WebLLM: high-performance in-browser LLM inference engine

ssaikatsg about 2 hours ago 7 commentsRead Article on github.com

Discussion Sentiment

50% Positive

Analyzed from 92 words in the discussion.

Discussion (7 Comments)Read Original on HackerNews

TekMol•about 1 hour ago

This seems to be the demo:

I am getting:

    WebGPUNotAvailableError: WebGPU is not supported in
    your current environment, but it is necessary to
    run the WebLLM engine.

On both, FireFox and Chromium on Linux.

toto007•about 1 hour ago

You can enable WebGPU support in Google Chrome by turning on hardware acceleration and activating the WebGPU flag. It Works.

refulgentis•about 1 hour ago

Project is de facto dead, used it for many years and had to rip it out 6 months ago, don't waste your time.

skybrian•about 1 hour ago

What did you switch to?

seamossfet•about 1 hour ago

We use the ONNX runtime for small models in the browser https://github.com/microsoft/onnxruntime

sroussey•about 1 hour ago

And ONNX is what Transformers.js uses as well, at least for the moment.

theanonymousone•about 1 hour ago

Transformers.js supports a lot of models since v4