Researchers created a system enabling language models to run privately offline in web browsers
Researchers created a system enabling language models to run privately and offline within web browsers using WebGPU technology.
The system uses small-batch integer kernels to verify multiple tokens at once while maintaining exact agreement with greedy decoding in the same quantized runtime.
A GPU-resident loop handles drafting, verification, acceptance and KV-cache maintenance without requiring a per-token CPU decision.
Across seven models ranging from 0.5B to 8B parameters, the runtime achieved speedups of 3.5 to 5.4 times on laptop GPUs, 4.8 to 6.2 times on integrated GPUs and 4.8 to 8.6 times on workstation GPUs compared to WebLLM.
Sources behind this report
Report history
No earlier revision dates are retained here. This does not establish that the report has never changed; read any dated updates and corrections.
This brief was made by AI and checked against its cited public sources. How the newsroom works