Docs

Everything runs in the browser: Kiln downloads a model's files from the Hugging Face Hub once, caches them, and runs the model on the GPU in a worker.

Install

npm install kiln-webgpu

Or from a CDN, with no bundler:

<script type="module">
  import { load } from "https://cdn.jsdelivr.net/npm/kiln-webgpu@0.2.0";
</script>

The package is one ES module with the model worker inside (started from a blob URL), so there is no worker file to serve. A page with a Content-Security-Policy needs worker-src blob:.

Loading a model

import { load } from "kiln-webgpu";

const llm = await load("Qwen/Qwen2.5-0.5B-Instruct", {
  quant: "q4",                                   // "auto" (default), "q4", "q8" or "f32"
  onProgress: (p) => console.log(p.file, p.loaded, p.total),
});
llm.info; // { family, backend: "webgpu" | "cpu", layers, maxSeq, contextLength, ... }

• A model id is org/repo (safetensors or pytorch_model.bin) or org/repo/file.gguf for one GGUF file.
• quant: "auto" stores float weights as q4 and keeps a GGUF's tensors at their own precision. Small models (Gemma 3 270M, Granite 4 350M) need "q8" or "f32".
• Files are cached in the browser (OPFS) as they download; interrupted downloads resume. listCache() and clearCache() manage the cache.
• Float weights loaded as q4 or q8 are also cached as converted (about a third to a half of the download on top), which roughly halves later loads; weightCache: false keeps only the download.
• Config, tokenizer and file-listing requests are cached as well and checked against the server in the background: a cached model loads offline, and an updated repo shows from the following load on.
• The context defaults to the model's, up to 8,192 tokens and 512 MiB of KV cache; maxSeq sets it.
• Without WebGPU, Kiln runs on the CPU (WASM SIMD; multithreaded on a cross-origin isolated page). device: "gpu" fails instead, device: "cpu" forces the CPU.

Supported language models: Llama, Mistral, Qwen2, Qwen3, SmolLM, Granite 3 and 4, Gemma 3 (images from 4B up) and SmolVLM.

Chat and completion

const messages = [
  { role: "system", content: "You are concise." },
  { role: "user", content: "Name three uses of a kiln." },
];
const stream = llm.chat(messages, { temperature: 0.7, maxTokens: 200 });
for await (const text of stream) output.append(text);
const stats = await stream.stats; // { promptTokens, cachedTokens, generatedTokens, decodeMs, stopReason, ... }

llm.complete("Once upon a time", { maxTokens: 50 }); // raw completion, no chat template

The model's own chat template renders the prompt. Each call reuses the cached K/V of the earlier prompt that shares the most tokens with it, so a follow-up only prefills what is new; the last conversation's cache also survives a page reload. Pass an AbortSignal as signal to stop a reply.

Options: maxTokens, temperature (0 is greedy), topK, topP, repetitionPenalty, seed, stopTokenIds, enableThinking (Qwen3), adapter, format, tools, documents, signal.

Structured output

Constrain the reply to a JSON Schema, a regex, a list of choices or a GBNF grammar. The constraint is enforced at every token, so the result always parses:

const schema = {
  type: "object",
  properties: { city: { type: "string" }, population: { type: "integer" } },
  required: ["city", "population"],
  additionalProperties: false,
};
let json = "";
for await (const t of llm.chat(messages, { format: { type: "json", schema } })) json += t;
JSON.parse(json);

llm.chat(messages, { format: { type: "choice", choices: ["yes", "no"] } });
llm.chat(messages, { format: { type: "regex", pattern: "\\d{3}-\\d{4}" } });
llm.chat(messages, { format: { type: "grammar", grammar: 'root ::= "yes" | "no"' } });

Schemas may use enum, const, anyOf / oneOf, $ref, pattern, format (date, time, date-time, email, uuid, ipv4), minimum / maximum / multipleOf, array and string lengths.

Tool calls

const tools = [{ type: "function", function: { name: "get_weather", parameters: { type: "object", properties: { city: { type: "string" } }, required: ["city"] } } }];
let text = "";
for await (const t of llm.chat(messages, { tools, toolChoice: "required" })) text += t;
parseToolCalls(text); // { content, toolCalls: [{ type: "function", function: { name: "get_weather", arguments: { city: "Oslo" } } }] }

Images in chat

const vlm = await load("HuggingFaceTB/SmolVLM-500M-Instruct");
const reply = vlm.chat([{ role: "user", content: [
  { type: "image", image: file },                 // Blob, File, ImageBitmap or ImageData
  { type: "text", text: "Read this invoice. Reply with JSON." },
] }], { format: { type: "json", schema } });

SmolVLM and Gemma 3 (4B and larger, from the transformers checkpoint or a GGUF with its mmproj file) take images. Asking again about the same image reuses its features.

Pipelines

transformers.js's pipeline(), with its task names, options and output shapes:

import { pipeline } from "kiln-webgpu";

const classifier = await pipeline("sentiment-analysis");
await classifier("I love this!"); // [{ label: "POSITIVE", score: 0.9998 }]

const transcriber = await pipeline("automatic-speech-recognition", "openai/whisper-base");
await transcriber(audioUrl, { return_timestamps: "word" });

Options take transformers.js's names (max_new_tokens, do_sample, top_k, pooling, aggregation_strategy, return_timestamps, …). dtype picks how weights are stored (q4, q8 or f32 names) and device is "auto", "webgpu" or "wasm". See From transformers.js for the differences.

transformers.js API

AutoTokenizer, AutoModel and its task heads, AutoModelForCausalLM.generate(), AutoProcessor, RawImage, read_audio, Tensor and env:

import { AutoTokenizer, AutoModel } from "kiln-webgpu";

const tokenizer = await AutoTokenizer.from_pretrained("sentence-transformers/all-MiniLM-L6-v2");
const model = await AutoModel.from_pretrained("sentence-transformers/all-MiniLM-L6-v2");
const { last_hidden_state } = await model(tokenizer(["Hello", "World"], { padding: true }));

Details and the full list are in the migration guide.

LoRA adapters

const llm = await load("HuggingFaceTB/SmolLM2-135M-Instruct");
await llm.loadAdapter("science", "Miladsaeedi70/smollm2-135m-scientific-sft-lora");
await llm.loadAdapter("linkedin", "ArtemMozharov/smollm2-135m-linkedin-translator");

await llm.setAdapter("science");                // from now on
llm.chat(messages, { adapter: "linkedin" });    // or for one call
await llm.setAdapter(null);                     // the base model

PEFT LoRA adapters (adapter_config.json and adapter_model.safetensors) on the attention and MLP projections. Each adapter is laid out on the GPU when it loads, so switching between loaded adapters takes a few milliseconds. DoRA, embedding adapters and modules_to_save are not supported.

Retrieval

import { Retriever } from "kiln-webgpu";

const rag = await Retriever.create();                 // all-MiniLM-L6-v2; or { model: "BAAI/bge-small-en-v1.5" }
await rag.add(documents);                             // strings or { id, text, metadata }
const hits = await rag.search("question", { k: 4 });
const { stream, sources } = await rag.ask(llm, "question", { k: 3, minScore: 0.3 });
const saved = rag.export();                           // plain data for IndexedDB; Retriever.fromSnapshot(saved) restores it

Documents are chunked at sentence boundaries by the embedder's own tokens; pooling and query prompts come from the model's sentence-transformers files. ask numbers the passages ([1], [2] …) in the system message in document order, so a follow-up over the same passages reuses the cached prompt. Small LLMs may still answer from their own knowledge; minScore keeps weak passages out.

One model, many tasks

import { tasks } from "kiln-webgpu";

const t = tasks(llm);
await t.classify("My card was charged twice", ["billing", "bug report", "feature request"]);
await t.extract(email, schema);
await t.ner(text, ["person", "drug", "hospital"]);
await t.answer("When did it open?", context);       // { answer, score, start, end } or { answer: null }
await t.summarize(text, { sentences: 2 });

classify returns the model's probabilities over your labels (each scored as its whole reply), not a sampled guess. extract always matches the schema. ner keeps only mentions found verbatim in the text. For standard entity types a dedicated NER model is still more thorough; the LLM's strength is types of your own.

Speech

const asr = await pipeline("automatic-speech-recognition", "openai/whisper-base");
await asr(fileOrUrl, { return_timestamps: true, chunk_length_s: 30, language: "french", task: "translate" });

const tts = await pipeline("text-to-speech");          // facebook/mms-tts-eng
const { audio, sampling_rate } = await tts("Hello!");

Whisper (timestamps, word timestamps, long audio), wav2vec2 and HuBERT CTC, audio classification, and SpeechT5, MMS-VITS and Kokoro voices. Audio is a URL, Blob, encoded bytes or 16 kHz samples.

Vision

const detector = await pipeline("zero-shot-object-detection");   // OWL-ViT
await detector(image, ["a cat", "a remote control"]);

const remover = await pipeline("background-removal");            // RMBG-1.4 (non-commercial license)
const cutout = await remover(image);                               // { width, height, channels: 4, data }

Images are resized and normalized exactly as Python's transformers does it (PIL's resampling, reimplemented), and can be URLs, Blobs, RawImage, ImageBitmap, ImageData or elements.

Performance and memory

• Prefer quant: "q4" for LLMs on integrated GPUs; f32 is for checking outputs.
• Keep one model loaded and reuse it: Kiln caches prompt prefixes across calls, conversations and reloads.
• await llm.startProfile() … await llm.stopProfile() gives GPU time per kernel.
• The benchmarks list end-to-end timings against transformers.js on the same machine.

Limits

• Kiln implements each architecture by hand, so it runs fewer architectures than transformers.js, which runs any ONNX export.
• One sequence decodes at a time per model; overlapping calls queue.
• Loading the same model twice in one page fails (its cached file is open in the first worker); share one instance.
• Repos must ship safetensors, pytorch_model.bin or GGUF weights; ONNX-only repos (most Xenova/ ones) do not load.