llamadart

Run local LLMs from Dart and Flutter with one API across native and web runtimes. llamadart routes GGUF models through llama.cpp and .litertlm models through LiteRT-LM.

Start Here

Need	Link
Install the package	Installation
Load a first model	Quickstart
Build chat history	First chat session
Check runtime support	Platform & backend matrix
Read API reference	pub.dev API docs
Try the Flutter demo	Hosted chat app

What It Supports

• GGUF model loading and generation through llama.cpp.
• .litertlm model loading and generation through LiteRT-LM.
• Native Dart and Flutter targets with downloaded runtime assets.
• Flutter Web through the experimental WebGPU bridge and LiteRT-LM web runtime.
• Streaming chat completions, tool-call parsing, multimodal GGUF projectors, structured JSON output, embeddings, LoRA, state persistence, and runtime diagnostics where the active backend supports them.

Unsupported runtime/option combinations are rejected explicitly instead of silently degrading. Check the support matrix before relying on a capability for a specific model format or platform.

Requirements

• Dart SDK >=3.10.7
• Flutter SDK >=3.38.0 for Flutter apps
• iOS deployment target 16.4 or newer for Flutter iOS apps
• macOS deployment target 14.0 or newer for Flutter macOS apps

Consumers do not need a local C++ toolchain. Native runtime archives are resolved by the package build hook on first build or run.

Install

For Dart or Flutter apps:

dependencies:
  llamadart: ^0.8.14

Flutter iOS/macOS apps that should link Apple XCFrameworks through Swift Package Manager should also add the runtime companion packages they need:

dependencies:
  llamadart: ^0.8.14
  llamadart_llama_cpp_flutter: ^0.0.9 # GGUF / llama.cpp
  llamadart_litert_lm_flutter: ^0.0.4 # iOS .litertlm / LiteRT-LM

At the current LiteRT-LM native pin, Flutter macOS .litertlm builds keep using the core native-assets fallback when the SwiftPM artifact set is incomplete for the selected architecture.

Then run:

dart pub get
# or
flutter pub get

First Generation

import 'package:llamadart/llamadart.dart';

Future<void> main() async {
  final engine = LlamaEngine(LlamaBackend());

  try {
    await engine.loadModelSource(
      ModelSource.parse(
        'hf://unsloth/SmolLM2-135M-Instruct-GGUF/'
        'SmolLM2-135M-Instruct-Q2_K.gguf',
      ),
    );

    final output = StringBuffer();
    await for (final chunk in engine.create(
      const [
        LlamaChatMessage.fromText(
          role: LlamaChatRole.user,
          text: 'Explain local inference in one sentence.',
        ),
      ],
      params: const GenerationParams(maxTokens: 48),
    )) {
      final text = chunk.choices.first.delta.content;
      if (text != null) {
        output.write(text);
      }
    }
    print(output.toString());
  } finally {
    await engine.dispose();
  }
}

For multi-turn chat, wrap the same engine in ChatSession and let it maintain history: First chat session.

Choosing a Runtime

Model format	Typical use	Runtime
GGUF	Broad llama.cpp compatibility, Metal/Vulkan/CUDA/CPU, WebGPU bridge	llama.cpp
.litertlm	LiteRT-LM deployments, Android GPU/NPU-oriented bundles, Gemma-family LiteRT packages	LiteRT-LM

LlamaBackend() routes by model file type. Use ModelParams for load-time controls such as context size, GPU layers, backend preference, LiteRT-LM backend selection, and WebGPU mem64 hints. See Runtime Parameters for the full list.

Current default runtime pins:

Runtime	Pin
Native llama.cpp / GGUF	leehack/llamadart-native@b9935
Native LiteRT-LM / .litertlm	leehack/litert-lm-native@v0.14.0-native.1
Web llama.cpp / GGUF	leehack/llama-web-bridge-assets@v0.1.18
Web LiteRT-LM / .litertlm	@litert-lm/core@0.14.0

Common Tasks

Task	Docs
Resolve local paths, URLs, and Hugging Face sources	Finding models
Pick native/Web/LiteRT backends	Backend selection
Stream text and collect output	Generation and streaming
Generate typed JSON	Structured output
Use tool calling	Tool calling
Use images, audio, or projectors	Multimodal
Generate embeddings	Embeddings
Load LoRA adapters	LoRA adapters
Save and restore KV state	API levels
Run Flutter Web / WebGPU	WebGPU bridge
Tune performance	Performance tuning

Examples

• Basic Dart CLI
• Flutter chat app
• HTTP server example
• TUI coding agent example

Validate Changes

For package changes:

dart format --output=none --set-exit-if-changed .
dart analyze
dart test -p vm -j 1 --exclude-tags local-only
dart test -p chrome --exclude-tags local-only

For docs changes:

dart run tool/testing/verify_release_docs_versions.dart
./tool/docs/build_site.sh
./tool/docs/validate_links.sh

For heavier local model checks, list the discoverable scenarios:

dart run tool/testing/run_local_e2e.dart --list
dart run tool/testing/test_matrix.dart --list

Contributing

Keep public behavior, examples, README, website docs, support matrices, and changelog entries aligned. For non-trivial PRs, record the relevant testing matrix rows and exact validation evidence in the PR body.

License

MIT. See LICENSE.