llamafile

What's Changed

The most exciting feature that this release of llamafile brings is the first version of transcribefile! 🎉 Our transcribefile is nothing more than a cosmocc-build of the CLI tool from transcribe.cpp, @cjpais' ggml-based Speech-to-Text library that supports 16+ model families 💙 For more details, see CJ's post and Mozilla.ai's annoucement.

In addition to this, a new sync with upstream llama.cpp (which will be soon replaced by a more recent one, as I let too much time pass before the release 😬); few improvements to Vulkan and ROCm acceleration; HTTPS support (both for the server and to download models from HF given their IDs); and pledge/SECCOMP sandboxing.

Thanks everyone for improving llamafile with issues and PRs!

• Probe GPU device count out-of-process on Windows (#988 follow-up) by @aittalam in https://github.com/mozilla-ai/llamafile/pull/994
• docs: redirect GitHub Pages docs to GitBook by @angpt in https://github.com/mozilla-ai/llamafile/pull/993
• ROCm shell script improvements by @aittalam in https://github.com/mozilla-ai/llamafile/pull/995
• docs(support): add GPU backend status table & acceleration verification (#996) by @Anai-Guo in https://github.com/mozilla-ai/llamafile/pull/997
• skill: guardrail llama.cpp update workflow with patch commands by @aittalam in https://github.com/mozilla-ai/llamafile/pull/998
• Update llama.cpp to b9747 (d6d8995) by @aittalam in https://github.com/mozilla-ai/llamafile/pull/1000
• chore: upgrade and digest pin all action digests by @RichardoC in https://github.com/mozilla-ai/llamafile/pull/1002
• CPU / GPU fixes + harness for upstream ggml's test-backend-ops by @aittalam in https://github.com/mozilla-ai/llamafile/pull/1004
• vulkan: clearer message when --gpu vulkan is explicitly unavailable by @aittalam in https://github.com/mozilla-ai/llamafile/pull/1006
• Fixes CTRL+C bug in chat mode by @aittalam in https://github.com/mozilla-ai/llamafile/pull/1009
• transcribefile: CPU build of transcribe.cpp + sync to df1a4ad by @aittalam in https://github.com/mozilla-ai/llamafile/pull/1007
• Updated runner to 16-core by @aittalam in https://github.com/mozilla-ai/llamafile/pull/1005
• transcribefile: Metal GPU support via llamafile's runtime loader by @aittalam in https://github.com/mozilla-ai/llamafile/pull/1010
• Add HTTPS support via cpp-httplib's mbedtls backend by @aittalam in https://github.com/mozilla-ai/llamafile/pull/1011
• feat(help): list all accepted llama.cpp options in --help by @RichardoC in https://github.com/mozilla-ai/llamafile/pull/1003
• Bring back pledge/SECCOMP sandboxing (#930) by @aittalam in https://github.com/mozilla-ai/llamafile/pull/1015

New Contributors

• @Anai-Guo made their first contribution in https://github.com/mozilla-ai/llamafile/pull/997
• @RichardoC made their first contribution in https://github.com/mozilla-ai/llamafile/pull/1002

Full Changelog: https://github.com/mozilla-ai/llamafile/compare/0.10.3...0.10.4

llamafile

<img src="docs/images/llamafile-640x640.png" width="320" height="320"

alt="[line drawing of llama animal head in front of slightly open manilla folder filled with files]">

License

ci status

Based on llama.cpp

Based on whisper.cpp

Discord

Mozilla Builders

llamafile lets you distribute and run LLMs with a single file.

llamafile is a Mozilla Builders project (see its announcement blog post), now revamped by Mozilla.ai.

Our goal is to make open LLMs much more

accessible to both developers and end users. We're doing that by

combining llama.cpp with Cosmopolitan Libc into one

framework that collapses all the complexity of LLMs down to

a single-file executable (called a "llamafile") that runs

locally on most operating systems and CPU architectures, with no installation.

llamafile also includes whisperfile, a single-file speech-to-text tool built on whisper.cpp and the same Cosmopolitan packaging. It supports transcription and translation of audio files across all the same platforms, with no installation required.

v0.10.*

llamafile versions starting from 0.10.0 use a new build system, aimed at keeping our code more easily

aligned with the latest versions of llama.cpp. This means they support more recent models and functionalities,

but at the same time they might be missing some of

the features you were accustomed to (check out this doc for a high-level description of what has been done). If you liked

the "classic experience" more, you will always be able to access the previous versions from our

releases page. Our pre-built llamafiles always

show which version of the server they have been bundled with (0.9. example, 0.10. example), so you will always know

which version of the software you are downloading.

We want to hear from you!

Whether you are a new user or a long-time fan, please share what you find most valuable about llamafile and what would make it more useful for you.

Read more via the blog and add your voice to the discussion here.

Quick Start

Download and run your first llamafile in minutes:

# Download an example model (Qwen3.5 0.8B)
curl -LO https://huggingface.co/mozilla-ai/llamafile_0.10/resolve/main/Qwen3.5-0.8B-Q8_0.llamafile

# Make it executable (macOS/Linux/BSD)
chmod +x Qwen3.5-0.8B-Q8_0.llamafile

# Run it
./Qwen3.5-0.8B-Q8_0.llamafile

We chose this model because that's the smallest one we have

built a llamafile for, so most likely to work out-of-the-box for you.

If you have powerful hardware and/or GPUs, feel free to choose

larger and more expressive models which should provide more accurate

responses.

Windows users: Rename the file to add .exe extension before running.

Note - Only executables under 4GB can run on Windows, so any llamafile above 4GB won't work. Download the llamafile binary and run it with any external weights/models(GGUF).

Documentation

Check the full documentation at docs.mozilla.ai/llamafile, or directly jump into one of the following subsections:

• Quickstart
• Pre-built llamafiles
• Running a llamafile
• Creating llamafiles
• Source installation
• Technical details
• Supported Systems
• Troubleshooting
• Whisperfile

Licensing

While the llamafile project is Apache 2.0-licensed, our changes

to llama.cpp and whisper.cpp are licensed under MIT (just like the projects

themselves) so as to remain compatible and upstreamable in the future,

should that be desired.

The llamafile logo on this page was generated with the assistance of DALL·E 3.

Star History Chart