Soup (soup-cli)

Soup is an open-source command-line tool that fine-tunes LLMs from one YAML config. The repository’s tagline: “Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.” Checked 2026-10-28 against the GitHub repository, its README and the PyPI entry.

Install	pipx install soup-cli (light CLI) or pip install "soup-cli[train]" (adds torch, transformers, peft, trl, datasets); extras for serve, UI and data
Runtime	Python 3.11 to 3.13 (3.10 dropped in v0.77.0); local GPU with QLoRA, Apple Silicon through an MLX backend, Unsloth backend; Docker image
License	Apache-2.0
Maintenance	Latest release v0.77.0, 2026-10-26; repo created 2026-02-20, pushed 2026-10-28; not archived. PyPI soup-cli 0.77.0. A young project that ships a minor version every few days and says in its own notes that v0.77 breaks configs that loaded on v0.76

What it gives you

• One config, one command. soup init --template chat writes a YAML; soup train runs it. Batch size, GPU detection and quantisation are chosen automatically (README).
• Many training methods. The docs list SFT, DPO, GRPO, PPO, KTO, ORPO, SimPO, IPO and BCO, plus tool-calling, process reward models, pre-training, distillation, classification, vision/audio/TTS and unlearning. PEFT options include DoRA, LoRA+, rsLoRA, VeRA, PiSSA and more.
• Layer streaming (BETA, opt-in with stream_layers: true). Keeps the frozen base out of VRAM and feeds it to the GPU one decoder layer at a time. The README’s own measurement: Llama-3.1-8B-Instruct with NF4 on an RTX 3050 Laptop 4 GB reached 119.6 tok/s with a 3.32 GB peak (measured on v0.72.2, before a v0.73.0 correctness repair), bit-exact against a normal resident run and reproduced on an H100, per the maintainers. A later post-repair run by a contributor reads 208.6 tok/s in a 2.40 GB peak on another RTX 3050 Laptop 4 GB; the README itself says the raw gain there is the clock speed and not a speedup. Treat these as the maintainers’ figures, not independently verified. The README links a paper (DOI on Zenodo), a benchmark folder and a Colab notebook that caps the process to 4 GB and checks the output is bit-identical.
• Broader tooling around training. Data engineering and quality scorecards, evaluation gates and probes, an OpenAI-compatible server (soup serve), export to GGUF, ONNX, TensorRT, BitNet and TQ1_0 (AWQ and GPTQ export were removed in v0.77.0 because both upstream projects are archived), a model registry, provenance records (soup registry push, soup ship --require-provenance), compliance init templates and a model-card generator.
• Local-first. Training on your own GPU with QLoRA, no cloud required (README).

Key differentiator

Layer streaming plus an unusually wide single-CLI scope: the stated aim is training an 8B model on a 4 GB GPU, and the same tool covers data prep, evaluation, serving, export and governance. Source: README. The alternatives below are better established for most teams.

Pick something else when

• You want a mature, widely adopted fine-tuning framework with a large community: Unsloth, Axolotl or LLaMA-Factory (all established projects).
• You only need preference tuning in Python code: Hugging Face TRL, which Soup builds on (the README names TRL objectives).
• You need a stable config schema: Soup’s v0.77.0 release refuses 31 config fields and three deprecated values that earlier versions accepted with a warning, so pin versions and run soup doctor --config soup.yaml after upgrades.
• Production, regulated training where bit-for-bit support matters: the layer-streaming feature is marked BETA by the project itself.

Limits to know

• The layer-streaming numbers come from the maintainers and one contributor; they have not been reproduced independently.
• The project is under nine months old, so its community size and long-term maintenance are still to be shown. Its GitHub star count is not recorded here.
• Speed, quality and model support relative to Unsloth, Axolotl and LLaMA-Factory are not compared in this note.

Related

• LangGraph (agent framework, unrelated to training; listed for the AI tooling neighbourhood)

Sources

• https://github.com/MakazhanAlpamys/Soup (README, releases)
• https://trysoup.dev
• https://pypi.org/project/soup-cli/