Light up your models 🔥

---

Flama

The production framework for Predictive and Generative AI.

Turn any model into a production API in a single line of code. Serve predictive and generative models on a Rust-powered core, and expose your tools to AI agents over the Model Context Protocol (MCP).

Flama is the Framework for Lightweight Applications, artificial intelligence Models, and Automation. It packages a model from any of the mainstream frameworks into a single portable format (the .flm file), so every model looks the same to your API no matter where it came from, and serves it over HTTP in seconds.

• 📦 Any framework, one format. Package scikit-learn, TensorFlow, PyTorch, or an LLM into a single portable .flm artifact.
• ⬇️ Models on demand. Download and package any model from the HuggingFace Hub with one command.
• 🤖 Generative AI serving. Serve LLMs with OpenAI-, Anthropic-, and Ollama-compatible endpoints, side by side.
• 💬 Chatbot out of the box. Every served model ships a polished streaming chat UI at /chat/, with Markdown, LaTeX, and Mermaid.
• 🔌 Native MCP. Expose tools, resources, and prompts to AI agents with a single decorator, schemas derived from your type hints.
• ⚡ Rust-powered core. Routing, JSON encoding, request parsing, and compression compiled to native code, shipped as plain wheels.
• 🚀 Production-ready first. Go from a packaged model to a running service over the CLI, in Python, with a spec file, or inside a container.

Installation

Flama is published on PyPI and ships native wheels for every supported Python version on Linux, macOS, and Windows. No Rust toolchain required. It needs one schema library, and Pydantic is the usual choice:

pip install "flama[pydantic]"

Everything else is an optional extra, so you install only what you need:

pip install "flama[typesystem]"    # or marshmallow, instead of pydantic
pip install "flama[database]"      # SQLAlchemy-backed resources
pip install "flama[llm]"           # generative AI serving (vLLM on Linux, MLX on Apple Silicon; Python 3.10 to 3.13)
pip install "flama[full]"          # everything

See the installation docs for details.

Quickstart

The same two commands serve both families of model: package it into a .flm, then point flama serve at the file. No application code either way.

Serve a predictive model

Package a model trained in any mainstream framework (this one needs scikit-learn):

import flama
from sklearn.datasets import load_iris
from sklearn.neural_network import MLPClassifier

X, y = load_iris(return_X_y=True)
model = MLPClassifier(activation="tanh", hidden_layer_sizes=(10,), max_iter=2000).fit(X, y)
flama.dump(model, family="ml", path="model.flm", metrics={"accuracy": model.score(X, y)})

Then serve it:

flama serve --model file=model.flm,url=/model,name=iris

You get two endpoints, an OpenAPI schema at /schema/, and Swagger UI at /docs/. GET /model/ returns the artifact's metadata; POST /model/predict/ runs inference:

curl -X POST http://127.0.0.1:8000/model/predict/ \
  -H "Content-Type: application/json" \
  -d '{"input": [[5.1, 3.5, 1.4, 0.2], [6.7, 3.0, 5.2, 2.3]]}'