Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.fabro.sh/llms.txt

Use this file to discover all available pages before exploring further.

LiteLLM can run as an OpenAI-compatible proxy in front of many model providers. Fabro includes a disabled litellm provider entry so you can opt in from settings.toml without changing Fabro code.

​
Prerequisites

  • A running LiteLLM proxy reachable from the Fabro process
  • At least one LiteLLM model name you want Fabro to route to
  • A LiteLLM key or placeholder key available to Fabro
Fabro’s built-in LiteLLM provider points at http://localhost:4000/v1. Change base_url if your proxy is hosted elsewhere.

​
Enable the provider

Add the provider override and one or more model entries to ~/.fabro/settings.toml:
settings.toml
_version = 1

[llm.providers.litellm]
enabled = true
base_url = "http://localhost:4000/v1"

[llm.models."litellm-gpt-5"]
provider = "litellm"
api_id = "gpt-5"
display_name = "LiteLLM GPT-5"
family = "litellm"
default = true

[llm.models."litellm-gpt-5".limits]
context_window = 128000
max_output = 8192

[llm.models."litellm-gpt-5".features]
tools = true
vision = false
reasoning = false
api_id is the model name Fabro sends to LiteLLM. It should match a model name configured in your LiteLLM proxy.

​
Configure credentials

The LiteLLM provider checks LITELLM_API_KEY from the Fabro process environment first, then the vault:LITELLM_API_KEY server secret. For a server-owned secret: 
fabro secret set LITELLM_API_KEY sk-proxy-key
For a process environment variable: 
export LITELLM_API_KEY=sk-proxy-key
If your local LiteLLM proxy does not enforce authentication, use a placeholder value such as anything; the OpenAI-compatible client still needs a credential value.

​
Use LiteLLM models

Once the provider is enabled and at least one model is declared, use the Fabro model ID like any other catalog model: 
fabro model list --provider litellm
fabro model test --model litellm-gpt-5
fabro run workflow.fabro --model litellm-gpt-5
In workflow stylesheets:
workflow.fabro
digraph Example {
    graph [
        model_stylesheet="
            * { model: litellm-gpt-5; }
        "
    ]

    start [shape=Mdiamond, label="Start"]
    work  [label="Work", prompt="Use the configured LiteLLM model."]
    exit  [shape=Msquare, label="Exit"]

    start -> work -> exit
}

​
Declaring more models

Declare each LiteLLM-routed model explicitly so Fabro knows its provider, context window, tool support, and routing defaults:
settings.toml
[llm.models."litellm-fast"]
provider = "litellm"
api_id = "fast-model"
display_name = "LiteLLM Fast"
family = "litellm"
aliases = ["fast"]

[llm.models."litellm-fast".limits]
context_window = 64000
max_output = 4096

[llm.models."litellm-fast".features]
tools = true
vision = false
reasoning = false
Only one model for a provider should set default = true. You may also mark one small/cheap utility model with small_default = true; Fabro uses it for metadata tasks such as generated run titles and falls back to the provider default when it is omitted.

​
Troubleshooting

“No API key configured” — Set vault:LITELLM_API_KEY with fabro secret set LITELLM_API_KEY ... or export LITELLM_API_KEY in the Fabro process environment. Connection refused — Confirm the LiteLLM proxy is running and that base_url is reachable from the Fabro process. For Docker deployments, localhost means the Fabro container unless you point it at a host or service name. Unknown model from LiteLLM — Check that the model’s api_id matches the model name configured in LiteLLM, then run fabro model test --model <fabro-model-id>.

​
Further reading

Models

How Fabro routes model IDs, providers, and fallbacks.

Settings Configuration

Full reference for [llm.providers.<id>] and [llm.models.<id>].