FreeLLMAPI - Setup Guide

Stop paying for AI API bills. Stack the free tiers from 14 providers behind one endpoint and get ~800M tokens/month.

What is this?

FreeLLMAPI is an open-source proxy that combines the free tiers from 14 different AI providers (Google, Groq, Cerebras, Mistral, etc.) into a single OpenAI-compatible API endpoint.

Instead of juggling 14 SDKs, 14 rate limits, and 14 places your code can break, you point your existing OpenAI code at localhost, and FreeLLMAPI handles everything. It picks the best available model, automatically fails over when one provider hits its rate limit, and tracks usage so you never exceed any free-tier cap.

The result: roughly 800 million free tokens per month across dozens of models, with one line of code changed in your app.

Key features

• One unified API key for all providers, your apps never see upstream keys
• Automatic failover, if a provider returns a 429 or error, the router retries on the next one instantly
• Per-key rate tracking (RPM, RPD, TPM, TPD) so you stay under every free-tier limit
• Sticky sessions, multi-turn conversations stay on the same model for 30 minutes
• Encrypted key storage (AES-256-GCM), your provider keys are encrypted at rest
• Admin dashboard with analytics, playground, and fallback chain editor
• Works with any OpenAI-compatible client (Python SDK, LangChain, LlamaIndex, curl, etc.)

Supported providers (14 total)

You don't need keys from all 14. Even 3 to 4 providers give you a solid amount of free tokens with good failover coverage.

Prerequisites

You need Node.js 20+ and npm installed on your computer.

node --version   # should show v20.x or higher
npm --version    # should show 10.x or higher

If not installed, download Node.js from nodejs.org, npm comes bundled with it.

Step 1, Clone the repo and install

Open your terminal and run:

git clone https://github.com/tashfeenahmed/freellmapi.git
cd freellmapi
npm install

This downloads the project and installs all dependencies.

Step 2, Set up the environment

The proxy encrypts your API keys at rest. You need to generate an encryption key:

cp .env.example .env
echo "ENCRYPTION_KEY=$(node -e "console.log(require('crypto').randomBytes(32).toString('hex'))")" >> .env

This creates a .env file with a random 256-bit encryption key. Your provider API keys will be encrypted with this before being stored in the local database.

Step 3, Start the server

npm run dev

This starts both the backend proxy server and the admin dashboard. You'll see output confirming both are running.

Step 4, Open the dashboard and add your keys

Open your browser and go to:

http://localhost:5173

This is the admin dashboard. Here's what to do:

1. Go to the Keys page, this is where you add your provider API keys
2. Add keys from your chosen providers, click "Add Key", select the provider, and paste your API key. Start with the free ones:
   • Google (Gemini), sign up at ai.google.dev, create an API key
   • Groq, sign up at console.groq.com, generate a key
   • Cerebras, sign up at cloud.cerebras.ai, get a key
   • Any others you want, every key you add increases your total token pool
3. Go to the Fallback Chain page, drag providers into your preferred order. The router tries your top-priority provider first, then falls down the chain if it hits a rate limit
4. Copy your unified API key, shown at the top of the Keys page. It looks like freellmapi-xxxxx. This is the only key your apps need

Step 5, Use it in your code

The entire point is that you change one line in your existing code, the base URL. Everything else stays the same.

Python (OpenAI SDK)

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:3001/v1",
    api_key="freellmapi-your-unified-key",  # from the dashboard
)

response = client.chat.completions.create(
    model="auto",  # let the router pick the best available model
    messages=[{"role": "user", "content": "Explain recursion in one paragraph."}],
)

print(response.choices[0].message.content)

Set model to "auto" and the router picks the best available model from your fallback chain. Or specify a model name directly (e.g., "gemini-2.5-flash") to target a specific one.

curl

curl http://localhost:3001/v1/chat/completions \
  -H "Authorization: Bearer freellmapi-your-unified-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Streaming

stream = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Write a haiku about coding."}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="", flush=True)

With any OpenAI-compatible tool

If you use LangChain, LlamaIndex, Continue, or any other tool that accepts an OpenAI base URL, just point it at http://localhost:3001/v1 with your unified key. No other changes needed.

How the router works

When your app sends a request:

1. The router looks at your fallback chain (the priority order you set in the dashboard)
2. It picks the highest-priority model that has a healthy key AND is under its rate limits
3. It decrypts the key, calls the provider
4. If the provider returns a 429 (rate limited) or 5xx (error), the router puts that key on a short cooldown and immediately retries with the next model in the chain
5. It tries up to 20 times before giving up

Every response includes headers telling you which provider served the request:

• X-Routed-Via: google/gemini-2.5-flash, which provider/model handled it
• X-Fallback-Attempts: 2, how many providers it tried before succeeding

The dashboard

The dashboard at http://localhost:5173 has three main sections:

• Keys, add, remove, and monitor your provider API keys. Each key shows a health status dot (green = healthy, yellow = rate limited, red = invalid)
• Fallback Chain, drag providers into your preferred priority order. Top = tried first
• Analytics, see request volume, success rate, tokens used, average latency, and per-provider breakdowns over 24h / 7d / 30d
• Playground, test prompts directly and see which provider served each response

Running in production

For a production-style build (single process, no hot reload):

npm run build
node server/dist/index.js

This serves both the API and dashboard on port 3001. Runs fine on minimal hardware, even a Raspberry Pi 4.

Important limitations

Be realistic about what this gives you:

• No frontier-tier reasoning. Free tiers top out around Llama 3.3 70B and Gemini 2.5 Pro. You won't get GPT-5 or Claude Opus quality through this.
• Quality drops as the day goes on. Your best models (Gemini 2.5 Pro, GPT-4o via GitHub Models) have the lowest daily caps. Once they exhaust, the router falls to weaker models. Resets at UTC midnight.
• Latency varies. Cerebras and Groq are extremely fast; others are slower. You get whichever is available.
• Free tiers can change any time. Providers regularly adjust their free offerings.
• No tool/function calling yet. Text-only chat completions for now.
• Single-user only. Don't expose this proxy to the internet, it's designed for personal use on your local machine.

Troubleshooting

"npm install" fails

Make sure you have Node.js 20+ installed. Run node --version to check. If you're on an older version, update from nodejs.org.

Dashboard won't load at localhost:5173

Make sure you ran npm run dev (not node server/dist/index.js). The dev command starts both the API server and the Vite dev server for the dashboard.

All requests failing

Check the Keys page in the dashboard. If all keys show red status dots, your API keys may be invalid or expired. Re-generate them from the provider's website.

Getting rate limited quickly

Reorder your fallback chain to put providers with higher limits at the top. Add more providers to increase your total pool.

Want to reset everything

Delete the SQLite database file and restart. Your encryption key in .env stays the same.

Quick reference

Links

• GitHub repo: github.com/tashfeenahmed/freellmapi
• Google AI keys: ai.google.dev
• Groq keys: console.groq.com
• Cerebras keys: cloud.cerebras.ai