BYOK AI gateway · velvet rope included
Your AI bill has a bouncer now.
Opus 4.8 does not get into the club to slugify a blog title. A 20ms doorman reads every request and sends the easy ones to a cheaper room — gpt-5-mini, or open models like GLM 5.2 and Qwen 3.7. We also recognize answers you've already paid for and trim the fat off each prompt. Same answers, meaningfully smaller tab — how much smaller depends on how repetitive your traffic actually is.
- base_url="https://api.openai.com/v1"
+ base_url="https://api.slashspend.ai/v1"
# that's the whole PR. ship it.
Your provider keys never touch our servers. No rewrites. Cancel by changing one line back.
“A frontier model. For a URL slug. Please sit down.”
The bouncer checks every request. Same answer, smaller tab.
“Wait — what is eating my money?”
You opened the credit card statement, saw the number, and made a noise only dogs can hear. Here's the autopsy: your priciest models keep getting hired for jobs an intern — or a 7B open model — could do, and you pay full freight every single time.
- 💸GPT-5 slugifying blog titles — a frontier model running .toLowerCase().replace(/ /g, "-").
- 🔁"Summarize our refund policy" answered 800 times, billed 800 times. You bought that answer once.
- 🧳4,000 tokens of chat history dragged along to ask one tiny follow-up. Cover charge by the pound.
- 🚪A provider wobble at 3pm, your retries stampede, and the meter just keeps spinning.
- Opus 4.8 · name a React component
- $14.20
- GPT-5 · slugify a blog title
- $9.40
- Gemini 2.5 Pro · parse JSON from a reply
- $6.10
- "Summarize refund policy" × 800
- $214.00
- Retries during the 3pm wobble
- $77.30
Illustrative. Your mileage (and horror) may vary.
What happens at the door
No card on file gets a markup, no prompt gets rewritten behind your back. The doorman just makes smarter calls than your code does at 2am.
Send the diva home — smart routing
A ~20ms doorman scores how much brain each request actually needs (RouteLLM-style, no extra round-trip). Trivial work drops to a cheaper tier — a mini model, or an open one like GLM 5.2 or Qwen 3.7 if you point routing there. A short-but-gnarly "prove this is undecidable" gets waved through to the big model — we never downgrade the questions that matter.
We remember your face — semantic cache
We match on meaning, not bytes. The cosine of the prompt embedding does the work, so a reworded question still hits. Answer comes back in <50ms at $0 — and we never store the prompt text, only its embedding: a fixed list of floats that's a lossy fingerprint of meaning, not a readable copy of what you sent.
Check your bags at the door — compression
We trim filler from your newest message and leave the conversation history byte-for-byte identical — so you stop paying by the pound without invalidating the provider's native prefix-cache discount. (Mutating history to save tokens is the classic own-goal. We don't.)
Lose the fluff with caveman mode
Flip on the header and models answer in dense, no-politeness prose — output tokens are the expensive ones, and agents/coding tools generate a firehose of them. Your token bill (and your context window) both thank you.
The back exit · automatic failover
If your provider face-plants mid-request — 429, 5xx, timeout — we walk you out the back to the next model in the chain (and back again) before your users notice. The party doesn't stop because one club lost power. Add an OpenRouter key and the chain ends with a strong open model (DeepSeek by default), so even all three frontier labs blinking at once still gets a served answer, not a 503.
If SlashSpend itself degrades, here's exactly what happens
No marketing language in this box. This is the actual fail-open logic, not a promise.
Every degraded response carries X-SlashSpend-Degraded: true — you can alert on it. We're not hiding it in a log you'll never read.
The behavioral contract — what never changes underneath you:
- — We never mutate your conversation history. Only filler in your newest message is trimmed; the history stays byte-for-byte, so your provider's prompt-cache discount keeps hitting.
- — We never store raw prompt text. A cache entry is an embedding vector or ciphertext, never plaintext (what a vector is — and isn't — is spelled out in the checklist below).
- — We never fail silently. If any layer degrades, the response is flagged with
X-SlashSpend-Degraded— we don't paper over a bad night. - — We never downgrade a model behind your back. Every routing and cache decision is visible at
/v1/trace, not buried. - — We never retry into a double-charge. Failover walks a fixed, ordered chain and hits each provider at most once; a status we don't understand is relayed as-is, not retried.
We don't wreck your prompt cache
Anthropic cache_control routes to the native Messages API, so your provider-side discount survives the hop — streaming included. Bursty traffic? Flip the 1h-tier knob or stabilize rotating timestamps and stop re-buying the same prefix.
New model drops Friday? Call it Friday.
Unrecognized models pass straight through untouched, and our registry auto-syncs pricing + context windows within 24h. You never wait on our deploy schedule to try the shiny new thing.
Every request leaves a receipt
A request-id on every response, a metadata-only decision trace at /v1/trace (routing, cache, failover, latency — never your prompt), and opt-in debug capture — one call, owner-scoped, gone in an hour — for when you need to open the hood.
Your cache is yours — and yours to torch
Cached responses live in your own namespace on a 1-hour TTL, encrypted at rest under a per-account key when a secret's set. Changed a prompt and want a clean slate? DELETE /v1/cache flushes the lot instantly. No support ticket, no waiting room.
“Doesn't OpenRouter already do this?”
Nope — and we run happily on top of it. OpenRouter's provider selection picks the best host for a model you already chose — the cheapest, fastest, most-up provider of, say, DeepSeek or Qwen. Useful! But it never asks the question that actually moves your bill: does this need to be an expensive model at all, and do we even need to call one?
| Optimizes… | OpenRouter routing | SlashSpend |
|---|---|---|
| The job | Which host runs the model you picked | Whether a cheaper tier (or no call) will do |
| Cross-tier downgrade (Opus → Haiku by difficulty) | ✗ | ✓ |
| Semantic cache — skip the model entirely | ✗ | ✓ |
| Prompt + output (caveman) compression | ✗ | ✓ |
| Cross-provider failover (OpenAI → Anthropic) | per-model only | ✓ |
| Sits on top of your OpenRouter key | (it is OpenRouter) | ✓ |
| Anthropic prompt-cache preserved (+ 1 h-tier knob) | ✗ | ✓ |
| Prefix stabilization (stops volatile tokens busting your cache) | ✗ | ✓ |
| Pricing | % fee on usage / credits | $99 flat, no % of spend |
Keep OpenRouter as one of your providers — pass your key, use any openrouter/ model (GLM 5.2, Qwen 3.7, your fine-tunes). We add the tier-routing, cache, and compression layer on top. Different layer of the stack; no turf war.
The paranoid-CTO checklist
You've been burned by a "free" middleware before. Good. Read this with your arms crossed.
“Won't a proxy add latency?”
We add roughly 20ms at the door on a normal request. The upside is that repeat traffic never reaches the model at all: a semantic-cache hit returns in ~30–50ms instead of the 1–3 seconds a fresh generation takes. So on cache-heavy workloads your p50 drops — not because a proxy is magically faster, but because the fastest request is the one you never have to send. On unique traffic that never hits cache, you're paying that ~20ms and nothing more. Watch it yourself: every response carries a latency breakdown at /v1/trace.
“Are you holding my provider keys?”
BYOKNo. In Local Key Mode your keys ride along as request headers, get used in memory, and are discarded. Never written to a database, a log, or any third party. Bring your own key; keep your enterprise agreements and billing tiers with the providers.
“What about my prompts and user data?”
No prompt storageWe never log prompt content or model responses. What we persist is metadata only — tokens, model, latency, cost — for 90 days, behind row-level security. "But the semantic cache stores something." Right, and here's the honest version so you can make your own call: it stores the prompt's embedding — a fixed-length list of floats — not the text, and we keep no copy of the original. An embedding is a lossy fingerprint of meaning, not an encoded copy; there's no key in our system that turns it back into your prompt. We're not going to claim it's mathematically impossible for anyone, ever, to approximate text from an embedding — that's an active research area and you'd be right to distrust anyone who did. We're telling you exactly what we hold: vectors in your own namespace, encrypted at rest, on a short TTL — no raw text, anywhere.
“Will this lock me in?”
No lock-inIt's a base URL — point it back at the provider any time and SlashSpend disappears, nothing to rip out. BYOK means your accounts, keys, rate-limit tier, and model previews were always yours; we never intermediate them. We even test the exit in CI: the same unmodified client must work against us and against the provider directly, or our build goes red.
“Do you break Anthropic's prompt-cache discount?”
No — requests with cache_control route to Anthropic's native Messages API, which honours the breakpoints, so your provider-side discount survives the proxy. The 1h-tier knob (X-SlashSpend-Cache-Tier: 1h) upgrades those breakpoints to the one-hour TTL tier — for bursty/agentic traffic you pay the cache-write premium once instead of re-paying the full prefix every five-minute eviction. Stabilize knob available too, for prompts with rotating timestamps that silently bust the cache.
“A new model just dropped — do I have to wait for you to ship an update?”
No on both counts. Unknown models are forwarded untouched (X-SlashSpend-Passthrough: true), so you can call a brand-new model the day it launches. The model registry also auto-syncs from the OpenRouter catalog every six hours, so context-window and pricing awareness arrives within 24 hours — automatically, no deploy on your side.
Don't take our word for it — read the no-lock-in pledge and the five-minute exit guide.
What should you actually expect to save?
Not a flat percentage — it's a function of how repetitive your traffic is. If a real slice of your calls are repeated questions, templated tasks, or work that a cheaper model can safely handle, you'll get close to our ~40% ceiling. If your traffic is mostly unique, hard, frontier-tier prompts, there's less for us to route around — sometimes close to nothing, and we'll say so rather than take your $99 for no reason. Your dashboard shows the real number for your account: what you actually paid vs. what you would have paid without us.
$99 a month. That's the whole menu.
No per-request markup. No percentage of your spend. No cut of your savings. The bouncer earns a flat salary — not a tip jar that grows with your tab.
A % of your spend
Sit with that incentive: they earn more when your bill is bigger. You'd be paying a company to shrink your spend that quietly profits when it grows. Weird. We didn't want that job.
$99/mo
Flat. Whether we save you $200 or $20,000, same price. If your team already spends more than ~$99/mo on these models, you'll likely save more than that in month one — and we've already paid for ourselves.
Start saving — $99/moPowered by Stripe · Cancel anytime · No lock-in
Flat really means flat.
- Is there hidden usage gating?
- No metered markup, ever. $99 whether you send 100 requests or 5 million.
- Is this only for small traffic?
- No — the 5M/mo number below is an abuse guard, not a plan limit. We email at 80% and raise it on request; it's a support ticket, not an upsell.
- What breaks at scale?
- Nothing. The same 20ms gateway, cache, and failover chain run whether you send 500 requests a month or 5 million — there's no separate "enterprise" code path that behaves differently. Watch it yourself at /v1/trace.
Payment activates your account. No free trial, because the math works on day one.
Abuse guard: 5M requests/mo, raised on request — it's not a plan limit. Details
Running well past 5M requests a month, or need an SLA, a security review, SSO, or invoice billing instead of a card? That's a conversation, not a checkbox — email sales@slashspend.ai.
Change one line. Keep the answers. Lose the markup.
Your provider keys never touch our servers. We never read your prompts. The bouncer just makes sure the right model answers the door.
Get on the list