OpenFill is in development. Inference is off, but is tested end to end, and will switch on at launch. All data currently on the site is for live testing: it will be erased at launch.

Skip to content
openfill

OpenAI-compatible API · priced by a live market

AI inference.You set the limit.

Every account starts with $1 of credit as a free trial. No card is needed.

Clearing level · market-preview · preview

$0.0383per 1M input tokens (90% cached, 2% output)

Per 1M tokens: $0.0167 cache-hit · $0.1665 cache-miss · $0.3330 output

A preview of the market, which opens soon. Its prices are simulated until then; fixed-price models run today.

Level, last 24 hours

Test price limit $0.0453

$0.0652$0
Cleared at once
66%
Longest wait
6h 49m
Mean charge
$0.0425

Drag the test price limit. It sets nothing, and the figures read the last 24 hours on this market.

market-preview in detail

Fixed-price models

Three published prices per 1M tokens

Each of these models is sold at published prices for cache-hit input, cache-miss input and output. A request is charged its tokens at those prices.

Every model and its prices
01

How the price is set

A bid line, a fill, and the gap between them

Clearing levelYour price limitFilledThe gap$0
13.2s
Wait
52%
Of the bid charged

You pay the clearing level

Bid
Your price limit. The request rests in the book at it.
Charge
The level at execution, up to your price limit.
Receipt
Level, bid and charge, in usage.x_market.
02

Where the capacity comes from

Dedicated nodes run full, and shared nodes are topped off

Keeping utilization at 100% instead of 30% allows us to offer a far better average price, as well as speed and reliability.

1 A dedicated node

100%in use

Capacity06:0012:0018:00
The level
Drag the price. A fixed price sells the day as it comes; the market lets the level fall until every slot is taken.

2 A partner's node, topped off

95%topped off at 02:00 · their traffic 5%

Capacity06:0012:0018:00
The partner's traffic
Scrub the day. The partner's traffic takes what it needs and the market tops off the rest, most of it while they are quiet.
03

Drop-in for the OpenAI SDK

One base URL and one parameter, in every tool

Change the base URL and add a price limit. With a default on your API key or account, the parameter is optional.

Read docs
import osfrom openai import OpenAI client = OpenAI(    base_url=(        "https://api.openfill.ai/v1"    ),    api_key=os.environ[        "OPENFILL_API_KEY"    ],) r = client.chat.completions.create(    model="market-preview",    messages=[        {"role": "user", "content": "hi"}    ],    extra_body={"price_limit": 0.10},)print(r.choices[0].message.content)print(r.usage)

Until the market opens, the same call runs today on any fixed-price model: put its id in place of market-preview.

04

An additional layer in the inference stack

Above the serving stack · under your code · any engine, any hardware

Your applicationAny client of the OpenAI API, with price_limit on each requestOpenAI SDKAI SDKLangChainLiteLLMClaude CodeCodexClineOpenCodeHow to call it
Request, with a price limitResponse, with the price paid
openfillThe market layerPrices each request against live demand and capacity, then admits it at the clearing levelHow the price is set
Admitted, at the clearing level
Serving stack
  • RouterSends an admitted request to a servervLLM Routerllm-dDynamo RouterSGLang RouterATOM Mesh
  • Inference engineRuns the model: prefill, then decodevLLMSGLangTensorRT-LLMATOM
  • KV cacheKeeps prompt state close to the acceleratorLMCacheMooncake StoreSGLang HiCachevLLM OffloadingDynamo KVBMMORI-UMBP
  • Data movementMoves bytes between accelerators, host memory and storageNIXLMooncakeMORI-IOmemcpyCUDAHIP
  • Accelerators and storageHBM on the accelerator, then DRAM, NVMe and remote storageNVIDIAAMDHuaweiCambriconHBMDRAMNVMeRemote FS
Create an account$1 of credit to start, no card neededPrice your own token mix