cplan / continuous batching

The first token
is the timetable.

End-to-end SLO attainment can look healthy while every user stares at an empty screen. This board puts the first token on the clock. Set your mean-TTFT budget, pick a load, and watch static batching miss the gate that continuous batching clears.

The decision

At a fixed first-token budget, which scheduler keeps the platform inside it as arrival rate rises?

simulated sweep

Arrival rate

Mean TTFT budget 300 ms

50 ms1,200 ms

4s SLO floor 99%

90%100%

Departures / mean TTFT —

Continuous — Static —

The gate, in milliseconds

dots = mean TTFT · line = your budget

The scheduling read

—

— continuous loads on time

— static loads on time

— TTFT cut at selected load

— static p99 at selected load

Source receipts/receipt.json Model scheduler simulator / 4s end-to-end SLO Budget and SLO floor are your inputs Open receipt ↗

The first tokenis the timetable.

—

The first token
is the timetable.