How Runway Characters Keeps AI Video at 24 FPS
Runway’s engineers brought a live video session down from eight H100 GPUs to four. Their measurements also show where the wait for a reply comes from.
Editorial visualizationRunway says it cut the hardware needed for a live video character from eight Nvidia H100 GPUs per session at launch to four. In a test on the four-GPU setup, its engineers brought the median time for the model’s diffusion step down from 240.9 to 159.3 milliseconds. That was quick enough to sustain 24 frames per second. The same system still took a median 2.13 seconds to start speaking after the user finished, in a much smaller sample of live conversations. Runway’s preprint and its November 5 engineering report show where those two clocks diverge.
The November research describes the infrastructure behind Runway Characters, an existing service that turns a reference image into a video character that listens and speaks. Its roughly 57-billion-parameter video model generates the next four visible frames at a time, using previous frames for context. At 24 frames per second, those four frames last 166.7 milliseconds. If generation keeps taking longer than that, a small playback buffer will eventually run dry. Preprint, workload and step budget
Where the frame budget went
| Configuration tested | Median diffusion step |
|---|---|
| Original stack on four H100s | 240.9 ms |
| Optimized stack without fused GPU communication | 159.3 ms |
| Optimized stack with fused GPU communication | 147.4 ms |
The biggest avoidable expense came from the model’s working memory, called a key-value cache. Each attention operation consults stored information about earlier frames. The old code repeatedly reformatted much of that information and shifted the stored window as it advanced. Runway switched to a circular buffer, formatting new entries when they arrive and reusing most of the ones already there. In the paper’s component timing, cache upkeep fell from 69.7 to 6.2 milliseconds. Preprint, cache design and timings
