At the recent VB Transform 2026 conference, Ryan Phillips, head of enterprise product at Runway ML, shared an intriguing story about how the company approaches technological bug fixing. Instead of continuing to spend weeks trying to engineer their way out of a stubborn backend model bug that caused AI-generated avatars to drift off-center during real-time video generation, Runway turned this limitation into a helpful frontend feature. This decision provides a valuable lesson for enterprise developers on how to optimize AI products without relying solely on backend foundation model upgrades.
Detailed Developments
According to Phillips, Runway developed "Runway Characters," a real-time video model that enables zero-latency, back-and-forth interactions with AI-generated avatars. However, during development, the engineering team discovered a persistent bug: characters would sway or drift away from the center of the frame during generation. The team spent weeks attempting to patch the core model architecture to eliminate the drift, but to no avail.
They ultimately realized that if the user's initial input image was perfectly centered, the generated video remained completely stable. Rather than wasting more time on a backend fix, Runway pivoted to a user experience solution by introducing a frontend feature called "Optimize for Image Quality." This tool automatically re-centers the user's image before video generation begins, transforming an engineering limitation into a helpful assistant feature in the eyes of users.
Technical & Technological Analysis
Delivering video globally at 24 frames per second (fps) requires optimizing every layer of the infrastructure stack. To achieve real-time latency, Runway relies on knowledge distillation, training a smaller, faster "student" model to mimic a massive "teacher" model, which cuts generation time by 80% to 90%. They then apply adversarial post-training (APT) to the distilled model to restore visual sharpness; however, it was the combination of distillation and APT that introduced the drift bug.
Furthermore, to maintain high performance, the team required deep system observability. When they noticed that 8% of API calls dropped to 16 fps, causing video stuttering, they deployed a Claude-powered AI agent alongside monitoring tools like Datadog and Sentry to trace the issue. The debugging session isolated the bottleneck to a single data center in the us-east-1 region, where they physically replaced the Nvidia GPUs to fully resolve the hardware-related anomaly.
Expert Opinions & Insights
According to Runway, building a robust AI product evaluation set (evals) is not strictly an engineering task. It requires cross-functional alignment across product, design, research, and sales to define what "quality" looks like. Runway uses simple tools like Excel spreadsheets to log daily tests and sets a strict pass-rate bar before shipping a model. Phillips suggested that enterprise developers can rely on large language models (LLMs) to automate visual grading, reducing manual evaluation bottlenecks.
Impact & Future
Runway's experience demonstrates that developing AI systems is rarely a linear process, often requiring teams to navigate what Phillips calls "failure hell" before achieving breakthroughs. Turning technical constraints into product features offers a new paradigm for tech companies deploying generative AI. In the future, the role of enterprise creatives will shift from designing single assets to defining the parameters, aesthetics, and worlds from which real-time models can generate content. This offers a highly relevant blueprint for developers and tech enterprises globally as they integrate AI solutions.