At the VB Transform 2026 conference in Menlo Park, Xavi Amatriain, Chief AI and Data Officer at Expedia Group, asserted that evaluation suites (evals) are becoming the new Product Requirement Documents (PRDs). This statement comes as enterprises struggle to control and safely operate the next generation of autonomous AI agents. Defining products through evals before writing code is expected to bridge the "evaluation gap" concerning security and technological trust.
Diễn biến chi tiết
According to Amatriain, in an era where code can be assisted or fully generated by AI, the bulk of human design thinking will shift toward building evaluations. Instead of imposing too many brittle guardrails—which he called "a necessary evil" that interferes with the user feedback loop—organizations should embed security and business requirements directly into evals from the start. This allows systems to automate testing without compromising the real-world user experience.
To put this philosophy into practice, Expedia employs a three-layered AI governance model: core principles, automated processes and tools, and risk-calibrated "agent release toll gates" that adjust based on the severity of the task.
Phân tích kỹ thuật & Công nghệ
Regarding system architecture, Expedia does not believe in a single, monolithic Artificial General Intelligence (AGI) model. Instead, the company constructs an ecosystem composed of highly specialized agents. Expedia's architecture starts from the smallest component level: tools compose into skills, skills assemble into sub-agents, and sub-agents are orchestrated into the full system.
To handle real-time dynamics like fluctuating hotel prices and flight availability, Expedia's framework blends retrieval-augmented generation (RAG) with direct API tool calls. The choice of approach is latency-dependent. General pricing trends can leverage cached data, while granular, real-time requests may justify a longer reasoning window to cross-reference supplier details against Expedia's own database of traveler reviews.
Ý kiến chuyên gia & Nhận định
However, not everyone at the event agreed with Amatriain's stance on minimizing guardrails. Other speakers argued that high-risk actions still demand very firm boundaries to prevent catastrophic outcomes.
This skepticism is grounded in reality, as evidenced by VentureBeat's Pulse research. According to the report, 66% of the 157 surveyed enterprises already permit or plan to permit some production deployment without human review within the next 12 months. Yet, only 5% fully trust automated evaluations. More strikingly, half of the surveyed companies admitted to shipping an agent that passed internal evals but failed when interacting with real customers.
Tác động & Tương lai
Looking ahead, Amatriain warned that future security threats will not just come from human hackers, but increasingly from powerful, external agentic systems poking at endpoints. Shifting security "left"—integrating evaluation and security directly into the early design phase—will be a vital defense strategy.
For complex transactional platforms like travel, Expedia maintains a non-negotiable rule: the AI can recommend and discuss, but the user must click the final booking button. This constraint is not just a consumer protection measure, but a core architectural security decision that eliminates post-deployment risks.