Bubble
Agent evaluation
Repeated prompts overfitted the agent, hiding regressions. Evals score for equivalence — the same app can be built in multiple valid ways.
- 1.5x accuracy improvement vs predecessor agent
Prompt iterations rubber-banded: each engineer's fix overcorrected the last. Evals are now a merge requirement—no eval, no commit.
The first agentic B2B support platform, where AI proactively solves customer problems using company backend tools rather than retrieving knowledge base answers, making response consistency the product that customers' customers experience directly.
Early prompt development was vibes-based, creating a rubber-banding failure mode where engineers over-corrected each other's work and introduced...
“We added a test so that every single AI prompt we check into the codebase needs to have a Braintrust playground ID. If it doesn't have a playground ID attached to it, the test fails.”
AI-powered customer support and ticketing platform for B2B teams.
AI observability and evaluation platform that helps developers build, test, and monitor LLM-powered applications.
Pylon's Prompt evaluation is part of this use case:
Related implementations across industries and use cases
Repeated prompts overfitted the agent, hiding regressions. Evals score for equivalence — the same app can be built in multiple valid ways.
Teams couldn't manually review hundreds of daily AI hotel calls. Audio models now evaluate raw recordings, routing exceptions to humans.
Debugging scattered across individual notebooks meant week-long fixes. One shared platform now catches errors before customers do.
Repeated prompts overfitted the agent, hiding regressions. Evals score for equivalence — the same app can be built in multiple valid ways.
Teams couldn't manually review hundreds of daily AI hotel calls. Audio models now evaluate raw recordings, routing exceptions to humans.
Three people, three different AI answers. Grounded data sources fixed that — finance now has years of vendor spend at its fingertips.
Thousands of tables tracking the same metrics differently. No one knew which to trust — not humans, not AI. One governed layer changed that.
Tournaments running simultaneously meant an hour of manual checks each. AI agents now run them in minutes, freeing the team to be proactive.
Large AI training jobs meant fighting for preemptible slots or leaving campus. Marlowe gave any lab guaranteed multi-node access on demand.
Prompt iterations rubber-banded: each engineer's fix overcorrected the last. Evals are now a merge requirement—no eval, no commit.
The first agentic B2B support platform, where AI proactively solves customer problems using company backend tools rather than retrieving knowledge base answers, making response consistency the product that customers' customers experience directly.
Early prompt development was vibes-based, creating a rubber-banding failure mode where engineers over-corrected each other's work and introduced...
“We added a test so that every single AI prompt we check into the codebase needs to have a Braintrust playground ID. If it doesn't have a playground ID attached to it, the test fails.”
AI-powered customer support and ticketing platform for B2B teams.
AI observability and evaluation platform that helps developers build, test, and monitor LLM-powered applications.
Pylon's Prompt evaluation is part of this use case:
Related implementations across industries and use cases
Repeated prompts overfitted the agent, hiding regressions. Evals score for equivalence — the same app can be built in multiple valid ways.
Teams couldn't manually review hundreds of daily AI hotel calls. Audio models now evaluate raw recordings, routing exceptions to humans.
Debugging scattered across individual notebooks meant week-long fixes. One shared platform now catches errors before customers do.
Repeated prompts overfitted the agent, hiding regressions. Evals score for equivalence — the same app can be built in multiple valid ways.
Teams couldn't manually review hundreds of daily AI hotel calls. Audio models now evaluate raw recordings, routing exceptions to humans.
Three people, three different AI answers. Grounded data sources fixed that — finance now has years of vendor spend at its fingertips.
Thousands of tables tracking the same metrics differently. No one knew which to trust — not humans, not AI. One governed layer changed that.
Tournaments running simultaneously meant an hour of manual checks each. AI agents now run them in minutes, freeing the team to be proactive.
Large AI training jobs meant fighting for preemptible slots or leaving campus. Marlowe gave any lab guaranteed multi-node access on demand.