Scattered spreadsheets couldn't catch AI hallucinations. Now, automated LLM judges evaluate every prompt change to block regressions.
A leading cloud storage platform developing a universal search tool that retrieves and organizes work across all of a user's connected applications.
Behind the search interface runs a complex chain of retrieval and inference steps where a single prompt tweak can ripple unpredictably to cause...
“With Braintrust, our science fiction writer can sit down, see something he doesn't like, test against it very quickly, and deploy his change to production. That's pretty remarkable.”
Cloud storage, file sharing, and collaboration platform for teams and individuals.
AI observability and evaluation platform that helps developers build, test, and monitor LLM-powered applications.
Dropbox's Model evaluation is part of this use case:
Related implementations across industries and use cases
Prompt iterations rubber-banded: each engineer's fix overcorrected the last. Evals are now a merge requirement—no eval, no commit.
Teams couldn't manually review hundreds of daily AI hotel calls. Audio models now evaluate raw recordings, routing exceptions to humans.
Sequential AI testing bottlenecked development. Engineers built a concurrent, code-first pipeline to evaluate agent responses in seconds.
Prompt iterations rubber-banded: each engineer's fix overcorrected the last. Evals are now a merge requirement—no eval, no commit.
Underperforming agents meant engineers rewriting prompts by hand and guessing. Now a self-correcting system reads scores and rewrites them.
Scattered data and basic coding tools bottlenecked engineers. A 9-agent AI workflow shifts them from writing code to directing AI teams.
Sequential AI testing bottlenecked development. Engineers built a concurrent, code-first pipeline to evaluate agent responses in seconds.
Tournaments running simultaneously meant an hour of manual checks each. AI agents now run them in minutes, freeing the team to be proactive.
Large AI training jobs meant fighting for preemptible slots or leaving campus. Marlowe gave any lab guaranteed multi-node access on demand.
Scattered spreadsheets couldn't catch AI hallucinations. Now, automated LLM judges evaluate every prompt change to block regressions.
A leading cloud storage platform developing a universal search tool that retrieves and organizes work across all of a user's connected applications.
Behind the search interface runs a complex chain of retrieval and inference steps where a single prompt tweak can ripple unpredictably to cause...
“With Braintrust, our science fiction writer can sit down, see something he doesn't like, test against it very quickly, and deploy his change to production. That's pretty remarkable.”
Cloud storage, file sharing, and collaboration platform for teams and individuals.
AI observability and evaluation platform that helps developers build, test, and monitor LLM-powered applications.
Dropbox's Model evaluation is part of this use case:
Related implementations across industries and use cases
Prompt iterations rubber-banded: each engineer's fix overcorrected the last. Evals are now a merge requirement—no eval, no commit.
Teams couldn't manually review hundreds of daily AI hotel calls. Audio models now evaluate raw recordings, routing exceptions to humans.
Sequential AI testing bottlenecked development. Engineers built a concurrent, code-first pipeline to evaluate agent responses in seconds.
Prompt iterations rubber-banded: each engineer's fix overcorrected the last. Evals are now a merge requirement—no eval, no commit.
Underperforming agents meant engineers rewriting prompts by hand and guessing. Now a self-correcting system reads scores and rewrites them.
Scattered data and basic coding tools bottlenecked engineers. A 9-agent AI workflow shifts them from writing code to directing AI teams.
Sequential AI testing bottlenecked development. Engineers built a concurrent, code-first pipeline to evaluate agent responses in seconds.
Tournaments running simultaneously meant an hour of manual checks each. AI agents now run them in minutes, freeing the team to be proactive.
Large AI training jobs meant fighting for preemptible slots or leaving campus. Marlowe gave any lab guaranteed multi-node access on demand.