Pylon
Prompt evaluation
Prompt iterations rubber-banded: each engineer's fix overcorrected the last. Evals are now a merge requirement—no eval, no commit.
- 10 mins of active debugging eliminated per on-call incident
Fixing AI tone quirks meant engineers in the loop every time. Now a writer and researcher test and ship prompt fixes themselves.
An AI companion app builder focused on authentic, non-romantic emotional connections through voice conversations and complex memory systems.
Conversation quality, emotional intelligence, and authentic behavior were too subjective and nuanced to capture through automated evals alone,...
“How will humanity build healthy relationships with AI? That's the question Tolan explores.”
AI-powered virtual companion app for emotional support and conversation.
AI observability and evaluation platform that helps developers build, test, and monitor LLM-powered applications.
Portola's Conversation quality review is part of this use case:
Related implementations across industries and use cases
Prompt iterations rubber-banded: each engineer's fix overcorrected the last. Evals are now a merge requirement—no eval, no commit.
Repeated prompts overfitted the agent, hiding regressions. Evals score for equivalence — the same app can be built in multiple valid ways.
Steep learning curves caused high user churn. Now, a native AI agent lets builders fluidly mix text prompts and point-and-click tools.
Prompt iterations rubber-banded: each engineer's fix overcorrected the last. Evals are now a merge requirement—no eval, no commit.
Repeated prompts overfitted the agent, hiding regressions. Evals score for equivalence — the same app can be built in multiple valid ways.
Three people, three different AI answers. Grounded data sources fixed that — finance now has years of vendor spend at its fingertips.
Thousands of tables tracking the same metrics differently. No one knew which to trust — not humans, not AI. One governed layer changed that.
Tournaments running simultaneously meant an hour of manual checks each. AI agents now run them in minutes, freeing the team to be proactive.
Large AI training jobs meant fighting for preemptible slots or leaving campus. Marlowe gave any lab guaranteed multi-node access on demand.
Fixing AI tone quirks meant engineers in the loop every time. Now a writer and researcher test and ship prompt fixes themselves.
An AI companion app builder focused on authentic, non-romantic emotional connections through voice conversations and complex memory systems.
Conversation quality, emotional intelligence, and authentic behavior were too subjective and nuanced to capture through automated evals alone,...
“How will humanity build healthy relationships with AI? That's the question Tolan explores.”
AI-powered virtual companion app for emotional support and conversation.
AI observability and evaluation platform that helps developers build, test, and monitor LLM-powered applications.
Portola's Conversation quality review is part of this use case:
Related implementations across industries and use cases
Prompt iterations rubber-banded: each engineer's fix overcorrected the last. Evals are now a merge requirement—no eval, no commit.
Repeated prompts overfitted the agent, hiding regressions. Evals score for equivalence — the same app can be built in multiple valid ways.
Steep learning curves caused high user churn. Now, a native AI agent lets builders fluidly mix text prompts and point-and-click tools.
Prompt iterations rubber-banded: each engineer's fix overcorrected the last. Evals are now a merge requirement—no eval, no commit.
Repeated prompts overfitted the agent, hiding regressions. Evals score for equivalence — the same app can be built in multiple valid ways.
Three people, three different AI answers. Grounded data sources fixed that — finance now has years of vendor spend at its fingertips.
Thousands of tables tracking the same metrics differently. No one knew which to trust — not humans, not AI. One governed layer changed that.
Tournaments running simultaneously meant an hour of manual checks each. AI agents now run them in minutes, freeing the team to be proactive.
Large AI training jobs meant fighting for preemptible slots or leaving campus. Marlowe gave any lab guaranteed multi-node access on demand.