LangSmith Engine
LangSmith Engine
Your proactive agent engineer
LangSmith Engine analyzes production traces, groups related failures, and recommends fixes so your team can improve agent quality faster.
Helping top teams ship great agents
Three ways to build, from full harness to full control
Each package solves a different problem. Choose where you want to start, then compose the rest of the stack around it.
Detect
Automatically detect issues in production traces
Engine analyzes LangSmith traces to look for patterns in unmet user expectations or agent failures. Based on your preferences, Engine will triage high severity issues, so you can understand where there’s opportunity for improvement or poor performance that needs to be addressed.
Engine can:
- Run automatically in the background
- Cluster related traces into issues
- Prioritize issues based on severity and your preferences
Fix
Engine writes the fix
For each issue, Engine summarizes the failure mode, identifies what needs to change, and writes the prompt or code fix. If you connect your codebase, Engine can open a GitHub PR with the proposed change, ready for your team to review and merge.
Engine can:
- Write prompt and code changes based on production failures
- Show diffs with explanations
- Open GitHub PRs for review
Prevent
Prevent issues from coming back
Like a good engineer, Engine writes tests so mistakes don’t reappear. It suggests online evaluators to track whether the issue stays resolved, and recommends examples to add to your offline eval datasets so you can catch regressions as you evolve your agent.
Engine can:
- Suggest online evals for production monitoring
- Recommend examples for offline experiments
- Build a feedback loop from traces to fixes to evals
Resources for LangSmith Engine
Get started with LangSmith Engine
Introducing LangSmith Engine
How We Built LangSmith Engine | Interrupt 26
FAQs for LangSmith Engine
What model do you use for LangSmith Engine?
LangSmith Engine uses LLMs provided through LangChain. We do not support bringing your own model for Engine so that we can control the full Engine experience and deliver the best results for customers. Please read more about our subprocessors here and our notice to customers here.
What can I share with our security team on using Engine?
All model providers operate under zero data retention and are contractually prohibited from training or fine-tuning on your data.
How is Engine priced?
LangSmith Engine is a standalone agent that consumes LangChain Compute Units while it works. LCUs are a normalized unit of work that account for compute, storage, memory, and LLM usage.
Usage depends on the number of traces Engine analyzes, the depth of analysis required, and the amount of work it performs. To learn more about pricing, visit our pricing page.