An agent that works in a demo and an agent you can let near a customer are different pieces of software. The difference is guardrails, evaluation and a handoff to a person.
.svg)
Agents are the newest thing on this site and the least settled, so we are going to be more careful here than a sales page usually is. These are the six failures we see most.
Demoed against clean examples and it looked close to magic. Against real messages, with typos, attachments, three questions at once and someone changing their mind halfway, it comes apart.
The happy path was built and the distribution of real inputs was never sampled. Before building, we pull a few hundred real cases and look at the tail, because the tail is where an agent either earns trust or destroys it.
The agent has been handed a set of tools and broad instructions. Nobody has enumerated what it must never do, so nobody can promise it will not do it.
Capability was scoped and authority was not. Those are different things. The useful discipline is defining the smallest set of actions that accomplish the job, and making anything outside that set impossible rather than discouraged.
When the agent cannot resolve something it either guesses or dead-ends. Either way the customer is stuck and nobody is watching.
Escalation was treated as an edge case rather than a core path. Knowing when to stop is as much a design problem as knowing what to do, and an agent that hands over cleanly at 20% of cases is worth far more than one that guesses at 100%.
Changes to the prompt or the tools get made on the strength of a few spot checks. Whether last week’s change helped is genuinely unknown.
There is no evaluation set. Without a fixed set of real cases with known good outcomes, every change is a guess and regressions are invisible. Building that set is unexciting and it is the difference between engineering and vibes.
In testing it cost almost nothing. At production volume, with retries and long context, the monthly bill is a line item somebody has to defend.
Multi-step agents can make many model calls per task, and cost scales with steps rather than with requests. We model cost per completed task at real volume early, because it occasionally rules a use case out and that is much cheaper to learn in week two.
The agent lives in a separate chat window. The work lives in your CRM, your ticketing system and your database, so somebody is still copying between them.
Integration is most of the engineering and none of the demo. An agent that cannot read and write the systems where work actually happens is a chatbot with extra steps, whatever it is called.
Six pieces. The model is one of them, and it is not the one that decides whether this works.
Contact us
Deciding what the agent does, and more importantly what it must never do. We sample a few hundred real inputs first, because the shape of the tail determines whether the job is viable at all.
%201%20(2).svg)
Connecting the agent to the systems where work actually happens: CRM, ticketing, databases, internal APIs. This is most of the engineering and it is what separates an agent from a chat window.
.svg)
Constraining actions so the unsafe ones are impossible rather than discouraged, with the scope of what the agent may touch defined explicitly and enforced in code.
A fixed set of real cases with known good outcomes, so a change can be measured instead of guessed at. Unexciting, and the thing that makes iteration possible rather than superstitious.
Designing the moment the agent stops. When it escalates, what context the person receives, and how the outcome comes back. An agent that hands over well beats one that guesses.
Watching what it does in production, what it costs per completed task, and what changed when a provider updated a model underneath you. Agents drift, and the drift is invisible without this.
Being straight again: VAULT publishes no named AI agent case study, so we are not going to show you one. Agents are new enough that anyone showing a decade of them is showing you something else. What we can show is the production engineering that agents have to be built on top of.
Integrations, data quality and knowing what a system does when it fails are not agent-specific skills, which is precisely the point. Most of what makes an agent trustworthy is ordinary software engineering done carefully.
Three steps to a plan, then phased delivery, run from our Chicago office. For agents the first phase is almost always a narrow pilot against real inputs with an evaluation set attached.
How we handle scope changes. Scope moves on almost every software project, and pretending otherwise is how budgets get broken quietly. We price and sequence in phases so a change is a decision you make at a milestone with the cost in front of you, rather than something you discover in an invoice. Our Clutch reviews are public, including the ones that mention budget and schedule, and we would rather you read them before the call than after it.
You talk to product people and an engineer rather than a sales rep, often the same day you get in touch. We want to hear the problem before the feature list, because the feature list is usually somebody’s first guess at a solution and it is worth testing.
You get a custom development plan within a week: scope, sequence, the team assigned, what the first milestone proves and what it costs. The plan is yours whether or not you continue with us, which is deliberate. It is hard to compare quotes when nobody will write down what they actually mean.
Phased execution with the named engineers who will do the work, not a bench you never see again after the pitch. You get a working increment early rather than everything landing at the end, because that is what makes a change of direction affordable instead of catastrophic.
For moving scope or ongoing support. Billed by time worked, so you can stop any time. Harder when you need a fixed budget first.
For a build with a clear finish line. One price per phase, agreed up front. Harder when each mid-phase change means a re-quote.
For engineers working inside your own team. Billed monthly per engineer, rolling with notice. Harder when nobody on your side can direct the work.
For discovery, a first release, then a live roadmap. Fixed to launch, then hourly or monthly. Harder for projects under about six months, where the model is overkill.
Start narrow. A short conversation with an engineer about one specific job, and if ordinary automation is the better answer we will tell you that instead. Or call 312‑298‑9598, or visit us at 20 N Wacker Dr in Chicago.
Contact us
Connect with one of our product specialists today.
.svg)
.svg)
