Can You Trust What the AI Just Told You?

An AI just answered a question about your business. Confidently. In complete sentences. With that calm, certain tone they all use.

Was it right?

That tone is the most dangerous thing about this whole technology, because it sounds exactly the same when the answer is wrong.

Hi! I am Bob Michaels, and this is Evolving the Web. Every two weeks I talk through the essays I just published, plainly, as one story. The last two weeks were about trust. What it takes to build AI systems you can actually rely on at work, where wrong answers cost money.

The first essay plants the flag for everything else, and the flag is one word. Grounded.

People use grounded to mean the AI can search documents before answering. That is plumbing. Plumbing is not the promise.

Here is the promise, and it is a contract with exactly three outcomes. Every answer either cites real evidence it actually read. Or it says plainly that it is not sure. Or it declines to answer. One of those three, every time, with a log of which one happened.

Cite, hedge, or decline. Nothing else.

Now here is why the contract matters. In one study last year, researchers ran sixteen hundred news questions through eight consumer AI search tools. More than sixty percent of the answers were wrong. And here is the detail that should stop you cold. One major chatbot, across the whole study, never once declined to answer.

Never once said I do not know.

A system that cannot say I do not know is not confident. It is unsafe. So here is the test I give every so called grounded system, and you can steal it. Ask it something its documents cannot possibly answer. The honest system declines. The dangerous one answers anyway, in that same calm voice.

The second essay is about where your company's knowledge should live when you build one of these systems, and there are two honest options.

Option one, the librarian. Your documents stay in a library, and when a question comes in, the system fetches just the relevant pages and answers from those. Option two, the open book exam. The newer models can hold enormous amounts of text at once, so you skip the librarian and hand the model the whole binder every time.

The vendors will happily sell either one as the answer. The truth is that the choice is not about fashion. It is about six boring questions. How much material is there. How often does it change. Does everyone have permission to see all of it. Do answers need citations. What does each approach cost. How fast must it respond.

Small pile of documents, stable, everyone allowed to read everything? The open book is simpler and genuinely fine. Big pile, changing weekly, where the intern must not see the executive folder? You need the librarian, because the librarian can check a library card. Most real companies end up with some of both, on purpose.

The only wrong answer is picking the architecture because a vendor said the old one is dead.

The third essay is a warning I wish more companies would hear before the invoice arrives, and it is this. One model is not a strategy.

Here is the fact almost nobody prices in. The AI models your systems are built on retire, on the vendor's calendar, not yours. The documented notice can be as short as a couple of weeks for preview models. The one your product depends on has a retirement date coming. You just do not know it yet.

You would never build a factory on equipment the landlord can haul away with two weeks notice and no replacement guaranteed to behave the same. But that is precisely the deal most companies have signed with their AI vendor, without reading it as a deal.

The fix is not paying five vendors at once. That is a spending habit, not a strategy. The fix is knowing your exit works. Write down what the model actually does for you, so a replacement can be tested against the job instead of against vibes. Know every place your code names a specific model. Keep your test results somewhere you own, outside every vendor's walls. And once a quarter, run your tests against one alternative model.

Staying with one provider is often the right call. Being unable to leave is not a call. It is a trap you can walk out of for the cost of one test a quarter.

The fourth essay is where it gets serious, because it is about agents. Not AI that answers questions. AI that does things. Sends the message, files the ticket, moves the money.

Every agent demo looks fantastic, and the demo proves almost nothing. The demo shows the happy path. Production is where the other paths live. The ambiguous request. The tool that times out halfway through. The task that half finished. The clever user trying to talk the agent into something it should never do.

So before an agent touches production, you test four kinds of situations, and the fourth is the one everyone skips. Normal cases, the job it was hired for. Edge cases, the weird but legitimate stuff. Adversarial cases, someone actively trying to trick it. And forbidden cases. You affirmatively test that the agent refuses the things it must never do. You do not hope. You check, and you write it down.

Then, and this is the sentence I most want you to keep, you score two numbers and you never blend them. Did it finish the job. And did it follow the rules. Separate numbers, forever, because an agent that finishes the job by breaking a rule did not succeed. It failed in the most expensive possible way, quietly.

The two week arc in one breath.

A trustworthy system cites, hedges, or declines, and logs which. Where knowledge lives is six boring questions, not a fashion choice. One model is not a strategy, and the exit costs one test a quarter. And an agent earns production by proving what it refuses to do, not by acing its own demo.

Every one of those is the same sentence in different clothes. Trust is not a feeling about the technology. It is a contract you can check.

The full essays are on the blog at bobmichaels dot ai, sources and checklists included, linked from the episode page.

Building systems that hold up under exactly these tests is what I do, as a consultant or fractionally. And the site itself is the demo. There is a chat window on every page, grounded the way this episode describes. Ask it something it cannot know, and watch it tell you so.

Thanks for listening. This week, ask your calmest, most confident AI something you know it cannot answer. What it does next tells you everything.