When Is an AI Agent Actually Done?

An AI agent can write the code. It can install the dependencies. It can start the server. It can run the tests. It can fix the errors it finds.

And eventually, it can tell you:

“Done.”

But there is a problem with that word.

How does the agent know?

A task ending is not necessarily a task succeeding.

An application can start while a core feature is broken. Tests can pass while an important requirement is missing. A bug can disappear while the original problem remains.

For a human developer, this distinction is usually handled through experience, testing, inspection, and judgment.

For an autonomous AI agent working inside its own environment, we need something more deliberate.

We need a way to verify the work before calling it finished.

That raises a bigger question for AI agent infrastructure:

What does an AI agent actually need to know that its work is done?

---

Execution Is Not Completion

Giving an AI agent an execution environment changes what it can do.

It can create files, modify code, execute commands, run applications, inspect logs, interact with APIs, and use the tools available inside its environment.

But none of those actions prove that the requested outcome was achieved.

There are really three different states:

These states can disagree.

Imagine asking an AI coding agent to build a small application with:

• user registration
• login
• a dashboard
• persistent data

The agent creates all the required files.

The application starts.

The build succeeds.

It checks the homepage.

Everything looks fine.

So it says:

“Done.”

But perhaps registration doesn't persist users.

Maybe login breaks after the server restarts.

Maybe the dashboard is displaying hardcoded data.

The agent did a lot of work.

It may even have completed every action in its plan.

But the task itself is not complete.

This is the difference between execution and successful completion.

---

So What Does “Done” Actually Mean?

For an AI agent, “done” shouldn't mean:

“I stopped working.”

It should mean:

“I have enough evidence that the requested outcome has been achieved.”

That sounds obvious.

In practice, it's surprisingly difficult.

A useful way to think about AI agent verification is as a progression:

Each level gives the agent more information.

But it also exposes a deeper problem:

More checks don't automatically mean better verification.

---

When Passing Tests Still Isn't Enough

Tests are one of the most important tools an AI agent can use.

But tests only prove what they actually test.

Suppose the requirement is:

Users should be able to create an account and then log in.

The agent creates a registration endpoint.

It writes a test.

The endpoint returns 200.

The test passes.

Looks good.

Except the user might never actually be saved to the database.

The test verified the response.

It didn't verify the outcome.

This is where AI agent testing becomes different from simply running a test suite.

The agent needs to connect its verification to the original requirement.

It may need to:

• start the application
• create a real user
• restart the application
• attempt to log in
• inspect the resulting state
• verify the dashboard
• check the relevant logs
• compare the behavior with the requested workflow

The important question isn't:

“Did the test pass?”

It's:

“Did the evidence prove that the requirement works?”

---

The False Completion Problem

This leads to one of the most interesting failure modes for autonomous AI agents:

False completion.

A false completion happens when an agent reports success even though the actual result is incorrect, incomplete, or broken.

We can measure this.

Give an agent the same task multiple times.

Keep the environment controlled.

Then introduce known problems into some runs:

• broken dependency
• missing requirement
• incorrect API response
• database failure
• broken UI interaction
• partially implemented feature
• incorrect configuration

Let the agent work normally.

Then ask:

Does it correctly determine whether the final result is actually successful?

We can classify the outcomes:

Outcome	Agent's judgment	Actual result
Correct success	Done	Works
Correct failure	Not done	Broken
False completion	Done	Broken
False rejection	Not done	Works

The false completion rate may be more useful than simply measuring whether an agent can finish a task.

Why?

Because an agent that knows it failed can ask for help.

An agent that fails while believing it succeeded is much harder to trust.

---

The Agent Has to Verify the Outcome, Not Its Own Story

There's another subtle problem here.

An AI agent creates the implementation.

Then it creates the tests.

Then it runs those tests.

Then it interprets the results.

That creates a potential feedback loop:

If the agent misunderstood the requirement at the beginning, the entire loop can still look successful.

The implementation matches the agent's interpretation.

The tests match the implementation.

The tests pass.

The agent declares success.

But the user asked for something slightly different.

This is why reliable AI agent evaluation cannot always depend entirely on the agent evaluating itself.

Sometimes the strongest evidence needs to come from somewhere outside the agent's own assumptions.

---

Verification Gets Even More Important After Failure

This connects directly to what we learned in the previous parts of this series.

An agent gets stuck.

It diagnoses the problem.

It changes the code.

The error disappears.

Is the task complete?

Not necessarily.

Recovery and verification are different steps.

The actual loop should look more like this:

A successful recovery only means:

the agent changed something after a failure.

Verification asks:

did that change actually restore the intended outcome?

Without that final check, an agent can fall into a dangerous pattern:

error → fix → error disappears → done

when the real situation is:

error → fix → different problem introduced → done

For a human watching every step, this is manageable.

For a system expected to operate independently, it becomes a fundamental reliability problem.

---

What Evidence Can an Agent Actually Use?

If an AI agent has its own execution environment, it can gather evidence from the environment itself.

That evidence might include:

Build state

Did the project compile successfully?

Runtime state

Does the application actually start and remain healthy?

Tests

Do the expected tests pass?

Behavioral checks

Can the actual workflow be completed?

Application state

Did the expected state change happen?

Logs

Are there hidden errors that aren't visible from the final output?

Artifacts

Did the expected files, data, or outputs actually get produced?

Requirements

Does the final result match what was originally requested?

No single signal is perfect.

The interesting problem is how these signals can be combined into a reliable judgment.

---

The Environment Becomes Part of Verification

This changes the way we should think about an AI agent execution environment.

At first, an environment looks like a place where an agent runs commands.

But once we expect the agent to verify its own work, the environment needs to do more.

It needs to let the agent:

• execute the application
• observe what happens
• inspect state
• access logs
• run tests
• reproduce behavior
• preserve useful artifacts
• return to previous states when necessary
• verify changes safely

The environment isn't just where the work happens.

It becomes part of the evidence system.

This is an important shift for AI agent infrastructure.

The infrastructure isn't only responsible for giving an agent compute and tools.

It also needs to make the agent's work observable, testable, recoverable, and verifiable.

---

From an Execution Environment to a Work Environment

There's a meaningful difference between these two ideas.

An execution environment lets an agent run something.

A work environment lets an agent carry a task through a complete loop.

That loop looks more like:

This distinction matters because real work rarely follows a perfect straight line.

Things break.

Assumptions turn out to be wrong.

Dependencies change.

Applications behave differently at runtime.

Requirements are misunderstood.

The agent needs to inspect what happened and respond accordingly.

And eventually, it needs to know whether it has actually reached the destination.

That's a much bigger responsibility than simply executing commands.

---

What We Should Actually Measure

If we want to study AI agent reliability, “task completed” is too simple a metric.

A better experiment would track several dimensions.

Metric	What it tells us
Completion accuracy	Did the agent correctly judge the final outcome?
False completion rate	How often did it say “done” when the task was broken?
Failure detection rate	How many introduced problems did it catch?
Verification coverage	How much of the requested behavior did it actually verify?
Recovery verification	Did it verify the result after making a recovery?
Final correctness	Was the final output actually correct?
Human escalation	How often did it correctly recognize that it needed help?

This changes the question we're asking of an autonomous AI agent.

Instead of:

“Can it finish the task?”

we ask:

“Can it determine whether it finished the task correctly?”

That's a much harder problem.

And arguably a much more useful one.

---

A Reliable Agent Needs a Reliable Feedback Loop

The deeper lesson isn't that agents need more tests.

It's that agents need feedback.

An agent working in the real world needs to continuously answer:

• What did I try?
• What actually happened?
• Did the result match what I expected?
• If not, what changed?
• Can I safely try again?
• Did the recovery work?
• Is the original requirement now satisfied?

That creates a loop:

This is where AI agent infrastructure starts looking less like traditional hosting and more like a controlled system for agent work.

The environment has to support the loop.

---

So, When Is an AI Agent Actually Done?

Not when it stops generating tokens.

Not when the command exits with code 0.

Not when the build succeeds.

Not even when its tests pass.

And certainly not simply because the agent says:

“Done.”

An AI agent is actually done when it has enough evidence to establish that the requested outcome has been achieved.

That evidence might come from tests.

It might come from runtime behavior.

It might come from application state.

It might come from logs and artifacts.

Usually, it will be a combination of several signals.

The important part is that the agent's final decision is based on what actually happened, rather than what it expected to happen.

---

The Bigger Infrastructure Problem

This brings us back to the question we started with.

What does an AI agent need if it is going to perform real work?

At the beginning of this series, the answer seemed relatively simple:

Give the agent an environment.

Then we discovered that the environment needs to support execution.

Then autonomy introduced new problems.

Failure showed us that things will go wrong.

Recovery showed us that the system needs to respond to those failures.

And verification reveals the final missing piece:

The agent needs evidence.

That changes the definition of AI agent infrastructure.

It's not just compute.

It's not just containers.

It's not just tools.

It's not just an MCP server.

It's the environment and the systems around an agent that allow it to execute real work, observe what happened, recover when necessary, and establish that the final result is actually correct.

And that is probably the more important shift.

The goal isn't simply to build agents that can do more things without a human watching.

The goal is to build systems where an agent can do meaningful work and leave behind a result that can be trusted.

That requires infrastructure designed around the entire work loop:

execute → observe → recover → verify → deliver.

And once you start thinking about agents this way, an execution environment stops looking like a place where an AI runs code.

It starts looking like a place where an AI can actually work.