Orchestrator, Not Autopilot: How ReconForge Runs LangGraph Agents in Parallel (Part 2)
Photo by Jon Handley / Unsplash

Orchestrator, Not Autopilot: How ReconForge Runs LangGraph Agents in Parallel (Part 2)

Have you ever worked with someone brilliant who had no manager? They start five things, finish two, and when you ask what happened to the other three you get a very confident story and nothing you can check. Now imagine that person is an AI, it has a shell, and it is pointed at a real target.

In Part 1 I drew the map of ReconForge: three layers, a small team of agents, and one rule about evidence. I also promised to explain how the middle layer keeps those agents in line. So this post is about the brain, and about one decision I would make again every time: ReconForge runs on an orchestrator, not on autopilot.

Autopilot Sounds Great Until You Have to Explain It

Autopilot is the popular design right now. You give one agent a goal, a pile of tools, and a loop. It decides what to do, does it, looks at the result, and decides again until it thinks the job is finished.

For a weekend project, that is honestly fun to watch. For a security assessment, I think it is the wrong shape. Ask an autopilot run three simple questions and see how it goes:

  • What ran, and in what order?
  • Why did it stop when it stopped?
  • If I run it again tomorrow, will it take the same path?

With a free running loop, the honest answer to all three is "it depends on what the model felt like." That is fine for brainstorming. It is not fine when a client is going to read the report.

So I flipped it around. The workflow is fixed and written down. The AI gets freedom inside each step, not over the steps themselves.

The Head Chef Does Not Cook

Remember the restaurant from Part 1? The orchestration layer was the kitchen. Let me zoom in on one person in that kitchen: the head chef standing at the pass.

The head chef does not chop, fry, or plate. They read the tickets, call out who starts what, check each dish as it comes back, and decide when the table is ready to be served. If the chef started cooking every dish personally, the whole kitchen would slow down to the speed of one pair of hands.

That is the orchestrator. It never runs a security tool and it never reads raw terminal output. Each agent does its own work and comes back with a short, structured summary: here is what I found. The orchestrator reads summaries, not noise. In my view, that one habit is what keeps it calm while five agents are busy around it.

One Question, Asked Over and Over

Here is the part I like most. The orchestrator is not following a script that says "step one, then step two, then step three." It keeps asking a single question:

Who has what they need to start?

orchestrator loop
The orchestrator loop: ask who is ready, send them off, collect their summaries, and look again until everyone is done.

It is a loop. Check who is ready, send them off, collect what they bring back, look at the board again. Nobody is ready at the very start except Recon, so Recon goes first. Once Recon comes back with a list of assets, Enumeration suddenly has what it needs. Once Enumeration finishes, two more agents are unblocked at the same moment.

I built this on LangGraph, which lets you describe the run as a graph of steps and then send several of them off at once. Stripped down to the idea, the readiness check looks like this:

def who_is_ready(state):
    ready = []

    if nothing_has_started(state):
        ready.append("recon")

    if state.discovered_assets and is_pending("enumeration"):
        ready.append("enumeration")

    # Two agents, same input, no reason to make one wait
    if state.enriched_assets:
        ready += still_pending("vulnerability", "compliance")

    # Reporting only goes when everyone else is finished
    if everyone_else_is_done(state) and is_pending("reporting"):
        ready.append("reporting")

    return [Send(agent, state) for agent in ready]

Notice there is no timer and no hardcoded order in there. An agent starts because its ingredients arrived, not because the clock said so.

Parallel Where It Is Safe, In Order Where It Matters

"Run the agents in parallel" sounds like an obvious win, but blind parallelism is a trap. If the Vulnerability agent starts at the same time as Recon, it has nothing to look at. You have just paid for an AI to stare at an empty table.

So the rule is simple: parallel where it is safe, in order where it matters.

Vulnerability and Compliance both need the same enriched asset list, and neither cares what the other is doing. They run side by side. If the target turns out to be several hosts, each of them can split the work per host too, so one slow machine does not hold up the rest.

Reporting is the opposite. It waits for everyone, every time. I would argue a report written from half the picture is worse than a report that arrives ten minutes later.

When One Agent Has a Bad Day

Agents fail. A model times out, a tool hangs, an answer comes back as nonsense. If you are building anything with LLMs, I think you should assume this from day one and design for it.

failure isolation
One agent failing costs one section of the report, not the run. The others finish, and Reporting is told who made it.

In ReconForge, one agent failing does not take the run down with it. Say Compliance falls over halfway through. The orchestrator marks it as failed, tells the UI, and carries on. Vulnerability finishes its work as if nothing happened. Then Reporting starts, and it is told exactly who finished and who did not, so it writes an honest report from what is actually there.

There is one more case worth handling. If an agent fails before producing anything, whoever was waiting on its output would wait forever. So the orchestrator closes those out as well. Nothing sits in "pending" until someone notices on Monday.

And there is a short list of problems that do stop everything: the person cancelled the run, or the AI provider itself is down. If the model is unreachable, every agent after this one is going to fail the same way. Marching on would only produce a longer list of identical errors.

What I Learned

A fixed workflow is not a limitation, it is the product. The predictable path is exactly what makes a run explainable. The agents still get to be clever, just inside their own lane.

Ask "who is ready?" instead of "what is next?" A readiness check gave me ordering and parallelism from the same few lines. I never had to write a schedule.

The orchestrator should read summaries, not terminals. The moment the coordinator starts digesting raw output, it becomes the slowest and most confused part of the system.

Decide what a failure costs before it happens. One agent failing should cost you one section of the report, not the whole run. To me, that was the difference between a demo and something people can rely on.

What's Next?

So the orchestrator hands out the work and keeps everyone in line. But what happens inside a single agent? How do you get a language model to hand back something a program can actually trust, instead of a nice paragraph?

That is Part 3: making an LLM return something you can trust.

Till next time! ✌️

Stay in the loop

Subscribe to get the latest posts delivered right to your inbox.

Subscribe