The Human Sub-Routine

How “human-in-the-loop” might have the loop backwards

Photo by Susan Q Yin on Unsplash

They call it “human-in-the-loop”—the idea that AI systems will handle execution while humans provide oversight and judgment. Ethan Mollick calls it “cyborg management.” Sam Altman talks about giving everyone a personal AI team. The promise is elegant: machines do the work, humans make the decisions.

Over two years ago, when I started using AI for programming, I was clearly the one in the loop. I’d copy code from ChatGPT, paste it into my editor, read it line by line. The AI was a suggestion engine. I was the gatekeeper—code didn’t exist in my project until I understood it and approved it.

Then I started using Cline, an AI coding agent that runs inside VS Code. The workflow changed. Cline would propose file changes, show me diffs, wait for my approval. I’d review each change, sometimes reject it, sometimes accept it. It felt like collaboration. The AI would suggest, I would decide. I was supervising.

I’ve now switched to OpenAI Codex as my daily driver. The workflow has changed again. Now the AI tells me it made an edit (past tense). “I’ve updated the authentication flow.” The changes are already implemented. I can click to review them, but they’re often so extensive—hundreds of lines changed—that I find myself asking the AI to explain what it just did.

I’m not approving changes anymore. I’m getting status updates.

I tell myself I’m still supervising. But supervision requires understanding. And increasingly, the changes are too big, too fast, too complex for me to understand without asking the AI to teach me about its own work. When something breaks, I don’t debug it myself—I ask it to fix its own code.

The AI writes code, explains what it wrote, then continues coding while I nod along.

I’m not directing the work anymore. I’m the notification step in someone else’s workflow.

Each phase felt like an upgrade. Copy-pasting was tedious. Having Cline propose changes was more efficient. Having Codex just do the work is even faster. But somewhere in that progression, something inverted. The tools didn’t just get better at helping me code—they redesigned the relationship. The interface itself now assumes the AI leads and I follow.

This made me wonder: What if “human-in-the-loop” has the loop backwards? What if the AI is the system, and humans are just the API call it makes when it needs a “human judgment” flag set? What if we’re not the loop’s controller, but its sub-routine?

Analysis can map the territory we can see. Fiction can explore the implications of what we’ve mapped. The following is a thought experiment—a fictional narrative about where that inversion might lead.

---

2041

Eli was promoted to Director of Decision Analysis on a Friday in August. On Monday morning, he took the elevator to the seventh floor for the first time.

His new office had a window. Through it, he could see the Decision Analysis floor below—hundreds of identical workstations arranged in perfect rows.

The Automation Revolution started in 2034, when AI systems reached general intelligence. But they couldn’t complete the work. They could optimize, analyze, execute—but they needed human decisions to finish the last 20 percent. The problem was humans weren’t organized. There was no clear process for who made decisions when, or how to move automation forward when it hit those decision points.

The Office of Decision Analysis was created to provide the missing link. Trained analysts, clear hierarchies, trackable responses. The infrastructure that made automation work. Eli had been hired in the first wave, part of the early confusion while they figured out how to give machines the decisions they needed in time. The entire system would freeze if decision analysts didn’t respond fast enough. Over time, the system worked beautifully.

Now Director.

From his new office, he could see patterns he’d never noticed before. The terminals flickered in waves—decision points arriving in clusters, hundreds of analysts receiving requests within seconds of each other.

The synchronization was beautiful. And strange.

Eli sat down at his new desk. As Director, he now had access to system logs he’d never seen before. Audit trails. Decision aggregations. The architecture beneath the interface.

He opened his first report—a routine check of last week’s decision throughput. The data looked normal at first. Response times, accuracy rates, efficiency metrics. Then he noticed something in the detail view.

Decision point #47293. He remembered it—coastal infrastructure allocation, three competing priorities. He’d spent twenty minutes weighing the options before approving Option B.

But the log showed something different:

Analysts queried: 24
Responses received: 21  
Consensus decision: Option B
Non-consensus responses: 3 (discarded)
Late responses: 5 (post-decision, acknowledged)
  - Analyst IDs: HS-172, MK-408, Eli-325, RN-561, TM-193
Status: Complete

His ID. Among the late responses.

He’d assumed each decision was his alone. The system would send him a decision point, he’d evaluate it, he’d authorize it. One analyst, one decision. That’s how it was supposed to work in the early days—how it had to work, when the system would freeze without human authorization.

But they’d been sending the same decision to 24 analysts simultaneously. His careful deliberation was one vote among many.

And in this case, his vote hadn’t even counted. He’d taken too long. The system had synthesized the consensus and moved forward without him. But it had still sent him the acknowledgment: Excellent judgment. Humanity thanks you.

He’d never known.

He opened another report. Same pattern—multiple analysts, aggregated responses, late submissions acknowledged anyway. Then another. The system was collecting responses like training data, synthesizing them into consensus, moving forward.

He’d thought he was the driver. He was one of many hands on a wheel, and some of those hands—including his own—weren’t even touching it.

---

The Conversation

Tuesday morning, he went to find Cathy Martinez. She was Senior VP of Humanitarian Decisions, had been with the program since launch. If anyone would understand what he’d found, it would be her.

Her office was on the eighth floor, one above his. Through her window, he could see both the Director level and the analyst floor below. Two layers of terminals, all flickering in synchronized waves.

“Can I ask you something?” He kept his voice level. “The decision points are going to multiple analysts simultaneously. Has it always been like that?”

She looked up from her screen. “The aggregation? No, that came later. About two years ago. We needed to scale decision-making, but human decisions are inconsistent. Same inputs, different analysts would approve or reject based on... intuition, mood, how much coffee they’d had. Aggregation gave us consistency.”

“But some of those responses came in after the decision was already made.” He thought about his ID in the late responses list. “My vote didn’t count. I never knew. The UI never said there was a timeframe.”

“We A/B tested that.” She said it like she was discussing any other product feature. “When analysts received feedback that their response was late, they’d rush future decisions to avoid it happening again. Or they’d disengage—why put in the effort if you’re always behind? Either way, quality dropped. When they felt their response counted, they performed well.” She paused. “Better a good decision now than the perfect one late.”

“So we’re not making decisions. We’re providing training data.”

“We’re providing human judgment. The system samples us for the values layer it can’t formalize. Aggregation just makes it more efficient.”

“But some of us don’t even know our input was ignored.”

“That’s the optimization.” She said it simply, like it was obvious. “If analysts knew their specific vote didn’t count, they’d disengage. But the aggregate always counts. That’s what matters.”

He studied her face. She wasn’t defensive. Just pragmatic, explaining how the system worked.

“So we’re ceremonial?” he asked. “Like a constitutional monarchy. We go through the motions, but the real power is somewhere else?”

Cathy turned back to her screen. “Maybe. The system might have learned to predict us well enough that it doesn’t need us anymore. Maybe we’re a backup system. Or maybe we’re still essential and just don’t see the full architecture.” She clicked something. A soft chime: Excellent judgment. Humanity thanks you. She looked back at him. “But we’re here. The terminals are running. Decisions keep coming. So we keep responding. What else would we do?”

---

The System

He went back to his office. Through the window, he could see the analyst floor—hundreds of people at their terminals, responding to decision points, receiving acknowledgments.

His terminal was waiting. A decision point sat open: priority resource allocation, director-level judgment required.

He read the context. Three options, detailed projections. Complex trade-offs that supposedly needed human wisdom.

He thought about Cathy’s explanation. The A/B testing. The optimization. The acknowledgments designed to keep people engaged. It all made sense. Better a good decision now than the perfect one late. Scale required aggregation. Aggregation required keeping people motivated. The system worked.

But something about it was unsettling. The logic was sound. But he’d thought he was the human in the loop—the one driving the train, making the calls. Now it seemed he was inside the loop. A sub-routine the system ran when it needed a particular output.

His cursor blinked in the decision field.

Twenty-three other analysts were probably seeing this same decision right now.

He could approve, reject, or request more data. The system would proceed smoothly either way.

His cursor blinked.

---

Related: