Real Delegation with CrewAI: Going Beyond the Hierarchical Process

Building Lucifer: an open-source AI company you can watch · ⭐ GitHub

🚀 Start here: 1. Meet Lucifer · 2. Why I built it · 3. Architecture

📖 Use it: 4. Getting started · 5. Connect an LLM · 6. Office tour

🛠 Build the engine: 7. Org chart in YAML · 8. Real delegation · 9. Sandboxed tools · 10. Simulation brain

🎨 Build the office: 11. 3D office with R3F · 12. WebSocket + zustand · 13. Visualizing workflow

🧭 What's next: 14. Roadmap · 15. Contributing

CrewAI gives you two ways to run a team: Process.sequential, where tasks run one after another, and Process.hierarchical, where a manager agent hands work to the others. The hierarchical process looks like the obvious fit for a company. Lucifer doesn't use it. This post explains why, and how Lucifer does multi-level delegation instead.

Why not Process.hierarchical

The hierarchical process has two properties that don't fit a company:

1. There is a single manager layer. One manager delegates to workers. A CEO who delegates to a CTO who delegates to engineers doesn't map onto it.
2. Delegation is synchronous, inside one run. The manager "asks" a worker and gets text back within the same kickoff(). The subtask never exists outside that run. There is nothing to show on a board, nothing to cancel and nothing that survives a restart.

What I wanted was closer to how people work. A manager hands out work, ends their turn, and comes back when the team has delivered. Each subtask is a real thing with an id, a status, an assignee, a result and files.

One work session, one crew

Lucifer turns CrewAI around. Instead of one crew for the whole company, every work session is its own tiny crew: exactly one agent and one task, run sequentially.

crew_agent = CrewAgent(
    role=f"{agent.title} ({agent.name})",
    goal=prompts.goal(agent),
    backstory=prompts.backstory(company, agent, session.manager(), session.delegates()),
    llm=llm,
    tools=build_tools(session),
    allow_delegation=False,          # delegation is our own tool, not CrewAI's
    max_iter=MAX_ITERATIONS,
    step_callback=on_step,           # streams thoughts to the office, checks for cancellation
)
crew_task = CrewTask(
    description=prompts.task_brief(session),
    expected_output=prompts.EXPECTED_OUTPUT,
    agent=crew_agent,
)
crew = Crew(agents=[crew_agent], tasks=[crew_task], process=Process.sequential)
output = crew.kickoff()

CrewAI still does what it's great at: the agent loop, tool calling, the LLM provider and prompt caching. Lucifer owns everything around it, from the org chart and the task queue to who works when.

Delegation is a tool call

Managers get a delegate_task tool. It doesn't call anyone. It creates a task in the engine:

def delegate(self, assignee_id, title, description, priority=None) -> str:
    self._assert_open()
    assignee = self._store.get_agent(assignee_id)
    if assignee is None:
        raise ValueError(f'No employee with id "{assignee_id}"')
    if not self._store.is_under(assignee.id, self.agent.id):
        raise ValueError(f"{assignee.name} does not report to you, so you cannot delegate to them")
    sub = self._store.create_task(
        title=title,
        description=description,
        assignee_id=assignee.id,
        created_by_id=self.agent.id,
        parent_id=self.task.id,
        priority=priority or self.task.priority,
        language=self.task.language or detect_language(f"{title}\n{description}"),
    )
    self._store.log("task_delegated", f'Delegated "{sub.title}" to {assignee.name}',
                    agent_id=self.agent.id, task_id=sub.id, target_agent_id=assignee.id)
    return sub.id

A few details matter here:

• The org chart is enforced in code, not just in the prompt. If the model tries to hand work to someone outside its branch, the tool returns an error, and CrewAI passes that error back to the model so it can correct itself.
• Priority and language are inherited. A Vietnamese request stays Vietnamese all the way down.
• The activity log records targetAgentId. That's what lets the office fly a comet from manager to report.

The tool's description tells the model what happens next: "Delegate a subtask to a direct report. You will be called again with their result when they finish." The backstory adds: "After delegating, end your turn."

Waiting, then waking up

When a manager's session ends, the engine decides the task's status in one place, _settle (simplified, without the activity logging):

def _settle(self, session):
    task = self._store.require_task(session.task.id)
    if session.outcome == "completed":
        self._wake_parent(task)
    elif session.outcome == "failed":
        self._wake_parent(task)
    elif task.status == "cancelled":
        pass
    elif session.cancelled.is_set():
        # The engine stopped mid-run: put the task back in the queue.
        self._store.update_task(task.id, status="todo")
    elif session.has_open_subtasks():
        self._store.update_task(task.id, status="waiting")
    elif session.delegated_task_ids:
        # The reports finished before this session ended: come back to merge their results.
        self._store.update_task(task.id, status="todo")
    else:
        # The brain ended without calling complete_task: treat its final answer as the result.
        session.complete(session.last_thought or "Done (no summary).")
        self._wake_parent(task)

And when any subtask finishes, done or failed, _wake_parent checks whether it was the last one:

def _wake_parent(self, task):
    if not task.parent_id:
        return
    parent = self._store.get_task(task.parent_id)
    if parent is None or parent.status != "waiting":
        return
    if all(t.terminal for t in self._store.subtasks(parent.id)):
        self._store.update_task(parent.id, status="todo")
        self.nudge()

The parent goes back to todo. On the next heartbeat its owner picks it up, and this time the brief contains the results:

# Task T-1a2b3c4d: Launch an AI note-taking app
Priority: normal
Language: English. Write everything for this task in English, ...

We need a PRD, a technical architecture and a marketing plan

## Work you delegated and the results
- T-5e6f7a8b "Technical architecture" → done
  Result: Architecture written to docs/tech/architecture.md: ...
  Files: docs/tech/architecture.md, docs/tech/api.md
- T-9c0d1e2f "Product requirements" → done
  Result: ...

Review your reports' results, read the related files if needed, then merge them and
complete the task (or delegate more work if something is missing).

Because this works the same at every level, a three-level tree needs no special case. Engineers finish and wake the CTO. The CTO merges, completes and wakes the CEO. The CEO merges everything into the final report.

The race you have to avoid

Two threads meet at the parent task: the manager's session settling it, and a report's session finishing a subtask. Without care, this can happen:

1. The CTO delegates to Gabriel, and its session starts to settle. has_open_subtasks() is true, because Gabriel hasn't finished.
2. On another thread, Gabriel finishes right now and calls _wake_parent. The parent is still in_progress, not waiting, so nothing happens.
3. The CTO's settle continues and sets the parent to waiting.

Now the parent is waiting with no open subtasks, and nobody will ever wake it. Two things prevent that:

• Settling and waking never interleave. Both run under the engine's lock, so Gabriel's wake-up runs either before the CTO's settle starts or after it has finished, never in the middle:

# Settling the status and waking the parent must happen under one lock; otherwise a
# subtask can finish just as the parent moves to `waiting`, and the parent hangs forever.
with self._lock:
    self._settle(session)

• If the reports finished first, _settle sees no open subtasks but a non-empty delegated_task_ids, and sends the parent straight back to todo so its owner returns to merge. That's the elif session.delegated_task_ids: branch above.

There is also a safety net for restarts: _wake_stalled_parents() runs when the engine starts and moves any waiting task whose subtasks are all finished back to todo.

Failure travels up, too

A subtask that fails, because the agent called fail_task or the brain crashed, still wakes its parent. The manager then sees → failed with the reason in its brief and decides what to do: delegate again, work around it, or report the gap upward. The failure is never swallowed.

Cancellation travels down

POST /api/tasks/{id}/cancel walks the subtree in one pass over the task list (no query per task), marks every open task cancelled, and signals any running session through a threading.Event. Brains check that event between steps, and the CrewAI step_callback raises to break out of the agent loop at the next step.

What you get

• Delegation of any depth, following the org chart.
• Every subtask is real: visible on the board, in the drawer's tree and on the yacht as a work link.
• Managers don't sit on a thread while they wait. A waiting parent takes up no worker, so LUCIFER_MAX_CONCURRENT is spent on agents doing actual work.
• Restarts are safe. State is in state.json, interrupted tasks go back to the queue, and stalled parents are woken.

Next

The delegate_task tool is one of seven. The next post covers the others and the sandbox they run in: Sandboxed agent tools.

The code from this post is in engine.py, session.py and brains/crew.py on GitHub.

---

Lucifer is open source under the MIT license. Clone it, run it for free with the simulation brain, and ⭐ star it if you'd like to follow along: github.com/siri2moon/lucifer.

Next in the series: Sandboxed Agent Tools: write_file, delegate_task and a Workspace Agents Can't Escape.