tool_dispatch.h File Reference

The single implementation of "execute an assistant message's tool_calls". More...

#include <neograph/api.h>
#include <neograph/types.h>
#include <neograph/tool.h>
#include <asio/awaitable.hpp>
#include <functional>
#include <memory>
#include <optional>
#include <string>
#include <vector>

Include dependency graph for tool_dispatch.h:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Classes
struct	neograph::ToolDecision
	A gate's verdict on one tool call. More...
struct	neograph::ToolGateContext
	What a ToolGate is allowed to know about the run it is gating. More...

Typedefs
using	neograph::ToolGate = std::function< asio::awaitable< ToolDecision >(ToolCall, ToolGateContext)>
	Called once per tool call, before any tool runs.

Functions
NEOGRAPH_API asio::awaitable< std::vector< ChatMessage > >	neograph::dispatch_tool_calls (std::vector< ToolCall > calls, std::vector< Tool * > tools, ToolGate gate={}, ToolGateContext gctx={}, ToolExecutionContext execution={})
	Execute every tool call in calls and return one tool message each.

Detailed Description

The single implementation of "execute an assistant message's tool_calls".

Before this existed the concept lived in two places — ToolDispatchNode (graph path) and Agent (standalone path) — and they had drifted: the node fanned the calls out concurrently, the agent ran them one at a time through the blocking execute() facade. Measured on three 300 ms async tools: 300 ms via the node, 900 ms via the agent (issue #87).

The performance gap was the symptom. The defect was the duplication: every capability added at this boundary — interception, per-call interrupt, token accounting — would have landed in one path and silently missed the other, exactly as concurrency did. Both callers now route through the function below and nowhere else.

Concurrency contract. Calls are launched together, so a tool that really suspends (one that overrides execute_async) overlaps with its siblings and may be entered concurrently with itself if the model asked for it twice. A tool that implements only the sync execute() never overlaps: the default Tool::execute_async body runs to completion before the coroutine's first suspension, so the calls serialize even inside the parallel group. That is why unifying the two paths cannot introduce a data race into an existing sync tool — see ToolDispatchParity.SyncToolsDoNotOverlapOnToolNode, which pins the behavior at 900 ms for three 300 ms sync tools.

Definition in file tool_dispatch.h.

Typedef Documentation

ToolGate

using neograph::ToolGate = typedef std::function<asio::awaitable<ToolDecision>(ToolCall, ToolGateContext)>

Called once per tool call, before any tool runs.

Awaitable, so a gate may do real work — ask a policy service, hit a database, prompt over a socket — without blocking the executor.

Set it through EngineConfig::tool_gate or GraphEngine::set_tool_gate (graph), or Agent::set_tool_gate (standalone). Unset means every call runs, exactly as before this existed.

Definition at line 146 of file tool_dispatch.h.

Function Documentation

dispatch_tool_calls()

NEOGRAPH_API asio::awaitable< std::vector< ChatMessage > > neograph::dispatch_tool_calls	(	std::vector< ToolCall >	calls,
		std::vector< Tool * >	tools,
		ToolGate	gate = {},
		ToolGateContext	gctx = {},
		ToolExecutionContext	execution = {}
	)

Execute every tool call in calls and return one tool message each.

Results come back in call order, not completion order, so the caller's message history is deterministic regardless of which tool finished first.

Failure contract (the same for one call and for a batch; see "Tool failure contract" in the reference):

• An ordinary per-call failure never throws. A call naming an unregistered tool (rejected), an unparseable argument payload, a tool whose lookup or execution throws, or a typed controller result such as timed_out all yield a tool message with tool_status set and content {"error", "status", "retryable", "effect_uncertain"}: the model sees the failure and can react to it.
• Run control signals are never turned into a tool message. A HookBoundaryBlocked (from a lifecycle Hook or from tool code), a NodeInterrupt (from the gate or from tool code) and cancellation of execution.cancel_token propagate with their original type and fields. In a batch the other calls run to completion first; if several calls stop the run, cancellation wins, then a Hook block, then an interrupt, then the earliest call.
• With an effect broker, a call whose effect is uncertain pauses the run with a NodeInterrupt whose payload kind is tool_effect_reconciliation.
• A call that would start at or after execution.deadline does not start: it yields a tool message with status expired and error run deadline elapsed before the tool started, and none of its Hooks run.
• With execution.hook_runtime, every call that starts crosses one BeforeToolExecution and one AfterToolExecution lifecycle event. Every caller supplies the same runtime here: llm::Agent its own, a graph node the run's (make_tool_execution_context).
• Tool output and failure text reach the model as well-formed UTF-8 (each ill-formed subsequence becomes U+FFFD).

Parameters

calls	Tool calls from the assistant message. Taken by value: this is a coroutine, and a reference parameter would dangle across the first suspension.
tools	Non-owning tool pointers to resolve names against. Also by value, for the same reason. The pointees must outlive the returned awaitable.
gate	Optional interception point (issue #89). Consulted for every call before any tool runs — see below. Default-constructed (empty) means no gating, and the behaviour is bit-for-bit what it was before.
gctx	What the gate is told about the run.
execution	Per-invocation context: cancel token, deadline, controller, identity, effect broker and lifecycle HookRuntime. Graph nodes build it with make_tool_execution_context(); llm::Agent fills it from its own configuration.

Exceptions

NodeInterrupt	when the gate returns Interrupt for any call, or tool code raises one. On the graph path the engine catches it, checkpoints, and hands the caller a paused RunResult. On the Agent path — which has no checkpoint machinery — it surfaces to the caller of Agent::run.
HookBoundaryBlocked	when a mandatory Hook blocks a call or tool code raises one.
graph::CancelledException	when the run's cancel token is cancelled.

Gating happens before execution, and that ordering is the design. A permission gate that runs the command and then asks is not a permission gate. Concretely: three calls, the middle one needs approval. Were the gate consulted after execution, the siblings would already have had their effects; the run pauses, the human approves, the engine resumes — and resume re-enters the node from the top, because a node that interrupted recorded no writes. The siblings run a second time. Approving an rm -rf would have double-committed the git commit beside it. So every call is decided first, while nothing has happened yet, and only then does anything run.

References neograph::dispatch_tool_calls().

Referenced by neograph::dispatch_tool_calls().