neograph::graph::GraphEngine Class Reference

Super-step loop execution engine for graph-based agent workflows. More...

#include <engine.h>

Public Member Functions
GraphAdmin	admin () noexcept
	Borrow the state-administration facade for this engine.
void	clear_node_cache ()
	Drop all cached entries (per-node enable state preserved).
std::string	fork (const std::string &source_thread_id, const std::string &new_thread_id, const std::string &checkpoint_id="")
	Fork a thread, creating a new thread from an existing checkpoint.
const std::string &	get_graph_name () const
	Get the graph name (from the JSON definition).
std::optional< json >	get_state (const std::string &thread_id) const
	Get the current state for a thread.
std::vector< Checkpoint >	get_state_history (const std::string &thread_id, int limit=100) const
	Get the checkpoint history for a thread (time-travel).
std::shared_ptr< Store >	get_store () const
	Get the cross-thread shared memory store.
std::optional< NestedCheckpoint >	inspect_nested_checkpoint (const std::string &root_thread_id, const std::vector< SubgraphPathStep > &path, std::shared_ptr< CheckpointStore > run_checkpoint_store={}) const
	Inspect a descendant's checkpoint by graph path and exact node invocation.
const NodeCache &	node_cache () const
	Borrow the engine's NodeCache for stats inspection.
std::optional< int >	resolved_attempt_timeout_ms (const std::string &node_name) const
	The attempt timeout node_name runs with, in milliseconds, after the resolution described at set_node_attempt_timeout_ms(); none when the node has no timeout.
RetryPolicy	resolved_retry_policy (const std::string &node_name) const
	The retry timing node_name runs with after the per-field resolution described at set_node_retry_policy().
RunResult	resume (const RunConfig &config, const json &resume_value, const GraphStreamCallback &cb, const RunMetadata &metadata)
	Resume with metadata for this new execution attempt.
RunResult	resume (const RunConfig &config, const json &resume_value=json(), const GraphStreamCallback &cb=nullptr)
	Resume while preserving caller cancellation settings.
RunResult	resume (const std::string &thread_id, const json &resume_value=json(), const GraphStreamCallback &cb=nullptr)
	Resume execution from a HITL interrupt.
asio::awaitable< RunResult >	resume_async (const std::string &thread_id, const json &resume_value=json(), const GraphStreamCallback &cb=nullptr)
	Async peer of resume — non-blocking coroutine surface.
asio::awaitable< RunResult >	resume_async (RunConfig config, json resume_value, GraphStreamCallback cb, RunMetadata metadata)
	Async resume peer with caller-supplied execution metadata.
asio::awaitable< RunResult >	resume_async (RunConfig config, json resume_value, GraphStreamCallback cb, RunMetadata metadata, RunResources resources)
	Async resume peer with metadata and invocation-scoped resources.
asio::awaitable< RunResult >	resume_async (RunConfig config, json resume_value=json(), GraphStreamCallback cb=nullptr)
	Async peer of the RunConfig-preserving resume overload.
RunResult	resume_from (RunConfig config, std::string checkpoint_id, json resume_value={}, GraphStreamCallback cb={}, RunMetadata metadata={}, RunResources resources={})
	Resume from one exact checkpoint ID.
asio::awaitable< RunResult >	resume_from_async (RunConfig config, std::string checkpoint_id, json resume_value={}, GraphStreamCallback cb={}, RunMetadata metadata={}, RunResources resources={})
	Async exact-checkpoint resume. Every argument is owned by its coroutine.
asio::awaitable< RunResult >	resume_from_until_safe_point_async (RunConfig config, std::string checkpoint_id, std::shared_ptr< GraphSafePointRequest > request, json resume_value={}, GraphStreamCallback cb={}, RunMetadata metadata={}, RunResources resources={})
	Exact resume peer that may stop at the next requested safe point.
RunResult	run (const RunConfig &config)
	Execute the graph synchronously (blocking).
RunResult	run (const RunConfig &config, const RunMetadata &metadata)
	Execute with optional C++ deadline and trace metadata.
asio::awaitable< RunResult >	run_async (RunConfig config)
	Async peer of run() — returns an awaitable yielding the result.
asio::awaitable< RunResult >	run_async (RunConfig config, RunMetadata metadata)
	Async peer that preserves the supplied C++ run metadata.
asio::awaitable< RunResult >	run_async (RunConfig config, RunMetadata metadata, RunResources resources)
	Async non-streaming run with invocation-scoped persistence and provider policy.
RunResult	run_stream (const RunConfig &config, const GraphStreamCallback &cb)
	Execute the graph with streaming event callbacks.
RunResult	run_stream (const RunConfig &config, const GraphStreamCallback &cb, const RunMetadata &metadata)
	Streaming execution with optional C++ deadline and trace metadata.
asio::awaitable< RunResult >	run_stream_async (RunConfig config, GraphStreamCallback cb)
	Async peer of run_stream — non-blocking coroutine surface.
asio::awaitable< RunResult >	run_stream_async (RunConfig config, GraphStreamCallback cb, RunMetadata metadata)
	Async streaming peer that preserves the supplied C++ run metadata.
asio::awaitable< RunResult >	run_stream_async (RunConfig config, GraphStreamCallback cb, RunMetadata metadata, RunResources resources)
	Async streaming peer with per-operation persistence overrides.
asio::awaitable< RunResult >	run_until_safe_point_async (RunConfig config, std::shared_ptr< GraphSafePointRequest > request, GraphStreamCallback cb={}, RunMetadata metadata={}, RunResources resources={})
	Execute normally until a one-shot host request captures a safe point.
void	set_checkpoint_store (std::shared_ptr< CheckpointStore > store)
	Set the checkpoint persistence store.
void	set_hook_runtime (std::shared_ptr<::neograph::HookRuntime > runtime)
	Set the host-owned lifecycle runtime: checkpoint publication, and the BeforeToolExecution/AfterToolExecution Hooks of every tool call a node makes through dispatch_tool_calls (tool_dispatch, the plan-and-execute executor, the deep-research researcher, host nodes using make_tool_execution_context).
void	set_node_attempt_timeout_ms (const std::string &node_name, std::optional< int > timeout_ms)
	Set or clear a host attempt timeout for one node (#364).
void	set_node_cache_enabled (const std::string &node_name, bool enabled)
	Enable or disable per-node result caching.
void	set_node_cache_enabled (const std::string &node_name, bool enabled, CacheKeyPolicy policy)
	Enable or disable caching with an explicit identity policy.
void	set_node_cache_max_entries (std::size_t max_entries)
	Set the global NodeCache entry bound.
void	set_node_retry_policy (const std::string &node_name, const RetryPolicy &policy)
	Set a retry policy for a specific node (overrides default).
void	set_retry_policy (const RetryPolicy &policy)
	Set the default retry policy for all nodes.
void	set_runtime_interposition (std::shared_ptr<::neograph::RuntimeInterpositionController > controller)
	Opt into controlled LLM dispatch for built-in LLMCallNode instances.
void	set_store (std::shared_ptr< Store > store)
	Set the cross-thread shared memory store.
void	set_tool_execution_controller (std::shared_ptr< ToolExecutionController > controller)
	Set the shared host resource-admission boundary for tools.
void	set_tool_gate (ToolGate gate)
	Intercept every tool call before it runs (issue #89).
void	set_worker_count (std::size_t n)
	Resize (or install) the engine-owned worker pool for parallel fan-out.
void	set_worker_count_auto ()
	Opt into a hardware_concurrency()-sized worker pool.
void	update_state (const std::string &thread_id, const json &channel_writes, const std::string &as_node="")
	Update the state for a thread by applying channel writes.
void	update_state_writes (const std::string &thread_id, const std::vector< ChannelWrite > &channel_writes, const std::string &as_node="")
	Update state with ordered writes that preserve ChannelWrite modes.
	~GraphEngine ()
	A node attempt that ignored its cancellation past its stop deadline (#364) can outlive its run and this engine; it keeps the node it runs, and the tools the engine owns, until it finishes.

Static Public Member Functions
static std::unique_ptr< GraphEngine >	build (const json &definition, EngineConfig config)
	Build a ready-to-run engine from one construction-time config.
static std::unique_ptr< GraphEngine >	build (const json &definition, EngineConfig config, EngineResources resources)
	Build with owned tools and a local-first registry.
static std::unique_ptr< GraphEngine >	build_strict (const json &definition, EngineConfig config)
	Build using strict topology validation when schema_version is missing or zero.
static std::unique_ptr< GraphEngine >	build_strict (const json &definition, EngineConfig config, EngineResources resources)
	Strict build overload with owned tools and a local registry.
static std::unique_ptr< GraphEngine >	compile (const json &definition, const NodeContext &default_context, std::shared_ptr< CheckpointStore > store=nullptr)
	Compile a graph from a JSON definition.
static std::unique_ptr< GraphEngine >	link (CompiledGraph graph, EngineConfig config, EngineResources resources)
	Link with owned tools and a per-engine registry overlay.
static std::unique_ptr< GraphEngine >	link (CompiledGraph graph, EngineConfig config, EngineResources resources, GraphGenerationIdentity generation)
	Link an engine permanently bound to one exact host generation.
static std::unique_ptr< GraphEngine >	link (CompiledGraph graph, EngineConfig config={})
	Link an already compiled graph into a configured runtime.
static std::unique_ptr< GraphEngine >	link (ValidatedTopology topology, EngineConfig config, EngineResources resources)
	Validated topology link with owned resources and local registry.
static std::unique_ptr< GraphEngine >	link (ValidatedTopology topology, EngineConfig config={})
	Instantiate and link a topology already proven semantically valid.

Detailed Description

Super-step loop execution engine for graph-based agent workflows.

GraphEngine compiles a JSON graph definition into an executable workflow, then runs it using the Pregel BSP model. Key capabilities:

• Parallel execution: Multiple independent nodes run concurrently on an engine-owned asio::thread_pool; sync and async entry points share it.
• Checkpointing: Full state snapshots at every super-step for time-travel.
• HITL: interrupt_before/after + resume() for human-in-the-loop workflows.
• Send/Command: Dynamic fan-out and routing overrides from nodes.
• Retry policies: Per-node exponential backoff on failure.

Thread safety

After build() or link() returns, the graph definition (nodes, edges, channels) is treated as immutable. A single GraphEngine instance is therefore safe to share across user threads that invoke run() / run_stream() / resume() concurrently with distinct thread_ids — each call constructs its own GraphState and the bundled InMemoryCheckpointStore is mutex-guarded. This lets you host multi-tenant agent workloads on a shared engine without an external async runtime; just dispatch onto std::async, a thread pool, or your existing event loop's worker.

Caveats:

• Configuration setters (set_*, own_tools) are construction-time only: call them before publishing the engine to any execution or admin thread.
• On this engine, administration (state/history reads, update, fork) and execution are mutually exclusive. An admin operation rejects with std::logic_error while any run/resume is in flight; a run/resume rejects while administration is in progress. This conservative engine-wide policy prevents same-thread checkpoint races without a mutex on each super-step. Drain/cancel and await a run before retrying administration.
• Concurrent executions on the same thread_id still produce unspecified checkpoint interleaving. Different engines using the same CheckpointStore are outside this engine's admission boundary; serialize those at the host.
• Custom GraphNode subclasses must be stateless or self-synchronized. Node instances are owned by the engine and shared across all runs.
• User-provided CheckpointStore / Store / Provider / Tool implementations must be thread-safe.

EngineConfig engine_config;
engine_config.node_context = context;
engine_config.checkpoint_store = checkpoint_store;
auto engine = GraphEngine::build(graph_json, std::move(engine_config));
RunConfig config;
config.thread_id = "session-1";
config.input = {{"messages", json::array({{{"role","user"},{"content","Hello"}}})}};
auto result = engine->run(config);

See also

RunConfig, RunResult, GraphNode

Note

New code can obtain state-administration operations through admin(). The corresponding methods remain on GraphEngine for compatibility. Pass all construction policy through EngineConfig and EngineResources; setters remain supported but are configuration, not runtime control.

Definition at line 648 of file engine.h.

Constructor & Destructor Documentation

~GraphEngine()

neograph::graph::GraphEngine::~GraphEngine

(

)

A node attempt that ignored its cancellation past its stop deadline (#364) can outlive its run and this engine; it keeps the node it runs, and the tools the engine owns, until it finishes.

The destructor does not wait for it.

Member Function Documentation

build() [1/2]

static std::unique_ptr< GraphEngine > neograph::graph::GraphEngine::build ( const json &  definition, EngineConfig  config  )

static

Build a ready-to-run engine from one construction-time config.

New code should prefer this entry point when it needs runtime Store, retry, worker, cache, or tool-gate configuration. It applies every option before returning the engine, avoiding a partially configured interval.

Parameters

definition	JSON graph definition.
config	Node construction context and engine runtime configuration.

Returns

A configured GraphEngine ready for execution.

Exceptions

std::runtime_error

If the graph definition is invalid.

build() [2/2]

static std::unique_ptr< GraphEngine > neograph::graph::GraphEngine::build ( const json &  definition, EngineConfig  config, EngineResources  resources  )

static

Build with owned tools and a local-first registry.

resources.tools is selected when NodeContext::tools is empty. Supplying both nonempty ToolSets is rejected as ambiguous.

build_strict()

static std::unique_ptr< GraphEngine > neograph::graph::GraphEngine::build_strict ( const json &  definition, EngineConfig  config  )

static

Build using strict topology validation when schema_version is missing or zero.

The caller's JSON is not mutated. Existing malformed or unsupported schema_version values are preserved so the compiler reports them rather than silently replacing them.

compile()

static std::unique_ptr< GraphEngine > neograph::graph::GraphEngine::compile ( const json &  definition, const NodeContext &  default_context, std::shared_ptr< CheckpointStore >  store = nullptr  )

static

Compile a graph from a JSON definition.

Parses the JSON graph definition, creates nodes via NodeFactory, resolves edges and conditions, and returns a ready-to-run engine.

Parameters

definition	JSON graph definition (nodes, edges, channels, etc.).
default_context	Default NodeContext providing provider, tools, model.
store	Optional checkpoint store for persistence (nullptr = no checkpointing).

Returns

A compiled GraphEngine ready for execution.

Exceptions

std::runtime_error

If the graph definition is invalid.

Note

Compatibility facade. This overload retains its original signature and behavior and delegates to build().

fork()

std::string neograph::graph::GraphEngine::fork	(	const std::string &	source_thread_id,
		const std::string &	new_thread_id,
		const std::string &	checkpoint_id = ""
	)

Fork a thread, creating a new thread from an existing checkpoint.

Copies the specified checkpoint (or the latest) to a new thread ID, enabling branching execution paths. resume(new_thread_id) continues the copied checkpoint's pending nodes. A completed end continuation executes nothing when resumed; editing its state does not schedule new work. To re-execute pending work, fork an exact earlier checkpoint from get_state_history() that still has nodes. A new user turn instead requires run() with resume_if_exists enabled. In-memory branches share the original live managed bank and pinned ceiling. A bounded durable managed-bank branch requires a real shared host bank or Program journal supplied at the original invocation; otherwise fork refuses before creating a checkpoint or effect. Unbounded durable forks are unchanged.

Parameters

source_thread_id	Thread to fork from.
new_thread_id	Thread ID for the new fork.
checkpoint_id	Specific checkpoint to fork from (empty = latest).

Returns

The checkpoint ID of the forked state.

get_graph_name()

const std::string & neograph::graph::GraphEngine::get_graph_name ( ) const

inline

Get the graph name (from the JSON definition).

Returns

Graph name string.

Definition at line 1253 of file engine.h.

get_state()

std::optional< json > neograph::graph::GraphEngine::get_state

(

const std::string &

thread_id

)

const

Get the current state for a thread.

Parameters

thread_id

Thread ID to look up.

Returns

Serialized state JSON, or std::nullopt if no checkpoint exists.

get_state_history()

std::vector< Checkpoint > neograph::graph::GraphEngine::get_state_history	(	const std::string &	thread_id,
		int	limit = 100
	)		const

Get the checkpoint history for a thread (time-travel).

Parameters

thread_id	Thread ID to look up.
limit	Maximum number of checkpoints to return (default: 100).

Returns

Vector of Checkpoint objects, newest first.

get_store()

std::shared_ptr< Store > neograph::graph::GraphEngine::get_store ( ) const

inline

Get the cross-thread shared memory store.

Returns

Shared pointer to the Store, or nullptr if not set.

Definition at line 1104 of file engine.h.

inspect_nested_checkpoint()

std::optional< NestedCheckpoint > neograph::graph::GraphEngine::inspect_nested_checkpoint	(	const std::string &	root_thread_id,
		const std::vector< SubgraphPathStep > &	path,
		std::shared_ptr< CheckpointStore >	run_checkpoint_store = {}
	)		const

Inspect a descendant's checkpoint by graph path and exact node invocation.

Pass the run-scoped root store if it overrode the engine's configured store. Returns nullopt for an absent checkpoint; rejects stateless paths and mismatched checkpoint IDs instead of reading another thread's state.

link() [1/2]

static std::unique_ptr< GraphEngine > neograph::graph::GraphEngine::link ( CompiledGraph  graph, EngineConfig  config, EngineResources  resources  )

static

Link with owned tools and a per-engine registry overlay.

Directly compiled graphs already retain their NodeContext ToolSet. The canonical build() overload also binds the registry automatically.

link() [2/2]

static std::unique_ptr< GraphEngine > neograph::graph::GraphEngine::link ( CompiledGraph  graph, EngineConfig  config = {}  )

static

Link an already compiled graph into a configured runtime.

This is the canonical runtime boundary for callers that explicitly run GraphCompiler and inspect or transform the resulting CompiledGraph. It performs semantic validation, consumes the graph by move, and applies every EngineConfig option without parsing the source JSON again.

Translation round-trip verification requires the original JSON and is therefore performed by build() before it delegates here. Callers using link() directly are responsible for calling GraphCompiler::verify_roundtrip when they need that source-to-IR guarantee.

EngineConfig::node_context is ignored because CompiledGraph already owns its instantiated nodes. The remaining runtime configuration is applied.

Parameters

graph	Compiled graph to consume.
config	Runtime configuration applied before the engine is returned.

Returns

A configured GraphEngine ready for execution.

Exceptions

std::runtime_error

If semantic validation fails in strict mode.

resume() [1/3]

RunResult neograph::graph::GraphEngine::resume	(	const RunConfig &	config,
		const json &	resume_value,
		const GraphStreamCallback &	cb,
		const RunMetadata &	metadata
	)

Resume with metadata for this new execution attempt.

Deadlines are not checkpointed because an absolute deadline from a previous attempt may be stale; callers intentionally provide a fresh RunMetadata value.

resume() [2/3]

RunResult neograph::graph::GraphEngine::resume	(	const RunConfig &	config,
		const json &	resume_value = json(),
		const GraphStreamCallback &	cb = nullptr
	)

Resume while preserving caller cancellation settings.

config.thread_id identifies the interrupted session. Input is ignored; the latest checkpoint remains the source of resumed graph state.

resume() [3/3]

RunResult neograph::graph::GraphEngine::resume	(	const std::string &	thread_id,
		const json &	resume_value = json(),
		const GraphStreamCallback &	cb = nullptr
	)

Resume execution from a HITL interrupt.

Loads the last checkpoint for the given thread, applies the resume value, and continues execution from the interrupted node.

Parameters

thread_id	Thread ID of the interrupted session.
resume_value	Optional value to inject before resuming (e.g., human input). It reaches RunContext::resume_value of the one invocation the resume answers, in the step it re-enters: the node or Send that raised NodeInterrupt, or the node an interrupt_before gate held. No other node sees it (#393).
cb	Optional streaming callback.

Returns

Execution result after resumption.

resume_async()

asio::awaitable< RunResult > neograph::graph::GraphEngine::resume_async	(	const std::string &	thread_id,
		const json &	resume_value = json(),
		const GraphStreamCallback &	cb = nullptr
	)

Async peer of resume — non-blocking coroutine surface.

All arguments are copied before this function returns, so the returned awaitable does not borrow the caller's strings, JSON, or callback object. Objects referenced by the callback's captures remain caller-owned.

resume_from()

RunResult neograph::graph::GraphEngine::resume_from	(	RunConfig	config,
		std::string	checkpoint_id,
		json	resume_value = {},
		GraphStreamCallback	cb = {},
		RunMetadata	metadata = {},
		RunResources	resources = {}
	)

Resume from one exact checkpoint ID.

The requested checkpoint must exist and belong to config.thread_id. This path never substitutes a newer checkpoint.

run() [1/2]

RunResult neograph::graph::GraphEngine::run

(

const RunConfig &

config

)

Execute the graph synchronously (blocking).

Thread-safe across distinct thread_ids on a shared engine instance: dispatch via std::async, a thread pool, or any executor. See the class-level "Thread safety" notes for caveats.

Parameters

config

Run configuration with thread ID, input, and limits.

Returns

Execution result with final state and metadata.

run() [2/2]

RunResult neograph::graph::GraphEngine::run	(	const RunConfig &	config,
		const RunMetadata &	metadata
	)

Execute with optional C++ deadline and trace metadata.

RunMetadata is separate from RunConfig to preserve the latter's stable ABI.

run_async()

asio::awaitable< RunResult > neograph::graph::GraphEngine::run_async

(

RunConfig

config

)

Async peer of run() — returns an awaitable yielding the result.

Callers driving an io_context can co_await engine->run_async(cfg) alongside other coroutines (typically multiple concurrent agents). The super-step loop, node dispatch, checkpoint I/O, parallel fan-out, and retry backoff are all coroutine-native (3.0) — the caller's executor is never blocked by engine work.

The config is taken by value so the awaitable owns its own copy in the coroutine frame. This makes the common asio::co_spawn(io.get_executor(), engine->run_async(stack_cfg), use_future) shape safe — stack_cfg may go out of scope before the awaitable resolves without dangling-referencing the config.

Parameters

config

Run configuration (moved into the coroutine frame).

Returns

Awaitable yielding the execution result.

run_stream()

RunResult neograph::graph::GraphEngine::run_stream	(	const RunConfig &	config,
		const GraphStreamCallback &	cb
	)

Execute the graph with streaming event callbacks.

Parameters

config	Run configuration.
cb	Callback invoked for each graph event (filtered by config.stream_mode).

Returns

Execution result.

run_stream_async()

asio::awaitable< RunResult > neograph::graph::GraphEngine::run_stream_async	(	RunConfig	config,
		GraphStreamCallback	cb
	)

Async peer of run_stream — non-blocking coroutine surface.

config and cb are taken by value for the same reason as run_async() — see that overload's docstring.

set_checkpoint_store()

void neograph::graph::GraphEngine::set_checkpoint_store

(

std::shared_ptr< CheckpointStore >

store

)

Set the checkpoint persistence store.

Parameters

store

Checkpoint store implementation.

set_hook_runtime()

void neograph::graph::GraphEngine::set_hook_runtime ( std::shared_ptr<::neograph::HookRuntime >  runtime )

inline

Set the host-owned lifecycle runtime: checkpoint publication, and the BeforeToolExecution/AfterToolExecution Hooks of every tool call a node makes through dispatch_tool_calls (tool_dispatch, the plan-and-execute executor, the deep-research researcher, host nodes using make_tool_execution_context).

RunResources::hook_runtime replaces it for the tool calls of one run. Configure before concurrent run() or resume() calls.

Definition at line 1091 of file engine.h.

set_node_attempt_timeout_ms()

void neograph::graph::GraphEngine::set_node_attempt_timeout_ms	(	const std::string &	node_name,
		std::optional< int >	timeout_ms
	)

Set or clear a host attempt timeout for one node (#364).

Each attempt of the node may run this long. A timed-out attempt is cancelled through its own CancelToken, its late completion is discarded, and it fails with NG.Timeout, which the node's retry policy retries and a failure edge can route. The next attempt starts only after the cancelled one has stopped; one that does not stop by its stop deadline (the timeout again, at least one second) fails the run with NG.EffectUncertain.

The timeout a node runs with is the shorter of the definition's node_retry_policies[node].attempt_timeout_ms and this host value, when either is set: like the other fields of a definition's per-node policy, the host may only tighten it, never lengthen or remove it. For a node without a definition timeout the host value applies as is.

Parameters

node_name	Name of the node.
timeout_ms	100 to 3 600 000 milliseconds; std::nullopt clears the host value.

Exceptions

std::invalid_argument

for an unknown node or a value out of range.

set_node_cache_enabled()

void neograph::graph::GraphEngine::set_node_cache_enabled	(	const std::string &	node_name,
		bool	enabled
	)

Enable or disable per-node result caching.

When enabled for a node, the executor hashes the input state and looks up an execution-local (node_name, scope, hash) key in the engine's NodeCache. On hit, the cached NodeResult is replayed without invoking the node — no LLM call, no tool execution. On miss, the node runs and the result is stored.

Cache is OFF by default. Only opt in for nodes that are pure (deterministic, no external side effects, no time dependence). The default CacheScope::Execution never reuses a result across run() or resume() calls, so it cannot cross tenant, Store, ToolGate, provider/model, or resume boundaries. Pass CacheScope::Reusable only for a node proven independent of all such context, optionally with a CacheKeyPolicy context key that declares stable, non-secret context partitions. Streaming runs (run_stream) bypass the cache for the affected nodes because cached hits cannot replay LLM_TOKEN events.

Parameters

node_name	Name of the node to enable / disable.
enabled	True to enable caching; false to disable.

Exceptions

std::invalid_argument

if the graph has no node named node_name.

set_node_cache_max_entries()

void neograph::graph::GraphEngine::set_node_cache_max_entries

(

std::size_t

max_entries

)

Set the global NodeCache entry bound.

Zero means unbounded. Safe during active runs; lowering the value may evict entries immediately.

set_node_retry_policy()

void neograph::graph::GraphEngine::set_node_retry_policy	(	const std::string &	node_name,
		const RetryPolicy &	policy
	)

Set a retry policy for a specific node (overrides default).

The timing a node runs with is resolved per field (#364): the default (set_retry_policy / EngineConfig::retry_policy, else the definition's retry_policy, else RetryPolicy{}), then the fields of the definition's per-node policy (node_retry_policies). For a node without such a policy, this host policy replaces the result, as before. For a node with one, the host may only tighten: max_retries takes the minimum, the delays and the multiplier the maximum, and jitter_pct is the host's; max_retries never exceeds 10 for such a node.

Parameters

node_name	Name of the node.
policy	Retry policy for this specific node.

Exceptions

std::invalid_argument

if the graph has no node named node_name.

set_retry_policy()

void neograph::graph::GraphEngine::set_retry_policy

(

const RetryPolicy &

policy

)

Set the default retry policy for all nodes.

Parameters

policy

Retry policy with backoff configuration.

set_runtime_interposition()

void neograph::graph::GraphEngine::set_runtime_interposition

(

std::shared_ptr<::neograph::RuntimeInterpositionController >

controller

)

Opt into controlled LLM dispatch for built-in LLMCallNode instances.

Custom nodes and direct Provider calls remain explicitly unmanaged.

set_store()

void neograph::graph::GraphEngine::set_store

(

std::shared_ptr< Store >

store

)

Set the cross-thread shared memory store.

Parameters

store

Store implementation for cross-thread data sharing.

See also

Store, InMemoryStore

set_tool_execution_controller()

void neograph::graph::GraphEngine::set_tool_execution_controller ( std::shared_ptr< ToolExecutionController >  controller )

inline

Set the shared host resource-admission boundary for tools.

Like the gate, this survives run() and resume(); configure before concurrent use.

Definition at line 1079 of file engine.h.

set_tool_gate()

void neograph::graph::GraphEngine::set_tool_gate ( ToolGate  gate )

inline

Intercept every tool call before it runs (issue #89).

A gate returns Allow (optionally rewriting the arguments), Deny (the model is told why) or Interrupt (pause and ask a human). It is consulted for every call in a batch before any tool runs, so an Interrupt has no side effects to undo and the resumed node cannot double-apply a sibling's effects. Reaches both dispatch paths — graph nodes and llm::Agent — because they share one dispatcher (issue #87).

It lives on the engine, not on RunConfig, and that is deliberate. resume() builds its own RunConfig internally, so a per-run gate would vanish the moment a human answered the very prompt it raised — the dangerous tool would then run unchecked, with the caller believing they had a permission system. A policy that disappears exactly when it is being exercised is worse than no policy. Set it once, here, and it holds for every run and every resume on this engine.

engine->set_tool_gate([](ToolCall call, ToolGateContext gctx)
        -> asio::awaitable<ToolDecision> {
    if (call.name != "shell") co_return ToolDecision::allow();
    if (!gctx.resume_value)
        co_return ToolDecision::interrupt("shell needs approval",
                                          json::parse(call.arguments));
    if (gctx.resume_value->value("approved", false))
        co_return ToolDecision::allow();
    co_return ToolDecision::deny("the human said no");
});

Definition at line 1075 of file engine.h.

set_worker_count()

void neograph::graph::GraphEngine::set_worker_count

(

std::size_t

n

)

Resize (or install) the engine-owned worker pool for parallel fan-out.

**compile() default is set_worker_count(1)** — no engine-owned thread pool, fan-out branches dispatch inline on the coroutine's own executor. That keeps sequential and single-Send workloads off the ~6-7 µs cross-thread submit path, and avoids silently exposing nodes that hold non-thread-safe state to concurrent execution. See docs/migration-v0.4-to-v1.0.md (Migration 3) for the full rationale.

Call this with n >= 2 to opt into a real engine-owned asio::thread_pool for multi-Send fan-out or multi-outgoing edges (e.g. set_worker_count(4) for a 4-way Send). For hardware_concurrency() sizing, see set_worker_count_auto().

Must be called before any concurrent run(); resizing rebuilds both the pool and the internal executor and is a hard error against in-flight runs. Values < 1 are clamped to 1.

Parameters

n	Number of worker threads in the fan-out pool.

See also

set_worker_count_auto()

set_worker_count_auto()

void neograph::graph::GraphEngine::set_worker_count_auto

(

)

Opt into a hardware_concurrency()-sized worker pool.

Equivalent to set_worker_count(std::thread::hardware_concurrency()), with a fallback of 4 if the runtime cannot detect. **compile() does NOT call this — its default is set_worker_count(1).** Use this once after compile() (and before any run()) to enable real parallel fan-out for multi-Send / multi-outgoing-edge workloads.

update_state()

void neograph::graph::GraphEngine::update_state	(	const std::string &	thread_id,
		const json &	channel_writes,
		const std::string &	as_node = ""
	)

Update the state for a thread by applying channel writes.

Loads the latest checkpoint, applies the writes, and saves a new checkpoint. Useful for injecting state externally (e.g., from a UI).

Parameters

thread_id	Thread ID to update.
channel_writes	JSON object of channel_name -> value pairs to write.
as_node	Optional node name to attribute the update to (for tracing).

update_state_writes()

void neograph::graph::GraphEngine::update_state_writes	(	const std::string &	thread_id,
		const std::vector< ChannelWrite > &	channel_writes,
		const std::string &	as_node = ""
	)

Update state with ordered writes that preserve ChannelWrite modes.

Unlike the JSON keyed form, this overload applies repeated channel writes in order and honors explicit overwrite intent.

---

The documentation for this class was generated from the following file:

• include/neograph/graph/engine.h