neograph::Provider Class Reference

Capability identity plus a single prepared, owned operation contract. More...

#include <provider.h>

Inheritance diagram for neograph::Provider:

Public Member Functions
template<class Outcome >
Outcome	complete_endpoint (sp::runtime::PreparedEndpoint< Outcome > prepared)
	Runs a prepared call on this provider's client and blocks for its outcome.
template<class Outcome >
asio::awaitable< Outcome >	complete_endpoint_async (sp::runtime::PreparedEndpoint< Outcome > prepared)
	The same call without blocking a thread.
bool	has_endpoints () const
	Whether this provider forwards SchemaProvider endpoint calls.
virtual bool	host_pumped () const noexcept
	Whether the SDK client behind this provider runs on a host-pumped executor (a WebAssembly host's event loop, #535).
template<EndpointRequest Request>
sp::runtime::PreparedEndpoint< EndpointOutcomeOf< Request > >	prepare_endpoint (Request request, sp::runtime::RunOptions options={})
	SchemaProvider's non-chat endpoints at this provider's origin (#531).

Static Public Member Functions
static std::optional< std::uint64_t >	invocation_upper_bound (const PreparedProviderRequest &request)
	Most model invocations one dispatch of this request can bill: the request's model invocations times the transport attempts its retry policy allows.
static std::string	request_content_digest (const PreparedProviderRequest &request)
	What the request asks for, apart from the policy that admitted it: its family, model, mode and origin, its encoded body with the members the descriptor names set apart by role (model, messages, stream and output cap), and for an endpoint call its endpoint, method, URL and upload (#541).
static std::string	request_digest (const PreparedProviderRequest &request)
	The dispatch digest of a prepared request: its content and everything the descriptor policy decided about sending it, including the policy's identity.
static std::string	request_policy_identity (const PreparedProviderRequest &request)
	The identity of the SchemaProvider descriptor policy that admitted the request, "sha256:" and 64 hex digits.

Protected Member Functions
virtual std::shared_ptr< sp::runtime::Client >	endpoint_runtime () const
	The SchemaProvider client whose endpoints this provider forwards, the same one for the provider's lifetime.

Static Protected Member Functions
static std::shared_ptr< sp::runtime::Client >	endpoint_runtime_of (const Provider &inner)
	For a provider that wraps another: forward the inner provider's endpoints by returning this from endpoint_runtime().
static PreparedProviderRequest	prepare_runtime (std::shared_ptr< sp::runtime::Client > client, ProviderRequest request, bool host_pumped=false)
	host_pumped is the client's executor (see host_pumped()): the prepared request then refuses to be dispatched from a blocking bridge.

Detailed Description

Capability identity plus a single prepared, owned operation contract.

Returned awaitables own all operation state, including the runtime client; neither request nor Provider object must survive coroutine scheduling.

Definition at line 387 of file provider.h.

Member Function Documentation

complete_endpoint_async()

template<class Outcome >

asio::awaitable< Outcome > neograph::Provider::complete_endpoint_async ( sp::runtime::PreparedEndpoint< Outcome >  prepared )

inline

The same call without blocking a thread.

The awaitable owns the client and the call; cancelling the awaiting coroutine cancels the call, and its outcome is then the SDK's Cancelled failure.

Definition at line 456 of file provider.h.

endpoint_runtime()

virtual std::shared_ptr< sp::runtime::Client > neograph::Provider::endpoint_runtime ( ) const

inlineprotectedvirtual

The SchemaProvider client whose endpoints this provider forwards, the same one for the provider's lifetime.

Null, the default, means none.

Reimplemented in neograph::llm::SchemaProvider, and neograph::observability::OpenInferenceProvider.

Definition at line 469 of file provider.h.

Referenced by endpoint_runtime_of().

host_pumped()

virtual bool neograph::Provider::host_pumped ( ) const

inlinevirtualnoexcept

Whether the SDK client behind this provider runs on a host-pumped executor (a WebAssembly host's event loop, #535).

Its calls finish only when the host runs its loop, so a synchronous entry point that waits for them on the calling thread (GraphEngine::run, Provider::dispatch, complete_endpoint) cannot work. The dispatch of such a call fails with the SDK's runtime.blocking.host_pumped refusal, nothing sent; use the _async forms on async::HostLoop.

Reimplemented in neograph::llm::SchemaProvider, and neograph::observability::OpenInferenceProvider.

Definition at line 427 of file provider.h.

invocation_upper_bound()

static std::optional< std::uint64_t > neograph::Provider::invocation_upper_bound ( const PreparedProviderRequest &  request )

static

Most model invocations one dispatch of this request can bill: the request's model invocations times the transport attempts its retry policy allows.

Empty when the prepared request does not state them.

prepare_endpoint()

template<EndpointRequest Request>

sp::runtime::PreparedEndpoint< EndpointOutcomeOf< Request > > neograph::Provider::prepare_endpoint ( Request  request, sp::runtime::RunOptions  options = {}  )

inline

SchemaProvider's non-chat endpoints at this provider's origin (#531).

They run on the client its chat calls use, with the same credential, admission slots, retry budget, deadlines and error classification. The descriptor policy admits each endpoint when the call is prepared; a refused call carries its error and completes as a Failure without reaching the network. Calls are buffered and sent as given: SchemaProvider::Defaults apply to chat calls only, and nothing here journals, reserves or charges a budget. A provider without SchemaProvider endpoints throws std::logic_error. Preparation throws sp::runtime::AdmissionError when no slot is free.

Definition at line 439 of file provider.h.

request_content_digest()

static std::string neograph::Provider::request_content_digest ( const PreparedProviderRequest &  request )

static

What the request asks for, apart from the policy that admitted it: its family, model, mode and origin, its encoded body with the members the descriptor names set apart by role (model, messages, stream and output cap), and for an endpoint call its endpoint, method, URL and upload (#541).

It does not change with the descriptor policy, the descriptor's routes, headers and member names, the retry policy or the observer limits. The body is the one complete statement of what is sent, so a different prompt, tool, option or message text, built in or the host's, is a different digest; an SDK that encodes the same request differently is too.

request_digest()

static std::string neograph::Provider::request_digest ( const PreparedProviderRequest &  request )

static

The dispatch digest of a prepared request: its content and everything the descriptor policy decided about sending it, including the policy's identity.

It changes when SchemaProvider's descriptor policy changes, even for a request of the same content. A journal that must recognise the same call across policy generations uses request_content_digest() and records request_policy_identity() beside it (#543).

request_policy_identity()

static std::string neograph::Provider::request_policy_identity ( const PreparedProviderRequest &  request )

static

The identity of the SchemaProvider descriptor policy that admitted the request, "sha256:" and 64 hex digits.

A journal records it beside request_content_digest().

---

The documentation for this class was generated from the following file:

• include/neograph/provider.h