Provider backed by a pinned, closed/versioned descriptor and immutable SDK policy. More...
#include <schema_provider.h>
Public Member Functions | |
| bool | host_pumped () const noexcept override |
| Whether the SDK client behind this provider runs on a host-pumped executor (a WebAssembly host's event loop, #535). | |
| SchemaProvider (sp::descriptor::ValidatedDescriptor descriptor, sp::runtime::Options options, std::shared_ptr< sp::runtime::Executor > executor, std::shared_ptr< sp::runtime::HostTransport > transport, Defaults defaults={}) | |
Runs on a host's event loop (#535): the SDK's work is posted to executor and every HTTP attempt goes through transport (sp::runtime::Executor and HostTransport, runtime/host.h; sp::runtime::js builds both over fetch). | |
| SchemaProvider (sp::descriptor::ValidatedDescriptor descriptor, sp::runtime::Options options={}, Defaults defaults={}) | |
| Runs on the SDK's worker pool and its libcurl transport. | |
 Public Member Functions inherited from neograph::Provider | |
| template<class Outcome > | |
| Outcome | complete_endpoint (sp::runtime::PreparedEndpoint< Outcome > prepared) |
| Runs a prepared call on this provider's client and blocks for its outcome. | |
| template<class Outcome > | |
| asio::awaitable< Outcome > | complete_endpoint_async (sp::runtime::PreparedEndpoint< Outcome > prepared) |
| The same call without blocking a thread. | |
| bool | has_endpoints () const |
| Whether this provider forwards SchemaProvider endpoint calls. | |
| template<EndpointRequest Request> | |
| sp::runtime::PreparedEndpoint< EndpointOutcomeOf< Request > > | prepare_endpoint (Request request, sp::runtime::RunOptions options={}) |
| SchemaProvider's non-chat endpoints at this provider's origin (#531). | |
Protected Member Functions | |
| std::shared_ptr< sp::runtime::Client > | endpoint_runtime () const override |
| The SchemaProvider client whose endpoints this provider forwards, the same one for the provider's lifetime. | |
Additional Inherited Members | |
| Static Public Member Functions inherited from neograph::Provider | |
| static std::optional< std::uint64_t > | invocation_upper_bound (const PreparedProviderRequest &request) |
| Most model invocations one dispatch of this request can bill: the request's model invocations times the transport attempts its retry policy allows. | |
| static std::string | request_content_digest (const PreparedProviderRequest &request) |
| What the request asks for, apart from the policy that admitted it: its family, model, mode and origin, its encoded body with the members the descriptor names set apart by role (model, messages, stream and output cap), and for an endpoint call its endpoint, method, URL and upload (#541). | |
| static std::string | request_digest (const PreparedProviderRequest &request) |
| The dispatch digest of a prepared request: its content and everything the descriptor policy decided about sending it, including the policy's identity. | |
| static std::string | request_policy_identity (const PreparedProviderRequest &request) |
| The identity of the SchemaProvider descriptor policy that admitted the request, "sha256:" and 64 hex digits. | |
| Static Protected Member Functions inherited from neograph::Provider | |
| static std::shared_ptr< sp::runtime::Client > | endpoint_runtime_of (const Provider &inner) |
| For a provider that wraps another: forward the inner provider's endpoints by returning this from endpoint_runtime(). | |
| static PreparedProviderRequest | prepare_runtime (std::shared_ptr< sp::runtime::Client > client, ProviderRequest request, bool host_pumped=false) |
host_pumped is the client's executor (see host_pumped()): the prepared request then refuses to be dispatched from a blocking bridge. | |
Detailed Description
Provider backed by a pinned, closed/versioned descriptor and immutable SDK policy.
Endpoint/header/model facts are data; codecs, replay and retry are compiled.
Definition at line 10 of file schema_provider.h.
Constructor & Destructor Documentation
◆ SchemaProvider() [1/2]
|
explicit |
Runs on the SDK's worker pool and its libcurl transport.
A build without that transport (WebAssembly, a single-thread build) throws the SDK's descriptor::ConfigError at "/transport": use the constructor below.
◆ SchemaProvider() [2/2]
| neograph::llm::SchemaProvider::SchemaProvider | ( | sp::descriptor::ValidatedDescriptor | descriptor, |
| sp::runtime::Options | options, | ||
| std::shared_ptr< sp::runtime::Executor > | executor, | ||
| std::shared_ptr< sp::runtime::HostTransport > | transport, | ||
| Defaults | defaults = {} |
||
| ) |
Runs on a host's event loop (#535): the SDK's work is posted to executor and every HTTP attempt goes through transport (sp::runtime::Executor and HostTransport, runtime/host.h; sp::runtime::js builds both over fetch).
Needs no libcurl, no OpenSSL and no thread. Pump the executor with async::HostLoop and use the _async entry points; the synchronous ones refuse (Provider::host_pumped). Throws the SDK's descriptor::ConfigError when transport is null, and when executor is null in a build without threads.
Member Function Documentation
◆ endpoint_runtime()
|
inlineoverrideprotectedvirtual |
The SchemaProvider client whose endpoints this provider forwards, the same one for the provider's lifetime.
Null, the default, means none.
Reimplemented from neograph::Provider.
Definition at line 41 of file schema_provider.h.
◆ host_pumped()
|
inlineoverridevirtualnoexcept |
Whether the SDK client behind this provider runs on a host-pumped executor (a WebAssembly host's event loop, #535).
Its calls finish only when the host runs its loop, so a synchronous entry point that waits for them on the calling thread (GraphEngine::run, Provider::dispatch, complete_endpoint) cannot work. The dispatch of such a call fails with the SDK's runtime.blocking.host_pumped refusal, nothing sent; use the _async forms on async::HostLoop.
Reimplemented from neograph::Provider.
Definition at line 39 of file schema_provider.h.
The documentation for this class was generated from the following file:
- include/neograph/llm/schema_provider.h
