neograph::llm::RateLimitedProvider Class Reference

Decorator that retries on RateLimitError according to its Retry-After hint. More...

#include <rate_limited_provider.h>

Inheritance diagram for neograph::llm::RateLimitedProvider:

Collaboration diagram for neograph::llm::RateLimitedProvider:

Classes
struct	Config
	Configuration for rate-limit handling. More...

Public Member Functions
asio::awaitable< ChatCompletion >	complete_async (const CompletionParams &params) override
	Async completion that retries on RateLimitError with a non- blocking asio::steady_timer sleep.
ChatCompletion	complete_stream (const CompletionParams &params, const StreamCallback &on_chunk) override
	Streaming retry path remains synchronous — complete_stream() has no awaitable peer on Provider, and async streaming through the schema/openai providers still uses httplib.
std::string	get_name () const override
	Get the provider name (e.g., "openai", "claude").
asio::awaitable< ChatCompletion >	invoke (const CompletionParams &params, StreamCallback on_chunk) override
	v1.0 single-dispatch override (Candidate 6 PR6).
Public Member Functions inherited from neograph::Provider
virtual ChatCompletion	complete (const CompletionParams &params)
	Perform a synchronous LLM completion.
virtual asio::awaitable< ChatCompletion >	complete_stream_async (const CompletionParams &params, const StreamCallback &on_chunk)
	Async streaming completion.

Static Public Member Functions
static std::unique_ptr< RateLimitedProvider >	create (std::shared_ptr< Provider > inner)
	Convenience: build with default Config.
static std::unique_ptr< RateLimitedProvider >	create (std::shared_ptr< Provider > inner, Config cfg)
	Build a decorator around an inner provider.

Detailed Description

Decorator that retries on RateLimitError according to its Retry-After hint.

Behaviour:

• Calls through to the inner provider.
• On RateLimitError, sleeps for retry_after_seconds() + 1s (or default_wait_seconds if the upstream didn't send a usable Retry-After), then retries.
• Capped at max_retries attempts. After that the final RateLimitError propagates to the caller.
• max_wait_seconds caps each individual sleep so a pathological Retry-After (server misconfigured, clock skew) can't stall the process — the decorator returns control to the caller, which can decide whether to keep going.

Non-rate-limit exceptions pass through untouched.

auto inner = SchemaProvider::create({.schema_path = "claude", ...});
auto provider = RateLimitedProvider::create(std::move(inner));
// use `provider` like any other Provider

Definition at line 44 of file rate_limited_provider.h.

Member Function Documentation

complete_async()

asio::awaitable< ChatCompletion > neograph::llm::RateLimitedProvider::complete_async ( const CompletionParams &  params )

overridevirtual

Async completion that retries on RateLimitError with a non- blocking asio::steady_timer sleep.

Sync complete() is inherited from Provider and routes through this via run_sync — the inner timer drives a fresh io_context, so caller-thread blocking semantics are preserved.

Reimplemented from neograph::Provider.

complete_stream()

ChatCompletion neograph::llm::RateLimitedProvider::complete_stream ( const CompletionParams &  params, const StreamCallback &  on_chunk  )

overridevirtual

Streaming retry path remains synchronous — complete_stream() has no awaitable peer on Provider, and async streaming through the schema/openai providers still uses httplib.

Once those migrate this method can grow an awaitable cousin.

Reimplemented from neograph::Provider.

create()

static std::unique_ptr< RateLimitedProvider > neograph::llm::RateLimitedProvider::create ( std::shared_ptr< Provider >  inner, Config  cfg  )

static

Build a decorator around an inner provider.

Parameters

inner	The Provider to wrap. Must be non-null.
cfg	Retry configuration.

Returns

A unique_ptr to the decorator.

get_name()

std::string neograph::llm::RateLimitedProvider::get_name ( ) const

overridevirtual

Get the provider name (e.g., "openai", "claude").

Opaque debug identifier, not a typed dispatch key. Different subclasses pick different conventions:

• OpenAIProvider always returns "openai".
• SchemaProvider returns whatever's in the schema's name field — could be "openai", "claude", "openai-responses", "gemini", or a user-defined schema id.
• RateLimitedProvider delegates to its inner provider.

Code branching on the exact string (e.g. if (get_name() == "openai-responses")) is brittle — a custom schema named "openai-responses-v2" slips through, or a future rename silently breaks the branch. Use it for logging, telemetry, or version-pinning diagnostics. For typed behaviour, add a typed ProviderKind accessor or branch on the schema's actual fields.

Returns

Opaque provider identifier string.

Implements neograph::Provider.

invoke()

asio::awaitable< ChatCompletion > neograph::llm::RateLimitedProvider::invoke ( const CompletionParams &  params, StreamCallback  on_chunk  )

overridevirtual

v1.0 single-dispatch override (Candidate 6 PR6).

Anchors invoke() on this decorator so engine provider->invoke(...) calls hit retry/backoff directly. v0.9 body routes through the existing 4-virtual overrides (which already wrap the inner provider with retry); v1.0 folds the retry loop into invoke() and deletes the legacy methods.

Reimplemented from neograph::Provider.

---

The documentation for this class was generated from the following file:

• include/neograph/llm/rate_limited_provider.h