Calling an Endpoint

Inference endpoints speak the OpenAI HTTP API. Any client that lets you set a base URL and an API key works, including the OpenAI SDKs, LangChain, LlamaIndex and plain HTTP.

Base URL

Each endpoint has its own URL, shown on the endpoint’s Overview tab and in the endpoint list. Add /v1 to it to get the base URL for an OpenAI client:

https://<your-endpoint-url>/v1

Authentication

Send a Saturn Cloud API token as a bearer token:

Authorization: Bearer <YOUR_SATURN_API_TOKEN>

Create a token by clicking your name in the sidebar and opening Access Keys. See API Tokens for scoped and unscoped tokens. For a shared (serverless) endpoint, the token must be scoped to the endpoint’s organization.

Requests are checked before they reach the model, and a request without a valid token is rejected.

Model name

The model field in each request selects what the endpoint serves:

The endpoint’s Overview tab has an example request with the correct value filled in. GET /v1/models lists the names an endpoint accepts.

Requests

Chat completions

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://<your-endpoint-url>/v1",
    api_key=os.environ["SATURN_TOKEN"],
)

response = client.chat.completions.create(
    model="<your-endpoint-name>",
    messages=[
        {"role": "system", "content": "You answer in one sentence."},
        {"role": "user", "content": "What is tensor parallelism?"},
    ],
    max_tokens=200,
)
print(response.choices[0].message.content)

Streaming

Set stream=True to receive tokens as they are generated:

stream = client.chat.completions.create(
    model="<your-endpoint-name>",
    messages=[{"role": "user", "content": "Explain KV caching."}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="", flush=True)

curl

curl https://<your-endpoint-url>/v1/chat/completions \
  -H "Authorization: Bearer $SATURN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<your-endpoint-name>",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

Request parameters such as temperature, top_p, max_tokens and stop work as in the OpenAI API. Features that depend on the model, such as tool calling and reasoning output, follow the model’s chat template, so check the model card for what the model supports.

Monitoring

The endpoint page has tabs for Metrics (request rate, latency, time to first token, token throughput, error rate and GPU use), Logs and Usage.