Llama APIPreview
API platform

Get started

Overview
Quickstart

Essentials

Models
API keys
SDKs & libraries
Rate limits

Features

Chat completion
Image understanding
Structured output
Tool calling
OpenAI compatibility
Moderation
Fine-tuning & evaluation
Accelerated inference

Guides

Chat & conversation
Tool calling
Moderation & security
Best practices

API reference

Chat completion
Models
Moderations

Resources

Data commitments
Legal
Log in to API

OpenAI compatibility

Llama API supports the OpenAI client libraries for Python and TypeScript as an alternative to the Llama API client, to help you integrate Llama API with existing OpenAI-based applications.The base path for OpenAI compatibility is:
Some features of the OpenAI client libraries are not supported on Llama API. See below for details on feature support.

Using the OpenAI client

Set up the OpenAI client using the base URL and your Llama API dashboard API key.
Python (OpenAI client)
Make calls to Llama API like you would for OpenAI. See below for an example of making a chat completion request.
Python (OpenAI client)

Feature support

Llama API supports the following OpenAI client library features:
Feature
Support Status
Chat completions
Supported
Model selection
Supported
Temperature/sampling
Supported
Streaming
Supported
Image understanding
Supported
Structured output (JSON mode)
Supported
Function calling (tools)
Supported

Compatibility API endpoints

The table below maps Llama API compatibility endpoints to OpenAI client functions.
SDK function
API endpoint
API method
client.models.list()
/compat/v1/models
GET
client.models.retrieve(model)
/compat/v1/models/{model}
GET
client.chat.completions.create(...)
/compat/v1/chat/completions
POST

Compatibility considerations

Model mapping

Use Llama model names instead of OpenAI model names with the compatibility endpoint, and use the Models endpoint for a complete list of available models.

Unsupported feature behaviour

Llama API does not support all OpenAI client library features or parameters, handling them in one of two ways:
  1. Silent handling: Most unsupported parameters are ignored silently.
  2. Error response: Some parameters will result in a HTTP 400 Bad Request error, to notify you about unsupported features.
Thoroughly test your implementation before migration to ensure compatibility works as expected.

Examples

List models

Use the OpenAI client to retrieve a list of available Llama API models:
Python (OpenAI client)

Retrieve model

Retrieve detailed information about a specific model:
Python (OpenAI client)

Chat completion

Create text completions from a conversation context:
Python (OpenAI client)

Streaming chat completion

Get chat completions as they are generated, one chunk at a time:
Python (OpenAI client)

Image understanding

Use the chat interface to process and understand Base64 encoded images:
Python (OpenAI client)

Structured outputs

Generate structured outputs using Pydantic models.
Python (OpenAI client)

Function calling (tools)

Enable models to call functions/tools defined by the developer.
Python (OpenAI client)

Error handling

Implement robust error handling for various error conditions with the compatibility endpoint:
Python (OpenAI client)
Was this page helpful?
Using the OpenAI client
Feature support
Compatibility API endpoints
Compatibility considerations
Examples