While chat completion is optimized for human-readable model responses, sometimes the audience for a model response is another process rather than an end user.Use JSON structured output to get model responses in a specific JSON format that you define, then use the structured response directly as part of your application’s business logic.JSON structured output offers several benefits over plain text responses:
•Consistent format: Guarantees the output follows your defined structure, reducing the need for complex parsing logic.
•Reduced processing errors: Minimizes errors caused by unexpected variations in the model's response format.
•Simpler integration: Enables you to directly use the model's output with APIs, databases, or other components that expect structured data.
•Better tool interaction: Works well with tool calling by providing structured data that external tools can easily use.
Use cases
Structured output is especially helpful for tasks such as:
•Extracting information: Pulling specific details such as names, dates, locations, or product information from unstructured text.
•Classifying data: Categorizing user input or text into predefined categories.
•Generating function arguments: Creating structured arguments for other functions or APIs based on natural language prompts.
•Generating configurations: Producing JSON-based configuration files from user requirements.
Combining with other Llama API features
You can effectively combine JSON structured output with other Llama API features:
•Tool calling: Use schemas to define a consistent JSON format for arguments passed to your tools or for results returned from your tools that the LLM needs to process.
•Image understanding: Extract structured data from images, such as detected objects or recognized text, and output it in JSON format.
•Chat completion: Use validated, structured data from one turn as reliable context for the next messages in a conversation.
How to use JSON structured output
To request a response with JSON structured output, specify a JSON schema in your request, by using the response_format parameter in your API request. Set its type field to json_schema and provide your desired JSON structure in the json_schema field. The model will then return output matching the schema you provided in the request.
Request example
Hand-written schema
Pydantic schema
Structured output with a hand-written JSON schema
Response example
The API returns the structured data as a JSON-formatted string within the completion_message.content.text field.
Structured output response
Next steps
Explore the full capabilities of Llama API with these resources:
•Extended Guide: See the Chat and conversation guide for examples of multi-turn conversations, memory management, and streaming.