OpenAI has unveiled “Structured Outputs”, a new API feature designed to address the long-standing challenge of reliably generating structured data from large language models (LLMs). The feature, available now, guarantees that model-generated outputs will adhere to developer-defined JSON Schemas.
Generating structured data from unstructured input is a cornerstone of many AI applications today. Developers leverage the OpenAI API to build sophisticated assistants capable of fetching data, answering complex questions via function calling, extracting structured data for seamless data entry, and enabling multi-step workflows where LLMs can take specific actions.
However, the inherent limitations of LLMs in consistently producing structured output have led developers to employ workarounds such as open-source tooling, intricate prompting techniques, and repeated request retries. These workarounds, while functional, add complexity and compromise efficiency.
OpenAI’s Structured Outputs promises to eliminate these workarounds. It achieves this by constraining OpenAI models to match developer-supplied schemas and by training models to better understand and adhere to complex data structures.
“Structured Outputs solves this problem by constraining OpenAI models to match developer-supplied schemas and by training our models to better understand complicated schemas,” OpenAI said in a blog post.
Internal evaluations using complex JSON schemas have shown remarkable results. The latest model, gpt-4o-2024-08-06, achieved a perfect 100% score in adherence to structured outputs, a significant improvement over the previous gpt-4-0613, which scored less than 40%.
Structured Outputs is offered in two key implementations:
- Function calling: This method, enabled by setting strict: true within a function definition, allows developers to define the precise structure of data returned by functions called by the model. This feature is compatible with all models supporting tools, including gpt-4-0613, gpt-3.5-turbo-0613, and later versions.
- Response format parameter: This approach allows developers to provide a JSON Schema via the new json_schema option within the response_format parameter. This is particularly useful when the model needs to respond directly to users in a structured format without invoking tools. Currently, this feature is supported by the latest GPT-4o models: gpt-4o-2024-08-06 and gpt-4o-mini-2024-07-18. Setting strict: true within the response_format ensures the model output conforms to the provided schema.
OpenAI has prioritised safety in Structured Outputs, ensuring it aligns with pre-existing safety policies. The model retains the ability to refuse unsafe requests, indicated to developers by a new refusal string value in API responses. This allows for programmatic detection of refusals, ensuring predictable behaviour and simplified error handling. Notably, the absence of a refusal value and a successful generation process (signified by finish_reason) guarantee a valid JSON output matching the developer-defined schema.
Native support for Structured Outputs has been integrated into OpenAI’s Python and Node SDKs, simplifying its use. Developers can define schemas for tools or response formats by providing a Pydantic or Zod object, which the SDKs automatically convert to JSON Schema. The SDKs also manage the deserialisation of JSON responses into typed data structures and handle potential refusals.