Content Understanding combines OCR and layout analysis with generative models so that you describe the fields you want and get them back as structured output. It works across documents, images, audio and video. Figure 9.1 shows the path from content to structured result.
Setting it up
- Provision a Microsoft Foundry resource in a supported region and configure default model deployments on it.
- An Azure OpenAI resource on its own doesn’t expose Content Understanding, and a project is a child of the Foundry resource rather than a separate place to enable it. Azure AI Search doesn’t extract fields.
What it returns
- Content Understanding goes beyond OCR: it performs content extraction (OCR and layout) and then applies a field schema, returning structured JSON with markdown content and the extracted fields.
Analyzers
An analyzer is the configuration that processes one kind of content.
| Analyzer | Content | Notes |
|---|---|---|
| Document (prebuilt-document, prebuilt-invoice, prebuilt-read, prebuilt-layout) | PDFs (including scanned), Office files, forms | OCR and layout, then fields |
| Image | Still images | Not PDFs |
| Audio (prebuilt-audio, prebuilt-audioSearch, prebuilt-callCenter) | Voicemails, calls | Transcription to WebVTT with diarization, speaker roles, optional word timestamps, then fields |
| Video | Video files | Keyframes and scenes |
Defining fields
fieldSchemadefines what is extracted. For each field you give a name, a type (string, number, date, object, array and so on), a description and a method (extract,classifyorgenerate).- Object and array fields return nested values, such as invoice line items.
- The schema is both the output contract and guidance for the model.
Calling it from code
- Analysis is asynchronous:
begin_analyze()returns anLROPollerat once, andpoller.result()polls until the job completes and returns theAnalysisResult. Results are in the JSON body, not in headers. status()only reports the job’s state;wait()blocks but returns nothing;get_resultsdoesn’t exist.- Synchronous analysis exists only in preview, for prebuilt-read and prebuilt-layout.