Generative AI training and extraction lets you:

• Use zero-shot and few-shot technology to get a high performing model with little to no training data using the foundation model.
• Use fine-tuning to further boost accuracy as you provide more and more training data.

Generative AI training methods

The training method you choose depends on the amount of documents you have available and the amount of effort available to put into training your model. There are three ways to train a generative AI model:

The following models are available for custom extractor. To change modelversions, seeManage processor versions.

Versions 1.4, 1.5 and 1.5 Pro support confidence scores.

To change the processor version in your project, reviewManaging processorversions.

To make a Quota Increase Request (QIR) for the default processor quota, followthe steps torequest a quota adjustment.

Initial setup

If not done so already, enable billing and theDocument AI APIs.

Build and evaluate a generative AI model

1. Create a processor anddefine fields you want to extract followingbest practices, which is important because it impacts extraction quality.

   • Go toWorkbench >Custom extractor >Create processor >Assign a name.

   

   • Go toGet started >Create new field.

   

   Note: The field names with the foundation model can greatly affect model accuracy and performance. A descriptive name is recommended.

2. Import documents

   • Import documents withauto-labeling and assign documents to the training and test set.
   • For zero-shot, only the schema is required. To evaluate the accuracy of the model, only a testing set is needed.
   • For few-shot, we recommend five training documents.
   • The number of needed testing documents depend on the use case. Generally, more testing documents is better.
   • Confirm or edit thelabels in the document.

3. Train model:

   • Select theBuild, thenCreate new version.
   • Enter a name and selectCreate.

   

1. Evaluation:

   • Go toEvaluate & test, select the version you just trained, then selectView full evaluation.

   

   • You now see metrics such asf1, precision, and recall for the entire document and each field.
   • Decide if performance meets your production goals. If it does not then reevaluate training and testing sets.

2. Set a new version as default:

   • Navigate toManage versions.
   • Select to expand the options, and then selectSet as default.
   Note: Verify theStatus of the version is deployed. If it is not, select the deploy version first.You can now set the new version as the default.

   

   Your model is now deployed. Documents sent to this processor use your custom version.You can evaluate themodel's performance to check if it requires further training.

Evaluation reference

The evaluation engine can do both exact match orfuzzy matching.For an exact match, the extracted value must exactly match the ground truth or is counted as a miss.

Fuzzy matching extractions that had slight differences such as capitalizationdifferences still count as a match. This can be changed at theEvaluation screen.

Fine-tuning

With fine-tuning, you use hundreds or thousands of documents for your training.

1. Create a processor and define fields you want to extract followingbest practices, which is important because it impacts extraction quality.

   Note: The field names with the foundation model can greatly affect model accuracy and performance. Be sure to give a descriptive name.

2. Import documents with auto-labeling, and assign documents to the training and test set.

3. Confirm or edit thelabels in the document.

4. Train model.

   • Select theBuild tab, and selectCreate New Version in theFine-tuning box.

   

   Note: Fine-tuning will tune a foundation model, which is recommended, whileTrain a custom model will train a conventional non-Generative AI based model.

   • Try out the default training parameters or values provided.If the results are unsatisfactory, experiment with these advanced options:

   • Training steps (between 100 and 400): Controls how often the weights areoptimized on a batch of data during the tuning.

     • Too low indicates a risk that the training ends before convergence (under-fitting).
     • Too high means the model might see the same batch of data multiple timesduring the training, which can lead to overfitting.
     • Fewer steps leads to faster training time. Higher counts can help for documentswith little template variation (and lower ones for those with more variation).

   • Learning rate multiplier (between 0.1 and 10): Controls how quickly the modelparameters are optimized on the training data. It roughly corresponds to the size of each training step.

     • Low rates mean small changes in the model weights at each training step.If too low, the model might not converge to a stable solution.
     • High rates indicate large changes, and too high can mean the model steps overthe optimal solution and converges instead to a suboptimal solution.
     • Training time is not affected by the choice of learning rate.

   • Give a name, select the required base processor version, and selectCreate.

   

5. Evaluation: Go toEvaluate & test, then select the version you just trained and selectView full evaluation.

   

   • You now see metrics such asf1, precision, and recallfor the entire document and each field.
   • Decide if performance meets your production goals. If not, then further trainingdocuments might be required.

6. Set a new version as the default:

   • Navigate toManage versions.
   • Select to expand the options, and selectSet as default.

   

   Note: Verify the status of the version is deployed. If it's not, then instead selectDeploy version.You can now set the new version as the default when its status is changed to deployed.

   Your model is now deployed and documents sent to this processor now use your custom version.You want to evaluate themodel's performance to check if it requires further training.

Auto-labeling with the foundation model

The foundation model can accurately extract fields for a variety of document types,but you can also provide additional training data to improve the accuracy of the modelfor specific document structures.

Document AI uses the label names you define and previous annotations to make itquicker and easier to label documents at scale with auto-labeling.

1. When you've created a custom processor, go to theGet Started tab.
2. SelectCreate New Field.

3. Give the label a descriptive, distinct name. ChooseExtract for values directly from the document orDerive for values inferred by the system. This improves the foundation model's accuracy and performance.

   

4. For extraction accuracy and performance add a description (such as added context, insights, and prior knowledge for each entity) for the kinds of entities it shouldpick up.

   Note: Regardless of content structure or variations, rules promote reliable and consistent extractions. To guide the model and minimize confusion in the document, include property descriptions with location information and text patterns of the property values.

   

   Note: It's possible to update the description. However, the change won't bepropagated to custom processor version. A snapshot of the description is inthe custom processor version.

5. Navigate to theBuild tab, then selectImport Documents.

   

6. Select the path of the documents and which set the documents should be imported into.Check the auto-labeling option and select the foundation model.

7. In theBuild tab, selectManage Dataset.

8. When you see your imported documents, select one of them.

   

The predictions from the model are now shown highlighted in purple.

1. Review each label predicted by the model, and verify it's correct.

2. If there are missing fields, add those as well.

   Note: It's important that all fields are as accurate as possible, or model performance will suffer. More details onlabeling.

   

3. When the document has been reviewed, selectMark as Labeled. The document is now ready to be used by the model.

4. Make sure the document is in either the testing or training set.

Three-level nesting

Custom Extractor now provides three levels of nesting. This feature provides better extraction for complex tables.

• processors.get
• processorVersions.get

The response of these is aProcessorVersion, which contains themodelType field inv1beta3 preview.

Procedure and example

We are using this sample:

1. SelectGet Started, and then create a field:

   • Create the top level.
   • In this sample, theofficer_appointments is used.
   • SelectThis is a parent label.
   • SelectOccurrence:Optional multiple.

   

2. SelectAdd child field. The second level label can now be created:

   • For this level label, createofficer.
   • SelectThis is a parent label.
   • SelectOccurrence:Optional multiple.

   

3. SelectAdd child field from second levelofficer. Create child labels for the third level of nesting.

4. When your schema is set, you can get predictions from documents with three levels of nesting using auto-labeling.

   

Label cross-page nested entities

Thepretrained-foundation-model-v1.5-2025-05-05 processor supports three-level nesting acrosspages.

1. Label an entity normally across a page.Note: The labeled entity will only be visible on the page where it's labeled,with the navigation bar changing page to page. By pinning the parent entity,this navigation bar persists.

   

2. Pin the parent entity with children you want to label across pages.

   

3. Navigate to the page with the child entity or entities to label.

For how to set the property descriptions, refer toUpdate document schema.

Send a processing request with property descriptions

If the document schema already has descriptions set, you can send a process requestwith the instructions atSend a processing request.

Fine-tune a processor with property descriptions

Before using any of the request data, make the following replacements:

• LOCATION: your processor'slocation, for example:
  • us - United States
  • eu - European Union
• PROJECT_ID: Your Google Cloud project ID.
• PROCESSOR_ID: the ID of your custom processor.
• DISPLAY_NAME: Display name for the processor.
• PRETRAINED_PROCESSOR_VERSION: the processor version identifier. Refer toSelect a processor version for more information. For example:
  • pretrained-TYPE-vX.X-YYYY-MM-DD
  • stable
  • rc
• TRAIN_STEPS: Training steps for model fine-tuning.
• LEARN_RATE_MULTIPLIER: Learning rate multiplier for model finetuning.
• DOCUMENT_SCHEMA: Schema for the processor. Refer toDocumentSchema representation.

HTTP method and URL:

POST https://LOCATION-documentai.googleapis.com/v1beta3/projects/PROJECT_ID/locations/LOCATION/processors/PROCESSOR_ID/processorVersions/PROCESSOR_VERSION:process

Request JSON body:

{  "rawDocument": {      "parent": "projects/PROJECT_ID/locations/LOCATION/processors/PROCESSOR_ID",         "processor_version": {             "name": "projects/PROJECT_ID/locations/LOCATION/processors/PROCESSOR_ID/processorVersions/DISPLAY_NAME",             "display_name": "DISPLAY_NAME",             "model_type": "MODEL_TYPE_GENERATIVE",         },         "base_processor_version": "projects/PROJECT_ID/locations/us/processors/PROCESSOR_ID/processorVersions/PRETRAINED_PROCESSOR_VERSION",         "foundation_model_tuning_options": {             "train_steps":TRAIN_STEPS,             "learning_rate_multiplier":LEARN_RATE_MULTIPLIER,         }         "document_schema":DOCUMENT_SCHEMA      }}

To send your request, choose one of these options:

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://LOCATION-documentai.googleapis.com/v1beta3/projects/PROJECT_ID/locations/LOCATION/processors/PROCESSOR_ID/processorVersions/PROCESSOR_VERSION:process"

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://LOCATION-documentai.googleapis.com/v1beta3/projects/PROJECT_ID/locations/LOCATION/processors/PROCESSOR_ID/processorVersions/PROCESSOR_VERSION:process" | Select-Object -Expand Content

Custom extractor with signature detection

This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of theService Specific Terms. Pre-GA products and features are available "as is" and might have limited support. For more information, see thelaunch stage descriptions.

(Public preview) Custom extractor supportssignature detection. This feature lets you detect the presence ofsignatures in the documents. Signature detection is available only by using thederived method type. You can specify a schema with the entity typesignaturefor such entities. The signature entities are derived using visual cues fromthe document.

For examples and configuration instructions, clickCustom extractor with derivedfield and signature detection.

Custom extractor with derived fields

This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of theService Specific Terms. Pre-GA products and features are available "as is" and might have limited support. For more information, see thelaunch stage descriptions.

The custom extractor supports derived fields. It lets you configure a field to bepopulated through intelligent inference or generation based on document context,rather than direct text extraction. You can employ this for use cases such asdeducing the country from an address, summarizing a document, counting items ina table, or detecting if an ID is authentic, without requiring the value to beexplicitly present in the text.

For examples and configuration instructions, clickCustom extractor with derivedfield and signature detection.

What's next

Learn aboutCustom extractor with derived field and signature detection.