Managing Document AI processors with Python
1. Overview

What is Document AI?
Document AI is a platform that lets you extract insights from your documents. At heart, it offers a growing list of document processors (also called parsers or splitters, depending on their functionality).
There are two ways you can manage Document AI processors:
- manually, from the web console;
- programmatically, using the Document AI API.
Here is an example screenshot showing your processor list, both from the web console and from Python code:

In this lab, you will focus on managing Document AI processors programmatically with the Python client library.
What you'll see
- How to set up your environment
- How to fetch processor types
- How to create processors
- How to list project processors
- How to use processors
- How to enable/disable processors
- How to manage processor versions
- How to delete processors
What you'll need
Survey
How will you use this tutorial?
How would you rate your experience with Python?
How would you rate your experience with Google Cloud services?
2. Setup and requirements
Self-paced environment setup
- Sign-in to theGoogle Cloud Console and create a new project or reuse an existing one. If you don't already have a Gmail or Google Workspace account, you mustcreate one.



- TheProject name is the display name for this project's participants. It is a character string not used by Google APIs. You can always update it.
- TheProject ID is unique across all Google Cloud projects and is immutable (cannot be changed after it has been set). The Cloud Console auto-generates a unique string; usually you don't care what it is. In most codelabs, you'll need to reference your Project ID (typically identified as
PROJECT_ID). If you don't like the generated ID, you might generate another random one. Alternatively, you can try your own, and see if it's available. It can't be changed after this step and remains for the duration of the project. - For your information, there is a third value, aProject Number, which some APIs use. Learn more about all three of these values in thedocumentation.
Caution: A project ID is globally unique and can't be used by anyone else after you've selected it. You are the only user of that ID. Even if a project is deleted, the ID can't be used again
Note: If you use a Gmail account, you can leave the default location set toNo organization. If you use a Google Workspace account, choose a location that makes sense for your organization.
- Next, you'll need toenable billing in the Cloud Console to use Cloud resources/APIs. Running through this codelab won't cost much, if anything at all. To shut down resources to avoid incurring billing beyond this tutorial, you can delete the resources you created or delete the project. New Google Cloud users are eligible for the$300 USD Free Trial program.
Start Cloud Shell
While Google Cloud can be operated remotely from your laptop, in this lab you are usingCloud Shell, a command line environment running in the Cloud.
Activate Cloud Shell
- From the Cloud Console, clickActivate Cloud Shell
.

If this is your first time starting Cloud Shell, you're presented with an intermediate screen describing what it is. If you were presented with an intermediate screen, clickContinue.

It should only take a few moments to provision and connect to Cloud Shell.

This virtual machine is loaded with all the development tools needed. It offers a persistent 5 GB home directory and runs in Google Cloud, greatly enhancing network performance and authentication. Much, if not all, of your work in this codelab can be done with a browser.
Once connected to Cloud Shell, you should see that you are authenticated and that the project is set to your project ID.
- Run the following command in Cloud Shell to confirm that you are authenticated:
gcloudauthlistCommand output
Credentialed AccountsACTIVE ACCOUNT* <my_account>@<my_domain.com>To set the active account, run: $ gcloud config set account `ACCOUNT`
Note: Thegcloud command-line tool is the powerful and unified command-line tool in Google Cloud. It comes preinstalled in Cloud Shell. You will notice its support for tab completion.You may be prompted to authenticate the first time you run a command. For more information, seegcloud command-line tool overview.
- Run the following command in Cloud Shell to confirm that the gcloud command knows about your project:
gcloudconfiglistprojectCommand output
[core]project = <PROJECT_ID>
If it is not, you can set it with this command:
gcloudconfigsetproject <PROJECT_ID>Command output
Updated property [core/project].
Note: If you're completing this tutorial outside of Cloud Shell, followSet up Application Default Credentials.
3. Environment setup
Before you can begin using Document AI, run the following command in Cloud Shell to enable the Document AI API:
gcloudservicesenabledocumentai.googleapis.comYou should see something like this:
Operation "operations/..." finished successfully.
Now, you can use Document AI!
Navigate to your home directory:
cd~Create a Python virtual environment to isolate the dependencies:
virtualenvvenv-docaiActivate the virtual environment:
sourcevenv-docai/bin/activateTo stop using the virtual environment and go back to your system Python version, you can use thedeactivate command.
Install IPython, the Document AI client library, and python-tabulate (which you'll use to pretty-print the request results):
pipinstallipythongoogle-cloud-documentaitabulateYou should see something like this:
...Installing collected packages: ..., tabulate, ipython, google-cloud-documentaiSuccessfully installed ... google-cloud-documentai-2.15.0 ...
Now, you're ready to use the Document AI client library!
If you're setting up your own Python development environment outside of Cloud Shell, follow theseguidelines.
Set the following environment variables:
exportPROJECT_ID=$(gcloudconfigget-valuecore/project)# Choose "us" or "eu"exportAPI_LOCATION="us"- Both environment variables will be used in your application.
API_LOCATIONlets you choose where the requests will take place ("us"for the United States,"eu"for the European Union). You can use processors in multiple locations, but you'll use a single location throughout this lab to keep it simple.
From now on, all steps should be completed in the same session.
If you start a new session, you'll need to restart from here after restoring your environment with the following specific parts from this section:
- Activate your virtual environment
- Set and check your environment variables
Make sure your environment variables are properly defined:
echo$PROJECT_IDecho$API_LOCATIONIn the next steps, you'll use an interactive Python interpreter calledIPython, which you just installed. Start a session by runningipython in Cloud Shell:
ipythonYou should see something like this:
Python 3.12.3 (main, Feb 4 2025, 14:48:35) [GCC 13.3.0]Type 'copyright', 'credits' or 'license' for more informationIPython 9.1.0 -- An enhanced Interactive Python. Type '?' for help.In [1]:
If needed, you can quit your IPython session with theexit command.
Copy the following code into your IPython session:
importosfromtypingimportIterator,MutableSequence,Optional,Sequence,Tupleimportgoogle.cloud.documentai_v1asdocaifromtabulateimporttabulatePROJECT_ID=os.getenv("PROJECT_ID","")API_LOCATION=os.getenv("API_LOCATION","")assertPROJECT_ID,"PROJECT_ID is undefined"assertAPI_LOCATIONin("us","eu"),"API_LOCATION is incorrect"# Test processorsdocument_ocr_display_name="document-ocr"form_parser_display_name="form-parser"test_processor_display_names_and_types=((document_ocr_display_name,"OCR_PROCESSOR"),(form_parser_display_name,"FORM_PARSER_PROCESSOR"),)defget_client()->docai.DocumentProcessorServiceClient:client_options={"api_endpoint":f"{API_LOCATION}-documentai.googleapis.com"}returndocai.DocumentProcessorServiceClient(client_options=client_options)defget_parent(client:docai.DocumentProcessorServiceClient)->str:returnclient.common_location_path(PROJECT_ID,API_LOCATION)defget_client_and_parent()->Tuple[docai.DocumentProcessorServiceClient,str]:client=get_client()parent=get_parent(client)returnclient,parent- This uses the
PROJECT_IDandAPI_LOCATIONenvironment variables defined earlier. - The
get_*()utility functions will be used in the next steps for processor requests.
You're ready to make your first request and fetch the processor types.
4. Fetching processor types
Before creating a processor in the next step, fetch the available processor types. You can retrieve this list withfetch_processor_types.
Add the following functions into your IPython session:
deffetch_processor_types()->MutableSequence[docai.ProcessorType]:client,parent=get_client_and_parent()response=client.fetch_processor_types(parent=parent)returnresponse.processor_typesdefprint_processor_types(processor_types:Sequence[docai.ProcessorType]):defsort_key(pt):return(notpt.allow_creation,pt.category,pt.type_)sorted_processor_types=sorted(processor_types,key=sort_key)data=processor_type_tabular_data(sorted_processor_types)headers=next(data)colalign=next(data)print(tabulate(data,headers,tablefmt="pretty",colalign=colalign))print(f"→ Processor types:{len(sorted_processor_types)}")defprocessor_type_tabular_data(processor_types:Sequence[docai.ProcessorType],)->Iterator[Tuple[str,str,str,str]]:deflocations(pt):return", ".join(sorted(loc.location_idforlocinpt.available_locations))yield("type","category","allow_creation","locations")yield("left","left","left","left")ifnotprocessor_types:yield("-","-","-","-")returnforptinprocessor_types:yield(pt.type_,pt.category,f"{pt.allow_creation}",locations(pt))List the processor types:
processor_types=fetch_processor_types()print_processor_types(processor_types)You should get something like the following:
+--------------------------------------+-------------+----------------+-----------+| type | category | allow_creation | locations |+--------------------------------------+-------------+----------------+-----------+| CUSTOM_CLASSIFICATION_PROCESSOR | CUSTOM | True | eu, us |...| FORM_PARSER_PROCESSOR | GENERAL | True | eu, us || LAYOUT_PARSER_PROCESSOR | GENERAL | True | eu, us || OCR_PROCESSOR | GENERAL | True | eu, us || BANK_STATEMENT_PROCESSOR | SPECIALIZED | True | eu, us || EXPENSE_PROCESSOR | SPECIALIZED | True | eu, us |...+--------------------------------------+-------------+----------------+-----------+→ Processor types: 19
- The
typecolumn shows the processor type string to use when creating a processor. - The
categorycolumn shows whether the processor is general, specialized, or a custom one. - The
allow_creationcolumn shows whether the processor is allowed for your project. To use a restricted processor type, you need to request access from theconsole and be approved beforehand.
If you get aPermissionDenied error, typeexit to quit IPython, and return to theSetup and requirements step.
Now, you have all the info needed to create processors in the next step.
5. Creating processors
To create a processor, callcreate_processor with a display name and a processor type.
Add the following function:
defcreate_processor(display_name:str,type:str)->docai.Processor:client,parent=get_client_and_parent()processor=docai.Processor(display_name=display_name,type_=type)returnclient.create_processor(parent=parent,processor=processor)At the project level, you identify processors by unique display names.
Create the test processors:
separator="="*80fordisplay_name,typeintest_processor_display_names_and_types:print(separator)print(f"Creating{display_name} ({type})...")try:create_processor(display_name,type)exceptExceptionaserr:print(err)print(separator)print("Done")You should get the following:
================================================================================Creating document-ocr (OCR_PROCESSOR)...================================================================================Creating form-parser (FORM_PARSER_PROCESSOR)...================================================================================Done
You have created new processors!
By default, upon creation, a processor is enabled and can be used right away. To prevent a processor from being used (for example, while setting up an architecture), you'll see how to disable a processor in a later step.
Next, see how to list the processors.
6. Listing project processors
list_processors returns the list of all the processors belonging to your project.
Add the following functions:
deflist_processors()->MutableSequence[docai.Processor]:client,parent=get_client_and_parent()response=client.list_processors(parent=parent)returnlist(response.processors)defprint_processors(processors:Optional[Sequence[docai.Processor]]=None):defsort_key(processor):returnprocessor.display_nameifprocessorsisNone:processors=list_processors()sorted_processors=sorted(processors,key=sort_key)data=processor_tabular_data(sorted_processors)headers=next(data)colalign=next(data)print(tabulate(data,headers,tablefmt="pretty",colalign=colalign))print(f"→ Processors:{len(sorted_processors)}")defprocessor_tabular_data(processors:Sequence[docai.Processor],)->Iterator[Tuple[str,str,str]]:yield("display_name","type","state")yield("left","left","left")ifnotprocessors:yield("-","-","-")returnforprocessorinprocessors:yield(processor.display_name,processor.type_,processor.state.name)Call the functions:
processors=list_processors()print_processors(processors)You should get the following:
+--------------+-----------------------+---------+| display_name | type | state |+--------------+-----------------------+---------+| document-ocr | OCR_PROCESSOR | ENABLED || form-parser | FORM_PARSER_PROCESSOR | ENABLED |+--------------+-----------------------+---------+→ Processors: 2
To retrieve a processor by its display name, add the following function:
defget_processor(display_name:str,processors:Optional[Sequence[docai.Processor]]=None,)->Optional[docai.Processor]:ifprocessorsisNone:processors=list_processors()forprocessorinprocessors:ifprocessor.display_name==display_name:returnprocessorreturnNoneTest the function:
processor=get_processor(document_ocr_display_name,processors)assertprocessorisnotNoneprint(processor)You should see something like this:
name: "projects/PROJECT_NUM/locations/LOCATION/processors/PROCESSOR_ID"type_: "OCR_PROCESSOR"display_name: "document-ocr"state: ENABLED...
Now, you know how to list your project processors and retrieve them by their display names. Next, see how to use a processor.
7. Using processors
Documents can be processed in two ways:
- Synchronously: Call
process_documentto analyze a single document and directly use the results. - Asynchronously: Call
batch_process_documentsto launch a batch processing on multiple or large documents.
Your test document (PDF) is a scanned questionnaire completed with handwritten answers. Download it into your working directory, directly from your IPython session:
!gsutilcpgs://cloud-samples-data/documentai/form.pdf.Check the content of your working directory:
!lsYou should have the following:
... form.pdf ... venv-docai ...
You can use the synchronousprocess_document method to analyze a local file. Add the following function:
defprocess_file(processor:docai.Processor,file_path:str,mime_type:str,)->docai.Document:client=get_client()withopen(file_path,"rb")asdocument_file:document_content=document_file.read()document=docai.RawDocument(content=document_content,mime_type=mime_type)request=docai.ProcessRequest(raw_document=document,name=processor.name)response=client.process_document(request)returnresponse.documentThe API returns adocument object containing the insights extracted by the chosen processor.
Because your document is a questionnaire, choose the form parser. In addition to extracting the text (printed and handwritten), which all processors do, this general processor detects form fields.
Analyze the document:
processor=get_processor(form_parser_display_name)assertprocessorisnotNonefile_path="./form.pdf"mime_type="application/pdf"document=process_file(processor,file_path,mime_type)All processors run an Optical Character Recognition (OCR) first pass on the document. Review the text detected by the OCR pass:
document.text.split("\n")You should see something like the following:
['FakeDoc M.D.', 'HEALTH INTAKE FORM', 'Please fill out the questionnaire carefully. The information you provide will be used to complete', 'your health profile and will be kept confidential.', 'Date:', '9/14/19', 'Name:', 'Sally Walker', 'DOB: 09/04/1986', 'Address: 24 Barney Lane', 'City: Towaco', 'State: NJ Zip: 07082', 'Email: Sally, walker@cmail.com', '_Phone #: (906) 917-3486', 'Gender: F', 'Marital Status:', ...]
Add the following functions to print out the detected form fields:
defprint_form_fields(document:docai.Document):sorted_form_fields=form_fields_sorted_by_ocr_order(document)data=form_field_tabular_data(sorted_form_fields,document)headers=next(data)colalign=next(data)print(tabulate(data,headers,tablefmt="pretty",colalign=colalign))print(f"→ Form fields:{len(sorted_form_fields)}")defform_field_tabular_data(form_fields:Sequence[docai.Document.Page.FormField],document:docai.Document,)->Iterator[Tuple[str,str,str]]:yield("name","value","confidence")yield("right","left","right")ifnotform_fields:yield("-","-","-")returnforform_fieldinform_fields:name_layout=form_field.field_namevalue_layout=form_field.field_valuename=text_from_layout(name_layout,document)value=text_from_layout(value_layout,document)confidence=value_layout.confidenceyield(name,value,f"{confidence:.1%}")Also add these utility functions:
defform_fields_sorted_by_ocr_order(document:docai.Document,)->MutableSequence[docai.Document.Page.FormField]:defsort_key(form_field):# Sort according to the field name detected positiontext_anchor=form_field.field_name.text_anchorreturntext_anchor.text_segments[0].start_indexiftext_anchorelse0fields=(fieldforpageindocument.pagesforfieldinpage.form_fields)returnsorted(fields,key=sort_key)deftext_from_layout(layout:docai.Document.Page.Layout,document:docai.Document,)->str:full_text=document.textsegs=layout.text_anchor.text_segmentstext="".join(full_text[seg.start_index:seg.end_index]forseginsegs)iftext.endswith("\n"):text=text[:-1]returntextPrint the detected form fields:
print_form_fields(document)You should get a printout like the following:
+-----------------+-------------------------+------------+| name | value | confidence |+-----------------+-------------------------+------------+| Date: | 9/14/19 | 83.0% || Name: | Sally Walker | 87.3% || DOB: | 09/04/1986 | 88.5% || Address: | 24 Barney Lane | 82.4% || City: | Towaco | 90.0% || State: | NJ | 89.4% || Zip: | 07082 | 91.4% || Email: | Sally, walker@cmail.com | 79.7% || _Phone #: | walker@cmail.com | 93.2% || | (906 | || Gender: | F | 88.2% || Marital Status: | Single | 85.2% || Occupation: | Software Engineer | 81.5% || Referred By: | None | 76.9% |...+-----------------+-------------------------+------------+→ Form fields: 17
Review the field names and values that have been detected (PDF). Here is the top half of the questionnaire:

You have analyzed a form that contains both printed and handwritten text. You have also detected its fields with high confidence. The result is that your pixels have been transformed into structured data!
8. Enabling and disabling processors
Withdisable_processor andenable_processor, you can control whether a processor can be used.
Add the following functions:
defupdate_processor_state(processor:docai.Processor,enable_processor:bool):client=get_client()ifenable_processor:request=docai.EnableProcessorRequest(name=processor.name)operation=client.enable_processor(request)else:request=docai.DisableProcessorRequest(name=processor.name)operation=client.disable_processor(request)operation.result()# Wait for operation to completedefenable_processor(processor:docai.Processor):update_processor_state(processor,True)defdisable_processor(processor:docai.Processor):update_processor_state(processor,False)Disable the form parser processor, and check the state of your processors:
processor=get_processor(form_parser_display_name)assertprocessorisnotNonedisable_processor(processor)print_processors()You should get the following:
+--------------+-----------------------+----------+| display_name | type | state |+--------------+-----------------------+----------+| document-ocr | OCR_PROCESSOR | ENABLED || form-parser | FORM_PARSER_PROCESSOR | DISABLED |+--------------+-----------------------+----------+→ Processors: 2
The form parser is disabled. If your application tries to use it, the API returns aFailedPrecondition: 400 Processor PROCESSOR_ID is not enabled error.
Re-enable the form parser processor:
enable_processor(processor)print_processors()You should get the following:
+--------------+-----------------------+---------+| display_name | type | state |+--------------+-----------------------+---------+| document-ocr | OCR_PROCESSOR | ENABLED || form-parser | FORM_PARSER_PROCESSOR | ENABLED |+--------------+-----------------------+---------+→ Processors: 2
Next, see how to manage processor versions.
9. Managing processor versions
Processors can be available in multiple versions. Check out how to use thelist_processor_versions andset_default_processor_version methods.
Add the following functions:
deflist_processor_versions(processor:docai.Processor,)->MutableSequence[docai.ProcessorVersion]:client=get_client()response=client.list_processor_versions(parent=processor.name)returnlist(response)defget_sorted_processor_versions(processor:docai.Processor,)->MutableSequence[docai.ProcessorVersion]:defsort_key(processor_version:docai.ProcessorVersion):returnprocessor_version.nameversions=list_processor_versions(processor)returnsorted(versions,key=sort_key)defprint_processor_versions(processor:docai.Processor):versions=get_sorted_processor_versions(processor)default_version_name=processor.default_processor_versiondata=processor_versions_tabular_data(versions,default_version_name)headers=next(data)colalign=next(data)print(tabulate(data,headers,tablefmt="pretty",colalign=colalign))print(f"→ Processor versions:{len(versions)}")defprocessor_versions_tabular_data(versions:Sequence[docai.ProcessorVersion],default_version_name:str,)->Iterator[Tuple[str,str,str]]:yield("version","display name","default")yield("left","left","left")ifnotversions:yield("-","-","-")returnforversioninversions:mapping=docai.DocumentProcessorServiceClient.parse_processor_version_path(version.name)processor_version=mapping["processor_version"]is_default="Y"ifversion.name==default_version_nameelse""yield(processor_version,version.display_name,is_default)List the available versions for the OCR processor:
processor=get_processor(document_ocr_display_name)assertprocessorisnotNoneprint_processor_versions(processor)You get the processor versions:
+--------------------------------+--------------------------+---------+| version | display name | default |+--------------------------------+--------------------------+---------+| pretrained-ocr-v1.0-2020-09-23 | Google Stable | || pretrained-ocr-v1.1-2022-09-12 | Google Release Candidate | || pretrained-ocr-v1.2-2022-11-10 | Google Release Candidate | || pretrained-ocr-v2.0-2023-06-02 | Google Stable | Y || pretrained-ocr-v2.1-2024-08-07 | Google Release Candidate | |+--------------------------------+--------------------------+---------+→ Processor versions: 5
You may useStable versions1.0 or2.0 in production, or also testRelease Candidate versions.
Now, add a function to change the default processor version:
defset_default_processor_version(processor:docai.Processor,version_name:str):client=get_client()request=docai.SetDefaultProcessorVersionRequest(processor=processor.name,default_processor_version=version_name,)operation=client.set_default_processor_version(request)operation.result()# Wait for operation to completeSwitch to the latest processor version:
processor=get_processor(document_ocr_display_name)assertprocessorisnotNoneversions=get_sorted_processor_versions(processor)new_version=versions[-1]# Latest versionset_default_processor_version(processor,new_version.name)# Update the processor infoprocessor=get_processor(document_ocr_display_name)assertprocessorisnotNoneprint_processor_versions(processor)You get the new version configuration:
+--------------------------------+--------------------------+---------+| version | display name | default |+--------------------------------+--------------------------+---------+| pretrained-ocr-v1.0-2020-09-23 | Google Stable | || pretrained-ocr-v1.1-2022-09-12 | Google Release Candidate | || pretrained-ocr-v1.2-2022-11-10 | Google Release Candidate | || pretrained-ocr-v2.0-2023-06-02 | Google Stable | || pretrained-ocr-v2.1-2024-08-07 | Google Release Candidate | Y |+--------------------------------+--------------------------+---------+→ Processor versions: 5
As a consequence, all document processing requests for this processor will now use the latest release candidate.
And next, the ultimate processor management method (deletion).
10. Deleting processors
Finally, check out how to use thedelete_processor method.
Add the following function:
defdelete_processor(processor:docai.Processor):client=get_client()operation=client.delete_processor(name=processor.name)operation.result()# Wait for operation to completeDelete your test processors:
processors_to_delete=[dnfordn,_intest_processor_display_names_and_types]print("Deleting processors...")forprocessorinlist_processors():ifprocessor.display_namenotinprocessors_to_delete:continueprint(f" Deleting{processor.display_name}...")delete_processor(processor)print("Done\n")print_processors()You should get the following:
Deleting processors... Deleting form-parser... Deleting document-ocr...Done+--------------+------+-------+| display_name | type | state |+--------------+------+-------+| - | - | - |+--------------+------+-------+→ Processors: 0
You've covered all the processor management methods! You're almost done...
11. Congratulations!

You learned how to manage Document AI processors using Python!
Clean up
To clean up your development environment, from Cloud Shell:
- If you're still in your IPython session, go back to the shell:
exit - Stop using the Python virtual environment:
deactivate - Delete your virtual environment folder:
cd ~ ; rm -rf ./venv-docai
To delete your Google Cloud project, from Cloud Shell:
- Retrieve your current project ID:
PROJECT_ID=$(gcloud config get-value core/project) - Make sure this is the project you want to delete:
echo $PROJECT_ID - Delete the project:
gcloud projects delete $PROJECT_ID
Learn more
- Try Document AI in your browser:https://cloud.google.com/document-ai/docs/drag-and-drop
- Document AI processor details:https://cloud.google.com/document-ai/docs/processors-list
- Python on Google Cloud:https://cloud.google.com/python
- Cloud Client Libraries for Python:https://github.com/googleapis/google-cloud-python
License
This work is licensed under a Creative Commons Attribution 2.0 Generic License.
Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.