Use custom classifier to classify documents. Build it from the ground up with yourown documents and custom classes. Its generative AI aspect allows few-shot learningand fine-tuning. These improve accuracy with fewer samples and corrections withiterative auto-labeling.

Custom classifier covers these three general use cases.

• Pretrained model: Use the pretrained generative AI foundation model toquickly classify documents with your supplied labels.
• Fine-tuning: Improve accuracy by training the generative AI foundation modelon your own data and labels.
• Train a custom model: Train a non-generative AI custom extractor using yourown data and labels.

Custom classifier model versions

Confidence scores are supported for custom classifier models inPreview. Forbest performance, use them with fine-tuned models.

Create a custom classifier in the Google Cloud console

You can create custom classifiers that are specifically suited to your documentsand trained and evaluated with your data. This processor identifies classes ofdocuments from a user-defined set of classes. You can then use this trained processoron additional documents. You typically would use a custom classifier on documentsthat are different types, then use the identification to pass the documents to anextraction processor to extract the entities.

For the general process to create and use a processor, see theHow tosection.

Guide me

Before you begin

Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Roles required to select or create a project

• Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
• Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.create permission.Learn how to grant roles.

Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

Go to project selector

If you're using an existing project for this guide,verify that you have the permissions required to complete this guide. If you created a new project, then you already have the required permissions.

Verify that billing is enabled for your Google Cloud project.

Enable the Document AI, Cloud Storage APIs.

Roles required to enable APIs

To enable APIs, you need the Service Usage Admin IAM role (roles/serviceusage.serviceUsageAdmin), which contains theserviceusage.services.enable permission.Learn how to grant roles.

Enable the APIs

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Roles required to select or create a project

• Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
• Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.create permission.Learn how to grant roles.

Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

Go to project selector

If you're using an existing project for this guide,verify that you have the permissions required to complete this guide. If you created a new project, then you already have the required permissions.

Verify that billing is enabled for your Google Cloud project.

Enable the Document AI, Cloud Storage APIs.

Roles required to enable APIs

To enable APIs, you need the Service Usage Admin IAM role (roles/serviceusage.serviceUsageAdmin), which contains theserviceusage.services.enable permission.Learn how to grant roles.

Enable the APIs

Required roles

To get the permissions that you need to create a custom classifier, ask your administrator to grant you the following IAM roles on your project:

• Document AI Administrator (roles/documentai.admin)
• Storage Admin (roles/storage.admin)

For more information about granting roles, seeManage access to projects, folders, and organizations.

You might also be able to get the required permissions throughcustom roles or otherpredefined roles.

Create a processor

Complete the following steps.

1. Go to theWorkbench

2. For custom document classifier, selectCreate processor.

   

3. In theCreate processor menu, enter a name for your processor, such asmy-custom-document-classifier.

   

4. Select the region closest to you.

5. SelectCreate. TheProcessor Details tab appears.

Configure dataset

To train this new processor, you must create a dataset with training andtesting data to help the processor identify the documents that you want to splitand classify. This dataset requires a new location. This can be an emptyCloud Storage bucketor folder, or you can allow an internally managed location.

After theProcessor Details tab appears, then you can:

1. SelectGoogle-managed storage in case you want to use Cloud Storage.
2. SelectI'll specify my own storage location if you want to use your ownstorage to use Customer-Managed Encryption Keys (CMEK), and follow theprocedure inCreate a dataset.

For more information about preparing your data for import, refer to theData preparation guide.

Define processor schema

You can create the processor schema either before or after you import documentsinto your dataset. The schema provides labels that you use to annotate documents.

1. On theBuild tab, selectManage Dataset >Edit Schema.TheEdit schema page opens.

2. SelectCreate label.

3. Enter the name for the label.

4. SelectCreate. Refer toDefine processor schemafor detailed instructions on creating and editing a schema.

   Note: When a processor is trained, labels cannot be deleted. Instead, you candisable any label you don't want to use.

5. Create each of the following labels for the processor schema.

   • computer_vision
   • crypto
   • med_tech
   • other
   Tip: Use the description field to enter a prompt which describes the label.This helps train the model and differentiate similarly written labels. Learnmore atlabel with property descriptions.

6. SelectSave when the labels are complete.

   

Assign annotated document to the training set

Now that you have labeled this example document, you can assign it to thetraining set.

1. On theManage Dataset tab, select theSelect Allcheckbox.

2. From theAssign to Setlist, selectTraining.

In theDocuments panel, you can find that one document has been assigned to thetraining set.

Import prelabeled data to the training and test sets

In this guide, you are provided with prelabeled data. If working on your ownproject, you have to determine how to label your data. Refer toLabeling options.

Document AI custom processors require a minimum of one document in both thetraining and test sets for each document type to be labeled. We recommend thatyou have at least 10 documents for each label for best performance. For 5 labels,you would need 50 documents to train and 50 to test. More training datatypically produces higher accuracy.

1. SelectImport documents.

2. Enter the following path inSource path. This bucket contains pre-labeled documents in theDocument JSON format.

   cloud-samples-data/documentai/Custom/Patents/JSON/Classification-InventionType

3. From theData split list, selectAuto-split. This automaticallysplits the documents to have 80% in the training set and 20% in the test set.Ignore theApply labels section.

4. SelectImport. The import might take several minutes to complete.

When the import is finished, you'll find the documents in theManage Datasettab.

Batch label documents at import

Optionally, after the schema has been configured, you can label all documents that are in aparticular directory at import to save time with labeling.

1. SelectImport documents.

2. Enter the following path inSource path. This bucket contains unlabeled documents in PDF format.

   cloud-samples-data/documentai/Custom/Patents/PDF-CDC-BatchLabel

3. From theData split list, selectAuto-split. This automatically splitsthe documents to have 80% in the training set and 20% in the test set.

4. In theApply labels section, selectChoose label.

5. For these sample documents, selectother.

6. SelectImport and wait for the process to finish. You can leave this pageand return later.When complete, you find the documents on theManage Dataset tab with the labelapplied.

Train the processor

Now that you have imported the training and test data, you can train the processor.Because training might take several hours, make sure you have set up the processorwith the appropriate data and labels before you begin training.

You can train fine-tuned and custom models with your labeled data. Fine-tunedmodels use generative AI. The custom models trains a unique large language Modelusing your labeled data. You need a minimum of two labels in the schema, with arecommended ten training documents and 10 test documents (minimum of 1).

1. SelectTrain New Version.

1. After deployment is complete, navigate to theEvaluate & Testtab.

   On this page, you can view evaluation metrics including the F1 score, precisionand recall for the full document, and individual labels. For more informationabout evaluation and statistics, refer toEvaluate processor.

2. Download a document that has not been involved in previous training or testingso that you can use it to evaluate the processor version. If using your own data,you would use a document set aside for this purpose.

   file_downloadDownload PDF

3. SelectUpload Test Document and select the document you just downloaded.

   TheCustom Document Classifier analysis page opens. The output demonstrateshow well the document was classified.

   You can also rerun the evaluation against a different test set or processorversion.

Auto-label newly imported documents

After deploying a trained processor version, you can useAuto-labeling to save time on labeling when importing new documents.

1. On theManage Dataset page,Import documents.

2. Copy and paste the following Cloud Storage path. This directory containsfive unlabeled patent PDFs. From theData split drop-down list, selectTraining.

   cloud-samples-data/documentai/Custom/Patents/PDF-CDC-AutoLabel

3. In theApply labels section, selectAuto-labeling.

4. Select an existing processor version to label the documents.

   • For example:2af620b2fd4d1fcf

5. SelectImport and wait for the process to finish. You can leave this pageand return later. When complete, the documents appear in theAuto-labeledsection of theManage Dataset page.

6. You cannot use auto-labeled documents for training or testing without markingthem as labeled. Go to theAuto-labeledsection to view the auto-labeled documents.

7. Select the first document to enter the labeling console.

8. Verify the label to ensure it's correct. Adjust if it's incorrect.

9. SelectMark as Labeled when finished.

10. Repeat the label verification for each auto-labeled document, then return totheManage Dataset page to assign the data for training.

Use the processor

You can manage your custom-trained processor versions just like any other processorversion. For more information, refer toManaging processor versions.

You can alsoSend a processing request to your customprocessor, and theresponse can be handled the same as other classifier processors.

Clean up

To avoid incurring charges to your Google Cloud account for the resources used on this page, follow these steps.

1. In the Google Cloud console navigation menu, selectDocument AI, thenMy Processors.

2. SelectMore actions in the same row as the processor you want to delete.

3. SelectDelete processor, enter the processor name, then selectDeleteagain to confirm.

What's next

• For more details, seeGuides.
• Review theprocessors list.
• Separate documents into readable chunks withLayout Parser.
• UseEnterprise Document OCR todetect and extract text.