Using Document AI with python from google and code from google codelabs returns wrong or empty result

Question

I tried the following code from codelabs.developers.google.com:

import pandas as pd
from google.cloud import documentai_v1 as documentai


def online_process(
    project_id: str,
    location: str,
    processor_id: str,
    file_path: str,
    mime_type: str,
) -> documentai.Document:
    """
    Processes a document using the Document AI Online Processing API.
    """

    opts = {"api_endpoint": f"{location}-documentai.googleapis.com"}

    # Instantiates a client
    documentai_client = documentai.DocumentProcessorServiceClient(client_options=opts)

    # The full resource name of the processor, e.g.:
    # projects/project-id/locations/location/processor/processor-id
    # You must create new processors in the Cloud Console first
    resource_name = documentai_client.processor_path(project_id, location, processor_id)

    # Read the file into memory
    with open(file_path, "rb") as file:
        file_content = file.read()

    # Load Binary Data into Document AI RawDocument Object
    raw_document = documentai.RawDocument(content=file_content, mime_type=mime_type)

    # Configure the process request
    request = documentai.ProcessRequest(name=resource_name, raw_document=raw_document)

    # Use the Document AI client to process the sample form
    result = documentai_client.process_document(request=request)

    return result.document


PROJECT_ID = "YOUR_PROJECT_ID"
LOCATION = "YOUR_PROJECT_LOCATION"  # Format is 'us' or 'eu'
PROCESSOR_ID = "PROCUREMENT_SPLITTER_ID"  # Create processor in Cloud Console

# The local file in your current working directory
FILE_PATH = "procurement_multi_document.pdf"
# Refer to https://cloud.google.com/document-ai/docs/processors-list
# for supported file types
MIME_TYPE = "application/pdf"

document = online_process(
    project_id=PROJECT_ID,
    location=LOCATION,
    processor_id=PROCESSOR_ID,
    file_path=FILE_PATH,
    mime_type=MIME_TYPE,
)

print("Document processing complete.")

types = []
confidence = []
pages = []

# Each Document.entity is a classification
for entity in document.entities:
    classification = entity.type_
    types.append(classification)
    confidence.append(f"{entity.confidence:.0%}")

    # entity.page_ref contains the pages that match the classification
    pages_list = []
    for page_ref in entity.page_anchor.page_refs:
        pages_list.append(page_ref.page)
    pages.append(pages_list)

# Create a Pandas Dataframe to print the values in tabular format.
df = pd.DataFrame({"Classification": types, "Confidence": confidence, "Pages": pages})

print(df)

Expected output (something like this):

$ python3 classification.py
Document processing complete.
         Classification Confidence Pages
0     invoice_statement       100%   [0]
1     receipt_statement        98%   [1]
2                 other        81%   [2]
3     utility_statement       100%   [3]
4  restaurant_statement       100%   [4]

Resulting output:

Document processing complete.
Empty DataFrame
Columns: [Classification, Confidence, Pages]
Index: []

I checked every other input to google documentAI and everything else seems fine. I can however get the text (ocr) using document.text without problem. Maybe I have reached the end of my documentAI quota? Any other reason or how to resolve this if possible? Thank you.

Answer 1

Document AI is a document understanding solution that uses machine learning to make unstructured data easier to understand and analyze. From the code that you have used is for Specialized Processor Procurement DocAI which takes input as invoice or receipts or utility statements which are unstructured documents and procure into structured data. The file types which are supported in Document AI are listed in thisdocument . The problem seems to be the type of file you are using. Try to use the file type listed in the documentation. For more information you can check this link .

I have tried the below code to get the expected output.

Code

import pandas as pd
from google.cloud import documentai_v1 as documentai


def online_process(
   project_id: "project",
   location: "us",
   processor_id: "processor-id",
   file_path: "file_path",
   mime_type: "mime_type",
) -> documentai.Document:
   
   opts = {"api_endpoint": f"{location}-documentai.googleapis.com"}

   # Instantiates a client
   documentai_client = documentai.DocumentProcessorServiceClient(client_options=opts)

   resource_name = documentai_client.processor_path(project_id, location, processor_id)

   # Read the file into memory
   with open(file_path, "rb") as file:
       file_content = file.read()

   # Load Binary Data into Document AI RawDocument Object
   raw_document = documentai.RawDocument(content=file_content, mime_type=mime_type)

   # Configure the process request
   request = documentai.ProcessRequest(name=resource_name, raw_document=raw_document)

   # Use the Document AI client to process the sample form
   result = documentai_client.process_document(request=request)

   return result.document


PROJECT_ID = "project"
LOCATION = "us" 
PROCESSOR_ID = "processor-id"  

# The local file in your current working directory
FILE_PATH = "file_path"
# Refer to https://cloud.google.com/document-ai/docs/processors-list
# for supported file types
MIME_TYPE = "mime_type"

document = online_process(
   project_id=PROJECT_ID,
   location=LOCATION,
   processor_id=PROCESSOR_ID,
   file_path=FILE_PATH,
   mime_type=MIME_TYPE,
)

print("Document processing complete.")

types = []
confidence = []
pages = []

# Each Document.entity is a classification
for entity in document.entities:
   classification = entity.type_
   types.append(classification)
   confidence.append(f"{entity.confidence:.0%}")

   # entity.page_ref contains the pages that match the classification
   pages_list = []
   for page_ref in entity.page_anchor.page_refs:
       pages_list.append(page_ref.page)
   pages.append(pages_list)

# Create a Pandas Dataframe to print the values in tabular format.
df = pd.DataFrame({"Classification": types, "Confidence": confidence, "Pages": pages})

print(df)

Outputs for different files :

1. procurement_multi_document.pdf

   

2. invoice.pdf

   

3. sample.pdf - pdf file only containing a paragraph

   

4. kitten.png - containing single image