我的代码从 PDF 个文件中提取文本，并比较信息。 执行大尺寸 Pdf 时，我的代码似乎失败了

Question

我可以使用我的代码来比较较小尺寸的 PDF，但是当它用于较大尺寸的 PDF 时，它会失败并显示各种错误消息。 下面是我的代码：

`

import pdfminer
import pandas as pd
from time import sleep
from tqdm import tqdm
from itertools import chain
import slate



# List of pdf files to process
pdf_files = ['file1.pdf', 'file2.pdf']

# Create a list to store the text from each PDF
pdf1_text = []
pdf2_text = []

# Iterate through each pdf file
for pdf_file in tqdm(pdf_files):
    # Open the pdf file
    with open(pdf_file, 'rb') as pdf_now:
        # Extract text using slate
        text = slate.PDF(pdf_now)
        text = text[0].split('\n')
        if pdf_file == pdf_files[0]:    
            pdf1_text.append(text)
        else:
            pdf2_text.append(text)

    sleep(20)

pdf1_text = list(chain.from_iterable(pdf1_text))
pdf2_text = list(chain.from_iterable(pdf2_text))

differences = set(pdf1_text).symmetric_difference(pdf2_text)

## Create a new dataframe to hold the differences
differences_df = pd.DataFrame(columns=['pdf1_text', 'pdf2_text'])

# Iterate through the differences and add them to the dataframe
for difference in differences:
    # Create a new row in the dataframe with the difference from pdf1 and pdf2
    differences_df = differences_df.append({'pdf1_text': difference if difference in pdf1_text else '',
                                            'pdf2_text': difference if difference in pdf2_text else ''}, ignore_index=True)

# Write the dataframe to an excel sheet
differences_df = differences_df.applymap(lambda x: x.encode('unicode_escape').decode('utf-8') if isinstance(x, str) else x)

differences_df.to_excel('differences.xlsx', index=False, engine='openpyxl')


import openpyxl

import re

# Load the Excel file into a dataframe
df = pd.read_excel("differences.xlsx")

# Create a condition to check the number of words in each cell
for column in ["pdf1_text", "pdf2_text"]:
    df[f"{column}_word_count"] = df[column].str.split().str.len()
    condition = df[f"{column}_word_count"] < 10
    # Drop the rows that meet the condition
    df = df[~condition]

for column in ["pdf1_text", "pdf2_text"]:
    df = df.drop(f"{column}_word_count", axis=1)


# Save the modified dataframe to a new Excel file
df.to_excel("differences.xlsx", index=False)

我得到的最后一个错误是这个。 任何人都可以通过代码请 go 帮助我找到实际问题是什么。

TypeError: %d format: a real number is required, not bytes

Answer 1

如果您真的想将脚本的速度提高至少一个数量级，我建议使用 PyMuPDF 而不是 PyPDF2 或 pdfminer。 我通常测量小 10 到 35 倍 (.) 的持续时间，当然，没有time.sleep() - 你为什么要人为地减慢处理速度？

以下是使用 PyMuPDF 阅读两个 PDF 的文本行的方式：

import fitz  # PyMuPDF

doc1 = fitz.open("file1.pdf")
doc2 = fitz.open("file2.pdf")

text1 = "\n".join([page.get_text() for page in doc1])
text2 = "\n".join([page.get_text() for page in doc2])

lines1 = text1.splitlines()
lines2 = text2.splitlines()

# then do your comparison ...