TypeError: 'PDFObjRef' object is not iterable

[sanjays-rao]

Initial Checks

• I confirm that I'm on the latest version

Description

Hi, need some help regarding the above error I'm facing while parsing my document. Few PDFs are not able to be parsed. Not able to understand why. It is an medical invoice PDF, where I am aiming to extract the text contents along with their bounding box coordinates.

Example Code

import openparse

basic_doc_path = "/home/sanjayr/Workspace/30-claims/42969914.pdf"
parser = openparse.DocumentParser()
parsed_basic_doc = parser.parse(basic_doc_path)

Python, open-parse & OS Version

python_version: 3.8.20
             operating_system: Linux
                   os_version: 5.15.0-1074-azure
           open-parse version: 0.7.0
                 install path: /home/sanjayr/.conda/envs/be-env/lib/python3.8/site-packages/openparse
               python version: 3.8.20 (default, Oct  3 2024, 15:24:27)  [GCC 11.2.0]
                     platform: Linux-5.15.0-1074-azure-x86_64-with-glibc2.17
             related packages: PyMuPDF-1.24.11 pydantic-2.10.4 tokenizers-0.20.3 transformers-4.46.3 torch-2.4.1 torchvision-0.19.1

[sanjays-rao]

----> 5 parsed_basic_doc = parser.parse(basic_doc_path)

File ~/.conda/envs/be-env/lib/python3.8/site-packages/openparse/doc_parser.py:100, in DocumentParser.parse(self, file, ocr)
95 doc = Pdf(file)
97 text_engine: Literal["pdfminer", "pymupdf"] = (
98 "pdfminer" if not ocr else "pymupdf"
99 )
--> 100 text_elems = text.ingest(doc, parsing_method=text_engine)
101 text_nodes = self._elems_to_nodes(text_elems)
103 table_nodes = []

File ~/.conda/envs/be-env/lib/python3.8/site-packages/openparse/text/parse.py:19, in ingest(doc, parsing_method)
12 """
13 Default to pdfminer-based implementation.
14
15 Optional use the PyMuPDF-based implementation which has identical behaviour but supports OCR. Important - see their licensing to view the terms of use:
16 https://mupdf.com/licensing/index.html
17 """
18 if parsing_method == "pdfminer":
---> 19 return pdfminer.ingest(doc)
20 elif parsing_method == "pymupdf":
21 return pymupdf.ingest(doc)

File ~/.conda/envs/be-env/lib/python3.8/site-packages/openparse/text/pdfminer/core.py:214, in ingest(pdf_input)
211 elements = []
212 page_layouts = pdf_input.extract_layout_pages()
--> 214 for page_num, page_layout in enumerate(page_layouts):
215 page_width = page_layout.width
216 page_height = page_layout.height

File ~/.conda/envs/be-env/lib/python3.8/site-packages/openparse/pdf.py:116, in Pdf.extract_layout_pages(self)
109 """
110 Yields layout objects for each page in the PDF using pdfminer.six.
111 """
112 assert (
113 self.file_path is not None
114 ), "PDF file path is required for this method for now."
--> 116 yield from extract_pages(self.file_path)

File ~/.conda/envs/be-env/lib/python3.8/site-packages/pdfminer/high_level.py:209, in extract_pages(pdf_file, password, page_numbers, maxpages, caching, laparams)
207 device = PDFPageAggregator(resource_manager, laparams=laparams)
208 interpreter = PDFPageInterpreter(resource_manager, device)
--> 209 for page in PDFPage.get_pages(
210 fp, page_numbers, maxpages=maxpages, password=password, caching=caching
211 ):
212 interpreter.process_page(page)
213 layout = device.get_result()

File ~/.conda/envs/be-env/lib/python3.8/site-packages/pdfminer/pdfpage.py:171, in PDFPage.get_pages(cls, fp, pagenos, maxpages, password, caching, check_extractable)
169 log.warning(warning_msg)
170 # Process each page contained in the document.
--> 171 for (pageno, page) in enumerate(cls.create_pages(doc)):
172 if pagenos and (pageno not in pagenos):
173 continue

File ~/.conda/envs/be-env/lib/python3.8/site-packages/pdfminer/pdfpage.py:127, in PDFPage.create_pages(cls, document)
125 objects = search(document.catalog["Pages"], document.catalog)
126 for (objid, tree) in objects:
--> 127 yield cls(document, objid, tree, next(page_labels))
128 pages = True
129 if not pages:
130 # fallback when /Pages is missing.

File ~/.conda/envs/be-env/lib/python3.8/site-packages/pdfminer/pdfpage.py:63, in PDFPage.init(self, doc, pageid, attrs, label)
59 self.lastmod = resolve1(self.attrs.get("LastModified"))
60 self.resources: Dict[object, object] = resolve1(
61 self.attrs.get("Resources", dict())
62 )
---> 63 mediabox_params: List[Any] = [
64 resolve1(mediabox_param) for mediabox_param in self.attrs["MediaBox"]
65 ]
66 self.mediabox: Rect = resolve1(mediabox_params)
67 if "CropBox" in self.attrs:

[Binary67]

This is probably the upstream issue from pdfminer.six (pdfminer/pdfminer.six#1004) and is already fixed in (pdfminer/pdfminer.six#1027). But the current requirements.txt and pyproject.toml is still using the old version. Perhaps we have to change it to pdfminer.six >= 20250324