multiprocessing.spawn on Linux imports paddleocr.py module instead of paddleocr package, causing ModuleNotFoundError for submodules

[DavGrg]

🔎 Search before asking

• I have searched the PaddleOCR Docs and found no similar bug report.
• I have searched the PaddleOCR Issues and found no similar bug report.
• I have searched the PaddleOCR Discussions and found no similar bug report.

🐛 Bug (问题描述)

When using Python’s multiprocessing with the spawn (or forkserver) start method, child processes re-import the paddleocr top‐level name from disk. In my environment I have the official PaddleOCR pip package installed under …/site-packages/paddleocr/init.py (alongside paddleocr/paddleocr.py and the tools/ subpackage), so I expect:

import paddleocr
print(paddleocr.__file__)
# -> .../site-packages/paddleocr/__init__.py
from paddleocr.tools.infer.utility import init_args  # should work

However, in the worker process spawned via:

import multiprocessing as mp

mp.set_start_method("spawn", force=True)
p = mp.Process(target=worker)
p.start()
p.join()

the interpreter instead binds paddleocr to …/site-packages/paddleocr/paddleocr.py (the standalone module), not the package’s __init__.py. As a result:

from paddleocr.tools.infer.utility import init_args
# ModuleNotFoundError: No module named 'paddleocr.tools'; 'paddleocr' is not a package

🏃‍♂️ Environment (运行环境)

OS Ubuntu 22.04.3 LTS
Python 3.12.0
PaddleOCR 2.10.0
PaddlePaddle 2.6.2

🌰 Minimal Reproducible Example (最小可复现问题的Demo)

import multiprocessing as mp
import paddleocr

def worker():
    try:
        from paddleocr.tools.infer.utility import init_args
        print("WORKER: import succeeded")
    except Exception as e:
        print("WORKER: import failed:", e)
    print("WORKER paddleocr loaded from:", paddleocr.__file__)

if __name__ == "__main__":
    mp.set_start_method("spawn", force=True)
    print("MAIN   paddleocr loaded from:", paddleocr.__file__)
    p = mp.Process(target=worker)
    p.start()
    p.join()

[Bobholamovic]

It appears that the issue is caused by the following two factors:

1. The child process inherits the sys.path from the parent process's Python interpreter, but it does not inherit sys.modules.
2. During the module loading phase, PaddleOCR 2.10 performs a dirty hack by prepending several paths to sys.path.

As a result, when import paddleocr is executed in the worker process, Python loads the module (file) at .../site-packages/paddleocr/paddleocr.py instead of the expected package (directory) .../site-packages/paddleocr, according to Python’s import resolution rules. This discrepancy leads to subsequent import errors.

To verify this behavior, you can slightly modify your demo code as follows:

import sys
print("MAIN" if __name__ == "__main__" else "WORKER", sys.path)

import multiprocessing as mp
import paddleocr

def worker():
    try:
        from paddleocr.tools.infer.utility import init_args
        print("WORKER: import succeeded")
    except Exception as e:
        print("WORKER: import failed:", e)
    print("WORKER paddleocr loaded from:", paddleocr.__file__)

if __name__ == "__main__":
    mp.set_start_method("spawn", force=True)
    print("MAIN   paddleocr loaded from:", paddleocr.__file__)
    p = mp.Process(target=worker)
    p.start()
    p.join()

You’ll get output similar to this:

MAIN ['/workspace/.venv1', '/usr/lib/python39.zip', '/usr/lib/python3.9', '/usr/lib/python3.9/lib-dynload', '/workspace/.venv1/lib/python3.9/site-packages']
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
MAIN   paddleocr loaded from: /workspace/.venv1/lib/python3.9/site-packages/paddleocr/__init__.py
WORKER ['/workspace/.venv1/lib/python3.9/site-packages/paddleocr', '/workspace/.venv1/lib/python3.9/site-packages/paddleocr', ... (many repeated paths) ..., '/workspace/.venv1', '/usr/lib/python39.zip', '/usr/lib/python3.9', '/usr/lib/python3.9/lib-dynload', '/workspace/.venv1/lib/python3.9/site-packages']
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
WORKER: import failed: No module named 'paddleocr.tools'; 'paddleocr' is not a package
WORKER paddleocr loaded from: /workspace/.venv1/lib/python3.9/site-packages/paddleocr/paddleocr.py

As you can see, sys.path in the worker process is already polluted from the start.

It should be noted that, according to the official Python documentation for the multiprocessing module, the spawn start method is described as:

The parent process starts a fresh Python interpreter process. The child process will only inherit those resources necessary to run the process object’s run() method.

However, the documentation does not clearly specify whether sys.path is preserved in the new interpreter (please correct me if I'm wrong). So the above conclusion is based solely on my empirical observations. I'm not sure if the multiprocessing module was designed to be this way.

Ultimately, this issue stems from a design flaw in PaddleOCR 2.10. Fortunately, the problem no longer exists in PaddleOCR 3.0.

If you're sticking with PaddleOCR 2.10 and want a workaround, consider delaying the import paddleocr statement until after the child process has started (i.e., after p.start()). This will help avoid polluting the child’s sys.path.

[DavGrg]

Thank you for your answer.
I haven't tried new PaddleOCRv3, however I did a workaround for PaddleOCRv2.
I made spawned process to see __init__.py file before paddleocr:

import os
import sys


for entry in sys.path:
    pkg_dir = os.path.join(entry, "paddleocr")
    init_py = os.path.join(pkg_dir, "__init__.py")
    if os.path.isdir(pkg_dir) and os.path.isfile(init_py):
        # Force Python to treat that as the package
        import paddleocr as _pkg
        _pkg.__path__ = [pkg_dir]
        return

this must be done before spawning any new process and paddleocr import