`multiprocessing.Queue`: Exceeding a certain amount of bytes in the queue prevents proper exit

[ebonnal]

Bug report

Bug description:

This terminates properly:

from multiprocessing import Queue
Queue().put(b"0" * 65514)
print("end")

while this prints 'end' and is then stuck:

from multiprocessing import Queue
Queue().put(b"0" * 65515)
print("end")

CPython versions tested on:

Latest identified version without the issue: 3.9.6
Oldest identified version with the issue: 3.9.18

Operating systems tested on:

Linux, macOS

[terryjreedy]

What is the most recent version with the issue? 65515 works for me in IDLE on Win10 with 3.12.8 and 3.14.0a1.

[ebonnal]

On Ubuntu (22.04) and Macos (15.1.1) I have not identified a version not having the issue anymore. The latest I have tested on Ubuntu is 3.12.1 and on Macos it's 3.14.0a0 and they both have the issue.

Interesting that you don't have it on Win10! Waiting for more linux/macos users to confirm that I'm not crazy 🙏🏻

[ZeroIntensity]

Confirmed it on Linux, for both 3.12 and main. FWIW, the script does print out end for me, but it hangs upon interpreter finalization, so it doesn't really show up when using IDLE or the REPL (which I suspect is why @terryjreedy couldn't reproduce).

pstack is showing that we're getting stuck waiting on a semaphore somewhere while joining a thread. I'll investigate.

[ebonnal]

Makes sense @ZeroIntensity, for your investigation note that it appeared between 3.9.6 (still ok) and 3.9.18 (has the issue).

[picnixz]

3.9 - 3.11 are security-only branches and this bug wouldn't categorize as a security issue IMO (if you were talking about the labels; for bisecting commits, using the main branch is fine)

[ZeroIntensity]

I do think this is possibly a security issue. It looks like this applies to any iterable greater than 65514, so if user input was passed to Queue.put, this is possibly a DOS. I'm speculating, though. I'll submit a patch once I figure it out and we'll go from there.

[picnixz]

(It could be a DOS but you should consider the possible attack vectors first IMO)