sys.setrecursionlimit no longer affecting pickle.dump

[Jiehong]

Bug report

Bug description:

This is a follow up of #112282 (updating doc).
It's also a different case of #128544 (import chains impacted).

In particular, on top of the doc no longer being correct for sys.setrecursionlimit, the regression needs to be fixed.

Note: this works in python 3.11 or below.

import pickle
import sys

nested_object = current = {}
for i in range(10000):
    current['nested'] = {}
    current = current['nested']

sys.setrecursionlimit(3750000)
x = pickle.dumps(nested_object)

Result of the run:

▶▶ python3.11 test-issue.py
# runs ok

▶▶ python3.12 test-issue.py
Traceback (most recent call last):
  File "~/test-issue.py", line 10, in <module>
    x = pickle.dumps(nested_object)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
RecursionError: maximum recursion depth exceeded while pickling an object

Impact: all programs trying to pickle data that is too nested cannot be ported to python 3.12 or 3.13 so far

CPython versions tested on:

3.12, 3.13

Operating systems tested on:

Linux, macOS, Windows

[markshannon]

With this program:

import pickle
import sys

nested_object = current = {}
for i in range(100000):
    next = {}
    current['nested'] = next
    current = next

for r in range(5000, 1000000, 5000):
    try:
        print("Trying with recursion limit", r)
        sys.setrecursionlimit(r)
        x = pickle.dumps(nested_object)
    except RecursionError:
        print("Recursion error")

With Python 3.11, I get:

Trying with recursion limit 5000
Recursion error
...
Trying with recursion limit 115000
Recursion error
Trying with recursion limit 120000
Segmentation fault (core dumped)

With 3.12, I get:

Trying with recursion limit 5000
Recursion error
...
Trying with recursion limit 995000
Recursion error

It is impossible to pickle an arbitrary long chain of dicts on any version of Python, there simply isn't enough C stack space.
Python 3.11 crashes. Python 3.12 does not. This doesn't seem like a regression.

[Jiehong]

Python 3.11 crashes. Python 3.12 does not. This doesn't seem like a regression.

For your particular program, indeed. I'm not saying pickle dumping is perfect for all structures. And for the program I provided, the result is opposite of yours, and this is a regression.

We've got a program in production that does pickle.dump, and without changing the program, it works fine from python 3.8 to python 3.11, yet it does not with python 3.12 and later with this exact message.

This, by definition, is a regression.

[picnixz]

I'm categorizing this one as a pickle bug for now, although I'm not sure if it's directly related to pickle or if it's related to something else (maybe the recursion limit is reached before due to some other changes in the interpreter).

cc @serhiy-storchaka

[serhiy-storchaka]

It is impossible to pickle an arbitrary long chain of dicts on any version of Python

But it was possible to pickle a chain of length 50000, and now it is not possible to pickle even a chain of length 5000. This is a regression.

It is interesting that a chain of length 4990 can be pickled with arbitrary small recursion limit, but pickling a chain of length 5000 fails for any recursion limit. This is suspicious.

I do not think that this issue is a pickle-specific.

[sergey-miryanov]

I will have a go at it if no one else is currently working on it.

[sergey-miryanov]

Py_SetRecursionLimit does not change PyThreadState.c_recursion_remaining, but pickle uses _Py_EnterRecursiveCall (-> _Py_MakeRecCheck and _Py_CheckRecursiveCall) that uses c_recursion_remaining.

After some searching, I believe this is related to gh-112215. @markshannon suggested using actual stack depth instead of count in 3.13, but I haven't seen any changes.

For now, the initial value of c_recursion_remaining is different for different platforms. This is why some run into this limit and some do not.