Options: Consistent handling of private-by-docstring `(*_)fields` properties

[richardebeling]

Edit:

As explained in #975 (comment), the properties fields, concrete_fields, local_concrete_fields, many_to_many and related_objects of the Option class are private according to their docstrings, even through they don't start with a leading underscore. This resolved my original issue. As a user, we should just use get_fields() and then filter the results as we wish.

However, @sterliakov noticed an inconsistency:

fields property is present in stubs, but marked as private in Django source too. I think it's worth discussing, and after final decision we should either drop fields or add other four properties too. Before my changes only fields and many_to_many were present.

So I'm leaving this issue open for now, although the original issue text below is outdated / has been resolved.

---

Problem description

In this commit of #909, the many_to_many property of the Option class was removed. In django, this is still available on current master.

After the removal, this code:

from django.db.models import Model

def test_mypy(model_instance: Model):
    return model_instance._meta.many_to_many

fails type checking with this error:

test.py:4: error: "Options[Any]" has no attribute "many_to_many"; maybe "local_many_to_many"?

Expected result

The code should pass, or is there any reason that many_to_many was removed from the stubs? I couldn't find any changelogs, and the message of the commit introducing the change doesn't give much insight either:

Define properties where possible, remove 'None' argument annotations, remove inadequate type in make_immutable_fields_list.

@sterliakov was there a reason to remove this?

Versions

Tested with django-stubs 1.11.0. The many_to_many property definition is missing on master if I see it correctly.

[sterliakov]

I'm sorry for a late response, have been a bit overloaded.

I don't insist on this change which was introduced only for consistency. AFAIC, it was initially decided not to document private methods in stubs. It applies to _underscored methods, of course, but the story here is similar.

There are five similar properties: fields, concrete_fields, local_concrete_fields, many_to_many and related_objects. They all have explicit note in docstring:

Private API intended only to be used by Django itself; get_fields() combined with filtering of field properties is the public API for obtaining this field list.

So they must be considered private and are not intended for external use. Thus my decision was to drop this property from stub.

I haven't noticed this before, but I have introduced another inconsistency: fields property is present in stubs, but marked as private in Django source too. I think it's worth discussing, and after final decision we should either drop fields or add other four properties too. Before my changes only fields and many_to_many were present.

[richardebeling]

Ah, alright, thanks. I didn't properly read the docstring and just assumed the property was intended to be public because it didn't start with a leading underscore.

In this case, I think we'll change our code to just use get_fields().

Changing the issue to be about consistent handling of these private-by-docstring properties, as you noted.

[GabDug]

Hey!

Stumbled upon this issue while working on stubtest errors for #1691.

Do we have an idea on which path to take?

Either way, I'll add a reference to this issue in the allowlist.

My 2 cents: as it's not prefixed by underscores, users will stumble on it and it might as well be typed. As a nice bonus, it will resolve a type: ignore from django mypy extension.