Incorrect evaluation of `|` between classes and non-classes

### Summary

The special cases added in https://github.com/astral-sh/ruff/pull/21195 are a little too broad. Here's our behaviour on `main`:

```py
class Foo:
    def __ror__(self, other) -> str:
        return "foo"

reveal_type(int | Foo())  # `UnionType`, but should be `str`
reveal_type(int | 42)  # `UnionType` (and no diagnostic), but this fails at runtime
```

### Version

_No response_