_PyBytes_Resize() dereferences NULL before reaching its PyErr_BadInternalCall() error path

[BHUVANSH855]

Bug report

Bug description:

_PyBytes_Resize() appears to handle invalid arguments differently from _PyTuple_Resize() and similar helper APIs.

When *pv == NULL, _PyBytes_Resize() dereferences the pointer through PyBytes_Check(v) before reaching its existing PyErr_BadInternalCall() error path.

Minimal reproducer:

from _testcapi import bytes_resize

bytes_resize(None, 0, False)

This results in a segmentation fault.

With UBSan enabled:

Include/object.h:234:14: runtime error: member access within null pointer of type 'struct PyObject'

GDB backtrace:

#0 _Py_TYPE_impl (ob=0x0) at Include/object.h:234
#1 _PyBytes_Resize() at Objects/bytesobject.c:3349
#2 bytes_resize() at Modules/_testcapi/bytes.c:35

The relevant implementation is:

v = *pv;
if (!PyBytes_Check(v) || newsize < 0) {
    *pv = 0;
    Py_DECREF(v);
    PyErr_BadInternalCall();
    return -1;
}

The crash occurs while evaluating PyBytes_Check(v), so the PyErr_BadInternalCall() path is never reached.

For comparison, _PyTuple_Resize() performs an explicit NULL check before the type check:

v = (PyTupleObject *)*pv;
if (v == NULL || !Py_IS_TYPE(v, &PyTuple_Type) ||
    (Py_SIZE(v) != 0 && !_PyObject_IsUniquelyReferenced(*pv))) {
    *pv = 0;
    Py_XDECREF(v);
    PyErr_BadInternalCall();
    return -1;
}

The corresponding _testcapi wrapper behaves accordingly:

>>> from _testcapi import _tuple_resize
>>> _tuple_resize(None, 0, False)
SystemError: Objects/tupleobject.c:1048: bad argument to internal function

whereas:

>>> from _testcapi import bytes_resize
>>> bytes_resize(None, 0, False)
# segmentation fault

I also noticed that Lib/test/test_capi/test_bytes.py already has the bytes_resize(None, ...) cases commented out as CRASHES.

All production callers I inspected appear to satisfy the documented precondition that *pv points to an existing bytes object, so this does not appear to affect normal CPython execution.

Is the current behavior for _PyBytes_Resize() intentional, or should it reject NULL consistently with _PyTuple_Resize() by reaching PyErr_BadInternalCall() instead of dereferencing the pointer first?

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

[picnixz]

Those are private APIs . You are not meant to use them directly

[BHUVANSH855]

That's correct, and all the production callers I checked do satisfy that precondition.

What made me investigate further was that _PyBytes_Resize() already has a PyErr_BadInternalCall() path, but *pv == NULL never reaches it because PyBytes_Check(v) dereferences the pointer first. _PyTuple_Resize() handles the same case by checking for NULL first and raising PyErr_BadInternalCall().

I wasn't sure if that difference was intentional, so I opened the issue to clarify the expected behavior.

[picnixz]

If pv == NULL happens then v=*pv would crash already. If pv is part of the inputs, just do not call it with NULL.

[BHUVANSH855]

If pv == NULL happens then v=*pv would crash already. If pv is part of the inputs, just do not call it with NULL.

Thanks for the clarification. We are talking about the same case: the reproducer isn't passing pv == NULL; it's passing a valid PyObject ** where *pv == NULL.

For example:

PyObject *obj = NULL;
_PyBytes_Resize(&obj, 0);

[picnixz]

Yeah but I think it is defensive programming here. We could add an assert though.

[BHUVANSH855]

I agree that the precondition is that *pv must point to a valid bytes object.

I mainly wanted to understand whether the current behavior was intentional. An assert() sounds like a reasonable way to make that contract explicit in debug builds.

[serhiy-storchaka]

Not all C API guard against NULL, even if they guard against invalid object type. It is rather exception. Although for _PyBytes_Resize() it would make more sense, since it sets *pv = NULL on error.

Would any code be simpler if add the internal check and remove the outer check? If yes, this is a good argument for such change.