Repository navigation
Undefined Behavior in _PyUnicodeWriter_WriteASCIIString: NULL pointer passed to memcpy when len is 0 #146196
Description
Activity
Could you check what versions affected and add MRE?
This is only in the main branch, and I'll push a PR with the fix myself shortly
Note that this will become valid code, see https://developers.redhat.com/articles/2024/12/11/making-memcpynull-null-0-well-defined#null_pointer_arithmetic. Both gcc and clang claim to conform this accepted proposal.
@ronaldoussoren Am I the only one, C2y?
I looked into that proposal (https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3322.pdf) too, BUT it’s for newer versions of C and will only be available in newer compilers. This problem is already relevant right now.- added a commit that references this issue
on Mar 20, 2026 Could you check what versions affected and add MRE?
@serhiy-storchaka recently added
writer.write_ascii(NULL, 0)test toLib/test/test_capi/test_unicode.py. It should trigger the undefined behavior.Ah yes, I confirm that the newly added test triggers the undefined behavior:
$ ./configure --with-pydebug --with-undefined-behavior-sanitizer $ make $ ./python -m test -v test_capi.test_unicode -m test_ascii (...) test_ascii (test.test_capi.test_unicode.PyUnicodeWriterTest.test_ascii) ... Objects/unicode_writer.c:494:9: runtime error: null pointer passed as argument 2, which is declared to never be null (...)So there is no need to add another test.
The good news is that other
PyUnicodeWriterfunctions don't emit undefined behavior (according to existing tests).The test was backported to 3.14.
Note that we get a crash for non-zero size. The case for zero size was not explicitly supported, the test was only added to prevent future regressions.
We should ether make the behavior well defined by adding the explicit check for 0 size, or comment out the test. We can also explicitly raise SystemError for NULL. Either is fine. But since there is no use case for passing NULL, commenting out the test may be preferable -- it adds zero overhead.
I close the issue, it has been fixed.
We should ether make the behavior well defined by adding the explicit check for 0 size, or comment out the test.
I merged the PR #146201 which fixes the undefined behavior by adding
if (len == 0)test.The test was backported to 3.14.
I also backported the change fixing the undefined behavior to 3.14, so we should be good.
Bug report
Bug description:
The function
_PyUnicodeWriter_WriteASCIIStringinObjects/unicodeobject.c(orObjects/unicode_writer.cin some versions) contains a potential Undefined Behavior. Whenlenis 0 andasciiisNULL, it callsmemcpywith aNULLsource pointer.According to the C standard, passing
NULLtomemcpyis undefined even if the count is zero.Proof of Concept
Clang's UndefinedBehaviorSanitizer (UBSan) reports:
../Objects/unicode_writer.c:494:36: runtime error: null pointer passed as argument 2, which is declared to never be nullThis happens in the following block:
Mitigation
The function should return early if
len == 0. This is a common pattern in CPython to avoid unnecessary work and prevent UB with memory functions.CPython versions tested on:
CPython main branch
Operating systems tested on:
Linux
Linked PRs