Repository navigation
Emscripten test worker sometimes fails witth "BlockingIOError: write could not complete without blocking" #94026
Description
Activity
The
fcntlmodule is currently not available on Emscripten. I hacked the module manually and was able to get flags from the stdout fd. Emscripten does not report that the fd is set to non-blocking mode.>>> import os, fcntl >>> fcntl.fcntl(1, fcntl.F_GETFL) 577 >>> fcntl.fcntl(1, fcntl.F_GETFL) & os.O_NONBLOCK 0I applied a hack on the buildbot worker host. Buildbot now starts NodeJS with
nice -n5. That seems to be good enough to make the tests pass.Why is it set to non-blocking mode? Can it be set to blocking mode? Can it be replaced with an fd set to blocking mode? Can TextIOWrapper and BufferedWriter be made to handle this internally? Can we use an alternate channel for communication between processes?
It is set to non-blocking mode because different rules apply to browser-like environments. It is not possible to change the property. In fact there is no way to even detect that the pipe fd is in non-blocking mode. I can only deduce that it is in non-blocking mode because tests sometimes fail when the system is under high load and the other side cannot read from the pipe fast enough.
Emscripten tries hard to mimic a POSIX-like OS on top of a JavaScript and WebAssembly engine. Ultimately it is bound by restriction of the runtime environment.
To handle the special case in Python's io module we would first have to detect the special case. That is not possible on Emscripten yet. The
fcntlmodule is not available because I disabledioctlas it leads to crashes. Thefcntl(F_GETFD)syscall does not returnO_NONBLOCKandfstatdoes not work on pipes and standard streams.We could only use a temporary file for communication. Socket's don't work as expected. IPC and socketpair are not available.
- added a commit that references this issue
on Jun 29, 2022
Bug report
Sometimes test worker processes on wasm32-emscripten build bot fails with
BlockingIOError: write could not complete without blocking. Example https://buildbot.python.org/all/#/builders/1044/builds/27/steps/10/logs/stdioThe error seems to occur at the end of a test job when the worker process returns its status to the control process. The control process runs CPython natively (
x86_64-pc-linux-gnuin case of the build bot). The worker processes is wasm32-emscripten CPython with NodeJS as runner. I guess that every now and then, the test worker process produces output faster than the control process can consume from the pipe that connects stdout of the worker and read pipe of the control process. The runtime does not like blocking IO and returnsEAGAIN.