Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upKill processes in container before copying output files #433
Comments
|
Another solution would be to open a file descriptor for accessing the directory in the container before starting the tool. As long as we keep this file descriptor open, we should be able to access the container's directories, so we can kill the container (and all subprocesses within it) immediately. This would probably be the cleanest solution, but requires rewriting |
This is at least an improvement for #433, although a proper solution should be implemented after support for Python 2 was dropped.
|
This is a lot of implementation effort unless we can make use of |
Currently, the following happens at the end of a run in container mode:
This means that if subprocesses change output files after the main process has terminated, the output files can be in an inconsistent state. Furthermore, subprocesses can continue accumulating CPU time while output files are copied.
We should probably kill all subprocesses and potentially do even more cleanup before copying output files. The problem is that the best way to kill the subprocesses is to kill the whole container, but we still need the container for copying output files (cf. 4bba456 and 78a801a). Using cgroups to kill the subprocesses is more inefficient and can be really problematic if the freezer cgroup is not available and a fork bomb runs in the container.