@thaJeztah @jfrazelle having just worked around this issue I think docker can (and should) workaround this problem.
The gist of what happens is:
- You have a container with N processes
docker stop is issued
- pid 1 gets a TERM
- pid 1 exists but other pids are still running some may have got TERMs
- In some weird kernel linux container condition the orphan processes are not torn down and trigger a failure mode in the kernel.
Instead I thing the "semantics" of
docker stop should be changed to:
- Send TERM to pid 1, wait up to 10 seconds for pid 1 to finish. (if not finished send a QUIT)
- Check if any other pids are still running in the container, if there are
a. In parallel send TERM + wait for up to 5 seconds for each of the remaining pids. If it is still running send a QUIT and wait.
- Stop container
This will change the semantics a bit on docker stop, but, on the other hand, introduce some sort of documented behavior for orphans and make
stop much more reliable even if kernel or aufs has bugs.