Skip to main content

You ever just add debug logs to the #Python standard library because it doesn’t have logs where they’re needed? :blobfoxgooglycry:

(I’m working with a heavily multi-processed program so pdb won’t do shit.)

#python
in reply to Doughnut Lollipop 【記録係】:blobfoxgooglymlem:

Doughnut Lollipop 【記録係】:blobfoxgooglymlem:
Doughnut Lollipop 【記録係】:blobfoxgooglymlem:
• •

I think I can fix it by using a Unix datagram socket to forward the logs to the logger thread, rather than trying to get multiprocessing.Queue to be inherited by child processes and their child processes. When I got rid of the latter logging approach, it stopped getting stuck. :blobfoxgooglytrash:

All of this because of the #GIL. :blobfoxangrylaugh: #Python

#python #gil
in reply to Doughnut Lollipop 【記録係】:blobfoxgooglymlem:

mr64bit
mr64bit
• •
in essence those shouldn't really be too different. sounds like a problem with forking a process with locked mutexes rather than the GIL
in reply to Doughnut Lollipop 【記録係】:blobfoxgooglymlem:

Doughnut Lollipop 【記録係】:blobfoxgooglymlem:
Doughnut Lollipop 【記録係】:blobfoxgooglymlem:
• •

Never, ever, use the fork context in multiprocessing. :blobfoxsweating: POSIX OSes (aside from macOS) have that as the default. :blobfoxangrylaugh: Always use the spawn context! :blobfoxdead:

Of course, unless you have an extreme edge case. :blobfoxgoogly: (Yes, the names are funny. :blobfoxgooglytrash: )

#Python #multiprocessing


multiprocessing — Process-based parallelism

Source code: Lib/multiprocessing/ Availability: not Emscripten, not WASI. This module does not work or is not available on WebAssembly platforms wasm32-emscripten and wasm32-wasi. See WebAssembly p...
Python documentation
#python #multiprocessing

Neil E. Hodges reshared this.

⇧