Do you use a good observability framework so you see where the slowdown is? I always built my own but there must be good ones out there. There used to be a good python profiler too.

I'm pretty sure I know where it is, when the queuing server gets a req from the firehose consumer it creates a worker which instantiates Pytorch and loads the ML model, and the way it spawns means that it loads model from disk every time. I have no idea how to fix that, I am not good at Python lol.