I am now more times facing the decision of shipping a new model, that I might not fully understand, but that seems reasonably good on an historical test, or to go back, spend 1-3 days going through the math and intuition to understand it.

I am the bottleneck, but I want it to be so. I think that's the feature of what we do and not the bug. I can't push something that will handle my own capital into prod, that I don't fully understand.
I am not talking here of a BS signal that people test and then call it a system... I am talking about full-fledged models, that seem to capture some behavioral aspect I'd like to capture, but that I don't fully understand the details of the underlying math that is driving risk controls, or even signal generation.
I don't care to be the bottleneck in my portfolio allocations. Over time I will understand the math and build the systems to handle similar exposures of future models. But right now I prioritize making sure I have went through the full scope of risks, and not be surprised by the inevitable unforeseen risk that is just lurking under added complexity.