This is such a tough situation to be in. ): I think it comes down to what thought gives YOU more peace. FInancial freedom? Or career freedom?

Growing up, my mom always helped us choose life changing decisons by visualizing both outcomes. If staying at your job will only get worse over time, the pay cut may be worth it. You could find flexible side hustles just like you do here on Medium to get extra spending money. But if the thought not much extra money sends you into a panic, maybe try to wait until more of your mortgage is paid off.