Oh it’s definitely inconvenient! No doubt about that. But convenience comes at a price … that price is being hackable at least to some extent.
I’d love a cellphone model that has embedded in it a full two “personalities” that are separated by a hardware block. It would be great to have a section of my cellphone that can NEVER EVER connect to the Internet on its own, with the capability for me to allow it for very short periods of time (for example to add an account to the authenticator). But that doesn’t exist yet, and even if it does, it can STILL be hacked if someone nefarious takes control of the phone and simulates me allowing access to the blocked off part of it.