I had my own set of keys and combinations I wanted to hit while mousing, mostly space and cut/copy/paste. I just made sure I had those on a layer activated by my left thumb on my left hand.

My layout would not work for you since I put navigation on the right hand (close to where it would be normally), but do whatever works for you.

Personally I use navigation keys much more on my split because they're right under my fingers. Having VIM movement keys everywhere is quite nice.