I felt this way for a long time about geospatial directions, but when your phone / gps dies and you have no map, it's kind of a terrifying feeling of helplessness
I skirt around this by instead of depending on turn by turn I study the main route and then only go to turn by turn at the endpoint. That way I gain the knowledge of the macro-infrastructure and main roads - I tend to give directions to out of towners even in cities I’ve been in for a week.