Yes, but that's why ~~good~~ fun enemy AI for games is both an art and a science.
My approach is that the path from the pathfinding algorithm is just a known good solution. The path the agent gets back from the algorithm doesn't have to be just a list of Vector3's for the window centers. It could get back a list of each window quad instead, and then aim for a random point in the next quad shifted by the distance between the two quads. You can also do some raycasts from each agent occasionally to do local-avoidance-type behavior (which you'll likely have to do anyway so that enemies spread out a bit and avoid non-static geometry).
I'd also ask if this isn't somewhat accurate for how people would do things? We have specific rules that we have internalized on what side of a road to drive on. But, absent those rules, it isn't uncommon to just drive down the center of the road. Largely expected on many surface streets that don't have brightly painted lanes and large shoulders.
To that end, you could always encode the rule of what side of a window to aim for? Something you probably want to do if you are going to be aiming for bidirectional agents going through them?
