You misunderstand the utility of masks, where it is significantly more an infected wearer of the mask who protects others by their wearing. So, being infections before you have symptoms is the problem, which is why people should - as is standard in Japan, for example - wear masks in public whenever they feel unwell or feel a 'cold' coming on.
So in fact whether you wear one or not is far less relevant.