Please use defer when there is only one code path as well unless it is a hot function and you have measured a noticeable performance decrease.

Code will get changed over time and one code path often becomes two or more. Chances are, the next developer who adds code will not realize that the resource needs to be freed, or closed and simply forgets to add the defer. The only thing where defer really gets into your way is for error handling (because it can get verbose if you handle errors in defer).

In the grand scheme of things, defer is not noticeablily slower.