This is a little like a system which says "we will use all free space behind your filesystems linear maximal extent as swap" and then discovering it gobbled all your freespace and DF reports you have no space. As long as the space comes back when you need it, you don't have to care. But, time machine, and swap both have a habit of grabbing stuff at high privilege you can't always directly impact without using abstruse methods. So, in practice while you can say "it doesn't matter" it can matter, depending: it can cause dismay, and it can interfere with your otherwise normal use of the machine.
"free" space being contextualised isn't entirely nice. better to be clear how its consumed, than say its free.
- free space is being used as swap until needed, then transparently released to be used
- "free" uniformly means being used this way, unlike "not free" which means it will not be released transparently when space is needed, so "free" is not being contextualized
- since the space is free but reported as "not free" by df, df has a bug
As someone who often plays near this limit, I can tell you that
> then transparently released to be used
is doing most of the heavy lifting here, because in practice it often is NOT transparently released.
To give a concrete example, imagine you have 500GB free, which is enough for you to unpack and analyze a single simulation output at a time. You unpack the first one, do your analysis, and then rm that folder and its archive to make space to pull down the next batch of results.
However, you then will find that for some reason, despite Finder saying that it has plenty of space, that trying to unpack another 500GB result will fail with an "out of space" error, unless you wait several hours/days for unnamed "background processes" to garbage collect that drive space. What that looks like in practice is that Finder will start to report superlinear space usage (e.g. for every 10GB you unpack, >>10GB of usage will be reported) until the operation fails.
Thus, I find that df actually is much better about predicting the actual expected behavior of the system than Finder itself. If there were a way to manually trigger an immediate cleanup via the command line, that would also be fine. As it is, I just work off of external drives w/o APFS, which avoids the issue entirely.
df just uses C library calls which use system calls.
If you're going to implement something like this then ffs do it right. I don't know the "easy" way to fix this, but in any case if something is supposed to be transparently free and available then standard tools using standard calls should see it as free and available, with specialized user space tools that can look behind the screens when needed.
For clarification, by "df has a bug" I meant that it doesn't produce the result it should given the situation, but not necessarily that the problem is located in df's own source code, vs. the kernel, other process, library etc.
> As long as the space comes back when you need it, you don't have to care.
That's a common assumption but it's also a lie: because you do have to care about wearing your media, reducing its lifespan. Abusing storage with heavy doses of r/w activity (which is what swapping does) is not good. Yes, the issue is not as bad in modern "disks" as in rust-spinners, and can often look like a matter of efficiency (because of how memory is physically "flipped", on modern drives it's more efficient to write a ton of data rather than a few bytes), but in reality you are wearing your disk more than you would if macOS kept its grubby hands to itself.
Of course this is not a downside for the people who will sell you replacement disks, or rather replacement machines (since drives are now soldered), who coincidentally happen to be the people who develop the disk-wearing OS.
> "free" space being contextualised isn't entirely nice. better to be clear how its consumed, than say its free.
I would guess that for more than 99% of users just saying "free" about swap/temp is the best option and it would just be confusing to say less disk is free or show multiple values. And if you really want to know you can just select "Get info" on the hard drive and see how much purgeable space you have.
Having really good background processes that makes sure the space is free when you need it and easy to understand error messages when it can't purge is of course great for everyone.
People on forums like this seem to consistently forget that the average computer user doesn’t even know the difference between storage and ram, much less understand concepts like swap. And why would they need to? It would cause way more headaches to be “honest” here than tell the convenient “lie.”
isn't this by design coz apple's business practice includes storage upsell?
if you are an avg mac consumer running out of space constantly and hitting that issue- you ask support/genius and they tell you to buy a bigger capacity more expensive mac or icloud storage as one of their "solutions".
Obviously no, because that's a very dubious way of getting sales.
(a) An issue that the vast majority of the users wont ever see.
(b) Those that would see it, would only see it after they already bought a Mac (with no storage upgrade option available to them to buy from Apple for their current Mac).
(c) So Apple's is hoping that a handful of those would do what? Buy a new Mac immediately? Totally unlikely. Get their next Macs 2-5 years down the line with more disk space?
(d) And all that combined with Apple already doing several things for the macOS to need less disk space, like the "sent online, storage restored, downloadable on demand" files feature introduced in the last OS, or a couple of rounds of prunning of the OS install size.
In general this is the kind of conspiracy theory that totally misunderstands business tactics, their feasibility, and their margin for profit.
any advice for buying a mac includes the tagline "get the max storage device that you can afford", i think mac's engineering is highly responsible for that mindset.
people like you can claim it's not profitable but it does lead to situations where ppl buy storage sizes they don't need for "peace of mind".
it's not as innocent as you imply it is.
edit: don't forget apple solders their storage in nowadays, so buying anxiety multiplies as you can't just fix it later.
APFS has a nice feature in that it can share free space between different volumes on the same physical device. So, for example, I can have a Time Machine backup and a general-purpose volume on my external SSD, and free space is automatically allocated to whichever volume needs it.
The downside is that the output of "df" can be confusing at first if you're not used to it!
(Maybe this feature is common on all OS's/filesystems now days, but I certainly could have used it back in my Linux-admin days. It was quite a revelation when I discovered it!)
Good question. I've only had it set up for a few months and so far Time Machine is only using 660 GB of my 2 TB SSD, backing up my MacBook's 512GB internal drive.
So Time Machine is pretty space-efficient, but it will keep growing over time until it fills the entire backup drive since there is no limit on how old the backups it keeps can be[1]. I guess when it starts filling up I will just manually delete some old backups to free up space if I need it. Apparently, another alternative is to limit the size (set quota) of Time Machine's backup volume in Disk Utility.
[1] Personally I wish this could be configured. I don't really care much about keeping old files, I just want to make sure I have a recent backup just in case my MacBook gets lost or stolen or it's SSD fails...
The way I set mine up is the Time Machine section of the drive has a quota capped at 1.5x the size of my internal drive, unfortunately you can’t edit quotas without formatting the drive. (Is there any functional difference between a quota and a partition then…? Weird…)
Yeah, that's annoying that you can't edit quotas. But they are functionally different to a partition because the quota just limits the maximum size of the volume. If it's not filled to its quota, any unused space is still available for other volumes to use.
Link from the same publication on this exact topic.
https://eclecticlight.co/2020/04/09/where-did-all-that-free-...