Convert GiB to TiB

A gibibyte is 1,073,741,824 bytes — about 7% more than a gigabyte. Windows measures in gibibytes and labels them GB, which is the whole of the missing-space mystery.

A tebibyte is 1,024 gibibytes. The gap against the terabyte has grown with each step: 2.4% at kilo, 4.9% at mega, 7.4% at giga, 10% at tera.

Going this way is a division, and by a whole number: 1,024 of these fit into one tebibyte with nothing left over. The awkwardness is only that the answers come out as fractions — a third of a tebibyte, a twelfth of one — rather than as the round figures the other direction gives you.

Nothing is lost to rounding even so. The division is exact, and if your answer will not sit still as a decimal — 0.0833… and friends — that is the fraction showing through, not an error creeping in.

One GiB is 1,024 of the unit below it; one GB is 1,000. On this page that is the difference between 0.001 TiB and 0.0009 TiB — 7.4 % — and the gap grows at every step up the scale, which is why it is a rounding error on a photograph and a visible chunk of a hard disk.

This is the whole of the missing-storage mystery, and on this page it is worth 7.4 %. A drive sold in GB holds exactly what the label says; Windows divides by 1,024 instead of 1,000, keeps the decimal name, and reports 0.0009 TiB where the box said 0.001. macOS has counted these in the decimal units since 10.6, which is why the same drive can look two sizes on two machines — nothing is missing and nobody is rounding, the same bytes have two names.

Drives are sold in decimal terabytes and pools are counted in binary units, so the first thing that happens to a purchase is a ten per cent haircut that is not a haircut at all. A 4 TB drive holds 4,000,000,000,000 bytes exactly as advertised, which is 3,725.29 GiB or 3.638 TiB. Eight of them are 29,802 GiB, or 29.10 TiB, and the invoice says 32 TB. Nothing has been lost between those two figures; they are the same bytes counted in different bases.

The second conversion is the one that removes real capacity, and it comes after: parity or mirroring, the reserve the filesystem keeps, and the fill ceiling anybody sensible plans to. Keeping the two separate is what makes a capacity plan checkable. State the raw figure in tebibytes first, then each deduction with a reason, rather than presenting one number that is 40 per cent below the invoice with no working shown.

This is the most common source of a gibibyte figure that will not reconcile. zpool list reports the raw size of the vdevs, parity included, because that is what the pool layer manages. zfs list reports what the dataset layer can store, parity excluded. On an eight-disk raidz2 pool the first will show roughly a third more than the second, and both are correct.

The rule that follows is to take capacity figures from the layer that matches the question. Planning how much data will fit is a zfs list question. Working out whether a replacement drive is large enough, or how much a resilver has to read, is a zpool list question. Converting the wrong one to tebibytes and putting it in a plan produces an array that runs out a third earlier than the spreadsheet said it would.

The headline arithmetic is simple. A mirror gives half. RAID 5 and raidz1 give n − 1 disks of data. RAID 6 and raidz2 give n − 2. A striped mirror gives half regardless of width. Eight 3,725.29 GiB drives in raidz2 give six drives of data, 22,351 GiB, or 21.83 TiB, and the same eight as four mirrored pairs give 14,901 GiB, or 14.55 TiB, with far better rebuild behaviour.

Padding takes a further slice that the headline arithmetic misses. ZFS allocates in units derived from the sector size, so records that do not divide evenly across the data disks leave gaps, and the effect is worst with small record sizes on wide vdevs. For a general file server with the default 128 KiB record size the loss is small; for a pool holding a database with an 8 KiB record size on ashift=12 disks it can run to a fifth of the nominal capacity, which is worth modelling before the pool is built rather than discovering afterwards.

A copy-on-write filesystem cannot free a file by overwriting a bit in place; it has to write new metadata, which requires free space. A pool that reached genuinely zero free space would therefore be unable to delete anything, which is a state with no way out. ZFS avoids it by holding back a share — a thirty-second of the pool by default — that ordinary writes cannot touch.

On a 32 TiB pool that is around 1 TiB set aside, and it is not visible as free space in the figure users see. It is a fixed proportion rather than a fixed amount, so it grows with the pool, and it is one of the deductions that should appear explicitly in a capacity plan. Btrfs has an analogous problem and handles it differently, through global reserve and the balance operation, but the underlying constraint is the same for any filesystem that never overwrites in place.

Allocation performance on a copy-on-write pool degrades as free space fragments, and it degrades sharply rather than gradually. The allocator has to search harder for contiguous space, writes become scattered, and read performance follows because the data landed scattered. The usual guidance is to plan on eighty per cent for a general-purpose pool and lower for one with a heavy random write load.

That turns 32 TiB of usable capacity into roughly 25.6 TiB of planning capacity, and treating the difference as a margin to be eaten into during a busy quarter is how a pool ends up slow and hard to recover. Converting the gibibyte total to tebibytes and applying the ceiling in the same step gives one number to plan against; keeping the raw figure in the plan invites somebody to use it.

A pool does not grow by an arbitrary amount. Adding capacity means adding another vdev of the same shape, or replacing every drive in an existing vdev with a larger one and letting it expand once the last is done. Recent ZFS releases can add a single disk to a raidz vdev, but the pre-existing data keeps its old parity ratio until it is rewritten, so the usable figure moves less than the arithmetic suggests.

The consequence for planning is that the growth increment is large and known in advance. An eight-drive raidz2 vdev of 4 TB disks adds 21.83 TiB at a time, so a plan that will exceed capacity in eighteen months has a purchasing decision with a discrete size rather than a rate. Working the increment out in tebibytes early makes it obvious whether the next step is one vdev or two, which is usually the question that matters.

A snapshot costs nothing when taken and grows as the live data diverges from it. A dataset holding 4 TiB with a month of daily snapshots and a heavy rewrite pattern can consume several times that, and none of the growth is visible in the size of the current data. zfs list -o space breaks it out into used, usedbysnapshots, usedbydataset and usedbyrefreservation, and that breakdown is where an unexplained shortfall usually turns out to live.

The planning consequence is that the usable tebibyte figure has to be reduced by an allowance for snapshot retention, and the allowance depends on the rewrite rate rather than on the size of the data. A media archive that is written once needs almost nothing; a virtual machine store where every guest rewrites its filesystem daily needs a large multiple. Where the rate is unknown, taking a fortnight of measurements is far cheaper than the alternative of finding out when the pool fills.