Convert kB to TiB

A kilobyte is 1,000 bytes when a standards body defines it and 1,024 when an operating system does. This page uses 1,000; the kibibyte is the unit that means 1,024.

A tebibyte is 1,024 gibibytes. The gap against the terabyte has grown with each step: 2.4% at kilo, 4.9% at mega, 7.4% at giga, 10% at tera.

Going this way is a division, and by a whole number: 1,099,511,628 of these fit into one tebibyte with nothing left over. The awkwardness is only that the answers come out as fractions — a third of a tebibyte, a twelfth of one — rather than as the round figures the other direction gives you.

Nothing is lost to rounding even so. The division is exact, and if your answer will not sit still as a decimal — 0.0833… and friends — that is the fraction showing through, not an error creeping in.

One TiB is 1,024 of the unit below it; one TB is 1,000. On this page that is the difference between 1099511627.776 kB and 1000000000 kB — 10 % — and the gap grows at every step up the scale, which is why it is a rounding error on a photograph and a visible chunk of a hard disk.

This is the whole of the missing-storage mystery, and on this page it is worth 10 %. A drive sold in TB holds exactly what the label says; Windows divides by 1,024 instead of 1,000, keeps the decimal name, and reports 1000000000 kB where the box said 1099511627.776. macOS has counted these in the decimal units since 10.6, which is why the same drive can look two sizes on two machines — nothing is missing and nobody is rounding, the same bytes have two names.

The forecast has four inputs and they are always the same: the size of one record, how many arrive per unit of time, how long they are kept, and what the storage layer multiplies that by. The first is in kilobytes because that is the scale of a log line or a metric sample. The last is in tebibytes because that is what a pool reports. Everything between them is multiplication, and the only hard part is not changing unit systems halfway.

A worked case: a billion records a day at 1 kB each is 1,000,000,000,000 bytes a day, which is a terabyte, which the pool will report as 0.909 TiB. Ninety days of that is 90 TB, which the pool will report as 81.85 TiB. Those two figures describe identical data, so a plan that computes 90 in one system and provisions 90 in the other is out by 8.15 units of storage — which is the whole of the trouble this pair causes.

Before worrying about 9 percent, get the factors that are worth several hundred percent. Compression is the first: structured logs and metrics commonly reduce to a fifth or a tenth of their raw size, so a forecast that ignores it overstates by an order of magnitude. Replication is the second and runs the other way: three copies is three times the storage, and erasure coding is typically 1.2 to 1.5 times.

Then there are the indexes, which are easy to forget because they are not the data. A log store that indexes fields adds a structure that can be a substantial fraction of the payload, and a time-series database keeps its own. The honest way to establish all of these is to run a representative sample through the actual system for a day and measure what it occupied — one measurement beats any amount of estimation.

Retention is usually set by habit or by the first number somebody suggested, and it is the input with the largest lever on cost. Halving the window halves the steady-state size exactly; the amount of value lost is rarely proportional, since the overwhelming majority of queries against operational data reach back hours or days rather than months. Seeing the storage figure for each candidate window next to each other changes the conversation.

Where a longer window is genuinely required — an audit obligation, a quarterly comparison — it is usually required for a subset rather than for everything. Splitting the stream so that the small, valuable part is kept for a year and the bulk for a fortnight gives most of the benefit at a fraction of the size, and it is a decision that has to be made before the data is written rather than after.

A store with a fixed retention window does not grow forever: it grows until the oldest data begins to expire, and then it holds roughly constant at the daily rate times the window. That plateau is the number to size for, and it arrives one retention period after the system starts. Sizing for the growth curve rather than the plateau buys capacity that will never be used; sizing below the plateau means the expiry policy starts doing your capacity management for you.

The plateau moves whenever the daily rate does, which is the part worth monitoring. New instrumentation, a traffic increase or a debug level left on in production all raise the rate, and the storage effect appears gradually over a full retention window rather than immediately. A dashboard of bytes ingested per day predicts the problem; one of bytes stored reports it a fortnight late.

It hides at the boundary between the calculation and the purchase. The forecast is naturally done in decimal units, because rates and record sizes are decimal and the arithmetic is easy. The pool is reported in binary units, because that is what filesystems do. If the plan says 90 TB and the pool says 82 TiB, those are the same amount and it looks like a shortfall; if the plan says 90 TiB and 90 TB is provisioned, it is a real shortfall of 9 percent and looks like a match.

The second case is the dangerous one because nothing signals it. Storage is bought in decimal terabytes, and a request for "90 terabytes" filled by a vendor selling decimal capacity against a requirement computed in tebibytes arrives nearly 9 TB short. Stating the requirement in bytes on the purchase order removes the ambiguity for the cost of one extra number.

A forecast is a chain of multiplications, and rounding at each step compounds in an unpredictable direction. Rate times size times window times replication times index overhead, each rounded to two figures, can land ten percent from the exact answer with every individual step looking reasonable. Multiply the exact numbers and round once, at the end, where the rounding is visible and deliberate.

That is also the point to decide how much precision the answer deserves. A storage forecast is worth two significant figures at best, because the inputs are estimates and the compression ratio alone carries more uncertainty than everything else combined. What it is not worth is false precision generated by a spreadsheet, since a figure quoted to five digits invites a reader to treat it as measured rather than projected.

It takes about 1.1 billion kilobytes to make a tebibyte, so a useful landmark is that one kilobyte per second, sustained, is about 31.5 GB a year — roughly 29.4 GiB — and a thousand kilobytes per second is about 31.5 TB a year, or 28.7 TiB. Most forecasts at this scale can be checked against those two numbers in a few seconds, and an answer that is not close to a multiple of them has an error somewhere in the chain. The same landmark works downwards: a hundred bytes a second is about 3.15 GB a year, which is where per-device telemetry usually sits before it is multiplied by the fleet.

The other check is the direction of the correction. Converting from decimal to binary always makes the number smaller, at this span by 9 percent, so a TiB figure that came out larger than the TB one has been multiplied where it should have been divided. That is the most frequent arithmetic mistake in this category and the easiest to catch, because it does not need the exact answer to spot.