Convert TiB to MiB

A tebibyte is 1,024 gibibytes. The gap against the terabyte has grown with each step: 2.4% at kilo, 4.9% at mega, 7.4% at giga, 10% at tera.

A mebibyte is 1,024 kibibytes, or 1,048,576 bytes. Linux tools and memory sizes generally mean this even when they print "MB".

Going this way is a multiplication, and by a whole number: one tebibyte is 1,048,576 mebibytes, exactly, and 1,048,576 is the definition rather than a measurement that came close.

That makes it one of the few conversions worth doing in your head, and it makes the answer checkable: divide back and you must land on the number you started with, exactly, with no remainder to explain away.

One TiB is 1,024 of the unit below it; one TB is 1,000. On this page that is the difference between 1048576 MiB and 953674.3164 MiB — 10 % — and the gap grows at every step up the scale, which is why it is a rounding error on a photograph and a visible chunk of a hard disk.

This is the whole of the missing-storage mystery, and on this page it is worth 10 %. A drive sold in TB holds exactly what the label says; Windows divides by 1,024 instead of 1,000, keeps the decimal name, and reports 953674.3164 MiB where the box said 1048576. macOS has counted these in the decimal units since 10.6, which is why the same drive can look two sizes on two machines — nothing is missing and nobody is rounding, the same bytes have two names.

Dividing a tebibyte by a mebibyte gives 1,048,576, and in almost every context where somebody performs this conversion that quotient is the point. How many parts will the upload have. How many chunks will the backup index hold. How many files will the split produce. The byte arithmetic is exact and uninteresting; the piece count runs into a limit, and the limit is usually on the count.

That reframes what a chunk size is. It is not a unit and not a performance knob in isolation — it is the divisor that decides whether the operation is possible at all. Choosing it by copying a default and only discovering the ceiling at the far end of a multi-hour transfer is the commonest way this goes wrong, and the arithmetic that would have prevented it takes a few seconds.

The S3 multipart protocol and the many implementations that copy it allow a maximum of 10,000 parts per upload, a minimum part size of 5 MiB for every part except the last, a maximum part size of 5 GiB, and a maximum object of 5 TiB. Three of those four are size limits and the one that actually bites is the count, because a client picks a chunk size once and then keeps producing parts until it runs out.

The consequence is a ceiling that is a property of the configuration rather than of the service: chunk size multiplied by 10,000. At 5 MiB that is 48.83 GiB. At 8 MiB it is 78.125 GiB. At 64 MiB it is 625 GiB, at 128 MiB it is 1.22 TiB and at 512 MiB it is 4.88 TiB, which is where the object ceiling itself starts to be the binding constraint. Every one of those figures is the chunk size in mebibytes times ten thousand, divided by 1,024 for a gibibyte answer and by 1,048,576 for a tebibyte one.

The AWS CLI defaults to an 8 MiB chunk with an 8 MiB threshold, and most SDKs pick something similar. That is a sensible default for the files people usually upload and it fails at 78.125 GiB, which is far enough above ordinary use that a team can run for years before meeting it. When they do, the failure lands after seventy-eight gibibytes have already been transferred, which is the worst possible place for it.

Raising it is a single configuration line — aws configure set default.s3.multipart_chunksize 128MB — and the value should be chosen against the largest object the system will ever produce, not the largest it produces today. A 128 MiB chunk covers 1.22 TiB and a 512 MiB chunk covers essentially the whole permitted range, at the cost of a larger retry unit and more memory held per concurrent part.

The floor is the total divided by 10,000. A tebibyte needs at least 104.86 MiB parts; a 5 TiB object needs at least 524.29 MiB. Rounding up to the next power of two gives 128 MiB and 1,024 MiB respectively, and the round figure is worth taking because it keeps the part count a clean 8,192 or 5,120 rather than something that has to be recomputed every time the object grows.

The ceiling is set by memory and by retry cost rather than by the protocol. A client holding eight concurrent 128 MiB parts has a gibibyte in flight, which is fine on a server and not fine in a container with a 512Mi limit. Where the uploader is itself running under a memory limit, the part size times the concurrency has to fit inside it with room for everything else, and that product is the number to check rather than either factor alone.

A failed part is retransmitted whole. On a connection that drops occasionally, small parts mean a failure costs little and large parts mean it costs a lot — a break at 90 per cent of a 1 GiB part discards 900 MiB of work. On a very reliable link the calculation reverses, because each part carries a request, a signature and a round trip, and 131,072 of them is a great deal of protocol overhead for one tebibyte.

The practical middle is to size parts so that one takes somewhere between one and five seconds on the link you actually have. At 100 Mbit/s that is roughly 16 to 64 MiB; on a gigabit link it is 128 to 512 MiB. Working it from the link speed rather than from a table means the chunk size adapts when the transfer moves from an office connection to a data centre, which is usually where the assumptions break.

Throughput on object storage comes from parts in flight, not from part size. A single stream is limited by round-trip time and the receiving endpoint's per-connection behaviour, and it will not saturate a fast link no matter how large the pieces are. Eight or sixteen concurrent parts typically will, and the improvement between one stream and eight is far larger than anything achievable by changing the chunk size.

That means the two settings should be chosen in the right order: pick the part size from the total and the part limit, then pick the concurrency from the link and the memory budget. Doing it the other way — raising the chunk size hoping for speed — produces a transfer that is no faster, uses more memory, and loses more work on every retry.

Deduplicating backup tools chunk with the opposite goal. Rather than a fixed mebibyte size they use content-defined boundaries with a target average — often around a mebibyte — so that inserting bytes at the start of a file shifts only the boundaries near the change rather than every boundary after it. Fixed-size chunking would make a one-byte insertion rewrite the entire chunk stream, which is why it is not used where deduplication matters.

The piece count still governs the cost. A tebibyte at a one-mebibyte average is on the order of a million chunks, each needing an index entry with a hash, so the repository index runs to tens or hundreds of mebibytes and has to be held in memory during a backup. Doubling the average chunk size halves the index and reduces the deduplication ratio, and where a repository has become slow rather than full, that trade is usually the one worth revisiting.