Storage space consumption is always a major bone of contention in all data centers. It seems 100 TB of new storage can fill up in a blink of an eye and then you have to buy some more. But what to do? Lets get below the covers to see what is happening.
When data is written to a volume (I am tempted to say disk, but since most disks are really virtual volumes, that may not actually write to a spinning disk, I will stick with volume) it is written by a file system (or disk space manager of some kind like ASM), to logical block addresses or LBAs (that are 512 bytes in size). Space in a volume is addressed in LBAs starting at zero and going to the highest address that the disk size allows (so clearly a 5 TB volume has way more LBAs than a 5 GB volume).
From the host servers perspective, if a volume claims it has 5 TB of space available, then the server believes it has the right to write 5 TB. It is quite common for storage controllers to allow storage administrators to over-allocate space. Meaning that for a fixed quantity of physical capacity (say 75 TB) you could allocate 150 TB of volumes. This is over allocation and is only made possible by thin provisioning sometimes combined with other space-saving methods (like compression and deduplication). Normally over-allocation occurs by creating over-allocated storage pools.
An over-allocated storage pool means the administrator can create virtual volumes whose total volume size (when summed together), exceeds the available storage capacity of that pool. In other words we can advertise more space than we actually have. This means the volumes in the pool had better be space efficient in nature.
Now genuine space efficient volume design should follow five principles:
- When data gets written to the volume, allocate as little space from the pool as possible to hold that data. In other words if I write 100KB to a volume, don’t allocate 100GB from the pool to that volume to hold that data.
- When zeros get written to the volume, allocate no space from the pool and preferably release the space occupied by those LBAs back to the pool. In other words, if I write 1 MB of zeros, don’t allocate 1 MB of pool space to hold those zeros. In fact, have a look at the LBAs I am writing to and if they include address ranges already allocated to the volume from the pool, see if we could de-allocate them from the volume and return that space to the pool.
- When allocated space is no longer needed, offer some way to release that space back to the pool (sounds like # 2 but is actually different). In other words, if I delete a 1 GB file, then that’s really 1 GB of volume space I don’t need anymore. The file system knows this, but does the underlying disk controller?
- If space is running short in the pool, give me plenty of warning so I can do something about it before everything goes wrong.
- If data is now being written in a thin fashion, then it is likely the data is not being written sequentially. When combined with other space-saving technologies this should ideally not create performance issues.
So how well does your storage system do in this regard? Over the next few posts I will explore these categories in greater depth. If you have any other characteristics I have missed, happy to add them.
