Block size

The size of data blocks in a file system can be specified at file system creation by using the -B option on the mmcrfs command or allowed to default to 256 KB. This value cannot be changed without re-creating the file system.

GPFS™ supports these block sizes for file systems: 64 KB, 128 KB, 256 KB, 512 KB, 1 MB, 2 MB, 4 MB, 8 MB and 16 MB (for GPFS Native RAID only). This value should be specified with the character K or M as appropriate, for example: 512K or 4M. You should choose the block size based on the application set that you plan to support and whether you are using RAID hardware. For more information about GPFS Native RAID, see IBM Spectrum Scale RAID: Administration in the Elastic Storage Server documentation on IBM® Knowledge Center.

The --metadata-block-size option on the mmcrfs command allows a different block size to be specified for the system storage pool, provided its usage is set to metadataOnly. This can be especially beneficial if the default block size is larger than 1 MB. Valid values are the same as those listed for the -B option.

If you plan to use RAID devices in your file system, a larger block size may be more effective and help avoid the penalties involved in small block write operations to RAID devices. For example, in a RAID configuration using 4 data disks and 1 parity disk (a 4+P RAID 5 configuration), which uses a 64 KB stripe size, the optimal file system block size would be an integral multiple of 256 KB (4 data disks × 64 KB stripe size = 256 KB). A block size of an integral multiple of 256 KB results in a single data write that encompasses the 4 data disks and a parity-write to the parity disk. If a block size smaller than 256 KB, such as 64 KB, is used with the same RAID configuration, write performance is degraded by the read-modify-write behavior. A 64 KB block size results in a single disk writing 64 KB and a subsequent read from the three remaining disks in order to compute the parity that is then written to the parity disk. The extra read degrades performance.

The choice of block size also affects the performance of certain metadata operations, in particular, block allocation performance. The GPFS block allocation map is stored in blocks, similar to regular files. When the block size is small:

It takes more blocks to store a given amount of data resulting in additional work to allocate those blocks
One block of allocation map data contains less information

Note: The choice of block size is particularly important for large file systems. For file systems larger than 100 TB, you should use a block size of at least 256 KB.