Most files aren't contiguous on a disk either, you know.

Which makes inability to punch wholes in the middle of the files without having to read/write huge chunks of data quite annoying, especially when you write a linker or an archiver. Reserving enough space for the header is quite non-trivial.

Filesystems try fairly hard to make files mostly contiguous on disk, but they don't do the same to directories.