From: Michael Vrable Date: Mon, 4 Jun 2007 17:40:33 +0000 (-0700) Subject: Design note: grouping also allows better compression. X-Git-Url: https://git.vrable.net/?a=commitdiff_plain;h=a7c068d940ab32b75dc2dfaf11146b363f95ca88;p=cumulus.git Design note: grouping also allows better compression. --- diff --git a/design.txt b/design.txt index 6aa53e0..e515359 100644 --- a/design.txt +++ b/design.txt @@ -84,6 +84,8 @@ associated with storing many small files. In any transfer protocol which is not pipelined, transferring many small files may be slower than transferring the same quantity of data in larger files. Small files may also lead to more wasted storage space due to internal fragmentation. +Grouping files together gives the chance for better compression, taking +advantage of inter-file similarity. Grouping is even more important if the snapshot format breaks files apart into blocks for storage, since the number of blocks could be far