X-Git-Url: http://git.vrable.net/?a=blobdiff_plain;f=format.txt;fp=format.txt;h=1e4ce24416fcc9a64cfba58253ec95cfce7f97cf;hb=f9deef3757145dab5eead5bc7e068defdb0aacd8;hp=7bf04745378df49c627e507ae13c19e9558db289;hpb=738a672b1a4a95eb24b785c9238594de99f8e314;p=cumulus.git diff --git a/format.txt b/format.txt index 7bf0474..1e4ce24 100644 --- a/format.txt +++ b/format.txt @@ -32,22 +32,11 @@ Object naming: - Each segment is assigned a unique 128-bit identifier (uuid). Each segment is stored as a separate file whose name is based on its uuid. - - Objects within a segment are numbered sequentially, with a 32-bit - counter. -Thus, each object may be referred to with a unique 160 (128 + 32) bit -identifier. - -Segment structure: -There are two main options: - - Streaming format: Each object is prepended with a header, and then - all (header, object) pairs are concatenated. This is inspired by - the tar file format. Can be written out in one pass and also - processed when read back in one pass. Well-adapted to streaming - transformations, such as compression. - - Indexed format: Each segment contains a table giving the starting - position and length of each object. This is somewhat similar to - PDF. Data can still be written out in a single pass, but reading - will require random access. + - Objects within a segment are numbered, using a 32-bit counter. + +Each segment is structured as a TAR file (optionally filtered through a +compressor such as gzip/bzip2, or encrypted). Objects are stored as +individual files. File attributes: Metadata for each file is stored in a dictionary. Dictionary keys include: