Track which objects are used in which snapshots in the local database.

[cumulus.git] / format.txt

diff --git a/format.txt b/format.txt
index 7bf0474..1e4ce24 100644 (file)
--- a/format.txt
+++ b/format.txt
@@ -32,22 +32,11 @@ Object naming:
   - Each segment is assigned a unique 128-bit identifier (uuid).  Each
     segment is stored as a separate file whose name is based on its
     uuid.
-  - Objects within a segment are numbered sequentially, with a 32-bit
-    counter.
-Thus, each object may be referred to with a unique 160 (128 + 32) bit
-identifier.
-
-Segment structure:
-There are two main options:
-  - Streaming format: Each object is prepended with a header, and then
-    all (header, object) pairs are concatenated.  This is inspired by
-    the tar file format.  Can be written out in one pass and also
-    processed when read back in one pass.  Well-adapted to streaming
-    transformations, such as compression.
-  - Indexed format: Each segment contains a table giving the starting
-    position and length of each object.  This is somewhat similar to
-    PDF.  Data can still be written out in a single pass, but reading
-    will require random access.
+  - Objects within a segment are numbered, using a 32-bit counter.
+
+Each segment is structured as a TAR file (optionally filtered through a
+compressor such as gzip/bzip2, or encrypted).  Objects are stored as
+individual files.
 
 File attributes: Metadata for each file is stored in a dictionary.
 Dictionary keys include: