X-Git-Url: http://git.vrable.net/?a=blobdiff_plain;f=doc%2Fformat.txt;h=2c40075221ac5736a4bf428980690bfb70f00c54;hb=HEAD;hp=c273e218c8d47a0c2d32927202510970a55333c7;hpb=748d04c5c242a1dfcc932220eabd14f778e386c9;p=cumulus.git

diff --git a/doc/format.txt b/doc/format.txt
index c273e21..2c40075 100644
--- a/doc/format.txt
+++ b/doc/format.txt
@@ -19,6 +19,30 @@ This document does not explain the rationale behind the format; for
 that, see design.txt.
 
 
+BACKUP REPOSITORY LAYOUT
+========================
+
+Cumulus backups are stored using a relatively simple layout.  Data files
+described below are written into one of several directories on the
+backup server, depending on their purpose:
+    snapshots/
+        Snapshot descriptor files, which quickly summarize each backup
+        snapshot stored.
+    segments0/
+    segments1/
+        Storage of the bulk of the backup data, in compressed/encrypted
+        form.  Technically any segment could be stored in either
+        directory (both directories will be searched when looking for a
+        segment).  However, data in segments0 might be faster to access
+        (but more expensive) depending on the storage backend.  The
+        intent is that segments0 can store filesystem tree metadata and
+        segments1 can store file contents.
+    meta/
+        Snapshot-specific metadata that is not core to the backup.  This
+        can include checksums of segments, some data for rebuilding
+        local database contents, etc.
+
+
 DATA CHECKSUMS
 ==============
 
@@ -32,8 +56,10 @@ format.  The general format used is
     <algorithm>=<hexdigits>
 
 <algorithm> identifies the checksum algorithm used, and allows new
-algorithms to be added later.  At the moment, the only permissible value
-is "sha1", indicating a SHA-1 checksum.
+algorithms to be added later.  Permissible values are:
+    "sha1": SHA-1
+    "sha224": SHA-224 (added in version 0.11)
+    "sha256": SHA-256 (added in version 0.11)
 
 <hexdigits> is a sequence of hexadecimal digits which encode the
 checksum value.  For sha1, <hexdigits> should be precisely 40 digits
@@ -69,6 +95,8 @@ fixed points; an example UUID is
 This segment could be stored in the filesystem as a file
     a704eeae-97f2-4f30-91a4-d4473956366b.tar
 The UUID used to name a segment is assigned when the segment is created.
+These files are stored in either the segments0 or segments1 directories
+on the backup server.
 
 Filters can be layered on top of the segment storage to provide
 compression, encryption, or other features.  For example, the example
@@ -99,8 +127,8 @@ object.
 
 NOTE: When naming an object, the segment portion consists of the UUID
 only.  Any extensions appended to the segment when storing it as a file
-in the filesystem (for example, .tar.bz2) are _not_ part of the name of
-the object.
+in the filesystem (for example, .tar.bz2) and path information (for
+example, segments0) are _not_ part of the name of the object.
 
 There are two additional components which may appear in an object name;
 both are optional.
@@ -121,18 +149,20 @@ is invalid to select using the slice syntax a range of bytes that does
 not fall within the original object.  The slice specification should be
 appended to an object name, for example:
     a704eeae-97f2-4f30-91a4-d4473956366b/000001ad[264+1000]
-selects only bytes 264..1263 from the original object.  As an
-abbreviation, the slice syntax
+selects only bytes 264..1263 from the original object.
+
+The slice syntax
     [<length>]
-is shorthand for
-    [0+<length>]
-In place of a traditional slice, the annotation
-    [=<length>]
-may be used.  This is somewhat similar to specifying [<length>], but
+indicates that all bytes of the object are to be used, but
 additionally asserts that the referenced object is exactly <length>
-bytes long--that is, this slice syntax does not change the bytes
-returned at all, but can be used to provide information about the
-underlying object store.
+bytes long.  Older versions of Cumulus can also use the syntax
+    [=<length>]
+as a synonym for length assertions, but this notation is deprecated.
+
+(In older versions of the format, the syntax [<length>] was a shorthand
+for [0+<length>]: that is, select the first <length> bytes of the object
+but make no assertions about the overall size.  The backup tool has not
+generated such slices since v0.8.)
 
 Both a checksum and a slice can be used.  In this case, the checksum is
 given first, followed by the slice.  The checksum is computed over the
@@ -148,7 +178,7 @@ object must have a slice specification appended to indicate the size of
 the object.  For example
     zero[1024]
 represents a block consisting of 1024 null bytes.  A checksum should not
-be given.  The slice syntax should use the abbreviated length-only form.
+be given.
 
 
 FILE METADATA LISTING