that, see design.txt.
+BACKUP REPOSITORY LAYOUT
+========================
+
+Cumulus backups are stored using a relatively simple layout. Data files
+described below are written into one of several directories on the
+backup server, depending on their purpose:
+ snapshots/
+ Snapshot descriptor files, which quickly summarize each backup
+ snapshot stored.
+ segments0/
+ segments1/
+ Storage of the bulk of the backup data, in compressed/encrypted
+ form. Technically any segment could be stored in either
+ directory (both directories will be searched when looking for a
+ segment). However, data in segments0 might be faster to access
+ (but more expensive) depending on the storage backend. The
+ intent is that segments0 can store filesystem tree metadata and
+ segments1 can store file contents.
+ meta/
+ Snapshot-specific metadata that is not core to the backup. This
+ can include checksums of segments, some data for rebuilding
+ local database contents, etc.
+
+
DATA CHECKSUMS
==============
<algorithm>=<hexdigits>
<algorithm> identifies the checksum algorithm used, and allows new
-algorithms to be added later. At the moment, the only permissible value
-is "sha1", indicating a SHA-1 checksum.
+algorithms to be added later. Permissible values are:
+ "sha1": SHA-1
+ "sha224": SHA-224 (added in version 0.11)
+ "sha256": SHA-256 (added in version 0.11)
<hexdigits> is a sequence of hexadecimal digits which encode the
checksum value. For sha1, <hexdigits> should be precisely 40 digits
This segment could be stored in the filesystem as a file
a704eeae-97f2-4f30-91a4-d4473956366b.tar
The UUID used to name a segment is assigned when the segment is created.
+These files are stored in either the segments0 or segments1 directories
+on the backup server.
Filters can be layered on top of the segment storage to provide
compression, encryption, or other features. For example, the example
NOTE: When naming an object, the segment portion consists of the UUID
only. Any extensions appended to the segment when storing it as a file
-in the filesystem (for example, .tar.bz2) are _not_ part of the name of
-the object.
+in the filesystem (for example, .tar.bz2) and path information (for
+example, segments0) are _not_ part of the name of the object.
There are two additional components which may appear in an object name;
both are optional.
not fall within the original object. The slice specification should be
appended to an object name, for example:
a704eeae-97f2-4f30-91a4-d4473956366b/000001ad[264+1000]
-selects only bytes 264..1263 from the original object. As an
-abbreviation, the slice syntax
+selects only bytes 264..1263 from the original object.
+
+The slice syntax
[<length>]
-is shorthand for
- [0+<length>]
-In place of a traditional slice, the annotation
- [=<length>]
-may be used. This is somewhat similar to specifying [<length>], but
+indicates that all bytes of the object are to be used, but
additionally asserts that the referenced object is exactly <length>
-bytes long--that is, this slice syntax does not change the bytes
-returned at all, but can be used to provide information about the
-underlying object store.
+bytes long. Older versions of Cumulus can also use the syntax
+ [=<length>]
+as a synonym for length assertions, but this notation is deprecated.
+
+(In older versions of the format, the syntax [<length>] was a shorthand
+for [0+<length>]: that is, select the first <length> bytes of the object
+but make no assertions about the overall size. The backup tool has not
+generated such slices since v0.8.)
Both a checksum and a slice can be used. In this case, the checksum is
given first, followed by the slice. The checksum is computed over the
the object. For example
zero[1024]
represents a block consisting of 1024 null bytes. A checksum should not
-be given. The slice syntax should use the abbreviated length-only form.
+be given.
FILE METADATA LISTING
projects / cumulus.git / blobdiff