From: Michael Vrable Date: Thu, 15 Nov 2007 23:38:59 +0000 (-0800) Subject: Changes to the metadata log format. X-Git-Url: http://git.vrable.net/?p=cumulus.git;a=commitdiff_plain;h=6c94114148c42e795b8abbef9532f399c24f3ed5 Changes to the metadata log format. A small collection of changes to make the snapshot format a little more rational (I hope). The intent is to also more or less merge the formats of the metadata log and statcache. There will likely be other changes in the future in the course of working on metadata scalability. The format is not yet finalized! WARNING: These changes will require new tools to read the generated backup snapshots. The version number in the format has been updated. This changeset does not include the necessary changes to the snapshot-parsing code. --- diff --git a/doc/format.txt b/doc/format.txt index 78f32c3..5c9d5fa 100644 --- a/doc/format.txt +++ b/doc/format.txt @@ -1,5 +1,6 @@ Backup Format Description for an LFS-Inspired Backup Solution + Version: "LBS Snapshot v0.6" NOTE: This format specification is not yet complete. Right now the code provides the best documentation of the format. @@ -166,7 +167,8 @@ for each field is specified in the field listing that follows. and those starting with 0x are intepreted as hexadecimal. Common fields (required in all stanzas): - name [encoded string]: Full path of the file archived. + path [encoded string]: Full path of the file archived. Note: In + previous versions (<= 0.2) the name of this field was "name". user [special]: The user ID of the file, as an integer, optionally followed by a space and the corresponding username, as an escaped string enclosed in parentheses. @@ -176,14 +178,19 @@ Common fields (required in all stanzas): mode [integer]: Unix mode bits for the file. type [special]: A single character which indicates the type of file. The type indicators are meant to be consistent with the - characters used to indicate file type in a directory listing: - - regular file + characters used with the -type option to find(1), and the file + type checks in test(1): + f regular file b block device c character device d directory l symlink p pipe s socket + Note that previous versions used '-' to indicate a regular file. + This character should not be generated in any new snapshots, but + may be encountered in old snapshots (those with a format version + <= 0.2). mtime [integer]: Modification time of the file. Optional common fields: @@ -199,6 +206,7 @@ Optional common fields: produced by the standard tool is // (where and specify the device of the containing filesystem and is the inode number of the file). + ctime [integer]: Change time for the inode. Special fields used for regular files: checksum [string]: Checksum of the file contents. @@ -211,6 +219,16 @@ Special fields used for regular files: references which should be parsed in the same manner as the data field. +Special fields used for symbolic links: + target[encoded string]: The target of the symlink, as returned by + readlink(2). Note: In old version of the format (<= 0.2), this + field was called "contents" instead of "target". + +Special fields used for block and character device files: + device[special]: The major and minor number of the device. Encoded + as "major/minor", where major is the major device number encoded + into an integer, and minor is the minor device number. + SNAPSHOT DESCRIPTOR =================== @@ -230,8 +248,8 @@ is %Y%m%dT%H%M%S (20070806T092239 means 2007-08-06 09:22:39). The contents of the descriptor are a set of RFC 822-style headers (much like the metadata listing). The fields which are defined are: - Format: The string "LBS Snapshot v0.2" which identifies this file as - an LBS backup descriptor. The version number (v0.2) might + Format: The string "LBS Snapshot v0.6" which identifies this file as + an LBS backup descriptor. The version number (v0.6) might change if there are changes to the format. It is expected that at some point, once the format is stabilized, the version identifier will be changed to v1.0. diff --git a/scandir.cc b/scandir.cc index e4fe21f..a165f4e 100644 --- a/scandir.cc +++ b/scandir.cc @@ -348,7 +348,7 @@ void dump_inode(const string& path, // Path within snapshot fprintf(stderr, "error reading symlink: %m\n"); } else if (len <= stat_buf.st_size) { buf[len] = '\0'; - file_info["contents"] = uri_encode(buf); + file_info["target"] = uri_encode(buf); } else if (len > stat_buf.st_size) { fprintf(stderr, "error reading symlink: name truncated\n"); } @@ -356,7 +356,7 @@ void dump_inode(const string& path, // Path within snapshot delete[] buf; break; case S_IFREG: - inode_type = '-'; + inode_type = 'f'; file_size = dumpfile(fd, file_info, path, stat_buf); file_info["size"] = encode_int(file_size); @@ -799,7 +799,7 @@ int main(int argc, char *argv[]) } FILE *descriptor = fdopen(descriptor_fd, "w"); - fprintf(descriptor, "Format: LBS Snapshot v0.2\n"); + fprintf(descriptor, "Format: LBS Snapshot v0.6\n"); fprintf(descriptor, "Producer: LBS %s\n", lbs_version); strftime(desc_buf, sizeof(desc_buf), "%Y-%m-%d %H:%M:%S %z", &time_buf); fprintf(descriptor, "Date: %s\n", desc_buf);