0.11 [???] - SNAPSHOT FORMAT CHANGES: - Timestamps encoded into the top-level snapshot file name are now UTC instead of the local time zone. This should avoid ambiguity and ensure sorted order is chronological order in the face of time zone changes. - The format identifier is now "Format: Cumulus Snapshot v0.nn" instead of "Format: LBS Snapshot v0.nn" (but the updated tools will accept either variation). - Additional checksum algorithms: sha224, sha256 (sha1 is still supported). - LOCAL DATABASE CHANGES: - Segment utilization data (per snapshot) is tracked differently, to allow better segment cleaning decisions. - New, greatly-enhanced file include/exclude filtering language. This is based on the filter language is rsync (though simplified) and allows glob-like patterns. It also supports filter rules merged at backup-time from files in the file system; this allows, for example, users to control which files in their home directories are included in backups. See doc/exclude.rst for details. - A new Python-based command-line interface to Cumulus. Eventually this should be the only command directly invoked by the user; it will automate backup expiration, segment cleaning, backup mirroring, etc. Configuration options are read from a config file so they can be applied consistently. 0.10 [2012-05-29] - Make a release that packages up various long-existing patches. - Add FTP and SFTP storage backends (from Ralf Schlatterbeck and Albert Dengg). - Various bugfixes. 0.9 [2009-07-28] - Rework storage layer, to make it easier to support local files, S3, and in the future more storage backends. cumulus-util now permits URLS for specifying storage locations, including s3:// for Amazon S3 storage. - The script-level interface between the main cumulus binary and remote storage has changed. It should not yet be considered completely finalized. - Add a tool (cumulus-sync) for copying snapshots between storage repositories. - Add a (mostly proof-of-concept) FUSE interface for accessing snapshots. - Implement a basic garbage-collection command for deleting unused segments. There still needs to be a way to select snapshots to delete (perhaps based on some type of schedule). - Assorted minor bugfixes. 0.8 [2008-08-01] - SNAPSHOT FORMAT CHANGE: The snapshot format has been extended to allow the local database to be partially rebuilt from the contents of the metadata log. This may be useful for disaster recovery and other scenarios. The changes are small, but old tools will not read the new backups. - Add the --rebuild-statcache option which will re-read all files instead of depending on the statcache file to be correct. This will additionally print warnings if a file has changed but would not have been detected as changed based on the statcache, and will add subfile incremental signatures for existing files that do not have them. 0.7 [2008-06-23] - LOCAL DATABASE CHANGE: The addition of subfile incrementals has requires an extension to the local database. The upgrade script contrib/upgrade0.7-localdb.sql should be run prior to running backups with this version. - Name change: the system is now known as Cumulus (replacing the old name of "LBS"). Some traces of the old name still remain. - Initial support for direct backups to remote storage. A sample script is provided for backing up to Amazon S3. Other scripts should be simple to write. The interface should not yet be considered stable, and may change in a future release. - Support for efficient subfile incremental backups. Changes to a small portion of a file do not require re-uploading substantially all of the file, even in cases where there is an insertion or deletion that shifts most of the file contents. - Allow selected files/directories to be restored, not simply an entire snapshot. Additionally, restore files in an order that should optimize performance (restore files based on how they are grouped into segments, instead of lexicographic order). Currently, the implementation of these changes requires that all metadata be loaded into memory when the restore tool runs, so it is more memory-intensive than the old version. This may be fixed in a future version; in the meantime, if the current restore tool requires too much memory, try the old restore tool or the restore.pl script. - Add a verbose ("-v") flag. By default, files will not be listed as they are backed up. The old behavior can be turned back on with -v. 0.6 [2008-02-19] - SNAPSHOT FORMAT CHANGE: A few minor tweaks have been made to the snapshot format. There is nothing substantial--this is primarily a cleaning up of the format before use becomes more widespread. An efficient and explicit means for representing sparse files has been added to the format. Updated tools will be needed to properly understand the new snapshots. Tools will continue to be able to read the old snapshot format. - LOCAL DATABASE CHANGE: The local database has been changed so that segment utilization data for past snapshots is stored more efficiently. The script in contrib/upgrade0.6-localdb.sql must be run on a database to upgrade it prior to running backups with the new tool. - The statcache implementation has been reworked, and the format changed. The first backup will run more slowly since the old statcache data will be ignored. New statcache data will be written to statcache2. The old statcache file can be deleted. - Metadata is shared between snapshots where possible. The --full-metadata option can be used to disable this (completely writing out all metadata with the new snapshot). - Add an "intent" field to snapshots, which specifies informally how long a snapshot is intended to be kept. For example, 1 can be used for daily snapshots, and 7 for weekly snapshots. Segment cleaning is now partly guided by intent values, but tuning of the cleaning algorithms is not yet finished. 0.5.1 [2007-11-13] - Have the lbs-util tool check the version number when reading a snapshot, and signal an error if the format is not recognized. - Include a sample script, contrib/parity-gen, for creating RAID-like parity sets to recover from some segment corruption. The par2 command (http://parchive.sourceforge.net/) is used to actually generate the parity sets; the parity-gen script simply automates maintaining the parity sets. 0.5 [2007-10-16] - Much improved Python interface for accessing and manipulating LBS archives and local database information. The interface should not yet be considered completely stable. - Python implementation of lbs-util now includes most of the features of the Perl implementation, plus some other new features. The Perl library and utility are deprecated and have been removed from this release. - Preliminary snapshot restore support in lbs-util. This is still not yet extensively tested. 0.4 [2007-08-24] - Documentation improvements: a getting started README, and a description of some of the implementation details. - Include an example filter script, lbs-filter-gpg, for encrypting segment data as it is written. - Add support for signed snapshot descriptor files (via the --signature-filter option). Due to the checksums contained in the snapshot descriptor, the signature covers the entire contents of the snapshot. - Preview release of lbs-util.py, a Python-based program for managing LBS snapshots. It currently implements a very simple form of automatic segment cleaning, though this isn't much tested. - Improved segment repacking support; after cleaning segments, objects can be written out in different groups based on age. This may help with long-term segment cleaning efficiency, but probably needs more tuning. 0.3 [2007-08-10] - LOCAL DATABASE CHANGE: A checksums file is now written out along with the snapshot descriptor that contains checksums of segments used in a snapshot. This will allow quick integrity checks of a snapshot, without needing to decompress or decrypt the segment files. The local database schema was modified to store the checksums for segments. - Build dependence on libtar has been dropped. All necessary support for writing TAR files is now directly included with the source. - The snapshot format is now documented, at least in part. See format.txt. - Snapshots now include link count and inode number for files with multiple hard links, so that a restore program could determine which files should be hard linked on restore. The reference restore.pl script does not use this. - Bugfix: Print a help message instead of crashing if no files are specified. - Bugfix: File descriptors of files being backed up were closed twice. In some cases this might have led to an unrelated file being closed (if the file descriptor was quickly re-used). - Preview of a new lbs-util command for maintaining snapshots. Functionality is currently limited. 0.2.2 [2007-07-27] - Update reference restore.pl script to handle octal/hexadecimal (format change introduced in 0.2). - Better support for mixing multiple backup schemes: the statcache file is now per-scheme, so performance should be better in cases where multiple different backups (of different source directories) are made, all sharing the same local database directory. 0.2.1 [2007-07-20] - Bugfix: Do not print an error when only one directory is specified to back up. 0.2 [2007-07-20] - SNAPSHOT FORMAT CHANGE: Metadata logs in backups can now contain octal and hexadecimal values. Old restore programs may not be able to read these new backups. The reference restore.pl script has not yet been updated. - LOCAL DATABASE CHANGE: Backups may be assigned a name with the --scheme= command-line option, to allow different backup sets to be mixed in the same directory. The database schema for the local database is changed. - Improved selection (inclusion/exclusion) of files to be included in a backup. - Device major/minor numbers are dumped for block and character devices. 0.1 [2007-07-14] - Initial release.