- LBS: An LFS-Inspired Backup Solution
+ Cumulus: Efficient Filesystem Backup to the Cloud
How to Build
------------
Dependencies:
- - libuuid
+ - libuuid (sometimes part of e2fsprogs)
- sqlite3
Building should be a simple matter of running "make". This will produce
-an executable called "lbs".
+an executable called "cumulus".
Setting up Backups
Two directories are needed for backups: one for storing the backup
snapshots themselves, and one for storing bookkeeping information to go
-with the backups. In this example, the first will be "/lbs", and the
-second "/lbs.db", but any directories will do. Only the first
-directory, /lbs, needs to be stored somewhere safe. The second is only
-used when creating new snapshots, and is not needed when restoring.
+with the backups. In this example, the first will be "/cumulus", and
+the second "/cumulus.db", but any directories will do. Only the first
+directory, /cumulus, needs to be stored somewhere safe. The second is
+only used when creating new snapshots, and is not needed when restoring.
1. Create the snapshot directory and the local database directory:
- $ mkdir /lbs /lbs.db
+ $ mkdir /cumulus /cumulus.db
2. Initialize the local database using the provided script schema.sql
from the source:
- $ sqlite3 /lbs.db/localdb.sqlite
+ $ sqlite3 /cumulus.db/localdb.sqlite
sqlite> .read schema.sql
sqlite> .exit
3. If encrypting or signing backups with gpg, generate appropriate
keypairs. The keys can be kept in a user keyring or in a separate
keyring just for backups; this example does the latter.
- $ mkdir /lbs.db/gpg; chmod 700 /lbs.db/gpg
- $ gpg --homedir /lbs.db/gpg --gen-key
+ $ mkdir /cumulus.db/gpg; chmod 700 /cumulus.db/gpg
+ $ gpg --homedir /cumulus.db/gpg --gen-key
(generate a keypair for encryption; enter a passphrase for
the secret key)
- $ gpg --homedir /lbs.db/gpg --gen-key
+ $ gpg --homedir /cumulus.db/gpg --gen-key
(generate a second keypair for signing; for automatic
signing do not use a passphrase to protect the secret key)
Be sure to store the secret key needed for decryption somewhere
To achieve better compression, the encryption key can be edited to
alter the preferred compression algorithms to list bzip2 before
zlib. Run
- $ gpg --homedir /lbs.db/gpg --edit-key <encryption key>
+ $ gpg --homedir /cumulus.db/gpg --edit-key <encryption key>
Command> pref
(prints a terse listing of preferences associated with the
key)
to be "Z3 Z2 Z1" which stands for "BZIP2, ZLIB, ZIP")
Command> save
- Copy the provided encryption filter program, lbs-filter-gpg,
+ Copy the provided encryption filter program, cumulus-filter-gpg,
somewhere it may be run from.
4. Create a script for launching the LBS backup process. A simple
version is:
#!/bin/sh
- export LBS_GPG_HOME=/lbs.db/gpg
+ export LBS_GPG_HOME=/cumulus.db/gpg
export LBS_GPG_ENC_KEY=<encryption key>
export LBS_GPG_SIGN_KEY=<signing key>
- lbs --dest=/lbs --localdb=/lbs.db
- --filter="lbs-filter-gpg --encrypt" --filter-extension=.gpg \
- --signature-filter="lbs-filter-gpg --clearsign" \
+ cumulus --dest=/cumulus --localdb=/cumulus.db \
+ --filter="cumulus-filter-gpg --encrypt" --filter-extension=.gpg \
+ --signature-filter="cumulus-filter-gpg --clearsign" \
/etc /home /other/paths/to/store
Make appropriate substitutions for the key IDs and any relevant
Data in these segments will be rewritten into new segments in future
backups to eliminate the dependence on the almost-empty old segments.
-Segment cleaning is currently a mostly manual process. An automatic
-tool for performing segment cleaning will be available in the future.
-
-Old backup snapshots can be pruned from the snapshot directory (/lbs) to
-recover space. Deleting an old backup snapshot is a simple matter of
-deleting the appropriate snapshot descriptor file (snapshot-*.lbs) and
-any associated checksums (snapshot-*.sha1sums). Segments used by that
-snapshot, but not any other snapshots, can be identified by running the
-clean-segments.pl script from the /lbs directory--this will perform a
-scan of the current directory to identify unreferenced segments, and
-will print a list to stdout. Assuming the list looks reasonable, the
-segments can be quickly deleted with
+The provided cumulus-util tool can perform the necessary cleaning. Run
+it with
+ $ cumulus-util --localdb=/cumulus.db clean
+Cleaning is still under development, and so may be improved in the
+future, but this version is intended to be functional.
+
+Old backup snapshots can be pruned from the snapshot directory
+(/cumulus) to recover space. A snapshot which is still referenced by
+the local database should not be deleted, however. Deleting an old
+backup snapshot is a simple matter of deleting the appropriate snapshot
+descriptor file (snapshot-*.lbs) and any associated checksums
+(snapshot-*.sha1sums). Segments used by that snapshot, but not any
+other snapshots, can be identified by running the clean-segments.pl
+script from the /cumulus directory--this will perform a scan of the
+current directory to identify unreferenced segments, and will print a
+list to stdout. Assuming the list looks reasonable, the segments can be
+quickly deleted with
$ rm `./clean-segments.pl`
+A tool to make this easier will be implemented later.
The clean-segments.pl script will also print out a warning message if
any snapshots appear to depend upon segments which are not present; this
Restoring a Snapshot
--------------------
-The restore.pl script is a simple (proof-of-concept, really) program for
-restoring the contents of an LBS snapshot. Ideally, it should be stored
-with the backup files so it is available if it is needed.
+The contrib/restore.pl script is a simple (proof-of-concept, really)
+program for restoring the contents of an LBS snapshot. Ideally, it
+should be stored with the backup files so it is available if it is
+needed.
The restore.pl script does not know how to decompress segments, so this
step must be performed manually. Create a temporary directory for
(*.lbs) in the temporary directory, and a directory where the restored
files should be written.
-A better recovery tool will be provided in the future.
+The cumulus-util program also now has some support for restoring
+snapsots (documentation coming soon).
projects / cumulus.git / blobdiff