Magma Backup and Recovery
Last updated at 2:59 pm UTC on 13 December 2011
Background
Every successful commit is numbered consecutively by the server. Each CommitPackage of each commit is serialized and appended to a log file known as "commits.log". Updates to the various repository files are flushed to disk only every five seconds, but several commits could have happened during that time. If the server is forcibly terminated by an outside event (such as a power outage), the commits.log file is read upon the next startup to replay those last few commits to the repository files.
In this way, any client that receives a successful response from a commit request can be assured their changes are secured in the database.
To manage the size of the commits.log file, every five minutes or, as defined by #logStartFrequency, it is safely closed and renamed into a subdirectory within the repository directory called "commits". Its name is also changed within this subdirectory to "commits.n.log", where n is the number of the highest commit in that file. The "commits.log" file, in the main repository directory, is known as the "active" commit log file. The old renamed files, in the commits subdirectory, are known as "archive" commit logs.
The active commit log file may also be safely archived on demand, without waiting for the (five minute) timer to expire by:
myRepositoryController archiveCommitLog
These old commit log files are generally of no use to the repository that wrote them, because it would only ever need up to the last five seconds worth of data in the active log. The old files can be useful to clients for a limited time, however.
They're also useful for copying to the commits directory of a "cold-backup" of this repository and simply opening it:
MagmaRepositoryController open: 'model'; close
On every open, Magma checks its commits directory for commit-log files and brings the repository up to date from those files.
Cleaning Up Old Commit Logs
Because of the variety of possible configurations, there is no way Magma can make always correct assumptions about when to clean up old commit logs. They will continue to accumulate in the "commits" directory until you ask your repositoryController to clean them up:
myRepositoryController deleteUnnecessaryCommitLogFiles
Making a Backup
Although backups can be taken while the server is running with no interruption of service, it could affect performance so it is better to simply keep a separate "offline" copy of the repository and update it by copying commit-log files to its "commits" directory and opening it.
Nevertheless, the way to make a full copy of a running repository is:
myMagmaSession modelBackupTo: aFileDirectory
aFileDirectory may be a FileDirectory or a ServerDirectory (for backing up straight to an FTP server).
aFileDirectory is created if it doesn't already exist. If a repository already exists at that location, a warning will be issued before deleting all files in the directory. Proceed past the warning and the active commit log is archived before a background (Smalltalk) Process is forked to copy the repository files. To ensure consistency, all flushing to the repository files from memory is suspended during this copy creating a potential increase in memory utilization if activity is high.
Once the files are copied, a bit is finally set in the target repository indicating it is in "restore mode," meaning it can now replay commit log files beginning with the next higher commit number. This is discussed in detail in the subsequent section, "Restoring from a Backup".
The backup can now be started as a separate, read-only repository, and keep itself up to date with the original even as subsequent commits come in (see Magma High Availability).
Restoring from a Backup
When a backup is taken, it is a snapshot of the persistent model up to a certain commit, identifiable by the #commitNumber of the repository. While the files are being copied, further commits continue to be received and written to the active commit.log file, but not the repository files because they are being copied. Therefore, the copied repository files are behind even before they are finished being written.
However, thanks to Magma's high-availability (HA) function, the warm-backup can be started and will automatically contact the original server to receive these final updates and all future ones in real time.
Restoring to a Point in Time
In addition to the commitNumber, Magma also stamps each record in the commit logs with the local DateAndTime.
In (hopefully) rare cases, an application bug or user error may result in an unintended commit with drastic consequences to the persistent model. If you know the approximate time the unintended commit occurred, a drastic solution is available; recover to a point in time.
In this case, a prior backup prior to the drastic event must be available. Put the commit logs timestamped up to and, even, a little beyond the drastic commit into the "commits" subdirectory.
When Magma starts, it will determine that the repository is behind the commit logs and, therefore, a forward recovery is needed. A MagmaForwardRecoveryNeeded warning message will be displayed:
"A forward recovery from 2007-08-02T22:01:09+00:00 is necessary. If you wish to recover to a point-in-time, press Debug and save the desired target DateAndTime in my 'targetRecoveryTime'."
Warm Backups
As mentioned, the backup is a nearly-exact copy of the repository. It differs by one bit from the original, its #restoreMode bit is set. When a repository is in this mode commits are not permitted because it expects to replay commits that occurred since the backup.
To do this, simply open the repository:
myRepositoryController := MagmaRepositoryController open: (MagmaLocalLocation path: '/path/to/your/repository')
Proceed past the warning and recovery to the highest available commit-log occurs automatically. Once up to date, future commits will be obtained automatically from the original repository. See Magma High Availability for more information.
