Solr Standalone Server Metadata Catalog Backup

The Need for a Backup & Recovery Plan

Prior to setting up backup for the Solr Metadata catalog, it is important to plan how backup and recovery will be executed. The amount and velocity of data entering the catalog differ depending on the use of the system. With this, there will be varying plans depending on the need. It is important to get a sense of how often the data changes in the catalog in order to determine how often the data should be backed up. When something goes wrong with the system and data is corrupted, how much time is there to recover? A plan must be put in place to remove corrupted data from the catalog and replace it with backed up data in a time span that fits deadlines. Equipment must also be purchased to maintain backups, and this equipment may be co-located with local production systems or remotely located at a different site. A backup schedule will also have to be determined so that it does not affect end users interacting with the production system.

Backing Up Data from the Solr Server Standalone Metadata Catalog

The Solr server contains a built-in backup system capable of saving full snapshot backups of the catalog data upon request. Backups are created by using a web based service. Through making a web based service call utilizing the web browser, a timestamped backup can be generated and saved to a local drive, or location where the backup device has been mounted.
The URL for the web call contains three parameters that allow for the customization of the backup:

command: allows for the command 'backup' to backup the catalog
location: allows for a file system location to place the backup to be specified
numberToKeep: allows the user to specify how many backups should be maintained. If the number of backups exceed the "numberToKeep" value, the system will replace the newest backup with the oldest one.

An example URL would look like the following:

http://127.0.0.1:8181/solr/replication?command=backup&location=d:/solr_data&numberToKeep=5

The IP address and port in the URL should be replaced with the IP address and port of the Solr Server. The following URL would run a backup, save the backup file in file location D:/solr_data, and it would keep up to (5) backup files at any time. To execute this backup, first ensure that the Solr server is running. Once the server is running, create the URL and copy it into a web browser window. Once the URL is executed, the following information is returned to the browser:

Solr Backup Response

<?xml version="1.0" encoding="UTF-8"?>
<response>
	<lst name="responseHeader">
		<int name="status">0</int>
		<int name="QTime">15</int>
	</lst>
	<str name="status">OK</str>
</response>

If the status equals 0, then that means there was success. Qtime shows the time it took to execute the backup (in milliseconds). Backup files are saved in directories which are given the name snapshot along with a timestamp. Within the directory are all of the files that contain the data from the catalog.

Restoring Data to the Solr Server Standalone Metadata Catalog

When data has been corrupted or information has accidentally been deleted, a restoration of the catalog must occur. The backup files acquired from the previous section will be used to restore data into the catalog.

The first step in the process is to choose which data backup will be used for restoring the catalog. A most recent backup maybe the correct choice, or the last stable backup may be a better option.
At this point one more backup may be executed to save the corrupted data just in case it needs to be revisited.
The next step is to shutdown the Solr server. The restoring of the catalog cannot occur while the server is running.
Next step is to find the index which contains all of the Solr data. This index is found at $DDF_INSTALL/solr/collection1/data/index. All of these files within the index directory should be deleted.
Now copy the files from the chosen backup directory into the index directory.
Restart the Solr server and the data should now be restored.

Suggestions For Managing Backup & Recovery

Here are some helpful suggestions for setting up data backups and recoveries:

Acquire a backup drive that is separate from the media that runs the server. Mount this drive as a directory and save backups to that location.
Ensure that the backup media has enough space to support the number of backups that need to be saved.
Run a scheduler program that calls the backup URL on a timed basis.
Put indicators in place that can detect when data corruption may have occurred.
Testing a backup before recovery is possible. A replicated "staging" Solr server instance can be stood up, and the backup can be copied to that system for testing before moving it to the "production" system.

Browser not supported