Submitting your feedback...
Knowledge Base Article
KB01032 - PI Data Archive Backup Best Practices
Product: PI Data Archive
Version(s): 3.x

This KB article was originally published as the February 2014 Technical Support Newsletter Tech Tip.

st-widget-{image: YouTube_Icon.jpg}Check the Youtube Playlist about How to setup backups for the PI Data Archive


Backing up the PI Server, also known as the PI Data Archive, is a critical step in preventing the loss of valuable data. Because this process involves backing up not only data, but also all of the tables and settings that allow the PI Data Archive to run as specified, it is vital that this process is followed in accordance to our best practices. In this month's Tech Tip, we cover many different topics regarding the PI Data Archive Backup, including the essential components within a backup, the recommended procedures for configuring PI Backups on both single and collective members, and how to double check the status of PI Backups so that you may be prepared for any incident.

What is a PI Data Archive backup?

st-widget-{image: YouTube_Icon.jpg}OSIsoft: What Are Backups of the PI Data Archive & Why Are They Important?

A PI Data Archive backup is a copy of both the data and configuration files that can be used to restore the system in the event of an incident.  This incident could come in the form of data corruption, unintended configuration change, or a hardware failure.  The aforementioned data and configuration files include the archive files, event queues, message logs, settings, timeout parameters, and other database files.  Note that simply backing up the archives files is NOT sufficient.  Configuration information such as the point database is essential to accessing the data within the archives.  To see the entire list of files that a backup will copy on your server, run the command below from the %piserver%\adm directory.

piartool -backup -identify -verbose -type FULL

In the above example, the backup type was specified.  There are several different backup types that can be performed.  Regardless of type, non-archive components will be included, but the inclusion of archive components varies based on type as summarized in the table below.  Archive files have a last backup time and a last modified time.  An archive has a modified status when the last modified time is more recent than the last backup time.
Backup Types
Type Archive Files Copied Effect on Last Backup Time
Copy All Not updated.
Full All Updated.
Incremental Only modified files Updated
Differential Only modified files Not Updated
Numarch/Cutoff A specified number of files more recent than a specified date  Updated
Table 1. The table above is a summary of backup types.

In addition to copying the files, the backup operation also validates the PI Snapshot and Base Subsystem databases as well as the Primary Archive.

Sometimes, data was inserted into an archive via backfilling or history recovery. This modifies the archive file and flags it for backup - even if the amount of data inserted is small compared to the archive size. 

What is the recommended procedure to back up the PI Data Archive?

  1. Establish a baseline backup on a dedicated drive – The first step of any PI System administrator’s backup strategy should be to establish a complete backup that can be used as a baseline.  The backup type Full should be used for this purpose as it will update the last backup time of the files.  Once this baseline is established, periodically scheduled Incremental backups will update the baseline backup with changed files instead of copying every file each time.

    The backup should be located on a dedicated physical drive.  While the best practice is to have separate physical drives for your operating system, PI Data Archive binaries, event queue, archives, and backups, you should separate the operating system, archives and backups at a minimum.  There are two reasons for this:
    • Safeguard your data against disk failure.  If the backup files are on the same drive as the archives, then there is no benefit in the event of hardware failure.
    • Improve performance.  Backup operations require a lot of disk activity to copy files.  While CPU and file system cache resource usage is inevitable during the backup operation, using a separate drive ensures that the impact on the physical disk performance of the PI Data Archive is minimized.
    • For detailed steps, check the Youtube video below:
  1. Create a scheduled task to update the backup – After the baseline backup is in place, it needs to be kept up to date with an automatically executed backup task.  The Incremental backup type is recommended as only files that have been modified since the last backup time are affected.  Due to the resource usage discussed above, this scheduled task should be scheduled for a time when there is typically the lowest demand on the PI Data Archive.  The default value used when a backup is installed using pibackup.bat is 3:15 AM.
    • For detailed steps, check the Youtube video below:
  1. Back up the files in the backup directory to a separate location – In addition to your daily backup task to a dedicated drive on the PI Data Archive, you should also back up the backup directory to another medium entirely separate from the PI Data Archive machine.  It is recommended to use a third-party backup tool to accomplish this, but as an alternative, the file pisitebackup.bat in the directory %piserver%\adm can be used to copy the local backup to a remote computer by following the instructions in the file pisitebackup.bat.example.
    This two-step backup strategy allows for data recovery even if the entire PI Data Archive is lost.  An added benefit is that a backup repository can be created to store the backups at different points in time to be restored to.  For example, if the backup repository contains weekly backups for the past month and a configuration issue is discovered two weeks after the fact, then the three-week-old backup could be used to restore the configuration.
  1. Validate your backups – Now that you have your plan to generate and update backups in place, you should also plan to verify your backups periodically.  Your data is only as protected up to your last good backup, so you will want a system in place to know if your backups are failing.  This leads us into our next topic.
st-widget-{image: repositorycorrected.png}
Figure 1. Maintaining a backup repository at a separate location allows for multiple points in time to which you can restore the PI Data Archive

How can I check the status of my backups?

st-widget-{image: YouTube_Icon.jpg}OSIsoft: How to Check PI Data Archive Backup History & Confirm Successful Backups 

PI System administrators can monitor the status of their backups within PI System Management Tools (PI SMT). In PI System Management Tools, expand Operation and select the Backups plugin. The two most important columns to focus on are the Start Time and the Status, which is shown in the figure below. The Start Time column indicates the time that the backup operation began, but more importantly, it represents the point in time up to when you could restore your data based on that backup.  The Status column indicates whether or not the backup operation succeeded.  A successful backup yields a status of "0 Success" as shown below, but in the event that a backup fails or is aborted, the status will include the error, for example: " [-16915] Backup was aborted."

st-widget-{image: backup history.png}

Figure 2. PI SMT Backups plugin can be used to view the backup history for the selected PI Server and execute ad hoc backups

While checking the PI SMT Backups plugin allows a PI System administrator to quickly check the overall status of their backups, there are more rigorous checks that can be performed less frequently to ensure the backups are created successfully.
  • Review the backup - There are log files of the form pibackup_DD-MMM-YY_hh:mm:ss in the backup directory that will indicate the type of backup run, the actions taken, and the status of each action.  This is the first place to look for more information if a backup has a bad status in PI SMT.
  • Verify your site-specific backup – If you are following the best practices and creating backups on a separate machine, verify that the files in this backup are in the appropriate location and are up to date.
  • Monitor free disk space - Be proactive by verifying that your backups do not fail due to a lack of disk space on their dedicated drive.  You can use the "Free Megabytes" or "% Free Space" performance counters to keep track of this information for the backup disk.
  • Test your backup – If you have a test environment, then you can periodically restore your backup in order to verify its status.

What special considerations are there for backing up PI Collectives?

Although any member of a collective can be reinitialized by the primary, this should not be seen as a replacement for backups.  First of all, there are several files included within backups that are not replicated between collective members including tuning parameters, message logs, and PI Batch components.  Another consideration is efficiency.  If the collective members are separated geographically, re-initializing a secondary member could take significantly longer than restoring it from a local backup.  Furthermore, since archive start and end times are not synchronized across members, it will most likely be necessary to backfill data to prevent an archive gap once the collective has been reinitialized.  It may still be necessary to reinitialize a collective member after it is restored from a backup if the configuration of the backup is out of sync with the primary member.  Consequently, if the primary is restored from a backup, all secondary nodes may need to be reinitialized.

When backing up PI collectives,  it is a good idea to stagger backups performed on collective members to limit any performance impact to a single server at a time.  It should also be noted that when a secondary server is reinitialized, the custom backup scripts pintbackup.bat and pisitebackup.bat are copied from the primary.  If used, these scripts should be generalized to apply to all members of the collective.

Where can I find out more about backups?

For more information about backups, refer to PI Data Archive documentation in Live Library.

If you experience problems with your backups or need assistance developing your backup strategy, contact OSIsoft Technical Support.
Article ID: KB01032 Created: 2014-04-16
Article Type: How-To Last Updated: 2018-07-25