Buffer Files

During operation, when the memory buffers become full, the PI API process appends additional data to local files. The buffering service extracts data from the buffer files and sends it to the PI Server when connectivity is restored. For most programs, no modifications are needed to the file buffering configuration since the file caches will be automatically filled and emptied by the PI API processes. However, important options for exceptional cases are discussed below: the path to the partition containing the buffer files, the size of each buffer file, and on UNIX, the number of buffer files.

The PI API file buffers are designed as first-in, first-out (FIFO) buffers. To ensure that the events are sent to the PI Server in chronological order, the buffering service sends the data in the same order as it is received.

The implementation of buffer files is different on Windows and UNIX platforms as described below. For additional information about the piclient.ini parameters mentioned in this section, including default values, refer to the parameter descriptions in "Configuring Buffering using bufserv".

Buffer File on Windows platforms

On Windows platforms, only one buffer file is allowed. Each buffered PI Server has its own buffer file that is independent of the files for other PI Servers. The data file is named APIBUF_[servername].DAT, where [servername] refers to the name used in piclient.ini to indicate the buffered server. When buffering to a collective, a separate file is created for each server.

The buffer file is limited to MAXFILESIZE kilobytes in size. Its location can be set using the FILEBUFPATH parameter if needed. If these parameters are not explicitly set in piclient.ini, default values are used.

Both PI Buffer Subsystem and API Buffer Server use the buffer file on Windows.

Buffer Files on UNIX platforms

On UNIX platforms, beginning with PI API version 1.6.4, multiple buffer files are allowed for each buffered PI Server. Each buffered PI Server has a master header file and from 0 to MAXFILECOUNT data files. When buffering to a collective, a separate master header file and a separate set of data files is created for each server. These are described in more detail below.

The location of these files can be set using the FILEBUFPATH parameter if needed.

Only API Buffer Server uses the buffer files on UNIX platforms. PI Buffer Subsystem is not available on UNIX.

Note: In PI API 1.6.4, the PI API does not limit the number of buffer files that can be created. Therefore it is possible for the buffer files to fill all of the available disk space and cause issues with the system. The MAXFILECOUNT parameter was added in PIAPI 1.6.6, and when used with MAXFILESIZE (and in some cases FILEBUFPATH), the maximum disk space consumed by the buffer files can be controlled. See "Disk space considerations on UNIX platforms" below.

The PI API programs and buffer server maintain a shared master header file named APIBUF_[server name].DAT, where [server name] refers to the name used in piclient.ini to indicate the buffered server. This file stores current information about the read location from which API Buffer Server is loading data and the write location to which PI API programs are appending new events.

The data files are named APIBUF_[server name]_####.DAT, where [server name] refers to the name used in piclient.ini to indicate the buffered server, and #### is the number used to indicate sequential file numbers. Each data file is limited to MAXFILESIZE kilobytes in size, and each file has a header with information about its own data contents. Data files can be created as needed up to the limit indicated by MAXFILECOUNT, or until the disk partition is filled, whichever comes first.

Buffer file data insertion begins with buffer file 0001 and when each buffer file reaches the maximum size, a new file is created and numbered sequentially. The buffer server removes events and sends them to the PI Server beginning with the lowest numbered file and proceeding until all files have been emptied. After each file is emptied and successfully sent to the PI Server, it is deleted.

The file number 0000 is reserved for cases when the buffer server must be stopped, but the PI Server is disconnected. In that case, API Buffer Server must cache the data stored in the first memory buffer in time order before all other data.

Disk space considerations on UNIX platforms

Some installations on UNIX platforms may need configuration changes to accommodate buffering for the quantity of data generated by the particular installed PI API program.

Running out of disk space can cause serious problems on some UNIX platforms. To stop this from happening, the MAXFILESIZE, MAXFILECOUNT, and FILEBUFPATH parameters can be set in piclient.ini to ensure that the buffer files do not fill all the available disk space. The total disk space used for buffering will be MAXFILESIZE times MAXFILECOUNT. By changing the MAXFILESIZE and MAXFILECOUNT, the total disk space used by the buffer files can be controlled. If needed, FILEBUFPATH can also be used to store the files in a location with more available disk space than the default.

Enabling Operational Intelligence