Replication by Priority

Description

Replicator provides the possibility to set priority to the CAB files. The feature is specifically designed for low bandwidth network to avoid sending/receiving huge CAB files. The following priority levels are available:

• High priority files contain the meta data or critical information.
• Low priority cab files contain all the data. 
  
  

In this respect, Adonis Replicator covers the following:

• Identify those replication task fields that should be added into the low priority CAB files.
• Identify the bandwidth thresholds that allow sending low priority files and any files (low and high), stop using a satellite connection to send low priority files and send any file.
• Identify the maximum response time to check a satellite connection.

Setup

To set up the bandwidth options, first, run Adonis Replicator. Within the application, navigate to Settings and click the Settings Wizard button. This opens the following dialog:

In the dialog, make sure that the Mail system field is set to FTP and click Next to proceed. The dialog appears:

Within the dialog, click the Satellite / Bandwidth button. Clicking the command makes the Satellite and bandwidth options dialog appear:

You can specify your preferences regarding the following points:

When a satellite connection to the Internet is used

• allow transfer of any files. If selected, either low or high priority files can be sent/received.
• forbid transfer of low priority files. If selected, low priority files cannot be sent/received.
• forbid transfer of any files. If selected, either low or high priority files cannot be sent/received.
  
  

Bandwidth thresholds 

• Stop transfer of low priority files when bandwidth is less than (KB/s). In the field, set the bandwidth limit (KB/s) for sending/receiving low priority files.
• Stop transfer of any files when bandwidth is less than (KB/s). In the field, set the bandwidth limit (KB/s) for sending/receiving any files.

If the Bandwidth thresholds options contain non-zero values, Replicator measures the bandwidth when downloading first files.

Advanced settings

• Minimum server response time via a satellite connection (milliseconds). In the field, specify the minimum time the server may respond.
• Duration of bandwidth measurement process (milliseconds). In the field, how long the measurement of bandwidth can take.
  
  

Click OK to save the changes.

Define Low Priority Fields

To define low priority fields, first, run Adonis Personnel Manager. Within the application, navigate to the Setup ribbon tab > Adonis Replicator > View -1 > Replication Tasks:

In the list of tasks, proceed as described below:

1. Select the task containing the fields that you want to be exported in the low priority CAB file.
2. In the Low Priority Fields column, double-click the cell where a low priority field is to be added. When the Browse  button appears, click it to view the list of fields to be added:
   
3. In the Select Values dialog, select the fields to be added as low priority fields. To select all the fields, click Tag All at the bottom of the dialog. To clear the selection, click Untag All at the bottom of the dialog.
4. Click OK to apply the selection.

As a result, the affected cell in the Replication Tasks grid appears as yellow-highlighted:

There is another way you can use to add low priority fields. Simply, enter LowPriorityFields='VALUE' where VALUE is a field name to be added with low priority and  ImportTableName=' PW001OLEDOCS ' in the replication tasks table.

The LowPriorityFields value must not contain:

• primary key fields
• foreign key fields
• fields used in unique constraint
• fields that do not allow nulls

Workflow

This section covers the workflow of Replicator in respect of low priority files at all the application phases. Based on the satellite and bandwidth options defined (see Setting Up Replicator), the application may have different procedures.

Receive phase

In case either Forbid transfer of low priority files or Forbid transfer of any files option is selected, Replicator calculates server response time and compares it with the value entered in the Minimum server response via a satellite connection field. Based on the specified option and calculated values of time response, Replicator starts downloading high priority files (files without ~1 at the end of their names). Only when all the high priority files are received, Replicator starts transmitting low priority files (if allowed by the options).

The bandwidth is measured a fixed period of time specified in the Duration of bandwidth measurement process (milliseconds) field. In case a file is small and, therefore, it is downloaded faster than the period ends, the calculated value is discarded and Replicator makes a new attempt to measure the bandwidth value while downloading the next file.

When the bandwidth is calculated, Replicator compares its value with the values in the fields. If the bandwidth appears to be lower that the threshold value, Replicator immediately stops downloading the current file and the Receive phase is terminated. In this case, Replicator adds a corresponding notification to the log file. Next time when Replicator starts and goes to the Receive phase, it will continue downloading the file that has not been downloaded completely. 

Import Phase

Replicator primarily imports high priority files and then low priority files.

Export Phase

For any tasks containing low priority fields, Replicator splits the result set of these tasks into low priority and high priority CAB files.

High priority files keep their usual names, while low priority files are identified with ~1 at the end of the file name.

Example: low priority 35~1~40~1234~1.cab                 high priority: 35~1~40~1234.cab

Send Phase

Replicator measures the bandwidth and response time. Based on the result of these measurements and the options set, Replicator start sending the CAB files. High priority files are primarily sent, followed by low priority based on the results of the bandwidth and response measurements. High priority files are always prioritized over low priority files even if they have been created in the previous replicator session.

Fields added to Database

Below, you can find the fields that Replicator automatically adds to the database:

Replicating Large Files

CAB files can become very large when exporting a database and all scanned documents. To avoid using large files, Replicator allows splitting the files into multiple CAB files. To do this, add the TOP (x) statement to the replication task select statement.

Let us consider the example of the replication task exporting the scanned documents PW001OLEDOCS:

SELECT TOP(1000)  * FROM dbo.PW001OLEDOCS WHERE repl_ModifiedDate >= #LR# AND repl_ModifiedBySite <> #DSN#

As a result, the CAB files will only contain 1000 PW001OLEDOCS records in each cab file. If PW001OLEDOCS contains 4500 records, Replicator will create 5 CAB files instead of 1, each containing 1000 PW001OLEDOCS records except for the last cab file that contains only 500 records.