The dsync function (Bit Level Replication) of File Replication Pro (FRP) is the means by which FRP uses CPU and Memory resources to parse a file on the source and destination servers to extract the changed bits (delta) of the file and replicate those changed bit saving bandwidth and time (in the case of larger files) rather than replicating the entire file.
The steps for dsync are as follows.
On the source file server:
If the time date stamp of the file has changed.....
Parse the file for changed bits since the last replication. (This operation requires more CPU and Memory which could be wasteful if applied to small files <10mb).
Create a temp file to hold the extracted bits with the filename prefix of $FRPdsync_ (This temporarily requires drive space equal to the size of the changes extracted.)
Replicate the $FRPdsync_myfilename.ext file to the other server. (This temp file can be only a few % the size of the original file saving time and bandwidth)
After the temp file is sucessufly replicated, it is deleted on the source server.
And example end of job notation of the dsync function is shown below.
** FRP efficiencies including Bit Level saved you 24514958384 Bytes of transfer (92.31781%) **
On the destination server:
The file with a $FRPdsync_myfilename.ext filename prefix is received.
FRP parses the destination file myfilename.ext for changes and then merges the $FRPdsync_myfilename.ext file into the myfilename.ext file.
The temp file $FRPdsync_myfilename.ext is deleted.
Please note that the process of dysnc requires as much in the way of CPU and Memory resources on the destination server at it did on the source server.
--- End of dsync process -----
Adjustments which can be made to dsync
CAUTION: Be very careful with theses adjustments. Causing FRP to dsync files less than 10mb in size can be very resource intensive if there are thousands or more files of this size to be examined. By default FRP will only dsync up to 20 files at a time in parallel and those files must have changed time stamps as well as the file must be larger than 10mb. Incorrect settings of dsync values can actually slow the replication process while consuming more server resources.
It is possible to change the size of files considered for dsync transfer by going to "Advanced Server Properties" of the source/initiator server in the GUI. Each server is configured separately from the others so if you want to change all of your servers you must do each one.
The values for dsync are set in bytes. The defalts are:
Minimum file size 10000000 bytes = 9.53674316 megabytes
Maximum file size 10000000000 bytes = 9.31322575 gigabytes
If desired or needed it is possible to disable dsync transfer by setting both values related to dsync transfer (max and min file sizes) to 1
dsync errors
Normal recovery in an error condition - If the connection fails, the server reboots or has some event, or if FRP becomes starved for memory (see setting Java Heap memory for FRP) during the dsync process it can fail dsync but then reverts automatically to normal transfer so long as the file is unlocked. The event will be logged as an error but will note in the next log entries that it reverted to normal transfer and completed the replication.
If the dsync process is interrupted by something it may result in orphaned temp files on the source and or destination servers. These files will still have the dsync filename prefix on them. In that case you may find files with the names $FRPdsync_myfilename.ext that remain on the drive for more than an hours or two. These orphaned files can be safely deleted using a file browser or by a command line delete command. In some severe cases of may orphaned files it might be necessary for you to create a delete script that is recursive and will run the path of the replication to delete all $FRPdsync_myfilename.ext files it finds.
Corrective Actions
Ensure good connectivity between the servers.
Troubleshoot for problems on the server that cause reboots of the server or stoppage of the FRP service.
Ensure that no other process is attempting to lock any part of the FRP installation or the $FRPdsync_ files.
Increase the Java Heap memory available to FRP for running resource intensive operations.
Turn off resource intensive options in FRP such as Compression and Encryption.
Decrease the number of parallel operations in FRP to conserve resources (be careful this will also reduce replication speed).