Scannex ip.buffer User Manual
© UK 2007-2021 Scannex Electronics Ltd. All rights reserved worldwide.
11.4.1. Overwrite and Append
Data is only deleted from the ip.buffer when the transfer completes successfully. If the
push fails and the ip.buffer reattempts delivery, it will transfer the file again.
When using the “Overwrite” command the ip.buffer will overwrite any existing file on the
server. In this mode it is suggested to use the “%Q”, or “%6Q” option in the filename to
avoid any possible duplication of data (where the server has a partial transfer plus a
complete transfer of the same information from the ip.buffer's reattempt). For example, if
the ip.buffer begins transferring “channel1.dat.00001234” and the transfer fails, the
ip.buffer will attempt the delivery of “channel1.dat.00001234” again the next time. The 8-
digit-hex number is only incremented on a successful delivery.
When using the “
Append
” command there is no easy way to determine when there has
been a failed transfer. Unless the FTP server can “unwind” the data on a failure, you
should use the “
Overwrite
” command instead. However, it is possible to make use of the
Data Markers – the prefix and suffix – to create markers that indicate a transfer was started
and completed successfully. But your application software has to handle this separately.
See section 11.10.1 for Data Markers.
11.4.2. Tmp File & Rename mode
The “Tmp file & Rename” option provides a good 'interlock' mechanism.
This method is especially useful when your server-side data processing works
by processing the file and then deleting or archiving the file.
The ip.buffer will work like this:
•
Logs into FTP server
•
Checks whether the file already exists (using the “
MDTM
” command in FTP)
◦
If the file exists:
▪
If the target uses the
%Q
sequence the ip.buffer deletes the file
▪
else then the push is aborted (the rename would fail)
◦
If the MDTM command fails (perhaps because not supported) then the ip.buffer
continues (but the rename itself may fail)
•
Transfers the file to
f.tmp
•
When successful, the ip.buffer instructs the server to rename
f.tmp
to
filename
.
As long as the server-side mechanisms ignore any “.tmp” files then everything
will be correctly interlocked
18
.
16
Available in version 2.80+
17
Version 2.91+. When the previous push transferred the data, the server renamed the file but the
success response was lost due to network outage, etc. The ip.buffer considers this a failure as it
cannot be sure the file was renamed.
Page 98
Scannex ip.buffer User Manual
© UK 2007-2021 Scannex Electronics Ltd. All rights reserved worldwide.
11.4.1. Overwrite and Append
Data is only deleted from the ip.buffer when the transfer completes successfully. If the
push fails and the ip.buffer reattempts delivery, it will transfer the file again.
When using the “Overwrite” command the ip.buffer will overwrite any existing file on the
server. In this mode it is suggested to use the “%Q”, or “%6Q” option in the filename to
avoid any possible duplication of data (where the server has a partial transfer plus a
complete transfer of the same information from the ip.buffer's reattempt). For example, if
the ip.buffer begins transferring “channel1.dat.00001234” and the transfer fails, the
ip.buffer will attempt the delivery of “channel1.dat.00001234” again the next time. The 8-
digit-hex number is only incremented on a successful delivery.
When using the “
Append
” command there is no easy way to determine when there has
been a failed transfer. Unless the FTP server can “unwind” the data on a failure, you
should use the “
Overwrite
” command instead. However, it is possible to make use of the
Data Markers – the prefix and suffix – to create markers that indicate a transfer was started
and completed successfully. But your application software has to handle this separately.
See section 11.10.1 for Data Markers.
11.4.2. Tmp File & Rename mode
The “Tmp file & Rename” option provides a good 'interlock' mechanism.
This method is especially useful when your server-side data processing works
by processing the file and then deleting or archiving the file.
The ip.buffer will work like this:
•
Logs into FTP server
•
Checks whether the file already exists (using the “
MDTM
” command in FTP)
◦
If the file exists:
▪
If the target uses the
%Q
sequence the ip.buffer deletes the file
▪
else then the push is aborted (the rename would fail)
◦
If the MDTM command fails (perhaps because not supported) then the ip.buffer
continues (but the rename itself may fail)
•
Transfers the file to
f.tmp
•
When successful, the ip.buffer instructs the server to rename
f.tmp
to
filename
.
As long as the server-side mechanisms ignore any “.tmp” files then everything
will be correctly interlocked
18
.
16
Available in version 2.80+
17
Version 2.91+. When the previous push transferred the data, the server renamed the file but the
success response was lost due to network outage, etc. The ip.buffer considers this a failure as it
cannot be sure the file was renamed.
Page 98
