Edit Host Configuration

This dialog is used to configure the destination host. It will also allow one to change the ordering of the host names in the afd_ctrl dialog by simply dragging and dropping a selected host name. It is only possible to have one of this dialog open at the same time.

[labeled picture of edit_hc dialog]
 1  Host Switching When the toggle button for Host switching is turned on (inserted) it is possible to have two addresses for a single host. The second host is always regarded as the secondary host. The user may can change to the secondary host or main host by pressing the Switch host button. In the Host 1 and Host 2 fields characters can be entered that distinguish the two hosts from each other. The character will be appended to the end of the alias host name.
 2  Host/IP Here you may specify the real host name or IP number. This has the advantage that you can give remote hosts names that are not in the local /etc/hosts file or the name server. If a secondary host is available, the second host field will be active. When the toggle button at the bottom of this boxed is set and the active real hostname matches that of the current (ie. the host where AFD is running) or if it matches any of the entries in $AFD_WORK_DIR/etc/local_interface.list it will use scheme 'file' to deliver the data.
 3  Proxy Name If the remote server is a proxy, enter the sequence on how to enter the proxy here. Currently you can only specify the order of the user name and the password. To enter the user name just enter $U<user name> and the password is entered with $P<password>. You may specify as many user and password elements as you want, each one must however be separated by a ;. If you only put a $U followed directly by a ; it will take the user name from the recipient entry. The same goes for the password. If the remote server requires the ACCT (account) directive instead of the USER to enter the login name you may also use $A instead of $U.
 4  Alias Hostname In this list widget, all host alias names found in the DIR_CONFIG are listed in the order that they are found there. In order to make a change for any host it must first be selected by clicking on it. By dragging the host name with the middle mouse button, one can change the ordering in the afd_ctrl dialog. When dragging the cursor will turn into a [image of a bee], the feelers at the top left are the hot spot of the cursor. The changes will only become effective when the update button was pressed.
 5  Text Fields
Transfer timeout The time how long the AFD should wait for a reply from the remote site. If one sets the Interrupt button it will cancel the transmission of a single file after the given timeout.
Ignore errors+warnings If set, errors and warnings for this host will be set to offline. If the value of 'Maximum errors' is reached the host will turn 'blue'.
Do not delete data When set no data will be deleted even when option age-limit is set or directory option delete queued files is set. This can be useful when the target host is taken down for maintenance and you do not want to loose any data.
Maximum errors If max. errors is reached the destination identifier turns 'red'. If error retries reaches twice max. errors the queue of this host will be paused.
Successful retries This is only used when there is a secondary host and automatic switch over is active. It is the number of successful transfers to the secondary host, before it tries to switch back to the main host to see if it is alive again. If this is set to 0 it will NOT automatically switch back to the original host after a failover.
Retry interval If an error occurs, this is the delay (in seconds) before another transfer is initiated.
Keep connected The number of seconds that the connection should be kept open for retrieving and sending files, when the toggle button 'Both' is set. If the toggle button 'Fetch' is set, this value is only set when retrieving files. When 'Send' is set the value is only valid for sending files.
Warn time When the given time has elapsed with no data being send to this host, the script/program in AFD_WORK_DIR/etc/action/target/warn/ with the <Alias hostname> as filename is executed with the parameter 'start'. As soon as data has been send successful the script/program is called again with the parameter 'stop'.
 6  Transfer rate limit With this option the transfer rate to the given host can be limited. The Value is in Kilobytes per second. See also group.transfer_rate_limit file, if you want to group several hosts to one limit.
 7  Socket send buffer size Changes the socket send buffer size. This can improve throughput on high bandwidth connections or when the RTT (round-trip time) gets large. Ensure that you understand how to determine this value otherwise you will be wasting memory and can increase latency with small files.
 8  Socket receive buffer size Changes the socket receive buffer size. See description above.
 9  Check for duplicates Performs a duplicate on files that are send to this host. The options have the following meaning:
Option Description
Timeout Time in seconds when this CRC value is to be discarded from the database.
Reference If Alias (default) is set, only the host alias is taken as reference when checking if the data has already been transmitted. If instead Recipient is used, the full recipient is taken as reference (eg. ftp://donald@ducktown/data). This is useful when you wish to send the same data to more then one user and/or directory on the same host.
Check Type What type of check is to be performed, the following values are possible:
Name Perform check only on the filename.
Name+size Perform check on the filename and the size of the file.
Name no suffix Perform check only on the filename without last suffix.
Content Creates the checksum on the file content only.
Name+content Creates checksum from file name and content.
Action What action is to be taken when we find a duplicate. The following values are possible:
Delete Delete the file.
Store Store the duplicate file in the following directory: $AFD_WORK_DIR/files/store/<id>. Where <id> is the directory id.
Warn Only warn in SYSTEM_LOG.
CRC type CRC-32 is a simple software CRC algorithm that has a reasonable performance. CRC32c computes better checksums at the cost of slightly higher memory usage but computes twice as fast as the first algorithm CRC-32. If you CPU supports SSE4.2, this calculation is done hardware and approximatly six time faster then CRC-32.
 10  Options
Max. parallel transfers The maximum number of parallel transfers for this host.
Transfer Blocksize The block size being used to send files to the given host.
File size offset When transmitting large files and the transfer gets interrupted, the AFD can append a file on the remote site. For this it needs to know the file size on the remote site. And to get the size it does a dir 'filename' at the remote site. Due to different replies of the FTP servers, the position of the file size is needed. You can easily determine this value simply doing an FTP to the remote site and a dir and count the spaces to the file size. For example: 
             -rw-r--r--   1 afd      mts-soft   14971 Jan  3 17:16
                       ^^^ ^   ^^^^^^        ^^^
                        |  |     |            |
                        |  |     |            |
                        1  2     3            4
i.e. the file size offset is 4.
 11  Protocol specific Protocol specific options. Currently these are only for FTP and FTPS, except for Fast cd which can also be used for SFTP.
FTP Mode The mode to use when transferring or receiving files. Can be either active, passive, extended active or extended passive. If you experience problems with the data connection you might try passive, since firewalls only allow passive connections. When using passive (NOT extended passive) it is possible to enable redirecting of the data part to another system. This is however very uncommon and should best be left disabled.
Use LIST When AFD gets a an FTP directory listing it tries to get the date and size by via the MLST command and if that is not supported gets the information by sending the MDTM and SIZE command for each file in the listing. On connection with a high latency this can take a long time. In this case you can set this option and AFD will use the LIST command (ls -al) and try determine from this output the date and size of each file.
Clear Control Connection With FTPS it is always a problem to get it running behind a firewall, because the firewall cannot read which data port is going to be opened for transmitting the data, since the control connection is encrypted. When this option is set AFD will drop encryption by sending the CCC comand, after having logged in successfully.
Set idle time The idle time will be set to the value that is set in the Transfer timeout field. Note not all FTP servers support this.
STAT Keepalive Some firewalls timeout the control connection when there is no activity on it. This can happen transferring large files over a slow connection. This option, if set, will send keepalive commands over the control connection during transfer. Most FTP servers do not support this (only vsftpd does support indirectly).
Fast rename This will send the RNFR (rename from) and RNTO (rename to) commands in one go, not as two separate commands. It reduces latency of renaming, but note that some FTP-servers do not support this, especially those from M$soft.
Fast cd If set no CWD (change working directory) command is sent. This reduces latency, but does not always work (eg. VAX systems).
Ignore type I By default AFD will always send a type I command for binary mode during connection initialization. Since some FTP-servers have this set by default, it is not required to send this. Note however that this option is very dangerous and only works with few FTP-servers (vsftpd), it can easily corrupt data, because some FTP-servers use ASCII mode as default.
Allow burst AFD by default tries to append a new job to an existing sending job, to reduce the number of connections and the overall latency. It can however happen that some servers have problems with this. So with this option it is possible to disable bursting.
TCP Keepalive Sets the keepalive flag for the TCP connection. When the keepalive option is set and no data has been exchanged in either direction for 2 hours, TCP automatically sends a keepalive probe. If the system does support the socket() option TCP_KEEPALIVE, then the timeout value is set to that value entered in the 'Tansfer timeout' field minus 5 seconds and not the default 2 hours.
Seq. Locking When transmitting file "xxxx.yyyy" it adds the number of retries to the end of the file name "xxxx.yyyy-" during transmission. After the file was transmitted successful it is renamed to the original name "xxxx.yyyy". Note, when set appending will NOT be possible. This option is useful when the remote side has problems with still having this file open by some other process, since with every try it will open a new file with a different name. It will try to delete the previous before it opens a new one. This option can be combined with the other locking options.
Compression If the protocol does support compression enable this to reduce the volume send to this host. This does increase the CPU time used and not all data does compress well. This can currently only be enabled for protocol SCP and SFTP.
Keep time stamp This tries to preserve the original time stamp of the file being transmitted. Currently this only works with FTP, FILE and SFTP.
Sort file names Files are being sorted by their name before being send.
No ageing jobs When during tranmission an error occurs, the job with this error starts to age. This means its priority decreases so much that even jobs that have a lower priority become more important and can bypass an error job. If you set this button it will no longer do this.
Match size After file is transmitted check the size of the destination file with that of the local file just send. If they do not match, show an error and then resend the file.
 12  Status Line Reports what edit_hc did when the user presses the update button.
 13  Button Line Pressing the update button will activate all changes made. You may change several hosts before pressing the update button, the dialog will remember all the changes. With the remove button a host can be removed, but only if it is not in the DIR_CONFIG.
Copyright © 1997 - 2014 by H.Kiehl
Holger.Kiehl@dwd.de
Last updated: 01.12.2014
[red dot]Index [red dot]Home