Each XSEDE SP should be logging all transfer activity, and uploading all XSEDE-related transfer activity to the central GridFTP logging infrastructure (currently hosted at TACC). There are three steps to set up the automated upload of logs to the upload location: Setting up an account with access to the GridFTP logs and the upload location, setting up log rotation for your GridFTP transfer logs (not mandatory but desirable for ease of management), and setting up daily log uploads.
Account access to upload logs
You will need an account with access to the GridFTP log files which will be used for uploading these logs to the log collection server. The easiest way to do this is simply to configure your "globus" user (sometimes called "globust" or "gridftp") to perform the uploads, but you may choose to use a Unix Group setup or some other mechanism to allow a specific user account to access the logs for upload. This is at the discretion of the SP. Once you have selected a user account, you will need to create an SSH RSA keypair for configuring authentication to the upload server from this account. This key can be created using the command:
If you intend to use this key for automated uploads, leave the passphrase empty. Once you have created the keypair, the public key will be stored in ~/.ssh/id_rsa.pub ; copy the contents of this file into e-mail and send it to email@example.com to be granted access to the upload server. When you have received confirmation you may proceed with testing uploads to the central logging server.
Setting up Log Rotation (optional)
Most systems will use the "logrotate" software tool to manage the rotation of system logs, and this tool can easily be extended to support rotation of the GridFTP transfer logs. You will need to know the location and name of these transfer logs, and you will need to decide on the frequency of log rotation. You may need to consult your system-specific documentation for specifics on how to configure certain options for log rotation. On CentOS/RedHat-based systems, you can create a file named /etc/logrotate.d/gridftp with the following contents to configure daily log rotation (this example assumes you are following the examples from the XSEDE GridFTP installation guide. Replace /var/log/gridftp/ with the actual path to your GridFTP transfer logs if you are using a different location):
Setting up Daily log uploads
Regardless of whether you are using log rotation, SPs should upload logs for each of their GridFTP server instances on a daily basis to ensure that the XSEDE centralized transfer information service has fresh data. Assuming that your server is configured to rotate logs on a daily basis, you can use the following script in combination with cron to accomplish this daily upload. You will need to modify the script to set "SITENAME" which you will be given when you exchange your SSH key with Siva, and to ensure that you are selecting the right log location from which to copy files.
An example crontab entry to execute this script shortly after midnight each day would look like the following: