TRIUMF Grid Software and Applications

Installation of SC Tape System

Here are some rough guidelines on setting up the SC Tape software system from scratch on any node with a SCSI connector.

Hot-plugging SCSI

On boot-up a system tells you about its SCSI subsystems.  For a SCSI tape subsystem the node typically needs to be explicitly told to recognize tape drives on boot-up.  Here we document hot-plugging our SCSI tape subsystem into a running node; i.e. how to probe loadably modules to get a system to recognize SCSI devices without rebooting.
  • Find out which SCSI Host numbers have been claimed by the kernel.  This info can be pieced together from /var/log/dmesg and /proc/scsi/scsi.  Here we see that the Adaptec controller to which the library is attached is scsi4:
    # grep -iE "scsi. :" /var/log/dmesg 
    scsi0 : sata_sil
    scsi1 : sata_sil
    scsi2 : sata_sil
    scsi3 : sata_sil
    scsi4 : Adaptec AIC7XXX EISA/VLB/PCI SCSI HBA DRIVER, Rev 6.2.36
    scsi5 : 3ware 9000 Storage Controller
  • li>See if the tape library is already recognized.  In this case it is not loaded, since it is not seen in /proc/scsi/scsi.  Therefore we hot-plug it.  We now know the Host Number (scsi4).  As well, you need to know the SCSI ID of the tape library and the tape drive(s), and that these addresses do not conflict with any other attached device.  The SCSI IDs of the library and the drives can be configured on the library LCD display, or over the web.  Below I show 'hot-plugging' the tape library (scsi id 3) and a tape drive (scsi id 1).  The fields echo-ed into /proc/scsi/scsi are Host, Channel, ID and Lun (assume zero for Channel and Lun):
    # cat /proc/scsi/scsi 
    Attached devices:
    Host: scsi0 Channel: 00 Id: 00 Lun: 00
      Vendor: ATA      Model: WDC WD800JD-75JN Rev: 05.0
      Type:   Direct-Access                    ANSI SCSI revision: 05
    Host: scsi5 Channel: 00 Id: 00 Lun: 00
      Vendor: AMCC     Model: 9500S-4LP  DISK  Rev: 2.06
      Type:   Direct-Access                    ANSI SCSI revision: 03
    # echo "scsi add-single-device 4 0 3 0" >/proc/scsi/scsi
    # echo "scsi add-single-device 4 0 1 0" >/proc/scsi/scsi
    # cat /proc/scsi/scsi
    Attached devices:
    Host: scsi0 Channel: 00 Id: 00 Lun: 00
    Vendor: ATA Model: WDC WD800JD-75JN Rev: 05.0
    Type: Direct-Access ANSI SCSI revision: 05
    Host: scsi5 Channel: 00 Id: 00 Lun: 00
    Vendor: AMCC Model: 9500S-4LP DISK Rev: 2.06
    Type: Direct-Access ANSI SCSI revision: 03
    Host: scsi4 Channel: 00 Id: 03 Lun: 00
    Vendor: IBM Model: 4560SLX Rev: 0416
    Type: Medium Changer ANSI SCSI revision: 02
    Host: scsi4 Channel: 00 Id: 01 Lun: 00
    Vendor: QUANTUM Model: SDLT320 Rev: 3838
    Type: Sequential-Access ANSI SCSI revision: 0
  • Figure out which devices match to the changer and to the tape drive(s).   It is best to use logical names for devices and document them.  In the case of logical devices you will need to make sure that you set correct symbolic links.  Also, give ownership of these devices to the user ID of the tape library owner.  In the example below we see that the changer is device /dev/sg2 and the tape drive is device /dev/st0:
    # grep scsi4 /var/log/messages
    Nov 21 12:25:29 sc4 kernel: (scsi4:A:3): 20.000MB/s transfers ...
    Nov 21 12:25:29 sc4 kernel: Attached scsi generic sg2 at scsi4, channel 0, id 3, lun 0...
    (scsi4:A:3): 20.000MB/s transfers (10.000MHz, offset 15, 16bit)
    Attached scsi generic sg2 at scsi4, channel 0, id 3, lun 0, type 8
    Nov 21 12:30:24 sc4 kernel: (scsi4:A:1): 80.000MB/s transfers ...
    Nov 21 12:30:24 sc4 kernel: Attached scsi tape st0 at scsi4, channel 0, id 1, lun 0
    Nov 21 12:30:24 sc4 kernel: Attached scsi generic sg3 at scsi4, channel 0, id 1, lun 0...
    (scsi4:A:1): 80.000MB/s transfers (40.000MHz, offset 62, 16bit)
    Attached scsi tape st0 at scsi4, channel 0, id 1, lun 0

    # ln -s /dev/nst0 /dev/tape
    # ln -s /dev/sg2 /dev/changer
    # chown trteam /dev/nst0 /dev/sg2

Setting up the mysql tables

  • Then verify that these same device names are referenced in the mysql database.  Update the library table so that it conforms with your device assignment.  Also make sure that there is one row in the operations table for each tape device, and that it conforms with the library name in the library table.
    mysql> select name, hostname, ndrives, status, changerdev,tapedev0 from library;
    +-----------------+------------------+---------+---------+--------------+-----------+
    | name | hostname | ndrives | status | changerdev | tapedev0 |
    +-----------------+------------------+---------+---------+--------------+-----------+
    | 4560SLX-1 | fts.triumf.ca | 1 | online | /dev/changer | /dev/tape |
    | 4560SLX-2 | sc4.triumf.ca | 1 | online | /dev/changer | /dev/tape |
    | OVERLAND neo | tw00.triumf.ca | 1 | offline | /dev/sg1 | /dev/nst0 |
    | OVERLAND amanda | amanda.triumf.ca | 1 | offline | /dev/sg1 | /dev/nst0 |
    +-----------------+------------------+---------+---------+--------------+-----------+
    4 rows in set (0.00 sec)

    mysql> select num,libname from operations;
    +-----+-----------------+
    | num | libname |
    +-----+-----------------+
    | 0 | 4560SLX-1 |
    | 1 | 4560SLX-2 |
    | 2 | OVERLAND neo |
    | 3 | OVERLAND amanda |
    +-----+-----------------+
    4 rows in set (0.00 sec)

Enable Tape Devices from Boot-up

Add these few lines to /etc/rc.local so that the tape devices are available after a reboot.  In case you are also starting Tape Library processes from boot-up which depend on these devices then these lines need to go into the Tape Library startup script instead. Note that /etc/rc.local is the last process run at startup.  An alternative idea is to run a cronjob that checks for the Tape Library process running periodically, and then restarts it if it is not running.
  • # tail /etc/rc.local
    modprobe sg
    modprobe st
    rm -f /dev/tape /dev/changer
    ln -s /dev/sg2 /dev/changer
    ln -s /dev/nst0 /dev/tape
    chown trteam /dev/sg2 /dev/nst0
Of course if the scsi bus is reconfigured (for example, adding a new external disk drive to the scsi chain) then you need to verify that the device paths are all correct after any such change.