ceph/src/spdk/doc/iscsi/getting_started.md

   1 # Getting Started Guide {#iscsi_getting_started}
   2
   3 The Intel(R) Storage Performance Development Kit iSCSI target application is named `iscsi_tgt`.
   4 This following section describes how to run iscsi from your cloned package.
   5
   6 # Prerequisites {#iscsi_prereqs}
   7
   8 This guide starts by assuming that you can already build the standard SPDK distribution on your
   9 platform. The SPDK iSCSI target has been known to work on several Linux distributions, namely
  10 Ubuntu 14.04, 15.04, and 15.10, Fedora 21, 22, and 23, and CentOS 7.
  11
  12 Once built, the binary will be in `app/iscsi_tgt`.
  13
  14 # Configuring iSCSI Target {#iscsi_config}
  15
  16 A `iscsi_tgt` specific configuration file is used to configure the iSCSI target. A fully documented
  17 example configuration file is located at `etc/spdk/iscsi.conf.in`.
  18
  19 The configuration file is used to configure the SPDK iSCSI target. This file defines the following:
  20 TCP ports to use as iSCSI portals; general iSCSI parameters; initiator names and addresses to allow
  21 access to iSCSI target nodes; number and types of storage backends to export over iSCSI LUNs; iSCSI
  22 target node mappings between portal groups, initiator groups, and LUNs.
  23
  24 Each LUN in an iSCSI target node is associated with an SPDK block device.  See @ref bdev_getting_started
  25 for details on configuring SPDK block devices.  The block device to LUN mappings are specified in the
  26 configuration file as:
  27
  28 ~~~~
  29 [TargetNodeX]
  30   LUN0 Malloc0
  31   LUN1 Nvme0n1
  32 ~~~~
  33
  34 This exports a malloc'd target. The disk is a RAM disk that is a chunk of memory allocated by iscsi in
  35 user space. It will use offload engine to do the copy job instead of memcpy if the system has enough DMA
  36 channels.
  37
  38 You should make a copy of the example configuration file, modify it to suit your environment, and
  39 then run the iscsi_tgt application and pass it the configuration file using the -c option. Right now,
  40 the target requires elevated privileges (root) to run.
  41
  42 ~~~
  43 app/iscsi_tgt/iscsi_tgt -c /path/to/iscsi.conf
  44 ~~~
  45
  46 # Configuring iSCSI Initiator {#iscsi_initiator}
  47
  48 The Linux initiator is open-iscsi.
  49
  50 Installing open-iscsi package
  51 Fedora:
  52 ~~~
  53 yum install -y iscsi-initiator-utils
  54 ~~~
  55
  56 Ubuntu:
  57 ~~~
  58 apt-get install -y open-iscsi
  59 ~~~
  60
  61 ## Setup
  62
  63 Edit /etc/iscsi/iscsid.conf
  64 ~~~
  65 node.session.cmds_max = 4096
  66 node.session.queue_depth = 128
  67 ~~~
  68
  69 iscsid must be restarted or receive SIGHUP for changes to take effect. To send SIGHUP, run:
  70 ~~~
  71 killall -HUP iscsid
  72 ~~~
  73
  74 Recommended changes to /etc/sysctl.conf
  75 ~~~
  76 net.ipv4.tcp_timestamps = 1
  77 net.ipv4.tcp_sack = 0
  78
  79 net.ipv4.tcp_rmem = 10000000 10000000 10000000
  80 net.ipv4.tcp_wmem = 10000000 10000000 10000000
  81 net.ipv4.tcp_mem = 10000000 10000000 10000000
  82 net.core.rmem_default = 524287
  83 net.core.wmem_default = 524287
  84 net.core.rmem_max = 524287
  85 net.core.wmem_max = 524287
  86 net.core.optmem_max = 524287
  87 net.core.netdev_max_backlog = 300000
  88 ~~~
  89
  90 ### Discovery
  91
  92 Assume target is at 192.168.1.5
  93 ~~~
  94 iscsiadm -m discovery -t sendtargets -p 192.168.1.5
  95 ~~~
  96
  97 ### Connect to target
  98
  99 ~~~
 100 iscsiadm -m node --login
 101 ~~~
 102
 103 At this point the iSCSI target should show up as SCSI disks. Check dmesg to see what
 104 they came up as.
 105
 106 ### Disconnect from target
 107
 108 ~~~
 109 iscsiadm -m node --logout
 110 ~~~
 111
 112 ### Deleting target node cache
 113
 114 ~~~
 115 iscsiadm -m node -o delete
 116 ~~~
 117
 118 This will cause the initiator to forget all previously discovered iSCSI target nodes.
 119
 120 ### Finding /dev/sdX nodes for iSCSI LUNs
 121
 122 ~~~
 123 iscsiadm -m session -P 3 | grep "Attached scsi disk" | awk '{print $4}'
 124 ~~~
 125
 126 This will show the /dev node name for each SCSI LUN in all logged in iSCSI sessions.
 127
 128 ### Tuning
 129
 130 After the targets are connected, they can be tuned. For example if /dev/sdc is
 131 an iSCSI disk then the following can be done:
 132 Set noop to scheduler
 133
 134 ~~~
 135 echo noop > /sys/block/sdc/queue/scheduler
 136 ~~~
 137
 138 Disable merging/coalescing (can be useful for precise workload measurements)
 139
 140 ~~~
 141 echo "2" > /sys/block/sdc/queue/nomerges
 142 ~~~
 143
 144 Increase requests for block queue
 145
 146 ~~~
 147 echo "1024" > /sys/block/sdc/queue/nr_requests
 148 ~~~