ceph/src/spdk/test/vhost/hotplug/test_plan.md

   1 #Vhost hotattach and hotdetach test plan
   2
   3 ## Objective
   4 The purpose of these tests is to verify that SPDK vhost remains stable during
   5 hot-attach and hot-detach operations performed on SCSI controllers devices.
   6 Hot-attach is a scenario where a device is added to controller already in use by
   7 guest VM, while in hot-detach device is removed from controller when already in use.
   8
   9 ## Test Cases Description
  10 1. FIO I/O traffic is run during hot-attach and detach operations.
  11 By default FIO uses default_integrity*.job config files located in
  12 test/vhost/hotfeatures/fio_jobs directory.
  13 2. FIO mode of operation in random write (randwrite) with verification enabled
  14 which results in also performing read operations.
  15 3. Test case descriptions below contain manual steps for testing.
  16 Automated tests are located in test/vhost/hotfeatures.
  17
  18 ### Hotattach, Hotdetach Test Cases prerequisites
  19 1. Run vhost with 8 empty controllers. Prepare 16 nvme disks.
  20 If you don't have 16 disks use split.
  21 2. In test cases fio status is checked after every run if there are any errors.
  22
  23 ### Hotattach Test Cases prerequisites
  24 1. Run vms, first with ctrlr-1 and ctrlr-2 and second one with ctrlr-3 and ctrlr-4.
  25
  26 ## Test Case 1
  27 1. Attach NVMe to Ctrlr 1
  28 2. Run fio integrity on attached device
  29
  30 ## Test Case 2
  31 1. Run fio integrity on attached device from test case 1
  32 2. During fio attach another NVMe to Ctrlr 1
  33 3. Run fio integrity on both devices
  34
  35 ## Test Case 3
  36 1. Run fio integrity on attached devices from previous test cases
  37 2. During fio attach NVMe to Ctrl2
  38 3. Run fio integrity on all devices
  39
  40 ## Test Case 4
  41 2. Run fio integrity on attached device from previous test cases
  42 3. During fio attach NVMe to Ctrl3/VM2
  43 4. Run fio integrity on all devices
  44 5. Reboot VMs
  45 6. Run fio integrity again on all devices
  46
  47
  48 ### Hotdetach Test Cases prerequisites
  49 1. Run vms, first with ctrlr-5 and ctrlr-6 and second with ctrlr-7 and ctrlr-8.
  50
  51 ## Test Case 1
  52 1. Run fio on all devices
  53 2. Detatch NVMe from Ctrl5 during fio
  54 3. Check vhost or VMs did not crash
  55 4. Check that detatched device is gone from VM
  56 5. Check that fio job run on detached device stopped and failed
  57
  58 ## Test Case 2
  59 1. Attach NVMe to Ctrlr 5
  60 2. Run fio on 1 device from Ctrl 5
  61 3. Detatch NVMe from Ctrl5 during fio traffic
  62 4. Check vhost or VMs did not crash
  63 5. Check that fio job run on detached device stopped and failed
  64 6. Check that detatched device is gone from VM
  65
  66 ## Test Case 3
  67 1. Attach NVMe to Ctrlr 5
  68 2. Run fio with integrity on all devices, except one
  69 3. Detatch NVMe without traffic during fio running on other devices
  70 4. Check vhost or VMs did not crash
  71 5. Check that fio jobs did not fail
  72 6. Check that detatched device is gone from VM
  73
  74 ## Test Case 4
  75 1. Attach NVMe to Ctrlr 5
  76 2. Run fio on 1 device from Ctrl 5
  77 3. Run separate fio with integrity on all other devices (all VMs)
  78 4. Detatch NVMe from Ctrl1 during fio traffic
  79 5. Check vhost or VMs did not crash
  80 6. Check that fio job run on detached device stopped and failed
  81 7. Check that other fio jobs did not fail
  82 8. Check that detatched device is gone from VM
  83 9. Reboot VMs
  84 10. Check that detatched device is gone from VM
  85 11. Check that all other devices are in place
  86 12. Run fio integrity on all remianing devices