]>
Commit | Line | Data |
---|---|---|
f67539c2 TL |
1 | ============== |
2 | RBD on Windows | |
3 | ============== | |
4 | ||
5 | The ``rbd`` command can be used to create, remove, import, export, map or | |
6 | unmap images exactly like it would on Linux. Make sure to check the | |
7 | `RBD basic commands`_ guide. | |
8 | ||
9 | ``librbd.dll`` is also available for applications that can natively use Ceph. | |
10 | ||
11 | Please check the `installation guide`_ to get started. | |
12 | ||
13 | Windows service | |
14 | =============== | |
15 | On Windows, ``rbd-wnbd`` daemons are managed by a centralized service. This allows | |
16 | decoupling the daemons from the Windows session from which they originate. At | |
17 | the same time, the service is responsible of recreating persistent mappings, | |
18 | usually when the host boots. | |
19 | ||
20 | Note that only one such service may run per host. | |
21 | ||
22 | By default, all image mappings are persistent. Non-persistent mappings can be | |
23 | requested using the ``-onon-persistent`` ``rbd`` flag. | |
24 | ||
25 | Persistent mappings are recreated when the service starts, unless explicitly | |
26 | unmapped. The service disconnects the mappings when being stopped. This also | |
27 | allows adjusting the Windows service start order so that RBD images can be | |
28 | mapped before starting services that may depend on it, such as VMMS. | |
29 | ||
30 | In order to be able to reconnect the images, ``rbd-wnbd`` stores mapping | |
31 | information in the Windows registry at the following location: | |
32 | ``SYSTEM\CurrentControlSet\Services\rbd-wnbd``. | |
33 | ||
34 | The following command can be used to configure the service. Please update | |
35 | the ``rbd-wnbd.exe`` path accordingly:: | |
36 | ||
37 | New-Service -Name "ceph-rbd" ` | |
38 | -Description "Ceph RBD Mapping Service" ` | |
39 | -BinaryPathName "c:\ceph\rbd-wnbd.exe service" ` | |
40 | -StartupType Automatic | |
41 | ||
42 | Note that the Ceph MSI installer takes care of creating the ``ceph-rbd`` | |
43 | Windows service. | |
44 | ||
45 | Usage | |
46 | ===== | |
47 | ||
48 | Integration | |
49 | ----------- | |
50 | ||
51 | RBD images can be exposed to the OS and host Windows partitions or they can be | |
52 | attached to Hyper-V VMs in the same way as iSCSI disks. | |
53 | ||
54 | Starting with Openstack Wallaby, the Nova Hyper-V driver can attach RBD Cinder | |
55 | volumes to Hyper-V VMs. | |
56 | ||
57 | Mapping images | |
58 | -------------- | |
59 | ||
60 | The workflow and CLI is similar to the Linux counterpart, with a few | |
61 | notable differences: | |
62 | ||
63 | * device paths cannot be requested. The disk number and path will be picked by | |
64 | Windows. If a device path is provided by the used when mapping an image, it | |
65 | will be used as an identifier, which can also be used when unmapping the | |
66 | image. | |
67 | * the ``show`` command was added, which describes a specific mapping. | |
68 | This can be used for retrieving the disk path. | |
69 | * the ``service`` command was added, allowing ``rbd-wnbd`` to run as a Windows service. | |
20effc67 | 70 | All mappings are by default persistent, being recreated when the service |
f67539c2 TL |
71 | stops, unless explicitly unmapped. The service disconnects the mappings |
72 | when being stopped. | |
73 | * the ``list`` command also includes a ``status`` column. | |
74 | ||
75 | The purpose of the ``service`` mode is to ensure that mappings survive reboots | |
76 | and that the Windows service start order can be adjusted so that RBD images can | |
77 | be mapped before starting services that may depend on it, such as VMMS. | |
78 | ||
79 | The mapped images can either be consumed by the host directly or exposed to | |
80 | Hyper-V VMs. | |
81 | ||
82 | Hyper-V VM disks | |
83 | ---------------- | |
84 | ||
85 | The following sample imports an RBD image and boots a Hyper-V VM using it:: | |
86 | ||
87 | # Feel free to use any other image. This one is convenient to use for | |
88 | # testing purposes because it's very small (~15MB) and the login prompt | |
89 | # prints the pre-configured password. | |
90 | wget http://download.cirros-cloud.net/0.5.1/cirros-0.5.1-x86_64-disk.img ` | |
91 | -OutFile cirros-0.5.1-x86_64-disk.img | |
92 | ||
93 | # We'll need to make sure that the imported images are raw (so no qcow2 or vhdx). | |
94 | # You may get qemu-img from https://cloudbase.it/qemu-img-windows/ | |
95 | # You can add the extracted location to $env:Path or update the path accordingly. | |
96 | qemu-img convert -O raw cirros-0.5.1-x86_64-disk.img cirros-0.5.1-x86_64-disk.raw | |
97 | ||
98 | rbd import cirros-0.5.1-x86_64-disk.raw | |
99 | # Let's give it a hefty 100MB size. | |
100 | rbd resize cirros-0.5.1-x86_64-disk.raw --size=100MB | |
101 | ||
102 | rbd device map cirros-0.5.1-x86_64-disk.raw | |
103 | ||
104 | # Let's have a look at the mappings. | |
105 | rbd device list | |
106 | Get-Disk | |
107 | ||
108 | $mappingJson = rbd-wnbd show cirros-0.5.1-x86_64-disk.raw --format=json | |
109 | $mappingJson = $mappingJson | ConvertFrom-Json | |
110 | ||
111 | $diskNumber = $mappingJson.disk_number | |
112 | ||
113 | New-VM -VMName BootFromRBD -MemoryStartupBytes 512MB | |
114 | # The disk must be turned offline before it can be passed to Hyper-V VMs | |
115 | Set-Disk -Number $diskNumber -IsOffline $true | |
116 | Add-VMHardDiskDrive -VMName BootFromRBD -DiskNumber $diskNumber | |
117 | Start-VM -VMName BootFromRBD | |
118 | ||
119 | Windows partitions | |
120 | ------------------ | |
121 | ||
122 | The following sample creates an empty RBD image, attaches it to the host and | |
123 | initializes a partition:: | |
124 | ||
125 | rbd create blank_image --size=1G | |
126 | rbd device map blank_image -onon-persistent | |
127 | ||
128 | $mappingJson = rbd-wnbd show blank_image --format=json | |
129 | $mappingJson = $mappingJson | ConvertFrom-Json | |
130 | ||
131 | $diskNumber = $mappingJson.disk_number | |
132 | ||
133 | # The disk must be online before creating or accessing partitions. | |
134 | Set-Disk -Number $diskNumber -IsOffline $false | |
135 | ||
20effc67 | 136 | # Initialize the disk, partition it and create a filesystem. |
f67539c2 TL |
137 | Get-Disk -Number $diskNumber | ` |
138 | Initialize-Disk -PassThru | ` | |
139 | New-Partition -AssignDriveLetter -UseMaximumSize | ` | |
140 | Format-Volume -Force -Confirm:$false | |
141 | ||
142 | Limitations | |
143 | ----------- | |
144 | ||
20effc67 TL |
145 | CSV support |
146 | ~~~~~~~~~~~ | |
147 | ||
f67539c2 TL |
148 | At the moment, the Microsoft Failover Cluster can't use WNBD disks as |
149 | Cluster Shared Volumes (CSVs) underlying storage. The main reason is that | |
150 | ``WNBD`` and ``rbd-wnbd`` don't support the *SCSI Persistent Reservations* | |
151 | feature yet. | |
152 | ||
20effc67 TL |
153 | Hyper-V disk addressing |
154 | ~~~~~~~~~~~~~~~~~~~~~~~ | |
155 | ||
156 | .. warning:: | |
157 | Hyper-V identifies passthrough VM disks by number instead of SCSI ID, although | |
158 | the disk number can change across host reboots. This means that the VMs can end | |
159 | up using incorrect disks after rebooting the host, which is an important | |
160 | security concern. This issue also affects iSCSI and Fibre Channel disks. | |
161 | ||
162 | There are a few possible ways of avoiding this Hyper-V limitation: | |
163 | ||
164 | * use an NTFS/ReFS partition to store VHDX image files instead of directly | |
165 | attaching the RBD image. This may slightly impact the IO performance. | |
166 | * use the Hyper-V ``AutomaticStartAction`` setting to prevent the VMs from | |
167 | booting with the incorrect disks and have a script that updates VM disks | |
168 | attachments before powering them back on. The ``ElementName`` field of the | |
169 | `Msvm_StorageAllocationSettingData`_ `WMI`_ class may be used to label VM | |
170 | disk attachments. | |
171 | * use the Openstack Hyper-V driver, which automatically refreshes the VM disk | |
172 | attachments before powering them back on. | |
173 | ||
f67539c2 TL |
174 | Troubleshooting |
175 | =============== | |
176 | ||
177 | Please consult the `Windows troubleshooting`_ page. | |
178 | ||
179 | .. _Windows troubleshooting: ../../install/windows-troubleshooting | |
180 | .. _installation guide: ../../install/windows-install | |
181 | .. _RBD basic commands: ../rados-rbd-cmds | |
182 | .. _WNBD driver: https://github.com/cloudbase/wnbd | |
20effc67 TL |
183 | .. _Msvm_StorageAllocationSettingData: https://docs.microsoft.com/en-us/windows/win32/hyperv_v2/msvm-storageallocationsettingdata |
184 | .. _WMI: https://docs.microsoft.com/en-us/windows/win32/wmisdk/wmi-start-page |