]>
Commit | Line | Data |
---|---|---|
11fdf7f2 TL |
1 | # dev environment setup |
2 | ||
3 | 1. start vms with _only_ the ceph packages installed | |
4 | ||
5 | In `src/pybind/mgr/ssh` run `vagrant up` to create a cluster with a monitor, | |
6 | manager, and osd nodes. The osd node will have two small extra disks attached. | |
7 | ||
8 | 2. generate an `ssh_config` file for the vm hosts | |
9 | ||
10 | Execute `vagrant ssh-config > /path/to/ssh_config` to generate a ssh | |
11 | configuration file that contains hosts, usernames, and keys that will be used by | |
12 | the bootstrap cluster / ssh orchestrator to establish ssh connections to the | |
13 | vagrant vms. | |
14 | ||
15 | 3. install ssh orchestrator dependencies | |
16 | ||
17 | The primary dependency is the `remoto` package that contains a Python SSH client | |
18 | for connecting to remote nodes and executing commands. | |
19 | ||
20 | Install with `dnf install python3-remoto`. The version must be >= 0.0.35. At the | |
21 | time of writing this version is being packaged and is not available. To install | |
22 | from source: | |
23 | ||
24 | ``` | |
25 | git clone https://github.com/ceph/remoto | |
26 | cd remoto | |
27 | python3 setup.py sdist | |
28 | pip3 install --prefix=/usr dist/remoto-0.0.35.tar.gz | |
29 | ``` | |
30 | ||
31 | 4. start the bootstrap cluster (in this case a `vstart.sh` cluster) | |
32 | ||
33 | Start with a network binding to which the vms can route traffic: | |
34 | ||
35 | `vstart.sh -n -i 192.168.121.1` | |
36 | ||
37 | The following is a manual method for finding this address. TODO: documenting a | |
38 | automated/deterministic method would be very helpful. | |
39 | ||
40 | First, ensure that your firewall settings permit each VM to communicate with the | |
41 | host. On Fedora, the `trusted` profile is sufficient: `firewall-cmd | |
42 | --set-default-zone trusted` and also allows traffic on Ceph ports. Then ssh into | |
43 | one of the vm nodes and ping the default gateway, which happens to be setup as | |
44 | the host machine. | |
45 | ||
46 | ``` | |
47 | [nwatkins@smash ssh]$ vagrant ssh mon0 -c "getent hosts gateway" | |
48 | 192.168.121.1 gateway | |
49 | ``` | |
50 | ||
51 | 5. setup the ssh orchestrator backend | |
52 | ||
53 | Enable and configure the ssh orchestrator as the active backend: | |
54 | ||
55 | ``` | |
56 | ceph mgr module enable ssh | |
57 | ceph orchestrator set backend ssh | |
58 | ||
59 | # optional: this document assumes the orchestrator CLI is enabled | |
60 | ceph mgr module enable orchestrator_cli | |
61 | ``` | |
62 | ||
63 | Configure the ssh orchestrator by setting the `ssh_config` option to point at | |
64 | the ssh configuration file generated above: | |
65 | ||
66 | ``` | |
67 | ceph config set mgr mgr/ssh/ssh_config_file /path/to/config | |
68 | ``` | |
69 | ||
70 | The setting can be confirmed by retrieving the configuration settings: | |
71 | ||
72 | ``` | |
73 | [nwatkins@smash build]$ ceph config get mgr. | |
74 | WHO MASK LEVEL OPTION VALUE RO | |
75 | mgr advanced mgr/orchestrator_cli/orchestrator ssh * | |
76 | mgr advanced mgr/ssh/ssh_config_file /home/nwatkins/src/ceph/src/pybind/mgr/ssh/config * | |
77 | ``` | |
78 | ||
79 | An SSH config file can also be provided through standard input that avoids the | |
80 | need to have an accessible file path. Use the following command: | |
81 | ||
82 | ||
83 | ``` | |
84 | ceph ssh set-ssh-config -i <path to ssh_config> | |
85 | ``` | |
86 | ||
87 | The next set of instructions we should move to the docs folder | |
88 | ||
89 | ceph orchestrator host add osd0 | |
90 | ceph orchestrator host add mgr0 | |
91 | ceph orchestrator host add mon0 | |
92 | ceph orchestrator device ls | |
93 | ceph orchestrator mgr update 3 mgr0 mgr1 |