ceph/src/pybind/mgr/ssh/README.md

   1 # dev environment setup
   2
   3 1. start vms with _only_ the ceph packages installed
   4
   5 In `src/pybind/mgr/ssh` run `vagrant up` to create a cluster with a monitor,
   6 manager, and osd nodes. The osd node will have two small extra disks attached.
   7
   8 2. generate an `ssh_config` file for the vm hosts
   9
  10 Execute `vagrant ssh-config > /path/to/ssh_config` to generate a ssh
  11 configuration file that contains hosts, usernames, and keys that will be used by
  12 the bootstrap cluster / ssh orchestrator to establish ssh connections to the
  13 vagrant vms.
  14
  15 3. install ssh orchestrator dependencies
  16
  17 The primary dependency is the `remoto` package that contains a Python SSH client
  18 for connecting to remote nodes and executing commands.
  19
  20 Install with `dnf install python3-remoto`. The version must be >= 0.0.35. At the
  21 time of writing this version is being packaged and is not available. To install
  22 from source:
  23
  24 ```
  25 git clone https://github.com/ceph/remoto
  26 cd remoto
  27 python3 setup.py sdist
  28 pip3 install --prefix=/usr dist/remoto-0.0.35.tar.gz
  29 ```
  30
  31 4. start the bootstrap cluster (in this case a `vstart.sh` cluster)
  32
  33 Start with a network binding to which the vms can route traffic:
  34
  35   `vstart.sh -n -i 192.168.121.1`
  36
  37 The following is a manual method for finding this address. TODO: documenting a
  38 automated/deterministic method would be very helpful.
  39
  40 First, ensure that your firewall settings permit each VM to communicate with the
  41 host.  On Fedora, the `trusted` profile is sufficient: `firewall-cmd
  42 --set-default-zone trusted` and also allows traffic on Ceph ports. Then ssh into
  43 one of the vm nodes and ping the default gateway, which happens to be setup as
  44 the host machine.
  45
  46 ```
  47 [nwatkins@smash ssh]$ vagrant ssh mon0 -c "getent hosts gateway"
  48 192.168.121.1   gateway
  49 ```
  50
  51 5. setup the ssh orchestrator backend
  52
  53 Enable and configure the ssh orchestrator as the active backend:
  54
  55 ```
  56 ceph mgr module enable ssh
  57 ceph orchestrator set backend ssh
  58
  59 # optional: this document assumes the orchestrator CLI is enabled
  60 ceph mgr module enable orchestrator_cli
  61 ```
  62
  63 Configure the ssh orchestrator by setting the `ssh_config` option to point at
  64 the ssh configuration file generated above:
  65
  66 ```
  67 ceph config set mgr mgr/ssh/ssh_config_file /path/to/config
  68 ```
  69
  70 The setting can be confirmed by retrieving the configuration settings:
  71
  72 ```
  73 [nwatkins@smash build]$ ceph config get mgr.
  74 WHO    MASK LEVEL    OPTION                            VALUE                                             RO
  75 mgr         advanced mgr/orchestrator_cli/orchestrator ssh                                               *
  76 mgr         advanced mgr/ssh/ssh_config_file           /home/nwatkins/src/ceph/src/pybind/mgr/ssh/config *
  77 ```
  78
  79 An SSH config file can also be provided through standard input that avoids the
  80 need to have an accessible file path. Use the following command:
  81
  82
  83 ```
  84 ceph ssh set-ssh-config -i <path to ssh_config>
  85 ```
  86
  87 The next set of instructions we should move to the docs folder
  88
  89 ceph orchestrator host add osd0
  90 ceph orchestrator host add mgr0
  91 ceph orchestrator host add mon0
  92 ceph orchestrator device ls
  93 ceph orchestrator mgr update 3 mgr0 mgr1