README.dev

   1 = Setup PVE Development Environment =
   2
   3 0.  Read https://pve.proxmox.com/wiki/Developer_Documentation
   4 1.  Install Debian 9 'stretch'
   5 2.  Configure the network interface(s)
   6 3.  Change the IP address of your hostname for proper name resolution
   7     in /etc/hosts
   8     Using 127.0.1.1 will not work, so change it to an IP address from your
   9     local network!
  10
  11 4:  Check that the Debian repositories are set properly.
  12     See https://wiki.debian.org/SourcesList for more information.
  13
  14 5.  Optional: Install openssh-server and connect via ssh to the host.
  15
  16     run: apt-get update && apt-get install openssh-server
  17     Connect via ssh to host and switch user to root
  18
  19 6.  Configure 'pvetest' repository in /etc/apt/sources.list.d/:
  20
  21     run: echo "deb http://download.proxmox.com/debian stretch pvetest" > /etc/apt/sources.list.d/pve-development.list
  22
  23 7.  Add the repository key:
  24
  25     run: wget -O- "http://download.proxmox.com/debian/proxmox-ve-release-5.x.gpg" | apt-key add -
  26
  27 8.  run: apt-get update && apt-get dist-upgrade
  28 9.  run: apt-get install proxmox-ve
  29 10. run: mv /etc/apt/sources.list.d/pve-enterprise.list /etc/apt/sources.list.d/pve-enterprise.list.bak
  30
  31 11. You should now have a working Proxmox VE installation.
  32     Open a browser: https://<host_IP_address>:8006 e.g. https://10.0.0.90:8006
  33
  34
  35 = Install build prerequisites for development environment =
  36
  37 12. run:
  38
  39 apt-get install build-essential git git-email debhelper \
  40 autotools-dev autogen dh-autoreconf dkms doxygen check pkg-config \
  41 groff quilt dpatch automake autoconf libtool lintian libdevel-cycle-perl \
  42 libjson-perl libcommon-sense-perl liblinux-inotify2-perl libio-stringy-perl \
  43 libstring-shellquote-perl dh-systemd rpm2cpio libsqlite3-dev sqlite3 \
  44 libglib2.0-dev librrd-dev librrds-perl rrdcached libdigest-hmac-perl \
  45 libxml-parser-perl gdb libcrypt-openssl-random-perl \
  46 libcrypt-openssl-rsa-perl libnet-ldap-perl libauthen-pam-perl \
  47 libjson-xs-perl libterm-readline-gnu-perl oathtool libmime-base32-perl \
  48 liboath0 libpci-dev texi2html libsdl1.2-dev libgnutls28-dev \
  49 libspice-protocol-dev xfslibs-dev libnuma-dev libaio-dev \
  50 pve-libspice-server-dev libusbredirparser-dev glusterfs-common \
  51 libusb-1.0-0-dev librbd-dev libpopt-dev iproute bridge-utils numactl \
  52 glusterfs-common ceph-common python-ceph libgoogle-perftools4 \
  53 libfile-chdir-perl lvm2 glusterfs-client liblockfile-simple-perl \
  54 libsystemd-dev libreadline-gplv2-dev libio-multiplex-perl \
  55 libnetfilter-log-dev libipset3 ipset socat libsasl2-dev libogg-dev \
  56 python-pyparsing libfilesys-df-perl libcrypt-ssleay-perl \
  57 libfile-readbackwards-perl libanyevent-perl libanyevent-http-perl \
  58 unzip liblocale-po-perl libfile-sync-perl cstream \
  59 lzop dtach apt-transport-https hdparm gdisk parted ttf-dejavu-core \
  60 liblzma-dev dosfstools mtools libxen-dev libfuse-dev corosync-dev \
  61 libcpg-dev libquorum-dev libcmap-dev libuuid-perl \
  62 libqb-dev libapparmor-dev docbook2x libcap-dev dh-apparmor \
  63 graphviz libseccomp-dev libglib-perl libgtk3-perl libnss3-dev libdlm-dev \
  64 libudev-dev asciidoc-dblatex source-highlight libiscsi-dev libiscsi7 \
  65 librsvg2-bin
  66
  67
  68 = Compile PVE packages from Source =
  69
  70 13: Download and install git repositories as Proxmox modules:
  71
  72     run: mkdir /root/proxmox && cd /root/proxmox
  73
  74     run: git clone git://git.proxmox.com/git/pve-common.git
  75
  76     'pve-common.git' is some kind of starting repository and needed for some
  77     other repositories as dependency.
  78     Install this to get an idea of how the installation process is working.
  79
  80     See https://git.proxmox.com/ for all available repositories.
  81
  82 14: Most packages can be installed with 'make dinstall' command.
  83     run: cd pve-common && make dinstall
  84
  85 15: Reboot the system.
  86 16. Learn to use the quilt patch scripts.
  87 17. Happy coding!
  88
  89
  90 = REST vs. SOAP =
  91
  92 We decided to change our SOAP API (1.X) and use a REST like API. The
  93 concept is described in [1] (Resource Oriented Architecture
  94 (ROA)). The main advantage is that we are able to remove a lot of code
  95 (the whole SOAP stack) to reduce software complexity.
  96
  97 We also moved away from server side content generation. Instead we use
  98 the ExtJS Rich Internet Application Framework
  99 (http://www.sencha.com).
 100
 101 That framework, like any other AJAX toolkit, can talk directly to the
 102 REST API using JSON. So we were able to remove the server side
 103 template toolkit completely.
 104
 105 = JSON and JSON Schema =
 106
 107 We use JSON as data format, because it is simple and parse-able by any
 108 web browser.
 109
 110 Additionally, we use JSON Schema [2] to formally describe our API. So
 111 we can automatically generate the whole API Documentation, and we can
 112 verify all parameters and return values.
 113
 114 A great side effect was that we are able to use JSON Schema to
 115 produce command line argument parsers automatically. In fact, the REST
 116 API and the command line tools use the same code.
 117
 118 Object linkage is done using the JSON Hyper Schema (links property).
 119
 120 A small utility called 'pvesh' exposes the whole REST API on the command
 121 line.
 122
 123 So here is a summary of the advantage:
 124
 125    - easy, human readable data format (native web browser format)
 126    - automatic parameter verification (we can also verify return values)
 127    - automatic generation of API documentation
 128    - easy way to create command line tools (using same API).
 129
 130 = API Implementation (PVE::RESTHandler) =
 131
 132 All classes exposing methods on the API use PVE::RESTHandler as base class.
 133
 134   use base qw(PVE::RESTHandler);
 135
 136 To expose methods, one needs to call register_method():
 137
 138   __PACKAGE__->register_method ($schema);
 139
 140 Where $schema is a PVE method schema as described in
 141 PVE::JSONSchema. It includes a description of parameters and return
 142 values, and a reference to the actual code
 143
 144 __PACKAGE__->register_method ({
 145     name => 'echo',
 146     path => 'echo',
 147     method => 'GET',
 148     description => "simple return value of parameter 'text'",
 149     parameters => {
 150         additionalProperties => 0,
 151         properties => {
 152             text => {
 153                  type => 'string',
 154             }
 155         },
 156     },
 157     returns => {
 158         type => 'string',
 159     },
 160     code => sub {
 161         my ($param) = @_;
 162
 163         return $param->{text};
 164     }
 165 });
 166
 167 The 'name' property is only used if you want to call the method
 168 directly from Perl. You can do that using:
 169
 170   print __PACKAGE__->echo({ text => "a test" });
 171
 172 We use Perl's AUTOLOAD feature to implement this. Note: You need to
 173 pass parameters a HASH reference.
 174
 175 There is a special helper method called cli_handler(). This is used by
 176 the CLIHandler Class for command line tools, where you want to pass
 177 arguments as array of strings. This uses Getopt::Long to parse parameters.
 178
 179 There is a second way to map names to methods - using the 'path'
 180 property.  And you can register subclasses. That way you can set up a
 181 filesystem like hierarchy to access methods.
 182
 183 Here is an example:
 184 ----------------------------
 185 package C1;
 186
 187 __PACKAGE__->register_method ({
 188     subclass => "C2",
 189     path => 'sub2',
 190 });
 191
 192
 193 __PACKAGE__->register_method ({
 194     name => 'list1',
 195     path => 'index',
 196     method => 'GET',
 197     ...
 198 });
 199
 200 package C2;
 201
 202 __PACKAGE__->register_method ({
 203     name => 'list2',
 204     path => 'index',
 205     method => 'GET',
 206     ...
 207 });
 208 -------------------------------
 209
 210 The utily method find_handler (in PVE::RESTHandler) can be use to do
 211 'path' related method lookups.
 212
 213 C1->find_handler('GET', "/index")      => C1::list1
 214 C1->find_handler('GET', "/sub2/index") => C2::list2
 215
 216 The HTTP server use the URL (a path) to find the corresponding method.
 217
 218
 219 = References =
 220
 221 [1] RESTful Web Services
 222 Web services for the real world
 223
 224 By
 225     Leonard Richardson, Sam Ruby
 226 Publisher:
 227     O'Reilly Media
 228 Released:
 229     May 2007
 230
 231 [2] JSON Schema links: http://json-schema.org/