README

   1 KERNEL SOURCE:
   2 ==============
   3
   4 We currently use the Ubuntu kernel sources, available from our mirror:
   5
   6  https://git.proxmox.com/?p=mirror_ubuntu-kernels.git;a=summary
   7
   8 Ubuntu will maintain those kernels till:
   9
  10  https://wiki.ubuntu.com/Kernel/Dev/ExtendedStable
  11  or
  12  https://pve.proxmox.com/pve-docs/chapter-pve-faq.html#faq-support-table
  13
  14  whatever happens to be earlier.
  15
  16
  17 Additional/Updated Modules:
  18 ---------------------------
  19
  20 - include native OpenZFS filesystem kernel modules for Linux
  21
  22   * https://github.com/zfsonlinux/
  23
  24   For licensing questions, see: http://open-zfs.org/wiki/Talk:FAQ
  25
  26
  27 SUBMODULE
  28 =========
  29
  30 We track the current upstream repository as submodule. Besides obvious
  31 advantages over tracking binary tar archives this also has some implications.
  32
  33 For building the submodule directory gets copied into build/ and a few patches
  34 get applied with the `patch` tool. From a git point-of-view, the copied
  35 directory remains clean even with extra patches applied since it does not
  36 contain a .git directory, but a reference to the (still pristine) submodule:
  37
  38 $ cat build/ubuntu-kernel/.git
  39
  40 If you mistakenly cloned the upstream repo as "normal" clone (not via the
  41 submodule mechanics) this means that you have a real .git directory with its
  42 independent objects and tracking info when copying for building, thus git
  43 operates on the copied directory - and "sees" that it was dirtied by `patch`,
  44 and thus the kernel buildsystem sees this too and will add a '+' to the version
  45 as a result. This changes the output directories for modules and other build
  46 artefacts and let's then the build fail on packaging.
  47
  48 So always ensure that you really checked it out as submodule, not as full
  49 "normal" clone. You can also explicitly set the LOCALVERSION variable to
  50 undefined with: `export LOCALVERSION= but that should only be done for test
  51 builds.
  52
  53 RELATED PACKAGES:
  54 =================
  55
  56 proxmox-ve
  57 ----------
  58
  59 top level meta package, depends on current default kernel series meta package.
  60
  61 git clone git://git.proxmox.com/git/proxmox-ve.git
  62
  63 proxmox-default-kernel
  64 ----------------------
  65
  66 Depends on default kernel and header meta package, e.g., proxmox-kernel-6.2 /
  67 proxmox-headers-6.2.
  68
  69 git clone git://git.proxmox.com/git/pve-kernel-meta.git
  70
  71 proxmox-kernel-X.Y
  72 ------------------
  73
  74 Depends on the latest kernel (or header, in case of proxmox-headers-X.Y)
  75 package within a certain series.
  76
  77 e.g., proxmox-kernel-6.2 depends on proxmox-kernel-6.2.16-6-pve
  78
  79 pve-firmware
  80 ------------
  81
  82 Contains the firmware for all released PVE kernels.
  83
  84 git clone git://git.proxmox.com/git/pve-firmware.git
  85
  86
  87 NOTES:
  88 ======
  89
  90 ABI versions, package versions and package name:
  91 ------------------------------------------------
  92
  93 We follow debian's versioning w.r.t ABI changes:
  94
  95 https://kernel-team.pages.debian.net/kernel-handbook/ch-versions.html
  96 https://wiki.debian.org/DebianKernelABIChanges
  97
  98 The debian/rules file has a target comparing the build kernel's ABI against the
  99 version stored in the repository and indicates when an ABI bump is necessary.
 100 An ABI bump within one upstream version consists of incrementing the KREL
 101 variable in the Makefile, rebuilding the packages and running 'make abiupdate'
 102 (the 'abiupdate' target in 'Makefile' contains the steps for consistently
 103 updating the repository).
 104
 105 Watchdog blacklist
 106 ------------------
 107
 108 By default, all watchdog modules are black-listed because it is totally undefined
 109 which device is actually used for /dev/watchdog.
 110 We ship this list in /lib/modprobe.d/blacklist_proxmox-kernel-<VERSION>.conf
 111 The user typically edit /etc/modules to enable a specific watchdog device.
 112
 113 Debug kernel and modules
 114 ------------------------
 115
 116 In order to build a -dbgsym package containing an unstripped copy of the kernel
 117 image and modules, enable the 'pkg.proxmox-kernel.debug' build profile (e.g. by
 118 exporting DEB_BUILD_PROFILES='pkg.proxmox-kernel.debug'). The resulting package can
 119 be used together with 'crash'/'kdump-tools' to debug kernel crashes.
 120
 121 Note: the -dbgsym package is only valid for the proxmox-kernel packages produced by
 122 the same build. A kernel/module from a different build will likely not match,
 123 even if both builds are of the same kernel and package version.
 124
 125 Additional information
 126 ----------------------
 127
 128 We use the default configuration provided by Ubuntu, and apply
 129 the following modifications:
 130
 131 NOTE: For the exact and current list see debian/rules (PVE_CONFIG_OPTS)
 132
 133 - enable INTEL_MEI_WDT=m (to allow disabling via patch)
 134
 135 - disable CONFIG_SND_PCM_OSS (enabled by default in Ubuntu, not needed)
 136
 137 - switch CONFIG_TRANSPARENT_HUGEPAGE to MADVISE from ALWAYS
 138
 139 - enable CONFIG_CEPH_FS=m (request from user)
 140
 141 - enable common CONFIG_BLK_DEV_XXX to avoid hardware detection
 142   problems (udev, update-initramfs have serious problems without that)
 143
 144          CONFIG_BLK_DEV_SD=y
 145          CONFIG_BLK_DEV_SR=y
 146          CONFIG_BLK_DEV_DM=y
 147
 148 - compile NBD and RBD modules
 149          CONFIG_BLK_DEV_NBD=m
 150          CONFIG_BLK_DEV_RBD=m
 151
 152 - enable IBM JFS file system as module
 153   requested by users (bug #64)
 154
 155 - enable apple HFS and HFSPLUS as module
 156   requested by users
 157
 158 - enable CONFIG_BCACHE=m (requested by user)
 159
 160 - enable CONFIG_BRIDGE=y
 161   to avoid warnings on boot, e.g. that net.bridge.bridge-nf-call-iptables is an unknown key
 162
 163 - enable CONFIG_DEFAULT_SECURITY_APPARMOR
 164   We need this for lxc
 165
 166 - set CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE=y
 167   because if not set, it can give some dynamic memory or cpu frequencies
 168   change, and vms can crash (mainly windows guest).
 169   see http://forum.proxmox.com/threads/18238-Windows-7-x64-VMs-crashing-randomly-during-process-termination?p=93273#post93273
 170
 171 - use 'deadline' as default scheduler
 172   This is the suggested setting for KVM. We also measure bad fsync performance with ext4 and cfq.
 173
 174 - disable CONFIG_INPUT_EVBUG
 175   Module evbug is not blacklisted on debian, so we simply disable it to avoid
 176   key-event logs (which is a big security problem)
 177
 178 - enable CONFIG_MODVERSIONS (needed for ABI tracking)
 179
 180 - switch default UNWINDER to FRAME_POINTER
 181   the recently introduced ORC_UNWINDER is not 100% stable yet, especially in combination with ZFS
 182
 183 - enable CONFIG_PAGE_TABLE_ISOLATION (Meltdown mitigation)