KERNEL SOURCE:
==============
-We currently use the Ubuntu kernel sources, available from:
+We currently use the Ubuntu kernel sources, available from our mirror:
- http://kernel.ubuntu.com/git/ubuntu/ubuntu-xenial.git/
+ https://git.proxmox.com/?p=mirror_ubuntu-kernels.git;a=summary
Ubuntu will maintain those kernels till:
https://wiki.ubuntu.com/Kernel/Dev/ExtendedStable
+ or
+ https://pve.proxmox.com/pve-docs/chapter-pve-faq.html#faq-support-table
+
+ whatever happens to be earlier.
Additional/Updated Modules:
---------------------------
-- include latest e1000e driver from intel/sourceforge
+- include native OpenZFS filesystem kernel modules for Linux
+
+ * https://github.com/zfsonlinux/
-- include latest ixgbe driver from intel/sourceforge
+ For licensing questions, see: http://open-zfs.org/wiki/Talk:FAQ
- - include latest igb driver from intel/sourceforge
-# Note: hpsa does not compile with kernel 3.19.8
-#- include latest HPSA driver (HP Smart Array)
-#
-# * http://sourceforge.net/projects/cciss/
+BUILD
+=====
-- include native OpenZFS filesystem kernel modules for Linux
+As this is packaging for the Linux kernel with some extra integrations, like
+ZFS, this repo cannot be handled like a plain Linux kernel git repository.
- * https://github.com/zfsonlinux/
+The actual Linux kernel source lives in a git submodule.
+
+For a build you should init the submodules and then handle it like most our
+Debian packaging builds. If unsure you can follow this:
+
+Installing Build-Dependencies
+-----------------------------
+
+You can either just check the package metadata template `debian/control.in`
+and install the packages listed in the `Build-Depends` section manually
+(replace `debhelper-compat` with just `debhelper`) or use a more automated way
+described below:
+
+ # install base build-dependencies and helpers
+ apt update
+ apt install devscripts
+
+ # create build-directory so that we got final packaging control files from the
+ # .in templates generated
+ make build-dir-fresh
+
+ # install build-dependencies (replace BUILD-DIR with actual one)
+ mk-build-deps -ir BUILD-DIR/debian/control
- For licensing questions, see: http://open-zfs.org/wiki/Talk:FAQ
-- include latest DRBD 9 driver, see http://drbd.linbit.com/home/what-is-drbd/
+Package Build
+-------------
+ # start the actual build
+ make deb
-FIRMWARE:
+For simple KConfig modifications you can adapt the list in `debian/rules` file.
+For quick code changes to the actual kernel code you can do them directly in
+the submodule/ubuntu-kernels directory, then re-create the build-directory, e.g.:
+
+ make clean
+ # now build again, explicitly creating the build-dir isn't required anymore
+ # after one has the build-dependencies already installed.
+ make deb
+
+
+Modify-Build-Test Cycles
+------------------------
+
+Ideally you avoid the need for doing a full package build and just directly
+build linux from the ubuntu-kernels or the mainline (stable) repo with copying
+over a build-config of a proxmox-kernel to that as .config and then using the
+`make olddefconfig` target.
+
+If you need full package builds you can try to make changes inside the
+BUILD-DIR directly and then continue build from there, e.g., using
+`dpkg-buildpackage -b -uc -us --no-pre-clean`. Depending on what stage you want
+to continue build you might need to touch, or remove some *.prepared files.
+Just check `debian/rules` for how kernel build progress is tracked by make.
+
+SUBMODULE
=========
-We create our own firmware package, which includes the firmware for
-all proxmox-ve kernels. So far this include
+We track the current upstream repository as submodule. Besides obvious
+advantages over tracking binary tar archives this also has some implications.
+
+For building the submodule directory gets copied into build/ and a few patches
+get applied with the `patch` tool. From a git point-of-view, the copied
+directory remains clean even with extra patches applied since it does not
+contain a .git directory, but a reference to the (still pristine) submodule:
+
+$ cat build/ubuntu-kernel/.git
+
+If you mistakenly cloned the upstream repo as "normal" clone (not via the
+submodule mechanics) this means that you have a real .git directory with its
+independent objects and tracking info when copying for building, thus git
+operates on the copied directory - and "sees" that it was dirtied by `patch`,
+and thus the kernel buildsystem sees this too and will add a '+' to the version
+as a result. This changes the output directories for modules and other build
+artefacts and let's then the build fail on packaging.
+
+So always ensure that you really checked it out as submodule, not as full
+"normal" clone. You can also explicitly set the LOCALVERSION variable to
+undefined with: `export LOCALVERSION= but that should only be done for test
+builds.
+
+RELATED PACKAGES:
+=================
+
+proxmox-ve
+----------
+
+top level meta package, depends on current default kernel series meta package.
+
+git clone git://git.proxmox.com/git/proxmox-ve.git
-pve-kernel-2.6.18
-pve-kernel-2.6.24
-pve-kernel-2.6.32
-pve-kernel-3.10.0
-pve-kernel-3.19.0
+proxmox-default-kernel
+----------------------
+
+Depends on default kernel and header meta package, e.g., proxmox-kernel-6.2 /
+proxmox-headers-6.2.
+
+git clone git://git.proxmox.com/git/pve-kernel-meta.git
+
+proxmox-kernel-X.Y
+------------------
+
+Depends on the latest kernel (or header, in case of proxmox-headers-X.Y)
+package within a certain series.
+
+e.g., proxmox-kernel-6.2 depends on proxmox-kernel-6.2.16-6-pve
+
+NOTE: Since Proxmox VE 8, based on Debian 12 Bookworm, the kernel ABI is bumped
+with every version bump due to module signing. Since then the meta package was
+pulled into the kernel repo, before that it lived in pve-kernel-meta.git.
-We use 'find-firmware.pl' to extract lists of required firmeware
-files. The script 'assemble-firmware.pl' is used to read those lists
-and copy the files from various source directory into a target
-directory.
+pve-firmware
+------------
-We do not include firmeware for some wireless HW when there is a
-separate debian package for that, for example:
+Contains the firmware for all released PVE kernels.
-zd1211-firmware
-atmel-firmware
-bluez-firmware
+git clone git://git.proxmox.com/git/pve-firmware.git
-PATCHES:
---------
+NOTES:
+======
- bridge-patch.diff: Avoid bridge problems with changing MAC
- see also: http://forum.openvz.org/index.php?t=msg&th=5291
+ABI versions, package versions and package name:
+------------------------------------------------
- Behaviour after 2.6.27 has changed slighly - after setting mac address
- of bridge device, then address won't change. So we could omit
- that patch, requiring to set hwaddress in /etc/network/interfaces.
+We follow debian's versioning w.r.t ABI changes:
+
+https://kernel-team.pages.debian.net/kernel-handbook/ch-versions.html
+https://wiki.debian.org/DebianKernelABIChanges
+
+The debian/rules file has a target comparing the build kernel's ABI against the
+version stored in the repository and indicates when an ABI bump is necessary.
+An ABI bump within one upstream version consists of incrementing the KREL
+variable in the Makefile, rebuilding the packages and running 'make abiupdate'
+(the 'abiupdate' target in 'Makefile' contains the steps for consistently
+updating the repository).
Watchdog blacklist
------------------
By default, all watchdog modules are black-listed because it is totally undefined
which device is actually used for /dev/watchdog.
-We ship this list in /lib/modprobe.d/blacklist_pve-kernel-<VERSION>.conf
+We ship this list in /lib/modprobe.d/blacklist_proxmox-kernel-<VERSION>.conf
The user typically edit /etc/modules to enable a specific watchdog device.
+Debug kernel and modules
+------------------------
+
+In order to build a -dbgsym package containing an unstripped copy of the kernel
+image and modules, enable the 'pkg.proxmox-kernel.debug' build profile (e.g. by
+exporting DEB_BUILD_PROFILES='pkg.proxmox-kernel.debug'). The resulting package can
+be used together with 'crash'/'kdump-tools' to debug kernel crashes.
+
+Note: the -dbgsym package is only valid for the proxmox-kernel packages produced by
+the same build. A kernel/module from a different build will likely not match,
+even if both builds are of the same kernel and package version.
+
Additional information
----------------------
We use the default configuration provided by Ubuntu, and apply
-the following modification:
+the following modifications:
+
+NOTE: For the exact and current list see debian/rules (PVE_CONFIG_OPTS)
+
+- enable INTEL_MEI_WDT=m (to allow disabling via patch)
-see Makefile (PVE_CONFIG_OPTS)
+- disable CONFIG_SND_PCM_OSS (enabled by default in Ubuntu, not needed)
+
+- switch CONFIG_TRANSPARENT_HUGEPAGE to MADVISE from ALWAYS
- enable CONFIG_CEPH_FS=m (request from user)
- enable common CONFIG_BLK_DEV_XXX to avoid hardware detection
- problems (udev, undate-initramfs have serious problems without that)
+ problems (udev, update-initramfs have serious problems without that)
CONFIG_BLK_DEV_SD=y
CONFIG_BLK_DEV_SR=y
CONFIG_BLK_DEV_DM=y
-- add workaround for Debian bug #807000 (see
- https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=807000)
-
- CONFIG_BLK_DEV_NVME=y
-
- compile NBD and RBD modules
CONFIG_BLK_DEV_NBD=m
CONFIG_BLK_DEV_RBD=m
-- set LOOP_MIN_COUNT to 8 (debian defaults)
- CONFIG_BLK_DEV_LOOP_MIN_COUNT=8
-
-- disable module signatures (CONFIG_MODULE_SIG)
-
-- enable IBM JFS file system
+- enable IBM JFS file system as module
+ requested by users (bug #64)
- This is disabled in RHEL kernel for no real reason, so we enable
- it as requested by users (bug #64)
-
-- enable apple HFS and HFSPLUS
-
- This is disabled in RHEL kernel for no real reason, so we enable
- it as requested by users
+- enable apple HFS and HFSPLUS as module
+ requested by users
- enable CONFIG_BCACHE=m (requested by user)
- enable CONFIG_BRIDGE=y
-
- Else we get warnings on boot, that
- net.bridge.bridge-nf-call-iptables is an unknown key
+ to avoid warnings on boot, e.g. that net.bridge.bridge-nf-call-iptables is an unknown key
- enable CONFIG_DEFAULT_SECURITY_APPARMOR
-
We need this for lxc
-
-- set CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE=y
+- set CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE=y
because if not set, it can give some dynamic memory or cpu frequencies
change, and vms can crash (mainly windows guest).
-
see http://forum.proxmox.com/threads/18238-Windows-7-x64-VMs-crashing-randomly-during-process-termination?p=93273#post93273
- use 'deadline' as default scheduler
-
- This is the suggested setting for KVM. We also measure bad fsync
- performance with ext4 and cfq.
+ This is the suggested setting for KVM. We also measure bad fsync performance with ext4 and cfq.
- disable CONFIG_INPUT_EVBUG
+ Module evbug is not blacklisted on debian, so we simply disable it to avoid
+ key-event logs (which is a big security problem)
- Module evbug is not blacklisted on debian, so we simply disable it
- to avoid key-event logs (which is a big security problem)
-
-Testing final kernel with kvm
------------------------------
+- enable CONFIG_MODVERSIONS (needed for ABI tracking)
-kvm -kernel data/boot/vmlinuz-3.19.8-1-pve -initrd initrd.img-3.19.8-1-pve -append "vga=791 video=vesafb:ywrap,mtrr" /dev/zero
+- switch default UNWINDER to FRAME_POINTER
+ the recently introduced ORC_UNWINDER is not 100% stable yet, especially in combination with ZFS
+- enable CONFIG_PAGE_TABLE_ISOLATION (Meltdown mitigation)
projects / pve-kernel.git / blobdiff