Status
~~~~~~
-[thumbnail="pmg-gui-server-status.png", big=1]
+[thumbnail="screenshot/pmg-gui-server-status.png", big=1]
This page shows statistics about server CPU, memory, disk and network
usage. You can select the displayed time span from the upper right.
Services
~~~~~~~~
-[thumbnail="pmg-gui-service-status.png", big=1]
+[thumbnail="screenshot/pmg-gui-service-status.png", big=1]
This panel lists all the major services used for mail processing and
cluster synchronization. If necessary, you can start, stop or restart
-----
+[[pmg_updates]]
Updates
~~~~~~~
-[thumbnail="pmg-gui-updates.png", big=1]
+[thumbnail="screenshot/pmg-gui-updates.png", big=1]
We release software updates on a regular basis, and it is recommended
to always run the latest available version. This page shows the
Syslog and Tasks
~~~~~~~~~~~~~~~~
-[thumbnail="pmg-gui-syslog.png", big=1]
+[thumbnail="screenshot/pmg-gui-syslog.png", big=1]
The Syslog page gives you a quick real-time log view. You can use the
xref:pmg_tracking_center[Tracking Center] to search the logs.
Spam
~~~~
-[thumbnail="pmg-gui-spam-quarantine.png", big=1]
+[thumbnail="screenshot/pmg-gui-spam-quarantine.png", big=1]
This panel lets you inspect the mail quarantine. Emails can be safely
previewed and if desired, delivered to the original user.
Tracking Center
---------------
-[thumbnail="pmg-gui-tracking-center.png", big=1]
+[thumbnail="screenshot/pmg-gui-tracking-center.png", big=1]
Email processing is a complex task and involves several service
daemons. Each daemon logs information to the syslog service. The
Postfix Queue Administration
----------------------------
-[thumbnail="pmg-gui-queue-admin-summary.png", big=1]
+[thumbnail="screenshot/pmg-gui-queue-admin-summary.png", big=1]
Mail-queues are one of the central concepts of the SMTP protocol. Once a
mail server accepts a mail for further processing it saves it to a queue.
Deferred Mail
~~~~~~~~~~~~~
-[thumbnail="pmg-gui-queue-admin-deferred.png"]
+[thumbnail="screenshot/pmg-gui-queue-admin-deferred.png"]
In the 'Deferred Mail' tab, you can examine each deferred email
separately. In addition to providing contact information about the
queued emails.
Here, you can also flush or delete each deferred email independently.
+
+
+[[pmg_firmware_updates]]
+Firmware Updates
+----------------
+Firmware updates from this chapter should be applied when running {pmg} or
+Debian on a bare-metal server. Whether configuring firmware updates is
+appropriate within a virtualized environment, e.g. when using device
+pass-through, depends strongly on your setup and is therefore out of scope.
+
+In addition to regular software updates, firmware updates are also important for
+reliable and secure operation.
+
+When obtaining and applying firmware updates, a combination of available options
+is recommended to get them as early as possible or at all.
+
+The term firmware is usually divided linguistically into microcode (for CPUs)
+and firmware (for other devices).
+
+
+[[pmg_firmware_persistent]]
+Persistent Firmware
+~~~~~~~~~~~~~~~~~~~
+This section is suitable for all devices. Updated microcode, which is usually
+included in a BIOS/UEFI update, is stored on the motherboard, whereas other
+firmware is stored on the respective device. This persistent method is
+especially important for the CPU, as it enables the earliest possible regular
+loading of the updated microcode at boot time.
+
+CAUTION: With some updates, such as for BIOS/UEFI or storage controller, the
+device configuration could be reset. Please follow the vendor's instructions
+carefully and back up the current configuration.
+
+Please check with your vendor which update methods are available.
+
+* Convenient update methods for servers can include Dell's Lifecycle Manager or
+Service Packs from HPE.
+
+* Sometimes there are Linux utilities available as well. Examples are
+https://network.nvidia.com/support/firmware/mlxup-mft/['mlxup'] for NVIDIA
+ConnectX or
+https://techdocs.broadcom.com/us/en/storage-and-ethernet-connectivity/ethernet-nic-controllers/bcm957xxx/adapters/software-installation/updating-the-firmware/manually-updating-the-adapter-firmware-on-linuxesx.html['bnxtnvm'/'niccli']
+for Broadcom network cards.
+
+* https://fwupd.org[LVFS] could also be an option if there is a cooperation with
+a https://fwupd.org/lvfs/vendors/[vendor] and
+https://fwupd.org/lvfs/devices/[supported hardware] in use. The technical
+requirement for this is that the system was manufactured after 2014, is booted
+via UEFI and the easiest way is to mount the EFI partition from which you boot
+(`mount /dev/disk/by-partuuid/<from efibootmgr -v> /boot/efi`) before installing
+'fwupd'.
+
+TIP: If the update instructions require a host reboot, please do not forget
+about it.
+
+
+[[pmg_firmware_runtime_files]]
+Runtime Firmware Files
+~~~~~~~~~~~~~~~~~~~~~~
+This method stores firmware on the {pmg} operating system and will pass it to a
+device if its xref:pmg_firmware_persistent[persisted firmware] is less recent.
+It is supported by devices such as network and graphics cards, but not by those
+that rely on persisted firmware such as the motherboard and hard disks.
+
+In {pmg} the package `pve-firmware` is already installed by default. Therefore,
+with the normal xref:pmg_updates[system updates (APT)], included firmware of
+common hardware is automatically kept up to date.
+
+An additional xref:pmg_debian_firmware_repo[Debian Firmware Repository] exists,
+but is not configured by default.
+
+If you try to install an additional firmware package but it conflicts, APT will
+abort the installation. Perhaps the particular firmware can be obtained in
+another way.
+
+
+[[pmg_firmware_cpu]]
+CPU Microcode Updates
+~~~~~~~~~~~~~~~~~~~~~
+Microcode updates are intended to fix found security vulnerabilities and other
+serious CPU bugs. While the CPU performance can be affected, a patched microcode
+is usually still more performant than an unpatched microcode where the kernel
+itself has to do mitigations. Depending on the CPU type, it is possible that
+performance results of the flawed factory state can no longer be achieved
+without knowingly running the CPU in an unsafe state.
+
+To get an overview of present CPU vulnerabilities and their mitigations, run
+`lscpu`. Current real-world known vulnerabilities can only show up if the {pmg}
+host is xref:pmg_updates[up to date], its version not
+xref:faq-support-table[end of life], and has at least been rebooted since the
+last kernel update.
+
+Besides the recommended microcode update via
+xref:pmg_firmware_persistent[persistent] BIOS/UEFI updates, there is also an
+independent method via *Early OS Microcode Updates*. It is convenient to use and
+also quite helpful when the motherboard vendor no longer provides BIOS/UEFI
+updates. Regardless of the method in use, a reboot is always needed to apply a
+microcode update.
+
+
+Set up Early OS Microcode Updates
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+To set up microcode updates that are applied early on boot by the Linux kernel,
+you need to:
+
+. Enable the xref:pmg_debian_firmware_repo[Debian Firmware Repository]
+. Get the latest available packages: `apt update` (or use the web interface,
+ under Administration -> Updates)
+. Install the CPU-vendor specific microcode package:
+ - For Intel CPUs: `apt install intel-microcode`
+ - For AMD CPUs: `apt install amd64-microcode`
+. Reboot the {pmg} host
+
+Any future microcode update will also require a reboot to be loaded.
+
+
+Microcode Version
+^^^^^^^^^^^^^^^^^
+To get the current running microcode revision for comparison or debugging
+purposes:
+
+----
+# grep microcode /proc/cpuinfo | uniq
+microcode : 0xf0
+----
+
+A microcode package has updates for many different CPUs. But updates
+specifically for your CPU might not come often. So, just looking at the date on
+the package won't tell you when the company actually released an update for your
+specific CPU.
+
+If you've installed a new microcode package and rebooted your {pmg} host, and
+this new microcode is newer than both, the version baked into the CPU and the
+one from the motherboard's firmware, you'll see a message in the system log
+saying "microcode updated early".
+
+----
+# dmesg | grep microcode
+[ 0.000000] microcode: microcode updated early to revision 0xf0, date = 2021-11-12
+[ 0.896580] microcode: Microcode Update Driver: v2.2.
+----
+
+
+[[pmg_firmware_troubleshooting]]
+Troubleshooting
+^^^^^^^^^^^^^^^
+For debugging purposes, the set up Early OS Microcode Update applied regularly
+at system boot can be temporarily disabled as follows:
+
+. Reboot the host to get to the GRUB menu (hold `SHIFT` if it is hidden)
+. At the desired {pmg} boot entry press `E`
+. Go to the line which starts with `linux` and append separated by a space
+*`dis_ucode_ldr`*
+. Press `CTRL-X` to boot this time without an Early OS Microcode Update
+
+If a problem related to a recent microcode update is suspected, a package
+downgrade should be considered instead of package removal
+(`apt purge <intel-microcode|amd64-microcode>`). Otherwise, a too old
+xref:pmg_firmware_persistent[persisted] microcode might be loaded, even
+though a more recent one would run without problems.
+
+A downgrade is possible if an earlier microcode package version is
+available in the Debian repository, as shown in this example:
+
+----
+# apt list -a intel-microcode
+Listing... Done
+intel-microcode/stable-security,now 3.20230808.1~deb12u1 amd64 [installed]
+intel-microcode/stable 3.20230512.1 amd64
+----
+----
+# apt install intel-microcode=3.202305*
+...
+Selected version '3.20230512.1' (Debian:12.1/stable [amd64]) for 'intel-microcode'
+...
+dpkg: warning: downgrading intel-microcode from 3.20230808.1~deb12u1 to 3.20230512.1
+...
+intel-microcode: microcode will be updated at next boot
+...
+----
+
+To apply an older microcode potentially included in the microcode package for
+your CPU type, reboot now.
+
+[TIP]
+====
+It makes sense to hold the downgraded package for a while and try more recent
+versions again at a later time. Even if the package version is the same in the
+future, system updates may have fixed the experienced problem in the meantime.
+----
+# apt-mark hold intel-microcode
+intel-microcode set on hold.
+----
+----
+# apt-mark unhold intel-microcode
+# apt update
+# apt upgrade
+----
+====
+
+
+include::system-booting.adoc[]
projects / pmg-docs.git / blobdiff