import quincy beta 17.1.0

[ceph.git] / ceph / doc / cephadm / troubleshooting.rst

diff --git a/ceph/doc/cephadm/troubleshooting.rst b/ceph/doc/cephadm/troubleshooting.rst
index 4ce76ee8166af1d458886275add75b884e3aab9d..b7d295b1296c3d9167e92a397353e7e625e88cd4 100644 (file)
--- a/ceph/doc/cephadm/troubleshooting.rst
+++ b/ceph/doc/cephadm/troubleshooting.rst
@@ -273,7 +273,7 @@ To call miscellaneous like ``ceph-objectstore-tool`` or
     0: [v2:127.0.0.1:3300/0,v1:127.0.0.1:6789/0] mon.myhostname
 
 This command sets up the environment in a way that is suitable
-for extended daemon maintenance and running the deamon interactively. 
+for extended daemon maintenance and running the daemon interactively. 
 
 .. _cephadm-restore-quorum:
 
@@ -324,7 +324,7 @@ Get the container image::
 
   ceph config get "mgr.hostname.smfvfd" container_image
 
-Create a file ``config-json.json`` which contains the information neccessary to deploy
+Create a file ``config-json.json`` which contains the information necessary to deploy
 the daemon:
 
 .. code-block:: json
@@ -338,3 +338,33 @@ Deploy the daemon::
 
   cephadm --image <container-image> deploy --fsid <fsid> --name mgr.hostname.smfvfd --config-json config-json.json
 
+Analyzing core dumps
+---------------------
+
+In case a Ceph daemon crashes, cephadm supports analyzing core dumps. To enable core dumps, run
+
+.. prompt:: bash #
+
+  ulimit -c unlimited
+
+core dumps will now be written to ``/var/lib/systemd/coredump``.
+
+.. note::
+
+  core dumps are not namespaced by the kernel, which means
+  they will be written to ``/var/lib/systemd/coredump`` on
+  the container host. 
+
+Now, wait for the crash to happen again. (To simulate the crash of a daemon, run e.g. ``killall -3 ceph-mon``)
+
+Install debug packages by entering the cephadm shell and install ``ceph-debuginfo``::
+
+  # cephadm shell --mount /var/lib/systemd/coredump
+  [ceph: root@host1 /]# dnf install ceph-debuginfo gdb zstd
+  [ceph: root@host1 /]# unzstd /mnt/coredump/core.ceph-*.zst
+  [ceph: root@host1 /]# gdb /usr/bin/ceph-mon /mnt/coredump/core.ceph-...
+  (gdb) bt
+  #0  0x00007fa9117383fc in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
+  #1  0x00007fa910d7f8f0 in std::condition_variable::wait(std::unique_lock<std::mutex>&) () from /lib64/libstdc++.so.6
+  #2  0x00007fa913d3f48f in AsyncMessenger::wait() () from /usr/lib64/ceph/libceph-common.so.2
+  #3  0x0000563085ca3d7e in main ()