cpuview: fix ABBA deadlock in find_proc_stat_node

Thanks to detailed report from Nikhil it was discovered
that on some workloads reads from lxcfs getting stuck.

After analysis of kernel crashdump it was found, that many
"mtail" processes waiting on read() from /proc/stat file.

First suspect was my last commit that fixes use-after-free,
but unfortunately it also adds ABBA deadlock.

Thread 1                                                   Thread 2

find_proc_stat_node():
rwlock_read                                               rwlock_read
	mutex_lock(some_node) [taken]                     	mutex_lock(some_node) [wait T1]
rwlock_unlock
rwlock_wrlock (prune_proc_stat_history call) [wait T2]

BOOM. That's deadlock.

Fix is simple, let's just move prune_proc_stat_history call
before taking mutex on cg_proc_stat node.

Fixes: 54db3e71b ("cpuview: fix possible use-after-free in find_proc_stat_node")
Issue #471

Reported-by: Nikhil Kshirsagar <nikhil.kshirsagar@canonical.com>
Signed-off-by: Alexander Mikhalitsyn <aleksandr.mikhalitsyn@canonical.com>

diff --git a/src/proc_cpuview.c b/src/proc_cpuview.c
index 678c937820ba7c38aaac12472feb1900ebfff244..89f366684881bacc921f6f47728c5816247909f2 100644 (file)
--- a/src/proc_cpuview.c
+++ b/src/proc_cpuview.c
@@ -307,6 +307,7 @@ static struct cg_proc_stat *find_proc_stat_node(struct cg_proc_stat_head *head,
 {
        struct cg_proc_stat *node;
 
+       prune_proc_stat_history();
        pthread_rwlock_rdlock(&head->lock);
 
        if (!head->next) {
@@ -327,7 +328,6 @@ static struct cg_proc_stat *find_proc_stat_node(struct cg_proc_stat_head *head,
 
 out:
        pthread_rwlock_unlock(&head->lock);
-       prune_proc_stat_history();
        return node;
 }