ceph/src/rocksdb/docs/_posts/2015-10-27-getthreadlist.markdown

   1 ---
   2 title: GetThreadList
   3 layout: post
   4 author: yhciang
   5 category: blog
   6 redirect_from:
   7   - /blog/2261/getthreadlist/
   8 ---
   9
  10 We recently added a new API, called `GetThreadList()`, that exposes the RocksDB background thread activity. With this feature, developers will be able to obtain the real-time information about the currently running compactions and flushes such as the input / output size, elapsed time, the number of bytes it has written. Below is an example output of `GetThreadList`.  To better illustrate the example, we have put a sample output of `GetThreadList` into a table where each column represents a thread status:
  11
  12 <!--truncate-->
  13
  14 <table width="637" >
  15 <tbody >
  16 <tr style="border:2px solid #000000" >
  17
  18 <td style="padding:3px" >ThreadID
  19 </td>
  20
  21 <td >140716395198208
  22 </td>
  23
  24 <td >140716416169728
  25 </td>
  26 </tr>
  27 <tr >
  28
  29 <td style="padding:3px" >DB
  30 </td>
  31
  32 <td >db1
  33 </td>
  34
  35 <td >db2
  36 </td>
  37 </tr>
  38 <tr >
  39
  40 <td style="padding:3px" >CF
  41 </td>
  42
  43 <td >default
  44 </td>
  45
  46 <td >picachu
  47 </td>
  48 </tr>
  49 <tr >
  50
  51 <td style="padding:3px" >ThreadType
  52 </td>
  53
  54 <td >High Pri
  55 </td>
  56
  57 <td >Low Pri
  58 </td>
  59 </tr>
  60 <tr >
  61
  62 <td style="padding:3px" >Operation
  63 </td>
  64
  65 <td >Flush
  66 </td>
  67
  68 <td >Compaction
  69 </td>
  70 </tr>
  71 <tr >
  72
  73 <td style="padding:3px" >ElapsedTime
  74 </td>
  75
  76 <td >143.459 ms
  77 </td>
  78
  79 <td >607.538 ms
  80 </td>
  81 </tr>
  82 <tr >
  83
  84 <td style="padding:3px" >Stage
  85 </td>
  86
  87 <td >FlushJob::WriteLevel0Table
  88 </td>
  89
  90 <td >CompactionJob::Install
  91 </td>
  92 </tr>
  93 <tr >
  94
  95 <td style="vertical-align:top;padding:3px" >OperationProperties
  96 </td>
  97
  98 <td style="vertical-align:top;padding:3px" >
  99 BytesMemtables 4092938
 100 BytesWritten 1050701
 101 </td>
 102
 103 <td style="vertical-align:top" >
 104 BaseInputLevel 1
 105 BytesRead 4876417
 106 BytesWritten 4140109
 107 IsDeletion 0
 108 IsManual 0
 109 IsTrivialMove 0
 110 JobID 146
 111 OutputLevel 2
 112 TotalInputBytes 4883044
 113 </td>
 114 </tr>
 115 </tbody>
 116 </table>
 117
 118 In the above output, we can see `GetThreadList()` reports the activity of two threads: one thread running flush job (middle column) and the other thread running a compaction job (right-most column).  In each thread status, it shows basic information about the thread such as thread id, it's target db / column family, and the job it is currently doing and the current status of the job.  For instance, we can see thread 140716416169728 is doing compaction on the `picachu` column family in database `db2`.  In addition, we can see the compaction has been running for 600 ms, and it has read 4876417 bytes out of 4883044 bytes. This indicates the compaction is about to complete.  The stage property indicates which code block the thread is currently executing.  For instance, thread 140716416169728 is currently running `CompactionJob::Install`, which further indicates the compaction job is almost done.
 119
 120 Below we briefly describe its API.
 121
 122
 123 ## How to Enable it?
 124
 125
 126 To enable thread-tracking of a rocksdb instance, simply set `enable_thread_tracking` to true in its DBOptions:
 127
 128 ```c++
 129 // If true, then the status of the threads involved in this DB will
 130 // be tracked and available via GetThreadList() API.
 131 //
 132 // Default: false
 133 bool enable_thread_tracking;
 134 ```
 135
 136
 137
 138 ## The API
 139
 140
 141 The GetThreadList API is defined in [include/rocksdb/env.h](https://github.com/facebook/rocksdb/blob/master/include/rocksdb/env.h#L317-L318), which is an Env
 142 function:
 143
 144 ```c++
 145 virtual Status GetThreadList(std::vector* thread_list)
 146 ```
 147
 148 Since an Env can be shared across multiple rocksdb instances, the output of
 149 `GetThreadList()` include the background activity of all the rocksdb instances
 150 that using the same Env.
 151
 152 The `GetThreadList()` API simply returns a vector of `ThreadStatus`, each describes
 153 the current status of a thread. The `ThreadStatus` structure, defined in
 154 [include/rocksdb/thread_status.h](https://github.com/facebook/rocksdb/blob/master/include/rocksdb/thread_status.h), contains the following information:
 155
 156 ```c++
 157 // An unique ID for the thread.
 158 const uint64_t thread_id;
 159
 160 // The type of the thread, it could be HIGH_PRIORITY,
 161 // LOW_PRIORITY, and USER
 162 const ThreadType thread_type;
 163
 164 // The name of the DB instance where the thread is currently
 165 // involved with. It would be set to empty string if the thread
 166 // does not involve in any DB operation.
 167 const std::string db_name;
 168
 169 // The name of the column family where the thread is currently
 170 // It would be set to empty string if the thread does not involve
 171 // in any column family.
 172 const std::string cf_name;
 173
 174 // The operation (high-level action) that the current thread is involved.
 175 const OperationType operation_type;
 176
 177 // The elapsed time in micros of the current thread operation.
 178 const uint64_t op_elapsed_micros;
 179
 180 // An integer showing the current stage where the thread is involved
 181 // in the current operation.
 182 const OperationStage operation_stage;
 183
 184 // A list of properties that describe some details about the current
 185 // operation. Same field in op_properties[] might have different
 186 // meanings for different operations.
 187 uint64_t op_properties[kNumOperationProperties];
 188
 189 // The state (lower-level action) that the current thread is involved.
 190 const StateType state_type;
 191 ```
 192
 193 If you are interested in the background thread activity of your RocksDB application, please feel free to give `GetThreadList()` a try :)