ceph/src/seastar/doc/network-connection-load-balancing.md

   1 # Motivation
   2
   3 In sharded systems like seastar it is important for work to be
   4 distributed equally between all shards to achieve maximum performance
   5 from the system. Networking subsystem has its part in distributing work
   6 equally. For instance if on a server all connections will be served by
   7 single shard only, the system will be working with the speed of this
   8 one shard and all other shards will be underutilized.
   9
  10 # Common ways to distribute work received over network between shards
  11
  12 Two common ways to distribute work between shards are:
  13  - do the work at a shard that received it
  14  - shard that does actual work depends on a data been processed
  15    (one way to do it is to hash(data) % smp_count = shard)
  16
  17 # Load Balancing
  18
  19 Those two ways asks for different strategy to distribute connections
  20 between shards.  The first one will work best if each cpu will have the
  21 same amount of connections (assuming each connection gets same amount of
  22 works) the second will work best if data will arrive to a shard where
  23 it is going to be processed and actual connection distribution does
  24 not matter.
  25
  26 Seastar's posix stack supports both of those strategies. Desired
  27 one can be chosen by specifying load balancing algorithm in
  28 listen_options provided to reactor::listen() call. Available options
  29 are:
  30
  31 - load_balancing_algorithm::connection_distribution
  32
  33   Make sure that new connection will be placed to a shard with smallest
  34   amount of connections of the same type.
  35
  36 -  load_balancing_algorithm::port
  37
  38    Destination shard is chosen as a function of client's local port:
  39    shard = port_number % num_shards.  This allows a client to make sure that
  40    a connection will be processed by a specific shard by choosing its local
  41    port accordingly (the knowledge about amount of shards in the server is
  42    needed and can be negotiated by different channel).