PCI: Avoid race while enabling upstream bridges

PCI: Avoid race while enabling upstream bridges

When we enable a device, we first enable any upstream bridges.  If a bridge
has multiple downstream devices and we enable them simultaneously, the race
to enable the upstream bridge may cause problems.  Consider this hierarchy:

  bridge A --+-- device B
             +-- device C

If drivers for B and C call pci_enable_device() simultaneously, both will
attempt to enable A, which involves setting PCI_COMMAND_MASTER via
pci_set_master() and PCI_COMMAND_MEMORY via pci_enable_resources().

In the following sequence, B's update to set A's PCI_COMMAND_MEMORY is
lost, and neither B nor C will work correctly:

      B                                C
  pci_set_master(A)
    cmd = read(A, PCI_COMMAND)
    cmd |= PCI_COMMAND_MASTER
                                   pci_set_master(A)
                                     cmd = read(A, PCI_COMMAND)
                                     cmd |= PCI_COMMAND_MASTER
    write(A, PCI_COMMAND, cmd)
  pci_enable_device(A)
    pci_enable_resources(A)
      cmd = read(A, PCI_COMMAND)
      cmd |= PCI_COMMAND_MEMORY
      write(A, PCI_COMMAND, cmd)
                                     write(A, PCI_COMMAND, cmd)

Avoid this race by holding a new pci_bridge_mutex while enabling a bridge.
This ensures that both PCI_COMMAND_MASTER and PCI_COMMAND_MEMORY will be
updated before another thread can start enabling the bridge.

Note that although pci_enable_bridge() is recursive, it enables any
upstream bridges *before* acquiring the mutex.  When it acquires the mutex
and calls pci_set_master() and pci_enable_device(), any upstream bridges
have already been enabled so pci_enable_device() will not deadlock by
calling pci_enable_bridge() again.

Signed-off-by: Srinath Mannam <srinath.mannam@broadcom.com>
[bhelgaas: changelog, comment]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>