match the driver to the device. For example:
# echo "046d c315" > /sys/bus/usb/drivers/foo/remove_id
-What: /sys/bus/usb/device/.../avoid_reset
+What: /sys/bus/usb/device/.../avoid_reset_quirk
Date: December 2009
Contact: Oliver Neukum <oliver@neukum.org>
Description:
--- /dev/null
+ Dynamic DMA mapping Guide
+ =========================
+
+ David S. Miller <davem@redhat.com>
+ Richard Henderson <rth@cygnus.com>
+ Jakub Jelinek <jakub@redhat.com>
+
+This is a guide to device driver writers on how to use the DMA API
+with example pseudo-code. For a concise description of the API, see
+DMA-API.txt.
+
+Most of the 64bit platforms have special hardware that translates bus
+addresses (DMA addresses) into physical addresses. This is similar to
+how page tables and/or a TLB translates virtual addresses to physical
+addresses on a CPU. This is needed so that e.g. PCI devices can
+access with a Single Address Cycle (32bit DMA address) any page in the
+64bit physical address space. Previously in Linux those 64bit
+platforms had to set artificial limits on the maximum RAM size in the
+system, so that the virt_to_bus() static scheme works (the DMA address
+translation tables were simply filled on bootup to map each bus
+address to the physical page __pa(bus_to_virt())).
+
+So that Linux can use the dynamic DMA mapping, it needs some help from the
+drivers, namely it has to take into account that DMA addresses should be
+mapped only for the time they are actually used and unmapped after the DMA
+transfer.
+
+The following API will work of course even on platforms where no such
+hardware exists.
+
+Note that the DMA API works with any bus independent of the underlying
+microprocessor architecture. You should use the DMA API rather than
+the bus specific DMA API (e.g. pci_dma_*).
+
+First of all, you should make sure
+
+#include <linux/dma-mapping.h>
+
+is in your driver. This file will obtain for you the definition of the
+dma_addr_t (which can hold any valid DMA address for the platform)
+type which should be used everywhere you hold a DMA (bus) address
+returned from the DMA mapping functions.
+
+ What memory is DMA'able?
+
+The first piece of information you must know is what kernel memory can
+be used with the DMA mapping facilities. There has been an unwritten
+set of rules regarding this, and this text is an attempt to finally
+write them down.
+
+If you acquired your memory via the page allocator
+(i.e. __get_free_page*()) or the generic memory allocators
+(i.e. kmalloc() or kmem_cache_alloc()) then you may DMA to/from
+that memory using the addresses returned from those routines.
+
+This means specifically that you may _not_ use the memory/addresses
+returned from vmalloc() for DMA. It is possible to DMA to the
+_underlying_ memory mapped into a vmalloc() area, but this requires
+walking page tables to get the physical addresses, and then
+translating each of those pages back to a kernel address using
+something like __va(). [ EDIT: Update this when we integrate
+Gerd Knorr's generic code which does this. ]
+
+This rule also means that you may use neither kernel image addresses
+(items in data/text/bss segments), nor module image addresses, nor
+stack addresses for DMA. These could all be mapped somewhere entirely
+different than the rest of physical memory. Even if those classes of
+memory could physically work with DMA, you'd need to ensure the I/O
+buffers were cacheline-aligned. Without that, you'd see cacheline
+sharing problems (data corruption) on CPUs with DMA-incoherent caches.
+(The CPU could write to one word, DMA would write to a different one
+in the same cache line, and one of them could be overwritten.)
+
+Also, this means that you cannot take the return of a kmap()
+call and DMA to/from that. This is similar to vmalloc().
+
+What about block I/O and networking buffers? The block I/O and
+networking subsystems make sure that the buffers they use are valid
+for you to DMA from/to.
+
+ DMA addressing limitations
+
+Does your device have any DMA addressing limitations? For example, is
+your device only capable of driving the low order 24-bits of address?
+If so, you need to inform the kernel of this fact.
+
+By default, the kernel assumes that your device can address the full
+32-bits. For a 64-bit capable device, this needs to be increased.
+And for a device with limitations, as discussed in the previous
+paragraph, it needs to be decreased.
+
+Special note about PCI: PCI-X specification requires PCI-X devices to
+support 64-bit addressing (DAC) for all transactions. And at least
+one platform (SGI SN2) requires 64-bit consistent allocations to
+operate correctly when the IO bus is in PCI-X mode.
+
+For correct operation, you must interrogate the kernel in your device
+probe routine to see if the DMA controller on the machine can properly
+support the DMA addressing limitation your device has. It is good
+style to do this even if your device holds the default setting,
+because this shows that you did think about these issues wrt. your
+device.
+
+The query is performed via a call to dma_set_mask():
+
+ int dma_set_mask(struct device *dev, u64 mask);
+
+The query for consistent allocations is performed via a call to
+dma_set_coherent_mask():
+
+ int dma_set_coherent_mask(struct device *dev, u64 mask);
+
+Here, dev is a pointer to the device struct of your device, and mask
+is a bit mask describing which bits of an address your device
+supports. It returns zero if your card can perform DMA properly on
+the machine given the address mask you provided. In general, the
+device struct of your device is embedded in the bus specific device
+struct of your device. For example, a pointer to the device struct of
+your PCI device is pdev->dev (pdev is a pointer to the PCI device
+struct of your device).
+
+If it returns non-zero, your device cannot perform DMA properly on
+this platform, and attempting to do so will result in undefined
+behavior. You must either use a different mask, or not use DMA.
+
+This means that in the failure case, you have three options:
+
+1) Use another DMA mask, if possible (see below).
+2) Use some non-DMA mode for data transfer, if possible.
+3) Ignore this device and do not initialize it.
+
+It is recommended that your driver print a kernel KERN_WARNING message
+when you end up performing either #2 or #3. In this manner, if a user
+of your driver reports that performance is bad or that the device is not
+even detected, you can ask them for the kernel messages to find out
+exactly why.
+
+The standard 32-bit addressing device would do something like this:
+
+ if (dma_set_mask(dev, DMA_BIT_MASK(32))) {
+ printk(KERN_WARNING
+ "mydev: No suitable DMA available.\n");
+ goto ignore_this_device;
+ }
+
+Another common scenario is a 64-bit capable device. The approach here
+is to try for 64-bit addressing, but back down to a 32-bit mask that
+should not fail. The kernel may fail the 64-bit mask not because the
+platform is not capable of 64-bit addressing. Rather, it may fail in
+this case simply because 32-bit addressing is done more efficiently
+than 64-bit addressing. For example, Sparc64 PCI SAC addressing is
+more efficient than DAC addressing.
+
+Here is how you would handle a 64-bit capable device which can drive
+all 64-bits when accessing streaming DMA:
+
+ int using_dac;
+
+ if (!dma_set_mask(dev, DMA_BIT_MASK(64))) {
+ using_dac = 1;
+ } else if (!dma_set_mask(dev, DMA_BIT_MASK(32))) {
+ using_dac = 0;
+ } else {
+ printk(KERN_WARNING
+ "mydev: No suitable DMA available.\n");
+ goto ignore_this_device;
+ }
+
+If a card is capable of using 64-bit consistent allocations as well,
+the case would look like this:
+
+ int using_dac, consistent_using_dac;
+
+ if (!dma_set_mask(dev, DMA_BIT_MASK(64))) {
+ using_dac = 1;
+ consistent_using_dac = 1;
+ dma_set_coherent_mask(dev, DMA_BIT_MASK(64));
+ } else if (!dma_set_mask(dev, DMA_BIT_MASK(32))) {
+ using_dac = 0;
+ consistent_using_dac = 0;
+ dma_set_coherent_mask(dev, DMA_BIT_MASK(32));
+ } else {
+ printk(KERN_WARNING
+ "mydev: No suitable DMA available.\n");
+ goto ignore_this_device;
+ }
+
+dma_set_coherent_mask() will always be able to set the same or a
+smaller mask as dma_set_mask(). However for the rare case that a
+device driver only uses consistent allocations, one would have to
+check the return value from dma_set_coherent_mask().
+
+Finally, if your device can only drive the low 24-bits of
+address you might do something like:
+
+ if (dma_set_mask(dev, DMA_BIT_MASK(24))) {
+ printk(KERN_WARNING
+ "mydev: 24-bit DMA addressing not available.\n");
+ goto ignore_this_device;
+ }
+
+When dma_set_mask() is successful, and returns zero, the kernel saves
+away this mask you have provided. The kernel will use this
+information later when you make DMA mappings.
+
+There is a case which we are aware of at this time, which is worth
+mentioning in this documentation. If your device supports multiple
+functions (for example a sound card provides playback and record
+functions) and the various different functions have _different_
+DMA addressing limitations, you may wish to probe each mask and
+only provide the functionality which the machine can handle. It
+is important that the last call to dma_set_mask() be for the
+most specific mask.
+
+Here is pseudo-code showing how this might be done:
+
+ #define PLAYBACK_ADDRESS_BITS DMA_BIT_MASK(32)
+ #define RECORD_ADDRESS_BITS DMA_BIT_MASK(24)
+
+ struct my_sound_card *card;
+ struct device *dev;
+
+ ...
+ if (!dma_set_mask(dev, PLAYBACK_ADDRESS_BITS)) {
+ card->playback_enabled = 1;
+ } else {
+ card->playback_enabled = 0;
+ printk(KERN_WARNING "%s: Playback disabled due to DMA limitations.\n",
+ card->name);
+ }
+ if (!dma_set_mask(dev, RECORD_ADDRESS_BITS)) {
+ card->record_enabled = 1;
+ } else {
+ card->record_enabled = 0;
+ printk(KERN_WARNING "%s: Record disabled due to DMA limitations.\n",
+ card->name);
+ }
+
+A sound card was used as an example here because this genre of PCI
+devices seems to be littered with ISA chips given a PCI front end,
+and thus retaining the 16MB DMA addressing limitations of ISA.
+
+ Types of DMA mappings
+
+There are two types of DMA mappings:
+
+- Consistent DMA mappings which are usually mapped at driver
+ initialization, unmapped at the end and for which the hardware should
+ guarantee that the device and the CPU can access the data
+ in parallel and will see updates made by each other without any
+ explicit software flushing.
+
+ Think of "consistent" as "synchronous" or "coherent".
+
+ The current default is to return consistent memory in the low 32
+ bits of the bus space. However, for future compatibility you should
+ set the consistent mask even if this default is fine for your
+ driver.
+
+ Good examples of what to use consistent mappings for are:
+
+ - Network card DMA ring descriptors.
+ - SCSI adapter mailbox command data structures.
+ - Device firmware microcode executed out of
+ main memory.
+
+ The invariant these examples all require is that any CPU store
+ to memory is immediately visible to the device, and vice
+ versa. Consistent mappings guarantee this.
+
+ IMPORTANT: Consistent DMA memory does not preclude the usage of
+ proper memory barriers. The CPU may reorder stores to
+ consistent memory just as it may normal memory. Example:
+ if it is important for the device to see the first word
+ of a descriptor updated before the second, you must do
+ something like:
+
+ desc->word0 = address;
+ wmb();
+ desc->word1 = DESC_VALID;
+
+ in order to get correct behavior on all platforms.
+
+ Also, on some platforms your driver may need to flush CPU write
+ buffers in much the same way as it needs to flush write buffers
+ found in PCI bridges (such as by reading a register's value
+ after writing it).
+
+- Streaming DMA mappings which are usually mapped for one DMA
+ transfer, unmapped right after it (unless you use dma_sync_* below)
+ and for which hardware can optimize for sequential accesses.
+
+ This of "streaming" as "asynchronous" or "outside the coherency
+ domain".
+
+ Good examples of what to use streaming mappings for are:
+
+ - Networking buffers transmitted/received by a device.
+ - Filesystem buffers written/read by a SCSI device.
+
+ The interfaces for using this type of mapping were designed in
+ such a way that an implementation can make whatever performance
+ optimizations the hardware allows. To this end, when using
+ such mappings you must be explicit about what you want to happen.
+
+Neither type of DMA mapping has alignment restrictions that come from
+the underlying bus, although some devices may have such restrictions.
+Also, systems with caches that aren't DMA-coherent will work better
+when the underlying buffers don't share cache lines with other data.
+
+
+ Using Consistent DMA mappings.
+
+To allocate and map large (PAGE_SIZE or so) consistent DMA regions,
+you should do:
+
+ dma_addr_t dma_handle;
+
+ cpu_addr = dma_alloc_coherent(dev, size, &dma_handle, gfp);
+
+where device is a struct device *. This may be called in interrupt
+context with the GFP_ATOMIC flag.
+
+Size is the length of the region you want to allocate, in bytes.
+
+This routine will allocate RAM for that region, so it acts similarly to
+__get_free_pages (but takes size instead of a page order). If your
+driver needs regions sized smaller than a page, you may prefer using
+the dma_pool interface, described below.
+
+The consistent DMA mapping interfaces, for non-NULL dev, will by
+default return a DMA address which is 32-bit addressable. Even if the
+device indicates (via DMA mask) that it may address the upper 32-bits,
+consistent allocation will only return > 32-bit addresses for DMA if
+the consistent DMA mask has been explicitly changed via
+dma_set_coherent_mask(). This is true of the dma_pool interface as
+well.
+
+dma_alloc_coherent returns two values: the virtual address which you
+can use to access it from the CPU and dma_handle which you pass to the
+card.
+
+The cpu return address and the DMA bus master address are both
+guaranteed to be aligned to the smallest PAGE_SIZE order which
+is greater than or equal to the requested size. This invariant
+exists (for example) to guarantee that if you allocate a chunk
+which is smaller than or equal to 64 kilobytes, the extent of the
+buffer you receive will not cross a 64K boundary.
+
+To unmap and free such a DMA region, you call:
+
+ dma_free_coherent(dev, size, cpu_addr, dma_handle);
+
+where dev, size are the same as in the above call and cpu_addr and
+dma_handle are the values dma_alloc_coherent returned to you.
+This function may not be called in interrupt context.
+
+If your driver needs lots of smaller memory regions, you can write
+custom code to subdivide pages returned by dma_alloc_coherent,
+or you can use the dma_pool API to do that. A dma_pool is like
+a kmem_cache, but it uses dma_alloc_coherent not __get_free_pages.
+Also, it understands common hardware constraints for alignment,
+like queue heads needing to be aligned on N byte boundaries.
+
+Create a dma_pool like this:
+
+ struct dma_pool *pool;
+
+ pool = dma_pool_create(name, dev, size, align, alloc);
+
+The "name" is for diagnostics (like a kmem_cache name); dev and size
+are as above. The device's hardware alignment requirement for this
+type of data is "align" (which is expressed in bytes, and must be a
+power of two). If your device has no boundary crossing restrictions,
+pass 0 for alloc; passing 4096 says memory allocated from this pool
+must not cross 4KByte boundaries (but at that time it may be better to
+go for dma_alloc_coherent directly instead).
+
+Allocate memory from a dma pool like this:
+
+ cpu_addr = dma_pool_alloc(pool, flags, &dma_handle);
+
+flags are SLAB_KERNEL if blocking is permitted (not in_interrupt nor
+holding SMP locks), SLAB_ATOMIC otherwise. Like dma_alloc_coherent,
+this returns two values, cpu_addr and dma_handle.
+
+Free memory that was allocated from a dma_pool like this:
+
+ dma_pool_free(pool, cpu_addr, dma_handle);
+
+where pool is what you passed to dma_pool_alloc, and cpu_addr and
+dma_handle are the values dma_pool_alloc returned. This function
+may be called in interrupt context.
+
+Destroy a dma_pool by calling:
+
+ dma_pool_destroy(pool);
+
+Make sure you've called dma_pool_free for all memory allocated
+from a pool before you destroy the pool. This function may not
+be called in interrupt context.
+
+ DMA Direction
+
+The interfaces described in subsequent portions of this document
+take a DMA direction argument, which is an integer and takes on
+one of the following values:
+
+ DMA_BIDIRECTIONAL
+ DMA_TO_DEVICE
+ DMA_FROM_DEVICE
+ DMA_NONE
+
+One should provide the exact DMA direction if you know it.
+
+DMA_TO_DEVICE means "from main memory to the device"
+DMA_FROM_DEVICE means "from the device to main memory"
+It is the direction in which the data moves during the DMA
+transfer.
+
+You are _strongly_ encouraged to specify this as precisely
+as you possibly can.
+
+If you absolutely cannot know the direction of the DMA transfer,
+specify DMA_BIDIRECTIONAL. It means that the DMA can go in
+either direction. The platform guarantees that you may legally
+specify this, and that it will work, but this may be at the
+cost of performance for example.
+
+The value DMA_NONE is to be used for debugging. One can
+hold this in a data structure before you come to know the
+precise direction, and this will help catch cases where your
+direction tracking logic has failed to set things up properly.
+
+Another advantage of specifying this value precisely (outside of
+potential platform-specific optimizations of such) is for debugging.
+Some platforms actually have a write permission boolean which DMA
+mappings can be marked with, much like page protections in the user
+program address space. Such platforms can and do report errors in the
+kernel logs when the DMA controller hardware detects violation of the
+permission setting.
+
+Only streaming mappings specify a direction, consistent mappings
+implicitly have a direction attribute setting of
+DMA_BIDIRECTIONAL.
+
+The SCSI subsystem tells you the direction to use in the
+'sc_data_direction' member of the SCSI command your driver is
+working on.
+
+For Networking drivers, it's a rather simple affair. For transmit
+packets, map/unmap them with the DMA_TO_DEVICE direction
+specifier. For receive packets, just the opposite, map/unmap them
+with the DMA_FROM_DEVICE direction specifier.
+
+ Using Streaming DMA mappings
+
+The streaming DMA mapping routines can be called from interrupt
+context. There are two versions of each map/unmap, one which will
+map/unmap a single memory region, and one which will map/unmap a
+scatterlist.
+
+To map a single region, you do:
+
+ struct device *dev = &my_dev->dev;
+ dma_addr_t dma_handle;
+ void *addr = buffer->ptr;
+ size_t size = buffer->len;
+
+ dma_handle = dma_map_single(dev, addr, size, direction);
+
+and to unmap it:
+
+ dma_unmap_single(dev, dma_handle, size, direction);
+
+You should call dma_unmap_single when the DMA activity is finished, e.g.
+from the interrupt which told you that the DMA transfer is done.
+
+Using cpu pointers like this for single mappings has a disadvantage,
+you cannot reference HIGHMEM memory in this way. Thus, there is a
+map/unmap interface pair akin to dma_{map,unmap}_single. These
+interfaces deal with page/offset pairs instead of cpu pointers.
+Specifically:
+
+ struct device *dev = &my_dev->dev;
+ dma_addr_t dma_handle;
+ struct page *page = buffer->page;
+ unsigned long offset = buffer->offset;
+ size_t size = buffer->len;
+
+ dma_handle = dma_map_page(dev, page, offset, size, direction);
+
+ ...
+
+ dma_unmap_page(dev, dma_handle, size, direction);
+
+Here, "offset" means byte offset within the given page.
+
+With scatterlists, you map a region gathered from several regions by:
+
+ int i, count = dma_map_sg(dev, sglist, nents, direction);
+ struct scatterlist *sg;
+
+ for_each_sg(sglist, sg, count, i) {
+ hw_address[i] = sg_dma_address(sg);
+ hw_len[i] = sg_dma_len(sg);
+ }
+
+where nents is the number of entries in the sglist.
+
+The implementation is free to merge several consecutive sglist entries
+into one (e.g. if DMA mapping is done with PAGE_SIZE granularity, any
+consecutive sglist entries can be merged into one provided the first one
+ends and the second one starts on a page boundary - in fact this is a huge
+advantage for cards which either cannot do scatter-gather or have very
+limited number of scatter-gather entries) and returns the actual number
+of sg entries it mapped them to. On failure 0 is returned.
+
+Then you should loop count times (note: this can be less than nents times)
+and use sg_dma_address() and sg_dma_len() macros where you previously
+accessed sg->address and sg->length as shown above.
+
+To unmap a scatterlist, just call:
+
+ dma_unmap_sg(dev, sglist, nents, direction);
+
+Again, make sure DMA activity has already finished.
+
+PLEASE NOTE: The 'nents' argument to the dma_unmap_sg call must be
+ the _same_ one you passed into the dma_map_sg call,
+ it should _NOT_ be the 'count' value _returned_ from the
+ dma_map_sg call.
+
+Every dma_map_{single,sg} call should have its dma_unmap_{single,sg}
+counterpart, because the bus address space is a shared resource (although
+in some ports the mapping is per each BUS so less devices contend for the
+same bus address space) and you could render the machine unusable by eating
+all bus addresses.
+
+If you need to use the same streaming DMA region multiple times and touch
+the data in between the DMA transfers, the buffer needs to be synced
+properly in order for the cpu and device to see the most uptodate and
+correct copy of the DMA buffer.
+
+So, firstly, just map it with dma_map_{single,sg}, and after each DMA
+transfer call either:
+
+ dma_sync_single_for_cpu(dev, dma_handle, size, direction);
+
+or:
+
+ dma_sync_sg_for_cpu(dev, sglist, nents, direction);
+
+as appropriate.
+
+Then, if you wish to let the device get at the DMA area again,
+finish accessing the data with the cpu, and then before actually
+giving the buffer to the hardware call either:
+
+ dma_sync_single_for_device(dev, dma_handle, size, direction);
+
+or:
+
+ dma_sync_sg_for_device(dev, sglist, nents, direction);
+
+as appropriate.
+
+After the last DMA transfer call one of the DMA unmap routines
+dma_unmap_{single,sg}. If you don't touch the data from the first dma_map_*
+call till dma_unmap_*, then you don't have to call the dma_sync_*
+routines at all.
+
+Here is pseudo code which shows a situation in which you would need
+to use the dma_sync_*() interfaces.
+
+ my_card_setup_receive_buffer(struct my_card *cp, char *buffer, int len)
+ {
+ dma_addr_t mapping;
+
+ mapping = dma_map_single(cp->dev, buffer, len, DMA_FROM_DEVICE);
+
+ cp->rx_buf = buffer;
+ cp->rx_len = len;
+ cp->rx_dma = mapping;
+
+ give_rx_buf_to_card(cp);
+ }
+
+ ...
+
+ my_card_interrupt_handler(int irq, void *devid, struct pt_regs *regs)
+ {
+ struct my_card *cp = devid;
+
+ ...
+ if (read_card_status(cp) == RX_BUF_TRANSFERRED) {
+ struct my_card_header *hp;
+
+ /* Examine the header to see if we wish
+ * to accept the data. But synchronize
+ * the DMA transfer with the CPU first
+ * so that we see updated contents.
+ */
+ dma_sync_single_for_cpu(&cp->dev, cp->rx_dma,
+ cp->rx_len,
+ DMA_FROM_DEVICE);
+
+ /* Now it is safe to examine the buffer. */
+ hp = (struct my_card_header *) cp->rx_buf;
+ if (header_is_ok(hp)) {
+ dma_unmap_single(&cp->dev, cp->rx_dma, cp->rx_len,
+ DMA_FROM_DEVICE);
+ pass_to_upper_layers(cp->rx_buf);
+ make_and_setup_new_rx_buf(cp);
+ } else {
+ /* Just sync the buffer and give it back
+ * to the card.
+ */
+ dma_sync_single_for_device(&cp->dev,
+ cp->rx_dma,
+ cp->rx_len,
+ DMA_FROM_DEVICE);
+ give_rx_buf_to_card(cp);
+ }
+ }
+ }
+
+Drivers converted fully to this interface should not use virt_to_bus any
+longer, nor should they use bus_to_virt. Some drivers have to be changed a
+little bit, because there is no longer an equivalent to bus_to_virt in the
+dynamic DMA mapping scheme - you have to always store the DMA addresses
+returned by the dma_alloc_coherent, dma_pool_alloc, and dma_map_single
+calls (dma_map_sg stores them in the scatterlist itself if the platform
+supports dynamic DMA mapping in hardware) in your driver structures and/or
+in the card registers.
+
+All drivers should be using these interfaces with no exceptions. It
+is planned to completely remove virt_to_bus() and bus_to_virt() as
+they are entirely deprecated. Some ports already do not provide these
+as it is impossible to correctly support them.
+
+ Optimizing Unmap State Space Consumption
+
+On many platforms, dma_unmap_{single,page}() is simply a nop.
+Therefore, keeping track of the mapping address and length is a waste
+of space. Instead of filling your drivers up with ifdefs and the like
+to "work around" this (which would defeat the whole purpose of a
+portable API) the following facilities are provided.
+
+Actually, instead of describing the macros one by one, we'll
+transform some example code.
+
+1) Use DEFINE_DMA_UNMAP_{ADDR,LEN} in state saving structures.
+ Example, before:
+
+ struct ring_state {
+ struct sk_buff *skb;
+ dma_addr_t mapping;
+ __u32 len;
+ };
+
+ after:
+
+ struct ring_state {
+ struct sk_buff *skb;
+ DEFINE_DMA_UNMAP_ADDR(mapping);
+ DEFINE_DMA_UNMAP_LEN(len);
+ };
+
+2) Use dma_unmap_{addr,len}_set to set these values.
+ Example, before:
+
+ ringp->mapping = FOO;
+ ringp->len = BAR;
+
+ after:
+
+ dma_unmap_addr_set(ringp, mapping, FOO);
+ dma_unmap_len_set(ringp, len, BAR);
+
+3) Use dma_unmap_{addr,len} to access these values.
+ Example, before:
+
+ dma_unmap_single(dev, ringp->mapping, ringp->len,
+ DMA_FROM_DEVICE);
+
+ after:
+
+ dma_unmap_single(dev,
+ dma_unmap_addr(ringp, mapping),
+ dma_unmap_len(ringp, len),
+ DMA_FROM_DEVICE);
+
+It really should be self-explanatory. We treat the ADDR and LEN
+separately, because it is possible for an implementation to only
+need the address in order to perform the unmap operation.
+
+ Platform Issues
+
+If you are just writing drivers for Linux and do not maintain
+an architecture port for the kernel, you can safely skip down
+to "Closing".
+
+1) Struct scatterlist requirements.
+
+ Struct scatterlist must contain, at a minimum, the following
+ members:
+
+ struct page *page;
+ unsigned int offset;
+ unsigned int length;
+
+ The base address is specified by a "page+offset" pair.
+
+ Previous versions of struct scatterlist contained a "void *address"
+ field that was sometimes used instead of page+offset. As of Linux
+ 2.5., page+offset is always used, and the "address" field has been
+ deleted.
+
+2) More to come...
+
+ Handling Errors
+
+DMA address space is limited on some architectures and an allocation
+failure can be determined by:
+
+- checking if dma_alloc_coherent returns NULL or dma_map_sg returns 0
+
+- checking the returned dma_addr_t of dma_map_single and dma_map_page
+ by using dma_mapping_error():
+
+ dma_addr_t dma_handle;
+
+ dma_handle = dma_map_single(dev, addr, size, direction);
+ if (dma_mapping_error(dev, dma_handle)) {
+ /*
+ * reduce current DMA mapping usage,
+ * delay and try again later or
+ * reset driver.
+ */
+ }
+
+ Closing
+
+This document, and the API itself, would not be in it's current
+form without the feedback and suggestions from numerous individuals.
+We would like to specifically mention, in no particular order, the
+following people:
+
+ Russell King <rmk@arm.linux.org.uk>
+ Leo Dagum <dagum@barrel.engr.sgi.com>
+ Ralf Baechle <ralf@oss.sgi.com>
+ Grant Grundler <grundler@cup.hp.com>
+ Jay Estabrook <Jay.Estabrook@compaq.com>
+ Thomas Sailer <sailer@ife.ee.ethz.ch>
+ Andrea Arcangeli <andrea@suse.de>
+ Jens Axboe <jens.axboe@oracle.com>
+ David Mosberger-Tang <davidm@hpl.hp.com>
+++ /dev/null
- Dynamic DMA mapping Guide
- =========================
-
- David S. Miller <davem@redhat.com>
- Richard Henderson <rth@cygnus.com>
- Jakub Jelinek <jakub@redhat.com>
-
-This is a guide to device driver writers on how to use the DMA API
-with example pseudo-code. For a concise description of the API, see
-DMA-API.txt.
-
-Most of the 64bit platforms have special hardware that translates bus
-addresses (DMA addresses) into physical addresses. This is similar to
-how page tables and/or a TLB translates virtual addresses to physical
-addresses on a CPU. This is needed so that e.g. PCI devices can
-access with a Single Address Cycle (32bit DMA address) any page in the
-64bit physical address space. Previously in Linux those 64bit
-platforms had to set artificial limits on the maximum RAM size in the
-system, so that the virt_to_bus() static scheme works (the DMA address
-translation tables were simply filled on bootup to map each bus
-address to the physical page __pa(bus_to_virt())).
-
-So that Linux can use the dynamic DMA mapping, it needs some help from the
-drivers, namely it has to take into account that DMA addresses should be
-mapped only for the time they are actually used and unmapped after the DMA
-transfer.
-
-The following API will work of course even on platforms where no such
-hardware exists.
-
-Note that the DMA API works with any bus independent of the underlying
-microprocessor architecture. You should use the DMA API rather than
-the bus specific DMA API (e.g. pci_dma_*).
-
-First of all, you should make sure
-
-#include <linux/dma-mapping.h>
-
-is in your driver. This file will obtain for you the definition of the
-dma_addr_t (which can hold any valid DMA address for the platform)
-type which should be used everywhere you hold a DMA (bus) address
-returned from the DMA mapping functions.
-
- What memory is DMA'able?
-
-The first piece of information you must know is what kernel memory can
-be used with the DMA mapping facilities. There has been an unwritten
-set of rules regarding this, and this text is an attempt to finally
-write them down.
-
-If you acquired your memory via the page allocator
-(i.e. __get_free_page*()) or the generic memory allocators
-(i.e. kmalloc() or kmem_cache_alloc()) then you may DMA to/from
-that memory using the addresses returned from those routines.
-
-This means specifically that you may _not_ use the memory/addresses
-returned from vmalloc() for DMA. It is possible to DMA to the
-_underlying_ memory mapped into a vmalloc() area, but this requires
-walking page tables to get the physical addresses, and then
-translating each of those pages back to a kernel address using
-something like __va(). [ EDIT: Update this when we integrate
-Gerd Knorr's generic code which does this. ]
-
-This rule also means that you may use neither kernel image addresses
-(items in data/text/bss segments), nor module image addresses, nor
-stack addresses for DMA. These could all be mapped somewhere entirely
-different than the rest of physical memory. Even if those classes of
-memory could physically work with DMA, you'd need to ensure the I/O
-buffers were cacheline-aligned. Without that, you'd see cacheline
-sharing problems (data corruption) on CPUs with DMA-incoherent caches.
-(The CPU could write to one word, DMA would write to a different one
-in the same cache line, and one of them could be overwritten.)
-
-Also, this means that you cannot take the return of a kmap()
-call and DMA to/from that. This is similar to vmalloc().
-
-What about block I/O and networking buffers? The block I/O and
-networking subsystems make sure that the buffers they use are valid
-for you to DMA from/to.
-
- DMA addressing limitations
-
-Does your device have any DMA addressing limitations? For example, is
-your device only capable of driving the low order 24-bits of address?
-If so, you need to inform the kernel of this fact.
-
-By default, the kernel assumes that your device can address the full
-32-bits. For a 64-bit capable device, this needs to be increased.
-And for a device with limitations, as discussed in the previous
-paragraph, it needs to be decreased.
-
-Special note about PCI: PCI-X specification requires PCI-X devices to
-support 64-bit addressing (DAC) for all transactions. And at least
-one platform (SGI SN2) requires 64-bit consistent allocations to
-operate correctly when the IO bus is in PCI-X mode.
-
-For correct operation, you must interrogate the kernel in your device
-probe routine to see if the DMA controller on the machine can properly
-support the DMA addressing limitation your device has. It is good
-style to do this even if your device holds the default setting,
-because this shows that you did think about these issues wrt. your
-device.
-
-The query is performed via a call to dma_set_mask():
-
- int dma_set_mask(struct device *dev, u64 mask);
-
-The query for consistent allocations is performed via a call to
-dma_set_coherent_mask():
-
- int dma_set_coherent_mask(struct device *dev, u64 mask);
-
-Here, dev is a pointer to the device struct of your device, and mask
-is a bit mask describing which bits of an address your device
-supports. It returns zero if your card can perform DMA properly on
-the machine given the address mask you provided. In general, the
-device struct of your device is embedded in the bus specific device
-struct of your device. For example, a pointer to the device struct of
-your PCI device is pdev->dev (pdev is a pointer to the PCI device
-struct of your device).
-
-If it returns non-zero, your device cannot perform DMA properly on
-this platform, and attempting to do so will result in undefined
-behavior. You must either use a different mask, or not use DMA.
-
-This means that in the failure case, you have three options:
-
-1) Use another DMA mask, if possible (see below).
-2) Use some non-DMA mode for data transfer, if possible.
-3) Ignore this device and do not initialize it.
-
-It is recommended that your driver print a kernel KERN_WARNING message
-when you end up performing either #2 or #3. In this manner, if a user
-of your driver reports that performance is bad or that the device is not
-even detected, you can ask them for the kernel messages to find out
-exactly why.
-
-The standard 32-bit addressing device would do something like this:
-
- if (dma_set_mask(dev, DMA_BIT_MASK(32))) {
- printk(KERN_WARNING
- "mydev: No suitable DMA available.\n");
- goto ignore_this_device;
- }
-
-Another common scenario is a 64-bit capable device. The approach here
-is to try for 64-bit addressing, but back down to a 32-bit mask that
-should not fail. The kernel may fail the 64-bit mask not because the
-platform is not capable of 64-bit addressing. Rather, it may fail in
-this case simply because 32-bit addressing is done more efficiently
-than 64-bit addressing. For example, Sparc64 PCI SAC addressing is
-more efficient than DAC addressing.
-
-Here is how you would handle a 64-bit capable device which can drive
-all 64-bits when accessing streaming DMA:
-
- int using_dac;
-
- if (!dma_set_mask(dev, DMA_BIT_MASK(64))) {
- using_dac = 1;
- } else if (!dma_set_mask(dev, DMA_BIT_MASK(32))) {
- using_dac = 0;
- } else {
- printk(KERN_WARNING
- "mydev: No suitable DMA available.\n");
- goto ignore_this_device;
- }
-
-If a card is capable of using 64-bit consistent allocations as well,
-the case would look like this:
-
- int using_dac, consistent_using_dac;
-
- if (!dma_set_mask(dev, DMA_BIT_MASK(64))) {
- using_dac = 1;
- consistent_using_dac = 1;
- dma_set_coherent_mask(dev, DMA_BIT_MASK(64));
- } else if (!dma_set_mask(dev, DMA_BIT_MASK(32))) {
- using_dac = 0;
- consistent_using_dac = 0;
- dma_set_coherent_mask(dev, DMA_BIT_MASK(32));
- } else {
- printk(KERN_WARNING
- "mydev: No suitable DMA available.\n");
- goto ignore_this_device;
- }
-
-dma_set_coherent_mask() will always be able to set the same or a
-smaller mask as dma_set_mask(). However for the rare case that a
-device driver only uses consistent allocations, one would have to
-check the return value from dma_set_coherent_mask().
-
-Finally, if your device can only drive the low 24-bits of
-address you might do something like:
-
- if (dma_set_mask(dev, DMA_BIT_MASK(24))) {
- printk(KERN_WARNING
- "mydev: 24-bit DMA addressing not available.\n");
- goto ignore_this_device;
- }
-
-When dma_set_mask() is successful, and returns zero, the kernel saves
-away this mask you have provided. The kernel will use this
-information later when you make DMA mappings.
-
-There is a case which we are aware of at this time, which is worth
-mentioning in this documentation. If your device supports multiple
-functions (for example a sound card provides playback and record
-functions) and the various different functions have _different_
-DMA addressing limitations, you may wish to probe each mask and
-only provide the functionality which the machine can handle. It
-is important that the last call to dma_set_mask() be for the
-most specific mask.
-
-Here is pseudo-code showing how this might be done:
-
- #define PLAYBACK_ADDRESS_BITS DMA_BIT_MASK(32)
- #define RECORD_ADDRESS_BITS DMA_BIT_MASK(24)
-
- struct my_sound_card *card;
- struct device *dev;
-
- ...
- if (!dma_set_mask(dev, PLAYBACK_ADDRESS_BITS)) {
- card->playback_enabled = 1;
- } else {
- card->playback_enabled = 0;
- printk(KERN_WARNING "%s: Playback disabled due to DMA limitations.\n",
- card->name);
- }
- if (!dma_set_mask(dev, RECORD_ADDRESS_BITS)) {
- card->record_enabled = 1;
- } else {
- card->record_enabled = 0;
- printk(KERN_WARNING "%s: Record disabled due to DMA limitations.\n",
- card->name);
- }
-
-A sound card was used as an example here because this genre of PCI
-devices seems to be littered with ISA chips given a PCI front end,
-and thus retaining the 16MB DMA addressing limitations of ISA.
-
- Types of DMA mappings
-
-There are two types of DMA mappings:
-
-- Consistent DMA mappings which are usually mapped at driver
- initialization, unmapped at the end and for which the hardware should
- guarantee that the device and the CPU can access the data
- in parallel and will see updates made by each other without any
- explicit software flushing.
-
- Think of "consistent" as "synchronous" or "coherent".
-
- The current default is to return consistent memory in the low 32
- bits of the bus space. However, for future compatibility you should
- set the consistent mask even if this default is fine for your
- driver.
-
- Good examples of what to use consistent mappings for are:
-
- - Network card DMA ring descriptors.
- - SCSI adapter mailbox command data structures.
- - Device firmware microcode executed out of
- main memory.
-
- The invariant these examples all require is that any CPU store
- to memory is immediately visible to the device, and vice
- versa. Consistent mappings guarantee this.
-
- IMPORTANT: Consistent DMA memory does not preclude the usage of
- proper memory barriers. The CPU may reorder stores to
- consistent memory just as it may normal memory. Example:
- if it is important for the device to see the first word
- of a descriptor updated before the second, you must do
- something like:
-
- desc->word0 = address;
- wmb();
- desc->word1 = DESC_VALID;
-
- in order to get correct behavior on all platforms.
-
- Also, on some platforms your driver may need to flush CPU write
- buffers in much the same way as it needs to flush write buffers
- found in PCI bridges (such as by reading a register's value
- after writing it).
-
-- Streaming DMA mappings which are usually mapped for one DMA
- transfer, unmapped right after it (unless you use dma_sync_* below)
- and for which hardware can optimize for sequential accesses.
-
- This of "streaming" as "asynchronous" or "outside the coherency
- domain".
-
- Good examples of what to use streaming mappings for are:
-
- - Networking buffers transmitted/received by a device.
- - Filesystem buffers written/read by a SCSI device.
-
- The interfaces for using this type of mapping were designed in
- such a way that an implementation can make whatever performance
- optimizations the hardware allows. To this end, when using
- such mappings you must be explicit about what you want to happen.
-
-Neither type of DMA mapping has alignment restrictions that come from
-the underlying bus, although some devices may have such restrictions.
-Also, systems with caches that aren't DMA-coherent will work better
-when the underlying buffers don't share cache lines with other data.
-
-
- Using Consistent DMA mappings.
-
-To allocate and map large (PAGE_SIZE or so) consistent DMA regions,
-you should do:
-
- dma_addr_t dma_handle;
-
- cpu_addr = dma_alloc_coherent(dev, size, &dma_handle, gfp);
-
-where device is a struct device *. This may be called in interrupt
-context with the GFP_ATOMIC flag.
-
-Size is the length of the region you want to allocate, in bytes.
-
-This routine will allocate RAM for that region, so it acts similarly to
-__get_free_pages (but takes size instead of a page order). If your
-driver needs regions sized smaller than a page, you may prefer using
-the dma_pool interface, described below.
-
-The consistent DMA mapping interfaces, for non-NULL dev, will by
-default return a DMA address which is 32-bit addressable. Even if the
-device indicates (via DMA mask) that it may address the upper 32-bits,
-consistent allocation will only return > 32-bit addresses for DMA if
-the consistent DMA mask has been explicitly changed via
-dma_set_coherent_mask(). This is true of the dma_pool interface as
-well.
-
-dma_alloc_coherent returns two values: the virtual address which you
-can use to access it from the CPU and dma_handle which you pass to the
-card.
-
-The cpu return address and the DMA bus master address are both
-guaranteed to be aligned to the smallest PAGE_SIZE order which
-is greater than or equal to the requested size. This invariant
-exists (for example) to guarantee that if you allocate a chunk
-which is smaller than or equal to 64 kilobytes, the extent of the
-buffer you receive will not cross a 64K boundary.
-
-To unmap and free such a DMA region, you call:
-
- dma_free_coherent(dev, size, cpu_addr, dma_handle);
-
-where dev, size are the same as in the above call and cpu_addr and
-dma_handle are the values dma_alloc_coherent returned to you.
-This function may not be called in interrupt context.
-
-If your driver needs lots of smaller memory regions, you can write
-custom code to subdivide pages returned by dma_alloc_coherent,
-or you can use the dma_pool API to do that. A dma_pool is like
-a kmem_cache, but it uses dma_alloc_coherent not __get_free_pages.
-Also, it understands common hardware constraints for alignment,
-like queue heads needing to be aligned on N byte boundaries.
-
-Create a dma_pool like this:
-
- struct dma_pool *pool;
-
- pool = dma_pool_create(name, dev, size, align, alloc);
-
-The "name" is for diagnostics (like a kmem_cache name); dev and size
-are as above. The device's hardware alignment requirement for this
-type of data is "align" (which is expressed in bytes, and must be a
-power of two). If your device has no boundary crossing restrictions,
-pass 0 for alloc; passing 4096 says memory allocated from this pool
-must not cross 4KByte boundaries (but at that time it may be better to
-go for dma_alloc_coherent directly instead).
-
-Allocate memory from a dma pool like this:
-
- cpu_addr = dma_pool_alloc(pool, flags, &dma_handle);
-
-flags are SLAB_KERNEL if blocking is permitted (not in_interrupt nor
-holding SMP locks), SLAB_ATOMIC otherwise. Like dma_alloc_coherent,
-this returns two values, cpu_addr and dma_handle.
-
-Free memory that was allocated from a dma_pool like this:
-
- dma_pool_free(pool, cpu_addr, dma_handle);
-
-where pool is what you passed to dma_pool_alloc, and cpu_addr and
-dma_handle are the values dma_pool_alloc returned. This function
-may be called in interrupt context.
-
-Destroy a dma_pool by calling:
-
- dma_pool_destroy(pool);
-
-Make sure you've called dma_pool_free for all memory allocated
-from a pool before you destroy the pool. This function may not
-be called in interrupt context.
-
- DMA Direction
-
-The interfaces described in subsequent portions of this document
-take a DMA direction argument, which is an integer and takes on
-one of the following values:
-
- DMA_BIDIRECTIONAL
- DMA_TO_DEVICE
- DMA_FROM_DEVICE
- DMA_NONE
-
-One should provide the exact DMA direction if you know it.
-
-DMA_TO_DEVICE means "from main memory to the device"
-DMA_FROM_DEVICE means "from the device to main memory"
-It is the direction in which the data moves during the DMA
-transfer.
-
-You are _strongly_ encouraged to specify this as precisely
-as you possibly can.
-
-If you absolutely cannot know the direction of the DMA transfer,
-specify DMA_BIDIRECTIONAL. It means that the DMA can go in
-either direction. The platform guarantees that you may legally
-specify this, and that it will work, but this may be at the
-cost of performance for example.
-
-The value DMA_NONE is to be used for debugging. One can
-hold this in a data structure before you come to know the
-precise direction, and this will help catch cases where your
-direction tracking logic has failed to set things up properly.
-
-Another advantage of specifying this value precisely (outside of
-potential platform-specific optimizations of such) is for debugging.
-Some platforms actually have a write permission boolean which DMA
-mappings can be marked with, much like page protections in the user
-program address space. Such platforms can and do report errors in the
-kernel logs when the DMA controller hardware detects violation of the
-permission setting.
-
-Only streaming mappings specify a direction, consistent mappings
-implicitly have a direction attribute setting of
-DMA_BIDIRECTIONAL.
-
-The SCSI subsystem tells you the direction to use in the
-'sc_data_direction' member of the SCSI command your driver is
-working on.
-
-For Networking drivers, it's a rather simple affair. For transmit
-packets, map/unmap them with the DMA_TO_DEVICE direction
-specifier. For receive packets, just the opposite, map/unmap them
-with the DMA_FROM_DEVICE direction specifier.
-
- Using Streaming DMA mappings
-
-The streaming DMA mapping routines can be called from interrupt
-context. There are two versions of each map/unmap, one which will
-map/unmap a single memory region, and one which will map/unmap a
-scatterlist.
-
-To map a single region, you do:
-
- struct device *dev = &my_dev->dev;
- dma_addr_t dma_handle;
- void *addr = buffer->ptr;
- size_t size = buffer->len;
-
- dma_handle = dma_map_single(dev, addr, size, direction);
-
-and to unmap it:
-
- dma_unmap_single(dev, dma_handle, size, direction);
-
-You should call dma_unmap_single when the DMA activity is finished, e.g.
-from the interrupt which told you that the DMA transfer is done.
-
-Using cpu pointers like this for single mappings has a disadvantage,
-you cannot reference HIGHMEM memory in this way. Thus, there is a
-map/unmap interface pair akin to dma_{map,unmap}_single. These
-interfaces deal with page/offset pairs instead of cpu pointers.
-Specifically:
-
- struct device *dev = &my_dev->dev;
- dma_addr_t dma_handle;
- struct page *page = buffer->page;
- unsigned long offset = buffer->offset;
- size_t size = buffer->len;
-
- dma_handle = dma_map_page(dev, page, offset, size, direction);
-
- ...
-
- dma_unmap_page(dev, dma_handle, size, direction);
-
-Here, "offset" means byte offset within the given page.
-
-With scatterlists, you map a region gathered from several regions by:
-
- int i, count = dma_map_sg(dev, sglist, nents, direction);
- struct scatterlist *sg;
-
- for_each_sg(sglist, sg, count, i) {
- hw_address[i] = sg_dma_address(sg);
- hw_len[i] = sg_dma_len(sg);
- }
-
-where nents is the number of entries in the sglist.
-
-The implementation is free to merge several consecutive sglist entries
-into one (e.g. if DMA mapping is done with PAGE_SIZE granularity, any
-consecutive sglist entries can be merged into one provided the first one
-ends and the second one starts on a page boundary - in fact this is a huge
-advantage for cards which either cannot do scatter-gather or have very
-limited number of scatter-gather entries) and returns the actual number
-of sg entries it mapped them to. On failure 0 is returned.
-
-Then you should loop count times (note: this can be less than nents times)
-and use sg_dma_address() and sg_dma_len() macros where you previously
-accessed sg->address and sg->length as shown above.
-
-To unmap a scatterlist, just call:
-
- dma_unmap_sg(dev, sglist, nents, direction);
-
-Again, make sure DMA activity has already finished.
-
-PLEASE NOTE: The 'nents' argument to the dma_unmap_sg call must be
- the _same_ one you passed into the dma_map_sg call,
- it should _NOT_ be the 'count' value _returned_ from the
- dma_map_sg call.
-
-Every dma_map_{single,sg} call should have its dma_unmap_{single,sg}
-counterpart, because the bus address space is a shared resource (although
-in some ports the mapping is per each BUS so less devices contend for the
-same bus address space) and you could render the machine unusable by eating
-all bus addresses.
-
-If you need to use the same streaming DMA region multiple times and touch
-the data in between the DMA transfers, the buffer needs to be synced
-properly in order for the cpu and device to see the most uptodate and
-correct copy of the DMA buffer.
-
-So, firstly, just map it with dma_map_{single,sg}, and after each DMA
-transfer call either:
-
- dma_sync_single_for_cpu(dev, dma_handle, size, direction);
-
-or:
-
- dma_sync_sg_for_cpu(dev, sglist, nents, direction);
-
-as appropriate.
-
-Then, if you wish to let the device get at the DMA area again,
-finish accessing the data with the cpu, and then before actually
-giving the buffer to the hardware call either:
-
- dma_sync_single_for_device(dev, dma_handle, size, direction);
-
-or:
-
- dma_sync_sg_for_device(dev, sglist, nents, direction);
-
-as appropriate.
-
-After the last DMA transfer call one of the DMA unmap routines
-dma_unmap_{single,sg}. If you don't touch the data from the first dma_map_*
-call till dma_unmap_*, then you don't have to call the dma_sync_*
-routines at all.
-
-Here is pseudo code which shows a situation in which you would need
-to use the dma_sync_*() interfaces.
-
- my_card_setup_receive_buffer(struct my_card *cp, char *buffer, int len)
- {
- dma_addr_t mapping;
-
- mapping = dma_map_single(cp->dev, buffer, len, DMA_FROM_DEVICE);
-
- cp->rx_buf = buffer;
- cp->rx_len = len;
- cp->rx_dma = mapping;
-
- give_rx_buf_to_card(cp);
- }
-
- ...
-
- my_card_interrupt_handler(int irq, void *devid, struct pt_regs *regs)
- {
- struct my_card *cp = devid;
-
- ...
- if (read_card_status(cp) == RX_BUF_TRANSFERRED) {
- struct my_card_header *hp;
-
- /* Examine the header to see if we wish
- * to accept the data. But synchronize
- * the DMA transfer with the CPU first
- * so that we see updated contents.
- */
- dma_sync_single_for_cpu(&cp->dev, cp->rx_dma,
- cp->rx_len,
- DMA_FROM_DEVICE);
-
- /* Now it is safe to examine the buffer. */
- hp = (struct my_card_header *) cp->rx_buf;
- if (header_is_ok(hp)) {
- dma_unmap_single(&cp->dev, cp->rx_dma, cp->rx_len,
- DMA_FROM_DEVICE);
- pass_to_upper_layers(cp->rx_buf);
- make_and_setup_new_rx_buf(cp);
- } else {
- /* Just sync the buffer and give it back
- * to the card.
- */
- dma_sync_single_for_device(&cp->dev,
- cp->rx_dma,
- cp->rx_len,
- DMA_FROM_DEVICE);
- give_rx_buf_to_card(cp);
- }
- }
- }
-
-Drivers converted fully to this interface should not use virt_to_bus any
-longer, nor should they use bus_to_virt. Some drivers have to be changed a
-little bit, because there is no longer an equivalent to bus_to_virt in the
-dynamic DMA mapping scheme - you have to always store the DMA addresses
-returned by the dma_alloc_coherent, dma_pool_alloc, and dma_map_single
-calls (dma_map_sg stores them in the scatterlist itself if the platform
-supports dynamic DMA mapping in hardware) in your driver structures and/or
-in the card registers.
-
-All drivers should be using these interfaces with no exceptions. It
-is planned to completely remove virt_to_bus() and bus_to_virt() as
-they are entirely deprecated. Some ports already do not provide these
-as it is impossible to correctly support them.
-
- Optimizing Unmap State Space Consumption
-
-On many platforms, dma_unmap_{single,page}() is simply a nop.
-Therefore, keeping track of the mapping address and length is a waste
-of space. Instead of filling your drivers up with ifdefs and the like
-to "work around" this (which would defeat the whole purpose of a
-portable API) the following facilities are provided.
-
-Actually, instead of describing the macros one by one, we'll
-transform some example code.
-
-1) Use DEFINE_DMA_UNMAP_{ADDR,LEN} in state saving structures.
- Example, before:
-
- struct ring_state {
- struct sk_buff *skb;
- dma_addr_t mapping;
- __u32 len;
- };
-
- after:
-
- struct ring_state {
- struct sk_buff *skb;
- DEFINE_DMA_UNMAP_ADDR(mapping);
- DEFINE_DMA_UNMAP_LEN(len);
- };
-
-2) Use dma_unmap_{addr,len}_set to set these values.
- Example, before:
-
- ringp->mapping = FOO;
- ringp->len = BAR;
-
- after:
-
- dma_unmap_addr_set(ringp, mapping, FOO);
- dma_unmap_len_set(ringp, len, BAR);
-
-3) Use dma_unmap_{addr,len} to access these values.
- Example, before:
-
- dma_unmap_single(dev, ringp->mapping, ringp->len,
- DMA_FROM_DEVICE);
-
- after:
-
- dma_unmap_single(dev,
- dma_unmap_addr(ringp, mapping),
- dma_unmap_len(ringp, len),
- DMA_FROM_DEVICE);
-
-It really should be self-explanatory. We treat the ADDR and LEN
-separately, because it is possible for an implementation to only
-need the address in order to perform the unmap operation.
-
- Platform Issues
-
-If you are just writing drivers for Linux and do not maintain
-an architecture port for the kernel, you can safely skip down
-to "Closing".
-
-1) Struct scatterlist requirements.
-
- Struct scatterlist must contain, at a minimum, the following
- members:
-
- struct page *page;
- unsigned int offset;
- unsigned int length;
-
- The base address is specified by a "page+offset" pair.
-
- Previous versions of struct scatterlist contained a "void *address"
- field that was sometimes used instead of page+offset. As of Linux
- 2.5., page+offset is always used, and the "address" field has been
- deleted.
-
-2) More to come...
-
- Handling Errors
-
-DMA address space is limited on some architectures and an allocation
-failure can be determined by:
-
-- checking if dma_alloc_coherent returns NULL or dma_map_sg returns 0
-
-- checking the returned dma_addr_t of dma_map_single and dma_map_page
- by using dma_mapping_error():
-
- dma_addr_t dma_handle;
-
- dma_handle = dma_map_single(dev, addr, size, direction);
- if (dma_mapping_error(dev, dma_handle)) {
- /*
- * reduce current DMA mapping usage,
- * delay and try again later or
- * reset driver.
- */
- }
-
- Closing
-
-This document, and the API itself, would not be in it's current
-form without the feedback and suggestions from numerous individuals.
-We would like to specifically mention, in no particular order, the
-following people:
-
- Russell King <rmk@arm.linux.org.uk>
- Leo Dagum <dagum@barrel.engr.sgi.com>
- Ralf Baechle <ralf@oss.sgi.com>
- Grant Grundler <grundler@cup.hp.com>
- Jay Estabrook <Jay.Estabrook@compaq.com>
- Thomas Sailer <sailer@ife.ee.ethz.ch>
- Andrea Arcangeli <andrea@suse.de>
- Jens Axboe <jens.axboe@oracle.com>
- David Mosberger-Tang <davidm@hpl.hp.com>
5.3 swappiness
Similar to /proc/sys/vm/swappiness, but affecting a hierarchy of groups only.
- Following cgroups' swapiness can't be changed.
+ Following cgroups' swappiness can't be changed.
- root cgroup (uses /proc/sys/vm/swappiness).
- a cgroup which uses hierarchy and it has child cgroup.
- a cgroup which uses hierarchy and not the root of hierarchy.
--- /dev/null
+ ================
+ CIRCULAR BUFFERS
+ ================
+
+By: David Howells <dhowells@redhat.com>
+ Paul E. McKenney <paulmck@linux.vnet.ibm.com>
+
+
+Linux provides a number of features that can be used to implement circular
+buffering. There are two sets of such features:
+
+ (1) Convenience functions for determining information about power-of-2 sized
+ buffers.
+
+ (2) Memory barriers for when the producer and the consumer of objects in the
+ buffer don't want to share a lock.
+
+To use these facilities, as discussed below, there needs to be just one
+producer and just one consumer. It is possible to handle multiple producers by
+serialising them, and to handle multiple consumers by serialising them.
+
+
+Contents:
+
+ (*) What is a circular buffer?
+
+ (*) Measuring power-of-2 buffers.
+
+ (*) Using memory barriers with circular buffers.
+ - The producer.
+ - The consumer.
+
+
+==========================
+WHAT IS A CIRCULAR BUFFER?
+==========================
+
+First of all, what is a circular buffer? A circular buffer is a buffer of
+fixed, finite size into which there are two indices:
+
+ (1) A 'head' index - the point at which the producer inserts items into the
+ buffer.
+
+ (2) A 'tail' index - the point at which the consumer finds the next item in
+ the buffer.
+
+Typically when the tail pointer is equal to the head pointer, the buffer is
+empty; and the buffer is full when the head pointer is one less than the tail
+pointer.
+
+The head index is incremented when items are added, and the tail index when
+items are removed. The tail index should never jump the head index, and both
+indices should be wrapped to 0 when they reach the end of the buffer, thus
+allowing an infinite amount of data to flow through the buffer.
+
+Typically, items will all be of the same unit size, but this isn't strictly
+required to use the techniques below. The indices can be increased by more
+than 1 if multiple items or variable-sized items are to be included in the
+buffer, provided that neither index overtakes the other. The implementer must
+be careful, however, as a region more than one unit in size may wrap the end of
+the buffer and be broken into two segments.
+
+
+============================
+MEASURING POWER-OF-2 BUFFERS
+============================
+
+Calculation of the occupancy or the remaining capacity of an arbitrarily sized
+circular buffer would normally be a slow operation, requiring the use of a
+modulus (divide) instruction. However, if the buffer is of a power-of-2 size,
+then a much quicker bitwise-AND instruction can be used instead.
+
+Linux provides a set of macros for handling power-of-2 circular buffers. These
+can be made use of by:
+
+ #include <linux/circ_buf.h>
+
+The macros are:
+
+ (*) Measure the remaining capacity of a buffer:
+
+ CIRC_SPACE(head_index, tail_index, buffer_size);
+
+ This returns the amount of space left in the buffer[1] into which items
+ can be inserted.
+
+
+ (*) Measure the maximum consecutive immediate space in a buffer:
+
+ CIRC_SPACE_TO_END(head_index, tail_index, buffer_size);
+
+ This returns the amount of consecutive space left in the buffer[1] into
+ which items can be immediately inserted without having to wrap back to the
+ beginning of the buffer.
+
+
+ (*) Measure the occupancy of a buffer:
+
+ CIRC_CNT(head_index, tail_index, buffer_size);
+
+ This returns the number of items currently occupying a buffer[2].
+
+
+ (*) Measure the non-wrapping occupancy of a buffer:
+
+ CIRC_CNT_TO_END(head_index, tail_index, buffer_size);
+
+ This returns the number of consecutive items[2] that can be extracted from
+ the buffer without having to wrap back to the beginning of the buffer.
+
+
+Each of these macros will nominally return a value between 0 and buffer_size-1,
+however:
+
+ [1] CIRC_SPACE*() are intended to be used in the producer. To the producer
+ they will return a lower bound as the producer controls the head index,
+ but the consumer may still be depleting the buffer on another CPU and
+ moving the tail index.
+
+ To the consumer it will show an upper bound as the producer may be busy
+ depleting the space.
+
+ [2] CIRC_CNT*() are intended to be used in the consumer. To the consumer they
+ will return a lower bound as the consumer controls the tail index, but the
+ producer may still be filling the buffer on another CPU and moving the
+ head index.
+
+ To the producer it will show an upper bound as the consumer may be busy
+ emptying the buffer.
+
+ [3] To a third party, the order in which the writes to the indices by the
+ producer and consumer become visible cannot be guaranteed as they are
+ independent and may be made on different CPUs - so the result in such a
+ situation will merely be a guess, and may even be negative.
+
+
+===========================================
+USING MEMORY BARRIERS WITH CIRCULAR BUFFERS
+===========================================
+
+By using memory barriers in conjunction with circular buffers, you can avoid
+the need to:
+
+ (1) use a single lock to govern access to both ends of the buffer, thus
+ allowing the buffer to be filled and emptied at the same time; and
+
+ (2) use atomic counter operations.
+
+There are two sides to this: the producer that fills the buffer, and the
+consumer that empties it. Only one thing should be filling a buffer at any one
+time, and only one thing should be emptying a buffer at any one time, but the
+two sides can operate simultaneously.
+
+
+THE PRODUCER
+------------
+
+The producer will look something like this:
+
+ spin_lock(&producer_lock);
+
+ unsigned long head = buffer->head;
+ unsigned long tail = ACCESS_ONCE(buffer->tail);
+
+ if (CIRC_SPACE(head, tail, buffer->size) >= 1) {
+ /* insert one item into the buffer */
+ struct item *item = buffer[head];
+
+ produce_item(item);
+
+ smp_wmb(); /* commit the item before incrementing the head */
+
+ buffer->head = (head + 1) & (buffer->size - 1);
+
+ /* wake_up() will make sure that the head is committed before
+ * waking anyone up */
+ wake_up(consumer);
+ }
+
+ spin_unlock(&producer_lock);
+
+This will instruct the CPU that the contents of the new item must be written
+before the head index makes it available to the consumer and then instructs the
+CPU that the revised head index must be written before the consumer is woken.
+
+Note that wake_up() doesn't have to be the exact mechanism used, but whatever
+is used must guarantee a (write) memory barrier between the update of the head
+index and the change of state of the consumer, if a change of state occurs.
+
+
+THE CONSUMER
+------------
+
+The consumer will look something like this:
+
+ spin_lock(&consumer_lock);
+
+ unsigned long head = ACCESS_ONCE(buffer->head);
+ unsigned long tail = buffer->tail;
+
+ if (CIRC_CNT(head, tail, buffer->size) >= 1) {
+ /* read index before reading contents at that index */
+ smp_read_barrier_depends();
+
+ /* extract one item from the buffer */
+ struct item *item = buffer[tail];
+
+ consume_item(item);
+
+ smp_mb(); /* finish reading descriptor before incrementing tail */
+
+ buffer->tail = (tail + 1) & (buffer->size - 1);
+ }
+
+ spin_unlock(&consumer_lock);
+
+This will instruct the CPU to make sure the index is up to date before reading
+the new item, and then it shall make sure the CPU has finished reading the item
+before it writes the new tail pointer, which will erase the item.
+
+
+Note the use of ACCESS_ONCE() in both algorithms to read the opposition index.
+This prevents the compiler from discarding and reloading its cached value -
+which some compilers will do across smp_read_barrier_depends(). This isn't
+strictly needed if you can be sure that the opposition index will _only_ be
+used the once.
+
+
+===============
+FURTHER READING
+===============
+
+See also Documentation/memory-barriers.txt for a description of Linux's memory
+barrier facilities.
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <linux/timer.h>
#include <linux/connector.h>
- information about the BeOS filesystem for Linux.
bfs.txt
- info for the SCO UnixWare Boot Filesystem (BFS).
+ceph.txt
+ - info for the Ceph Distributed File System
cifs.txt
- description of the CIFS filesystem.
coda.txt
mount -t 9p `namespace`/acme /mnt/9 -o trans=unix,uname=$USER
+For server running on QEMU host with virtio transport:
+
+ mount -t 9p -o trans=virtio <mount_tag> /mnt/9
+
+where mount_tag is the tag associated by the server to each of the exported
+mount points. Each 9P export is seen by the client as a virtio device with an
+associated "mount_tag" property. Available mount tags can be
+seen by reading /sys/bus/virtio/drivers/9pnet_virtio/virtio<n>/mount_tag files.
+
OPTIONS
=======
fd - used passed file descriptors for connection
(see rfdno and wfdno)
virtio - connect to the next virtio channel available
- (from lguest or KVM with trans_virtio module)
+ (from QEMU with trans_virtio module)
rdma - connect to a specified RDMA channel
uname=name user name to attempt mount as on the remote server. The
port=n port to connect to on the remote server
- noextend force legacy mode (no 9p2000.u semantics)
+ noextend force legacy mode (no 9p2000.u or 9p2000.L semantics)
+
+ version=name Select 9P protocol version. Valid options are:
+ 9p2000 - Legacy mode (same as noextend)
+ 9p2000.u - Use 9P2000.u protocol
+ 9p2000.L - Use 9P2000.L protocol
dfltuid attempt to mount as a particular uid
--- /dev/null
+Ceph Distributed File System
+============================
+
+Ceph is a distributed network file system designed to provide good
+performance, reliability, and scalability.
+
+Basic features include:
+
+ * POSIX semantics
+ * Seamless scaling from 1 to many thousands of nodes
+ * High availability and reliability. No single point of failure.
+ * N-way replication of data across storage nodes
+ * Fast recovery from node failures
+ * Automatic rebalancing of data on node addition/removal
+ * Easy deployment: most FS components are userspace daemons
+
+Also,
+ * Flexible snapshots (on any directory)
+ * Recursive accounting (nested files, directories, bytes)
+
+In contrast to cluster filesystems like GFS, OCFS2, and GPFS that rely
+on symmetric access by all clients to shared block devices, Ceph
+separates data and metadata management into independent server
+clusters, similar to Lustre. Unlike Lustre, however, metadata and
+storage nodes run entirely as user space daemons. Storage nodes
+utilize btrfs to store data objects, leveraging its advanced features
+(checksumming, metadata replication, etc.). File data is striped
+across storage nodes in large chunks to distribute workload and
+facilitate high throughputs. When storage nodes fail, data is
+re-replicated in a distributed fashion by the storage nodes themselves
+(with some minimal coordination from a cluster monitor), making the
+system extremely efficient and scalable.
+
+Metadata servers effectively form a large, consistent, distributed
+in-memory cache above the file namespace that is extremely scalable,
+dynamically redistributes metadata in response to workload changes,
+and can tolerate arbitrary (well, non-Byzantine) node failures. The
+metadata server takes a somewhat unconventional approach to metadata
+storage to significantly improve performance for common workloads. In
+particular, inodes with only a single link are embedded in
+directories, allowing entire directories of dentries and inodes to be
+loaded into its cache with a single I/O operation. The contents of
+extremely large directories can be fragmented and managed by
+independent metadata servers, allowing scalable concurrent access.
+
+The system offers automatic data rebalancing/migration when scaling
+from a small cluster of just a few nodes to many hundreds, without
+requiring an administrator carve the data set into static volumes or
+go through the tedious process of migrating data between servers.
+When the file system approaches full, new nodes can be easily added
+and things will "just work."
+
+Ceph includes flexible snapshot mechanism that allows a user to create
+a snapshot on any subdirectory (and its nested contents) in the
+system. Snapshot creation and deletion are as simple as 'mkdir
+.snap/foo' and 'rmdir .snap/foo'.
+
+Ceph also provides some recursive accounting on directories for nested
+files and bytes. That is, a 'getfattr -d foo' on any directory in the
+system will reveal the total number of nested regular files and
+subdirectories, and a summation of all nested file sizes. This makes
+the identification of large disk space consumers relatively quick, as
+no 'du' or similar recursive scan of the file system is required.
+
+
+Mount Syntax
+============
+
+The basic mount syntax is:
+
+ # mount -t ceph monip[:port][,monip2[:port]...]:/[subdir] mnt
+
+You only need to specify a single monitor, as the client will get the
+full list when it connects. (However, if the monitor you specify
+happens to be down, the mount won't succeed.) The port can be left
+off if the monitor is using the default. So if the monitor is at
+1.2.3.4,
+
+ # mount -t ceph 1.2.3.4:/ /mnt/ceph
+
+is sufficient. If /sbin/mount.ceph is installed, a hostname can be
+used instead of an IP address.
+
+
+
+Mount Options
+=============
+
+ ip=A.B.C.D[:N]
+ Specify the IP and/or port the client should bind to locally.
+ There is normally not much reason to do this. If the IP is not
+ specified, the client's IP address is determined by looking at the
+ address it's connection to the monitor originates from.
+
+ wsize=X
+ Specify the maximum write size in bytes. By default there is no
+ maximum. Ceph will normally size writes based on the file stripe
+ size.
+
+ rsize=X
+ Specify the maximum readahead.
+
+ mount_timeout=X
+ Specify the timeout value for mount (in seconds), in the case
+ of a non-responsive Ceph file system. The default is 30
+ seconds.
+
+ rbytes
+ When stat() is called on a directory, set st_size to 'rbytes',
+ the summation of file sizes over all files nested beneath that
+ directory. This is the default.
+
+ norbytes
+ When stat() is called on a directory, set st_size to the
+ number of entries in that directory.
+
+ nocrc
+ Disable CRC32C calculation for data writes. If set, the storage node
+ must rely on TCP's error correction to detect data corruption
+ in the data payload.
+
+ noasyncreaddir
+ Disable client's use its local cache to satisfy readdir
+ requests. (This does not change correctness; the client uses
+ cached metadata only when a lease or capability ensures it is
+ valid.)
+
+
+More Information
+================
+
+For more information on Ceph, see the home page at
+ http://ceph.newdream.net/
+
+The Linux kernel client source tree is available at
+ git://ceph.newdream.net/git/ceph-client.git
+ git://git.kernel.org/pub/scm/linux/kernel/git/sage/ceph-client.git
+
+and the source for the full system is at
+ git://ceph.newdream.net/git/ceph.git
all files in that instance (if CONFIG_NUMA is enabled) - which can be
adjusted on the fly via 'mount -o remount ...'
-mpol=default prefers to allocate memory from the local node
+mpol=default use the process allocation policy
+ (see set_mempolicy(2))
mpol=prefer:Node prefers to allocate memory from the given Node
mpol=bind:NodeList allocates memory only from nodes in NodeList
mpol=interleave prefers to allocate from each node in turn
mpol=interleave:NodeList allocates from each node of NodeList in turn
+mpol=local prefers to allocate memory from the local node
NodeList format is a comma-separated list of decimal numbers and ranges,
a range being two hyphen-separated decimal numbers, the smallest and
Christoph Rohland <cr@sap.com>, 1.12.01
Updated:
Hugh Dickins, 4 June 2007
+Updated:
+ KOSAKI Motohiro, 16 Mar 2010
0x92 00-0F drivers/usb/mon/mon_bin.c
0x93 60-7F linux/auto_fs.h
0x94 all fs/btrfs/ioctl.h
+0x97 00-7F fs/ceph/ioctl.h Ceph file system
0x99 00-0F 537-Addinboard driver
<mailto:buk@buks.ipn.de>
0xA0 all linux/sdp/sdp.h Industrial Device Project
direct expression of inheritance, so other techniques - such as structure
embedding - must be used.
-So, for example, the UIO code has a structure that defines the memory
-region associated with a uio device:
+(As an aside, for those familiar with the kernel linked list implementation,
+this is analogous as to how "list_head" structs are rarely useful on
+their own, but are invariably found embedded in the larger objects of
+interest.)
-struct uio_mem {
+So, for example, the UIO code in drivers/uio/uio.c has a structure that
+defines the memory region associated with a uio device:
+
+ struct uio_map {
struct kobject kobj;
- unsigned long addr;
- unsigned long size;
- int memtype;
- void __iomem *internal_addr;
-};
+ struct uio_mem *mem;
+ };
-If you have a struct uio_mem structure, finding its embedded kobject is
+If you have a struct uio_map structure, finding its embedded kobject is
just a matter of using the kobj member. Code that works with kobjects will
often have the opposite problem, however: given a struct kobject pointer,
what is the pointer to the containing structure? You must avoid tricks
(such as assuming that the kobject is at the beginning of the structure)
and, instead, use the container_of() macro, found in <linux/kernel.h>:
- container_of(pointer, type, member)
+ container_of(pointer, type, member)
+
+where:
+
+ * "pointer" is the pointer to the embedded kobject,
+ * "type" is the type of the containing structure, and
+ * "member" is the name of the structure field to which "pointer" points.
+
+The return value from container_of() is a pointer to the corresponding
+container type. So, for example, a pointer "kp" to a struct kobject
+embedded *within* a struct uio_map could be converted to a pointer to the
+*containing* uio_map structure with:
+
+ struct uio_map *u_map = container_of(kp, struct uio_map, kobj);
+
+For convenience, programmers often define a simple macro for "back-casting"
+kobject pointers to the containing type. Exactly this happens in the
+earlier drivers/uio/uio.c, as you can see here:
+
+ struct uio_map {
+ struct kobject kobj;
+ struct uio_mem *mem;
+ };
-where pointer is the pointer to the embedded kobject, type is the type of
-the containing structure, and member is the name of the structure field to
-which pointer points. The return value from container_of() is a pointer to
-the given type. So, for example, a pointer "kp" to a struct kobject
-embedded within a struct uio_mem could be converted to a pointer to the
-containing uio_mem structure with:
+ #define to_map(map) container_of(map, struct uio_map, kobj)
- struct uio_mem *u_mem = container_of(kp, struct uio_mem, kobj);
+where the macro argument "map" is a pointer to the struct kobject in
+question. That macro is subsequently invoked with:
-Programmers often define a simple macro for "back-casting" kobject pointers
-to the containing type.
+ struct uio_map *map = to_map(kobj);
Initialization of kobjects
Example code to copy from
For a more complete example of using ksets and kobjects properly, see the
-sample/kobject/kset-example.c code.
+example programs samples/kobject/{kobject-example.c,kset-example.c},
+which will be built as loadable modules if you select CONFIG_SAMPLE_KOBJECT.
============================
By: David Howells <dhowells@redhat.com>
+ Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Contents:
- And then there's the Alpha.
+ (*) Example uses.
+
+ - Circular buffers.
+
(*) References.
See the subsection on "Cache Coherency" above.
+============
+EXAMPLE USES
+============
+
+CIRCULAR BUFFERS
+----------------
+
+Memory barriers can be used to implement circular buffering without the need
+of a lock to serialise the producer with the consumer. See:
+
+ Documentation/circular-buffers.txt
+
+for details.
+
+
==========
REFERENCES
==========
--- /dev/null
+ STMicroelectronics 10/100/1000 Synopsys Ethernet driver
+
+Copyright (C) 2007-2010 STMicroelectronics Ltd
+Author: Giuseppe Cavallaro <peppe.cavallaro@st.com>
+
+This is the driver for the MAC 10/100/1000 on-chip Ethernet controllers
+(Synopsys IP blocks); it has been fully tested on STLinux platforms.
+
+Currently this network device driver is for all STM embedded MAC/GMAC
+(7xxx SoCs).
+
+DWC Ether MAC 10/100/1000 Universal version 3.41a and DWC Ether MAC 10/100
+Universal version 4.0 have been used for developing the first code
+implementation.
+
+Please, for more information also visit: www.stlinux.com
+
+1) Kernel Configuration
+The kernel configuration option is STMMAC_ETH:
+ Device Drivers ---> Network device support ---> Ethernet (1000 Mbit) --->
+ STMicroelectronics 10/100/1000 Ethernet driver (STMMAC_ETH)
+
+2) Driver parameters list:
+ debug: message level (0: no output, 16: all);
+ phyaddr: to manually provide the physical address to the PHY device;
+ dma_rxsize: DMA rx ring size;
+ dma_txsize: DMA tx ring size;
+ buf_sz: DMA buffer size;
+ tc: control the HW FIFO threshold;
+ tx_coe: Enable/Disable Tx Checksum Offload engine;
+ watchdog: transmit timeout (in milliseconds);
+ flow_ctrl: Flow control ability [on/off];
+ pause: Flow Control Pause Time;
+ tmrate: timer period (only if timer optimisation is configured).
+
+3) Command line options
+Driver parameters can be also passed in command line by using:
+ stmmaceth=dma_rxsize:128,dma_txsize:512
+
+4) Driver information and notes
+
+4.1) Transmit process
+The xmit method is invoked when the kernel needs to transmit a packet; it sets
+the descriptors in the ring and informs the DMA engine that there is a packet
+ready to be transmitted.
+Once the controller has finished transmitting the packet, an interrupt is
+triggered; So the driver will be able to release the socket buffers.
+By default, the driver sets the NETIF_F_SG bit in the features field of the
+net_device structure enabling the scatter/gather feature.
+
+4.2) Receive process
+When one or more packets are received, an interrupt happens. The interrupts
+are not queued so the driver has to scan all the descriptors in the ring during
+the receive process.
+This is based on NAPI so the interrupt handler signals only if there is work to be
+done, and it exits.
+Then the poll method will be scheduled at some future point.
+The incoming packets are stored, by the DMA, in a list of pre-allocated socket
+buffers in order to avoid the memcpy (Zero-copy).
+
+4.3) Timer-Driver Interrupt
+Instead of having the device that asynchronously notifies the frame receptions, the
+driver configures a timer to generate an interrupt at regular intervals.
+Based on the granularity of the timer, the frames that are received by the device
+will experience different levels of latency. Some NICs have dedicated timer
+device to perform this task. STMMAC can use either the RTC device or the TMU
+channel 2 on STLinux platforms.
+The timers frequency can be passed to the driver as parameter; when change it,
+take care of both hardware capability and network stability/performance impact.
+Several performance tests on STM platforms showed this optimisation allows to spare
+the CPU while having the maximum throughput.
+
+4.4) WOL
+Wake up on Lan feature through Magic Frame is only supported for the GMAC
+core.
+
+4.5) DMA descriptors
+Driver handles both normal and enhanced descriptors. The latter has been only
+tested on DWC Ether MAC 10/100/1000 Universal version 3.41a.
+
+4.6) Ethtool support
+Ethtool is supported. Driver statistics and internal errors can be taken using:
+ethtool -S ethX command. It is possible to dump registers etc.
+
+4.7) Jumbo and Segmentation Offloading
+Jumbo frames are supported and tested for the GMAC.
+The GSO has been also added but it's performed in software.
+LRO is not supported.
+
+4.8) Physical
+The driver is compatible with PAL to work with PHY and GPHY devices.
+
+4.9) Platform information
+Several information came from the platform; please refer to the
+driver's Header file in include/linux directory.
+
+struct plat_stmmacenet_data {
+ int bus_id;
+ int pbl;
+ int has_gmac;
+ void (*fix_mac_speed)(void *priv, unsigned int speed);
+ void (*bus_setup)(unsigned long ioaddr);
+#ifdef CONFIG_STM_DRIVERS
+ struct stm_pad_config *pad_config;
+#endif
+ void *bsp_priv;
+};
+
+Where:
+- pbl (Programmable Burst Length) is maximum number of
+ beats to be transferred in one DMA transaction.
+ GMAC also enables the 4xPBL by default.
+- fix_mac_speed and bus_setup are used to configure internal target
+ registers (on STM platforms);
+- has_gmac: GMAC core is on board (get it at run-time in the next step);
+- bus_id: bus identifier.
+
+struct plat_stmmacphy_data {
+ int bus_id;
+ int phy_addr;
+ unsigned int phy_mask;
+ int interface;
+ int (*phy_reset)(void *priv);
+ void *priv;
+};
+
+Where:
+- bus_id: bus identifier;
+- phy_addr: physical address used for the attached phy device;
+ set it to -1 to get it at run-time;
+- interface: physical MII interface mode;
+- phy_reset: hook to reset HW function.
+
+TODO:
+- Continue to make the driver more generic and suitable for other Synopsys
+ Ethernet controllers used on other architectures (i.e. ARM).
+- 10G controllers are not supported.
+- MAC uses Normal descriptors and GMAC uses enhanced ones.
+ This is a limit that should be reviewed. MAC could want to
+ use the enhanced structure.
+- Checksumming: Rx/Tx csum is done in HW in case of GMAC only.
+- Review the timer optimisation code to use an embedded device that seems to be
+ available in new chip generations.
- fsl,qe-num-snums: define how many serial number(SNUM) the QE can use for the
threads.
+Optional properties:
+- fsl,firmware-phandle:
+ Usage: required only if there is no fsl,qe-firmware child node
+ Value type: <phandle>
+ Definition: Points to a firmware node (see "QE Firmware Node" below)
+ that contains the firmware that should be uploaded for this QE.
+ The compatible property for the firmware node should say,
+ "fsl,qe-firmware".
+
Recommended properties
- brg-frequency : the internal clock source frequency for baud-rate
generators in Hz.
reg = <0 c000>;
};
};
+
+* QE Firmware Node
+
+This node defines a firmware binary that is embedded in the device tree, for
+the purpose of passing the firmware from bootloader to the kernel, or from
+the hypervisor to the guest.
+
+The firmware node itself contains the firmware binary contents, a compatible
+property, and any firmware-specific properties. The node should be placed
+inside a QE node that needs it. Doing so eliminates the need for a
+fsl,firmware-phandle property. Other QE nodes that need the same firmware
+should define an fsl,firmware-phandle property that points to the firmware node
+in the first QE node.
+
+The fsl,firmware property can be specified in the DTS (possibly using incbin)
+or can be inserted by the boot loader at boot time.
+
+Required properties:
+ - compatible
+ Usage: required
+ Value type: <string>
+ Definition: A standard property. Specify a string that indicates what
+ kind of firmware it is. For QE, this should be "fsl,qe-firmware".
+
+ - fsl,firmware
+ Usage: required
+ Value type: <prop-encoded-array>, encoded as an array of bytes
+ Definition: A standard property. This property contains the firmware
+ binary "blob".
+
+Example:
+ qe1@e0080000 {
+ compatible = "fsl,qe";
+ qe_firmware:qe-firmware {
+ compatible = "fsl,qe-firmware";
+ fsl,firmware = [0x70 0xcd 0x00 0x00 0x01 0x46 0x45 ...];
+ };
+ ...
+ };
+
+ qe2@e0090000 {
+ compatible = "fsl,qe";
+ fsl,firmware-phandle = <&qe_firmware>;
+ ...
+ };
cpu_relax();
The cpu_relax() call can lower CPU power consumption or yield to a
-hyperthreaded twin processor; it also happens to serve as a memory barrier,
-so, once again, volatile is unnecessary. Of course, busy-waiting is
-generally an anti-social act to begin with.
+hyperthreaded twin processor; it also happens to serve as a compiler
+barrier, so, once again, volatile is unnecessary. Of course, busy-
+waiting is generally an anti-social act to begin with.
There are still a few rare situations where volatile makes sense in the
kernel:
S: Maintained
ARM/NOMADIK ARCHITECTURE
-M: Alessandro Rubini <rubini@unipv.it>
-M: STEricsson <STEricsson_nomadik_linux@list.st.com>
-L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
-S: Maintained
-F: arch/arm/mach-nomadik/
-F: arch/arm/plat-nomadik/
+M: Alessandro Rubini <rubini@unipv.it>
+M: STEricsson <STEricsson_nomadik_linux@list.st.com>
+L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
+S: Maintained
+F: arch/arm/mach-nomadik/
+F: arch/arm/plat-nomadik/
ARM/OPENMOKO NEO FREERUNNER (GTA02) MACHINE SUPPORT
M: Nelson Castillo <arhuaco@freaks-unidos.net>
F: arch/powerpc/oprofile/*cell*
F: arch/powerpc/platforms/cell/
+CEPH DISTRIBUTED FILE SYSTEM CLIENT
+M: Sage Weil <sage@newdream.net>
+L: ceph-devel@vger.kernel.org
+W: http://ceph.newdream.net/
+T: git git://git.kernel.org/pub/scm/linux/kernel/git/sage/ceph-client.git
+S: Supported
+F: Documentation/filesystems/ceph.txt
+F: fs/ceph
+
CERTIFIED WIRELESS USB (WUSB) SUBSYSTEM:
M: David Vrabel <david.vrabel@csr.com>
L: linux-usb@vger.kernel.org
F: drivers/scsi/dpt/
DRBD DRIVER
-P: Philipp Reisner
-P: Lars Ellenberg
-M: drbd-dev@lists.linbit.com
-L: drbd-user@lists.linbit.com
-W: http://www.drbd.org
-T: git git://git.drbd.org/linux-2.6-drbd.git drbd
-T: git git://git.drbd.org/drbd-8.3.git
-S: Supported
-F: drivers/block/drbd/
-F: lib/lru_cache.c
-F: Documentation/blockdev/drbd/
+P: Philipp Reisner
+P: Lars Ellenberg
+M: drbd-dev@lists.linbit.com
+L: drbd-user@lists.linbit.com
+W: http://www.drbd.org
+T: git git://git.drbd.org/linux-2.6-drbd.git drbd
+T: git git://git.drbd.org/drbd-8.3.git
+S: Supported
+F: drivers/block/drbd/
+F: lib/lru_cache.c
+F: Documentation/blockdev/drbd/
DRIVER CORE, KOBJECTS, AND SYSFS
M: Greg Kroah-Hartman <gregkh@suse.de>
ISDN SUBSYSTEM
M: Karsten Keil <isdn@linux-pingi.de>
L: isdn4linux@listserv.isdn4linux.de (subscribers-only)
+L: netdev@vger.kernel.org
W: http://www.isdn4linux.de
T: git git://git.kernel.org/pub/scm/linux/kernel/git/kkeil/isdn-2.6.git
S: Maintained
F: include/linux/kexec.h
F: kernel/kexec.c
+KEYS/KEYRINGS:
+M: David Howells <dhowells@redhat.com>
+L: keyrings@linux-nfs.org
+S: Maintained
+F: Documentation/keys.txt
+F: include/linux/key.h
+F: include/linux/key-type.h
+F: include/keys/
+F: security/keys/
+
KGDB
M: Jason Wessel <jason.wessel@windriver.com>
L: kgdb-bugreport@lists.sourceforge.net
LTP (Linux Test Project)
M: Rishikesh K Rajak <risrajak@linux.vnet.ibm.com>
M: Garrett Cooper <yanegomi@gmail.com>
-M: Mike Frysinger <vapier@gentoo.org>
-M: Subrata Modak <subrata@linux.vnet.ibm.com>
+M: Mike Frysinger <vapier@gentoo.org>
+M: Subrata Modak <subrata@linux.vnet.ibm.com>
L: ltp-list@lists.sourceforge.net (subscribers-only)
W: http://ltp.sourceforge.net/
T: git git://git.kernel.org/pub/scm/linux/kernel/git/galak/ltp.git
F: sound/soc/codecs/twl4030*
TIPC NETWORK LAYER
-M: Per Liden <per.liden@ericsson.com>
M: Jon Maloy <jon.maloy@ericsson.com>
M: Allan Stephens <allan.stephens@windriver.com>
L: tipc-discussion@lists.sourceforge.net
X86 PLATFORM DRIVERS
M: Matthew Garrett <mjg@redhat.com>
L: platform-driver-x86@vger.kernel.org
-T: git git://git.kernel.org/pub/scm/linux/kernel/git/mjg59/platform-drivers-x86.git
+T: git git://git.kernel.org/pub/scm/linux/kernel/git/mjg59/platform-drivers-x86.git
S: Maintained
F: drivers/platform/x86
VERSION = 2
PATCHLEVEL = 6
SUBLEVEL = 34
-EXTRAVERSION = -rc1
+EXTRAVERSION = -rc3
NAME = Man-Eating Seals of Antiquity
# *DOCUMENTATION*
* based significantly on the arch/alpha/boot/main.c of Linus Torvalds
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include <generated/utsrelease.h>
#include <linux/mm.h>
* and the decompression code from MILO.
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include <generated/utsrelease.h>
#include <linux/mm.h>
* This file is the bootloader for the Linux/AXP kernel
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include <generated/utsrelease.h>
#include <linux/mm.h>
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#define __ALPHA_MARVEL__H__
#include <linux/types.h>
-#include <linux/pci.h>
#include <linux/spinlock.h>
#include <asm/compiler.h>
#define MCPCIA_ONE_HAE_WINDOW 1
#include <linux/types.h>
-#include <linux/pci.h>
#include <asm/compiler.h>
/*
#define __ALPHA_TITAN__H__
#include <linux/types.h>
-#include <linux/pci.h>
#include <asm/compiler.h>
/*
#define __ALPHA_TSUNAMI__H__
#include <linux/types.h>
-#include <linux/pci.h>
#include <asm/compiler.h>
/*
#include <linux/sched.h>
#include <linux/ptrace.h>
#include <linux/interrupt.h>
-#include <linux/slab.h>
#include <linux/random.h>
#include <linux/init.h>
#include <linux/irq.h>
#include <linux/syscalls.h>
#include <linux/unistd.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/user.h>
#include <linux/utsname.h>
#include <linux/time.h>
#include <linux/uio.h>
#include <linux/vfs.h>
#include <linux/rcupdate.h>
+#include <linux/slab.h>
#include <asm/fpu.h>
#include <asm/io.h>
#include <linux/pci.h>
#include <linux/init.h>
#include <linux/bootmem.h>
+#include <linux/gfp.h>
#include <linux/capability.h>
#include <linux/mm.h>
#include <linux/errno.h>
*/
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/pci.h>
static int hose_mmap_page_range(struct pci_controller *hose,
#include <linux/kernel.h>
#include <linux/mm.h>
#include <linux/pci.h>
-#include <linux/slab.h>
+#include <linux/gfp.h>
#include <linux/bootmem.h>
#include <linux/scatterlist.h>
#include <linux/log2.h>
#include <linux/stddef.h>
#include <linux/unistd.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/user.h>
#include <linux/time.h>
#include <linux/major.h>
#include <linux/reboot.h>
#include <linux/tty.h>
#include <linux/console.h>
+#include <linux/slab.h>
#include <asm/reg.h>
#include <asm/uaccess.h>
#include <linux/errno.h>
#include <linux/ptrace.h>
#include <linux/user.h>
-#include <linux/slab.h>
#include <linux/security.h>
#include <linux/signal.h>
*/
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/init.h>
#include <linux/delay.h>
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/init.h>
#include <linux/delay.h>
*/
#include <linux/kernel.h>
+#include <linux/gfp.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/proc_fs.h>
dp264_device_interrupt(unsigned long vector)
{
#if 1
- printk("dp264_device_interrupt: NOT IMPLEMENTED YET!! \n");
+ printk("dp264_device_interrupt: NOT IMPLEMENTED YET!!\n");
#else
unsigned long pld;
unsigned int i;
static void
titan_device_interrupt(unsigned long vector)
{
- printk("titan_device_interrupt: NOT IMPLEMENTED YET!! \n");
+ printk("titan_device_interrupt: NOT IMPLEMENTED YET!!\n");
}
static void
#include <linux/module.h>
#include <linux/init.h>
#include <linux/kallsyms.h>
+#include <linux/ratelimit.h>
#include <asm/gentrap.h>
#include <asm/uaccess.h>
do_entUnaUser(void __user * va, unsigned long opcode,
unsigned long reg, struct pt_regs *regs)
{
- static int cnt = 0;
- static unsigned long last_time;
+ static DEFINE_RATELIMIT_STATE(ratelimit, 5 * HZ, 5);
unsigned long tmp1, tmp2, tmp3, tmp4;
unsigned long fake_reg, *reg_addr = &fake_reg;
with the unaliged access. */
if (!test_thread_flag (TIF_UAC_NOPRINT)) {
- if (cnt >= 5 && time_after(jiffies, last_time + 5 * HZ)) {
- cnt = 0;
- }
- if (++cnt < 5) {
+ if (__ratelimit(&ratelimit)) {
printk("%s(%d): unaligned trap at %016lx: %p %lx %ld\n",
current->comm, task_pid_nr(current),
regs->pc - 4, va, opcode, reg);
}
- last_time = jiffies;
}
if (test_thread_flag (TIF_UAC_SIGBUS))
goto give_sigbus;
#include <linux/init.h>
#include <linux/bootmem.h> /* max_low_pfn */
#include <linux/vmalloc.h>
+#include <linux/gfp.h>
#include <asm/system.h>
#include <asm/uaccess.h>
#include <linux/string.h>
#include <linux/mutex.h>
#include <linux/clk.h>
+#include <linux/slab.h>
#include <asm/clkdev.h>
#include <mach/clkdev.h>
#include <linux/ptrace.h>
#include <linux/interrupt.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/init.h>
#include <linux/ioport.h>
#include <linux/irq.h>
save->LCM_GPO = locomo_readl(lchip->base + LOCOMO_GPO); /* GPIO */
locomo_writel(0x00, lchip->base + LOCOMO_GPO);
save->LCM_SPICT = locomo_readl(lchip->base + LOCOMO_SPI + LOCOMO_SPICT); /* SPI */
- locomo_writel(0x40, lchip->base + LOCOMO_SPICT);
+ locomo_writel(0x40, lchip->base + LOCOMO_SPI + LOCOMO_SPICT);
save->LCM_GPE = locomo_readl(lchip->base + LOCOMO_GPE); /* GPIO */
locomo_writel(0x00, lchip->base + LOCOMO_GPE);
save->LCM_ASD = locomo_readl(lchip->base + LOCOMO_ASD); /* ADSTART */
/* Longtime timer */
locomo_writel(0, lchip->base + LOCOMO_LTINT);
/* SPI */
- locomo_writel(0, lchip->base + LOCOMO_SPIIE);
+ locomo_writel(0, lchip->base + LOCOMO_SPI + LOCOMO_SPIIE);
locomo_writel(6 + 8 + 320 + 30 - 10, lchip->base + LOCOMO_ASD);
r = locomo_readl(lchip->base + LOCOMO_ASD);
udelay(DAC_SCL_HIGH_HOLD_TIME); /* 4.7 usec */
if (locomo_readl(mapbase + LOCOMO_DAC) & LOCOMO_DAC_SDAOEB) { /* High is error */
printk(KERN_WARNING "locomo: m62332_senddata Error 1\n");
- return;
+ goto out;
}
/* Send Sub address (LSB is channel select) */
udelay(DAC_SCL_HIGH_HOLD_TIME); /* 4.7 usec */
if (locomo_readl(mapbase + LOCOMO_DAC) & LOCOMO_DAC_SDAOEB) { /* High is error */
printk(KERN_WARNING "locomo: m62332_senddata Error 2\n");
- return;
+ goto out;
}
/* Send DAC data */
udelay(DAC_SCL_HIGH_HOLD_TIME); /* 4.7 usec */
if (locomo_readl(mapbase + LOCOMO_DAC) & LOCOMO_DAC_SDAOEB) { /* High is error */
printk(KERN_WARNING "locomo: m62332_senddata Error 3\n");
- return;
}
+out:
/* stop */
r = locomo_readl(mapbase + LOCOMO_DAC);
r &= ~(LOCOMO_DAC_SCLOEB);
#include <asm/glue.h>
#include <asm/shmparam.h>
#include <asm/cachetype.h>
+#include <asm/outercache.h>
#define CACHE_COLOUR(vaddr) ((vaddr & (SHMLBA - 1)) >> PAGE_SHIFT)
void (*dma_flush_range)(const void *, const void *);
};
-struct outer_cache_fns {
- void (*inv_range)(unsigned long, unsigned long);
- void (*clean_range)(unsigned long, unsigned long);
- void (*flush_range)(unsigned long, unsigned long);
-};
-
/*
* Select the calling method
*/
#endif
-#ifdef CONFIG_OUTER_CACHE
-
-extern struct outer_cache_fns outer_cache;
-
-static inline void outer_inv_range(unsigned long start, unsigned long end)
-{
- if (outer_cache.inv_range)
- outer_cache.inv_range(start, end);
-}
-static inline void outer_clean_range(unsigned long start, unsigned long end)
-{
- if (outer_cache.clean_range)
- outer_cache.clean_range(start, end);
-}
-static inline void outer_flush_range(unsigned long start, unsigned long end)
-{
- if (outer_cache.flush_range)
- outer_cache.flush_range(start, end);
-}
-
-#else
-
-static inline void outer_inv_range(unsigned long start, unsigned long end)
-{ }
-static inline void outer_clean_range(unsigned long start, unsigned long end)
-{ }
-static inline void outer_flush_range(unsigned long start, unsigned long end)
-{ }
-
-#endif
-
/*
* Copy user data from/to a page which is mapped into a different
* processes address space. Really, we want to allow our "user
#define __ASM_CLKDEV_H
struct clk;
+struct device;
struct clk_lookup {
struct list_head node;
#ifndef __ASSEMBLY__
struct irqaction;
+struct pt_regs;
extern void migrate_irqs(void);
extern void asm_do_IRQ(unsigned int, struct pt_regs *);
--- /dev/null
+/*
+ * arch/arm/include/asm/outercache.h
+ *
+ * Copyright (C) 2010 ARM Ltd.
+ * Written by Catalin Marinas <catalin.marinas@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#ifndef __ASM_OUTERCACHE_H
+#define __ASM_OUTERCACHE_H
+
+struct outer_cache_fns {
+ void (*inv_range)(unsigned long, unsigned long);
+ void (*clean_range)(unsigned long, unsigned long);
+ void (*flush_range)(unsigned long, unsigned long);
+#ifdef CONFIG_OUTER_CACHE_SYNC
+ void (*sync)(void);
+#endif
+};
+
+#ifdef CONFIG_OUTER_CACHE
+
+extern struct outer_cache_fns outer_cache;
+
+static inline void outer_inv_range(unsigned long start, unsigned long end)
+{
+ if (outer_cache.inv_range)
+ outer_cache.inv_range(start, end);
+}
+static inline void outer_clean_range(unsigned long start, unsigned long end)
+{
+ if (outer_cache.clean_range)
+ outer_cache.clean_range(start, end);
+}
+static inline void outer_flush_range(unsigned long start, unsigned long end)
+{
+ if (outer_cache.flush_range)
+ outer_cache.flush_range(start, end);
+}
+
+#else
+
+static inline void outer_inv_range(unsigned long start, unsigned long end)
+{ }
+static inline void outer_clean_range(unsigned long start, unsigned long end)
+{ }
+static inline void outer_flush_range(unsigned long start, unsigned long end)
+{ }
+
+#endif
+
+#ifdef CONFIG_OUTER_CACHE_SYNC
+static inline void outer_sync(void)
+{
+ if (outer_cache.sync)
+ outer_cache.sync();
+}
+#else
+static inline void outer_sync(void)
+{ }
+#endif
+
+#endif /* __ASM_OUTERCACHE_H */
#include <linux/linkage.h>
#include <linux/irqflags.h>
+#include <asm/outercache.h>
+
#define __exception __attribute__((section(".exception.text")))
struct thread_info;
#define dmb() __asm__ __volatile__ ("" : : : "memory")
#endif
-#if __LINUX_ARM_ARCH__ >= 7 || defined(CONFIG_SMP)
-#define mb() dmb()
+#ifdef CONFIG_ARCH_HAS_BARRIERS
+#include <mach/barriers.h>
+#elif __LINUX_ARM_ARCH__ >= 7 || defined(CONFIG_SMP)
+#define mb() do { dsb(); outer_sync(); } while (0)
#define rmb() dmb()
-#define wmb() dmb()
+#define wmb() mb()
#else
#define mb() do { if (arch_is_coherent()) dmb(); else barrier(); } while (0)
#define rmb() do { if (arch_is_coherent()) dmb(); else barrier(); } while (0)
#define smp_rmb() barrier()
#define smp_wmb() barrier()
#else
-#define smp_mb() mb()
-#define smp_rmb() rmb()
-#define smp_wmb() wmb()
+#define smp_mb() dmb()
+#define smp_rmb() dmb()
+#define smp_wmb() dmb()
#endif
#define read_barrier_depends() do { } while(0)
#include <linux/ioport.h>
#include <linux/interrupt.h>
#include <linux/irq.h>
-#include <linux/slab.h>
#include <linux/random.h>
#include <linux/smp.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/kprobes.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/stop_machine.h>
#include <linux/stringify.h>
#include <asm/traps.h>
/*
* Setup an empty pt_regs. Fill SP and PC fields as
* they're needed by longjmp_break_handler.
+ *
+ * We allocate some slack between the original SP and start of
+ * our fabricated regs. To be precise we want to have worst case
+ * covered which is STMFD with all 16 regs so we allocate 2 *
+ * sizeof(struct_pt_regs)).
+ *
+ * This is to prevent any simulated instruction from writing
+ * over the regs when they are accessing the stack.
*/
"sub sp, %0, %1 \n\t"
"ldr r0, ="__stringify(JPROBE_MAGIC_ADDR)"\n\t"
"ldmia sp, {r0 - pc} \n\t"
:
: "r" (kcb->jprobe_saved_regs.ARM_sp),
- "I" (sizeof(struct pt_regs)),
+ "I" (sizeof(struct pt_regs) * 2),
"J" (offsetof(struct pt_regs, ARM_sp)),
"J" (offsetof(struct pt_regs, ARM_pc)),
"J" (offsetof(struct pt_regs, ARM_cpsr))
#include <linux/mm.h>
#include <linux/elf.h>
#include <linux/vmalloc.h>
-#include <linux/slab.h>
#include <linux/fs.h>
#include <linux/string.h>
+#include <linux/gfp.h>
#include <asm/pgtable.h>
#include <asm/sections.h>
#include <linux/mm.h>
#include <linux/stddef.h>
#include <linux/unistd.h>
-#include <linux/slab.h>
#include <linux/user.h>
#include <linux/delay.h>
#include <linux/reboot.h>
#include <linux/module.h>
#include <linux/errno.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/sem.h>
#include <linux/msg.h>
#include <linux/file.h>
#include <linux/ipc.h>
#include <linux/uaccess.h>
+#include <linux/slab.h>
/* Fork a new task - this creates a new program thread.
* This is called indirectly via a small wrapper
rsb ip, ip, #32
addne pc, pc, ip @ C is always clear here
b 7f
-6: nop
+6: W(nop)
W(ldr) r3, [r1, #-4]!
W(ldr) r4, [r1, #-4]!
W(ldr) r5, [r1, #-4]!
add pc, pc, ip
nop
- nop
+ W(nop)
W(str) r3, [r0, #-4]!
W(str) r4, [r0, #-4]!
W(str) r5, [r0, #-4]!
#include <linux/mm.h>
#include <linux/sched.h>
#include <linux/hardirq.h> /* for in_atomic() */
+#include <linux/gfp.h>
#include <asm/current.h>
#include <asm/page.h>
#include <linux/timex.h>
#include <linux/signal.h>
#include <linux/clk.h>
+#include <linux/gfp.h>
#include <mach/hardware.h>
#include <asm/irq.h>
#include <linux/interrupt.h>
#include <linux/irqreturn.h>
#include <linux/proc_fs.h>
+#include <linux/slab.h>
#include <mach/timer.h>
#include <linux/leds.h>
#include <linux/mtd/mtd.h>
#include <linux/mtd/partitions.h>
+#include <linux/slab.h>
#include <linux/mtd/nand.h>
#include <linux/input.h>
#include <linux/spi/spi.h>
#include <linux/interrupt.h>
#include <linux/platform_device.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <mach/edma.h>
*/
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/mman.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/cpufreq.h>
-#include <linux/slab.h>
#include <linux/sched.h>
#include <linux/smp.h>
#include <linux/init.h>
#include <linux/amba/bus.h>
#include <linux/amba/clcd.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <asm/clkdev.h>
#include <mach/clkdev.h>
#include <linux/list.h>
#include <linux/platform_device.h>
#include <linux/dma-mapping.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/sysdev.h>
#include <linux/amba/bus.h>
#include <linux/amba/clcd.h>
#include <linux/amba/mmci.h>
#include <linux/io.h>
+#include <linux/gfp.h>
#include <asm/clkdev.h>
#include <mach/clkdev.h>
*/
#include <linux/kernel.h>
#include <linux/pci.h>
-#include <linux/slab.h>
#include <linux/ioport.h>
#include <linux/interrupt.h>
#include <linux/spinlock.h>
*/
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/jiffies.h>
#include <asm/irq.h>
#include <linux/pci.h>
#include <linux/pm.h>
#include <linux/string.h>
-#include <linux/slab.h>
#include <linux/serial_core.h>
#include <linux/serial_8250.h>
#include <linux/mtd/physmap.h>
#include <linux/pci.h>
#include <linux/pm.h>
#include <linux/string.h>
-#include <linux/slab.h>
#include <linux/serial_core.h>
#include <linux/serial_8250.h>
#include <linux/mtd/physmap.h>
#include <linux/kernel.h>
#include <linux/pci.h>
#include <linux/string.h>
-#include <linux/slab.h>
#include <linux/serial_core.h>
#include <linux/serial_8250.h>
#include <linux/mtd/physmap.h>
#include <linux/pci.h>
#include <linux/pm.h>
#include <linux/string.h>
-#include <linux/slab.h>
#include <linux/serial_core.h>
#include <linux/serial_8250.h>
#include <linux/mtd/physmap.h>
#include <linux/kernel.h>
#include <linux/pci.h>
#include <linux/string.h>
-#include <linux/slab.h>
#include <linux/serial_core.h>
#include <linux/serial_8250.h>
#include <linux/mtd/physmap.h>
#include <linux/kernel.h>
#include <linux/pci.h>
#include <linux/string.h>
-#include <linux/slab.h>
#include <linux/serial_core.h>
#include <linux/serial_8250.h>
#include <linux/mtd/physmap.h>
#include <linux/bitops.h>
#include <linux/pci.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/serial.h>
#include <linux/tty.h>
#include <linux/bitops.h>
#include <linux/pci.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/io.h>
#include <linux/bitops.h>
#include <linux/pci.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/io.h>
#include <linux/bitops.h>
#include <linux/pci.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/io.h>
#include <linux/bitops.h>
#include <linux/pci.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/serial.h>
#include <linux/tty.h>
#include <linux/mm.h>
#include <linux/init.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/io.h>
*/
#define PHYS_OFFSET (0x00000000)
-#define IXP23XX_PCI_SDRAM_OFFSET (*((volatile int *)IXP23XX_PCI_SDRAM_BAR) & 0xfffffff0))
+#define IXP23XX_PCI_SDRAM_OFFSET (*((volatile int *)IXP23XX_PCI_SDRAM_BAR) & 0xfffffff0)
#define __phys_to_bus(x) ((x) + (IXP23XX_PCI_SDRAM_OFFSET - PHYS_OFFSET))
#define __bus_to_phys(x) ((x) - (IXP23XX_PCI_SDRAM_OFFSET - PHYS_OFFSET))
#include <linux/mm.h>
#include <linux/init.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/io.h>
#include <linux/serial.h>
#include <linux/tty.h>
#include <linux/serial_8250.h>
-#include <linux/slab.h>
#include <linux/i2c-gpio.h>
#include <asm/types.h>
#include <asm/setup.h>
#include <linux/serial.h>
#include <linux/tty.h>
#include <linux/serial_8250.h>
-#include <linux/slab.h>
#include <asm/types.h>
#include <asm/setup.h>
#include <linux/serial.h>
#include <linux/tty.h>
#include <linux/serial_8250.h>
-#include <linux/slab.h>
#include <asm/types.h>
#include <asm/setup.h>
#include <linux/serial.h>
#include <linux/tty.h>
#include <linux/serial_8250.h>
-#include <linux/slab.h>
#include <asm/types.h>
#include <asm/setup.h>
#include <asm/memory.h>
#include <linux/serial.h>
#include <linux/tty.h>
#include <linux/serial_8250.h>
-#include <linux/slab.h>
#include <linux/i2c-gpio.h>
#include <linux/io.h>
#include <linux/mtd/mtd.h>
#include <linux/io.h>
#include <linux/kernel.h>
#include <linux/module.h>
-#include <linux/slab.h>
#include <mach/npe.h>
#define DEBUG_MSG 0
#include <linux/serial.h>
#include <linux/tty.h>
#include <linux/serial_8250.h>
-#include <linux/slab.h>
#include <asm/types.h>
#include <asm/setup.h>
.desc = "SWR Button",
.active_low = 1,
}, {
- .code = KEY_F1,
+ .code = KEY_WPS_BUTTON,
.gpio = 46,
- .desc = "WPS Button(F1)",
+ .desc = "WPS Button",
.active_low = 1,
},
};
#include <linux/kernel.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/mbus.h>
#include <asm/irq.h>
#include <asm/mach/pci.h>
*/
#include <linux/init.h>
+#include <linux/gfp.h>
#include <linux/device.h>
#include <linux/dma-mapping.h>
#include <linux/sysdev.h>
#define UART2_BASE (APB_PHYS_BASE + 0x17000)
#define UART3_BASE (APB_PHYS_BASE + 0x18000)
-static volatile unsigned long *UART = (unsigned long *)UART2_BASE;
+static volatile unsigned long *UART;
static inline void putc(char c)
{
static inline void arch_decomp_setup(void)
{
+ /* default to UART2 */
+ UART = (unsigned long *)UART2_BASE;
+
if (machine_is_avengers_lite())
UART = (unsigned long *)UART3_BASE;
}
#include <linux/delay.h>
#include <linux/dma-mapping.h>
#include <linux/fsl_devices.h>
+#include <linux/gfp.h>
#include <linux/gpio.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/usb/otg.h>
#include <linux/usb/ulpi.h>
#include <linux/fsl_devices.h>
+#include <linux/gfp.h>
#include <media/soc_camera.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/usb/otg.h>
#include <linux/interrupt.h>
#include <linux/i2c.h>
#include <linux/spi/spi.h>
+#include <linux/slab.h>
#include <linux/platform_device.h>
#include <linux/types.h>
#include <linux/amba/bus.h>
#include <linux/amba/clcd.h>
#include <linux/err.h>
+#include <linux/gfp.h>
#include <asm/irq.h>
#include <linux/device.h>
#include <linux/firmware.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <linux/io.h>
#include <mach/hardware.h>
#include <linux/spinlock.h>
#include <linux/interrupt.h>
#include <linux/irq.h>
+#include <linux/slab.h>
#include <mach/hardware.h>
#include <mach/gpio.h>
*/
#include <linux/platform_device.h>
#include <linux/serial_8250.h>
+#include <linux/slab.h>
#include <mach/regs-board-a9m9750dev.h>
#include <mach/board.h>
*/
#include <linux/io.h>
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <asm/page.h>
#include <asm/mach/map.h>
#include <linux/err.h>
#include <linux/io.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <mach/irqs.h>
#include <plat/dma.h>
#include <linux/clk.h>
#include <linux/io.h>
#include <linux/cpufreq.h>
+#include <linux/slab.h>
#include <plat/clock.h>
#include <plat/sram.h>
#include <linux/device.h>
#include <linux/jiffies.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/stringify.h>
#include <plat/iommu.h>
#include <linux/err.h>
#include <linux/io.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <mach/irqs.h>
#include <plat/dma.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/list.h>
#include <linux/ctype.h>
#include <linux/err.h>
#include <linux/io.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <plat/clock.h>
#include <plat/board.h>
#include <linux/gpio.h>
#include <linux/clk.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <plat/sram.h>
#include <plat/clockdomain.h>
#include <linux/kernel.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/mbus.h>
#include <asm/irq.h>
#include <asm/mach/pci.h>
.desc = "Reset Button",
.active_low = 1,
}, {
- .code = KEY_WLAN,
+ .code = KEY_WPS_BUTTON,
.gpio = 2,
.desc = "WPS Button",
.active_low = 1,
#include <linux/dma-mapping.h>
#include <linux/clk.h>
#include <linux/io.h>
+#include <linux/gfp.h>
#include <asm/system.h>
#include <mach/hardware.h>
#include <linux/delay.h>
#include <linux/clk.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <asm/cacheflush.h>
config MACH_HIMALAYA
bool "HTC Himalaya Support"
select CPU_PXA26x
- select FB_W100
config MACH_MAGICIAN
bool "Enable HTC Magician Support"
config SHARPSL_PM
bool
select APM_EMULATION
+ select SHARPSL_PM_MAX1111
+
+config SHARPSL_PM_MAX1111
+ bool
+ depends on !CORGI_SSP_DEPRECATED
+ select HWMON
+ select SENSORS_MAX1111
config CORGI_SSP_DEPRECATED
bool
bool "Toshiba e740"
default y
depends on ARCH_PXA_ESERIES
- select FB_W100
help
Say Y here if you intend to run this kernel on a Toshiba
e740 family PDA.
bool "Toshiba e750"
default y
depends on ARCH_PXA_ESERIES
- select FB_W100
help
Say Y here if you intend to run this kernel on a Toshiba
e750 family PDA.
bool "Toshiba e800"
default y
depends on ARCH_PXA_ESERIES
- select FB_W100
help
Say Y here if you intend to run this kernel on a Toshiba
e800 family PDA.
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/platform_device.h>
#include <mach/hardware.h>
#include <linux/sched.h>
#include <linux/init.h>
#include <linux/cpufreq.h>
+#include <linux/slab.h>
#include <mach/pxa3xx-regs.h>
pxa_set_btuart_info(NULL);
pxa_set_stuart_info(NULL);
- /* SPI chip select directions - all other directions should
- * be handled by drivers.*/
- gpio_direction_output(37, 0);
-
platform_add_devices(imote2_devices, ARRAY_SIZE(imote2_devices));
pxa2xx_set_spi_info(1, &pxa_ssp_master_0_info);
#define BTUART_BASE (0x40200000)
#define STUART_BASE (0x40700000)
-static unsigned long uart_base = FFUART_BASE;
-static unsigned int uart_shift = 2;
-static unsigned int uart_is_pxa = 1;
+static unsigned long uart_base;
+static unsigned int uart_shift;
+static unsigned int uart_is_pxa;
static inline unsigned char uart_read(int offset)
{
static inline void arch_decomp_setup(void)
{
+ /* initialize to default */
+ uart_base = FFUART_BASE;
+ uart_shift = 2;
+ uart_is_pxa = 1;
+
if (machine_is_littleton() || machine_is_intelmote2()
|| machine_is_csb726() || machine_is_stargate2()
|| machine_is_cm_x300() || machine_is_balloon3())
#include <linux/mtd/physmap.h>
#include <linux/usb/gpio_vbus.h>
#include <linux/regulator/max1586.h>
+#include <linux/slab.h>
#include <asm/mach-types.h>
#include <asm/mach/arch.h>
#include <linux/module.h>
#include <linux/suspend.h>
#include <linux/errno.h>
+#include <linux/slab.h>
#include <mach/pm.h>
#include <linux/lis3lv02d.h>
#include <linux/pda_power.h>
#include <linux/power_supply.h>
-#include <linux/pda_power.h>
-#include <linux/power_supply.h>
#include <linux/regulator/max8660.h>
#include <linux/regulator/machine.h>
#include <linux/regulator/fixed.h>
.active_low = 0,
.wakeup = 0,
.debounce_interval = 5, /* ms */
- .desc = "on/off button",
+ .desc = "on_off button",
},
};
pxa_set_btuart_info(NULL);
pxa_set_stuart_info(NULL);
- /* spi chip selects */
- gpio_direction_output(37, 0);
- gpio_direction_output(24, 0);
- gpio_direction_output(39, 0);
-
platform_add_devices(ARRAY_AND_SIZE(stargate2_devices));
pxa2xx_set_spi_info(1, &pxa_ssp_master_0_info);
#include <linux/delay.h>
#include <linux/fs.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/major.h>
#include <linux/module.h>
#include <linux/smsc911x.h>
#include <linux/ata_platform.h>
#include <linux/amba/mmci.h>
+#include <linux/gfp.h>
#include <asm/clkdev.h>
#include <asm/system.h>
*
* DMA functions specific to RiscPC architecture
*/
-#include <linux/slab.h>
#include <linux/mman.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/dmapool.h>
#include <linux/sysdev.h>
#include <linux/errno.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/clk.h>
#include <linux/err.h>
#include <linux/module.h>
#include <linux/platform_device.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <mach/hardware.h>
#include <mach/jornada720.h>
#include <linux/ioport.h>
#include <linux/serial_core.h>
#include <linux/platform_device.h>
-#include <linux/slab.h>
#include <mach/hardware.h>
#include <asm/mach-types.h>
#include <linux/mutex.h>
#include <linux/spi/spi.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
/*
* WARNING! Do not include this pl022-specific controller header
* for any generic driver. It is only done in this dummy chip
#include <linux/regulator/machine.h>
#include <linux/gpio.h>
#include <linux/amba/mmci.h>
+#include <linux/slab.h>
#include "mmc.h"
#include "padmux.h"
#include <linux/clockchips.h>
#include <linux/cnt32_to_63.h>
#include <linux/io.h>
+#include <linux/gfp.h>
#include <asm/clkdev.h>
#include <asm/system.h>
*/
#include <linux/kernel.h>
#include <linux/pci.h>
-#include <linux/slab.h>
#include <linux/ioport.h>
#include <linux/interrupt.h>
#include <linux/spinlock.h>
#include <linux/timer.h>
#include <linux/init.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/mtd/physmap.h>
#include <linux/mtd/mtd.h>
config OUTER_CACHE
bool
+config OUTER_CACHE_SYNC
+ bool
+ help
+ The outer cache has a outer_cache_fns.sync function pointer
+ that can be used to drain the write buffer of the outer cache.
+
config CACHE_FEROCEON_L2
bool "Enable the Feroceon L2 cache controller"
depends on ARCH_KIRKWOOD || ARCH_MV78XX0
REALVIEW_EB_A9MP || ARCH_MX35 || ARCH_MX31 || MACH_REALVIEW_PBX || ARCH_NOMADIK || ARCH_OMAP4
default y
select OUTER_CACHE
+ select OUTER_CACHE_SYNC
help
This option enables the L2x0 PrimeCell.
int
default 6 if ARM_L1_CACHE_SHIFT_6
default 5
+
+config ARCH_HAS_BARRIERS
+ bool
+ help
+ This option allows the use of custom mandatory barriers
+ included via the mach/barriers.h file.
}
#endif
+static void l2x0_cache_sync(void)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&l2x0_lock, flags);
+ cache_sync();
+ spin_unlock_irqrestore(&l2x0_lock, flags);
+}
+
static inline void l2x0_inv_all(void)
{
unsigned long flags;
outer_cache.inv_range = l2x0_inv_range;
outer_cache.clean_range = l2x0_clean_range;
outer_cache.flush_range = l2x0_flush_range;
+ outer_cache.sync = l2x0_cache_sync;
printk(KERN_INFO "L2X0 cache controller enabled\n");
}
*/
#include <linux/module.h>
#include <linux/mm.h>
-#include <linux/slab.h>
+#include <linux/gfp.h>
#include <linux/errno.h>
#include <linux/list.h>
#include <linux/init.h>
#include <linux/vmalloc.h>
#include <linux/init.h>
#include <linux/pagemap.h>
+#include <linux/gfp.h>
#include <asm/bugs.h>
#include <asm/cacheflush.h>
#include <linux/initrd.h>
#include <linux/sort.h>
#include <linux/highmem.h>
+#include <linux/gfp.h>
#include <asm/mach-types.h>
#include <asm/sections.h>
* published by the Free Software Foundation.
*/
#include <linux/mm.h>
+#include <linux/gfp.h>
#include <linux/highmem.h>
#include <asm/pgalloc.h>
#include <linux/io.h>
#include <linux/clk.h>
#include <linux/debugfs.h>
+#include <linux/slab.h>
#include <mach/audmux.h>
#include <mach/hardware.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/err.h>
#include <linux/clk.h>
#include <linux/io.h>
#include <linux/init.h>
#include <linux/platform_device.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <mach/hardware.h>
#include <asm/mach-types.h>
#include <linux/interrupt.h>
#include <linux/irq.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <asm/system.h>
#include <mach/hardware.h>
#include <linux/err.h>
#include <linux/clk.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <linux/uaccess.h>
#include <linux/platform_device.h>
#include <linux/debugfs.h>
#include <linux/err.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/ioport.h>
#include <linux/clk.h>
*/
#include <linux/err.h>
+#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/device.h>
#include <linux/scatterlist.h>
#include <linux/interrupt.h>
#include <linux/device.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <plat/mailbox.h>
#include <linux/clk.h>
#include <linux/delay.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <plat/dma.h>
#include <plat/mcbsp.h>
#include <linux/kernel.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/err.h>
#include <linux/io.h>
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/interrupt.h>
#include <linux/errno.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/err.h>
#include <linux/clk.h>
#include <linux/io.h>
#include <linux/sysdev.h>
#include <linux/kobject.h>
#include <linux/sysfs.h>
+#include <linux/slab.h>
#include <asm/mach/arch.h>
#include <asm/mach/map.h>
#include <linux/serial_core.h>
#include <linux/platform_device.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <asm/mach/arch.h>
#include <asm/mach/map.h>
#include <linux/cpufreq.h>
#include <linux/seq_file.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <mach/map.h>
#include <mach/regs-mem.h>
#include <linux/delay.h>
#include <linux/clk.h>
#include <linux/err.h>
+#include <linux/slab.h>
#include <linux/amba/pl093.h>
#include <linux/platform_device.h>
#include <linux/sched.h>
#include <linux/list.h>
+#include <linux/slab.h>
#include <linux/err.h>
#include <linux/clk.h>
#include <linux/interrupt.h>
#include <linux/string.h>
#include <linux/platform_device.h>
#include <linux/fb.h>
+#include <linux/gfp.h>
#include <mach/irqs.h>
#include <mach/map.h>
* published by the Free Software Foundation.
*/
+#include <linux/gfp.h>
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/platform_device.h>
* published by the Free Software Foundation.
*/
+#include <linux/gfp.h>
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/platform_device.h>
* published by the Free Software Foundation.
*/
+#include <linux/gfp.h>
#include <linux/kernel.h>
#include <linux/platform_device.h>
* published by the Free Software Foundation.
*/
+#include <linux/gfp.h>
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/platform_device.h>
#include <linux/init.h>
#include <linux/crc32.h>
#include <linux/ioport.h>
+#include <linux/slab.h>
#include <plat/pm.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/err.h>
#include <linux/clk.h>
#include <linux/io.h>
* http://www.opensource.org/licenses/gpl-license.html
* http://www.gnu.org/copyleft/gpl.html
*/
+#include <linux/gfp.h>
#include <linux/kernel.h>
#include <linux/device.h>
#include <linux/dmapool.h>
#
# http://www.arm.linux.org.uk/developer/machines/?action=new
#
-# Last update: Sat Feb 20 14:16:15 2010
+# Last update: Sat Mar 20 15:35:41 2010
#
# machine_is_xxx CONFIG_xxxx MACH_TYPE_xxx number
#
aquila MACH_AQUILA AQUILA 2676
spark_sls_hw2 MACH_SPARK_SLS_HW2 SPARK_SLS_HW2 2677
sheeva_esata MACH_ESATA_SHEEVAPLUG ESATA_SHEEVAPLUG 2678
-surf7x30 MACH_SURF7X30 SURF7X30 2679
+msm7x30_surf MACH_MSM7X30_SURF MSM7X30_SURF 2679
micro2440 MACH_MICRO2440 MICRO2440 2680
am2440 MACH_AM2440 AM2440 2681
tq2440 MACH_TQ2440 TQ2440 2682
mioa702 MACH_MIOA702 MIOA702 2691
hpmin MACH_HPMIN HPMIN 2692
ak880xak MACH_AK880XAK AK880XAK 2693
+arm926tomap850 MACH_ARM926TOMAP850 ARM926TOMAP850 2694
+lkevm MACH_LKEVM LKEVM 2695
+mw6410 MACH_MW6410 MW6410 2696
+terastation_wxl MACH_TERASTATION_WXL TERASTATION_WXL 2697
+cpu8000e MACH_CPU8000E CPU8000E 2698
+catania MACH_CATANIA CATANIA 2699
+tokyo MACH_TOKYO TOKYO 2700
+msm7201a_surf MACH_MSM7201A_SURF MSM7201A_SURF 2701
+msm7201a_ffa MACH_MSM7201A_FFA MSM7201A_FFA 2702
+msm7x25_surf MACH_MSM7X25_SURF MSM7X25_SURF 2703
+msm7x25_ffa MACH_MSM7X25_FFA MSM7X25_FFA 2704
+msm7x27_surf MACH_MSM7X27_SURF MSM7X27_SURF 2705
+msm7x27_ffa MACH_MSM7X27_FFA MSM7X27_FFA 2706
+msm7x30_ffa MACH_MSM7X30_FFA MSM7X30_FFA 2707
+qsd8x50_surf MACH_QSD8X50_SURF QSD8X50_SURF 2708
+qsd8x50_comet MACH_QSD8X50_COMET QSD8X50_COMET 2709
+qsd8x50_ffa MACH_QSD8X50_FFA QSD8X50_FFA 2710
+qsd8x50a_surf MACH_QSD8X50A_SURF QSD8X50A_SURF 2711
+qsd8x50a_ffa MACH_QSD8X50A_FFA QSD8X50A_FFA 2712
+adx_xgcp10 MACH_ADX_XGCP10 ADX_XGCP10 2713
+mcgwumts2a MACH_MCGWUMTS2A MCGWUMTS2A 2714
+mobikt MACH_MOBIKT MOBIKT 2715
+mx53_evk MACH_MX53_EVK MX53_EVK 2716
+igep0030 MACH_IGEP0030 IGEP0030 2717
+axell_h40_h50_ctrl MACH_AXELL_H40_H50_CTRL AXELL_H40_H50_CTRL 2718
+dtcommod MACH_DTCOMMOD DTCOMMOD 2719
+gould MACH_GOULD GOULD 2720
+siberia MACH_SIBERIA SIBERIA 2721
+sbc3530 MACH_SBC3530 SBC3530 2722
+qarm MACH_QARM QARM 2723
+mips MACH_MIPS MIPS 2724
+mx27grb MACH_MX27GRB MX27GRB 2725
+sbc8100 MACH_SBC8100 SBC8100 2726
+saarb MACH_SAARB SAARB 2727
+omap3mini MACH_OMAP3MINI OMAP3MINI 2728
+cnmbook7se MACH_CNMBOOK7SE CNMBOOK7SE 2729
+catan MACH_CATAN CATAN 2730
+harmony MACH_HARMONY HARMONY 2731
+tonga MACH_TONGA TONGA 2732
+cybook_orizon MACH_CYBOOK_ORIZON CYBOOK_ORIZON 2733
+htcrhodiumcdma MACH_HTCRHODIUMCDMA HTCRHODIUMCDMA 2734
+epc_g45 MACH_EPC_G45 EPC_G45 2735
+epc_lpc3250 MACH_EPC_LPC3250 EPC_LPC3250 2736
+mxc91341evb MACH_MXC91341EVB MXC91341EVB 2737
+rtw1000 MACH_RTW1000 RTW1000 2738
+bobcat MACH_BOBCAT BOBCAT 2739
+trizeps6 MACH_TRIZEPS6 TRIZEPS6 2740
+msm7x30_fluid MACH_MSM7X30_FLUID MSM7X30_FLUID 2741
+nedap9263 MACH_NEDAP9263 NEDAP9263 2742
+netgear_ms2110 MACH_NETGEAR_MS2110 NETGEAR_MS2110 2743
+bmx MACH_BMX BMX 2744
+netstream MACH_NETSTREAM NETSTREAM 2745
+vpnext_rcu MACH_VPNEXT_RCU VPNEXT_RCU 2746
+vpnext_mpu MACH_VPNEXT_MPU VPNEXT_MPU 2747
+bcmring_tablet_v1 MACH_BCMRING_TABLET_V1 BCMRING_TABLET_V1 2748
+sgarm10 MACH_SGARM10 SGARM10 2749
+cm_t3517 MACH_CM_T3517 CM_T3517 2750
+omap3_cps MACH_OMAP3_CPS OMAP3_CPS 2751
+axar1500_receiver MACH_AXAR1500_RECEIVER AXAR1500_RECEIVER 2752
+wbd222 MACH_WBD222 WBD222 2753
+mt65xx MACH_MT65XX MT65XX 2754
+msm8x60_surf MACH_MSM8X60_SURF MSM8X60_SURF 2755
+msm8x60_sim MACH_MSM8X60_SIM MSM8X60_SIM 2756
+vmc300 MACH_VMC300 VMC300 2757
+tcc8000_sdk MACH_TCC8000_SDK TCC8000_SDK 2758
+nanos MACH_NANOS NANOS 2759
+stamp9g10 MACH_STAMP9G10 STAMP9G10 2760
+stamp9g45 MACH_STAMP9G45 STAMP9G45 2761
+h6053 MACH_H6053 H6053 2762
+smint01 MACH_SMINT01 SMINT01 2763
+prtlvt2 MACH_PRTLVT2 PRTLVT2 2764
*/
elf_hwcap |= HWCAP_VFP;
#ifdef CONFIG_VFPv3
- if (VFP_arch >= 3) {
+ if (VFP_arch >= 2) {
elf_hwcap |= HWCAP_VFPv3;
/*
#include <linux/fs.h>
#include <linux/pm.h>
#include <linux/ptrace.h>
+#include <linux/slab.h>
#include <linux/reboot.h>
#include <linux/tick.h>
#include <linux/uaccess.h>
#include <linux/init.h>
#include <linux/platform_device.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <linux/gpio.h>
#include <linux/spi/spi.h>
#include <linux/usb/atmel_usba_udc.h>
#include <linux/irq.h>
#include <linux/platform_device.h>
#include <linux/random.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <mach/smc.h>
*/
#include <linux/dma-mapping.h>
+#include <linux/gfp.h>
#include <asm/addrspace.h>
#include <asm/cacheflush.h>
*/
#include <linux/kernel.h>
+#include <linux/gfp.h>
#include <linux/mm.h>
#include <linux/swap.h>
#include <linux/init.h>
#include <linux/mm.h>
#include <linux/module.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <asm/pgtable.h>
#include <asm/addrspace.h>
#ifndef __BLACKFIN_MMU_CONTEXT_H__
#define __BLACKFIN_MMU_CONTEXT_H__
-#include <linux/gfp.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#include <asm/setup.h>
#include <asm/page.h>
#include <linux/interrupt.h>
#include <linux/percpu.h>
#include <linux/bitops.h>
-#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/kthread.h>
#include <linux/unistd.h>
#include <linux/unistd.h>
#include <linux/user.h>
#include <linux/uaccess.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#include <linux/tick.h>
#include <linux/fs.h>
#include <linux/suspend.h>
#include <linux/sched.h>
#include <linux/proc_fs.h>
+#include <linux/slab.h>
#include <linux/io.h>
#include <linux/irq.h>
#include <linux/smp.h>
#include <linux/seq_file.h>
#include <linux/irq.h>
+#include <linux/slab.h>
#include <asm/atomic.h>
#include <asm/cacheflush.h>
#include <asm/mmu_context.h>
* Licensed under the GPL-2 or later.
*/
+#include <linux/gfp.h>
#include <linux/swap.h>
#include <linux/bootmem.h>
#include <linux/uaccess.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/sched.h>
#include <linux/proc_fs.h>
#include <linux/spinlock.h>
#include <linux/rtc.h>
+#include <linux/slab.h>
#include <asm/blackfin.h>
#include <asm/mem_map.h>
#include "blackfin_sram.h"
#include <linux/module.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/smp_lock.h>
#include <linux/errno.h>
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/major.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/poll.h>
#include <linux/init.h>
*/
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/err.h>
#include <linux/fs.h>
-#include <linux/slab.h>
#include <arch/svinto.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/kernel.h>
#include <linux/fs.h>
if ((res->flags & IORESOURCE_IO) && (start & 0x300))
start = (start + 0x3ff) & ~0x3ff;
- return start
+ return start;
}
int pcibios_enable_resources(struct pci_dev *dev, int mask)
#include <linux/mm.h>
#include <linux/string.h>
#include <linux/pci.h>
+#include <linux/gfp.h>
#include <asm/io.h>
void *dma_alloc_coherent(struct device *dev, size_t size,
#include <linux/errno.h>
#include <linux/major.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/smp_lock.h>
#include <linux/interrupt.h>
#include <linux/poll.h>
*/
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/err.h>
#include <linux/fs.h>
-#include <linux/slab.h>
#include <hwregs/reg_rdwr.h>
#include <hwregs/reg_map.h>
#include <hwregs/timer_defs.h>
#include <linux/sched.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/signal.h>
#include <linux/errno.h>
#include <linux/ioport.h>
#include <linux/interrupt.h>
#include <linux/timex.h>
-#include <linux/slab.h>
#include <linux/random.h>
#include <linux/init.h>
#include <linux/seq_file.h>
#include <linux/fs.h>
#include <linux/string.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#if 0
#define DEBUGP printk
#include <linux/errno.h>
#include <linux/kernel.h>
#include <linux/proc_fs.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <asm/ptrace.h>
#include <asm/uaccess.h>
*
*/
+#include <linux/gfp.h>
#include <linux/init.h>
#include <linux/bootmem.h>
#include <asm/tlb.h>
#include <linux/ioport.h>
#include <linux/interrupt.h>
#include <linux/timex.h>
-#include <linux/slab.h>
#include <linux/random.h>
#include <linux/init.h>
#include <linux/kernel_stat.h>
* 2 of the License, or (at your option) any later version.
*/
-#include <linux/slab.h>
#include <linux/sysctl.h>
#include <linux/proc_fs.h>
#include <linux/init.h>
*/
#include <linux/types.h>
-#include <linux/slab.h>
#include <linux/dma-mapping.h>
#include <linux/list.h>
#include <linux/pci.h>
if ((res->flags & IORESOURCE_IO) && (start & 0x300))
start = (start + 0x3ff) & ~0x3ff;
- return start
+ return start;
}
r = &dev->resource[idx];
if (!r->start)
continue;
- if (pci_claim_resource(dev, idx) < 0)
- printk(KERN_ERR "PCI: Cannot allocate resource region %d of bridge %s\n", idx, pci_name(dev));
+ pci_claim_resource(dev, idx);
}
}
pcibios_allocate_bus_resources(&bus->children);
DBG("PCI: Resource %08lx-%08lx (f=%lx, d=%d, p=%d)\n",
r->start, r->end, r->flags, disabled, pass);
if (pci_claim_resource(dev, idx) < 0) {
- printk(KERN_ERR "PCI: Cannot allocate resource region %d of device %s\n", idx, pci_name(dev));
/* We'll assign a new address later */
r->end -= r->start;
r->start = 0;
#include <linux/kernel.h>
#include <linux/pci.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/irq.h>
#include <linux/init.h>
#include <linux/ioport.h>
#include <linux/delay.h>
-#include <linux/slab.h>
#include <asm/segment.h>
#include <asm/io.h>
#include <linux/init.h>
#include <linux/pci.h>
#include <linux/hardirq.h>
+#include <linux/gfp.h>
#include <asm/pgalloc.h>
#include <asm/io.h>
#include <linux/signal.h>
#include <linux/sched.h>
#include <linux/pagemap.h>
+#include <linux/gfp.h>
#include <linux/swap.h>
#include <linux/mm.h>
#include <linux/kernel.h>
*/
#include <linux/sched.h>
-#include <linux/slab.h>
+#include <linux/gfp.h>
#include <linux/mm.h>
#include <linux/highmem.h>
#include <linux/quicklist.h>
#include <linux/stddef.h>
#include <linux/unistd.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/user.h>
#include <linux/interrupt.h>
#include <linux/reboot.h>
#include <linux/fs.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/system.h>
#include <linux/highmem.h>
#include <linux/pagemap.h>
#include <linux/bootmem.h>
-#include <linux/slab.h>
+#include <linux/gfp.h>
#include <asm/setup.h>
#include <asm/segment.h>
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/types.h>
-#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <asm/setup.h>
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/types.h>
-#include <linux/slab.h>
#include <asm/setup.h>
#include <asm/segment.h>
#ifndef _ASM_DMI_H
#define _ASM_DMI_H 1
+#include <linux/slab.h>
#include <asm/io.h>
/* Use normal IO mappings for DMI */
#include <linux/module.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/acpi.h>
#include <asm/acpi-ext.h>
#include <linux/efi.h>
#include <linux/mmzone.h>
#include <linux/nodemask.h>
+#include <linux/slab.h>
#include <acpi/processor.h>
#include <asm/io.h>
#include <asm/iosapic.h>
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/cpufreq.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/time.h>
#include <linux/efi.h>
#include <linux/kexec.h>
#include <linux/kernel.h>
#include <linux/list.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/smp.h>
#include <linux/string.h>
#include <linux/bootmem.h>
#include <linux/interrupt.h>
#include <linux/ioport.h>
#include <linux/kernel_stat.h>
-#include <linux/slab.h>
#include <linux/ptrace.h>
#include <linux/random.h> /* for rand_initialize_irq() */
#include <linux/signal.h>
#include <linux/cpumask.h>
#include <linux/kdebug.h>
#include <linux/cpu.h>
+#include <linux/gfp.h>
#include <asm/delay.h>
#include <asm/machvec.h>
#include <linux/smp.h>
#include <linux/workqueue.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include <asm/delay.h>
#include <asm/machvec.h>
/* Glue code to lib/swiotlb.c */
#include <linux/pci.h>
+#include <linux/gfp.h>
#include <linux/cache.h>
#include <linux/module.h>
#include <linux/dma-mapping.h>
#include <linux/rcupdate.h>
#include <linux/completion.h>
#include <linux/tracehook.h>
+#include <linux/slab.h>
#include <asm/errno.h>
#include <asm/intrinsics.h>
#include <linux/kallsyms.h>
#include <linux/kernel.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/notifier.h>
#include <linux/personality.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/stddef.h>
#include <linux/thread_info.h>
#include <linux/unistd.h>
*/
#include <linux/kernel.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/errno.h>
#include <linux/ptrace.h>
#include <linux/kernel.h>
#include <linux/mm.h>
#include <linux/node.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/bootmem.h>
#include <linux/nodemask.h>
#include <linux/init.h>
#include <linux/errno.h>
#include <linux/string.h>
-#include <linux/slab.h>
#include <linux/efi.h>
#include <linux/genalloc.h>
+#include <linux/gfp.h>
#include <asm/page.h>
#include <asm/pal.h>
#include <asm/system.h>
#include <linux/module.h>
#include <linux/errno.h>
#include <linux/percpu.h>
-#include <linux/gfp.h>
#include <linux/fs.h>
+#include <linux/slab.h>
#include <linux/smp.h>
#include <linux/kvm_host.h>
#include <linux/kvm.h>
#include <linux/acpi.h>
#include <linux/efi.h>
#include <linux/nodemask.h>
+#include <linux/slab.h>
#include <asm/pgalloc.h>
#include <asm/tlb.h>
#include <asm/meminit.h>
#include <linux/hugetlb.h>
#include <linux/pagemap.h>
#include <linux/module.h>
-#include <linux/slab.h>
#include <linux/sysctl.h>
#include <linux/log2.h>
#include <asm/mman.h>
#include <linux/smp.h>
#include <linux/mm.h>
#include <linux/bootmem.h>
+#include <linux/slab.h>
#include <asm/delay.h>
#include <asm/mmu_context.h>
#include <linux/bootmem.h>
#include <linux/string.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <asm/sn/bte.h>
#include <asm/sn/sn_sal.h>
#include "xtalk/hubdev.h"
#include <linux/acpi.h>
+#include <linux/slab.h>
/*
*/
#include <linux/bootmem.h>
+#include <linux/slab.h>
#include <asm/sn/types.h>
#include <asm/sn/addrs.h>
#include <asm/sn/sn_feature_sets.h>
* Copyright (C) 1992 - 1997, 2000-2006 Silicon Graphics, Inc. All rights reserved.
*/
+#include <linux/slab.h>
#include <asm/sn/types.h>
#include <asm/sn/addrs.h>
#include <asm/sn/io.h>
#include <linux/spinlock.h>
#include <linux/init.h>
#include <linux/rculist.h>
+#include <linux/slab.h>
#include <asm/sn/addrs.h>
#include <asm/sn/arch.h>
#include <asm/sn/intr.h>
#include <linux/pci.h>
#include <linux/cpumask.h>
#include <linux/msi.h>
+#include <linux/slab.h>
#include <asm/sn/addrs.h>
#include <asm/sn/intr.h>
* a description of how these routines should be used.
*/
+#include <linux/gfp.h>
#include <linux/module.h>
#include <linux/dma-mapping.h>
#include <asm/dma.h>
#include <linux/interrupt.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <asm/sn/addrs.h>
#include <asm/sn/geo.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
#include <linux/bitmap.h>
+#include <linux/slab.h>
#include <asm/sn/sn_sal.h>
#include <asm/sn/addrs.h>
#include <asm/sn/io.h>
#include <linux/types.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <asm/sn/sn_sal.h>
#include <asm/sn/addrs.h>
#include <linux/module.h>
#include <linux/vmalloc.h>
+#include <linux/slab.h>
#include <linux/mm.h>
#include <xen/interface/xen.h>
*/
#include <linux/fs.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/ptrace.h>
#include <linux/unistd.h>
-#include <linux/slab.h>
#include <linux/hardirq.h>
#include <asm/io.h>
#include <linux/bitops.h>
#include <linux/nodemask.h>
#include <linux/pfn.h>
+#include <linux/gfp.h>
#include <asm/types.h>
#include <asm/processor.h>
#include <asm/page.h>
#include <linux/types.h>
#include <linux/errno.h>
#include <linux/miscdevice.h>
-#include <linux/slab.h>
#include <linux/smp_lock.h>
#include <linux/ioport.h>
#include <linux/capability.h>
#include <linux/device.h>
#include <linux/kernel.h>
#include <linux/scatterlist.h>
+#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <asm/pgalloc.h>
#include <linux/sched.h>
#include <linux/kernel.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include <linux/fs.h>
#include <linux/smp.h>
#include <linux/smp_lock.h>
#include <linux/stddef.h>
#include <linux/unistd.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/user.h>
#include <linux/reboot.h>
#include <linux/init_task.h>
#include <linux/kernel.h>
#include <linux/delay.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/time.h>
#include <linux/rtc.h>
#include <linux/mm.h>
#include <linux/types.h>
#include <linux/init.h>
#include <linux/bootmem.h>
+#include <linux/gfp.h>
#include <asm/setup.h>
#include <asm/uaccess.h>
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/types.h>
-#include <linux/slab.h>
#include <linux/init.h>
#include <linux/pagemap.h>
+#include <linux/gfp.h>
#include <asm/setup.h>
#include <asm/segment.h>
#include <linux/types.h>
#include <linux/init.h>
#include <linux/bootmem.h>
+#include <linux/gfp.h>
#include <asm/setup.h>
#include <asm/uaccess.h>
#include <linux/types.h>
#include <linux/errno.h>
#include <linux/miscdevice.h>
-#include <linux/slab.h>
#include <linux/smp_lock.h>
#include <linux/ioport.h>
#include <linux/capability.h>
#include <linux/module.h>
#include <linux/kernel.h>
+#include <linux/gfp.h>
#include <linux/mm.h>
#include <linux/list.h>
#include <linux/bitops.h>
#include <linux/mm.h>
#include <linux/bootmem.h>
-#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <asm/sun3x.h>
*/
#include <linux/types.h>
+#include <linux/gfp.h>
#include <linux/mm.h>
#include <linux/device.h>
#include <linux/dma-mapping.h>
#include <linux/stddef.h>
#include <linux/unistd.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/user.h>
#include <linux/interrupt.h>
#include <linux/reboot.h>
#include <linux/fs.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/system.h>
#include <linux/highmem.h>
#include <linux/pagemap.h>
#include <linux/bootmem.h>
-#include <linux/slab.h>
+#include <linux/gfp.h>
#include <asm/setup.h>
#include <asm/segment.h>
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/types.h>
-#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <asm/setup.h>
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/types.h>
-#include <linux/slab.h>
#include <asm/segment.h>
#include <asm/page.h>
config HAVE_LATENCYTOP_SUPPORT
def_bool y
-config PCI
- def_bool n
-
config DTC
def_bool y
echo '* linux.bin - Create raw binary'
echo ' linux.bin.gz - Create compressed raw binary'
echo ' simpleImage.<dt> - ELF image with $(arch)/boot/dts/<dt>.dts linked in'
- echo ' - stripped elf with fdt blob
+ echo ' - stripped elf with fdt blob'
echo ' simpleImage.<dt>.unstrip - full ELF image with fdt blob'
echo ' *_defconfig - Select default config from arch/microblaze/configs'
echo ''
echo ' name of a dts file from the arch/microblaze/boot/dts/ directory'
echo ' (minus the .dts extension).'
endef
+
+MRPROPER_FILES += $(boot)/simpleImage.*
endif
$(obj)/linux.bin: vmlinux FORCE
- [ -n $(CONFIG_INITRAMFS_SOURCE) ] && [ ! -e $(CONFIG_INITRAMFS_SOURCE) ] && \
- touch $(CONFIG_INITRAMFS_SOURCE) || echo "No CPIO image"
$(call if_changed,objcopy)
$(call if_changed,uimage)
@echo 'Kernel: $@ is ready' ' (#'`cat .version`')'
$(obj)/%.dtb: $(dtstree)/%.dts FORCE
$(call if_changed,dtc)
-clean-kernel += linux.bin linux.bin.gz simpleImage.*
-
-clean-files += *.dtb simpleImage.*.unstrip
+clean-files += *.dtb simpleImage.*.unstrip linux.bin.ub
#include <asm/ptrace.h>
#include <asm/setup.h>
#include <asm/registers.h>
-#include <asm/segment.h>
#include <asm/entry.h>
#include <asm/current.h>
+++ /dev/null
-/*
- * Copyright (C) 2008-2009 Michal Simek <monstr@monstr.eu>
- * Copyright (C) 2008-2009 PetaLogix
- * Copyright (C) 2006 Atmark Techno, Inc.
- *
- * This file is subject to the terms and conditions of the GNU General Public
- * License. See the file "COPYING" in the main directory of this archive
- * for more details.
- */
-
-#ifndef _ASM_MICROBLAZE_SEGMENT_H
-#define _ASM_MICROBLAZE_SEGMENT_H
-
-# ifndef __ASSEMBLY__
-
-typedef struct {
- unsigned long seg;
-} mm_segment_t;
-
-/*
- * On Microblaze the fs value is actually the top of the corresponding
- * address space.
- *
- * The fs value determines whether argument validity checking should be
- * performed or not. If get_fs() == USER_DS, checking is performed, with
- * get_fs() == KERNEL_DS, checking is bypassed.
- *
- * For historical reasons, these macros are grossly misnamed.
- *
- * For non-MMU arch like Microblaze, KERNEL_DS and USER_DS is equal.
- */
-# define MAKE_MM_SEG(s) ((mm_segment_t) { (s) })
-
-# ifndef CONFIG_MMU
-# define KERNEL_DS MAKE_MM_SEG(0)
-# define USER_DS KERNEL_DS
-# else
-# define KERNEL_DS MAKE_MM_SEG(0xFFFFFFFF)
-# define USER_DS MAKE_MM_SEG(TASK_SIZE - 1)
-# endif
-
-# define get_ds() (KERNEL_DS)
-# define get_fs() (current_thread_info()->addr_limit)
-# define set_fs(val) (current_thread_info()->addr_limit = (val))
-
-# define segment_eq(a, b) ((a).seg == (b).seg)
-
-# endif /* __ASSEMBLY__ */
-#endif /* _ASM_MICROBLAZE_SEGMENT_H */
#ifndef __ASSEMBLY__
# include <linux/types.h>
# include <asm/processor.h>
-# include <asm/segment.h>
/*
* low level task data that entry.S needs immediate access to
__u32 fsr;
};
+typedef struct {
+ unsigned long seg;
+} mm_segment_t;
+
struct thread_info {
struct task_struct *task; /* main task structure */
struct exec_domain *exec_domain; /* execution domain */
extern void _tlbia(void);
#define __tlbia() { preempt_disable(); _tlbia(); preempt_enable(); }
+#define __tlbie(x) { _tlbie(x); }
static inline void local_flush_tlb_all(void)
{ __tlbia(); }
{ __tlbia(); }
static inline void local_flush_tlb_page(struct vm_area_struct *vma,
unsigned long vmaddr)
- { _tlbie(vmaddr); }
+ { __tlbie(vmaddr); }
static inline void local_flush_tlb_range(struct vm_area_struct *vma,
unsigned long start, unsigned long end)
{ __tlbia(); }
#include <asm/mmu.h>
#include <asm/page.h>
#include <asm/pgtable.h>
-#include <asm/segment.h>
#include <linux/string.h>
#define VERIFY_READ 0
#define VERIFY_WRITE 1
-#define __clear_user(addr, n) (memset((void *)(addr), 0, (n)), 0)
-
-#ifndef CONFIG_MMU
-
-extern int ___range_ok(unsigned long addr, unsigned long size);
-
-#define __range_ok(addr, size) \
- ___range_ok((unsigned long)(addr), (unsigned long)(size))
-
-#define access_ok(type, addr, size) (__range_ok((addr), (size)) == 0)
-#define __access_ok(add, size) (__range_ok((addr), (size)) == 0)
-
-/* Undefined function to trigger linker error */
-extern int bad_user_access_length(void);
-
-/* FIXME this is function for optimalization -> memcpy */
-#define __get_user(var, ptr) \
-({ \
- int __gu_err = 0; \
- switch (sizeof(*(ptr))) { \
- case 1: \
- case 2: \
- case 4: \
- (var) = *(ptr); \
- break; \
- case 8: \
- memcpy((void *) &(var), (ptr), 8); \
- break; \
- default: \
- (var) = 0; \
- __gu_err = __get_user_bad(); \
- break; \
- } \
- __gu_err; \
-})
+/*
+ * On Microblaze the fs value is actually the top of the corresponding
+ * address space.
+ *
+ * The fs value determines whether argument validity checking should be
+ * performed or not. If get_fs() == USER_DS, checking is performed, with
+ * get_fs() == KERNEL_DS, checking is bypassed.
+ *
+ * For historical reasons, these macros are grossly misnamed.
+ *
+ * For non-MMU arch like Microblaze, KERNEL_DS and USER_DS is equal.
+ */
+# define MAKE_MM_SEG(s) ((mm_segment_t) { (s) })
-#define __get_user_bad() (bad_user_access_length(), (-EFAULT))
+# ifndef CONFIG_MMU
+# define KERNEL_DS MAKE_MM_SEG(0)
+# define USER_DS KERNEL_DS
+# else
+# define KERNEL_DS MAKE_MM_SEG(0xFFFFFFFF)
+# define USER_DS MAKE_MM_SEG(TASK_SIZE - 1)
+# endif
-/* FIXME is not there defined __pu_val */
-#define __put_user(var, ptr) \
-({ \
- int __pu_err = 0; \
- switch (sizeof(*(ptr))) { \
- case 1: \
- case 2: \
- case 4: \
- *(ptr) = (var); \
- break; \
- case 8: { \
- typeof(*(ptr)) __pu_val = (var); \
- memcpy(ptr, &__pu_val, sizeof(__pu_val)); \
- } \
- break; \
- default: \
- __pu_err = __put_user_bad(); \
- break; \
- } \
- __pu_err; \
-})
+# define get_ds() (KERNEL_DS)
+# define get_fs() (current_thread_info()->addr_limit)
+# define set_fs(val) (current_thread_info()->addr_limit = (val))
-#define __put_user_bad() (bad_user_access_length(), (-EFAULT))
+# define segment_eq(a, b) ((a).seg == (b).seg)
-#define put_user(x, ptr) __put_user((x), (ptr))
-#define get_user(x, ptr) __get_user((x), (ptr))
+/*
+ * The exception table consists of pairs of addresses: the first is the
+ * address of an instruction that is allowed to fault, and the second is
+ * the address at which the program should continue. No registers are
+ * modified, so it is entirely up to the continuation code to figure out
+ * what to do.
+ *
+ * All the routines below use bits of fixup code that are out of line
+ * with the main instruction path. This means when everything is well,
+ * we don't even have to jump over them. Further, they do not intrude
+ * on our cache or tlb entries.
+ */
+struct exception_table_entry {
+ unsigned long insn, fixup;
+};
-#define copy_to_user(to, from, n) (memcpy((to), (from), (n)), 0)
-#define copy_from_user(to, from, n) (memcpy((to), (from), (n)), 0)
+/* Returns 0 if exception not found and fixup otherwise. */
+extern unsigned long search_exception_table(unsigned long);
-#define __copy_to_user(to, from, n) (copy_to_user((to), (from), (n)))
-#define __copy_from_user(to, from, n) (copy_from_user((to), (from), (n)))
-#define __copy_to_user_inatomic(to, from, n) \
- (__copy_to_user((to), (from), (n)))
-#define __copy_from_user_inatomic(to, from, n) \
- (__copy_from_user((to), (from), (n)))
+#ifndef CONFIG_MMU
-static inline unsigned long clear_user(void *addr, unsigned long size)
+/* Check against bounds of physical memory */
+static inline int ___range_ok(unsigned long addr, unsigned long size)
{
- if (access_ok(VERIFY_WRITE, addr, size))
- size = __clear_user(addr, size);
- return size;
+ return ((addr < memory_start) ||
+ ((addr + size) > memory_end));
}
-/* Returns 0 if exception not found and fixup otherwise. */
-extern unsigned long search_exception_table(unsigned long);
+#define __range_ok(addr, size) \
+ ___range_ok((unsigned long)(addr), (unsigned long)(size))
-extern long strncpy_from_user(char *dst, const char *src, long count);
-extern long strnlen_user(const char *src, long count);
+#define access_ok(type, addr, size) (__range_ok((addr), (size)) == 0)
-#else /* CONFIG_MMU */
+#else
/*
* Address is valid if:
/* || printk("access_ok failed for %s at 0x%08lx (size %d), seg 0x%08x\n",
type?"WRITE":"READ",addr,size,get_fs().seg)) */
-/*
- * All the __XXX versions macros/functions below do not perform
- * access checking. It is assumed that the necessary checks have been
- * already performed before the finction (macro) is called.
- */
+#endif
-#define get_user(x, ptr) \
-({ \
- access_ok(VERIFY_READ, (ptr), sizeof(*(ptr))) \
- ? __get_user((x), (ptr)) : -EFAULT; \
-})
+#ifdef CONFIG_MMU
+# define __FIXUP_SECTION ".section .fixup,\"ax\"\n"
+# define __EX_TABLE_SECTION ".section __ex_table,\"a\"\n"
+#else
+# define __FIXUP_SECTION ".section .discard,\"ax\"\n"
+# define __EX_TABLE_SECTION ".section .discard,\"a\"\n"
+#endif
-#define put_user(x, ptr) \
-({ \
- access_ok(VERIFY_WRITE, (ptr), sizeof(*(ptr))) \
- ? __put_user((x), (ptr)) : -EFAULT; \
+extern unsigned long __copy_tofrom_user(void __user *to,
+ const void __user *from, unsigned long size);
+
+/* Return: number of not copied bytes, i.e. 0 if OK or non-zero if fail. */
+static inline unsigned long __must_check __clear_user(void __user *to,
+ unsigned long n)
+{
+ /* normal memset with two words to __ex_table */
+ __asm__ __volatile__ ( \
+ "1: sb r0, %2, r0;" \
+ " addik %0, %0, -1;" \
+ " bneid %0, 1b;" \
+ " addik %2, %2, 1;" \
+ "2: " \
+ __EX_TABLE_SECTION \
+ ".word 1b,2b;" \
+ ".previous;" \
+ : "=r"(n) \
+ : "0"(n), "r"(to)
+ );
+ return n;
+}
+
+static inline unsigned long __must_check clear_user(void __user *to,
+ unsigned long n)
+{
+ might_sleep();
+ if (unlikely(!access_ok(VERIFY_WRITE, to, n)))
+ return n;
+
+ return __clear_user(to, n);
+}
+
+/* put_user and get_user macros */
+extern long __user_bad(void);
+
+#define __get_user_asm(insn, __gu_ptr, __gu_val, __gu_err) \
+({ \
+ __asm__ __volatile__ ( \
+ "1:" insn " %1, %2, r0;" \
+ " addk %0, r0, r0;" \
+ "2: " \
+ __FIXUP_SECTION \
+ "3: brid 2b;" \
+ " addik %0, r0, %3;" \
+ ".previous;" \
+ __EX_TABLE_SECTION \
+ ".word 1b,3b;" \
+ ".previous;" \
+ : "=&r"(__gu_err), "=r"(__gu_val) \
+ : "r"(__gu_ptr), "i"(-EFAULT) \
+ ); \
})
+/**
+ * get_user: - Get a simple variable from user space.
+ * @x: Variable to store result.
+ * @ptr: Source address, in user space.
+ *
+ * Context: User context only. This function may sleep.
+ *
+ * This macro copies a single simple variable from user space to kernel
+ * space. It supports simple types like char and int, but not larger
+ * data types like structures or arrays.
+ *
+ * @ptr must have pointer-to-simple-variable type, and the result of
+ * dereferencing @ptr must be assignable to @x without a cast.
+ *
+ * Returns zero on success, or -EFAULT on error.
+ * On error, the variable @x is set to zero.
+ */
+
#define __get_user(x, ptr) \
({ \
unsigned long __gu_val; \
__get_user_asm("lw", (ptr), __gu_val, __gu_err); \
break; \
default: \
- __gu_val = 0; __gu_err = -EINVAL; \
+ /* __gu_val = 0; __gu_err = -EINVAL;*/ __gu_err = __user_bad();\
} \
x = (__typeof__(*(ptr))) __gu_val; \
__gu_err; \
})
-#define __get_user_asm(insn, __gu_ptr, __gu_val, __gu_err) \
+
+#define get_user(x, ptr) \
({ \
- __asm__ __volatile__ ( \
- "1:" insn " %1, %2, r0; \
- addk %0, r0, r0; \
- 2: \
- .section .fixup,\"ax\"; \
- 3: brid 2b; \
- addik %0, r0, %3; \
- .previous; \
- .section __ex_table,\"a\"; \
- .word 1b,3b; \
- .previous;" \
- : "=r"(__gu_err), "=r"(__gu_val) \
- : "r"(__gu_ptr), "i"(-EFAULT) \
- ); \
+ access_ok(VERIFY_READ, (ptr), sizeof(*(ptr))) \
+ ? __get_user((x), (ptr)) : -EFAULT; \
+})
+
+#define __put_user_asm(insn, __gu_ptr, __gu_val, __gu_err) \
+({ \
+ __asm__ __volatile__ ( \
+ "1:" insn " %1, %2, r0;" \
+ " addk %0, r0, r0;" \
+ "2: " \
+ __FIXUP_SECTION \
+ "3: brid 2b;" \
+ " addik %0, r0, %3;" \
+ ".previous;" \
+ __EX_TABLE_SECTION \
+ ".word 1b,3b;" \
+ ".previous;" \
+ : "=&r"(__gu_err) \
+ : "r"(__gu_val), "r"(__gu_ptr), "i"(-EFAULT) \
+ ); \
})
+#define __put_user_asm_8(__gu_ptr, __gu_val, __gu_err) \
+({ \
+ __asm__ __volatile__ (" lwi %0, %1, 0;" \
+ "1: swi %0, %2, 0;" \
+ " lwi %0, %1, 4;" \
+ "2: swi %0, %2, 4;" \
+ " addk %0, r0, r0;" \
+ "3: " \
+ __FIXUP_SECTION \
+ "4: brid 3b;" \
+ " addik %0, r0, %3;" \
+ ".previous;" \
+ __EX_TABLE_SECTION \
+ ".word 1b,4b,2b,4b;" \
+ ".previous;" \
+ : "=&r"(__gu_err) \
+ : "r"(&__gu_val), "r"(__gu_ptr), "i"(-EFAULT) \
+ ); \
+})
+
+/**
+ * put_user: - Write a simple value into user space.
+ * @x: Value to copy to user space.
+ * @ptr: Destination address, in user space.
+ *
+ * Context: User context only. This function may sleep.
+ *
+ * This macro copies a single simple value from kernel space to user
+ * space. It supports simple types like char and int, but not larger
+ * data types like structures or arrays.
+ *
+ * @ptr must have pointer-to-simple-variable type, and @x must be assignable
+ * to the result of dereferencing @ptr.
+ *
+ * Returns zero on success, or -EFAULT on error.
+ */
+
#define __put_user(x, ptr) \
({ \
__typeof__(*(ptr)) volatile __gu_val = (x); \
case 1: \
__put_user_asm("sb", (ptr), __gu_val, __gu_err); \
break; \
- case 2: \
+ case 2: \
__put_user_asm("sh", (ptr), __gu_val, __gu_err); \
break; \
case 4: \
__put_user_asm_8((ptr), __gu_val, __gu_err); \
break; \
default: \
- __gu_err = -EINVAL; \
+ /*__gu_err = -EINVAL;*/ __gu_err = __user_bad(); \
} \
__gu_err; \
})
-#define __put_user_asm_8(__gu_ptr, __gu_val, __gu_err) \
-({ \
-__asm__ __volatile__ (" lwi %0, %1, 0; \
- 1: swi %0, %2, 0; \
- lwi %0, %1, 4; \
- 2: swi %0, %2, 4; \
- addk %0,r0,r0; \
- 3: \
- .section .fixup,\"ax\"; \
- 4: brid 3b; \
- addik %0, r0, %3; \
- .previous; \
- .section __ex_table,\"a\"; \
- .word 1b,4b,2b,4b; \
- .previous;" \
- : "=&r"(__gu_err) \
- : "r"(&__gu_val), \
- "r"(__gu_ptr), "i"(-EFAULT) \
- ); \
-})
+#ifndef CONFIG_MMU
-#define __put_user_asm(insn, __gu_ptr, __gu_val, __gu_err) \
-({ \
- __asm__ __volatile__ ( \
- "1:" insn " %1, %2, r0; \
- addk %0, r0, r0; \
- 2: \
- .section .fixup,\"ax\"; \
- 3: brid 2b; \
- addik %0, r0, %3; \
- .previous; \
- .section __ex_table,\"a\"; \
- .word 1b,3b; \
- .previous;" \
- : "=r"(__gu_err) \
- : "r"(__gu_val), "r"(__gu_ptr), "i"(-EFAULT) \
- ); \
-})
+#define put_user(x, ptr) __put_user((x), (ptr))
-/*
- * Return: number of not copied bytes, i.e. 0 if OK or non-zero if fail.
- */
-static inline int clear_user(char *to, int size)
-{
- if (size && access_ok(VERIFY_WRITE, to, size)) {
- __asm__ __volatile__ (" \
- 1: \
- sb r0, %2, r0; \
- addik %0, %0, -1; \
- bneid %0, 1b; \
- addik %2, %2, 1; \
- 2: \
- .section __ex_table,\"a\"; \
- .word 1b,2b; \
- .section .text;" \
- : "=r"(size) \
- : "0"(size), "r"(to)
- );
- }
- return size;
-}
+#else /* CONFIG_MMU */
-#define __copy_from_user(to, from, n) copy_from_user((to), (from), (n))
+#define put_user(x, ptr) \
+({ \
+ access_ok(VERIFY_WRITE, (ptr), sizeof(*(ptr))) \
+ ? __put_user((x), (ptr)) : -EFAULT; \
+})
+#endif /* CONFIG_MMU */
+
+/* copy_to_from_user */
+#define __copy_from_user(to, from, n) \
+ __copy_tofrom_user((__force void __user *)(to), \
+ (void __user *)(from), (n))
#define __copy_from_user_inatomic(to, from, n) \
copy_from_user((to), (from), (n))
-#define copy_to_user(to, from, n) \
- (access_ok(VERIFY_WRITE, (to), (n)) ? \
- __copy_tofrom_user((void __user *)(to), \
- (__force const void __user *)(from), (n)) \
- : -EFAULT)
+static inline long copy_from_user(void *to,
+ const void __user *from, unsigned long n)
+{
+ might_sleep();
+ if (access_ok(VERIFY_READ, from, n))
+ return __copy_from_user(to, from, n);
+ return n;
+}
-#define __copy_to_user(to, from, n) copy_to_user((to), (from), (n))
+#define __copy_to_user(to, from, n) \
+ __copy_tofrom_user((void __user *)(to), \
+ (__force const void __user *)(from), (n))
#define __copy_to_user_inatomic(to, from, n) copy_to_user((to), (from), (n))
-#define copy_from_user(to, from, n) \
- (access_ok(VERIFY_READ, (from), (n)) ? \
- __copy_tofrom_user((__force void __user *)(to), \
- (void __user *)(from), (n)) \
- : -EFAULT)
+static inline long copy_to_user(void __user *to,
+ const void *from, unsigned long n)
+{
+ might_sleep();
+ if (access_ok(VERIFY_WRITE, to, n))
+ return __copy_to_user(to, from, n);
+ return n;
+}
+/*
+ * Copy a null terminated string from userspace.
+ */
extern int __strncpy_user(char *to, const char __user *from, int len);
-extern int __strnlen_user(const char __user *sstr, int len);
-#define strncpy_from_user(to, from, len) \
- (access_ok(VERIFY_READ, from, 1) ? \
- __strncpy_user(to, from, len) : -EFAULT)
-#define strnlen_user(str, len) \
- (access_ok(VERIFY_READ, str, 1) ? __strnlen_user(str, len) : 0)
+#define __strncpy_from_user __strncpy_user
-#endif /* CONFIG_MMU */
-
-extern unsigned long __copy_tofrom_user(void __user *to,
- const void __user *from, unsigned long size);
+static inline long
+strncpy_from_user(char *dst, const char __user *src, long count)
+{
+ if (!access_ok(VERIFY_READ, src, 1))
+ return -EFAULT;
+ return __strncpy_from_user(dst, src, count);
+}
/*
- * The exception table consists of pairs of addresses: the first is the
- * address of an instruction that is allowed to fault, and the second is
- * the address at which the program should continue. No registers are
- * modified, so it is entirely up to the continuation code to figure out
- * what to do.
+ * Return the size of a string (including the ending 0)
*
- * All the routines below use bits of fixup code that are out of line
- * with the main instruction path. This means when everything is well,
- * we don't even have to jump over them. Further, they do not intrude
- * on our cache or tlb entries.
+ * Return 0 on exception, a value greater than N if too long
*/
-struct exception_table_entry {
- unsigned long insn, fixup;
-};
+extern int __strnlen_user(const char __user *sstr, int len);
+
+static inline long strnlen_user(const char __user *src, long n)
+{
+ if (!access_ok(VERIFY_READ, src, 1))
+ return 0;
+ return __strnlen_user(src, n);
+}
#endif /* __ASSEMBLY__ */
#endif /* __KERNEL__ */
*/
#include <linux/init.h>
-#include <linux/slab.h>
#include <asm/cpuinfo.h>
#include <asm/pvr.h>
#include <linux/device.h>
#include <linux/dma-mapping.h>
+#include <linux/gfp.h>
#include <linux/dma-debug.h>
#include <asm/bug.h>
#include <asm/cacheflush.h>
static unsigned long get_dma_direct_offset(struct device *dev)
{
- if (dev)
+ if (likely(dev))
return (unsigned long)dev->archdata.dma_data;
return PCI_DRAM_OFFSET; /* FIXME Not sure if is correct */
.text
ENTRY(_start)
+#if CONFIG_KERNEL_BASE_ADDR == 0
+ brai TOPHYS(real_start)
+ .org 0x100
+real_start:
+#endif
+
mfs r1, rmsr
andi r1, r1, ~2
mts rmsr, r1
tophys(r4,r4) /* convert to phys address */
ori r3, r0, COMMAND_LINE_SIZE - 1 /* number of loops */
_copy_command_line:
- lbu r2, r5, r6 /* r7=r5+r6 - r5 contain pointer to command line */
- sb r2, r4, r6 /* addr[r4+r6]= r7*/
+ lbu r2, r5, r6 /* r2=r5+r6 - r5 contain pointer to command line */
+ sb r2, r4, r6 /* addr[r4+r6]= r2*/
addik r6, r6, 1 /* increment counting */
bgtid r3, _copy_command_line /* loop for all entries */
addik r3, r3, -1 /* descrement loop */
* virtual to physical.
*/
nop
- addik r3, r0, 63 /* Invalidate all TLB entries */
+ addik r3, r0, MICROBLAZE_TLB_SIZE -1 /* Invalidate all TLB entries */
_invalidate:
mts rtlbx, r3
mts rtlbhi, r0 /* flush: ensure V is clear */
mfs r5, rmsr;
nop
swi r5, r1, 0;
- mfs r3, resr
+ mfs r4, resr
nop
- mfs r4, rear;
+ mfs r3, rear;
nop
#ifndef CONFIG_MMU
- andi r5, r3, 0x1000; /* Check ESR[DS] */
+ andi r5, r4, 0x1000; /* Check ESR[DS] */
beqi r5, not_in_delay_slot; /* Branch if ESR[DS] not set */
mfs r17, rbtr; /* ESR[DS] set - return address in BTR */
nop
swi r17, r1, PT_R17
#endif
- andi r5, r3, 0x1F; /* Extract ESR[EXC] */
+ andi r5, r4, 0x1F; /* Extract ESR[EXC] */
#ifdef CONFIG_MMU
/* Calculate exception vector offset = r5 << 2 */
addk r6, r5, r5; /* << 1 */
addk r6, r6, r6; /* << 2 */
+#ifdef DEBUG
/* counting which exception happen */
lwi r5, r0, 0x200 + TOPHYS(r0_ram)
addi r5, r5, 1
lwi r5, r6, 0x200 + TOPHYS(r0_ram)
addi r5, r5, 1
swi r5, r6, 0x200 + TOPHYS(r0_ram)
+#endif
/* end */
/* Load the HW Exception vector */
lwi r6, r6, TOPHYS(_MB_HW_ExceptionVectorTable)
swi r18, r1, PT_R18
or r5, r1, r0
- andi r6, r3, 0x1F; /* Load ESR[EC] */
+ andi r6, r4, 0x1F; /* Load ESR[EC] */
lwi r7, r0, PER_CPU(KM) /* MS: saving current kernel mode to regs */
swi r7, r1, PT_MODE
mfs r7, rfsr
*/
handle_unaligned_ex:
/* Working registers already saved: R3, R4, R5, R6
- * R3 = ESR
- * R4 = EAR
+ * R4 = ESR
+ * R3 = EAR
*/
#ifdef CONFIG_MMU
- andi r6, r3, 0x1000 /* Check ESR[DS] */
+ andi r6, r4, 0x1000 /* Check ESR[DS] */
beqi r6, _no_delayslot /* Branch if ESR[DS] not set */
mfs r17, rbtr; /* ESR[DS] set - return address in BTR */
nop
RESTORE_STATE;
bri unaligned_data_trap
#endif
- andi r6, r3, 0x3E0; /* Mask and extract the register operand */
+ andi r6, r4, 0x3E0; /* Mask and extract the register operand */
srl r6, r6; /* r6 >> 5 */
srl r6, r6;
srl r6, r6;
/* Store the register operand in a temporary location */
sbi r6, r0, TOPHYS(ex_reg_op);
- andi r6, r3, 0x400; /* Extract ESR[S] */
+ andi r6, r4, 0x400; /* Extract ESR[S] */
bnei r6, ex_sw;
ex_lw:
- andi r6, r3, 0x800; /* Extract ESR[W] */
+ andi r6, r4, 0x800; /* Extract ESR[W] */
beqi r6, ex_lhw;
- lbui r5, r4, 0; /* Exception address in r4 */
+ lbui r5, r3, 0; /* Exception address in r3 */
/* Load a word, byte-by-byte from destination address
and save it in tmp space */
sbi r5, r0, TOPHYS(ex_tmp_data_loc_0);
- lbui r5, r4, 1;
+ lbui r5, r3, 1;
sbi r5, r0, TOPHYS(ex_tmp_data_loc_1);
- lbui r5, r4, 2;
+ lbui r5, r3, 2;
sbi r5, r0, TOPHYS(ex_tmp_data_loc_2);
- lbui r5, r4, 3;
+ lbui r5, r3, 3;
sbi r5, r0, TOPHYS(ex_tmp_data_loc_3);
- /* Get the destination register value into r3 */
- lwi r3, r0, TOPHYS(ex_tmp_data_loc_0);
+ /* Get the destination register value into r4 */
+ lwi r4, r0, TOPHYS(ex_tmp_data_loc_0);
bri ex_lw_tail;
ex_lhw:
- lbui r5, r4, 0; /* Exception address in r4 */
+ lbui r5, r3, 0; /* Exception address in r3 */
/* Load a half-word, byte-by-byte from destination
address and save it in tmp space */
sbi r5, r0, TOPHYS(ex_tmp_data_loc_0);
- lbui r5, r4, 1;
+ lbui r5, r3, 1;
sbi r5, r0, TOPHYS(ex_tmp_data_loc_1);
- /* Get the destination register value into r3 */
- lhui r3, r0, TOPHYS(ex_tmp_data_loc_0);
+ /* Get the destination register value into r4 */
+ lhui r4, r0, TOPHYS(ex_tmp_data_loc_0);
ex_lw_tail:
/* Get the destination register number into r5 */
lbui r5, r0, TOPHYS(ex_reg_op);
andi r6, r6, 0x800; /* Extract ESR[W] */
beqi r6, ex_shw;
/* Get the word - delay slot */
- swi r3, r0, TOPHYS(ex_tmp_data_loc_0);
+ swi r4, r0, TOPHYS(ex_tmp_data_loc_0);
/* Store the word, byte-by-byte into destination address */
- lbui r3, r0, TOPHYS(ex_tmp_data_loc_0);
- sbi r3, r4, 0;
- lbui r3, r0, TOPHYS(ex_tmp_data_loc_1);
- sbi r3, r4, 1;
- lbui r3, r0, TOPHYS(ex_tmp_data_loc_2);
- sbi r3, r4, 2;
- lbui r3, r0, TOPHYS(ex_tmp_data_loc_3);
- sbi r3, r4, 3;
+ lbui r4, r0, TOPHYS(ex_tmp_data_loc_0);
+ sbi r4, r3, 0;
+ lbui r4, r0, TOPHYS(ex_tmp_data_loc_1);
+ sbi r4, r3, 1;
+ lbui r4, r0, TOPHYS(ex_tmp_data_loc_2);
+ sbi r4, r3, 2;
+ lbui r4, r0, TOPHYS(ex_tmp_data_loc_3);
+ sbi r4, r3, 3;
bri ex_handler_done;
ex_shw:
/* Store the lower half-word, byte-by-byte into destination address */
- swi r3, r0, TOPHYS(ex_tmp_data_loc_0);
- lbui r3, r0, TOPHYS(ex_tmp_data_loc_2);
- sbi r3, r4, 0;
- lbui r3, r0, TOPHYS(ex_tmp_data_loc_3);
- sbi r3, r4, 1;
+ swi r4, r0, TOPHYS(ex_tmp_data_loc_0);
+ lbui r4, r0, TOPHYS(ex_tmp_data_loc_2);
+ sbi r4, r3, 0;
+ lbui r4, r0, TOPHYS(ex_tmp_data_loc_3);
+ sbi r4, r3, 1;
ex_sw_end: /* Exception handling of store word, ends. */
ex_handler_done:
*/
mfs r11, rpid
nop
- bri 4
- mfs r3, rear /* Get faulting address */
- nop
/* If we are faulting a kernel address, we have to use the
* kernel page tables.
*/
- ori r4, r0, CONFIG_KERNEL_START
- cmpu r4, r3, r4
- bgti r4, ex3
+ ori r5, r0, CONFIG_KERNEL_START
+ cmpu r5, r3, r5
+ bgti r5, ex3
/* First, check if it was a zone fault (which means a user
* tried to access a kernel or read-protected page - always
* a SEGV). All other faults here must be stores, so no
* need to check ESR_S as well. */
- mfs r4, resr
- nop
andi r4, r4, 0x800 /* ESR_Z - zone protection */
bnei r4, ex2
* tried to access a kernel or read-protected page - always
* a SEGV). All other faults here must be stores, so no
* need to check ESR_S as well. */
- mfs r4, resr
- nop
andi r4, r4, 0x800 /* ESR_Z */
bnei r4, ex2
/* get current task address */
* R3 = ESR
*/
- mfs r3, rear /* Get faulting address */
- nop
RESTORE_STATE;
bri page_fault_instr_trap
*/
handle_data_tlb_miss_exception:
/* Working registers already saved: R3, R4, R5, R6
- * R3 = ESR
+ * R3 = EAR, R4 = ESR
*/
mfs r11, rpid
nop
- bri 4
- mfs r3, rear /* Get faulting address */
- nop
/* If we are faulting a kernel address, we have to use the
* kernel page tables. */
- ori r4, r0, CONFIG_KERNEL_START
- cmpu r4, r3, r4
+ ori r6, r0, CONFIG_KERNEL_START
+ cmpu r4, r3, r6
bgti r4, ex5
ori r4, r0, swapper_pg_dir
mts rpid, r0 /* TLB will have 0 TID */
* Many of these bits are software only. Bits we don't set
* here we (properly should) assume have the appropriate value.
*/
+ brid finish_tlb_load
andni r4, r4, 0x0ce2 /* Make sure 20, 21 are zero */
-
- bri finish_tlb_load
ex7:
/* The bailout. Restore registers to pre-exception conditions
* and call the heavyweights to help us out.
*/
mfs r11, rpid
nop
- bri 4
- mfs r3, rear /* Get faulting address */
- nop
/* If we are faulting a kernel address, we have to use the
* kernel page tables.
lwi r4, r5, 0 /* Get Linux PTE */
andi r6, r4, _PAGE_PRESENT
- beqi r6, ex7
+ beqi r6, ex10
ori r4, r4, _PAGE_ACCESSED
swi r4, r5, 0
* Many of these bits are software only. Bits we don't set
* here we (properly should) assume have the appropriate value.
*/
+ brid finish_tlb_load
andni r4, r4, 0x0ce2 /* Make sure 20, 21 are zero */
-
- bri finish_tlb_load
ex10:
/* The bailout. Restore registers to pre-exception conditions
* and call the heavyweights to help us out.
andi r5, r5, (MICROBLAZE_TLB_SIZE-1)
ori r6, r0, 1
cmp r31, r5, r6
- blti r31, sem
+ blti r31, ex12
addik r5, r6, 1
- sem:
+ ex12:
/* MS: save back current TLB index */
swi r5, r0, TOPHYS(tlb_index)
nop
/* Done...restore registers and get out of here. */
- ex12:
mts rpid, r11
nop
bri 4
* We avoid flushing the pinned 0, 1 and possibly 2 entries.
*/
.globl _tlbia;
+.type _tlbia, @function
.align 4;
_tlbia:
- addik r12, r0, 63 /* flush all entries (63 - 3) */
+ addik r12, r0, MICROBLAZE_TLB_SIZE - 1 /* flush all entries (63 - 3) */
/* isync */
_tlbia_1:
mts rtlbx, r12
/* sync */
rtsd r15, 8
nop
+ .size _tlbia, . - _tlbia
/*
* Flush MMU TLB for a particular address (in r5)
*/
.globl _tlbie;
+.type _tlbie, @function
.align 4;
_tlbie:
mts rtlbsx, r5 /* look up the address in TLB */
rtsd r15, 8
nop
+ .size _tlbie, . - _tlbie
+
/*
* Allocate TLB entry for early console
*/
.globl early_console_reg_tlb_alloc;
+.type early_console_reg_tlb_alloc, @function
.align 4;
early_console_reg_tlb_alloc:
/*
* Load a TLB entry for the UART, so that microblaze_progress() can use
* the UARTs nice and early. We use a 4k real==virtual mapping.
*/
- ori r4, r0, 63
+ ori r4, r0, MICROBLAZE_TLB_SIZE - 1
mts rtlbx, r4 /* TLB slot 2 */
or r4,r5,r0
rtsd r15, 8
nop
+ .size early_console_reg_tlb_alloc, . - early_console_reg_tlb_alloc
+
/*
* Copy a whole page (4096 bytes).
*/
#define DCACHE_LINE_BYTES (4 * 4)
.globl copy_page;
+.type copy_page, @function
.align 4;
copy_page:
ori r11, r0, (PAGE_SIZE/DCACHE_LINE_BYTES) - 1
addik r11, r11, -1
rtsd r15, 8
nop
+
+ .size copy_page, . - copy_page
#include <linux/kernel.h>
#include <linux/elf.h>
#include <linux/vmalloc.h>
-#include <linux/slab.h>
#include <linux/fs.h>
#include <linux/string.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/mod_devicetable.h>
-#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/of.h>
#include <linux/of_device.h>
#include <linux/bitops.h>
#include <asm/system.h>
#include <asm/pgalloc.h>
+#include <asm/uaccess.h> /* for USER_DS macros */
#include <asm/cacheflush.h>
void show_regs(struct pt_regs *regs)
void default_idle(void)
{
- if (!hlt_counter) {
+ if (likely(hlt_counter)) {
+ while (!need_resched())
+ cpu_relax();
+ } else {
clear_thread_flag(TIF_POLLING_NRFLAG);
smp_mb__after_clear_bit();
local_irq_disable();
cpu_sleep();
local_irq_enable();
set_thread_flag(TIF_POLLING_NRFLAG);
- } else
- while (!need_resched())
- cpu_relax();
+ }
}
void cpu_idle(void)
}
#endif /* CONFIG_MTD_UCLINUX_EBSS */
+#if defined(CONFIG_EARLY_PRINTK) && defined(CONFIG_SERIAL_UARTLITE_CONSOLE)
+#define eprintk early_printk
+#else
+#define eprintk printk
+#endif
+
void __init machine_early_init(const char *cmdline, unsigned int ram,
unsigned int fdt, unsigned int msr)
{
setup_early_printk(NULL);
#endif
- early_printk("Ramdisk addr 0x%08x, ", ram);
+ eprintk("Ramdisk addr 0x%08x, ", ram);
if (fdt)
- early_printk("FDT at 0x%08x\n", fdt);
+ eprintk("FDT at 0x%08x\n", fdt);
else
- early_printk("Compiled-in FDT at 0x%08x\n",
+ eprintk("Compiled-in FDT at 0x%08x\n",
(unsigned int)_fdt_start);
#ifdef CONFIG_MTD_UCLINUX
- early_printk("Found romfs @ 0x%08x (0x%08x)\n",
+ eprintk("Found romfs @ 0x%08x (0x%08x)\n",
romfs_base, romfs_size);
- early_printk("#### klimit %p ####\n", old_klimit);
+ eprintk("#### klimit %p ####\n", old_klimit);
BUG_ON(romfs_size < 0); /* What else can we do? */
- early_printk("Moved 0x%08x bytes from 0x%08x to 0x%08x\n",
+ eprintk("Moved 0x%08x bytes from 0x%08x to 0x%08x\n",
romfs_size, romfs_base, (unsigned)&_ebss);
- early_printk("New klimit: 0x%08x\n", (unsigned)klimit);
+ eprintk("New klimit: 0x%08x\n", (unsigned)klimit);
#endif
#if CONFIG_XILINX_MICROBLAZE0_USE_MSR_INSTR
if (msr)
- early_printk("!!!Your kernel has setup MSR instruction but "
+ eprintk("!!!Your kernel has setup MSR instruction but "
"CPU don't have it %d\n", msr);
#else
if (!msr)
- early_printk("!!!Your kernel not setup MSR instruction but "
+ eprintk("!!!Your kernel not setup MSR instruction but "
"CPU have it %d\n", msr);
#endif
#include <linux/semaphore.h>
#include <linux/uaccess.h>
#include <linux/unistd.h>
+#include <linux/slab.h>
#include <asm/syscalls.h>
__enable_hw_exceptions();
}
-static int kstack_depth_to_print = 24;
+static unsigned long kstack_depth_to_print = 24;
static int __init kstack_setup(char *s)
{
- kstack_depth_to_print = strict_strtoul(s, 0, NULL);
-
- return 1;
+ return !strict_strtoul(s, 0, &kstack_depth_to_print);
}
__setup("kstack=", kstack_setup);
lib-y += memcpy.o memmove.o
endif
-lib-$(CONFIG_NO_MMU) += uaccess.o
-lib-$(CONFIG_MMU) += uaccess_old.o
+lib-y += uaccess_old.o
*/
#include <linux/linkage.h>
-
+ .text
.globl memcpy
+ .type memcpy, @function
.ent memcpy
memcpy:
rtsd r15, 8
nop
+.size memcpy, . - memcpy
.end memcpy
/*----------------------------------------------------------------------------*/
.globl memmove
+ .type memmove, @function
.ent memmove
memmove:
rtsd r15, 8
nop
+.size memmove, . - memmove
.end memmove
const uint32_t *i_src;
uint32_t *i_dst;
- if (c >= 4) {
+ if (likely(c >= 4)) {
unsigned value, buf_hold;
/* Align the dstination to a word boundry. */
#ifdef __HAVE_ARCH_MEMSET
void *memset(void *v_src, int c, __kernel_size_t n)
{
-
char *src = v_src;
#ifdef CONFIG_OPT_LIB_FUNCTION
uint32_t *i_src;
- uint32_t w32;
+ uint32_t w32 = 0;
#endif
/* Truncate c to 8 bits */
c = (c & 0xFF);
#ifdef CONFIG_OPT_LIB_FUNCTION
- /* Make a repeating word out of it */
- w32 = c;
- w32 |= w32 << 8;
- w32 |= w32 << 16;
+ if (unlikely(c)) {
+ /* Make a repeating word out of it */
+ w32 = c;
+ w32 |= w32 << 8;
+ w32 |= w32 << 16;
+ }
- if (n >= 4) {
+ if (likely(n >= 4)) {
/* Align the destination to a word boundary */
/* This is done in an endian independant manner */
switch ((unsigned) src & 3) {
+++ /dev/null
-/*
- * Copyright (C) 2006 Atmark Techno, Inc.
- *
- * This file is subject to the terms and conditions of the GNU General Public
- * License. See the file "COPYING" in the main directory of this archive
- * for more details.
- */
-
-#include <linux/string.h>
-#include <asm/uaccess.h>
-
-#include <asm/bug.h>
-
-long strnlen_user(const char __user *src, long count)
-{
- return strlen(src) + 1;
-}
-
-#define __do_strncpy_from_user(dst, src, count, res) \
- do { \
- char *tmp; \
- strncpy(dst, src, count); \
- for (tmp = dst; *tmp && count > 0; tmp++, count--) \
- ; \
- res = (tmp - dst); \
- } while (0)
-
-long __strncpy_from_user(char *dst, const char __user *src, long count)
-{
- long res;
- __do_strncpy_from_user(dst, src, count, res);
- return res;
-}
-
-long strncpy_from_user(char *dst, const char __user *src, long count)
-{
- long res = -EFAULT;
- if (access_ok(VERIFY_READ, src, 1))
- __do_strncpy_from_user(dst, src, count, res);
- return res;
-}
-
-unsigned long __copy_tofrom_user(void __user *to,
- const void __user *from, unsigned long size)
-{
- memcpy(to, from, size);
- return 0;
-}
.text
.globl __strncpy_user;
+.type __strncpy_user, @function
.align 4;
__strncpy_user:
3:
rtsd r15,8
nop
-
+ .size __strncpy_user, . - __strncpy_user
.section .fixup, "ax"
.align 2
.text
.globl __strnlen_user;
+.type __strnlen_user, @function
.align 4;
__strnlen_user:
addik r3,r6,0
3:
rtsd r15,8
nop
-
+ .size __strnlen_user, . - __strnlen_user
.section .fixup,"ax"
4:
*/
.text
.globl __copy_tofrom_user;
+.type __copy_tofrom_user, @function
.align 4;
__copy_tofrom_user:
/*
* r7, r3 - count
* r4 - tempval
*/
- addik r3,r7,0
- beqi r3,3f
-1:
- lbu r4,r6,r0
- addik r6,r6,1
-2:
- sb r4,r5,r0
- addik r3,r3,-1
- bneid r3,1b
- addik r5,r5,1 /* delay slot */
+ beqid r7, 3f /* zero size is not likely */
+ andi r3, r7, 0x3 /* filter add count */
+ bneid r3, 4f /* if is odd value then byte copying */
+ or r3, r5, r6 /* find if is any to/from unaligned */
+ andi r3, r3, 0x3 /* mask unaligned */
+ bneid r3, 1f /* it is unaligned -> then jump */
+ or r3, r0, r0
+
+/* at least one 4 byte copy */
+5: lw r4, r6, r3
+6: sw r4, r5, r3
+ addik r7, r7, -4
+ bneid r7, 5b
+ addik r3, r3, 4
+ addik r3, r7, 0
+ rtsd r15, 8
+ nop
+4: or r3, r0, r0
+1: lbu r4,r6,r3
+2: sb r4,r5,r3
+ addik r7,r7,-1
+ bneid r7,1b
+ addik r3,r3,1 /* delay slot */
3:
+ addik r3,r7,0
rtsd r15,8
nop
-
+ .size __copy_tofrom_user, . - __copy_tofrom_user
.section __ex_table,"a"
- .word 1b,3b,2b,3b
+ .word 1b,3b,2b,3b,5b,3b,6b,3b
#include <linux/highmem.h>
#include <linux/pci.h>
#include <linux/interrupt.h>
+#include <linux/gfp.h>
#include <asm/pgalloc.h>
#include <linux/io.h>
regs->esr = error_code;
/* On a kernel SLB miss we can only check for a valid exception entry */
- if (kernel_mode(regs) && (address >= TASK_SIZE)) {
+ if (unlikely(kernel_mode(regs) && (address >= TASK_SIZE))) {
printk(KERN_WARNING "kernel task_size exceed");
_exception(SIGSEGV, regs, code, address);
}
}
#endif /* CONFIG_KGDB */
- if (in_atomic() || !mm) {
+ if (unlikely(in_atomic() || !mm)) {
if (kernel_mode(regs))
goto bad_area_nosemaphore;
* source. If this is invalid we can skip the address space check,
* thus avoiding the deadlock.
*/
- if (!down_read_trylock(&mm->mmap_sem)) {
+ if (unlikely(!down_read_trylock(&mm->mmap_sem))) {
if (kernel_mode(regs) && !search_exception_tables(regs->pc))
goto bad_area_nosemaphore;
}
vma = find_vma(mm, address);
- if (!vma)
+ if (unlikely(!vma))
goto bad_area;
if (vma->vm_start <= address)
goto good_area;
- if (!(vma->vm_flags & VM_GROWSDOWN))
+ if (unlikely(!(vma->vm_flags & VM_GROWSDOWN)))
goto bad_area;
- if (!is_write)
+ if (unlikely(!is_write))
goto bad_area;
/*
* before setting the user r1. Thus we allow the stack to
* expand to 1MB without further checks.
*/
- if (address + 0x100000 < vma->vm_end) {
+ if (unlikely(address + 0x100000 < vma->vm_end)) {
/* get user regs even if this fault is in kernel mode */
struct pt_regs *uregs = current->thread.regs;
code = SEGV_ACCERR;
/* a write */
- if (is_write) {
- if (!(vma->vm_flags & VM_WRITE))
+ if (unlikely(is_write)) {
+ if (unlikely(!(vma->vm_flags & VM_WRITE)))
goto bad_area;
/* a read */
} else {
/* protection fault */
- if (error_code & 0x08000000)
+ if (unlikely(error_code & 0x08000000))
goto bad_area;
- if (!(vma->vm_flags & (VM_READ | VM_EXEC)))
+ if (unlikely(!(vma->vm_flags & (VM_READ | VM_EXEC))))
goto bad_area;
}
goto do_sigbus;
BUG();
}
- if (fault & VM_FAULT_MAJOR)
+ if (unlikely(fault & VM_FAULT_MAJOR))
current->maj_flt++;
else
current->min_flt++;
#include <linux/initrd.h>
#include <linux/pagemap.h>
#include <linux/pfn.h>
+#include <linux/slab.h>
#include <linux/swap.h>
#include <asm/page.h>
for (addr = begin; addr < end; addr += PAGE_SIZE) {
ClearPageReserved(virt_to_page(addr));
init_page_count(virt_to_page(addr));
- memset((void *)addr, 0xcc, PAGE_SIZE);
free_page(addr);
totalram_pages++;
}
}
#ifndef CONFIG_MMU
-/* Check against bounds of physical memory */
-int ___range_ok(unsigned long addr, unsigned long size)
-{
- return ((addr < memory_start) ||
- ((addr + size) > memory_end));
-}
-EXPORT_SYMBOL(___range_ok);
-
int page_is_ram(unsigned long pfn)
{
return __range_ok(pfn, 0);
err = 0;
set_pte_at(&init_mm, va, pg, pfn_pte(pa >> PAGE_SHIFT,
__pgprot(flags)));
- if (mem_init_done)
+ if (unlikely(mem_init_done))
flush_HPTE(0, va, pmd_val(*pd));
/* flush_HPTE(0, va, pg); */
}
#include <linux/syscalls.h>
#include <linux/irq.h>
#include <linux/vmalloc.h>
+#include <linux/slab.h>
#include <asm/processor.h>
#include <asm/io.h>
#include <linux/irq.h>
#include <linux/list.h>
#include <linux/of.h>
+#include <linux/slab.h>
#include <asm/processor.h>
#include <asm/io.h>
#include <linux/mm.h>
#include <linux/bootmem.h>
#include <linux/spinlock.h>
+#include <linux/gfp.h>
#include <asm/mipsregs.h>
#include <asm/jazz.h>
#include <asm/io.h>
#include <linux/kernel_stat.h>
#include <linux/module.h>
#include <linux/proc_fs.h>
-#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/random.h>
#include <linux/sched.h>
#include <linux/time.h>
#include <linux/times.h>
#include <linux/poll.h>
-#include <linux/slab.h>
#include <linux/skbuff.h>
#include <linux/filter.h>
#include <linux/shm.h>
#include <linux/compat.h>
#include <linux/vfs.h>
#include <linux/ipc.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/scm.h>
#include <linux/stddef.h>
#include <linux/unistd.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/mman.h>
#include <linux/personality.h>
#include <linux/sys.h>
#include <linux/fs.h>
#include <linux/init.h>
#include <asm/uaccess.h>
-#include <linux/slab.h>
#include <linux/list.h>
#include <linux/vmalloc.h>
#include <linux/elf.h>
#include <linux/kernel_stat.h>
#include <linux/module.h>
#include <linux/ftrace.h>
+#include <linux/slab.h>
#include <asm/cpu.h>
#include <asm/processor.h>
#include <linux/string.h>
#include <linux/syscalls.h>
#include <linux/file.h>
-#include <linux/slab.h>
#include <linux/utsname.h>
#include <linux/unistd.h>
#include <linux/sem.h>
#include <linux/module.h>
#include <linux/ipc.h>
#include <linux/uaccess.h>
+#include <linux/slab.h>
#include <asm/asm.h>
#include <asm/branch.h>
*/
#include <linux/init.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/kernel_stat.h>
#include <asm/mips-boards/simint.h>
#include <linux/module.h>
#include <linux/scatterlist.h>
#include <linux/string.h>
+#include <linux/gfp.h>
#include <asm/cache.h>
#include <asm/io.h>
#include <linux/mm.h>
#include <linux/hugetlb.h>
#include <linux/pagemap.h>
-#include <linux/slab.h>
#include <linux/err.h>
#include <linux/sysctl.h>
#include <asm/mman.h>
#include <linux/proc_fs.h>
#include <linux/pfn.h>
#include <linux/hardirq.h>
+#include <linux/gfp.h>
#include <asm/asm-offsets.h>
#include <asm/bootinfo.h>
#include <asm/addrspace.h>
#include <asm/byteorder.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <asm/cacheflush.h>
#include <asm/io.h>
#include <linux/irq.h>
#include <linux/sched.h>
#include <linux/smp.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/io.h>
#include <linux/kernel_stat.h>
* along with this program; if not, write to the Free Software
* Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
-#include <linux/slab.h>
#include <linux/reboot.h>
#include <pnx833x.h>
#include <linux/init.h>
#include <linux/irq.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/kernel_stat.h>
#include <linux/random.h>
#include <linux/proc_fs.h>
#include <linux/irq.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/kernel_stat.h>
#include <linux/random.h>
* Reset the PNX8550 board.
*
*/
-#include <linux/slab.h>
#include <asm/reboot.h>
#include <glb.h>
#include <linux/types.h>
#include <linux/pci.h>
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <asm/io.h>
#include <linux/string.h>
#include <linux/interrupt.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include <asm/addrspace.h>
#include <asm/bootinfo.h>
#include <linux/types.h>
#include <linux/pci.h>
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <asm/pci.h>
#include <asm/io.h>
#include <linux/ioport.h>
#include <linux/irq.h>
#include <linux/timex.h>
-#include <linux/slab.h>
#include <linux/random.h>
#include <linux/bitops.h>
#include <asm/bootinfo.h>
#include <linux/mm.h>
#include <linux/platform_device.h>
#include <linux/module.h>
+#include <linux/gfp.h>
#include <asm/page.h>
#include <linux/swap.h>
#include <linux/highmem.h>
#include <linux/init.h>
#include <linux/irq.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/kernel_stat.h>
#include <linux/kernel.h>
#include <linux/interrupt.h>
#include <linux/ioport.h>
#include <linux/timex.h>
-#include <linux/slab.h>
#include <linux/random.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/ioport.h>
#include <linux/timex.h>
-#include <linux/slab.h>
#include <linux/smp.h>
#include <linux/random.h>
#include <linux/kernel.h>
#include <linux/irq.h>
#include <linux/bitops.h>
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/random.h>
#include <linux/sched.h>
#include <linux/smp.h>
#include <linux/spinlock.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/kernel_stat.h>
#include <asm/errno.h>
#include <linux/types.h>
#include <linux/init.h>
#include <linux/interrupt.h>
-#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/fs.h>
#include <linux/errno.h>
#include <linux/spinlock.h>
#include <linux/smp.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/kernel_stat.h>
#include <asm/errno.h>
#include <asm/txx9/pci.h>
#ifdef CONFIG_TOSHIBA_FPCIB0
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <asm/i8259.h>
#include <asm/txx9/smsc_fdc37m81x.h>
#endif
#include <linux/mtd/physmap.h>
#include <linux/leds.h>
#include <linux/sysdev.h>
+#include <linux/slab.h>
#include <asm/bootinfo.h>
#include <asm/time.h>
#include <asm/reboot.h>
* Support for TX4938 in 2.6 - Manish Lachwani (mlachwani@mvista.com)
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/device.h>
#include <linux/spi/spi.h>
#include <linux/spi/eeprom.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/platform_device.h>
#include <linux/leds.h>
#include <linux/interrupt.h>
#include <linux/stddef.h>
#include <linux/unistd.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/user.h>
#include <linux/interrupt.h>
#include <linux/delay.h>
#include <linux/percpu.h>
#include <linux/err.h>
#include <linux/fs.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/pgtable.h>
#include <asm/system.h>
#include <linux/stddef.h>
#include <linux/unistd.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/user.h>
#include <linux/tty.h>
#include <linux/ioport.h>
#include <linux/mm.h>
#include <linux/string.h>
#include <linux/pci.h>
+#include <linux/gfp.h>
#include <asm/io.h>
static unsigned long pci_sram_allocated = 0xbc000000;
#include <linux/types.h>
#include <linux/ptrace.h>
#include <linux/mman.h>
-#include <linux/slab.h>
#include <linux/fs.h>
#include <linux/mm.h>
#include <linux/swap.h>
#include <linux/highmem.h>
#include <linux/pagemap.h>
#include <linux/bootmem.h>
+#include <linux/gfp.h>
#include <asm/processor.h>
#include <asm/system.h>
#include <linux/sched.h>
#include <linux/kernel.h>
#include <linux/errno.h>
+#include <linux/gfp.h>
#include <linux/mm.h>
#include <linux/swap.h>
#include <linux/smp.h>
#include <linux/highmem.h>
-#include <linux/slab.h>
#include <linux/pagemap.h>
#include <linux/spinlock.h>
#include <linux/quicklist.h>
#include <linux/kernel.h>
#include <linux/pci.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/irq.h>
#include <asm/io.h>
#include <linux/fs.h>
#include <linux/sched.h>
#include <linux/file.h>
-#include <linux/slab.h>
#include <linux/ptrace.h>
+#include <linux/slab.h>
#include <asm/errno.h>
#include <asm/uaccess.h>
#include <linux/string.h>
#include <linux/kernel.h>
#include <linux/bug.h>
+#include <linux/slab.h>
#include <asm/unwind.h>
*/
#include <linux/init.h>
+#include <linux/gfp.h>
#include <linux/mm.h>
#include <linux/pci.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/types.h>
#include <linux/scatterlist.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/pci.h>
-#include <linux/slab.h>
#include <linux/types.h>
#include <asm/io.h>
#include <linux/personality.h>
#include <linux/ptrace.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/stddef.h>
#include <linux/unistd.h>
#include <linux/kallsyms.h>
*/
#include <linux/compat.h>
-#include <linux/slab.h>
#include <linux/module.h>
#include <linux/unistd.h>
#include <linux/init.h>
*/
#include <linux/types.h>
#include <linux/spinlock.h>
-#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/module.h>
#include <linux/mm.h>
#include <linux/bootmem.h>
+#include <linux/gfp.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/pci.h> /* for hppa_dma_ops and pcxl_dma_ops */
It is recommended that you build a soft-float userspace instead.
-config IOMMU_VMERGE
- bool "Enable IOMMU virtual merging"
- depends on PPC64
- default y
- help
- Cause IO segments sent to a device for DMA to be merged virtually
- by the IOMMU when they happen to have been allocated contiguously.
- This doesn't add pressure to the IOMMU allocator. However, some
- drivers don't support getting large merged segments coming back
- from *_map_sg().
-
- Most drivers don't have this problem; it is safe to say Y here.
-
config IOMMU_HELPER
def_bool PPC64
#define PPC_LLARX(t, a, b, eh) PPC_LDARX(t, a, b, eh)
#define PPC_STLCX stringify_in_c(stdcx.)
#define PPC_CNTLZL stringify_in_c(cntlzd)
+#define PPC_LR_STKOFF 16
/* Move to CR, single-entry optimized version. Only available
* on POWER4 and later.
#define PPC_STLCX stringify_in_c(stwcx.)
#define PPC_CNTLZL stringify_in_c(cntlzw)
#define PPC_MTOCRF stringify_in_c(mtcrf)
+#define PPC_LR_STKOFF 4
#endif
#define PPC_INST_LDARX 0x7c0000a8
#define PPC_INST_LSWI 0x7c0004aa
#define PPC_INST_LSWX 0x7c00042a
-#define PPC_INST_LWARX 0x7c000029
+#define PPC_INST_LWARX 0x7c000028
#define PPC_INST_LWSYNC 0x7c2004ac
#define PPC_INST_LXVD2X 0x7c000698
#define PPC_INST_MCRXR 0x7c000400
#define __PPC_T_TLB(t) (((t) & 0x3) << 21)
#define __PPC_WC(w) (((w) & 0x3) << 21)
/*
- * Only use the larx hint bit on 64bit CPUs. Once we verify it doesn't have
- * any side effects on all 32bit processors, we can do this all the time.
+ * Only use the larx hint bit on 64bit CPUs. e500v1/v2 based CPUs will treat a
+ * larx with EH set as an illegal instruction.
*/
#ifdef CONFIG_PPC64
#define __PPC_EH(eh) (((eh) & 0x1) << 0)
static inline long syscall_get_error(struct task_struct *task,
struct pt_regs *regs)
{
- return (regs->ccr & 0x1000) ? -regs->gpr[3] : 0;
+ return (regs->ccr & 0x10000000) ? -regs->gpr[3] : 0;
}
static inline long syscall_get_return_value(struct task_struct *task,
int error, long val)
{
if (error) {
- regs->ccr |= 0x1000L;
+ regs->ccr |= 0x10000000L;
regs->gpr[3] = -error;
} else {
- regs->ccr &= ~0x1000L;
+ regs->ccr &= ~0x10000000L;
regs->gpr[3] = val;
}
}
#include <linux/notifier.h>
#include <linux/of.h>
#include <linux/percpu.h>
+#include <linux/slab.h>
#include <asm/prom.h>
#include "cacheinfo.h"
#include <linux/device.h>
#include <linux/dma-mapping.h>
#include <linux/dma-debug.h>
+#include <linux/gfp.h>
#include <linux/lmb.h>
#include <asm/bug.h>
#include <asm/abs_addr.h>
rlwimi r12, r11, 32-19, 27, 31 /* extract WIMGE from pte */
#else
rlwimi r12, r11, 26, 27, 31 /* extract WIMGE from pte */
-#endif
-#ifdef CONFIG_SMP
- ori r12, r12, MAS2_M
#endif
mtspr SPRN_MAS2, r12
lwz r3,_MSR(r1)
oris r3,r3,MSR_SPE@h
stw r3,_MSR(r1) /* enable use of SPE after return */
+#ifdef CONFIG_PRINTK
lis r3,87f@h
ori r3,r3,87f@l
mr r4,r2 /* current */
lwz r5,_NIP(r1)
bl printk
+#endif
b ret_from_except
+#ifdef CONFIG_PRINTK
87: .string "SPE used in kernel (task=%p, pc=%x) \n"
+#endif
.align 4,0
#endif /* CONFIG_SPE */
#include <linux/dma-mapping.h>
#include <linux/interrupt.h>
#include <linux/of.h>
+#include <linux/slab.h>
#include <linux/of_platform.h>
#include <asm/ibmebus.h>
#include <asm/abs_addr.h>
#define DBG(...)
-#ifdef CONFIG_IOMMU_VMERGE
-static int novmerge = 0;
-#else
-static int novmerge = 1;
-#endif
-
+static int novmerge;
static int protect4gb = 1;
static void __iommu_free(struct iommu_table *, dma_addr_t, unsigned int);
#include <linux/preempt.h>
#include <linux/module.h>
#include <linux/kdebug.h>
+#include <linux/slab.h>
#include <asm/cacheflush.h>
#include <asm/sstep.h>
#include <asm/uaccess.h>
#include <linux/proc_fs.h>
#include <linux/init.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/iseries/hv_lp_config.h>
#include <asm/lppaca.h>
_GLOBAL(__restore_cpu_power7)
/* place holder */
blr
+
+/*
+ * Get a minimal set of registers for our caller's nth caller.
+ * r3 = regs pointer, r5 = n.
+ *
+ * We only get R1 (stack pointer), NIP (next instruction pointer)
+ * and LR (link register). These are all we can get in the
+ * general case without doing complicated stack unwinding, but
+ * fortunately they are enough to do a stack backtrace, which
+ * is all we need them for.
+ */
+_GLOBAL(perf_arch_fetch_caller_regs)
+ mr r6,r1
+ cmpwi r5,0
+ mflr r4
+ ble 2f
+ mtctr r5
+1: PPC_LL r6,0(r6)
+ bdnz 1b
+ PPC_LL r4,PPC_LR_STKOFF(r6)
+2: PPC_LL r7,0(r6)
+ PPC_LL r7,PPC_LR_STKOFF(r7)
+ PPC_STL r6,GPR1-STACK_FRAME_OVERHEAD(r3)
+ PPC_STL r4,_NIP-STACK_FRAME_OVERHEAD(r3)
+ PPC_STL r7,_LINK-STACK_FRAME_OVERHEAD(r3)
+ blr
#include <linux/init.h>
#include <linux/module.h>
#include <linux/mod_devicetable.h>
-#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/of.h>
#include <linux/of_device.h>
#include <linux/syscalls.h>
#include <linux/irq.h>
#include <linux/vmalloc.h>
+#include <linux/slab.h>
#include <asm/processor.h>
#include <asm/io.h>
#include <linux/irq.h>
#include <linux/list.h>
#include <linux/of.h>
+#include <linux/slab.h>
#include <asm/processor.h>
#include <asm/io.h>
#include <linux/pci.h>
#include <linux/string.h>
#include <linux/init.h>
+#include <linux/gfp.h>
#include <asm/io.h>
#include <asm/prom.h>
#include <linux/init.h>
#include <linux/mm.h>
#include <linux/proc_fs.h>
-#include <linux/slab.h>
#include <linux/kernel.h>
#include <asm/machdep.h>
#include <linux/completion.h>
#include <linux/cpumask.h>
#include <linux/lmb.h>
+#include <linux/slab.h>
#include <asm/prom.h>
#include <asm/rtas.h>
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/proc_fs.h>
#include <asm/delay.h>
#include <asm/uaccess.h>
#include <linux/spinlock.h>
#include <linux/cpu.h>
#include <linux/workqueue.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/io.h>
#include <asm/serial.h>
#include <asm/udbg.h>
#include <asm/mmu_context.h>
-#include <asm/swiotlb.h>
#include "setup.h"
ppc_md.setup_arch();
if ( ppc_md.progress ) ppc_md.progress("arch: exit", 0x3eab);
-#ifdef CONFIG_SWIOTLB
- if (ppc_swiotlb_enable)
- swiotlb_init(1);
-#endif
-
paging_init();
/* Initialize the MMU context management stuff */
#include <asm/xmon.h>
#include <asm/udbg.h>
#include <asm/kexec.h>
-#include <asm/swiotlb.h>
#include <asm/mmu_context.h>
#include "setup.h"
if (ppc_md.setup_arch)
ppc_md.setup_arch();
-#ifdef CONFIG_SWIOTLB
- if (ppc_swiotlb_enable)
- swiotlb_init(1);
-#endif
-
paging_init();
/* Initialize the MMU context management stuff */
#include <linux/smp.h>
#include <linux/unistd.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <asm/atomic.h>
#include <asm/smp.h>
#include <asm/time.h>
#include <linux/stddef.h>
#include <linux/unistd.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/user.h>
#include <linux/interrupt.h>
#include <linux/ptrace.h>
#include <linux/elf.h>
#include <linux/ipc.h>
+#include <linux/slab.h>
#include <asm/ptrace.h>
#include <asm/types.h>
#include <linux/stddef.h>
#include <linux/unistd.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/user.h>
#include <linux/interrupt.h>
#include <linux/init.h>
#include <linux/types.h>
#include <linux/device.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/console.h>
#include <linux/module.h>
#include <linux/mm.h>
*/
#include <linux/kvm_host.h>
+#include <linux/slab.h>
#include <linux/err.h>
#include <asm/reg.h>
#include <asm/kvm_ppc.h>
#include <asm/kvm_book3s.h>
#include <asm/mmu_context.h>
+#include <linux/gfp.h>
#include <linux/sched.h>
#include <linux/vmalloc.h>
#include <linux/errno.h>
#include <linux/err.h>
#include <linux/kvm_host.h>
+#include <linux/gfp.h>
#include <linux/module.h>
#include <linux/vmalloc.h>
#include <linux/fs.h>
*/
#include <linux/kvm_host.h>
+#include <linux/slab.h>
#include <linux/err.h>
#include <asm/reg.h>
*/
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include <linux/kvm.h>
#include <linux/kvm_host.h>
#include <linux/vmalloc.h>
#include <linux/hrtimer.h>
#include <linux/fs.h>
+#include <linux/slab.h>
#include <asm/cputable.h>
#include <asm/uaccess.h>
#include <asm/kvm_ppc.h>
*/
#include <linux/device.h> /* devres_*(), devm_ioremap_release() */
+#include <linux/gfp.h>
#include <linux/io.h> /* ioremap_flags() */
#include <linux/module.h> /* EXPORT_SYMBOL() */
*/
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <linux/hugetlb.h>
#include <asm/pgtable.h>
#include <asm/pgalloc.h>
#include <linux/initrd.h>
#include <linux/pagemap.h>
#include <linux/lmb.h>
+#include <linux/gfp.h>
#include <asm/pgalloc.h>
#include <asm/prom.h>
#include <linux/poison.h>
#include <linux/lmb.h>
#include <linux/hugetlb.h>
+#include <linux/slab.h>
#include <asm/pgalloc.h>
#include <asm/page.h>
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/string.h>
+#include <linux/gfp.h>
#include <linux/types.h>
#include <linux/mm.h>
#include <linux/stddef.h>
#include <asm/sparsemem.h>
#include <asm/vdso.h>
#include <asm/fixmap.h>
+#include <asm/swiotlb.h>
#include "mmu_decl.h"
struct page *page;
unsigned long reservedpages = 0, codesize, initsize, datasize, bsssize;
+#ifdef CONFIG_SWIOTLB
+ if (ppc_swiotlb_enable)
+ swiotlb_init(1);
+#endif
+
num_physpages = lmb.memory.size >> PAGE_SHIFT;
high_memory = (void *) __va(max_low_pfn * PAGE_SIZE);
#include <linux/spinlock.h>
#include <linux/idr.h>
#include <linux/module.h>
+#include <linux/gfp.h>
#include <asm/mmu_context.h>
#include <linux/bootmem.h>
#include <linux/notifier.h>
#include <linux/cpu.h>
+#include <linux/slab.h>
#include <asm/mmu_context.h>
#include <asm/tlbflush.h>
*/
#include <linux/kernel.h>
+#include <linux/gfp.h>
#include <linux/mm.h>
#include <linux/init.h>
#include <linux/percpu.h>
#include <linux/init.h>
#include <linux/highmem.h>
#include <linux/lmb.h>
+#include <linux/slab.h>
#include <asm/pgtable.h>
#include <asm/pgalloc.h>
#include <linux/init.h>
#include <linux/bootmem.h>
#include <linux/lmb.h>
+#include <linux/slab.h>
#include <asm/pgalloc.h>
#include <asm/page.h>
#include <linux/errno.h>
#include <linux/kernel.h>
#include <linux/gfp.h>
-#include <linux/slab.h>
#include <linux/types.h>
#include <linux/mm.h>
#include <linux/hugetlb.h>
#include <linux/notifier.h>
#include <linux/numa.h>
#include <linux/oprofile.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include "pr_util.h"
#include <linux/string.h>
#include <linux/uaccess.h>
#include <linux/elf.h>
+#include <linux/slab.h>
#include "pr_util.h"
#include <linux/delay.h>
#include <linux/of_gpio.h>
#include <linux/of_i2c.h>
+#include <linux/slab.h>
#include <asm/machdep.h>
#include <asm/prom.h>
#include <linux/of.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/of_gpio.h>
#include <linux/io.h>
#include <linux/of_platform.h>
#include <linux/of_platform.h>
#include <linux/of_gpio.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/watchdog.h>
#include <linux/miscdevice.h>
#include <linux/uaccess.h>
if (rc)
goto err_bcom_rx_irq;
+ lpbfifo.dma_irqs_enabled = 1;
+
/* Request the Bestcomm transmit (memory --> fifo) task and IRQ */
lpbfifo.bcom_tx_task =
bcom_gen_bd_tx_init(2, res.start + LPBFIFO_REG_FIFO_DATA,
#include <linux/fsl_devices.h>
#include <linux/mdio-bitbang.h>
#include <linux/of_mdio.h>
+#include <linux/slab.h>
#include <linux/of_platform.h>
#include <asm/io.h>
#include <linux/irq.h>
#include <linux/types.h>
#include <linux/bootmem.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/prom.h>
#include <linux/gpio.h>
#include <linux/of.h>
#include <linux/of_gpio.h>
+#include <linux/slab.h>
#include <asm/prom.h>
#include <asm/machdep.h>
#include <linux/of_platform.h>
#include <linux/of_gpio.h>
#include <linux/gpio.h>
+#include <linux/slab.h>
#define GEF_GPIO_DIRECT 0x00
#define GEF_GPIO_IN 0x04
*/
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/init.h>
#include <linux/time.h>
#include <linux/msi.h>
#include <linux/of_platform.h>
#include <linux/debugfs.h>
+#include <linux/slab.h>
#include <asm/dcr.h>
#include <asm/machdep.h>
#include <linux/pci_regs.h>
#include <linux/of.h>
#include <linux/of_device.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/irq.h>
#include <linux/kernel.h>
#include <linux/pci.h>
#include <linux/string.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/bootmem.h>
#include <linux/delay.h>
#include <linux/notifier.h>
#include <linux/of.h>
#include <linux/of_platform.h>
+#include <linux/slab.h>
#include <linux/lmb.h>
#include <asm/prom.h>
#include <linux/types.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/smp.h>
#include <linux/reboot.h>
#include <linux/kexec.h>
#include <linux/mm.h>
#include <linux/stddef.h>
#include <linux/unistd.h>
-#include <linux/slab.h>
#include <linux/user.h>
#include <linux/reboot.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/of_platform.h>
+#include <linux/slab.h>
#include <linux/io.h>
#include <asm/ppc-pci.h>
#include <linux/list.h>
#include <linux/module.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/wait.h>
#include <linux/mm.h>
#include <linux/io.h>
#include <linux/list.h>
#include <linux/module.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/wait.h>
#include <linux/mm.h>
#include <linux/io.h>
#include <linux/file.h>
#include <linux/fdtable.h>
#include <linux/fs.h>
+#include <linux/gfp.h>
#include <linux/list.h>
#include <linux/module.h>
#include <linux/syscalls.h>
#include <linux/poll.h>
#include <linux/ptrace.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/time.h>
#include <linux/kernel.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <asm/spu.h>
#include <linux/sched.h>
#include <linux/kernel.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include <linux/completion.h>
#include <linux/vmalloc.h>
#include <linux/smp.h>
#include <linux/module.h>
#include <linux/mount.h>
#include <linux/namei.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <linux/kernel.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/spinlock.h>
#include <asm/uaccess.h>
#include <asm/prom.h>
#include <linux/stddef.h>
#include <linux/unistd.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/user.h>
#include <linux/tty.h>
#include <linux/major.h>
#include <linux/list.h>
#include <linux/pci.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <asm/iommu.h>
#include <asm/vio.h>
#include <linux/dma-mapping.h>
#include <linux/bcd.h>
#include <linux/rtc.h>
+#include <linux/slab.h>
#include <asm/time.h>
#include <asm/uaccess.h>
#include <linux/kernel.h>
#include <linux/list.h>
#include <linux/string.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/pci.h>
*/
#include <linux/of.h>
#include <linux/init.h>
-#include <linux/gfp.h>
+#include <linux/slab.h>
#include <linux/completion.h>
#include <linux/proc_fs.h>
#include <linux/module.h>
*/
#include <linux/module.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/vmalloc.h>
#include <linux/string.h>
#include <linux/stddef.h>
#include <linux/unistd.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/user.h>
#include <linux/tty.h>
#include <linux/string.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/of.h>
#include <asm/pasemi_dma.h>
#include <linux/io.h>
#include <linux/module.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#include <linux/errno.h>
#include <linux/ioport.h>
#include <linux/console.h>
#include <linux/pci.h>
#include <linux/of_platform.h>
+#include <linux/gfp.h>
#include <asm/prom.h>
#include <asm/system.h>
#include <linux/sched.h>
#include <linux/adb.h>
#include <linux/pmu.h>
-#include <linux/slab.h>
#include <linux/cpufreq.h>
#include <linux/init.h>
#include <linux/sysdev.h>
#include <linux/kernel.h>
#include <linux/delay.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/cpufreq.h>
#include <linux/init.h>
#include <linux/completion.h>
#include <linux/timer.h>
#include <linux/mutex.h>
#include <linux/i2c.h>
+#include <linux/slab.h>
#include <asm/keylargo.h>
#include <asm/uninorth.h>
#include <asm/io.h>
#include <linux/string.h>
#include <linux/nvram.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/errno.h>
#include <linux/adb.h>
#include <linux/delay.h>
#include <linux/kernel.h>
#include <linux/spinlock.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/mutex.h>
#include <linux/stddef.h>
#include <linux/unistd.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/user.h>
#include <linux/tty.h>
#include <linux/string.h>
#include <linux/kernel.h>
#include <linux/kthread.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/reboot.h>
#include <asm/firmware.h>
#include <linux/module.h>
#include <linux/memory_hotplug.h>
#include <linux/lmb.h>
+#include <linux/slab.h>
#include <asm/cell-regs.h>
#include <asm/firmware.h>
#include <linux/ctype.h>
#include <linux/lmb.h>
#include <linux/of.h>
+#include <linux/slab.h>
#include <asm/prom.h>
#include <linux/kernel.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/mmzone.h>
#include <linux/io.h>
#include <linux/mm.h>
#include <linux/module.h>
#include <linux/dma-mapping.h>
#include <linux/err.h>
+#include <linux/slab.h>
#include <asm/udbg.h>
#include <asm/lv1call.h>
#include <linux/delay.h>
#include <linux/errno.h>
#include <linux/fs.h>
+#include <linux/gfp.h>
#include <linux/init.h>
#include <linux/kthread.h>
#include <linux/module.h>
#include <linux/proc_fs.h>
#include <linux/spinlock.h>
#include <linux/cpu.h>
+#include <linux/slab.h>
#include "offline_states.h"
#include <asm/prom.h>
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/debugfs.h>
#include <asm/smp.h>
#include <asm/system.h>
#include <linux/list.h>
#include <linux/pci.h>
#include <linux/rbtree.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <asm/atomic.h>
#include <asm/pci-bridge.h>
#include <linux/list.h>
#include <linux/mutex.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/workqueue.h>
#include <asm/eeh_event.h>
#include <asm/ppc-pci.h>
#include <linux/types.h>
#include <linux/errno.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/spinlock.h>
#include <asm/uaccess.h>
#include <asm/nvram.h>
*
*/
+#include <linux/gfp.h>
#include <linux/init.h>
#include <linux/kobject.h>
#include <linux/mm.h>
#include <linux/interrupt.h>
#include <linux/timex.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/irq.h>
#include <linux/random.h>
#include <linux/kref.h>
#include <linux/notifier.h>
#include <linux/proc_fs.h>
+#include <linux/slab.h>
#include <asm/prom.h>
#include <asm/machdep.h>
#include <linux/proc_fs.h>
#include <linux/init.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/rtas.h>
#include <asm/prom.h>
#include <linux/mm.h>
#include <linux/stddef.h>
#include <linux/unistd.h>
-#include <linux/slab.h>
#include <linux/user.h>
#include <linux/tty.h>
#include <linux/major.h>
#include <linux/irq.h>
#include <linux/module.h>
#include <linux/spinlock.h>
+#include <linux/slab.h>
#include <asm/page.h>
#include <asm/pgtable.h>
#include <asm/8xx_immap.h>
#include <linux/of_device.h>
#include <linux/spinlock.h>
#include <linux/of.h>
+#include <linux/slab.h>
#include <asm/udbg.h>
#include <asm/io.h>
#include <linux/init.h>
#include <linux/types.h>
-#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/spinlock.h>
#include <linux/string.h>
#include <linux/vmalloc.h>
#include <linux/suspend.h>
#include <linux/lmb.h>
+#include <linux/gfp.h>
#include <asm/io.h>
#include <asm/prom.h>
#include <asm/iommu.h>
#include <linux/of.h>
#include <linux/spinlock.h>
#include <linux/bitops.h>
+#include <linux/slab.h>
#include <asm/fsl_gtm.h>
#define GTCFR_STP(x) ((x) & 1 ? 1 << 5 : 1 << 1)
#include <linux/bootmem.h>
#include <linux/msi.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/of_platform.h>
#include <sysdev/fsl_soc.h>
#include <asm/prom.h>
#include <linux/bootmem.h>
#include <linux/lmb.h>
#include <linux/log2.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/prom.h>
#include <linux/rio_drv.h>
#include <linux/of_platform.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <linux/of.h>
#include <linux/of_gpio.h>
#include <linux/gpio.h>
+#include <linux/slab.h>
#define MPC8XXX_GPIO_PINS 32
#include <linux/bootmem.h>
#include <linux/spinlock.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <asm/ptrace.h>
#include <asm/signal.h>
*
*/
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/bitmap.h>
#include <asm/msi_bitmap.h>
#include <linux/of.h>
#include <linux/init.h>
#include <linux/of_platform.h>
+#include <linux/slab.h>
static __initdata struct {
const char *compatible;
*/
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <linux/completion.h>
#include <linux/spinlock.h>
#include <linux/workqueue.h>
#include <linux/of_gpio.h>
#include <linux/gpio.h>
#include <linux/types.h>
+#include <linux/slab.h>
#define GPIO_MASK(gpio) (0x80000000 >> (gpio))
#define GPIO_MASK2(gpio) (0xc0000000 >> ((gpio) * 2))
#include <linux/of.h>
#include <linux/bootmem.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/pci-bridge.h>
#include <linux/of.h>
#include <linux/of_gpio.h>
#include <linux/gpio.h>
+#include <linux/slab.h>
#include <asm/qe.h>
struct qe_gpio_chip {
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/errno.h>
-#include <linux/slab.h>
#include <linux/stddef.h>
#include <linux/spinlock.h>
#include <linux/module.h>
#include <linux/of.h>
#include <linux/of_gpio.h>
#include <linux/gpio.h>
+#include <linux/slab.h>
#include <asm/prom.h>
#include "simple_gpio.h"
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/pci.h>
-#include <linux/slab.h>
#include <linux/irq.h>
#include <linux/interrupt.h>
#include <linux/module.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/kernel_stat.h>
#include <linux/pagemap.h>
#include <linux/module.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/kernel_stat.h>
#include <linux/netdevice.h>
/* Symbols defined by linker scripts */
extern char input_data[];
extern int input_len;
-extern int _text;
-extern int _end;
+extern char _text, _end;
+extern char _bss, _ebss;
static void error(char *m);
unsigned long output_addr;
unsigned char *output;
+ check_ipl_parmblock((void *) 0, (unsigned long) output + SZ__bss_start);
+ memset(&_bss, 0, &_ebss - &_bss);
free_mem_ptr = (unsigned long)&_end;
free_mem_end_ptr = free_mem_ptr + HEAP_SIZE;
output = (unsigned char *) ((free_mem_end_ptr + 4095UL) & -4096UL);
- check_ipl_parmblock((void *) 0, (unsigned long) output + SZ__bss_start);
-
#ifdef CONFIG_BLK_DEV_INITRD
/*
* Move the initrd right behind the end of the decompressed
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/random.h>
+#include <linux/slab.h>
#include <asm/debug.h>
#include <asm/uaccess.h>
#include <linux/types.h>
#include <linux/errno.h>
-#include <linux/gfp.h>
#include <linux/slab.h>
#include <linux/string.h>
#include <linux/vmalloc.h>
#include <linux/fs.h>
#include <linux/namei.h>
#include <linux/vfs.h>
+#include <linux/slab.h>
#include <linux/pagemap.h>
-#include <linux/gfp.h>
#include <linux/time.h>
#include <linux/parser.h>
#include <linux/sysfs.h>
#endif /* CONFIG_PFAULT */
extern void cmma_init(void);
+extern int memcpy_real(void *, void *, size_t);
#define finish_arch_switch(prev) do { \
set_fs(current->thread.mm_segment); \
" l %0,%2\n"
"0: nr %0,%5\n"
" lr %1,%0\n"
- " or %0,%2\n"
- " or %1,%3\n"
+ " or %0,%3\n"
+ " or %1,%4\n"
" cs %0,%1,%2\n"
" jnl 1f\n"
" xr %1,%0\n"
" l %0,%2\n"
"0: nr %0,%5\n"
" lr %1,%0\n"
- " or %0,%2\n"
- " or %1,%3\n"
+ " or %0,%3\n"
+ " or %1,%4\n"
" cs %0,%1,%2\n"
" jnl 1f\n"
" xr %1,%0\n"
#include <linux/sem.h>
#include <linux/msg.h>
#include <linux/shm.h>
-#include <linux/slab.h>
#include <linux/uio.h>
#include <linux/quota.h>
#include <linux/module.h>
#include <linux/ptrace.h>
#include <linux/fadvise.h>
#include <linux/ipc.h>
+#include <linux/slab.h>
#include <asm/types.h>
#include <asm/uaccess.h>
lhi %r1,2 # mode 2 = esame (dump)
sigp %r1,%r0,0x12 # switch to esame mode
sam64 # switch to 64 bit mode
+ larl %r13,4f
+ lmh %r0,%r15,0(%r13) # clear high-order half
jg startup_continue
+4: .fill 16,4,0x0
#else
mvi __LC_AR_MODE_ID,0 # set ESA flag (mode 0)
l %r13,4f-.LPG0(%r13)
larl %r1,sched_clock_base_cc
mvc 0(8,%r1),__LC_LAST_UPDATE_CLOCK
larl %r13,.LPG1 # get base
- lmh %r0,%r15,.Lzero64-.LPG1(%r13) # clear high-order half
lctlg %c0,%c15,.Lctl-.LPG1(%r13) # load control registers
lg %r12,.Lparmaddr-.LPG1(%r13) # pointer to parameter area
# move IPL device to lowcore
.L4malign:.quad 0xffffffffffc00000
.Lscan2g:.quad 0x80000000 + 0x20000 - 8 # 2GB + 128K - 8
.Lnop: .long 0x07000700
-.Lzero64:.fill 16,4,0x0
.Lparmaddr:
.quad PARMAREA
.align 64
#include <linux/reboot.h>
#include <linux/ctype.h>
#include <linux/fs.h>
+#include <linux/gfp.h>
#include <asm/ipl.h>
#include <asm/smp.h>
#include <asm/setup.h>
#include <asm/cacheflush.h>
#include <asm/sections.h>
#include <linux/module.h>
+#include <linux/slab.h>
DEFINE_PER_CPU(struct kprobe *, current_kprobe) = NULL;
DEFINE_PER_CPU(struct kprobe_ctlblk, kprobe_ctlblk);
#include <linux/fs.h>
#include <linux/smp.h>
#include <linux/stddef.h>
+#include <linux/slab.h>
#include <linux/unistd.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/user.h>
#include <linux/interrupt.h>
#include <linux/stddef.h>
#include <linux/unistd.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/user.h>
#include <linux/tty.h>
#include <linux/ioport.h>
* Setup lowcore for boot cpu
*/
BUILD_BUG_ON(sizeof(struct _lowcore) != LC_PAGES * 4096);
- lc = __alloc_bootmem(LC_PAGES * PAGE_SIZE, LC_PAGES * PAGE_SIZE, 0);
+ lc = __alloc_bootmem_low(LC_PAGES * PAGE_SIZE, LC_PAGES * PAGE_SIZE, 0);
lc->restart_psw.mask = PSW_BASE_BITS | PSW_DEFAULT_KEY;
lc->restart_psw.addr =
PSW_ADDR_AMODE | (unsigned long) restart_int_handler;
#ifndef CONFIG_64BIT
if (MACHINE_HAS_IEEE) {
lc->extended_save_area_addr = (__u32)
- __alloc_bootmem(PAGE_SIZE, PAGE_SIZE, 0);
+ __alloc_bootmem_low(PAGE_SIZE, PAGE_SIZE, 0);
/* enable extended save area */
__ctl_set_bit(14, 29);
}
#include <linux/cpu.h>
#include <linux/timex.h>
#include <linux/bootmem.h>
+#include <linux/slab.h>
#include <asm/asm-offsets.h>
#include <asm/ipl.h>
#include <asm/setup.h>
zfcpdump_save_areas[cpu] = kmalloc(sizeof(struct save_area), GFP_KERNEL);
while (raw_sigp(phy_cpu, sigp_stop_and_store_status) == sigp_busy)
cpu_relax();
- memcpy(zfcpdump_save_areas[cpu],
- (void *)(unsigned long) store_prefix() + SAVE_AREA_BASE,
- sizeof(struct save_area));
+ memcpy_real(zfcpdump_save_areas[cpu],
+ (void *)(unsigned long) store_prefix() + SAVE_AREA_BASE,
+ sizeof(struct save_area));
}
struct save_area *zfcpdump_save_areas[NR_CPUS + 1];
#include <linux/init.h>
#include <linux/delay.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <asm/ebcdic.h>
#include <asm/sysinfo.h>
#include <asm/cpcmd.h>
#include <linux/notifier.h>
#include <linux/clocksource.h>
#include <linux/clockchips.h>
+#include <linux/gfp.h>
#include <asm/uaccess.h>
#include <asm/delay.h>
#include <asm/s390_ext.h>
#include <linux/kvm_host.h>
#include <linux/hrtimer.h>
#include <linux/signal.h>
+#include <linux/slab.h>
#include <asm/asm-offsets.h>
#include <asm/uaccess.h>
#include "kvm-s390.h"
*/
#include <linux/kvm.h>
+#include <linux/gfp.h>
#include <linux/errno.h>
#include <asm/current.h>
#include <asm/debug.h>
#include <linux/kvm.h>
#include <linux/kvm_host.h>
+#include <linux/slab.h>
#include "gaccess.h"
#include "kvm-s390.h"
#include <linux/fs.h>
#include <linux/init.h>
#include <linux/module.h>
+#include <linux/gfp.h>
#include <linux/sched.h>
#include <linux/sysctl.h>
#include <linux/ctype.h>
#include <linux/pfn.h>
#include <linux/poison.h>
#include <linux/initrd.h>
+#include <linux/gfp.h>
#include <asm/processor.h>
#include <asm/system.h>
#include <asm/uaccess.h>
}
return copied < 0 ? -EFAULT : 0;
}
+
+int memcpy_real(void *dest, void *src, size_t count)
+{
+ register unsigned long _dest asm("2") = (unsigned long) dest;
+ register unsigned long _len1 asm("3") = (unsigned long) count;
+ register unsigned long _src asm("4") = (unsigned long) src;
+ register unsigned long _len2 asm("5") = (unsigned long) count;
+ unsigned long flags;
+ int rc = -EFAULT;
+
+ if (!count)
+ return 0;
+ flags = __raw_local_irq_stnsm(0xf8UL);
+ asm volatile (
+ "0: mvcle %1,%2,0x0\n"
+ "1: jo 0b\n"
+ " lhi %0,0x0\n"
+ "2:\n"
+ EX_TABLE(1b,2b)
+ : "+d" (rc), "+d" (_dest), "+d" (_src), "+d" (_len1),
+ "+d" (_len2), "=m" (*((long *) dest))
+ : "m" (*((long *) src))
+ : "cc", "memory");
+ __raw_local_irq_ssm(flags);
+ return rc;
+}
#include <linux/errno.h>
#include <linux/types.h>
#include <linux/mm.h>
+#include <linux/gfp.h>
#include <linux/init.h>
#define ESSA_SET_STABLE 1
#include <linux/sched.h>
#include <linux/kernel.h>
#include <linux/errno.h>
+#include <linux/gfp.h>
#include <linux/mm.h>
#include <linux/swap.h>
#include <linux/smp.h>
#include <linux/highmem.h>
-#include <linux/slab.h>
#include <linux/pagemap.h>
#include <linux/spinlock.h>
#include <linux/module.h>
#include <linux/module.h>
#include <linux/list.h>
#include <linux/hugetlb.h>
+#include <linux/slab.h>
#include <asm/pgalloc.h>
#include <asm/pgtable.h>
#include <asm/setup.h>
#include <linux/mm.h>
#include <linux/mman.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/unistd.h>
#include <linux/syscalls.h>
#include <asm/syscalls.h>
#include <linux/errno.h>
#include <linux/bootmem.h>
#include <linux/kernel.h>
+#include <linux/gfp.h>
#include <linux/init.h>
#include <linux/mm.h>
#include <linux/mman.h>
pd->mac_addr[i] = mac_read(a, 0x10 + i);
msleep(10);
}
+
+ i2c_put_adapter(a);
}
#else
static void __init sh_eth_init(struct sh_eth_plat_data *pd)
* and change SW41 to use 720p
*/
+/*
+ * about sound
+ *
+ * This setup.c supports FSI slave mode.
+ * Please change J20, J21, J22 pin to 1-2 connection.
+ */
+
/* Heartbeat */
static struct resource heartbeat_resource = {
.start = PA_LED,
.rate = 0, /* unknown */
};
+/* change J20, J21, J22 pin to 1-2 connection to use slave mode */
struct sh_fsi_platform_info fsi_info = {
.porta_flags = SH_FSI_BRS_INV |
SH_FSI_OUT_SLAVE_MODE |
#
# Automatically generated make config: don't edit
-# Linux kernel version: 2.6.33-rc2
-# Mon Jan 4 11:20:36 2010
+# Linux kernel version: 2.6.34-rc2
+# Mon Mar 29 02:21:58 2010
#
CONFIG_SUPERH=y
CONFIG_SUPERH32=y
CONFIG_GENERIC_HWEIGHT=y
CONFIG_GENERIC_HARDIRQS=y
CONFIG_GENERIC_HARDIRQS_NO__DO_IRQ=y
-CONFIG_GENERIC_IRQ_PROBE=y
CONFIG_IRQ_PER_CPU=y
+CONFIG_SPARSE_IRQ=y
CONFIG_GENERIC_GPIO=y
CONFIG_GENERIC_TIME=y
CONFIG_GENERIC_CLOCKEVENTS=y
CONFIG_ARCH_HAS_DEFAULT_IDLE=y
CONFIG_ARCH_HAS_CPU_IDLE_WAIT=y
CONFIG_DMA_NONCOHERENT=y
+CONFIG_NEED_DMA_MAP_STATE=y
CONFIG_DEFCONFIG_LIST="/lib/modules/$UNAME_RELEASE/.config"
CONFIG_CONSTRUCTORS=y
CONFIG_HAVE_KERNEL_GZIP=y
CONFIG_HAVE_KERNEL_BZIP2=y
CONFIG_HAVE_KERNEL_LZMA=y
+CONFIG_HAVE_KERNEL_LZO=y
CONFIG_KERNEL_GZIP=y
# CONFIG_KERNEL_BZIP2 is not set
# CONFIG_KERNEL_LZMA is not set
+# CONFIG_KERNEL_LZO is not set
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
CONFIG_SYSVIPC_SYSCTL=y
# CONFIG_TREE_RCU_TRACE is not set
# CONFIG_IKCONFIG is not set
CONFIG_LOG_BUF_SHIFT=14
-CONFIG_GROUP_SCHED=y
-CONFIG_FAIR_GROUP_SCHED=y
-# CONFIG_RT_GROUP_SCHED is not set
-CONFIG_USER_SCHED=y
-# CONFIG_CGROUP_SCHED is not set
# CONFIG_CGROUPS is not set
-CONFIG_SYSFS_DEPRECATED=y
-CONFIG_SYSFS_DEPRECATED_V2=y
+# CONFIG_SYSFS_DEPRECATED_V2 is not set
# CONFIG_RELAY is not set
# CONFIG_NAMESPACES is not set
# CONFIG_BLK_DEV_INITRD is not set
#
# Kernel Performance Events And Counters
#
-# CONFIG_PERF_EVENTS is not set
+CONFIG_PERF_EVENTS=y
# CONFIG_PERF_COUNTERS is not set
CONFIG_VM_EVENT_COUNTERS=y
CONFIG_COMPAT_BRK=y
# CONFIG_SLOB is not set
# CONFIG_PROFILING is not set
CONFIG_HAVE_OPROFILE=y
-CONFIG_HAVE_IOREMAP_PROT=y
CONFIG_HAVE_KPROBES=y
CONFIG_HAVE_KRETPROBES=y
CONFIG_HAVE_ARCH_TRACEHOOK=y
CONFIG_HAVE_DMA_ATTRS=y
CONFIG_HAVE_CLK=y
CONFIG_HAVE_DMA_API_DEBUG=y
+CONFIG_HAVE_HW_BREAKPOINT=y
#
# GCOV-based kernel profiling
CONFIG_QUICKLIST=y
CONFIG_MMU=y
CONFIG_PAGE_OFFSET=0x80000000
-CONFIG_FORCE_MAX_ZONEORDER=11
+CONFIG_FORCE_MAX_ZONEORDER=12
CONFIG_MEMORY_START=0x08000000
CONFIG_MEMORY_SIZE=0x10000000
CONFIG_29BIT=y
-# CONFIG_PMB_ENABLE is not set
-# CONFIG_X2TLB is not set
+# CONFIG_PMB is not set
+CONFIG_X2TLB=y
CONFIG_VSYSCALL=y
CONFIG_ARCH_FLATMEM_ENABLE=y
CONFIG_ARCH_SPARSEMEM_ENABLE=y
CONFIG_MAX_ACTIVE_REGIONS=1
CONFIG_ARCH_POPULATES_NODE_MAP=y
CONFIG_ARCH_SELECT_MEMORY_MODEL=y
+CONFIG_IOREMAP_FIXED=y
+CONFIG_UNCACHED_MAPPING=y
CONFIG_PAGE_SIZE_4KB=y
# CONFIG_PAGE_SIZE_8KB is not set
# CONFIG_PAGE_SIZE_16KB is not set
CONFIG_SPLIT_PTLOCK_CPUS=4
# CONFIG_PHYS_ADDR_T_64BIT is not set
CONFIG_ZONE_DMA_FLAG=0
-CONFIG_NR_QUICK=2
+CONFIG_NR_QUICK=1
# CONFIG_KSM is not set
CONFIG_DEFAULT_MMAP_MIN_ADDR=4096
# CONFIG_PREEMPT_VOLUNTARY is not set
CONFIG_PREEMPT=y
CONFIG_GUSA=y
-# CONFIG_SPARSE_IRQ is not set
#
# Boot options
CONFIG_ENTRY_OFFSET=0x00001000
CONFIG_CMDLINE_OVERWRITE=y
# CONFIG_CMDLINE_EXTEND is not set
-CONFIG_CMDLINE="console=tty0, console=ttySC0,115200 root=/dev/nfs ip=dhcp mem=120M memchunk.vpu=4m"
+CONFIG_CMDLINE="console=tty0, console=ttySC0,115200 root=/dev/nfs ip=dhcp mem=248M memchunk.vpu=8m memchunk.veu0=4m"
#
# Bus options
CONFIG_SUSPEND_FREEZER=y
# CONFIG_HIBERNATION is not set
CONFIG_PM_RUNTIME=y
+CONFIG_PM_OPS=y
# CONFIG_CPU_IDLE is not set
CONFIG_NET=y
# Networking options
#
CONFIG_PACKET=y
-# CONFIG_PACKET_MMAP is not set
CONFIG_UNIX=y
# CONFIG_NET_KEY is not set
CONFIG_INET=y
# CONFIG_NET_PKTGEN is not set
# CONFIG_HAMRADIO is not set
# CONFIG_CAN is not set
-# CONFIG_IRDA is not set
+CONFIG_IRDA=y
+
+#
+# IrDA protocols
+#
+# CONFIG_IRLAN is not set
+# CONFIG_IRCOMM is not set
+# CONFIG_IRDA_ULTRA is not set
+
+#
+# IrDA options
+#
+# CONFIG_IRDA_CACHE_LAST_LSAP is not set
+# CONFIG_IRDA_FAST_RR is not set
+# CONFIG_IRDA_DEBUG is not set
+
+#
+# Infrared-port device drivers
+#
+
+#
+# SIR device drivers
+#
+# CONFIG_IRTTY_SIR is not set
+
+#
+# Dongle support
+#
+CONFIG_SH_SIR=y
+# CONFIG_KINGSUN_DONGLE is not set
+# CONFIG_KSDAZZLE_DONGLE is not set
+# CONFIG_KS959_DONGLE is not set
+
+#
+# FIR device drivers
+#
+# CONFIG_USB_IRDA is not set
+# CONFIG_SIGMATEL_FIR is not set
+# CONFIG_MCS_FIR is not set
# CONFIG_BT is not set
# CONFIG_AF_RXRPC is not set
CONFIG_WIRELESS=y
# CONFIG_MTD_NAND_NANDSIM is not set
# CONFIG_MTD_NAND_PLATFORM is not set
# CONFIG_MTD_ALAUDA is not set
+# CONFIG_MTD_NAND_SH_FLCTL is not set
# CONFIG_MTD_ONENAND is not set
#
# CONFIG_ICS932S401 is not set
# CONFIG_ENCLOSURE_SERVICES is not set
# CONFIG_ISL29003 is not set
+# CONFIG_SENSORS_TSL2550 is not set
# CONFIG_DS1682 is not set
# CONFIG_TI_DAC7512 is not set
# CONFIG_C2PORT is not set
#
# SCSI device support
#
+CONFIG_SCSI_MOD=y
# CONFIG_RAID_ATTRS is not set
CONFIG_SCSI=y
CONFIG_SCSI_DMA=y
# CONFIG_INPUT_MOUSE is not set
# CONFIG_INPUT_JOYSTICK is not set
# CONFIG_INPUT_TABLET is not set
-# CONFIG_INPUT_TOUCHSCREEN is not set
+CONFIG_INPUT_TOUCHSCREEN=y
+# CONFIG_TOUCHSCREEN_ADS7846 is not set
+# CONFIG_TOUCHSCREEN_AD7877 is not set
+# CONFIG_TOUCHSCREEN_AD7879_I2C is not set
+# CONFIG_TOUCHSCREEN_AD7879_SPI is not set
+# CONFIG_TOUCHSCREEN_AD7879 is not set
+# CONFIG_TOUCHSCREEN_DYNAPRO is not set
+# CONFIG_TOUCHSCREEN_EETI is not set
+# CONFIG_TOUCHSCREEN_FUJITSU is not set
+# CONFIG_TOUCHSCREEN_GUNZE is not set
+# CONFIG_TOUCHSCREEN_ELO is not set
+# CONFIG_TOUCHSCREEN_WACOM_W8001 is not set
+# CONFIG_TOUCHSCREEN_MCS5000 is not set
+# CONFIG_TOUCHSCREEN_MTOUCH is not set
+# CONFIG_TOUCHSCREEN_INEXIO is not set
+# CONFIG_TOUCHSCREEN_MK712 is not set
+# CONFIG_TOUCHSCREEN_PENMOUNT is not set
+# CONFIG_TOUCHSCREEN_TOUCHRIGHT is not set
+# CONFIG_TOUCHSCREEN_TOUCHWIN is not set
+# CONFIG_TOUCHSCREEN_USB_COMPOSITE is not set
+# CONFIG_TOUCHSCREEN_TOUCHIT213 is not set
+CONFIG_TOUCHSCREEN_TSC2007=y
+# CONFIG_TOUCHSCREEN_W90X900 is not set
# CONFIG_INPUT_MISC is not set
#
CONFIG_SERIAL_SH_SCI_CONSOLE=y
CONFIG_SERIAL_CORE=y
CONFIG_SERIAL_CORE_CONSOLE=y
+# CONFIG_SERIAL_TIMBERDALE is not set
CONFIG_UNIX98_PTYS=y
# CONFIG_DEVPTS_MULTIPLE_INSTANCES is not set
-CONFIG_LEGACY_PTYS=y
-CONFIG_LEGACY_PTY_COUNT=256
+# CONFIG_LEGACY_PTYS is not set
# CONFIG_IPMI_HANDLER is not set
CONFIG_HW_RANDOM=y
# CONFIG_HW_RANDOM_TIMERIOMEM is not set
# CONFIG_I2C_OCORES is not set
CONFIG_I2C_SH_MOBILE=y
# CONFIG_I2C_SIMTEC is not set
+# CONFIG_I2C_XILINX is not set
#
# External I2C/SMBus adapter drivers
#
# CONFIG_I2C_PCA_PLATFORM is not set
# CONFIG_I2C_STUB is not set
-
-#
-# Miscellaneous I2C Chip support
-#
-# CONFIG_SENSORS_TSL2550 is not set
# CONFIG_I2C_DEBUG_CORE is not set
# CONFIG_I2C_DEBUG_ALGO is not set
# CONFIG_I2C_DEBUG_BUS is not set
-# CONFIG_I2C_DEBUG_CHIP is not set
CONFIG_SPI=y
CONFIG_SPI_MASTER=y
#
# Memory mapped GPIO expanders:
#
+# CONFIG_GPIO_IT8761E is not set
#
# I2C GPIO expanders:
#
+# CONFIG_GPIO_MAX7300 is not set
# CONFIG_GPIO_MAX732X is not set
# CONFIG_GPIO_PCA953X is not set
# CONFIG_GPIO_PCF857X is not set
+# CONFIG_GPIO_ADP5588 is not set
#
# PCI GPIO expanders:
#
# Multifunction device drivers
#
-# CONFIG_MFD_CORE is not set
+CONFIG_MFD_CORE=y
+# CONFIG_MFD_88PM860X is not set
# CONFIG_MFD_SM501 is not set
-# CONFIG_MFD_SH_MOBILE_SDHI is not set
+CONFIG_MFD_SH_MOBILE_SDHI=y
# CONFIG_HTC_PASIC3 is not set
+# CONFIG_HTC_I2CPLD is not set
# CONFIG_TPS65010 is not set
# CONFIG_TWL4030_CORE is not set
# CONFIG_MFD_TMIO is not set
# CONFIG_PMIC_DA903X is not set
# CONFIG_PMIC_ADP5520 is not set
+# CONFIG_MFD_MAX8925 is not set
# CONFIG_MFD_WM8400 is not set
# CONFIG_MFD_WM831X is not set
# CONFIG_MFD_WM8350_I2C is not set
+# CONFIG_MFD_WM8994 is not set
# CONFIG_MFD_PCF50633 is not set
# CONFIG_MFD_MC13783 is not set
# CONFIG_AB3100_CORE is not set
# CONFIG_EZX_PCAP is not set
-# CONFIG_MFD_88PM8607 is not set
# CONFIG_AB4500_CORE is not set
# CONFIG_REGULATOR is not set
CONFIG_MEDIA_SUPPORT=y
# CONFIG_SOC_CAMERA_MT9M001 is not set
# CONFIG_SOC_CAMERA_MT9M111 is not set
# CONFIG_SOC_CAMERA_MT9T031 is not set
-# CONFIG_SOC_CAMERA_MT9T112 is not set
+CONFIG_SOC_CAMERA_MT9T112=y
# CONFIG_SOC_CAMERA_MT9V022 is not set
# CONFIG_SOC_CAMERA_RJ54N1 is not set
-# CONFIG_SOC_CAMERA_TW9910 is not set
+CONFIG_SOC_CAMERA_TW9910=y
# CONFIG_SOC_CAMERA_PLATFORM is not set
# CONFIG_SOC_CAMERA_OV772X is not set
# CONFIG_SOC_CAMERA_OV9640 is not set
# CONFIG_RADIO_SI470X is not set
# CONFIG_USB_MR800 is not set
# CONFIG_RADIO_TEA5764 is not set
+# CONFIG_RADIO_SAA7706H is not set
# CONFIG_RADIO_TEF6862 is not set
# CONFIG_DAB is not set
#
# CONFIG_FB_S1D13XXX is not set
CONFIG_FB_SH_MOBILE_LCDC=y
+# CONFIG_FB_TMIO is not set
# CONFIG_FB_VIRTUAL is not set
# CONFIG_FB_METRONOME is not set
# CONFIG_FB_MB862XX is not set
# CONFIG_LOGO_SUPERH_MONO is not set
# CONFIG_LOGO_SUPERH_VGA16 is not set
CONFIG_LOGO_SUPERH_CLUT224=y
-# CONFIG_SOUND is not set
+CONFIG_SOUND=y
+CONFIG_SOUND_OSS_CORE=y
+CONFIG_SOUND_OSS_CORE_PRECLAIM=y
+CONFIG_SND=y
+CONFIG_SND_TIMER=y
+CONFIG_SND_PCM=y
+CONFIG_SND_JACK=y
+CONFIG_SND_SEQUENCER=y
+CONFIG_SND_SEQ_DUMMY=y
+CONFIG_SND_OSSEMUL=y
+CONFIG_SND_MIXER_OSS=y
+CONFIG_SND_PCM_OSS=y
+CONFIG_SND_PCM_OSS_PLUGINS=y
+# CONFIG_SND_SEQUENCER_OSS is not set
+# CONFIG_SND_DYNAMIC_MINORS is not set
+CONFIG_SND_SUPPORT_OLD_API=y
+CONFIG_SND_VERBOSE_PROCFS=y
+# CONFIG_SND_VERBOSE_PRINTK is not set
+# CONFIG_SND_DEBUG is not set
+# CONFIG_SND_RAWMIDI_SEQ is not set
+# CONFIG_SND_OPL3_LIB_SEQ is not set
+# CONFIG_SND_OPL4_LIB_SEQ is not set
+# CONFIG_SND_SBAWE_SEQ is not set
+# CONFIG_SND_EMU10K1_SEQ is not set
+# CONFIG_SND_DRIVERS is not set
+# CONFIG_SND_SPI is not set
+CONFIG_SND_SUPERH=y
+# CONFIG_SND_USB is not set
+CONFIG_SND_SOC=y
+
+#
+# SoC Audio support for SuperH
+#
+CONFIG_SND_SOC_SH4_FSI=y
+# CONFIG_SND_FSI_AK4642 is not set
+CONFIG_SND_FSI_DA7210=y
+CONFIG_SND_SOC_I2C_AND_SPI=y
+# CONFIG_SND_SOC_ALL_CODECS is not set
+CONFIG_SND_SOC_DA7210=y
+# CONFIG_SOUND_PRIME is not set
CONFIG_HID_SUPPORT=y
CONFIG_HID=y
# CONFIG_HIDRAW is not set
#
# Special HID drivers
#
+# CONFIG_HID_3M_PCT is not set
# CONFIG_HID_A4TECH is not set
# CONFIG_HID_APPLE is not set
# CONFIG_HID_BELKIN is not set
# CONFIG_HID_KENSINGTON is not set
# CONFIG_HID_LOGITECH is not set
# CONFIG_HID_MICROSOFT is not set
+# CONFIG_HID_MOSART is not set
# CONFIG_HID_MONTEREY is not set
# CONFIG_HID_NTRIG is not set
+# CONFIG_HID_ORTEK is not set
# CONFIG_HID_PANTHERLORD is not set
# CONFIG_HID_PETALYNX is not set
+# CONFIG_HID_QUANTA is not set
# CONFIG_HID_SAMSUNG is not set
# CONFIG_HID_SONY is not set
+# CONFIG_HID_STANTUM is not set
# CONFIG_HID_SUNPLUS is not set
# CONFIG_HID_GREENASIA is not set
# CONFIG_HID_SMARTJOYPLUS is not set
# CONFIG_USB_SL811_HCD is not set
CONFIG_USB_R8A66597_HCD=y
# CONFIG_USB_HWA_HCD is not set
+# CONFIG_USB_GADGET_MUSB_HDRC is not set
#
# USB Device Class drivers
# CONFIG_USB_RIO500 is not set
# CONFIG_USB_LEGOTOWER is not set
# CONFIG_USB_LCD is not set
-# CONFIG_USB_BERRY_CHARGE is not set
# CONFIG_USB_LED is not set
# CONFIG_USB_CYPRESS_CY7C63 is not set
# CONFIG_USB_CYTHERM is not set
# CONFIG_USB_IOWARRIOR is not set
# CONFIG_USB_TEST is not set
# CONFIG_USB_ISIGHTFW is not set
-# CONFIG_USB_VST is not set
-# CONFIG_USB_GADGET is not set
+CONFIG_USB_GADGET=y
+# CONFIG_USB_GADGET_DEBUG_FILES is not set
+# CONFIG_USB_GADGET_DEBUG_FS is not set
+CONFIG_USB_GADGET_VBUS_DRAW=2
+CONFIG_USB_GADGET_SELECTED=y
+# CONFIG_USB_GADGET_AT91 is not set
+# CONFIG_USB_GADGET_ATMEL_USBA is not set
+# CONFIG_USB_GADGET_FSL_USB2 is not set
+# CONFIG_USB_GADGET_LH7A40X is not set
+# CONFIG_USB_GADGET_OMAP is not set
+# CONFIG_USB_GADGET_PXA25X is not set
+CONFIG_USB_GADGET_R8A66597=y
+CONFIG_USB_R8A66597=y
+# CONFIG_USB_GADGET_PXA27X is not set
+# CONFIG_USB_GADGET_S3C_HSOTG is not set
+# CONFIG_USB_GADGET_IMX is not set
+# CONFIG_USB_GADGET_S3C2410 is not set
+# CONFIG_USB_GADGET_M66592 is not set
+# CONFIG_USB_GADGET_AMD5536UDC is not set
+# CONFIG_USB_GADGET_FSL_QE is not set
+# CONFIG_USB_GADGET_CI13XXX is not set
+# CONFIG_USB_GADGET_NET2280 is not set
+# CONFIG_USB_GADGET_GOKU is not set
+# CONFIG_USB_GADGET_LANGWELL is not set
+# CONFIG_USB_GADGET_DUMMY_HCD is not set
+CONFIG_USB_GADGET_DUALSPEED=y
+# CONFIG_USB_ZERO is not set
+# CONFIG_USB_AUDIO is not set
+# CONFIG_USB_ETH is not set
+# CONFIG_USB_GADGETFS is not set
+CONFIG_USB_FILE_STORAGE=m
+# CONFIG_USB_FILE_STORAGE_TEST is not set
+# CONFIG_USB_MASS_STORAGE is not set
+# CONFIG_USB_G_SERIAL is not set
+# CONFIG_USB_MIDI_GADGET is not set
+# CONFIG_USB_G_PRINTER is not set
+# CONFIG_USB_CDC_COMPOSITE is not set
+# CONFIG_USB_G_NOKIA is not set
+# CONFIG_USB_G_MULTI is not set
#
# OTG and related infrastructure
# MMC/SD/SDIO Host Controller Drivers
#
# CONFIG_MMC_SDHCI is not set
-# CONFIG_MMC_AT91 is not set
-# CONFIG_MMC_ATMELMCI is not set
CONFIG_MMC_SPI=y
-# CONFIG_MMC_TMIO is not set
+CONFIG_MMC_TMIO=y
# CONFIG_MEMSTICK is not set
# CONFIG_NEW_LEDS is not set
# CONFIG_ACCESSIBILITY is not set
# CONFIG_RTC_DRV_DS1374 is not set
# CONFIG_RTC_DRV_DS1672 is not set
# CONFIG_RTC_DRV_MAX6900 is not set
-# CONFIG_RTC_DRV_RS5C372 is not set
+CONFIG_RTC_DRV_RS5C372=y
# CONFIG_RTC_DRV_ISL1208 is not set
# CONFIG_RTC_DRV_X1205 is not set
-CONFIG_RTC_DRV_PCF8563=y
+# CONFIG_RTC_DRV_PCF8563 is not set
# CONFIG_RTC_DRV_PCF8583 is not set
# CONFIG_RTC_DRV_M41T80 is not set
# CONFIG_RTC_DRV_BQ32K is not set
CONFIG_UIO=y
# CONFIG_UIO_PDRV is not set
CONFIG_UIO_PDRV_GENIRQ=y
-# CONFIG_UIO_SMX is not set
-# CONFIG_UIO_SERCOS3 is not set
#
# TI VLYNQ
# CONFIG_EFS_FS is not set
# CONFIG_JFFS2_FS is not set
# CONFIG_UBIFS_FS is not set
+# CONFIG_LOGFS is not set
# CONFIG_CRAMFS is not set
# CONFIG_SQUASHFS is not set
# CONFIG_VXFS_FS is not set
# CONFIG_RPCSEC_GSS_KRB5 is not set
# CONFIG_RPCSEC_GSS_SPKM3 is not set
# CONFIG_SMB_FS is not set
+# CONFIG_CEPH_FS is not set
# CONFIG_CIFS is not set
# CONFIG_NCP_FS is not set
# CONFIG_CODA_FS is not set
CONFIG_DEBUG_BUGVERBOSE=y
# CONFIG_DEBUG_MEMORY_INIT is not set
# CONFIG_RCU_CPU_STALL_DETECTOR is not set
+# CONFIG_LKDTM is not set
# CONFIG_LATENCYTOP is not set
CONFIG_SYSCTL_SYSCALL_CHECK=y
CONFIG_HAVE_FUNCTION_TRACER=y
#
CONFIG_BITREVERSE=y
CONFIG_GENERIC_FIND_LAST_BIT=y
-# CONFIG_CRC_CCITT is not set
+CONFIG_CRC_CCITT=y
# CONFIG_CRC16 is not set
CONFIG_CRC_T10DIF=y
CONFIG_CRC_ITU_T=y
#include <linux/platform_device.h>
#include <linux/mm.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <asm/dma.h>
DEFINE_SPINLOCK(dma_spin_lock);
#include <linux/interrupt.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <asm/dma.h>
#include <asm/dmabrg.h>
#include <asm/io.h>
#include <linux/sched.h>
#include <linux/timer.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <asm/heartbeat.h>
#define DRV_NAME "heartbeat"
#include <linux/kernel.h>
#include <linux/io.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include "pcie-sh7786.h"
#include <asm/sizes.h>
* for more details.
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/interrupt.h>
#include <linux/platform_device.h>
#define VSYSCALL_AUX_ENT \
if (vdso_enabled) \
- NEW_AUX_ENT(AT_SYSINFO_EHDR, VDSO_BASE);
+ NEW_AUX_ENT(AT_SYSINFO_EHDR, VDSO_BASE); \
+ else \
+ NEW_AUX_ENT(AT_IGNORE, 0);
#else
#define VSYSCALL_AUX_ENT
#endif /* CONFIG_VSYSCALL */
#ifdef CONFIG_SH_FPU
#define FPU_AUX_ENT NEW_AUX_ENT(AT_FPUCW, FPSCR_INIT)
#else
-#define FPU_AUX_ENT
+#define FPU_AUX_ENT NEW_AUX_ENT(AT_IGNORE, 0)
#endif
extern int l1i_cache_shape, l1d_cache_shape, l2_cache_shape;
#else
+static inline int
+pmb_bolt_mapping(unsigned long virt, phys_addr_t phys,
+ unsigned long size, pgprot_t prot)
+{
+ return -EINVAL;
+}
+
static inline void __iomem *
pmb_remap_caller(phys_addr_t phys, unsigned long size,
pgprot_t prot, void *caller)
#define MMUCR 0xFF000010 /* MMU Control Register */
+#define MMU_ITLB_ADDRESS_ARRAY 0xF2000000
+#define MMU_ITLB_ADDRESS_ARRAY2 0xF2800000
#define MMU_UTLB_ADDRESS_ARRAY 0xF6000000
#define MMU_UTLB_ADDRESS_ARRAY2 0xF6800000
#define MMU_PAGE_ASSOC_BIT 0x80
#define MMUCR_URB 0x00FC0000
#define MMUCR_URB_SHIFT 18
#define MMUCR_URB_NENTRIES 64
+#define MMUCR_URC 0x0000FC00
+#define MMUCR_URC_SHIFT 10
#if defined(CONFIG_32BIT) && defined(CONFIG_CPU_SUBTYPE_ST40)
#define MMUCR_SE (1 << 4)
#define WTCNT 0xffcc0000 /*WDTST*/
#define WTST WTCNT
#define WTBST 0xffcc0008 /*WDTBST*/
+/* Register definitions */
+#elif defined(CONFIG_CPU_SUBTYPE_SH7722) || \
+ defined(CONFIG_CPU_SUBTYPE_SH7723) || \
+ defined(CONFIG_CPU_SUBTYPE_SH7724)
+#define WTCNT 0xa4520000
+#define WTCSR 0xa4520004
#else
/* Register definitions */
#define WTCNT 0xffc00008
#include <linux/sched.h>
+#include <linux/slab.h>
#include <asm/processor.h>
#include <asm/fpu.h>
#include <linux/clk.h>
#include <linux/compiler.h>
-#include <linux/slab.h>
#include <linux/io.h>
#include <linux/spinlock.h>
#include <asm/suspend.h>
return -ENODEV;
cpus_allowed = current->cpus_allowed;
- set_cpus_allowed(current, cpumask_of_cpu(cpu));
+ set_cpus_allowed_ptr(current, cpumask_of(cpu));
BUG_ON(smp_processor_id() != cpu);
freqs.flags = 0;
cpufreq_notify_transition(&freqs, CPUFREQ_PRECHANGE);
- set_cpus_allowed(current, cpus_allowed);
+ set_cpus_allowed_ptr(current, &cpus_allowed);
clk_set_rate(cpuclk, freq);
cpufreq_notify_transition(&freqs, CPUFREQ_POSTCHANGE);
#include <linux/mm.h>
#include <linux/elf.h>
#include <linux/ftrace.h>
+#include <linux/slab.h>
#include <asm/dwarf.h>
#include <asm/unwinder.h>
#include <asm/sections.h>
unsigned char *end, struct module *mod)
{
struct rb_node **rb_node = &cie_root.rb_node;
- struct rb_node *parent;
+ struct rb_node *parent = *rb_node;
struct dwarf_cie *cie;
unsigned long flags;
int count;
unsigned char *end, struct module *mod)
{
struct rb_node **rb_node = &fde_root.rb_node;
- struct rb_node *parent;
+ struct rb_node *parent = *rb_node;
struct dwarf_fde *fde;
struct dwarf_cie *cie;
unsigned long flags;
}
}
-void __cpuinit select_idle_routine(void)
+void __init select_idle_routine(void)
{
/*
* If a platform has set its own idle routine, leave it alone.
#include <linux/ptrace.h>
#include <linux/preempt.h>
#include <linux/kdebug.h>
+#include <linux/slab.h>
#include <asm/cacheflush.h>
#include <asm/uaccess.h>
sh_pmu->disable_all();
}
-int register_sh_pmu(struct sh_pmu *pmu)
+int __cpuinit register_sh_pmu(struct sh_pmu *pmu)
{
if (sh_pmu)
return -EBUSY;
#include <linux/mm.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/sched.h>
struct kmem_cache *task_xstate_cachep = NULL;
*/
#include <linux/module.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include <linux/elfcore.h>
#include <linux/kallsyms.h>
#include <linux/fs.h>
#include <linux/fs.h>
#include <linux/ptrace.h>
#include <linux/reboot.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/io.h>
return error;
}
-/*
- * These bracket the sleeping functions..
- */
-extern void interruptible_sleep_on(wait_queue_head_t *q);
-
-#define mid_sched ((unsigned long) interruptible_sleep_on)
-
#ifdef CONFIG_FRAME_POINTER
static int in_sh64_switch_to(unsigned long pc)
{
#include <linux/errno.h>
#include <linux/ptrace.h>
#include <linux/user.h>
-#include <linux/slab.h>
#include <linux/security.h>
#include <linux/signal.h>
#include <linux/io.h>
* for more details.
*/
#include <linux/kernel.h>
+#include <linux/module.h>
#include <asm/dwarf.h>
#ifdef CONFIG_DWARF_UNWINDER
}
#endif
+
+EXPORT_SYMBOL_GPL(return_address);
unsigned int cpu;
struct mm_struct *mm = &init_mm;
+ enable_mmu();
atomic_inc(&mm->mm_count);
atomic_inc(&mm->mm_users);
current->active_mm = mm;
* for more details.
*/
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/gfp.h>
#include <linux/dma-debug.h>
#include <linux/io.h>
#include <linux/module.h>
+#include <linux/gfp.h>
#include <asm/cacheflush.h>
#include <asm/addrspace.h>
#include <linux/mm.h>
#include <linux/hugetlb.h>
#include <linux/pagemap.h>
-#include <linux/slab.h>
#include <linux/sysctl.h>
#include <asm/mman.h>
#include <linux/mm.h>
#include <linux/swap.h>
#include <linux/init.h>
+#include <linux/gfp.h>
#include <linux/bootmem.h>
#include <linux/proc_fs.h>
#include <linux/pagemap.h>
*/
#include <linux/vmalloc.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/pci.h>
#include <linux/io.h>
#include <linux/io.h>
#include <linux/bootmem.h>
#include <linux/proc_fs.h>
-#include <linux/slab.h>
#include <asm/fixmap.h>
#include <asm/page.h>
#include <asm/pgalloc.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#define PGALLOC_GFP GFP_KERNEL | __GFP_REPEAT | __GFP_ZERO
#include <linux/sysdev.h>
#include <linux/cpu.h>
#include <linux/module.h>
-#include <linux/slab.h>
#include <linux/bitops.h>
#include <linux/debugfs.h>
#include <linux/fs.h>
writel_uncached(data_val & ~PMB_V, data);
}
+#ifdef CONFIG_PM
static void set_pmb_entry(struct pmb_entry *pmbe)
{
unsigned long flags;
__set_pmb_entry(pmbe);
spin_unlock_irqrestore(&pmbe->lock, flags);
}
+#endif /* CONFIG_PM */
int pmb_bolt_mapping(unsigned long vaddr, phys_addr_t phys,
unsigned long size, pgprot_t prot)
writel_uncached(0, PMB_IRMCR);
/* Flush out the TLB */
- __raw_writel(__raw_readl(MMUCR) | MMUCR_TI, MMUCR);
+ local_flush_tlb_all();
ctrl_barrier();
}
jump_to_uncached();
__raw_writel(page, MMU_UTLB_ADDRESS_ARRAY | MMU_PAGE_ASSOC_BIT);
__raw_writel(asid, MMU_UTLB_ADDRESS_ARRAY2 | MMU_PAGE_ASSOC_BIT);
+ __raw_writel(page, MMU_ITLB_ADDRESS_ARRAY | MMU_PAGE_ASSOC_BIT);
+ __raw_writel(asid, MMU_ITLB_ADDRESS_ARRAY2 | MMU_PAGE_ASSOC_BIT);
back_to_cached();
}
+
+void local_flush_tlb_all(void)
+{
+ unsigned long flags, status;
+ int i;
+
+ /*
+ * Flush all the TLB.
+ */
+ local_irq_save(flags);
+ jump_to_uncached();
+
+ status = __raw_readl(MMUCR);
+ status = ((status & MMUCR_URB) >> MMUCR_URB_SHIFT);
+
+ if (status == 0)
+ status = MMUCR_URB_NENTRIES;
+
+ for (i = 0; i < status; i++)
+ __raw_writel(0x0, MMU_UTLB_ADDRESS_ARRAY | (i << 8));
+
+ for (i = 0; i < 4; i++)
+ __raw_writel(0x0, MMU_ITLB_ADDRESS_ARRAY | (i << 8));
+
+ back_to_cached();
+ ctrl_barrier();
+ local_irq_restore(flags);
+}
for (i = 0; i < ways; i++)
__raw_writel(data, addr + (i << 8));
}
+
+void local_flush_tlb_all(void)
+{
+ unsigned long flags, status;
+
+ /*
+ * Flush all the TLB.
+ *
+ * Write to the MMU control register's bit:
+ * TF-bit for SH-3, TI-bit for SH-4.
+ * It's same position, bit #2.
+ */
+ local_irq_save(flags);
+ status = __raw_readl(MMUCR);
+ status |= 0x04;
+ __raw_writel(status, MMUCR);
+ ctrl_barrier();
+ local_irq_restore(flags);
+}
__raw_writel(data, addr);
back_to_cached();
}
+
+void local_flush_tlb_all(void)
+{
+ unsigned long flags, status;
+ int i;
+
+ /*
+ * Flush all the TLB.
+ */
+ local_irq_save(flags);
+ jump_to_uncached();
+
+ status = __raw_readl(MMUCR);
+ status = ((status & MMUCR_URB) >> MMUCR_URB_SHIFT);
+
+ if (status == 0)
+ status = MMUCR_URB_NENTRIES;
+
+ for (i = 0; i < status; i++)
+ __raw_writel(0x0, MMU_UTLB_ADDRESS_ARRAY | (i << 8));
+
+ for (i = 0; i < 4; i++)
+ __raw_writel(0x0, MMU_ITLB_ADDRESS_ARRAY | (i << 8));
+
+ back_to_cached();
+ ctrl_barrier();
+ local_irq_restore(flags);
+}
local_irq_save(flags);
- /* Load the entry into the TLB */
- __update_tlb(vma, addr, pte);
-
- /* ... and wire it up. */
status = __raw_readl(MMUCR);
urb = (status & MMUCR_URB) >> MMUCR_URB_SHIFT;
- status &= ~MMUCR_URB;
+ status &= ~MMUCR_URC;
/*
* Make sure we're not trying to wire the last TLB entry slot.
urb = urb % MMUCR_URB_NENTRIES;
+ /*
+ * Insert this entry into the highest non-wired TLB slot (via
+ * the URC field).
+ */
+ status |= (urb << MMUCR_URC_SHIFT);
+ __raw_writel(status, MMUCR);
+ ctrl_barrier();
+
+ /* Load the entry into the TLB */
+ __update_tlb(vma, addr, pte);
+
+ /* ... and wire it up. */
+ status = __raw_readl(MMUCR);
+
+ status &= ~MMUCR_URB;
status |= (urb << MMUCR_URB_SHIFT);
+
__raw_writel(status, MMUCR);
ctrl_barrier();
local_irq_restore(flags);
}
}
-
-void local_flush_tlb_all(void)
-{
- unsigned long flags, status;
-
- /*
- * Flush all the TLB.
- *
- * Write to the MMU control register's bit:
- * TF-bit for SH-3, TI-bit for SH-4.
- * It's same position, bit #2.
- */
- local_irq_save(flags);
- status = __raw_readl(MMUCR);
- status |= 0x04;
- __raw_writel(status, MMUCR);
- ctrl_barrier();
- local_irq_restore(flags);
-}
#include <linux/init.h>
+#include <linux/module.h>
#include <asm/sizes.h>
#include <asm/page.h>
+#include <asm/addrspace.h>
/*
* This is the offset of the uncached section from its cached alias.
unsigned long cached_to_uncached = SZ_512M;
unsigned long uncached_size = SZ_512M;
unsigned long uncached_start, uncached_end;
+EXPORT_SYMBOL(uncached_start);
+EXPORT_SYMBOL(uncached_end);
int virt_addr_uncached(unsigned long kaddr)
{
return (kaddr >= uncached_start) && (kaddr < uncached_end);
}
+EXPORT_SYMBOL(virt_addr_uncached);
void __init uncached_init(void)
{
+#ifdef CONFIG_29BIT
+ uncached_start = P2SEG;
+#else
uncached_start = memory_end;
+#endif
uncached_end = uncached_start + uncached_size;
}
#
# Automatically generated make config: don't edit
-# Linux kernel version: 2.6.33
-# Wed Mar 3 02:54:29 2010
+# Linux kernel version: 2.6.34-rc3
+# Sat Apr 3 15:49:56 2010
#
CONFIG_64BIT=y
CONFIG_SPARC=y
CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK=y
CONFIG_GENERIC_HARDIRQS_NO__DO_IRQ=y
CONFIG_MMU=y
+CONFIG_NEED_DMA_MAP_STATE=y
CONFIG_ARCH_NO_VIRT_TO_BUS=y
CONFIG_OF=y
CONFIG_ARCH_SUPPORTS_DEBUG_PAGEALLOC=y
# CONFIG_ENCLOSURE_SERVICES is not set
# CONFIG_HP_ILO is not set
# CONFIG_ISL29003 is not set
+# CONFIG_SENSORS_TSL2550 is not set
# CONFIG_DS1682 is not set
# CONFIG_C2PORT is not set
#
# SCSI device support
#
+CONFIG_SCSI_MOD=y
CONFIG_RAID_ATTRS=m
CONFIG_SCSI=y
CONFIG_SCSI_DMA=y
CONFIG_SERIAL_CORE=y
CONFIG_SERIAL_CORE_CONSOLE=y
# CONFIG_SERIAL_JSM is not set
+# CONFIG_SERIAL_TIMBERDALE is not set
# CONFIG_SERIAL_GRLIB_GAISLER_APBUART is not set
CONFIG_UNIX98_PTYS=y
# CONFIG_DEVPTS_MULTIPLE_INSTANCES is not set
#
# CONFIG_I2C_OCORES is not set
# CONFIG_I2C_SIMTEC is not set
+# CONFIG_I2C_XILINX is not set
#
# External I2C/SMBus adapter drivers
#
# CONFIG_I2C_PCA_PLATFORM is not set
# CONFIG_I2C_STUB is not set
-
-#
-# Miscellaneous I2C Chip support
-#
-# CONFIG_SENSORS_TSL2550 is not set
# CONFIG_I2C_DEBUG_CORE is not set
# CONFIG_I2C_DEBUG_ALGO is not set
# CONFIG_I2C_DEBUG_BUS is not set
-# CONFIG_I2C_DEBUG_CHIP is not set
# CONFIG_SPI is not set
#
# CONFIG_SENSORS_ADM1029 is not set
# CONFIG_SENSORS_ADM1031 is not set
# CONFIG_SENSORS_ADM9240 is not set
+# CONFIG_SENSORS_ADT7411 is not set
# CONFIG_SENSORS_ADT7462 is not set
# CONFIG_SENSORS_ADT7470 is not set
-# CONFIG_SENSORS_ADT7473 is not set
# CONFIG_SENSORS_ADT7475 is not set
+# CONFIG_SENSORS_ASC7621 is not set
# CONFIG_SENSORS_ATXP1 is not set
# CONFIG_SENSORS_DS1621 is not set
# CONFIG_SENSORS_I5K_AMB is not set
# Multifunction device drivers
#
# CONFIG_MFD_CORE is not set
+# CONFIG_MFD_88PM860X is not set
# CONFIG_MFD_SM501 is not set
# CONFIG_HTC_PASIC3 is not set
# CONFIG_TWL4030_CORE is not set
# CONFIG_MFD_TMIO is not set
# CONFIG_PMIC_DA903X is not set
# CONFIG_PMIC_ADP5520 is not set
+# CONFIG_MFD_MAX8925 is not set
# CONFIG_MFD_WM8400 is not set
# CONFIG_MFD_WM831X is not set
# CONFIG_MFD_WM8350_I2C is not set
+# CONFIG_MFD_WM8994 is not set
# CONFIG_MFD_PCF50633 is not set
# CONFIG_AB3100_CORE is not set
-# CONFIG_MFD_88PM8607 is not set
+# CONFIG_LPC_SCH is not set
# CONFIG_REGULATOR is not set
# CONFIG_MEDIA_SUPPORT is not set
# CONFIG_FB_LEO is not set
CONFIG_FB_XVR500=y
CONFIG_FB_XVR2500=y
+CONFIG_FB_XVR1000=y
# CONFIG_FB_S1D13XXX is not set
# CONFIG_FB_NVIDIA is not set
# CONFIG_FB_RIVA is not set
# CONFIG_USB_RIO500 is not set
# CONFIG_USB_LEGOTOWER is not set
# CONFIG_USB_LCD is not set
-# CONFIG_USB_BERRY_CHARGE is not set
# CONFIG_USB_LED is not set
# CONFIG_USB_CYPRESS_CY7C63 is not set
# CONFIG_USB_CYTHERM is not set
# CONFIG_USB_IOWARRIOR is not set
# CONFIG_USB_TEST is not set
# CONFIG_USB_ISIGHTFW is not set
-# CONFIG_USB_VST is not set
# CONFIG_USB_GADGET is not set
#
# CONFIG_BEFS_FS is not set
# CONFIG_BFS_FS is not set
# CONFIG_EFS_FS is not set
+# CONFIG_LOGFS is not set
# CONFIG_CRAMFS is not set
# CONFIG_SQUASHFS is not set
# CONFIG_VXFS_FS is not set
# CONFIG_NFS_FS is not set
# CONFIG_NFSD is not set
# CONFIG_SMB_FS is not set
+# CONFIG_CEPH_FS is not set
# CONFIG_CIFS is not set
# CONFIG_NCP_FS is not set
# CONFIG_CODA_FS is not set
ino_t st_ino;
mode_t st_mode;
short st_nlink;
- uid16_t st_uid;
- gid16_t st_gid;
+ unsigned short st_uid;
+ unsigned short st_gid;
unsigned short st_rdev;
off_t st_size;
time_t st_atime;
#include <linux/kernel.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include <linux/init.h>
#include <linux/of_device.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/cpumask.h>
nop
.size stack_trace_flush,.-stack_trace_flush
+#ifdef CONFIG_PERF_EVENTS
+ .globl perf_arch_fetch_caller_regs
+ .type perf_arch_fetch_caller_regs,#function
+perf_arch_fetch_caller_regs:
+ /* We always read the %pstate into %o5 since we will use
+ * that to construct a fake %tstate to store into the regs.
+ */
+ rdpr %pstate, %o5
+ brz,pn %o2, 50f
+ mov %o2, %g7
+
+ /* Turn off interrupts while we walk around the register
+ * window by hand.
+ */
+ wrpr %o5, PSTATE_IE, %pstate
+
+ /* The %canrestore tells us how many register windows are
+ * still live in the chip above us, past that we have to
+ * walk the frame as saved on the stack. We stash away
+ * the %cwp in %g1 so we can return back to the original
+ * register window.
+ */
+ rdpr %cwp, %g1
+ rdpr %canrestore, %g2
+ sub %g1, 1, %g3
+
+ /* We have the skip count in %g7, if it hits zero then
+ * %fp/%i7 are the registers we need. Otherwise if our
+ * %canrestore count maintained in %g2 hits zero we have
+ * to start traversing the stack.
+ */
+10: brz,pn %g2, 4f
+ sub %g2, 1, %g2
+ wrpr %g3, %cwp
+ subcc %g7, 1, %g7
+ bne,pt %xcc, 10b
+ sub %g3, 1, %g3
+
+ /* We found the values we need in the cpu's register
+ * windows.
+ */
+ mov %fp, %g3
+ ba,pt %xcc, 3f
+ mov %i7, %g2
+
+50: mov %fp, %g3
+ ba,pt %xcc, 2f
+ mov %i7, %g2
+
+ /* We hit the end of the valid register windows in the
+ * cpu, start traversing the stack frame.
+ */
+4: mov %fp, %g3
+
+20: ldx [%g3 + STACK_BIAS + RW_V9_I7], %g2
+ subcc %g7, 1, %g7
+ bne,pn %xcc, 20b
+ ldx [%g3 + STACK_BIAS + RW_V9_I6], %g3
+
+ /* Restore the current register window position and
+ * re-enable interrupts.
+ */
+3: wrpr %g1, %cwp
+ wrpr %o5, %pstate
+
+2: stx %g3, [%o0 + PT_V9_FP]
+ sllx %o5, 8, %o5
+ stx %o5, [%o0 + PT_V9_TSTATE]
+ stx %g2, [%o0 + PT_V9_TPC]
+ add %g2, 4, %g2
+ retl
+ stx %g2, [%o0 + PT_V9_TNPC]
+ .size perf_arch_fetch_caller_regs,.-perf_arch_fetch_caller_regs
+#endif /* CONFIG_PERF_EVENTS */
+
#ifdef CONFIG_SMP
.globl hard_smp_processor_id
.type hard_smp_processor_id,#function
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <asm/hypervisor.h>
#include <asm/oplib.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/device.h>
#include <linux/dma-mapping.h>
#include <linux/kprobes.h>
#include <linux/module.h>
#include <linux/kdebug.h>
+#include <linux/slab.h>
#include <asm/signal.h>
#include <asm/cacheflush.h>
#include <asm/uaccess.h>
#include <linux/init.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include <linux/jiffies.h>
#include <linux/timer.h>
#include <linux/module.h>
#include <linux/errno.h>
#include <linux/mutex.h>
-#include <linux/slab.h>
#include <linux/of.h>
#include <linux/of_platform.h>
#include <linux/interrupt.h>
#include <linux/profile.h>
#include <linux/pm.h>
#include <linux/delay.h>
+#include <linux/gfp.h>
#include <asm/cacheflush.h>
#include <asm/tlbflush.h>
#include <linux/elf.h>
#include <linux/vmalloc.h>
#include <linux/fs.h>
+#include <linux/gfp.h>
#include <linux/string.h>
#include <linux/ctype.h>
-#include <linux/slab.h>
#include <linux/mm.h>
#include <asm/processor.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/mod_devicetable.h>
-#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/irq.h>
#include <linux/of_device.h>
*/
#include <linux/kernel.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <linux/irq.h>
#include "pci_impl.h"
callchain_store(entry, PERF_CONTEXT_USER);
callchain_store(entry, regs->tpc);
- ufp = regs->u_regs[UREG_I6];
+ ufp = regs->u_regs[UREG_I6] & 0xffffffffUL;
do {
struct sparc_stackf32 *usf, sf;
unsigned long pc;
#include <linux/mm.h>
#include <linux/stddef.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/user.h>
#include <linux/smp.h>
#include <linux/reboot.h>
#include <linux/delay.h>
#include <linux/pm.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <asm/auxio.h>
#include <asm/oplib.h>
*k++ = regs->u_regs[pos++];
reg_window = (unsigned long __user *) regs->u_regs[UREG_I6];
+ reg_window -= 16;
for (; count > 0 && pos < 32; count--) {
if (get_user(*k++, ®_window[pos++]))
return -EFAULT;
}
reg_window = (unsigned long __user *) regs->u_regs[UREG_I6];
+ reg_window -= 16;
for (; count > 0 && pos < 32; count--) {
if (get_user(reg, ®_window[pos++]) ||
put_user(reg, u++))
regs->u_regs[pos++] = *k++;
reg_window = (unsigned long __user *) regs->u_regs[UREG_I6];
+ reg_window -= 16;
for (; count > 0 && pos < 32; count--) {
if (put_user(*k++, ®_window[pos++]))
return -EFAULT;
}
reg_window = (unsigned long __user *) regs->u_regs[UREG_I6];
+ reg_window -= 16;
for (; count > 0 && pos < 32; count--) {
if (get_user(reg, u++) ||
put_user(reg, ®_window[pos++]))
*k++ = regs->u_regs[pos++];
reg_window = (compat_ulong_t __user *) regs->u_regs[UREG_I6];
+ reg_window -= 16;
if (target == current) {
for (; count > 0 && pos < 32; count--) {
if (get_user(*k++, ®_window[pos++]))
}
reg_window = (compat_ulong_t __user *) regs->u_regs[UREG_I6];
+ reg_window -= 16;
if (target == current) {
for (; count > 0 && pos < 32; count--) {
if (get_user(reg, ®_window[pos++]) ||
regs->u_regs[pos++] = *k++;
reg_window = (compat_ulong_t __user *) regs->u_regs[UREG_I6];
+ reg_window -= 16;
if (target == current) {
for (; count > 0 && pos < 32; count--) {
if (put_user(*k++, ®_window[pos++]))
}
reg_window = (compat_ulong_t __user *) regs->u_regs[UREG_I6];
+ reg_window -= 16;
if (target == current) {
for (; count > 0 && pos < 32; count--) {
if (get_user(reg, u++) ||
#include <linux/stddef.h>
#include <linux/unistd.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <asm/smp.h>
#include <linux/user.h>
#include <linux/screen_info.h>
#include <linux/bootmem.h>
#include <linux/vmalloc.h>
#include <linux/cpu.h>
+#include <linux/slab.h>
#include <asm/head.h>
#include <asm/ptrace.h>
#include <linux/sched.h>
#include <linux/ptrace.h>
#include <linux/interrupt.h>
-#include <linux/slab.h>
#include <linux/init.h>
#include <linux/of.h>
#include <linux/of_device.h>
#include <linux/ptrace.h>
#include <linux/smp.h>
#include <linux/interrupt.h>
-#include <linux/slab.h>
#include <linux/init.h>
#include <linux/ioport.h>
#include <linux/of.h>
#include <linux/sem.h>
#include <linux/msg.h>
#include <linux/shm.h>
-#include <linux/slab.h>
#include <linux/uio.h>
#include <linux/nfs_fs.h>
#include <linux/quota.h>
#include <linux/compat.h>
#include <linux/vfs.h>
#include <linux/ptrace.h>
+#include <linux/slab.h>
#include <asm/types.h>
#include <asm/uaccess.h>
unsigned long ret;
/* should return -EINVAL to userspace */
- if (set_cpus_allowed(current, cpumask_of_cpu(cpu)))
+ if (set_cpus_allowed_ptr(current, cpumask_of(cpu)))
return 0;
ret = func(arg);
- set_cpus_allowed(current, old_affinity);
+ set_cpus_allowed_ptr(current, &old_affinity);
return ret;
}
#include <linux/mm.h>
#include <linux/init.h>
#include <linux/kdebug.h>
+#include <linux/gfp.h>
#include <asm/smp.h>
#include <asm/delay.h>
return 0;
cpus_allowed = current->cpus_allowed;
- set_cpus_allowed(current, cpumask_of_cpu(cpu));
+ set_cpus_allowed_ptr(current, cpumask_of(cpu));
clock_tick = sparc64_get_clock_tick(cpu) / 1000;
estar = read_hbreg(HBIRD_ESTAR_MODE_ADDR);
- set_cpus_allowed(current, cpus_allowed);
+ set_cpus_allowed_ptr(current, &cpus_allowed);
return clock_tick / estar_to_divisor(estar);
}
return;
cpus_allowed = current->cpus_allowed;
- set_cpus_allowed(current, cpumask_of_cpu(cpu));
+ set_cpus_allowed_ptr(current, cpumask_of(cpu));
new_freq = clock_tick = sparc64_get_clock_tick(cpu) / 1000;
new_bits = index_to_estar_mode(index);
cpufreq_notify_transition(&freqs, CPUFREQ_POSTCHANGE);
- set_cpus_allowed(current, cpus_allowed);
+ set_cpus_allowed_ptr(current, &cpus_allowed);
}
static int us2e_freq_target(struct cpufreq_policy *policy,
return 0;
cpus_allowed = current->cpus_allowed;
- set_cpus_allowed(current, cpumask_of_cpu(cpu));
+ set_cpus_allowed_ptr(current, cpumask_of(cpu));
reg = read_safari_cfg();
ret = get_current_freq(cpu, reg);
- set_cpus_allowed(current, cpus_allowed);
+ set_cpus_allowed_ptr(current, &cpus_allowed);
return ret;
}
return;
cpus_allowed = current->cpus_allowed;
- set_cpus_allowed(current, cpumask_of_cpu(cpu));
+ set_cpus_allowed_ptr(current, cpumask_of(cpu));
new_freq = sparc64_get_clock_tick(cpu) / 1000;
switch (index) {
cpufreq_notify_transition(&freqs, CPUFREQ_POSTCHANGE);
- set_cpus_allowed(current, cpus_allowed);
+ set_cpus_allowed_ptr(current, &cpus_allowed);
}
static int us3_freq_target(struct cpufreq_policy *policy,
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/irq.h>
#include <linux/init.h>
#include <linux/mm.h>
#include <linux/hugetlb.h>
#include <linux/pagemap.h>
-#include <linux/slab.h>
#include <linux/sysctl.h>
#include <asm/mman.h>
#include <linux/bootmem.h>
#include <linux/pagemap.h>
#include <linux/poison.h>
+#include <linux/gfp.h>
#include <asm/sections.h>
#include <asm/system.h>
#include <linux/bootmem.h>
#include <linux/mm.h>
#include <linux/hugetlb.h>
-#include <linux/slab.h>
#include <linux/initrd.h>
#include <linux/swap.h>
#include <linux/pagemap.h>
#include <linux/percpu.h>
#include <linux/lmb.h>
#include <linux/mmzone.h>
+#include <linux/gfp.h>
#include <asm/head.h>
#include <asm/system.h>
"node=%d entry=%lu/%lu\n", start, block, nr,
node,
addr >> VMEMMAP_CHUNK_SHIFT,
- VMEMMAP_SIZE >> VMEMMAP_CHUNK_SHIFT);
+ VMEMMAP_SIZE);
}
}
return 0;
#include <linux/kernel.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/pagemap.h>
#include <linux/init.h>
#include <linux/seq_file.h>
#include <linux/kdebug.h>
#include <linux/log2.h>
+#include <linux/gfp.h>
#include <asm/bitext.h>
#include <asm/page.h>
#include <linux/kernel.h>
#include <linux/mm.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/bootmem.h>
#include <linux/highmem.h>
#include <linux/fs.h>
#include <linux/kernel.h>
#include <linux/preempt.h>
+#include <linux/slab.h>
#include <asm/system.h>
#include <asm/page.h>
#include <asm/tlbflush.h>
#include <linux/platform_device.h>
#include <linux/rtnetlink.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include "init.h"
#include "irq_kern.h"
#include "linux/interrupt.h"
#include "linux/list.h"
#include "linux/mutex.h"
+#include "linux/slab.h"
#include "linux/workqueue.h"
#include "asm/atomic.h"
#include "init.h"
#include "linux/ctype.h"
#include "linux/capability.h"
#include "linux/mm.h"
+#include "linux/slab.h"
#include "linux/vmalloc.h"
#include "linux/blkpg.h"
#include "linux/genhd.h"
#include "linux/smp_lock.h"
#include "linux/ptrace.h"
#include "linux/sched.h"
+#include "linux/slab.h"
#include "asm/current.h"
#include "asm/processor.h"
#include "asm/uaccess.h"
#include "linux/module.h"
#include "linux/sched.h"
#include "linux/seq_file.h"
+#include "linux/slab.h"
#include "as-layout.h"
#include "kern_util.h"
#include "os.h"
#include <linux/stddef.h>
#include <linux/bootmem.h>
-#include <linux/gfp.h>
#include <linux/highmem.h>
#include <linux/mm.h>
#include <linux/swap.h>
+#include <linux/slab.h>
#include <asm/fixmap.h>
#include <asm/page.h>
#include "as-layout.h"
#include <linux/stddef.h>
#include <linux/err.h>
#include <linux/hardirq.h>
-#include <linux/gfp.h>
#include <linux/mm.h>
#include <linux/module.h>
#include <linux/personality.h>
#include <linux/proc_fs.h>
#include <linux/ptrace.h>
#include <linux/random.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#include <linux/seq_file.h>
#include <linux/tick.h>
*/
#include "linux/sched.h"
+#include "linux/slab.h"
#include "kern_util.h"
#include "os.h"
#include "skas.h"
#include "linux/mm.h"
#include "linux/sched.h"
+#include "linux/slab.h"
#include "asm/pgalloc.h"
#include "asm/pgtable.h"
#include "as-layout.h"
#include <errno.h>
#include <sched.h>
#include <linux/limits.h>
+#include <linux/slab.h>
#include <sys/socket.h>
#include <sys/wait.h>
#include "kern_constants.h"
#include <linux/mm.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <asm/unistd.h>
#include "os.h"
#include "proc_mm.h"
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <asm/i387.h>
struct crypto_fpu_ctx {
#include <linux/fcntl.h>
#include <linux/ptrace.h>
#include <linux/user.h>
-#include <linux/slab.h>
#include <linux/binfmts.h>
#include <linux/personality.h>
#include <linux/init.h>
#include <linux/ptrace.h>
#include <linux/highuid.h>
#include <linux/sysctl.h>
+#include <linux/slab.h>
#include <asm/mman.h>
#include <asm/types.h>
#include <asm/uaccess.h>
#endif
FIX_DBGP_BASE,
FIX_EARLYCON_MEM_BASE,
+#ifdef CONFIG_PROVIDE_OHCI1394_DMA_INIT
+ FIX_OHCI1394_BASE,
+#endif
#ifdef CONFIG_X86_LOCAL_APIC
FIX_APIC_BASE, /* local (CPU) APIC) -- required for SMP or not */
#endif
(__end_of_permanent_fixed_addresses & (TOTAL_FIX_BTMAPS - 1))
: __end_of_permanent_fixed_addresses,
FIX_BTMAP_BEGIN = FIX_BTMAP_END + TOTAL_FIX_BTMAPS - 1,
-#ifdef CONFIG_PROVIDE_OHCI1394_DMA_INIT
- FIX_OHCI1394_BASE,
-#endif
#ifdef CONFIG_X86_32
FIX_WP_TEST,
#endif
typedef int vector_irq_t[NR_VECTORS];
DECLARE_PER_CPU(vector_irq_t, vector_irq);
+extern void setup_vector_irq(int cpu);
#ifdef CONFIG_X86_IO_APIC
extern void lock_vector_lock(void);
#define MSR_AMD64_PATCH_LEVEL 0x0000008b
#define MSR_AMD64_NB_CFG 0xc001001f
#define MSR_AMD64_PATCH_LOADER 0xc0010020
+#define MSR_AMD64_OSVW_ID_LENGTH 0xc0010140
+#define MSR_AMD64_OSVW_STATUS 0xc0010141
#define MSR_AMD64_IBSFETCHCTL 0xc0011030
#define MSR_AMD64_IBSFETCHLINAD 0xc0011031
#define MSR_AMD64_IBSFETCHPHYSAD 0xc0011032
#include <asm/paravirt.h>
#include <linux/bitops.h>
-#include <linux/slab.h>
#include <linux/list.h>
#include <linux/spinlock.h>
#include <linux/module.h>
#include <linux/dmi.h>
#include <linux/irq.h>
+#include <linux/slab.h>
#include <linux/bootmem.h>
#include <linux/ioport.h>
#include <linux/pci.h>
#include <linux/vmalloc.h>
#include <linux/memory.h>
#include <linux/stop_machine.h>
+#include <linux/slab.h>
#include <asm/alternative.h>
#include <asm/sections.h>
#include <asm/pgtable.h>
*/
#include <linux/pci.h>
-#include <linux/gfp.h>
#include <linux/bitmap.h>
+#include <linux/slab.h>
#include <linux/debugfs.h>
#include <linux/scatterlist.h>
#include <linux/dma-mapping.h>
#include <linux/pci.h>
#include <linux/acpi.h>
-#include <linux/gfp.h>
#include <linux/list.h>
+#include <linux/slab.h>
#include <linux/sysdev.h>
#include <linux/interrupt.h>
#include <linux/msi.h>
#include <linux/errno.h>
#include <linux/init.h>
#include <linux/sysdev.h>
+#include <linux/slab.h>
#include <linux/pm.h>
#include <linux/pci.h>
#include <linux/sfi.h>
#include <linux/errno.h>
#include <linux/acpi.h>
#include <linux/init.h>
+#include <linux/gfp.h>
#include <linux/nmi.h>
#include <linux/smp.h>
#include <linux/io.h>
#include <linux/freezer.h>
#include <linux/kthread.h>
#include <linux/jiffies.h> /* time_after() */
+#include <linux/slab.h>
#ifdef CONFIG_ACPI
#include <acpi/acpi_bus.h>
#endif
/* Mark the inuse vectors */
for_each_irq_desc(irq, desc) {
cfg = desc->chip_data;
+
+ /*
+ * If it is a legacy IRQ handled by the legacy PIC, this cpu
+ * will be part of the irq_cfg's domain.
+ */
+ if (irq < legacy_pic->nr_legacy_irqs && !IO_APIC_IRQ(irq))
+ cpumask_set_cpu(cpu, cfg->domain);
+
if (!cpumask_test_cpu(cpu, cfg->domain))
continue;
vector = cfg->vector;
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/sysdev.h>
#include <linux/sysctl.h>
#include <linux/percpu.h>
#include <linux/ctype.h>
#include <linux/sched.h>
#include <linux/timer.h>
+#include <linux/slab.h>
#include <linux/cpu.h>
#include <linux/init.h>
#include <linux/io.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/string.h>
-#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/acpi.h>
#include <asm/io.h>
#include <linux/cpufreq.h>
#include <linux/compiler.h>
#include <linux/dmi.h>
+#include <linux/slab.h>
#include <trace/events/power.h>
#include <linux/acpi.h>
#include <linux/module.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/cpufreq.h>
#include <linux/cpufreq.h>
#include <linux/pci.h>
#include <linux/errno.h>
+#include <linux/slab.h>
#include <asm/processor-cyrix.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/cpufreq.h>
#include <linux/timex.h>
#include <linux/init.h>
#include <linux/smp.h>
#include <linux/cpufreq.h>
-#include <linux/slab.h>
#include <linux/cpumask.h>
#include <linux/timex.h>
#include <linux/sched.h>
#include <linux/cpufreq.h>
#include <linux/compiler.h>
+#include <linux/slab.h>
#include <linux/acpi.h>
#include <linux/io.h>
#include <linux/init.h>
#include <linux/cpufreq.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/timex.h>
#include <linux/io.h>
#include <linux/sched.h> /* current */
#include <linux/delay.h>
#include <linux/compiler.h>
+#include <linux/gfp.h>
#include <asm/msr.h>
#include <asm/processor.h>
#include <linux/init.h>
#include <linux/cpufreq.h>
#include <linux/pci.h>
-#include <linux/slab.h>
#include <linux/sched.h>
#include "speedstep-lib.h"
#include <linux/moduleparam.h>
#include <linux/init.h>
#include <linux/cpufreq.h>
-#include <linux/slab.h>
#include <asm/msr.h>
#include <asm/tsc.h>
#include <linux/moduleparam.h>
#include <linux/init.h>
#include <linux/cpufreq.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/io.h>
#include <asm/ist.h>
#include <linux/kdebug.h>
#include <linux/cpu.h>
#include <linux/sched.h>
+#include <linux/gfp.h>
#include <asm/mce.h>
#include <asm/apic.h>
#include <linux/sched.h>
#include <linux/sysfs.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/kmod.h>
#include <linux/poll.h>
#include <linux/errno.h>
#include <linux/sched.h>
#include <linux/sysfs.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/cpu.h>
#include <linux/smp.h>
* Author: Andi Kleen
*/
+#include <linux/gfp.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/percpu.h>
#include <linux/module.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/io.h>
#include <linux/mm.h>
#include <linux/module.h>
#include <linux/ctype.h>
#include <linux/string.h>
+#include <linux/slab.h>
#include <linux/init.h>
#define LINE_SIZE 80
#include <linux/kdebug.h>
#include <linux/sched.h>
#include <linux/uaccess.h>
+#include <linux/slab.h>
#include <linux/highmem.h>
#include <linux/cpu.h>
#include <linux/bitops.h>
#include <asm/apic.h>
#include <asm/stacktrace.h>
#include <asm/nmi.h>
+#include <asm/compat.h>
static u64 perf_event_mask __read_mostly;
struct perf_event *event);
struct event_constraint *event_constraints;
- void (*cpu_prepare)(int cpu);
+ int (*cpu_prepare)(int cpu);
void (*cpu_starting)(int cpu);
void (*cpu_dying)(int cpu);
void (*cpu_dead)(int cpu);
x86_pmu_notifier(struct notifier_block *self, unsigned long action, void *hcpu)
{
unsigned int cpu = (long)hcpu;
+ int ret = NOTIFY_OK;
switch (action & ~CPU_TASKS_FROZEN) {
case CPU_UP_PREPARE:
if (x86_pmu.cpu_prepare)
- x86_pmu.cpu_prepare(cpu);
+ ret = x86_pmu.cpu_prepare(cpu);
break;
case CPU_STARTING:
x86_pmu.cpu_dying(cpu);
break;
+ case CPU_UP_CANCELED:
case CPU_DEAD:
if (x86_pmu.cpu_dead)
x86_pmu.cpu_dead(cpu);
break;
}
- return NOTIFY_OK;
+ return ret;
}
static void __init pmu_check_apic(void)
return len;
}
-static int copy_stack_frame(const void __user *fp, struct stack_frame *frame)
+#ifdef CONFIG_COMPAT
+static inline int
+perf_callchain_user32(struct pt_regs *regs, struct perf_callchain_entry *entry)
{
- unsigned long bytes;
+ /* 32-bit process in 64-bit kernel. */
+ struct stack_frame_ia32 frame;
+ const void __user *fp;
- bytes = copy_from_user_nmi(frame, fp, sizeof(*frame));
+ if (!test_thread_flag(TIF_IA32))
+ return 0;
+
+ fp = compat_ptr(regs->bp);
+ while (entry->nr < PERF_MAX_STACK_DEPTH) {
+ unsigned long bytes;
+ frame.next_frame = 0;
+ frame.return_address = 0;
+
+ bytes = copy_from_user_nmi(&frame, fp, sizeof(frame));
+ if (bytes != sizeof(frame))
+ break;
+
+ if (fp < compat_ptr(regs->sp))
+ break;
- return bytes == sizeof(*frame);
+ callchain_store(entry, frame.return_address);
+ fp = compat_ptr(frame.next_frame);
+ }
+ return 1;
+}
+#else
+static inline int
+perf_callchain_user32(struct pt_regs *regs, struct perf_callchain_entry *entry)
+{
+ return 0;
}
+#endif
static void
perf_callchain_user(struct pt_regs *regs, struct perf_callchain_entry *entry)
callchain_store(entry, PERF_CONTEXT_USER);
callchain_store(entry, regs->ip);
+ if (perf_callchain_user32(regs, entry))
+ return;
+
while (entry->nr < PERF_MAX_STACK_DEPTH) {
+ unsigned long bytes;
frame.next_frame = NULL;
frame.return_address = 0;
- if (!copy_stack_frame(fp, &frame))
+ bytes = copy_from_user_nmi(&frame, fp, sizeof(frame));
+ if (bytes != sizeof(frame))
break;
if ((unsigned long)fp < regs->sp)
return entry;
}
-#ifdef CONFIG_EVENT_TRACING
void perf_arch_fetch_caller_regs(struct pt_regs *regs, unsigned long ip, int skip)
{
regs->ip = ip;
regs->cs = __KERNEL_CS;
local_save_flags(regs->flags);
}
-#endif
return (hwc->config & 0xe0) == 0xe0;
}
+static inline int amd_has_nb(struct cpu_hw_events *cpuc)
+{
+ struct amd_nb *nb = cpuc->amd_nb;
+
+ return nb && nb->nb_id != -1;
+}
+
static void amd_put_event_constraints(struct cpu_hw_events *cpuc,
struct perf_event *event)
{
/*
* only care about NB events
*/
- if (!(nb && amd_is_nb_event(hwc)))
+ if (!(amd_has_nb(cpuc) && amd_is_nb_event(hwc)))
return;
/*
/*
* if not NB event or no NB, then no constraints
*/
- if (!(nb && amd_is_nb_event(hwc)))
+ if (!(amd_has_nb(cpuc) && amd_is_nb_event(hwc)))
return &unconstrained;
/*
return nb;
}
-static void amd_pmu_cpu_online(int cpu)
+static int amd_pmu_cpu_prepare(int cpu)
+{
+ struct cpu_hw_events *cpuc = &per_cpu(cpu_hw_events, cpu);
+
+ WARN_ON_ONCE(cpuc->amd_nb);
+
+ if (boot_cpu_data.x86_max_cores < 2)
+ return NOTIFY_OK;
+
+ cpuc->amd_nb = amd_alloc_nb(cpu, -1);
+ if (!cpuc->amd_nb)
+ return NOTIFY_BAD;
+
+ return NOTIFY_OK;
+}
+
+static void amd_pmu_cpu_starting(int cpu)
{
- struct cpu_hw_events *cpu1, *cpu2;
- struct amd_nb *nb = NULL;
+ struct cpu_hw_events *cpuc = &per_cpu(cpu_hw_events, cpu);
+ struct amd_nb *nb;
int i, nb_id;
if (boot_cpu_data.x86_max_cores < 2)
return;
- /*
- * function may be called too early in the
- * boot process, in which case nb_id is bogus
- */
nb_id = amd_get_nb_id(cpu);
- if (nb_id == BAD_APICID)
- return;
-
- cpu1 = &per_cpu(cpu_hw_events, cpu);
- cpu1->amd_nb = NULL;
+ WARN_ON_ONCE(nb_id == BAD_APICID);
raw_spin_lock(&amd_nb_lock);
for_each_online_cpu(i) {
- cpu2 = &per_cpu(cpu_hw_events, i);
- nb = cpu2->amd_nb;
- if (!nb)
+ nb = per_cpu(cpu_hw_events, i).amd_nb;
+ if (WARN_ON_ONCE(!nb))
continue;
- if (nb->nb_id == nb_id)
- goto found;
- }
- nb = amd_alloc_nb(cpu, nb_id);
- if (!nb) {
- pr_err("perf_events: failed NB allocation for CPU%d\n", cpu);
- raw_spin_unlock(&amd_nb_lock);
- return;
+ if (nb->nb_id == nb_id) {
+ kfree(cpuc->amd_nb);
+ cpuc->amd_nb = nb;
+ break;
+ }
}
-found:
- nb->refcnt++;
- cpu1->amd_nb = nb;
+
+ cpuc->amd_nb->nb_id = nb_id;
+ cpuc->amd_nb->refcnt++;
raw_spin_unlock(&amd_nb_lock);
}
-static void amd_pmu_cpu_offline(int cpu)
+static void amd_pmu_cpu_dead(int cpu)
{
struct cpu_hw_events *cpuhw;
raw_spin_lock(&amd_nb_lock);
- if (--cpuhw->amd_nb->refcnt == 0)
- kfree(cpuhw->amd_nb);
+ if (cpuhw->amd_nb) {
+ struct amd_nb *nb = cpuhw->amd_nb;
+
+ if (nb->nb_id == -1 || --nb->refcnt == 0)
+ kfree(nb);
- cpuhw->amd_nb = NULL;
+ cpuhw->amd_nb = NULL;
+ }
raw_spin_unlock(&amd_nb_lock);
}
.get_event_constraints = amd_get_event_constraints,
.put_event_constraints = amd_put_event_constraints,
- .cpu_prepare = amd_pmu_cpu_online,
- .cpu_dead = amd_pmu_cpu_offline,
+ .cpu_prepare = amd_pmu_cpu_prepare,
+ .cpu_starting = amd_pmu_cpu_starting,
+ .cpu_dead = amd_pmu_cpu_dead,
};
static __init int amd_pmu_init(void)
#include <linux/cpu.h>
#include <linux/notifier.h>
#include <linux/uaccess.h>
+#include <linux/gfp.h>
#include <asm/processor.h>
#include <asm/msr.h>
* Copyright (C) IBM Corporation, 2004. All rights reserved
*/
+#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/highmem.h>
#include <linux/crash_dump.h>
unsigned long return_address;
};
+struct stack_frame_ia32 {
+ u32 next_frame;
+ u32 return_address;
+};
+
static inline unsigned long rewind_frame_pointer(int n)
{
struct stack_frame *frame;
#include <linux/init.h>
#include <linux/start_kernel.h>
+#include <linux/mm.h>
#include <asm/setup.h>
#include <asm/sections.h>
#ifdef CONFIG_BLK_DEV_INITRD
/* Reserve INITRD */
if (boot_params.hdr.type_of_loader && boot_params.hdr.ramdisk_image) {
+ /* Assume only end is not page aligned */
u64 ramdisk_image = boot_params.hdr.ramdisk_image;
u64 ramdisk_size = boot_params.hdr.ramdisk_size;
- u64 ramdisk_end = ramdisk_image + ramdisk_size;
+ u64 ramdisk_end = PAGE_ALIGN(ramdisk_image + ramdisk_size);
reserve_early(ramdisk_image, ramdisk_end, "RAMDISK");
}
#endif
#ifdef CONFIG_BLK_DEV_INITRD
/* Reserve INITRD */
if (boot_params.hdr.type_of_loader && boot_params.hdr.ramdisk_image) {
+ /* Assume only end is not page aligned */
unsigned long ramdisk_image = boot_params.hdr.ramdisk_image;
unsigned long ramdisk_size = boot_params.hdr.ramdisk_size;
- unsigned long ramdisk_end = ramdisk_image + ramdisk_size;
+ unsigned long ramdisk_end = PAGE_ALIGN(ramdisk_image + ramdisk_size);
reserve_early(ramdisk_image, ramdisk_end, "RAMDISK");
}
#endif
#include <linux/sysdev.h>
#include <linux/delay.h>
#include <linux/errno.h>
+#include <linux/slab.h>
#include <linux/hpet.h>
#include <linux/init.h>
#include <linux/cpu.h>
#include <linux/module.h>
#include <linux/regset.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <asm/sigcontext.h>
#include <asm/processor.h>
#include <linux/ioport.h>
#include <linux/interrupt.h>
#include <linux/timex.h>
-#include <linux/slab.h>
#include <linux/random.h>
#include <linux/init.h>
#include <linux/kernel_stat.h>
#include <linux/ioport.h>
#include <linux/interrupt.h>
#include <linux/timex.h>
-#include <linux/slab.h>
#include <linux/random.h>
#include <linux/kprobes.h>
#include <linux/init.h>
x86_init.irqs.intr_init();
}
+/*
+ * Setup the vector to irq mappings.
+ */
+void setup_vector_irq(int cpu)
+{
+#ifndef CONFIG_X86_IO_APIC
+ int irq;
+
+ /*
+ * On most of the platforms, legacy PIC delivers the interrupts on the
+ * boot cpu. But there are certain platforms where PIC interrupts are
+ * delivered to multiple cpu's. If the legacy IRQ is handled by the
+ * legacy PIC, for the new cpu that is coming online, setup the static
+ * legacy vector to irq mapping:
+ */
+ for (irq = 0; irq < legacy_pic->nr_legacy_irqs; irq++)
+ per_cpu(vector_irq, cpu)[IRQ0_VECTOR + irq] = irq;
+#endif
+
+ __setup_vector_irq(cpu);
+}
+
static void __init smp_intr_init(void)
{
#ifdef CONFIG_SMP
* Shared support code for AMD K8 northbridges and derivates.
* Copyright 2006 Andi Kleen, SUSE Labs. Subject to GPLv2.
*/
-#include <linux/gfp.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/errno.h>
#include <linux/module.h>
#include <linux/debugfs.h>
#include <linux/uaccess.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/stat.h>
#include <linux/io.h>
* portion of kgdb because this operation requires mutexs to
* complete.
*/
+ hw_breakpoint_init(&attr);
attr.bp_addr = (unsigned long)kgdb_arch_init;
- attr.type = PERF_TYPE_BREAKPOINT;
attr.bp_len = HW_BREAKPOINT_LEN_1;
attr.bp_type = HW_BREAKPOINT_W;
attr.disabled = 1;
*/
#include <linux/errno.h>
+#include <linux/gfp.h>
#include <linux/sched.h>
#include <linux/string.h>
#include <linux/mm.h>
#include <linux/mm.h>
#include <linux/kexec.h>
#include <linux/string.h>
+#include <linux/gfp.h>
#include <linux/reboot.h>
#include <linux/numa.h>
#include <linux/ftrace.h>
#include <linux/kernel.h>
#include <linux/mca.h>
#include <linux/kprobes.h>
+#include <linux/slab.h>
#include <asm/system.h>
#include <asm/io.h>
#include <linux/proc_fs.h>
#include <linux/kernel.h>
#include <linux/bug.h>
#include <linux/mm.h>
+#include <linux/gfp.h>
#include <asm/system.h>
#include <asm/page.h>
#include <linux/cpu.h>
#include <linux/notifier.h>
#include <linux/uaccess.h>
+#include <linux/gfp.h>
#include <asm/processor.h>
#include <asm/msr.h>
#include <linux/dma-debug.h>
#include <linux/dmar.h>
#include <linux/bootmem.h>
+#include <linux/gfp.h>
#include <linux/pci.h>
#include <linux/kmemleak.h>
#include <linux/iommu-helper.h>
#include <linux/sysdev.h>
#include <linux/io.h>
+#include <linux/gfp.h>
#include <asm/atomic.h>
#include <asm/mtrr.h>
#include <asm/pgtable.h>
#include <linux/scatterlist.h>
#include <linux/string.h>
#include <linux/init.h>
+#include <linux/gfp.h>
#include <linux/pci.h>
#include <linux/mm.h>
}
/*
- * Check for AMD CPUs, which have potentially C1E support
+ * Check for AMD CPUs, where APIC timer interrupt does not wake up CPU from C1e.
+ * For more information see
+ * - Erratum #400 for NPT family 0xf and family 0x10 CPUs
+ * - Erratum #365 for family 0x11 (not affected because C1e not in use)
*/
static int __cpuinit check_c1e_idle(const struct cpuinfo_x86 *c)
{
+ u64 val;
if (c->x86_vendor != X86_VENDOR_AMD)
- return 0;
-
- if (c->x86 < 0x0F)
- return 0;
+ goto no_c1e_idle;
/* Family 0x0f models < rev F do not have C1E */
- if (c->x86 == 0x0f && c->x86_model < 0x40)
- return 0;
+ if (c->x86 == 0x0F && c->x86_model >= 0x40)
+ return 1;
- return 1;
+ if (c->x86 == 0x10) {
+ /*
+ * check OSVW bit for CPUs that are not affected
+ * by erratum #400
+ */
+ rdmsrl(MSR_AMD64_OSVW_ID_LENGTH, val);
+ if (val >= 2) {
+ rdmsrl(MSR_AMD64_OSVW_STATUS, val);
+ if (!(val & BIT(1)))
+ goto no_c1e_idle;
+ }
+ return 1;
+ }
+
+no_c1e_idle:
+ return 0;
}
static cpumask_var_t c1e_mask;
#include <linux/mm.h>
#include <linux/smp.h>
#include <linux/errno.h>
+#include <linux/slab.h>
#include <linux/ptrace.h>
#include <linux/regset.h>
#include <linux/tracehook.h>
#include <linux/stddef.h>
#include <linux/unistd.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/user.h>
#include <linux/delay.h>
#define MAX_MAP_CHUNK (NR_FIX_BTMAPS << PAGE_SHIFT)
static void __init relocate_initrd(void)
{
-
+ /* Assume only end is not page aligned */
u64 ramdisk_image = boot_params.hdr.ramdisk_image;
u64 ramdisk_size = boot_params.hdr.ramdisk_size;
+ u64 area_size = PAGE_ALIGN(ramdisk_size);
u64 end_of_lowmem = max_low_pfn_mapped << PAGE_SHIFT;
u64 ramdisk_here;
unsigned long slop, clen, mapaddr;
char *p, *q;
/* We need to move the initrd down into lowmem */
- ramdisk_here = find_e820_area(0, end_of_lowmem, ramdisk_size,
+ ramdisk_here = find_e820_area(0, end_of_lowmem, area_size,
PAGE_SIZE);
if (ramdisk_here == -1ULL)
/* Note: this includes all the lowmem currently occupied by
the initrd, we rely on that fact to keep the data intact. */
- reserve_early(ramdisk_here, ramdisk_here + ramdisk_size,
+ reserve_early(ramdisk_here, ramdisk_here + area_size,
"NEW RAMDISK");
initrd_start = ramdisk_here + PAGE_OFFSET;
initrd_end = initrd_start + ramdisk_size;
static void __init reserve_initrd(void)
{
+ /* Assume only end is not page aligned */
u64 ramdisk_image = boot_params.hdr.ramdisk_image;
u64 ramdisk_size = boot_params.hdr.ramdisk_size;
- u64 ramdisk_end = ramdisk_image + ramdisk_size;
+ u64 ramdisk_end = PAGE_ALIGN(ramdisk_image + ramdisk_size);
u64 end_of_lowmem = max_low_pfn_mapped << PAGE_SHIFT;
if (!boot_params.hdr.type_of_loader ||
#include <linux/cache.h>
#include <linux/interrupt.h>
#include <linux/cpu.h>
+#include <linux/gfp.h>
#include <asm/mtrr.h>
#include <asm/tlbflush.h>
#include <linux/nmi.h>
#include <linux/tboot.h>
#include <linux/stackprotector.h>
+#include <linux/gfp.h>
#include <asm/acpi.h>
#include <asm/desc.h>
end_local_APIC_setup();
map_cpu_to_logical_apicid();
- notify_cpu_starting(cpuid);
-
/*
* Need to setup vector mappings before we enable interrupts.
*/
- __setup_vector_irq(smp_processor_id());
+ setup_vector_irq(smp_processor_id());
/*
* Get our bogomips.
*
*/
smp_store_cpu_info(cpuid);
+ notify_cpu_starting(cpuid);
+
/*
* Allow the master to continue.
*/
#include <linux/seq_file.h>
#include <linux/proc_fs.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <asm/mmu_context.h>
#include <asm/uv/uv.h>
#include <linux/module.h>
#include <linux/rbtree.h>
+#include <linux/slab.h>
#include <linux/irq.h>
#include <asm/apic.h>
* Copyright (c) Dimitri Sivanich
*/
#include <linux/clockchips.h>
+#include <linux/slab.h>
#include <asm/uv/uv_mmrs.h>
#include <asm/uv/uv_hub.h>
#include <linux/mm.h>
#include <linux/highmem.h>
#include <linux/sched.h>
+#include <linux/gfp.h>
#include <asm/vmi.h>
#include <asm/io.h>
#include <asm/fixmap.h>
.smp_locks : AT(ADDR(.smp_locks) - LOAD_OFFSET) {
__smp_locks = .;
*(.smp_locks)
- __smp_locks_end = .;
. = ALIGN(PAGE_SIZE);
+ __smp_locks_end = .;
}
#ifdef CONFIG_X86_64
#define pr_fmt(fmt) "pit: " fmt
#include <linux/kvm_host.h>
+#include <linux/slab.h>
#include "irq.h"
#include "i8254.h"
* Port from Qemu.
*/
#include <linux/mm.h>
+#include <linux/slab.h>
#include <linux/bitops.h>
#include "irq.h"
#include <linux/io.h>
#include <linux/module.h>
#include <linux/math64.h>
+#include <linux/slab.h>
#include <asm/processor.h>
#include <asm/msr.h>
#include <asm/page.h>
#include <linux/hugetlb.h>
#include <linux/compiler.h>
#include <linux/srcu.h>
+#include <linux/slab.h>
#include <asm/page.h>
#include <asm/cmpxchg.h>
#include <linux/highmem.h>
#include <linux/sched.h>
#include <linux/ftrace_event.h>
+#include <linux/slab.h>
#include <asm/desc.h>
#include <linux/sched.h>
#include <linux/moduleparam.h>
#include <linux/ftrace_event.h>
+#include <linux/slab.h>
#include "kvm_cache_regs.h"
#include "x86.h"
#include <linux/cpufreq.h>
#include <linux/user-return-notifier.h>
#include <linux/srcu.h>
+#include <linux/slab.h>
#include <trace/events/kvm.h>
#undef TRACE_INCLUDE_FILE
#define CREATE_TRACE_POINTS
#include <linux/mm.h>
#include <linux/hugetlb.h>
#include <linux/pagemap.h>
-#include <linux/slab.h>
#include <linux/err.h>
#include <linux/sysctl.h>
#include <asm/mman.h>
+#include <linux/gfp.h>
#include <linux/initrd.h>
#include <linux/ioport.h>
#include <linux/swap.h>
void free_init_pages(char *what, unsigned long begin, unsigned long end)
{
- unsigned long addr = begin;
+ unsigned long addr;
+ unsigned long begin_aligned, end_aligned;
- if (addr >= end)
+ /* Make sure boundaries are page aligned */
+ begin_aligned = PAGE_ALIGN(begin);
+ end_aligned = end & PAGE_MASK;
+
+ if (WARN_ON(begin_aligned != begin || end_aligned != end)) {
+ begin = begin_aligned;
+ end = end_aligned;
+ }
+
+ if (begin >= end)
return;
+ addr = begin;
+
/*
* If debugging page accesses then do not free this memory but
* mark them not present - any buggy init-section access will
*/
#ifdef CONFIG_DEBUG_PAGEALLOC
printk(KERN_INFO "debug: unmapping init memory %08lx..%08lx\n",
- begin, PAGE_ALIGN(end));
+ begin, end);
set_memory_np(begin, (end - begin) >> PAGE_SHIFT);
#else
/*
for (; addr < end; addr += PAGE_SIZE) {
ClearPageReserved(virt_to_page(addr));
init_page_count(virt_to_page(addr));
- memset((void *)(addr & ~(PAGE_SIZE-1)),
- POISON_FREE_INITMEM, PAGE_SIZE);
+ memset((void *)addr, POISON_FREE_INITMEM, PAGE_SIZE);
free_page(addr);
totalram_pages++;
}
#ifdef CONFIG_BLK_DEV_INITRD
void free_initrd_mem(unsigned long start, unsigned long end)
{
- free_init_pages("initrd memory", start, end);
+ /*
+ * end could be not aligned, and We can not align that,
+ * decompresser could be confused by aligned initrd_end
+ * We already reserve the end partial page before in
+ * - i386_start_kernel()
+ * - x86_64_start_kernel()
+ * - relocate_initrd()
+ * So here We can do PAGE_ALIGN() safely to get partial page to be freed
+ */
+ free_init_pages("initrd memory", start, PAGE_ALIGN(end));
}
#endif
#include <linux/pfn.h>
#include <linux/poison.h>
#include <linux/bootmem.h>
-#include <linux/slab.h>
#include <linux/proc_fs.h>
#include <linux/memory_hotplug.h>
#include <linux/initrd.h>
#include <linux/cpumask.h>
+#include <linux/gfp.h>
#include <asm/asm.h>
#include <asm/bios_ebda.h>
#include <linux/module.h>
#include <linux/memory_hotplug.h>
#include <linux/nmi.h>
+#include <linux/gfp.h>
#include <asm/processor.h>
#include <asm/bios_ebda.h>
#include <linux/kdebug.h>
#include <linux/mutex.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <asm/cacheflush.h>
#include <asm/tlbflush.h>
#include <linux/errno.h>
#include <linux/module.h>
#include <linux/debugfs.h>
+#include <linux/slab.h>
#include <linux/uaccess.h>
#include <linux/io.h>
#include <linux/version.h>
#include <linux/bootmem.h>
#include <linux/module.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/interrupt.h>
#include <linux/seq_file.h>
#include <linux/debugfs.h>
#include <linux/pfn.h>
#include <linux/percpu.h>
+#include <linux/gfp.h>
#include <asm/e820.h>
#include <asm/processor.h>
#include <linux/debugfs.h>
#include <linux/kernel.h>
#include <linux/module.h>
-#include <linux/gfp.h>
+#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/fs.h>
#include <linux/rbtree.h>
#include <linux/mm.h>
+#include <linux/gfp.h>
#include <asm/pgalloc.h>
#include <asm/pgtable.h>
#include <asm/tlb.h>
#include <linux/swap.h>
#include <linux/smp.h>
#include <linux/highmem.h>
-#include <linux/slab.h>
#include <linux/pagemap.h>
#include <linux/spinlock.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/irq.h>
#include <linux/dmi.h>
+#include <linux/slab.h>
#include <asm/numa.h>
#include <asm/pci_x86.h>
struct acpi_resource_address64 addr;
acpi_status status;
unsigned long flags;
- struct resource *root;
- u64 start, end;
+ struct resource *root, *conflict;
+ u64 start, end, max_len;
status = resource_to_addr(acpi_res, &addr);
if (!ACPI_SUCCESS(status))
} else
return AE_OK;
+ max_len = addr.maximum - addr.minimum + 1;
+ if (addr.address_length > max_len) {
+ dev_printk(KERN_DEBUG, &info->bridge->dev,
+ "host bridge window length %#llx doesn't fit in "
+ "%#llx-%#llx, trimming\n",
+ (unsigned long long) addr.address_length,
+ (unsigned long long) addr.minimum,
+ (unsigned long long) addr.maximum);
+ addr.address_length = max_len;
+ }
+
start = addr.minimum + addr.translation_offset;
end = start + addr.address_length - 1;
return AE_OK;
}
- if (insert_resource(root, res)) {
+ conflict = insert_resource_conflict(root, res);
+ if (conflict) {
dev_err(&info->bridge->dev,
- "can't allocate host bridge window %pR\n", res);
+ "address space collision: host bridge window %pR "
+ "conflicts with %s %pR\n",
+ res, conflict->name, conflict);
} else {
pci_bus_add_resource(info->bus, res, 0);
info->res_num++;
#include <linux/ioport.h>
#include <linux/init.h>
#include <linux/dmi.h>
+#include <linux/slab.h>
#include <asm/acpi.h>
#include <asm/segment.h>
continue;
if (!r->start ||
pci_claim_resource(dev, idx) < 0) {
- dev_info(&dev->dev,
- "can't reserve window %pR\n",
- r);
/*
* Something is wrong with the region.
* Invalidate the resource to prevent
"BAR %d: reserving %pr (d=%d, p=%d)\n",
idx, r, disabled, pass);
if (pci_claim_resource(dev, idx) < 0) {
- dev_info(&dev->dev,
- "can't reserve %pR\n", r);
/* We'll assign a new address later */
r->end -= r->start;
r->start = 0;
#include <linux/kernel.h>
#include <linux/pci.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/dmi.h>
#include <linux/io.h>
#include <linux/sfi_acpi.h>
#include <linux/bitmap.h>
#include <linux/dmi.h>
+#include <linux/slab.h>
#include <asm/e820.h>
#include <asm/pci_x86.h>
#include <asm/acpi.h>
#include <linux/pci.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/uaccess.h>
#include <asm/pci_x86.h>
* Copyright (c) 2006 Rafael J. Wysocki <rjw@sisk.pl>
*/
+#include <linux/gfp.h>
#include <linux/suspend.h>
#include <linux/bootmem.h>
* Copyright (c) 2001 Patrick Mochel <mochel@osdl.org>
*/
+#include <linux/gfp.h>
#include <linux/smp.h>
#include <linux/suspend.h>
#include <asm/proto.h>
#include <linux/mm.h>
#include <linux/err.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/random.h>
#include <linux/elf.h>
#include <linux/init.h>
#include <linux/debugfs.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include "debugfs.h"
#include <linux/highmem.h>
#include <linux/console.h>
#include <linux/pci.h>
+#include <linux/gfp.h>
#include <xen/xen.h>
#include <xen/interface/xen.h>
#include <linux/debugfs.h>
#include <linux/bug.h>
#include <linux/module.h>
+#include <linux/gfp.h>
#include <asm/pgtable.h>
#include <asm/tlbflush.h>
*/
#include <linux/sched.h>
#include <linux/err.h>
+#include <linux/slab.h>
#include <linux/smp.h>
#include <asm/paravirt.h>
#include <linux/spinlock.h>
#include <linux/debugfs.h>
#include <linux/log2.h>
+#include <linux/gfp.h>
#include <asm/paravirt.h>
#include <linux/clockchips.h>
#include <linux/kernel_stat.h>
#include <linux/math64.h>
+#include <linux/gfp.h>
#include <asm/pvclock.h>
#include <asm/xen/hypervisor.h>
#include <linux/mm.h>
#include <linux/string.h>
#include <linux/pci.h>
+#include <linux/gfp.h>
#include <asm/io.h>
#include <asm/cacheflush.h>
#include <linux/stddef.h>
#include <linux/unistd.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/elf.h>
#include <linux/init.h>
#include <linux/prctl.h>
#include <linux/module.h>
#include <linux/mqueue.h>
#include <linux/fs.h>
+#include <linux/slab.h>
#include <asm/pgtable.h>
#include <asm/uaccess.h>
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/bootmem.h>
+#include <linux/gfp.h>
#include <linux/swap.h>
#include <linux/mman.h>
#include <linux/nodemask.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <asm/bootparam.h>
#include <asm/page.h>
#include <linux/sched.h>
#include <linux/console.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/major.h>
#include <linux/param.h>
#include <linux/module.h>
#include <linux/bio.h>
#include <linux/blkdev.h>
+#include <linux/gfp.h>
#include "blk.h"
#include <linux/kdev_t.h>
#include <linux/module.h>
#include <linux/err.h>
+#include <linux/slab.h>
#include "blk-cgroup.h"
static DEFINE_SPINLOCK(blkio_list_lock);
#include <linux/mempool.h>
#include <linux/bio.h>
#include <linux/scatterlist.h>
+#include <linux/slab.h>
#include "blk.h"
#include <linux/bio.h>
#include <linux/blkdev.h>
#include <linux/bootmem.h> /* for max_pfn/max_low_pfn */
+#include <linux/slab.h>
#include "blk.h"
#include <linux/bootmem.h> /* for max_pfn/max_low_pfn */
#include <linux/gcd.h>
#include <linux/jiffies.h>
+#include <linux/gfp.h>
#include "blk.h"
* Functions related to sysfs handling
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/bio.h>
#include <linux/blkdev.h>
#include <linux/module.h>
#include <linux/bio.h>
#include <linux/blkdev.h>
+#include <linux/slab.h>
#include "blk.h"
#include <linux/idr.h>
#include <linux/bsg.h>
#include <linux/smp_lock.h>
+#include <linux/slab.h>
#include <scsi/scsi.h>
#include <scsi/scsi_ioctl.h>
* Copyright (C) 2003 Jens Axboe <axboe@kernel.dk>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/blkdev.h>
#include <linux/elevator.h>
#include <linux/jiffies.h>
#include <linux/elevator.h>
#include <linux/fd.h>
#include <linux/hdreg.h>
+#include <linux/slab.h>
#include <linux/syscalls.h>
#include <linux/smp_lock.h>
#include <linux/types.h>
#include <linux/capability.h>
#include <linux/blkdev.h>
+#include <linux/gfp.h>
#include <linux/blkpg.h>
#include <linux/hdreg.h>
#include <linux/backing-dev.h>
#include <linux/elevator.h>
#include <linux/bio.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
struct noop_data {
#include <linux/list.h>
#include <linux/module.h>
#include <linux/rtnetlink.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include "internal.h"
#include <linux/notifier.h>
#include <linux/rtnetlink.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include "internal.h"
#include <linux/dma-mapping.h>
#include <linux/raid/pq.h>
#include <linux/async_tx.h>
+#include <linux/gfp.h>
/**
* pq_scribble_page - space to hold throwaway P or Q buffer for
*
*/
#include <linux/async_tx.h>
+#include <linux/gfp.h>
#include <linux/random.h>
#undef pr
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/scatterlist.h>
-#include <linux/slab.h>
#include <linux/string.h>
struct hmac_ctx {
#include <linux/mutex.h>
#include <linux/random.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include <linux/string.h>
static DEFINE_MUTEX(crypto_default_rng_lock);
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/string.h>
#include <crypto/hash.h>
#include <linux/err.h>
#include <linux/init.h>
+#include <linux/gfp.h>
#include <linux/module.h>
-#include <linux/slab.h>
#include <linux/scatterlist.h>
#include <linux/string.h>
#include <linux/moduleparam.h>
#define BH_TRACE 0
#include <linux/module.h>
+#include <linux/gfp.h>
#include <linux/raid/xor.h>
#include <linux/jiffies.h>
#include <asm/xor.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/types.h>
#ifdef CONFIG_ACPI_PROCFS_POWER
#include <linux/init.h>
#include <linux/types.h>
#include <linux/memory_hotplug.h>
+#include <linux/slab.h>
#include <acpi/acpi_drivers.h>
#define ACPI_MEMORY_DEVICE_CLASS "memory"
#include <linux/freezer.h>
#include <linux/cpu.h>
#include <linux/clockchips.h>
+#include <linux/slab.h>
#include <acpi/acpi_bus.h>
#include <acpi/acpi_drivers.h>
#include <linux/jiffies.h>
#include <linux/async.h>
#include <linux/dmi.h>
+#include <linux/slab.h>
#ifdef CONFIG_ACPI_PROCFS_POWER
#include <linux/proc_fs.h>
#include <linux/device.h>
#include <linux/proc_fs.h>
#include <linux/acpi.h>
+#include <linux/slab.h>
#ifdef CONFIG_X86
#include <asm/mpspec.h>
#endif
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
#include <linux/input.h>
+#include <linux/slab.h>
#include <acpi/acpi_bus.h>
#include <acpi/acpi_drivers.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/acpi.h>
#include <acpi/acpi_bus.h>
#include <linux/kernel.h>
#include <linux/moduleparam.h>
#include <linux/debugfs.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <acpi/acpi_drivers.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/types.h>
#include <linux/notifier.h>
#include <linux/interrupt.h>
#include <linux/list.h>
#include <linux/spinlock.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <acpi/acpi_bus.h>
#include <acpi/acpi_drivers.h>
#include <linux/proc_fs.h>
#include <linux/init.h>
#include <linux/poll.h>
+#include <linux/gfp.h>
#include <acpi/acpi_drivers.h>
#include <net/netlink.h>
#include <net/genetlink.h>
#include <linux/init.h>
#include <linux/list.h>
#include <linux/device.h>
+#include <linux/slab.h>
#include <linux/rwsem.h>
#include <linux/acpi.h>
#include <linux/pm.h>
#include <linux/pci.h>
#include <linux/acpi.h>
+#include <linux/slab.h>
#include <acpi/acpi_bus.h>
#include <acpi/acpi_drivers.h>
#include <linux/pm.h>
#include <linux/pci.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <acpi/acpi_bus.h>
#include <acpi/acpi_drivers.h>
#include <linux/pci.h>
#include <linux/pci-acpi.h>
#include <linux/acpi.h>
+#include <linux/slab.h>
#include <acpi/acpi_bus.h>
#include <acpi/acpi_drivers.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/pci.h>
#include <linux/acpi.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
#include <acpi/acpi_bus.h>
#include <linux/jiffies.h>
#include <linux/mutex.h>
#include <linux/dmi.h>
+#include <linux/slab.h>
#include <linux/kdev_t.h>
#include <linux/sched.h>
#include <linux/time.h>
* - Added _PDC for platforms with Intel CPUs
*/
#include <linux/dmi.h>
+#include <linux/slab.h>
#include <acpi/acpi_drivers.h>
#include <acpi/processor.h>
#include <linux/dmi.h>
#include <linux/moduleparam.h>
#include <linux/cpuidle.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/system.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/cpufreq.h>
+#include <linux/slab.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
#include <linux/acpi.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/cpufreq.h>
+#include <linux/slab.h>
#ifdef CONFIG_X86
#include <asm/cpufeature.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/sched.h>
#include <linux/cpufreq.h>
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/kernel.h>
#include <acpi/acpi_bus.h>
#include <acpi/acpi_drivers.h>
#include <linux/wait.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include "sbshc.h"
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/acpi.h>
#include <linux/signal.h>
#include <linux/kthread.h>
+#include <linux/dmi.h>
#include <acpi/acpi_drivers.h>
list_add_tail(&id->list, &device->pnp.ids);
}
+/*
+ * Old IBM workstations have a DSDT bug wherein the SMBus object
+ * lacks the SMBUS01 HID and the methods do not have the necessary "_"
+ * prefix. Work around this.
+ */
+static int acpi_ibm_smbus_match(struct acpi_device *device)
+{
+ acpi_handle h_dummy;
+ struct acpi_buffer path = {ACPI_ALLOCATE_BUFFER, NULL};
+ int result;
+
+ if (!dmi_name_in_vendors("IBM"))
+ return -ENODEV;
+
+ /* Look for SMBS object */
+ result = acpi_get_name(device->handle, ACPI_SINGLE_NAME, &path);
+ if (result)
+ return result;
+
+ if (strcmp("SMBS", path.pointer)) {
+ result = -ENODEV;
+ goto out;
+ }
+
+ /* Does it have the necessary (but misnamed) methods? */
+ result = -ENODEV;
+ if (ACPI_SUCCESS(acpi_get_handle(device->handle, "SBI", &h_dummy)) &&
+ ACPI_SUCCESS(acpi_get_handle(device->handle, "SBR", &h_dummy)) &&
+ ACPI_SUCCESS(acpi_get_handle(device->handle, "SBW", &h_dummy)))
+ result = 0;
+out:
+ kfree(path.pointer);
+ return result;
+}
+
static void acpi_device_set_id(struct acpi_device *device)
{
acpi_status status;
acpi_add_id(device, ACPI_BAY_HID);
else if (ACPI_SUCCESS(acpi_dock_match(device)))
acpi_add_id(device, ACPI_DOCK_HID);
+ else if (!acpi_ibm_smbus_match(device))
+ acpi_add_id(device, ACPI_SMBUS_IBM_HID);
break;
case ACPI_BUS_TYPE_POWER:
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/string.h>
#include <asm/uaccess.h>
#include <linux/module.h>
#include <linux/dmi.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/proc_fs.h>
#include <linux/jiffies.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/types.h>
#include <acpi/acpi_bus.h>
#include <linux/sort.h>
#include <linux/pci.h>
#include <linux/pci_ids.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <linux/dmi.h>
#include <acpi/acpi_bus.h>
#include <linux/dma-mapping.h>
#include <linux/device.h>
#include <linux/dmi.h>
+#include <linux/gfp.h>
#include <scsi/scsi_host.h>
#include <scsi/scsi_cmnd.h>
#include <linux/libata.h>
#include <linux/blkdev.h>
#include <linux/delay.h>
#include <linux/device.h>
+#include <linux/gfp.h>
#include <scsi/scsi_host.h>
#include <linux/libata.h>
#include <linux/dmi.h>
#include <linux/acpi.h>
#include <linux/libata.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <scsi/scsi_device.h>
#include "libata.h"
#include <linux/io.h>
#include <linux/async.h>
#include <linux/log2.h>
+#include <linux/slab.h>
#include <scsi/scsi.h>
#include <scsi/scsi_cmnd.h>
#include <scsi/scsi_host.h>
{
struct ata_eh_context *ehc = &dev->link->eh_context;
int print_info = ehc->i.flags & ATA_EHI_PRINTINFO;
+ bool unlock_hpa = ata_ignore_hpa || dev->flags & ATA_DFLAG_UNLOCK_HPA;
u64 sectors = ata_id_n_sectors(dev->id);
u64 native_sectors;
int rc;
/* If device aborted the command or HPA isn't going to
* be unlocked, skip HPA resizing.
*/
- if (rc == -EACCES || !ata_ignore_hpa) {
+ if (rc == -EACCES || !unlock_hpa) {
ata_dev_printk(dev, KERN_WARNING, "HPA support seems "
"broken, skipping HPA handling\n");
dev->horkage |= ATA_HORKAGE_BROKEN_HPA;
dev->n_native_sectors = native_sectors;
/* nothing to do? */
- if (native_sectors <= sectors || !ata_ignore_hpa) {
+ if (native_sectors <= sectors || !unlock_hpa) {
if (!print_info || native_sectors == sectors)
return 0;
goto fail;
/* verify n_sectors hasn't changed */
- if (dev->class == ATA_DEV_ATA && n_sectors &&
- dev->n_sectors != n_sectors) {
- ata_dev_printk(dev, KERN_WARNING, "n_sectors mismatch "
- "%llu != %llu\n",
- (unsigned long long)n_sectors,
- (unsigned long long)dev->n_sectors);
- /*
- * Something could have caused HPA to be unlocked
- * involuntarily. If n_native_sectors hasn't changed
- * and the new size matches it, keep the device.
- */
- if (dev->n_native_sectors == n_native_sectors &&
- dev->n_sectors > n_sectors &&
- dev->n_sectors == n_native_sectors) {
- ata_dev_printk(dev, KERN_WARNING,
- "new n_sectors matches native, probably "
- "late HPA unlock, continuing\n");
- /* keep using the old n_sectors */
- dev->n_sectors = n_sectors;
- } else {
- /* restore original n_[native]_sectors and fail */
- dev->n_native_sectors = n_native_sectors;
- dev->n_sectors = n_sectors;
- rc = -ENODEV;
- goto fail;
- }
+ if (dev->class != ATA_DEV_ATA || !n_sectors ||
+ dev->n_sectors == n_sectors)
+ return 0;
+
+ /* n_sectors has changed */
+ ata_dev_printk(dev, KERN_WARNING, "n_sectors mismatch %llu != %llu\n",
+ (unsigned long long)n_sectors,
+ (unsigned long long)dev->n_sectors);
+
+ /*
+ * Something could have caused HPA to be unlocked
+ * involuntarily. If n_native_sectors hasn't changed and the
+ * new size matches it, keep the device.
+ */
+ if (dev->n_native_sectors == n_native_sectors &&
+ dev->n_sectors > n_sectors && dev->n_sectors == n_native_sectors) {
+ ata_dev_printk(dev, KERN_WARNING,
+ "new n_sectors matches native, probably "
+ "late HPA unlock, continuing\n");
+ /* keep using the old n_sectors */
+ dev->n_sectors = n_sectors;
+ return 0;
}
- return 0;
+ /*
+ * Some BIOSes boot w/o HPA but resume w/ HPA locked. Try
+ * unlocking HPA in those cases.
+ *
+ * https://bugzilla.kernel.org/show_bug.cgi?id=15396
+ */
+ if (dev->n_native_sectors == n_native_sectors &&
+ dev->n_sectors < n_sectors && n_sectors == n_native_sectors &&
+ !(dev->horkage & ATA_HORKAGE_BROKEN_HPA)) {
+ ata_dev_printk(dev, KERN_WARNING,
+ "old n_sectors matches native, probably "
+ "late HPA lock, will try to unlock HPA\n");
+ /* try unlocking HPA */
+ dev->flags |= ATA_DFLAG_UNLOCK_HPA;
+ rc = -EIO;
+ } else
+ rc = -ENODEV;
+ /* restore original n_[native_]sectors and fail */
+ dev->n_native_sectors = n_native_sectors;
+ dev->n_sectors = n_sectors;
fail:
ata_dev_printk(dev, KERN_ERR, "revalidation failed (errno=%d)\n", rc);
return rc;
{ "HTS541080G9SA00", "MB4OC60D", ATA_HORKAGE_NONCQ, },
{ "HTS541010G9SA00", "MBZOC60D", ATA_HORKAGE_NONCQ, },
+ /* https://bugzilla.kernel.org/show_bug.cgi?id=15573 */
+ { "C300-CTFDDAC128MAG", "0001", ATA_HORKAGE_NONCQ, },
+
/* devices which puke on READ_NATIVE_MAX */
{ "HDS724040KLSA80", "KFAOA20N", ATA_HORKAGE_BROKEN_HPA, },
{ "WDC WD3200JD-00KLB0", "WD-WCAMR1130137", ATA_HORKAGE_BROKEN_HPA },
#include <linux/kernel.h>
#include <linux/libata.h>
+#include <linux/slab.h>
#include "libata.h"
const struct ata_port_operations sata_pmp_port_ops = {
*
*/
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/blkdev.h>
#include <linux/spinlock.h>
*/
#include <linux/kernel.h>
+#include <linux/gfp.h>
#include <linux/pci.h>
#include <linux/libata.h>
#include <linux/highmem.h>
{
struct ata_eh_info *ehi = &ap->link.eh_info;
u8 status, host_stat = 0;
+ bool bmdma_stopped = false;
VPRINTK("ata%u: protocol %d task_state %d\n",
ap->print_id, qc->tf.protocol, ap->hsm_task_state);
/* before we do anything else, clear DMA-Start bit */
ap->ops->bmdma_stop(qc);
+ bmdma_stopped = true;
if (unlikely(host_stat & ATA_DMA_ERR)) {
/* error when transfering data to/from memory */
/* check main status, clearing INTRQ if needed */
status = ata_sff_irq_status(ap);
- if (status & ATA_BUSY)
- goto idle_irq;
+ if (status & ATA_BUSY) {
+ if (bmdma_stopped) {
+ /* BMDMA engine is already stopped, we're screwed */
+ qc->err_mask |= AC_ERR_HSM;
+ ap->hsm_task_state = HSM_ST_ERR;
+ } else
+ goto idle_irq;
+ }
/* ack bmdma irq events */
ap->ops->sff_irq_clear(ap);
irqreturn_t ata_sff_interrupt(int irq, void *dev_instance)
{
struct ata_host *host = dev_instance;
+ bool retried = false;
unsigned int i;
- unsigned int handled = 0, polling = 0;
+ unsigned int handled, idle, polling;
unsigned long flags;
/* TODO: make _irqsave conditional on x86 PCI IDE legacy mode */
spin_lock_irqsave(&host->lock, flags);
+retry:
+ handled = idle = polling = 0;
for (i = 0; i < host->n_ports; i++) {
struct ata_port *ap = host->ports[i];
struct ata_queued_cmd *qc;
handled |= ata_sff_host_intr(ap, qc);
else
polling |= 1 << i;
- }
+ } else
+ idle |= 1 << i;
}
/*
* asserting IRQ line, nobody cared will ensue. Check IRQ
* pending status if available and clear spurious IRQ.
*/
- if (!handled) {
+ if (!handled && !retried) {
+ bool retry = false;
+
for (i = 0; i < host->n_ports; i++) {
struct ata_port *ap = host->ports[i];
!ap->ops->sff_irq_check(ap))
continue;
- if (printk_ratelimit())
- ata_port_printk(ap, KERN_INFO,
- "clearing spurious IRQ\n");
+ if (idle & (1 << i)) {
+ ap->ops->sff_check_status(ap);
+ ap->ops->sff_irq_clear(ap);
+ } else {
+ /* clear INTRQ and check if BUSY cleared */
+ if (!(ap->ops->sff_check_status(ap) & ATA_BUSY))
+ retry |= true;
+ /*
+ * With command in flight, we can't do
+ * sff_irq_clear() w/o racing with completion.
+ */
+ }
+ }
- ap->ops->sff_check_status(ap);
- ap->ops->sff_irq_clear(ap);
+ if (retry) {
+ retried = true;
+ goto retry;
}
}
#include <linux/blkdev.h>
#include <linux/delay.h>
#include <linux/device.h>
+#include <linux/gfp.h>
#include <scsi/scsi_host.h>
#include <acpi/acpi_bus.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/irq.h>
+#include <linux/slab.h>
#include <scsi/scsi_host.h>
#include <linux/ata.h>
#include <linux/libata.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/blkdev.h>
+#include <linux/gfp.h>
#include <scsi/scsi_host.h>
#include <linux/ata.h>
#include <linux/clk.h>
#include <linux/blkdev.h>
#include <linux/delay.h>
#include <linux/device.h>
+#include <linux/gfp.h>
#include <scsi/scsi_host.h>
#include <linux/libata.h>
#include <linux/init.h>
#include <linux/blkdev.h>
#include <linux/delay.h>
+#include <linux/gfp.h>
#include <scsi/scsi_host.h>
#include <linux/libata.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/blkdev.h>
+#include <linux/gfp.h>
#include <scsi/scsi_host.h>
#include <linux/ata.h>
#include <linux/libata.h>
#include <linux/init.h>
#include <linux/blkdev.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <scsi/scsi_host.h>
#include <linux/libata.h>
#include <linux/pmu.h>
#include <linux/scatterlist.h>
#include <linux/of.h>
+#include <linux/gfp.h>
#include <scsi/scsi.h>
#include <scsi/scsi_host.h>
#include <linux/kernel.h>
#include <linux/module.h>
-#include <linux/slab.h>
+#include <linux/gfp.h>
#include <linux/delay.h>
#include <linux/libata.h>
#include <linux/of_platform.h>
#include <linux/module.h>
#include <linux/libata.h>
#include <linux/irq.h>
+#include <linux/slab.h>
#include <linux/platform_device.h>
#include <linux/workqueue.h>
#include <scsi/scsi_host.h>
#include <linux/init.h>
#include <linux/blkdev.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <scsi/scsi_host.h>
#include <linux/ata.h>
#include <linux/libata.h>
*
*/
+#include <linux/gfp.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/platform_device.h>
#include <linux/blkdev.h>
#include <linux/delay.h>
#include <linux/device.h>
+#include <linux/gfp.h>
#include <scsi/scsi_host.h>
#include <linux/libata.h>
#include <linux/dmi.h>
#include <linux/init.h>
#include <linux/blkdev.h>
#include <linux/delay.h>
+#include <linux/gfp.h>
#include <scsi/scsi_host.h>
#include <linux/libata.h>
#include <linux/dmi.h>
u8 rev = isa->revision;
pci_dev_put(isa);
+ if ((id->device == 0x0415 || id->device == 0x3164) &&
+ (config->id != id->device))
+ continue;
+
if (rev >= config->rev_min && rev <= config->rev_max)
break;
}
{ PCI_VDEVICE(VIA, 0x3164), },
{ PCI_VDEVICE(VIA, 0x5324), },
{ PCI_VDEVICE(VIA, 0xC409), VIA_IDFLAG_SINGLE },
+ { PCI_VDEVICE(VIA, 0x9001), VIA_IDFLAG_SINGLE },
{ },
};
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/gfp.h>
#include <linux/pci.h>
#include <linux/init.h>
#include <linux/blkdev.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <scsi/scsi_host.h>
#include <scsi/scsi_cmnd.h>
* happy to assist.
*/
+#include <linux/gfp.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/pci.h>
#include <linux/ata_platform.h>
#include <linux/mbus.h>
#include <linux/bitops.h>
+#include <linux/gfp.h>
#include <scsi/scsi_host.h>
#include <scsi/scsi_cmnd.h>
#include <scsi/scsi_device.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/gfp.h>
#include <linux/pci.h>
#include <linux/init.h>
#include <linux/blkdev.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/gfp.h>
#include <linux/pci.h>
#include <linux/init.h>
#include <linux/blkdev.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/gfp.h>
#include <linux/pci.h>
#include <linux/init.h>
#include <linux/blkdev.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/gfp.h>
#include <linux/pci.h>
#include <linux/blkdev.h>
#include <linux/delay.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/blkdev.h>
#include <linux/delay.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/gfp.h>
#include <linux/pci.h>
#include <linux/init.h>
#include <linux/blkdev.h>
#include <linux/mm.h>
#include <linux/timer.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/byteorder.h>
#include <asm/uaccess.h>
#include <linux/mutex.h>
#include <linux/firmware.h>
#include <linux/ihex.h>
+#include <linux/slab.h>
#include <asm/atomic.h>
#include <asm/io.h>
#include <linux/atm_tcp.h>
#include <linux/bitops.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/atomic.h>
#include <linux/init.h>
#include <linux/atm_eni.h>
#include <linux/bitops.h>
+#include <linux/slab.h>
#include <asm/system.h>
#include <asm/io.h>
#include <asm/atomic.h>
#include <linux/init.h>
#include <linux/capability.h>
#include <linux/bitops.h>
+#include <linux/slab.h>
#include <asm/byteorder.h>
#include <asm/system.h>
#include <asm/string.h>
#include <linux/timer.h>
#include <linux/interrupt.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/byteorder.h>
#include <asm/uaccess.h>
#include <linux/init.h>
#include <linux/ioport.h>
#include <linux/wait.h>
+#include <linux/slab.h>
#include <asm/system.h>
#include <asm/io.h>
#include <linux/capability.h>
#include <linux/atm_idt77105.h>
#include <linux/spinlock.h>
+#include <linux/slab.h>
#include <asm/system.h>
#include <asm/param.h>
#include <asm/uaccess.h>
#include <linux/wait.h>
#include <linux/jiffies.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/uaccess.h>
#include <linux/uio.h>
#include <linux/init.h>
#include <linux/wait.h>
+#include <linux/slab.h>
#include <asm/system.h>
#include <asm/io.h>
#include <asm/atomic.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/atmdev.h>
#include <asm/io.h>
#include <linux/timer.h>
#include <linux/interrupt.h>
#include <linux/bitops.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/uaccess.h>
#include <asm/atomic.h>
#include <linux/firmware.h>
#include <linux/ctype.h>
#include <linux/swab.h>
+#include <linux/slab.h>
#define VERSION "0.07"
#define PTAG "solos-pci"
#include <linux/init.h>
#include <linux/capability.h>
#include <linux/atm_suni.h>
+#include <linux/slab.h>
#include <asm/system.h>
#include <asm/param.h>
#include <asm/uaccess.h>
#include <linux/atmdev.h>
#include <linux/sonet.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/atomic.h>
#include <linux/capability.h>
#include <linux/bitops.h>
#include <linux/wait.h>
+#include <linux/slab.h>
#include <asm/byteorder.h>
#include <asm/system.h>
#include <asm/string.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/fs.h>
+#include <linux/slab.h>
#include <linux/cdev.h>
#include <linux/delay.h>
#include <linux/device.h>
#include <linux/fb.h>
#include <linux/mm.h>
#include <linux/platform_device.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/uaccess.h>
#include <linux/cfag12864b.h>
return sysfs_create_group(&classdev->kobj, cont->grp);
for (i = 0; attrs[i]; i++) {
+ sysfs_attr_init(&attrs[i]->attr);
error = device_create_file(classdev, attrs[i]);
if (error)
return error;
#include <linux/device.h>
#include <linux/module.h>
#include <linux/errno.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/string.h>
#include "base.h"
* This is used to create a struct class pointer that can then be used
* in calls to device_create().
*
+ * Returns &struct class pointer on success, or ERR_PTR() on error.
+ *
* Note, the pointer created here is to be destroyed when finished by
* making a call to class_destroy().
*/
* 'module' symlink which points to the @owner directory
* in sysfs.
*
+ * Returns &struct device pointer on success, or ERR_PTR() on error.
+ *
* Note: You probably want to use root_device_register().
*/
struct device *__root_device_register(const char *name, struct module *owner)
* Any further sysfs files that might be required can be created using this
* pointer.
*
+ * Returns &struct device pointer on success, or ERR_PTR() on error.
+ *
* Note: the struct class passed to this function must have previously
* been created with a call to class_create().
*/
* Any further sysfs files that might be required can be created using this
* pointer.
*
+ * Returns &struct device pointer on success, or ERR_PTR() on error.
+ *
* Note: the struct class passed to this function must have previously
* been created with a call to class_create().
*/
#include <linux/topology.h>
#include <linux/device.h>
#include <linux/node.h>
+#include <linux/gfp.h>
#include "base.h"
}
#ifdef CONFIG_ARCH_CPU_PROBE_RELEASE
-static ssize_t cpu_probe_store(struct sys_device *dev,
- struct sysdev_attribute *attr,
- const char *buf,
+static ssize_t cpu_probe_store(struct sysdev_class *class,
+ struct sysdev_class_attribute *attr,
+ const char *buf,
size_t count)
{
return arch_cpu_probe(buf, count);
}
-static ssize_t cpu_release_store(struct sys_device *dev,
- struct sysdev_attribute *attr,
- const char *buf,
+static ssize_t cpu_release_store(struct sysdev_class *class,
+ struct sysdev_class_attribute *attr,
+ const char *buf,
size_t count)
{
return arch_cpu_release(buf, count);
}
-static SYSDEV_ATTR(probe, S_IWUSR, NULL, cpu_probe_store);
-static SYSDEV_ATTR(release, S_IWUSR, NULL, cpu_release_store);
+static SYSDEV_CLASS_ATTR(probe, S_IWUSR, NULL, cpu_probe_store);
+static SYSDEV_CLASS_ATTR(release, S_IWUSR, NULL, cpu_release_store);
#endif /* CONFIG_ARCH_CPU_PROBE_RELEASE */
#else /* ... !CONFIG_HOTPLUG_CPU */
#include <linux/device.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include "base.h"
#include <linux/cred.h>
#include <linux/sched.h>
#include <linux/init_task.h>
+#include <linux/slab.h>
static struct vfsmount *dev_mnt;
* Coherent per-device memory handling.
* Borrowed from i386
*/
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/dma-mapping.h>
*/
#include <linux/dma-mapping.h>
+#include <linux/gfp.h>
/*
* Managed DMA API
#include <linux/device.h>
#include <linux/module.h>
#include <linux/errno.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include "base.h"
#include <linux/kthread.h>
#include <linux/highmem.h>
#include <linux/firmware.h>
+#include <linux/slab.h>
#define to_dev(obj) container_of(obj, struct device, kobj)
/**
* firmware_timeout_store - set number of seconds to wait for firmware
* @class: device class pointer
+ * @attr: device attribute pointer
* @buf: buffer to scan for timeout value
* @count: number of bytes in @buf
*
fw_priv = dev_get_drvdata(f_dev);
fw_priv->fw = fw;
+ sysfs_bin_attr_init(&fw_priv->attr_data);
retval = sysfs_create_bin_file(&f_dev->kobj, &fw_priv->attr_data);
if (retval) {
dev_err(device, "%s: sysfs_create_bin_file failed\n", __func__);
#include <linux/mm.h>
#include <linux/mutex.h>
#include <linux/stat.h>
+#include <linux/slab.h>
#include <asm/atomic.h>
#include <asm/uaccess.h>
#include <linux/device.h>
#include <linux/module.h>
#include <linux/errno.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include "base.h"
#include <linux/cpu.h>
#include <linux/device.h>
#include <linux/swap.h>
+#include <linux/gfp.h>
static struct sysdev_class_attribute *node_state_attrs[];
int len = 0;
int i;
- /* buf currently PAGE_SIZE, need ~4 chars per node */
- BUILD_BUG_ON(MAX_NUMNODES*4 > PAGE_SIZE/2);
+ /*
+ * buf is currently PAGE_SIZE in length and each node needs 4 chars
+ * at the most (distance + space or newline).
+ */
+ BUILD_BUG_ON(MAX_NUMNODES * 4 > PAGE_SIZE);
for_each_online_node(i)
len += sprintf(buf + len, "%s%d", i ? " " : "", node_distance(nid, i));
* enumeration tasks, they don't fully conform to the Linux driver model.
* In particular, when such drivers are built as modules, they can't be
* "hotplugged".
+ *
+ * Returns &struct platform_device pointer on success, or ERR_PTR() on error.
*/
struct platform_device *platform_device_register_simple(const char *name,
int id,
* allocated for the device allows drivers using such devices to be
* unloaded without waiting for the last reference to the device to be
* dropped.
+ *
+ * Returns &struct platform_device pointer on success, or ERR_PTR() on error.
*/
struct platform_device *platform_device_register_data(
struct device *parent,
*
* Use this in legacy-style modules that probe hardware directly and
* register a single platform device and corresponding platform driver.
+ *
+ * Returns &struct platform_device pointer on success, or ERR_PTR() on error.
*/
struct platform_device * __init_or_module platform_create_bundle(
struct platform_driver *driver,
static __initdata LIST_HEAD(early_platform_device_list);
/**
- * early_platform_driver_register
+ * early_platform_driver_register - register early platform driver
* @epdrv: early_platform driver structure
* @buf: string passed from early_param()
+ *
+ * Helper function for early_platform_init() / early_platform_init_buffer()
*/
int __init early_platform_driver_register(struct early_platform_driver *epdrv,
char *buf)
}
/**
- * early_platform_add_devices - add a numbers of early platform devices
+ * early_platform_add_devices - adds a number of early platform devices
* @devs: array of early platform devices to add
* @num: number of early platform devices in array
+ *
+ * Used by early architecture code to register early platform devices and
+ * their platform data.
*/
void __init early_platform_add_devices(struct platform_device **devs, int num)
{
}
/**
- * early_platform_driver_register_all
+ * early_platform_driver_register_all - register early platform drivers
* @class_str: string to identify early platform driver class
+ *
+ * Used by architecture code to register all early platform drivers
+ * for a certain class. If omitted then only early platform drivers
+ * with matching kernel command line class parameters will be registered.
*/
void __init early_platform_driver_register_all(char *class_str)
{
}
/**
- * early_platform_match
+ * early_platform_match - find early platform device matching driver
* @epdrv: early platform driver structure
* @id: id to match against
*/
}
/**
- * early_platform_left
+ * early_platform_left - check if early platform driver has matching devices
* @epdrv: early platform driver structure
* @id: return true if id or above exists
*/
}
/**
- * early_platform_driver_probe_id
+ * early_platform_driver_probe_id - probe drivers matching class_str and id
* @class_str: string to identify early platform driver class
* @id: id to match against
* @nr_probe: number of platform devices to successfully probe before exiting
}
/**
- * early_platform_driver_probe
+ * early_platform_driver_probe - probe a class of registered drivers
* @class_str: string to identify early platform driver class
* @nr_probe: number of platform devices to successfully probe before exiting
* @user_only: only probe user specified early platform devices
+ *
+ * Used by architecture code to probe registered early platform drivers
+ * within a certain class. For probe to happen a registered early platform
+ * device matching a registered early platform driver is needed.
*/
int __init early_platform_driver_probe(char *class_str,
int nr_probe,
if (dev->bus && dev->bus->pm) {
pm_dev_dbg(dev, state, "EARLY ");
error = pm_noirq_op(dev, dev->bus->pm, state);
+ if (error)
+ goto End;
}
+ if (dev->type && dev->type->pm) {
+ pm_dev_dbg(dev, state, "EARLY type ");
+ error = pm_noirq_op(dev, dev->type->pm, state);
+ if (error)
+ goto End;
+ }
+
+ if (dev->class && dev->class->pm) {
+ pm_dev_dbg(dev, state, "EARLY class ");
+ error = pm_noirq_op(dev, dev->class->pm, state);
+ }
+
+End:
TRACE_RESUME(error);
return error;
}
{
int error = 0;
+ if (dev->class && dev->class->pm) {
+ pm_dev_dbg(dev, state, "LATE class ");
+ error = pm_noirq_op(dev, dev->class->pm, state);
+ if (error)
+ goto End;
+ }
+
+ if (dev->type && dev->type->pm) {
+ pm_dev_dbg(dev, state, "LATE type ");
+ error = pm_noirq_op(dev, dev->type->pm, state);
+ if (error)
+ goto End;
+ }
+
if (dev->bus && dev->bus->pm) {
pm_dev_dbg(dev, state, "LATE ");
error = pm_noirq_op(dev, dev->bus->pm, state);
}
+
+End:
return error;
}
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/pm.h>
#include <linux/device.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/fd.h>
#include <linux/hdreg.h>
#include <linux/backing-dev.h>
#include <linux/fs.h>
#include <linux/ioctl.h>
+#include <linux/slab.h>
#include <linux/genhd.h>
#include <linux/netdevice.h>
#include "aoe.h"
#include <linux/blkdev.h>
#include <linux/completion.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/smp_lock.h>
#include <linux/skbuff.h>
#include "aoe.h"
*/
#include <linux/ata.h>
+#include <linux/slab.h>
#include <linux/hdreg.h>
#include <linux/blkdev.h>
#include <linux/skbuff.h>
#include <linux/blkdev.h>
#include <linux/netdevice.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include "aoe.h"
static void dummy_timer(ulong);
* Ethernet portion of AoE driver
*/
+#include <linux/gfp.h>
#include <linux/hdreg.h>
#include <linux/blkdev.h>
#include <linux/netdevice.h>
#include <linux/blkdev.h>
#include <linux/bio.h>
#include <linux/highmem.h>
-#include <linux/gfp.h>
#include <linux/radix-tree.h>
#include <linux/buffer_head.h> /* invalidate_bh_lrus() */
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <linux/vmalloc.h>
#include <linux/string.h>
#include <linux/drbd.h>
+#include <linux/slab.h>
#include <asm/kmap_types.h>
#include "drbd_int.h"
#include <asm/uaccess.h>
#include <linux/fs.h>
#include <linux/file.h>
-#include <linux/slab.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
#include <linux/drbd.h>
#include <linux/fs.h>
#include <linux/kernel.h>
#include <linux/genhd.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/ioport.h>
#include <linux/init.h>
#include <linux/buffer_head.h> /* for invalidate_bdev() */
#include <linux/completion.h>
#include <linux/highmem.h>
-#include <linux/gfp.h>
#include <linux/kthread.h>
#include <linux/splice.h>
#include <linux/platform_device.h>
#include <linux/gpio.h>
#include <linux/mg_disk.h>
+#include <linux/slab.h>
#define MG_RES_SEC (CONFIG_MG_DISK_RES << 1)
#include <linux/compiler.h>
#include <linux/err.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <linux/net.h>
#include <linux/kthread.h>
#include <linux/device.h>
#include <linux/module.h>
#include <linux/fs.h>
+#include <linux/slab.h>
#include <scsi/osd_initiator.h>
#include <scsi/osd_attributes.h>
#include <scsi/osd_sec.h>
#include <linux/init.h>
#include <linux/module.h>
+#include <linux/gfp.h>
#include <linux/fs.h>
#include <linux/delay.h>
#include <linux/hdreg.h>
#include <linux/miscdevice.h>
#include <linux/freezer.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <scsi/scsi_cmnd.h>
#include <scsi/scsi_ioctl.h>
#include <scsi/scsi.h>
#include <linux/ata.h>
#include <linux/blkdev.h>
+#include <linux/slab.h>
#include <asm/lv1call.h>
#include <asm/ps3stor.h>
#include <linux/delay.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include <asm/cell-regs.h>
#include <asm/firmware.h>
#include <linux/module.h>
#include <linux/fd.h>
+#include <linux/slab.h>
#include <linux/blkdev.h>
#include <linux/hdreg.h>
#include <linux/kernel.h>
#include <linux/blkdev.h>
#include <linux/timer.h>
#include <linux/scatterlist.h>
+#include <linux/slab.h>
#include <scsi/scsi.h>
#define DRV_NAME "ub"
#include <linux/kernel.h>
#include <linux/mm.h>
#include <linux/mman.h>
+#include <linux/gfp.h>
#include <linux/ioctl.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/timer.h>
#include <linux/pci.h>
-#include <linux/slab.h>
#include <linux/dma-mapping.h>
#include <linux/fcntl.h> /* O_ACCMODE */
//#define DEBUG
#include <linux/spinlock.h>
+#include <linux/slab.h>
#include <linux/blkdev.h>
#include <linux/hdreg.h>
#include <linux/virtio.h>
#include <linux/blkpg.h>
#include <linux/delay.h>
#include <linux/io.h>
+#include <linux/gfp.h>
#include <asm/system.h>
#include <asm/uaccess.h>
#include <linux/hdreg.h>
#include <linux/cdrom.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/scatterlist.h>
#include <xen/xen.h>
#include <linux/module.h>
#include <linux/blkdev.h>
#include <linux/bitops.h>
+#include <linux/slab.h>
#include <asm/setup.h>
#include <asm/amigahw.h>
**/
#include <linux/debugfs.h>
+#include <linux/slab.h>
#include <net/bluetooth/bluetooth.h>
#include <net/bluetooth/hci_core.h>
#include <linux/kthread.h>
#include <linux/bitops.h>
+#include <linux/slab.h>
#include <net/bluetooth/bluetooth.h>
#define BTM_HEADER_LEN 4
**/
#include <linux/firmware.h>
+#include <linux/slab.h>
#include <linux/mmc/sdio_ids.h>
#include <linux/mmc/sdio_func.h>
#include <linux/pci.h>
#include <linux/init.h>
#include <linux/agp_backend.h>
-#include <linux/gfp.h>
#include <linux/page-flags.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include "agp.h"
#define AMD_MMBASE 0x14
#include <linux/module.h>
#include <linux/pci.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/pagemap.h>
#include <linux/miscdevice.h>
#include <linux/pm.h>
#include <linux/pci.h>
#include <linux/fs.h>
#include <linux/agpgart.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include "agp.h"
#include "compat_ioctl.h"
#include <linux/dma-mapping.h>
#include <linux/mm.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/cacheflush.h>
#include <asm/pgtable.h>
#include <linux/init.h>
#include <linux/agp_backend.h>
#include <linux/log2.h>
+#include <linux/slab.h>
#include <asm/acpi-ext.h>
#include <linux/module.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/pagemap.h>
#define IS_PINEVIEW (agp_bridge->dev->device == PCI_DEVICE_ID_INTEL_PINEVIEW_M_HB || \
agp_bridge->dev->device == PCI_DEVICE_ID_INTEL_PINEVIEW_HB)
+#define IS_SNB (agp_bridge->dev->device == PCI_DEVICE_ID_INTEL_SANDYBRIDGE_HB || \
+ agp_bridge->dev->device == PCI_DEVICE_ID_INTEL_SANDYBRIDGE_M_HB)
+
#define IS_G4X (agp_bridge->dev->device == PCI_DEVICE_ID_INTEL_EAGLELAKE_HB || \
agp_bridge->dev->device == PCI_DEVICE_ID_INTEL_Q45_HB || \
agp_bridge->dev->device == PCI_DEVICE_ID_INTEL_G45_HB || \
agp_bridge->dev->device == PCI_DEVICE_ID_INTEL_IRONLAKE_M_HB || \
agp_bridge->dev->device == PCI_DEVICE_ID_INTEL_IRONLAKE_MA_HB || \
agp_bridge->dev->device == PCI_DEVICE_ID_INTEL_IRONLAKE_MC2_HB || \
- agp_bridge->dev->device == PCI_DEVICE_ID_INTEL_SANDYBRIDGE_HB || \
- agp_bridge->dev->device == PCI_DEVICE_ID_INTEL_SANDYBRIDGE_M_HB)
+ IS_SNB)
extern int agp_memory_reserved;
#define SNB_GMCH_GMS_STOLEN_448M (0xe << 3)
#define SNB_GMCH_GMS_STOLEN_480M (0xf << 3)
#define SNB_GMCH_GMS_STOLEN_512M (0x10 << 3)
+#define SNB_GTT_SIZE_0M (0 << 8)
+#define SNB_GTT_SIZE_1M (1 << 8)
+#define SNB_GTT_SIZE_2M (2 << 8)
+#define SNB_GTT_SIZE_MASK (3 << 8)
static const struct aper_size_info_fixed intel_i810_sizes[] =
{
if (intel_private.ifp_resource.start)
return;
+ if (IS_SNB)
+ return;
+
/* setup a resource for this object */
intel_private.ifp_resource.name = "Intel Flush Page";
intel_private.ifp_resource.flags = IORESOURCE_MEM;
static void intel_i965_get_gtt_range(int *gtt_offset, int *gtt_size)
{
+ u16 snb_gmch_ctl;
+
switch (agp_bridge->dev->device) {
case PCI_DEVICE_ID_INTEL_GM45_HB:
case PCI_DEVICE_ID_INTEL_EAGLELAKE_HB:
case PCI_DEVICE_ID_INTEL_IRONLAKE_M_HB:
case PCI_DEVICE_ID_INTEL_IRONLAKE_MA_HB:
case PCI_DEVICE_ID_INTEL_IRONLAKE_MC2_HB:
+ *gtt_offset = *gtt_size = MB(2);
+ break;
case PCI_DEVICE_ID_INTEL_SANDYBRIDGE_HB:
case PCI_DEVICE_ID_INTEL_SANDYBRIDGE_M_HB:
- *gtt_offset = *gtt_size = MB(2);
+ *gtt_offset = MB(2);
+
+ pci_read_config_word(intel_private.pcidev, SNB_GMCH_CTRL, &snb_gmch_ctl);
+ switch (snb_gmch_ctl & SNB_GTT_SIZE_MASK) {
+ default:
+ case SNB_GTT_SIZE_0M:
+ printk(KERN_ERR "Bad GTT size mask: 0x%04x.\n", snb_gmch_ctl);
+ *gtt_size = MB(0);
+ break;
+ case SNB_GTT_SIZE_1M:
+ *gtt_size = MB(1);
+ break;
+ case SNB_GTT_SIZE_2M:
+ *gtt_size = MB(2);
+ break;
+ }
break;
default:
*gtt_offset = *gtt_size = KB(512);
#include <linux/pci.h>
#include <linux/init.h>
#include <linux/agp_backend.h>
-#include <linux/gfp.h>
#include <linux/page-flags.h>
#include <linux/mm.h>
#include <linux/jiffies.h>
#include <linux/acpi.h>
#include <linux/module.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/agp_backend.h>
#include <asm/sn/addrs.h>
*/
#include <linux/module.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/pagemap.h>
#include <linux/agp_backend.h>
#include <linux/module.h>
#include <linux/mutex.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/tty.h>
#include <linux/tty_driver.h>
#include <linux/tty_flip.h>
#include <linux/kernel.h>
#include <linux/wait.h>
#include <linux/string.h>
-#include <linux/slab.h>
#include <linux/ioport.h>
#include <linux/delay.h>
#include <linux/miscdevice.h>
#include <linux/cdev.h>
#include <linux/list.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include <asm/pgtable.h>
#include <asm/io.h>
#include <linux/bitops.h>
#include <linux/firmware.h>
#include <linux/device.h>
+#include <linux/slab.h>
#include <linux/io.h>
#include <linux/uaccess.h>
*/
#include <linux/module.h>
-#include <linux/slab.h> /* for kmalloc() and kfree() */
#include <linux/major.h>
#include <linux/types.h>
#include <linux/errno.h>
#include <linux/ctype.h>
#include <linux/tty.h>
#include <linux/tty_flip.h>
-#include <linux/slab.h>
#include <linux/smp_lock.h>
#include <linux/ioport.h>
#include <linux/interrupt.h>
#include <linux/interrupt.h>
#include <linux/tty_flip.h>
#include <linux/delay.h>
+#include <linux/gfp.h>
#include <asm/uaccess.h>
#define DEBUG
#include <linux/seq_file.h>
#include <linux/bitops.h>
#include <linux/clocksource.h>
+#include <linux/slab.h>
#include <asm/current.h>
#include <asm/uaccess.h>
#include <linux/spinlock.h>
#include <linux/delay.h>
#include <linux/freezer.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
spin_lock_irqsave(&hp->lock, flags);
/* Check and then increment for fast path open. */
if (hp->count++ > 0) {
+ tty_kref_get(tty);
spin_unlock_irqrestore(&hp->lock, flags);
hvc_kick();
return 0;
tty->driver_data = hp;
- hp->tty = tty;
+ hp->tty = tty_kref_get(tty);
spin_unlock_irqrestore(&hp->lock, flags);
spin_lock_irqsave(&hp->lock, flags);
hp->tty = NULL;
spin_unlock_irqrestore(&hp->lock, flags);
+ tty_kref_put(tty);
tty->driver_data = NULL;
kref_put(&hp->kref, destroy_hvc_struct);
printk(KERN_ERR "hvc_open: request_irq failed with rc %d.\n", rc);
return;
hp = tty->driver_data;
+
spin_lock_irqsave(&hp->lock, flags);
+ tty_kref_get(tty);
if (--hp->count == 0) {
/* We are done with the tty pointer now. */
hp->tty = NULL;
spin_unlock_irqrestore(&hp->lock, flags);
+ /* Put the ref obtained in hvc_open() */
+ tty_kref_put(tty);
+
if (hp->ops->notifier_del)
hp->ops->notifier_del(hp, hp->data);
spin_unlock_irqrestore(&hp->lock, flags);
}
+ tty_kref_put(tty);
kref_put(&hp->kref, destroy_hvc_struct);
}
spin_unlock_irqrestore(&hp->lock, flags);
if (hp->ops->notifier_hangup)
- hp->ops->notifier_hangup(hp, hp->data);
+ hp->ops->notifier_hangup(hp, hp->data);
while(temp_open_count) {
--temp_open_count;
+ tty_kref_put(tty);
kref_put(&hp->kref, destroy_hvc_struct);
}
}
}
/* No tty attached, just skip */
- tty = hp->tty;
+ tty = tty_kref_get(hp->tty);
if (tty == NULL)
goto bail;
tty_flip_buffer_push(tty);
}
+ if (tty)
+ tty_kref_put(tty);
return poll_mask;
}
struct tty_struct *tty;
spin_lock_irqsave(&hp->lock, flags);
- tty = hp->tty;
+ tty = tty_kref_get(hp->tty);
if (hp->index < MAX_NR_HVC_CONSOLES)
vtermnos[hp->index] = -1;
/*
* We 'put' the instance that was grabbed when the kref instance
* was initialized using kref_init(). Let the last holder of this
- * kref cause it to be removed, which will probably be the tty_hangup
+ * kref cause it to be removed, which will probably be the tty_vhangup
* below.
*/
kref_put(&hp->kref, destroy_hvc_struct);
/*
- * This function call will auto chain call hvc_hangup. The tty should
- * always be valid at this time unless a simultaneous tty close already
- * cleaned up the hvc_struct.
+ * This function call will auto chain call hvc_hangup.
*/
- if (tty)
- tty_hangup(tty);
+ if (tty) {
+ tty_vhangup(tty);
+ tty_kref_put(tty);
+ }
return 0;
}
EXPORT_SYMBOL_GPL(hvc_remove);
#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
#include <linux/types.h>
+#include <linux/slab.h>
#include <asm/ebcdic.h>
#include <linux/ctype.h>
#include <linux/delay.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/stat.h>
#include <linux/tty.h>
#include <linux/pci.h>
#include <linux/stop_machine.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <linux/device.h>
#include <linux/hw_random.h>
#include <linux/io.h>
+#include <linux/gfp.h>
#include <asm/octeon/octeon.h>
#include <asm/octeon/cvmx-rnm-defs.h>
#include <linux/io.h>
#include <linux/platform_device.h>
#include <linux/hw_random.h>
+#include <linux/gfp.h>
#define TX4939_RNG_RCSR 0x00000000
#define TX4939_RNG_ROR(n) (0x00000018 + (n) * 8)
bmc->device_id_attr.attr.name = "device_id";
bmc->device_id_attr.attr.mode = S_IRUGO;
bmc->device_id_attr.show = device_id_show;
+ sysfs_attr_init(&bmc->device_id_attr.attr);
bmc->provides_dev_sdrs_attr.attr.name = "provides_device_sdrs";
bmc->provides_dev_sdrs_attr.attr.mode = S_IRUGO;
bmc->provides_dev_sdrs_attr.show = provides_dev_sdrs_show;
+ sysfs_attr_init(&bmc->provides_dev_sdrs_attr.attr);
bmc->revision_attr.attr.name = "revision";
bmc->revision_attr.attr.mode = S_IRUGO;
bmc->revision_attr.show = revision_show;
+ sysfs_attr_init(&bmc->revision_attr.attr);
bmc->firmware_rev_attr.attr.name = "firmware_revision";
bmc->firmware_rev_attr.attr.mode = S_IRUGO;
bmc->firmware_rev_attr.show = firmware_rev_show;
+ sysfs_attr_init(&bmc->firmware_rev_attr.attr);
bmc->version_attr.attr.name = "ipmi_version";
bmc->version_attr.attr.mode = S_IRUGO;
bmc->version_attr.show = ipmi_version_show;
+ sysfs_attr_init(&bmc->version_attr.attr);
bmc->add_dev_support_attr.attr.name = "additional_device_support";
bmc->add_dev_support_attr.attr.mode = S_IRUGO;
bmc->add_dev_support_attr.show = add_dev_support_show;
+ sysfs_attr_init(&bmc->add_dev_support_attr.attr);
bmc->manufacturer_id_attr.attr.name = "manufacturer_id";
bmc->manufacturer_id_attr.attr.mode = S_IRUGO;
bmc->manufacturer_id_attr.show = manufacturer_id_show;
+ sysfs_attr_init(&bmc->manufacturer_id_attr.attr);
bmc->product_id_attr.attr.name = "product_id";
bmc->product_id_attr.attr.mode = S_IRUGO;
bmc->product_id_attr.show = product_id_show;
+ sysfs_attr_init(&bmc->product_id_attr.attr);
bmc->guid_attr.attr.name = "guid";
bmc->guid_attr.attr.mode = S_IRUGO;
bmc->guid_attr.show = guid_show;
+ sysfs_attr_init(&bmc->guid_attr.attr);
bmc->aux_firmware_rev_attr.attr.name = "aux_firmware_revision";
bmc->aux_firmware_rev_attr.attr.mode = S_IRUGO;
bmc->aux_firmware_rev_attr.show = aux_firmware_rev_show;
+ sysfs_attr_init(&bmc->aux_firmware_rev_attr.attr);
err = device_create_file(&bmc->dev->dev,
&bmc->device_id_attr);
#include <linux/timer.h>
#include <linux/delay.h>
#include <linux/ioport.h>
+#include <linux/slab.h>
#include <linux/uaccess.h>
#include <linux/io.h>
#include <linux/uio.h>
#include <linux/mutex.h>
#include <linux/smp_lock.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/uaccess.h>
#include <asm/system.h>
#include <linux/miscdevice.h>
#include <linux/kernel.h>
#include <linux/major.h>
-#include <linux/slab.h>
#include <linux/mutex.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
#include <linux/device.h>
#include <linux/tty.h>
#include <linux/kmod.h>
+#include <linux/gfp.h>
/*
* Head entry for the doubly linked miscdevice list
#include <linux/time.h>
#include <linux/math64.h>
#include <linux/smp_lock.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/sn/addrs.h>
#include <linux/pci.h>
#include <linux/init.h>
#include <linux/bitops.h>
+#include <linux/slab.h>
#include <asm/system.h>
#include <asm/io.h>
#include <linux/string.h>
#include <linux/fcntl.h>
#include <linux/ptrace.h>
-#include <linux/gfp.h>
#include <linux/ioport.h>
#include <linux/mm.h>
#include <linux/delay.h>
#include <linux/pci.h>
#include <linux/bitops.h>
+#include <linux/slab.h>
#include <asm/system.h>
#include <asm/io.h>
#include <linux/init.h>
#include <linux/kfifo.h>
#include <linux/uaccess.h>
+#include <linux/slab.h>
#include <asm/byteorder.h>
#include <linux/delay.h>
#include <linux/types.h>
#include <linux/errno.h>
#include <linux/miscdevice.h>
-#include <linux/slab.h>
#include <linux/ioport.h>
#include <linux/fcntl.h>
#include <linux/mc146818rtc.h>
#include <linux/netdevice.h>
#include <linux/ppp_channel.h>
#include <linux/ppp_defs.h>
+#include <linux/slab.h>
#include <linux/if_ppp.h>
#include <linux/skbuff.h>
#include <linux/parport.h>
#include <linux/ctype.h>
#include <linux/poll.h>
+#include <linux/slab.h>
#include <linux/major.h>
#include <linux/ppdev.h>
#include <linux/smp_lock.h>
#include <linux/fs.h>
#include <linux/miscdevice.h>
+#include <linux/slab.h>
#include <linux/uaccess.h>
#include <asm/lv1call.h>
#include <linux/uaccess.h>
#include <linux/bitops.h>
#include <linux/devpts_fs.h>
+#include <linux/slab.h>
#include <asm/system.h>
#include <linux/device.h>
#include <linux/mutex.h>
#include <linux/smp_lock.h>
+#include <linux/gfp.h>
#include <asm/uaccess.h>
*/
#include <linux/module.h>
-#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/delay.h>
#include <asm/io.h>
*/
#include <linux/module.h>
-#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/tty.h>
#include <linux/tty_flip.h>
*/
#include <linux/module.h>
-#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/tty.h>
#include <asm/io.h>
*/
#include <linux/module.h>
-#include <linux/slab.h>
#include <linux/errno.h>
#include <asm/io.h>
#include <asm/system.h>
#include <linux/module.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/tty.h>
#include <linux/string.h>
#include <linux/module.h>
#include <linux/bitops.h>
#include <linux/tty_flip.h>
+#include <linux/gfp.h>
#include <asm/system.h>
#include <asm/io.h>
#include <linux/interrupt.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <asm/byteorder.h>
#include <asm/sn/sn_sal.h>
#include <asm/unaligned.h>
#include <linux/err.h>
#include <linux/kfifo.h>
#include <linux/platform_device.h>
+#include <linux/gfp.h>
#include <asm/uaccess.h>
#include <asm/io.h>
#include <linux/pci.h>
#include <linux/init.h>
#include <linux/uaccess.h>
+#include <linux/gfp.h>
#include "specialix_io8.h"
#include "cd1865.h"
#include <linux/workqueue.h>
#include <linux/hrtimer.h>
#include <linux/oom.h>
+#include <linux/slab.h>
#include <asm/ptrace.h>
#include <asm/irq_regs.h>
*/
#include <linux/poll.h>
+#include <linux/slab.h>
#include <linux/mutex.h>
#include <linux/spinlock.h>
#include <linux/fs.h>
#include <linux/security.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <acpi/acpi.h>
#include "tpm.h"
*/
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include "tpm.h"
/* National definitions */
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/pnp.h>
+#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/wait.h>
#include "tpm.h"
*/
#include <linux/audit.h>
+#include <linux/slab.h>
#include <linux/tty.h>
struct tty_audit_buf {
{
int copied = 0;
do {
- int goal = min(size - copied, TTY_BUFFER_PAGE);
+ int goal = min_t(size_t, size - copied, TTY_BUFFER_PAGE);
int space = tty_buffer_request_room(tty, goal);
struct tty_buffer *tb = tty->buf.tail;
/* If there is no space then tb may be NULL */
{
int copied = 0;
do {
- int goal = min(size - copied, TTY_BUFFER_PAGE);
+ int goal = min_t(size_t, size - copied, TTY_BUFFER_PAGE);
int space = tty_buffer_request_room(tty, goal);
struct tty_buffer *tb = tty->buf.tail;
/* If there is no space then tb may be NULL */
list_del_init(&tty->tty_files);
file_list_unlock();
+ put_pid(tty->pgrp);
+ put_pid(tty->session);
free_tty_struct(tty);
}
static void tty_port_shutdown(struct tty_port *port)
{
mutex_lock(&port->mutex);
- if (port->ops->shutdown &&
+ if (port->ops->shutdown && !port->console &&
test_and_clear_bit(ASYNCB_INITIALIZED, &port->flags))
port->ops->shutdown(port);
mutex_unlock(&port->mutex);
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
#include <linux/smp_lock.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/ioctls.h>
#include <linux/list.h>
#include <linux/poll.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/virtio.h>
#include <linux/virtio_console.h>
struct virtio_device *vdev;
struct winsize ws;
+ /* The port could have been hot-unplugged */
+ if (!port)
+ return;
+
vdev = port->portdev->vdev;
if (virtio_has_feature(vdev, VIRTIO_CONSOLE_F_SIZE)) {
vdev->config->get(vdev,
*/
err = sysfs_create_group(&port->dev->kobj,
&port_attribute_group);
- if (err)
+ if (err) {
dev_err(port->dev,
"Error %d creating sysfs device attributes\n",
err);
-
+ } else {
+ /*
+ * Generate a udev event so that appropriate
+ * symlinks can be created based on udev
+ * rules.
+ */
+ kobject_uevent(&port->dev->kobj, KOBJ_CHANGE);
+ }
break;
case VIRTIO_CONSOLE_PORT_REMOVE:
/*
#include <linux/fcntl.h>
#include <linux/major.h>
#include <linux/delay.h>
-#include <linux/slab.h>
#include <linux/miscdevice.h>
#include <linux/console.h>
#include <linux/init.h>
ret = -EFAULT;
goto out;
}
- if (tmp.mode != VT_AUTO && tmp.mode != VT_PROCESS && tmp.mode != VT_PROCESS_AUTO) {
+ if (tmp.mode != VT_AUTO && tmp.mode != VT_PROCESS) {
ret = -EINVAL;
goto out;
}
* telling it that it has acquired. Also check if it has died and
* clean up (similar to logic employed in change_console())
*/
- if (vc->vt_mode.mode == VT_PROCESS || vc->vt_mode.mode == VT_PROCESS_AUTO) {
+ if (vc->vt_mode.mode == VT_PROCESS) {
/*
* Send the signal as privileged - kill_pid() will
* tell us if the process has gone or something else
* vt to auto control.
*/
vc = vc_cons[fg_console].d;
- if (vc->vt_mode.mode == VT_PROCESS || vc->vt_mode.mode == VT_PROCESS_AUTO) {
+ if (vc->vt_mode.mode == VT_PROCESS) {
/*
* Send the signal as privileged - kill_pid() will
* tell us if the process has gone or something else
*/
vc->vt_newvt = new_vc->vc_num;
if (kill_pid(vc->vt_pid, vc->vt_mode.relsig, 1) == 0) {
- if(vc->vt_mode.mode == VT_PROCESS)
- /*
- * It worked. Mark the vt to switch to and
- * return. The process needs to send us a
- * VT_RELDISP ioctl to complete the switch.
- */
- return;
- } else {
/*
- * The controlling process has died, so we revert back to
- * normal operation. In this case, we'll also change back
- * to KD_TEXT mode. I'm not sure if this is strictly correct
- * but it saves the agony when the X server dies and the screen
- * remains blanked due to KD_GRAPHICS! It would be nice to do
- * this outside of VT_PROCESS but there is no single process
- * to account for and tracking tty count may be undesirable.
+ * It worked. Mark the vt to switch to and
+ * return. The process needs to send us a
+ * VT_RELDISP ioctl to complete the switch.
*/
- reset_vc(vc);
+ return;
}
/*
- * Fall through to normal (VT_AUTO and VT_PROCESS_AUTO) handling of the switch...
+ * The controlling process has died, so we revert back to
+ * normal operation. In this case, we'll also change back
+ * to KD_TEXT mode. I'm not sure if this is strictly correct
+ * but it saves the agony when the X server dies and the screen
+ * remains blanked due to KD_GRAPHICS! It would be nice to do
+ * this outside of VT_PROCESS but there is no single process
+ * to account for and tracking tty count may be undesirable.
+ */
+ reset_vc(vc);
+
+ /*
+ * Fall through to normal (VT_AUTO) handling of the switch...
*/
}
#include <linux/fs.h>
#include <linux/cdev.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/uaccess.h>
#include <linux/clocksource.h>
#include <linux/clockchips.h>
#include <linux/sh_timer.h>
+#include <linux/slab.h>
struct sh_cmt_priv {
void __iomem *mapbase;
#include <linux/err.h>
#include <linux/clockchips.h>
#include <linux/sh_timer.h>
+#include <linux/slab.h>
struct sh_mtu2_priv {
void __iomem *mapbase;
#include <linux/clocksource.h>
#include <linux/clockchips.h>
#include <linux/sh_timer.h>
+#include <linux/slab.h>
struct sh_tmu_priv {
void __iomem *mapbase;
#include <linux/ktime.h>
#include <linux/init.h>
#include <linux/connector.h>
+#include <linux/gfp.h>
#include <asm/atomic.h>
#include <asm/unaligned.h>
#include <linux/netlink.h>
#include <linux/moduleparam.h>
#include <linux/connector.h>
+#include <linux/slab.h>
#include <linux/mutex.h>
#include <linux/proc_fs.h>
#include <linux/spinlock.h>
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/sysdev.h>
#include <linux/cpu.h>
#include <linux/sysfs.h>
#include <linux/kernel.h>
#include <linux/cpuidle.h>
#include <linux/sysfs.h>
+#include <linux/slab.h>
#include <linux/cpu.h>
#include "cpuidle.h"
#include <linux/platform_device.h>
#include <linux/init.h>
#include <linux/of_platform.h>
+#include <linux/slab.h>
#include <asm/dcr.h>
#include <asm/dcr-regs.h>
#include <asm/cacheflush.h>
#include <linux/rtnetlink.h>
#include <linux/interrupt.h>
#include <linux/spinlock.h>
+#include <linux/gfp.h>
#include <crypto/ctr.h>
#include <crypto/des.h>
#include <linux/kthread.h>
#include <linux/platform_device.h>
#include <linux/scatterlist.h>
+#include <linux/slab.h>
#include "mv_cesa.h"
/*
#include <linux/kernel.h>
#include <linux/percpu.h>
#include <linux/smp.h>
+#include <linux/slab.h>
#include <asm/byteorder.h>
#include <asm/processor.h>
#include <asm/i387.h>
#include <linux/io.h>
#include <linux/spinlock.h>
#include <linux/rtnetlink.h>
+#include <linux/slab.h>
#include <crypto/algapi.h>
#include <crypto/aes.h>
#include <linux/notifier.h>
#include <linux/device.h>
#include <linux/dca.h>
+#include <linux/slab.h>
#define DCA_VERSION "1.12.1"
#include <linux/kdev_t.h>
#include <linux/err.h>
#include <linux/dca.h>
+#include <linux/gfp.h>
static struct class *dca_class;
static struct idr dca_idr;
#include <linux/interrupt.h>
#include <linux/module.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include "at_hdmac_regs.h"
#include <linux/spinlock.h>
#include <linux/dmapool.h>
#include <linux/memory.h>
+#include <linux/gfp.h>
#include <mach/coh901318.h>
#include "coh901318_lli.h"
#include <linux/jiffies.h>
#include <linux/rculist.h>
#include <linux/idr.h>
+#include <linux/slab.h>
static DEFINE_MUTEX(dma_list_mutex);
static LIST_HEAD(dma_device_list);
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/random.h>
+#include <linux/slab.h>
#include <linux/wait.h>
static unsigned int test_buf_size = 16384;
#include <linux/init.h>
#include <linux/module.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/dmaengine.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/interrupt.h>
#include <linux/dmaengine.h>
#include <linux/init.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/interrupt.h>
#include <linux/dmaengine.h>
*/
#include <linux/pci.h>
+#include <linux/gfp.h>
#include <linux/dmaengine.h>
#include <linux/dma-mapping.h>
#include "registers.h"
#include <linux/pci.h>
#include <linux/interrupt.h>
#include <linux/dca.h>
+#include <linux/slab.h>
#include "dma.h"
#include "dma_v2.h"
#include "registers.h"
#include <linux/memory.h>
#include <linux/ioport.h>
#include <linux/raid/pq.h>
+#include <linux/slab.h>
#include <mach/adma.h>
#include <linux/dmaengine.h>
#include <linux/pagemap.h>
+#include <linux/slab.h>
#include <net/tcp.h> /* for memcpy_toiovec */
#include <asm/io.h>
#include <asm/uaccess.h>
#include <linux/dma-mapping.h>
#include <linux/interrupt.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <linux/of_device.h>
#include <linux/of_platform.h>
#include <linux/init.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/dma-mapping.h>
#include <linux/spinlock.h>
#include <linux/dma-mapping.h>
#include <linux/spinlock.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <linux/uaccess.h>
#include <linux/proc_fs.h>
#include <linux/of.h>
#include <linux/init.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/dmaengine.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/pci.h>
#include <linux/pci_ids.h>
-#include <linux/slab.h>
#include <linux/edac.h>
#include "edac_core.h"
#include <linux/edac.h>
#include <linux/of.h>
#include <linux/platform_device.h>
+#include <linux/gfp.h>
#include "edac_core.h"
#include "edac_module.h"
#include <linux/init.h>
#include <linux/pci.h>
#include <linux/pci_ids.h>
-#include <linux/slab.h>
#include <linux/edac.h>
#include "edac_core.h"
#include <linux/init.h>
#include <linux/pci.h>
#include <linux/pci_ids.h>
-#include <linux/slab.h>
#include <linux/edac.h>
#include "edac_core.h"
#include <linux/ctype.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include "edac_core.h"
#include "edac_module.h"
*/
#include <linux/ctype.h>
+#include <linux/slab.h>
#include <linux/bug.h>
#include "edac_core.h"
if (regs->nbsh & K8_NBSH_ERR_CPU_VAL)
pr_cont(", core: %u\n", (u8)(regs->nbsh & 0xf));
} else {
- pr_cont(", core: %d\n", fls((regs->nbsh & 0xf) - 1));
+ u8 assoc_cpus = regs->nbsh & 0xf;
+
+ if (assoc_cpus > 0)
+ pr_cont(", core: %d", fls(assoc_cpus) - 1);
+
+ pr_cont("\n");
}
pr_emerg("%s.\n", EXT_ERR_MSG(xec));
*/
#include <linux/module.h>
#include <linux/sysdev.h>
+#include <linux/slab.h>
#include <linux/ctype.h>
#include "edac_core.h"
#include <linux/init.h>
#include <linux/pci.h>
#include <linux/pci_ids.h>
-#include <linux/slab.h>
#include <linux/edac.h>
#include "edac_core.h"
#include <linux/init.h>
#include <linux/pci.h>
#include <linux/pci_ids.h>
-#include <linux/slab.h>
#include <linux/edac.h>
#include <linux/io.h>
#include "edac_core.h"
#include <linux/init.h>
#include <linux/pci.h>
#include <linux/pci_ids.h>
-#include <linux/slab.h>
#include <linux/edac.h>
#include <linux/delay.h>
#include <linux/mmzone.h>
#include <linux/pci.h>
#include <linux/pci_ids.h>
-#include <linux/slab.h>
#include <linux/edac.h>
#include "edac_core.h"
#include <linux/init.h>
#include <linux/pci.h>
#include <linux/pci_ids.h>
-#include <linux/slab.h>
#include <linux/edac.h>
#include "edac_core.h"
#include <linux/init.h>
#include <linux/pci.h>
#include <linux/pci_ids.h>
-#include <linux/slab.h>
#include <linux/edac.h>
#include "edac_core.h"
#include <linux/init.h>
#include <linux/pci.h>
#include <linux/pci_ids.h>
-#include <linux/slab.h>
#include <linux/edac.h>
#include "edac_core.h"
*/
#include <linux/module.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/ctype.h>
#include <linux/io.h>
#include <linux/mod_devicetable.h>
#include <linux/edac.h>
#include <linux/smp.h>
+#include <linux/gfp.h>
#include <linux/of_platform.h>
#include <linux/of_device.h>
#include <linux/module.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/io.h>
#include <linux/edac.h>
+#include <linux/gfp.h>
#include "edac_core.h"
#include "edac_module.h"
#include <linux/init.h>
#include <linux/pci.h>
#include <linux/pci_ids.h>
-#include <linux/slab.h>
#include <linux/edac.h>
#include "edac_core.h"
#include <linux/init.h>
#include <linux/pci.h>
#include <linux/pci_ids.h>
-#include <linux/slab.h>
#include <linux/edac.h>
#include "edac_core.h"
#include <linux/init.h>
#include <linux/pci.h>
#include <linux/pci_ids.h>
-#include <linux/slab.h>
#include <linux/edac.h>
#include "edac_core.h"
#include <linux/mutex.h>
#include <linux/poll.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/string.h>
#include <linux/time.h>
#include <linux/module.h>
#include <linux/mutex.h>
#include <linux/rwsem.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/string.h>
#include <linux/workqueue.h>
}
EXPORT_SYMBOL(fw_csr_string);
-static bool is_fw_unit(struct device *dev);
-
-static int match_unit_directory(const u32 *directory, u32 match_flags,
- const struct ieee1394_device_id *id)
+static void get_ids(const u32 *directory, int *id)
{
struct fw_csr_iterator ci;
- int key, value, match;
+ int key, value;
- match = 0;
fw_csr_iterator_init(&ci, directory);
while (fw_csr_iterator_next(&ci, &key, &value)) {
- if (key == CSR_VENDOR && value == id->vendor_id)
- match |= IEEE1394_MATCH_VENDOR_ID;
- if (key == CSR_MODEL && value == id->model_id)
- match |= IEEE1394_MATCH_MODEL_ID;
- if (key == CSR_SPECIFIER_ID && value == id->specifier_id)
- match |= IEEE1394_MATCH_SPECIFIER_ID;
- if (key == CSR_VERSION && value == id->version)
- match |= IEEE1394_MATCH_VERSION;
+ switch (key) {
+ case CSR_VENDOR: id[0] = value; break;
+ case CSR_MODEL: id[1] = value; break;
+ case CSR_SPECIFIER_ID: id[2] = value; break;
+ case CSR_VERSION: id[3] = value; break;
+ }
}
+}
+
+static void get_modalias_ids(struct fw_unit *unit, int *id)
+{
+ get_ids(&fw_parent_device(unit)->config_rom[5], id);
+ get_ids(unit->directory, id);
+}
+
+static bool match_ids(const struct ieee1394_device_id *id_table, int *id)
+{
+ int match = 0;
+
+ if (id[0] == id_table->vendor_id)
+ match |= IEEE1394_MATCH_VENDOR_ID;
+ if (id[1] == id_table->model_id)
+ match |= IEEE1394_MATCH_MODEL_ID;
+ if (id[2] == id_table->specifier_id)
+ match |= IEEE1394_MATCH_SPECIFIER_ID;
+ if (id[3] == id_table->version)
+ match |= IEEE1394_MATCH_VERSION;
- return (match & match_flags) == match_flags;
+ return (match & id_table->match_flags) == id_table->match_flags;
}
+static bool is_fw_unit(struct device *dev);
+
static int fw_unit_match(struct device *dev, struct device_driver *drv)
{
- struct fw_unit *unit = fw_unit(dev);
- struct fw_device *device;
- const struct ieee1394_device_id *id;
+ const struct ieee1394_device_id *id_table =
+ container_of(drv, struct fw_driver, driver)->id_table;
+ int id[] = {0, 0, 0, 0};
/* We only allow binding to fw_units. */
if (!is_fw_unit(dev))
return 0;
- device = fw_parent_device(unit);
- id = container_of(drv, struct fw_driver, driver)->id_table;
+ get_modalias_ids(fw_unit(dev), id);
- for (; id->match_flags != 0; id++) {
- if (match_unit_directory(unit->directory, id->match_flags, id))
+ for (; id_table->match_flags != 0; id_table++)
+ if (match_ids(id_table, id))
return 1;
- /* Also check vendor ID in the root directory. */
- if ((id->match_flags & IEEE1394_MATCH_VENDOR_ID) &&
- match_unit_directory(&device->config_rom[5],
- IEEE1394_MATCH_VENDOR_ID, id) &&
- match_unit_directory(unit->directory, id->match_flags
- & ~IEEE1394_MATCH_VENDOR_ID, id))
- return 1;
- }
-
return 0;
}
static int get_modalias(struct fw_unit *unit, char *buffer, size_t buffer_size)
{
- struct fw_device *device = fw_parent_device(unit);
- struct fw_csr_iterator ci;
+ int id[] = {0, 0, 0, 0};
- int key, value;
- int vendor = 0;
- int model = 0;
- int specifier_id = 0;
- int version = 0;
-
- fw_csr_iterator_init(&ci, &device->config_rom[5]);
- while (fw_csr_iterator_next(&ci, &key, &value)) {
- switch (key) {
- case CSR_VENDOR:
- vendor = value;
- break;
- case CSR_MODEL:
- model = value;
- break;
- }
- }
-
- fw_csr_iterator_init(&ci, unit->directory);
- while (fw_csr_iterator_next(&ci, &key, &value)) {
- switch (key) {
- case CSR_SPECIFIER_ID:
- specifier_id = value;
- break;
- case CSR_VERSION:
- version = value;
- break;
- }
- }
+ get_modalias_ids(unit, id);
return snprintf(buffer, buffer_size,
"ieee1394:ven%08Xmo%08Xsp%08Xver%08X",
- vendor, model, specifier_id, version);
+ id[0], id[1], id[2], id[3]);
}
static int fw_unit_uevent(struct device *dev, struct kobj_uevent_env *env)
#include <linux/firewire-constants.h>
#include <linux/kernel.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/vmalloc.h>
if (ret < 0)
*bandwidth = 0;
- if (allocate && ret < 0 && c >= 0) {
- deallocate_channel(card, irm_id, generation, c, buffer);
+ if (allocate && ret < 0) {
+ if (c >= 0)
+ deallocate_channel(card, irm_id, generation, c, buffer);
*channel = ret;
}
}
#include <linux/mutex.h>
#include <linux/netdevice.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <asm/unaligned.h>
#include <linux/dma-mapping.h>
#include <linux/firewire.h>
#include <linux/firewire-constants.h>
-#include <linux/gfp.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/io.h>
#include <linux/moduleparam.h>
#include <linux/pci.h>
#include <linux/pci_ids.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/string.h>
static char ohci_driver_name[] = KBUILD_MODNAME;
+#define PCI_DEVICE_ID_TI_TSB12LV22 0x8009
+
#define QUIRK_CYCLE_TIMER 1
#define QUIRK_RESET_PACKET 2
#define QUIRK_BE_HEADERS 4
static const struct {
unsigned short vendor, device, flags;
} ohci_quirks[] = {
+ {PCI_VENDOR_ID_TI, PCI_DEVICE_ID_TI_TSB12LV22, QUIRK_CYCLE_TIMER |
+ QUIRK_RESET_PACKET},
{PCI_VENDOR_ID_TI, PCI_ANY_ID, QUIRK_RESET_PACKET},
{PCI_VENDOR_ID_AL, PCI_ANY_ID, QUIRK_CYCLE_TIMER},
{PCI_VENDOR_ID_NEC, PCI_ANY_ID, QUIRK_CYCLE_TIMER},
#include <linux/platform_device.h>
#include <linux/dma-mapping.h>
#include <linux/errno.h>
+#include <linux/gfp.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/mc146818rtc.h>
*/
#include <linux/init.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/blkdev.h>
#include <linux/init.h>
#include <linux/dmi.h>
#include <linux/device.h>
+#include <linux/slab.h>
struct dmi_device_attribute{
struct device_attribute dev_attr;
#include <linux/dmi.h>
#include <linux/efi.h>
#include <linux/bootmem.h>
-#include <linux/slab.h>
#include <asm/dmi.h>
/*
#include <linux/sysfs.h>
#include <linux/kobject.h>
#include <linux/device.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <linux/limits.h>
#include <linux/module.h>
#include <linux/pci.h>
-#include <linux/slab.h>
#include <linux/stat.h>
#include <linux/string.h>
#include <linux/types.h>
#include <linux/module.h>
#include <linux/types.h>
#include <linux/bootmem.h>
+#include <linux/slab.h>
/*
* Data types ------------------------------------------------------------------
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/platform_device.h>
#include <linux/module.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/i2c.h>
#include <linux/gpio.h>
#include <linux/pci.h>
#include <linux/spinlock.h>
#include <linux/gpio.h>
+#include <linux/slab.h>
/* Steal the hardware definitions from the bttv driver. */
#include "../media/video/bt8xx/bt848.h"
#include <linux/seq_file.h>
#include <linux/gpio.h>
#include <linux/idr.h>
+#include <linux/slab.h>
/* Optional implementation infrastructure for GPIO interfaces.
#include <linux/irq.h>
#include <linux/io.h>
#include <linux/gpio.h>
+#include <linux/slab.h>
struct lnw_gpio_register {
u32 GPLR[2];
#include <linux/mutex.h>
#include <linux/i2c.h>
#include <linux/spi/max7301.h>
+#include <linux/slab.h>
static int max7300_i2c_write(struct device *dev, unsigned int reg,
unsigned int val)
#include <linux/init.h>
#include <linux/platform_device.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <linux/spi/spi.h>
#include <linux/spi/max7301.h>
#include <linux/mutex.h>
#include <linux/spi/max7301.h>
#include <linux/gpio.h>
+#include <linux/slab.h>
/*
* Pin configurations, see MAX7301 datasheet page 6
return ret;
}
EXPORT_SYMBOL_GPL(__max730x_remove);
+
+MODULE_AUTHOR("Juergen Beisert, Wolfram Sang");
+MODULE_LICENSE("GPL v2");
+MODULE_DESCRIPTION("MAX730x GPIO-Expanders, generic parts");
#include <linux/spi/spi.h>
#include <linux/spi/mc33880.h>
#include <linux/gpio.h>
+#include <linux/slab.h>
#define DRIVER_NAME "mc33880"
#include <linux/gpio.h>
#include <linux/spi/spi.h>
#include <linux/spi/mcp23s08.h>
+#include <linux/slab.h>
/* Registers are all 8 bits wide.
#include <linux/irq.h>
#include <linux/i2c.h>
#include <linux/i2c/pca953x.h>
+#include <linux/slab.h>
#ifdef CONFIG_OF_GPIO
#include <linux/of_platform.h>
#include <linux/of_gpio.h>
#include <linux/device.h>
#include <linux/amba/bus.h>
#include <linux/amba/pl061.h>
+#include <linux/slab.h>
#define GPIODIR 0x400
#define GPIOIS 0x404
#include <linux/io.h>
#include <linux/timb_gpio.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#define DRIVER_NAME "timb-gpio"
#include <linux/irq.h>
#include <linux/gpio.h>
#include <linux/platform_device.h>
-#include <linux/slab.h>
#include <linux/i2c/twl.h>
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/gpio.h>
#include <linux/mfd/core.h>
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/gpio.h>
#include <linux/mfd/core.h>
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/gpio.h>
#include <linux/mfd/core.h>
#include <linux/of_gpio.h>
#include <linux/io.h>
#include <linux/gpio.h>
+#include <linux/slab.h>
/* Register Offset Definitions */
#define XGPIO_DATA_OFFSET (0x0) /* Data register */
#include "drmP.h"
#include <linux/module.h>
+#include <linux/slab.h>
#if __OS_HAS_AGP
*/
#include <linux/vmalloc.h>
+#include <linux/slab.h>
#include <linux/log2.h>
#include <asm/shmparam.h>
#include "drmP.h"
* Jesse Barnes <jesse.barnes@intel.com>
*/
#include <linux/list.h>
+#include <linux/slab.h>
#include "drm.h"
#include "drmP.h"
#include "drm_crtc.h"
if (connector->status == connector_status_disconnected) {
DRM_DEBUG_KMS("%s is disconnected\n",
drm_get_connector_name(connector));
+ drm_mode_connector_update_edid_property(connector, NULL);
goto prune;
}
#include <linux/debugfs.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include "drmP.h"
#if defined(CONFIG_DEBUG_FS)
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/delay.h>
-#include <linux/slab.h>
#include <linux/init.h>
#include <linux/errno.h>
#include <linux/sched.h>
*/
#include <linux/debugfs.h>
+#include <linux/slab.h>
#include "drmP.h"
#include "drm_core.h"
* DEALINGS IN THE SOFTWARE.
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/i2c.h>
#include <linux/i2c-algo-bit.h>
#include "drmP.h"
mode->vsync_end = mode->vsync_start + vsync_pulse_width;
mode->vtotal = mode->vdisplay + vblank;
- /* perform the basic check for the detailed timing */
- if (mode->hsync_end > mode->htotal ||
- mode->vsync_end > mode->vtotal) {
- drm_mode_destroy(dev, mode);
- DRM_DEBUG_KMS("Incorrect detailed timing. "
- "Sync is beyond the blank.\n");
- return NULL;
- }
-
/* Some EDIDs have bogus h/vtotal values */
if (mode->hsync_end > mode->htotal)
mode->htotal = mode->hsync_end + 1;
*/
#include <linux/kernel.h>
#include <linux/sysrq.h>
+#include <linux/slab.h>
#include <linux/fb.h>
#include "drmP.h"
#include "drm_crtc.h"
.help_msg = "force-fb(V)",
.action_msg = "Restore framebuffer console",
};
+#else
+static struct sysrq_key_op sysrq_drm_fb_helper_restore_op = { };
#endif
static void drm_fb_helper_on(struct fb_info *info)
#include "drmP.h"
#include <linux/poll.h>
+#include <linux/slab.h>
#include <linux/smp_lock.h>
static int drm_open_helper(struct inode *inode, struct file *filp,
spin_unlock(&dev->count_lock);
}
out:
- mutex_lock(&dev->struct_mutex);
- if (minor->type == DRM_MINOR_LEGACY) {
- BUG_ON((dev->dev_mapping != NULL) &&
- (dev->dev_mapping != inode->i_mapping));
- if (dev->dev_mapping == NULL)
- dev->dev_mapping = inode->i_mapping;
+ if (!retcode) {
+ mutex_lock(&dev->struct_mutex);
+ if (minor->type == DRM_MINOR_LEGACY) {
+ if (dev->dev_mapping == NULL)
+ dev->dev_mapping = inode->i_mapping;
+ else if (dev->dev_mapping != inode->i_mapping)
+ retcode = -ENODEV;
+ }
+ mutex_unlock(&dev->struct_mutex);
}
- mutex_unlock(&dev->struct_mutex);
return retcode;
}
#include "drmP.h"
#include "drm_hashtab.h"
#include <linux/hash.h>
+#include <linux/slab.h>
int drm_ht_create(struct drm_open_hash *ht, unsigned int order)
{
#include "drmP.h"
#include <linux/interrupt.h> /* For task queue support */
+#include <linux/slab.h>
#include <linux/vgaarb.h>
/**
*/
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/dma-mapping.h>
#include "drmP.h"
*/
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include "drmP.h"
/***************************************************
*/
#include <linux/vmalloc.h>
+#include <linux/slab.h>
#include "drmP.h"
#define DEBUG_SCATTER 0
#include <linux/module.h>
#include <linux/moduleparam.h>
+#include <linux/slab.h>
#include "drmP.h"
#include "drm_core.h"
#include <linux/device.h>
#include <linux/kdev_t.h>
+#include <linux/gfp.h>
#include <linux/err.h>
#include "drm_sysfs.h"
#include "drmP.h"
#if defined(__ia64__)
#include <linux/efi.h>
+#include <linux/slab.h>
#endif
static void drm_vm_open(struct vm_area_struct *vma);
#include "i810_drv.h"
#include <linux/interrupt.h> /* For task queue support */
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/pagemap.h>
#define I810_BUF_FREE 2
#include <linux/interrupt.h> /* For task queue support */
#include <linux/pagemap.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#define I830_BUF_FREE 2
#include <linux/seq_file.h>
#include <linux/debugfs.h>
+#include <linux/slab.h>
#include "drmP.h"
#include "drm.h"
#include "i915_drm.h"
#include <linux/acpi.h>
#include <linux/pnp.h>
#include <linux/vga_switcheroo.h>
+#include <linux/slab.h>
/* Really want an OS-independent resettable timer. Would like to have
* this loop run for (eg) 3 sec, but have the timer reset every time
DRM_IOCTL_DEF(DRM_I915_GET_VBLANK_PIPE, i915_vblank_pipe_get, DRM_AUTH ),
DRM_IOCTL_DEF(DRM_I915_VBLANK_SWAP, i915_vblank_swap, DRM_AUTH),
DRM_IOCTL_DEF(DRM_I915_HWS_ADDR, i915_set_status_page, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
- DRM_IOCTL_DEF(DRM_I915_GEM_INIT, i915_gem_init_ioctl, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
- DRM_IOCTL_DEF(DRM_I915_GEM_EXECBUFFER, i915_gem_execbuffer, DRM_AUTH),
- DRM_IOCTL_DEF(DRM_I915_GEM_EXECBUFFER2, i915_gem_execbuffer2, DRM_AUTH),
- DRM_IOCTL_DEF(DRM_I915_GEM_PIN, i915_gem_pin_ioctl, DRM_AUTH|DRM_ROOT_ONLY),
- DRM_IOCTL_DEF(DRM_I915_GEM_UNPIN, i915_gem_unpin_ioctl, DRM_AUTH|DRM_ROOT_ONLY),
- DRM_IOCTL_DEF(DRM_I915_GEM_BUSY, i915_gem_busy_ioctl, DRM_AUTH),
- DRM_IOCTL_DEF(DRM_I915_GEM_THROTTLE, i915_gem_throttle_ioctl, DRM_AUTH),
- DRM_IOCTL_DEF(DRM_I915_GEM_ENTERVT, i915_gem_entervt_ioctl, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
- DRM_IOCTL_DEF(DRM_I915_GEM_LEAVEVT, i915_gem_leavevt_ioctl, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
- DRM_IOCTL_DEF(DRM_I915_GEM_CREATE, i915_gem_create_ioctl, 0),
- DRM_IOCTL_DEF(DRM_I915_GEM_PREAD, i915_gem_pread_ioctl, 0),
- DRM_IOCTL_DEF(DRM_I915_GEM_PWRITE, i915_gem_pwrite_ioctl, 0),
- DRM_IOCTL_DEF(DRM_I915_GEM_MMAP, i915_gem_mmap_ioctl, 0),
- DRM_IOCTL_DEF(DRM_I915_GEM_MMAP_GTT, i915_gem_mmap_gtt_ioctl, 0),
- DRM_IOCTL_DEF(DRM_I915_GEM_SET_DOMAIN, i915_gem_set_domain_ioctl, 0),
- DRM_IOCTL_DEF(DRM_I915_GEM_SW_FINISH, i915_gem_sw_finish_ioctl, 0),
- DRM_IOCTL_DEF(DRM_I915_GEM_SET_TILING, i915_gem_set_tiling, 0),
- DRM_IOCTL_DEF(DRM_I915_GEM_GET_TILING, i915_gem_get_tiling, 0),
- DRM_IOCTL_DEF(DRM_I915_GEM_GET_APERTURE, i915_gem_get_aperture_ioctl, 0),
- DRM_IOCTL_DEF(DRM_I915_GET_PIPE_FROM_CRTC_ID, intel_get_pipe_from_crtc_id, 0),
- DRM_IOCTL_DEF(DRM_I915_GEM_MADVISE, i915_gem_madvise_ioctl, 0),
- DRM_IOCTL_DEF(DRM_I915_OVERLAY_PUT_IMAGE, intel_overlay_put_image, DRM_MASTER|DRM_CONTROL_ALLOW),
- DRM_IOCTL_DEF(DRM_I915_OVERLAY_ATTRS, intel_overlay_attrs, DRM_MASTER|DRM_CONTROL_ALLOW),
+ DRM_IOCTL_DEF(DRM_I915_GEM_INIT, i915_gem_init_ioctl, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY|DRM_UNLOCKED),
+ DRM_IOCTL_DEF(DRM_I915_GEM_EXECBUFFER, i915_gem_execbuffer, DRM_AUTH|DRM_UNLOCKED),
+ DRM_IOCTL_DEF(DRM_I915_GEM_EXECBUFFER2, i915_gem_execbuffer2, DRM_AUTH|DRM_UNLOCKED),
+ DRM_IOCTL_DEF(DRM_I915_GEM_PIN, i915_gem_pin_ioctl, DRM_AUTH|DRM_ROOT_ONLY|DRM_UNLOCKED),
+ DRM_IOCTL_DEF(DRM_I915_GEM_UNPIN, i915_gem_unpin_ioctl, DRM_AUTH|DRM_ROOT_ONLY|DRM_UNLOCKED),
+ DRM_IOCTL_DEF(DRM_I915_GEM_BUSY, i915_gem_busy_ioctl, DRM_AUTH|DRM_UNLOCKED),
+ DRM_IOCTL_DEF(DRM_I915_GEM_THROTTLE, i915_gem_throttle_ioctl, DRM_AUTH|DRM_UNLOCKED),
+ DRM_IOCTL_DEF(DRM_I915_GEM_ENTERVT, i915_gem_entervt_ioctl, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY|DRM_UNLOCKED),
+ DRM_IOCTL_DEF(DRM_I915_GEM_LEAVEVT, i915_gem_leavevt_ioctl, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY|DRM_UNLOCKED),
+ DRM_IOCTL_DEF(DRM_I915_GEM_CREATE, i915_gem_create_ioctl, DRM_UNLOCKED),
+ DRM_IOCTL_DEF(DRM_I915_GEM_PREAD, i915_gem_pread_ioctl, DRM_UNLOCKED),
+ DRM_IOCTL_DEF(DRM_I915_GEM_PWRITE, i915_gem_pwrite_ioctl, DRM_UNLOCKED),
+ DRM_IOCTL_DEF(DRM_I915_GEM_MMAP, i915_gem_mmap_ioctl, DRM_UNLOCKED),
+ DRM_IOCTL_DEF(DRM_I915_GEM_MMAP_GTT, i915_gem_mmap_gtt_ioctl, DRM_UNLOCKED),
+ DRM_IOCTL_DEF(DRM_I915_GEM_SET_DOMAIN, i915_gem_set_domain_ioctl, DRM_UNLOCKED),
+ DRM_IOCTL_DEF(DRM_I915_GEM_SW_FINISH, i915_gem_sw_finish_ioctl, DRM_UNLOCKED),
+ DRM_IOCTL_DEF(DRM_I915_GEM_SET_TILING, i915_gem_set_tiling, DRM_UNLOCKED),
+ DRM_IOCTL_DEF(DRM_I915_GEM_GET_TILING, i915_gem_get_tiling, DRM_UNLOCKED),
+ DRM_IOCTL_DEF(DRM_I915_GEM_GET_APERTURE, i915_gem_get_aperture_ioctl, DRM_UNLOCKED),
+ DRM_IOCTL_DEF(DRM_I915_GET_PIPE_FROM_CRTC_ID, intel_get_pipe_from_crtc_id, DRM_UNLOCKED),
+ DRM_IOCTL_DEF(DRM_I915_GEM_MADVISE, i915_gem_madvise_ioctl, DRM_UNLOCKED),
+ DRM_IOCTL_DEF(DRM_I915_OVERLAY_PUT_IMAGE, intel_overlay_put_image, DRM_MASTER|DRM_CONTROL_ALLOW|DRM_UNLOCKED),
+ DRM_IOCTL_DEF(DRM_I915_OVERLAY_ATTRS, intel_overlay_attrs, DRM_MASTER|DRM_CONTROL_ALLOW|DRM_UNLOCKED),
};
int i915_max_ioctl = DRM_ARRAY_SIZE(i915_ioctls);
const static struct intel_device_info intel_sandybridge_d_info = {
.is_i965g = 1, .is_i9xx = 1, .need_gfx_hws = 1,
- .has_hotplug = 1,
+ .has_hotplug = 1, .is_gen6 = 1,
};
const static struct intel_device_info intel_sandybridge_m_info = {
.is_i965g = 1, .is_mobile = 1, .is_i9xx = 1, .need_gfx_hws = 1,
- .has_hotplug = 1,
+ .has_hotplug = 1, .is_gen6 = 1,
};
const static struct pci_device_id pciidlist[] = {
u8 is_g4x : 1;
u8 is_pineview : 1;
u8 is_ironlake : 1;
+ u8 is_gen6 : 1;
u8 has_fbc : 1;
u8 has_rc6 : 1;
u8 has_pipe_cxsr : 1;
#define IS_IRONLAKE_M(dev) ((dev)->pci_device == 0x0046)
#define IS_IRONLAKE(dev) (INTEL_INFO(dev)->is_ironlake)
#define IS_I9XX(dev) (INTEL_INFO(dev)->is_i9xx)
+#define IS_GEN6(dev) (INTEL_INFO(dev)->is_gen6)
#define IS_MOBILE(dev) (INTEL_INFO(dev)->is_mobile)
#define IS_GEN3(dev) (IS_I915G(dev) || \
#define I915_NEED_GFX_HWS(dev) (INTEL_INFO(dev)->need_gfx_hws)
-#define IS_GEN6(dev) ((dev)->pci_device == 0x0102)
-
/* With the 945 and later, Y tiling got adjusted so that it was 32 128-byte
* rows, which changed the alignment requirements and fence programming.
*/
#include "i915_drv.h"
#include "i915_trace.h"
#include "intel_drv.h"
+#include <linux/slab.h>
#include <linux/swap.h>
#include <linux/pci.h>
obj_priv->dirty = 0;
for (i = 0; i < page_count; i++) {
- if (obj_priv->pages[i] == NULL)
- break;
-
if (obj_priv->dirty)
set_page_dirty(obj_priv->pages[i]);
seqno = i915_add_request(dev, NULL, obj->write_domain);
if (seqno == 0)
return -ENOMEM;
-
- ret = i915_wait_request(dev, seqno);
- if (ret)
- return ret;
-
continue;
}
}
struct address_space *mapping;
struct inode *inode;
struct page *page;
- int ret;
if (obj_priv->pages_refcount++ != 0)
return 0;
mapping_gfp_mask (mapping) |
__GFP_COLD |
gfpmask);
- if (IS_ERR(page)) {
- ret = PTR_ERR(page);
- i915_gem_object_put_pages(obj);
- return ret;
- }
+ if (IS_ERR(page))
+ goto err_pages;
+
obj_priv->pages[i] = page;
}
i915_gem_object_do_bit_17_swizzle(obj);
return 0;
+
+err_pages:
+ while (i--)
+ page_cache_release(obj_priv->pages[i]);
+
+ drm_free_large(obj_priv->pages);
+ obj_priv->pages = NULL;
+ obj_priv->pages_refcount--;
+ return PTR_ERR(page);
}
static void sandybridge_write_fence_reg(struct drm_i915_fence_reg *reg)
ring->space += ring->Size;
}
+ if (IS_I9XX(dev) && !IS_GEN3(dev)) {
+ I915_WRITE(MI_MODE,
+ (VS_TIMER_DISPATCH) << 16 | VS_TIMER_DISPATCH);
+ }
+
return 0;
}
* need to ensure that any fence register is cleared.
*/
if (!i915_gem_object_fence_offset_ok(obj, args->tiling_mode))
- ret = i915_gem_object_unbind(obj);
+ ret = i915_gem_object_unbind(obj);
+ else if (obj_priv->fence_reg != I915_FENCE_REG_NONE)
+ ret = i915_gem_object_put_fence_reg(obj);
else
- ret = i915_gem_object_put_fence_reg(obj);
+ i915_gem_release_mmap(obj);
+
if (ret != 0) {
WARN(ret != -ERESTARTSYS,
"failed to reset object for tiling switch");
*/
#include <linux/sysrq.h>
+#include <linux/slab.h>
#include "drmP.h"
#include "drm.h"
#include "i915_drm.h"
#define INSTDONE 0x02090
#define NOPID 0x02094
#define HWSTAM 0x02098
+
+#define MI_MODE 0x0209c
+# define VS_TIMER_DISPATCH (1 << 6)
+
#define SCPD0 0x0209c /* 915+ only */
#define IER 0x020a0
#define IIR 0x020a4
#define FBC_CTL_PERIODIC (1<<30)
#define FBC_CTL_INTERVAL_SHIFT (16)
#define FBC_CTL_UNCOMPRESSIBLE (1<<14)
-#define FBC_C3_IDLE (1<<13)
+#define FBC_CTL_C3_IDLE (1<<13)
#define FBC_CTL_STRIDE_SHIFT (5)
#define FBC_CTL_FENCENO (1<<0)
#define FBC_COMMAND 0x0320c
#define DISPLAY_PORT_PLL_BIOS_1 0x46010
#define DISPLAY_PORT_PLL_BIOS_2 0x46014
+#define PCH_DSPCLK_GATE_D 0x42020
+# define DPFDUNIT_CLOCK_GATE_DISABLE (1 << 7)
+# define DPARBUNIT_CLOCK_GATE_DISABLE (1 << 5)
+
+#define PCH_3DCGDIS0 0x46020
+# define MARIUNIT_CLOCK_GATE_DISABLE (1 << 18)
+# define SVSMUNIT_CLOCK_GATE_DISABLE (1 << 1)
+
#define FDI_PLL_FREQ_CTL 0x46030
#define FDI_PLL_FREQ_CHANGE_REQUEST (1<<24)
#define FDI_PLL_FREQ_LOCK_LIMIT_MASK 0xfff00
edp = find_section(bdb, BDB_EDP);
if (!edp) {
if (SUPPORTS_EDP(dev_priv->dev) && dev_priv->edp_support) {
- DRM_DEBUG_KMS("No eDP BDB found but eDP panel supported,\
- assume 18bpp panel color depth.\n");
+ DRM_DEBUG_KMS("No eDP BDB found but eDP panel "
+ "supported, assume 18bpp panel color "
+ "depth.\n");
dev_priv->edp_bpp = 18;
}
return;
*/
#include <linux/i2c.h>
+#include <linux/slab.h>
#include "drmP.h"
#include "drm.h"
#include "drm_crtc.h"
#include <linux/input.h>
#include <linux/i2c.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include "drmP.h"
#include "intel_drv.h"
#include "i915_drm.h"
/* enable it... */
fbc_ctl = FBC_CTL_EN | FBC_CTL_PERIODIC;
if (IS_I945GM(dev))
- fbc_ctl |= FBC_C3_IDLE; /* 945 needs special SR handling */
+ fbc_ctl |= FBC_CTL_C3_IDLE; /* 945 needs special SR handling */
fbc_ctl |= (dev_priv->cfb_pitch & 0xff) << FBC_CTL_STRIDE_SHIFT;
fbc_ctl |= (interval & 0x2fff) << FBC_CTL_INTERVAL_SHIFT;
if (obj_priv->tiling_mode != I915_TILING_NONE)
* specs, but enable as much else as we can.
*/
if (HAS_PCH_SPLIT(dev)) {
+ uint32_t dspclk_gate = VRHUNIT_CLOCK_GATE_DISABLE;
+
+ if (IS_IRONLAKE(dev)) {
+ /* Required for FBC */
+ dspclk_gate |= DPFDUNIT_CLOCK_GATE_DISABLE;
+ /* Required for CxSR */
+ dspclk_gate |= DPARBUNIT_CLOCK_GATE_DISABLE;
+
+ I915_WRITE(PCH_3DCGDIS0,
+ MARIUNIT_CLOCK_GATE_DISABLE |
+ SVSMUNIT_CLOCK_GATE_DISABLE);
+ }
+
+ I915_WRITE(PCH_DSPCLK_GATE_D, dspclk_gate);
return;
} else if (IS_G4X(dev)) {
uint32_t dspclk_gate;
*/
#include <linux/i2c.h>
+#include <linux/slab.h>
#include "drmP.h"
#include "drm.h"
#include "drm_crtc.h"
* Eric Anholt <eric@anholt.net>
*/
#include <linux/i2c.h>
+#include <linux/slab.h>
#include "drmP.h"
#include "drm.h"
#include "drm_crtc.h"
#include <linux/string.h>
#include <linux/mm.h>
#include <linux/tty.h>
-#include <linux/slab.h>
#include <linux/sysrq.h>
#include <linux/delay.h>
#include <linux/fb.h>
*/
#include <linux/i2c.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include "drmP.h"
#include "drm.h"
* Eric Anholt <eric@anholt.net>
*/
#include <linux/i2c.h>
+#include <linux/slab.h>
#include <linux/i2c-id.h>
#include <linux/i2c-algo-bit.h>
#include "drmP.h"
#include <acpi/button.h>
#include <linux/dmi.h>
#include <linux/i2c.h>
+#include <linux/slab.h>
#include "drmP.h"
#include "drm.h"
#include "drm_crtc.h"
I915_WRITE(PFIT_CONTROL, lvds_priv->pfit_control);
}
-/* Some lid devices report incorrect lid status, assume they're connected */
-static const struct dmi_system_id bad_lid_status[] = {
- {
- .ident = "Compaq nx9020",
- .matches = {
- DMI_MATCH(DMI_SYS_VENDOR, "Hewlett-Packard"),
- DMI_MATCH(DMI_BOARD_NAME, "3084"),
- },
- },
- {
- .ident = "Samsung SX20S",
- .matches = {
- DMI_MATCH(DMI_SYS_VENDOR, "Samsung Electronics"),
- DMI_MATCH(DMI_BOARD_NAME, "SX20S"),
- },
- },
- {
- .ident = "Aspire One",
- .matches = {
- DMI_MATCH(DMI_SYS_VENDOR, "Acer"),
- DMI_MATCH(DMI_PRODUCT_NAME, "Aspire one"),
- },
- },
- {
- .ident = "Aspire 1810T",
- .matches = {
- DMI_MATCH(DMI_SYS_VENDOR, "Acer"),
- DMI_MATCH(DMI_PRODUCT_NAME, "Aspire 1810T"),
- },
- },
- {
- .ident = "PC-81005",
- .matches = {
- DMI_MATCH(DMI_SYS_VENDOR, "MALATA"),
- DMI_MATCH(DMI_PRODUCT_NAME, "PC-81005"),
- },
- },
- {
- .ident = "Clevo M5x0N",
- .matches = {
- DMI_MATCH(DMI_SYS_VENDOR, "CLEVO Co."),
- DMI_MATCH(DMI_BOARD_NAME, "M5x0N"),
- },
- },
- { }
-};
-
/**
* Detect the LVDS connection.
*
/* ACPI lid methods were generally unreliable in this generation, so
* don't even bother.
*/
- if (IS_GEN2(dev))
+ if (IS_GEN2(dev) || IS_GEN3(dev))
return connector_status_connected;
- if (!dmi_check_system(bad_lid_status) && !acpi_lid_open())
- status = connector_status_disconnected;
-
return status;
}
* DEALINGS IN THE SOFTWARE.
*/
+#include <linux/slab.h>
#include <linux/i2c.h>
#include <linux/fb.h>
#include "drmP.h"
drmmode_obj = drm_mode_object_find(dev, put_image_rec->crtc_id,
DRM_MODE_OBJECT_CRTC);
- if (!drmmode_obj)
- return -ENOENT;
+ if (!drmmode_obj) {
+ ret = -ENOENT;
+ goto out_free;
+ }
crtc = to_intel_crtc(obj_to_crtc(drmmode_obj));
new_bo = drm_gem_object_lookup(dev, file_priv,
put_image_rec->bo_handle);
- if (!new_bo)
- return -ENOENT;
+ if (!new_bo) {
+ ret = -ENOENT;
+ goto out_free;
+ }
mutex_lock(&dev->mode_config.mutex);
mutex_lock(&dev->struct_mutex);
mutex_unlock(&dev->struct_mutex);
mutex_unlock(&dev->mode_config.mutex);
drm_gem_object_unreference_unlocked(new_bo);
+out_free:
kfree(params);
return ret;
* Eric Anholt <eric@anholt.net>
*/
#include <linux/i2c.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include "drmP.h"
#include "drm.h"
nouveau_dp.o nouveau_grctx.o \
nv04_timer.o \
nv04_mc.o nv40_mc.o nv50_mc.o \
- nv04_fb.o nv10_fb.o nv40_fb.o \
+ nv04_fb.o nv10_fb.o nv40_fb.o nv50_fb.o \
nv04_fifo.o nv10_fifo.o nv40_fifo.o nv50_fifo.o \
nv04_graph.o nv10_graph.o nv20_graph.o \
nv40_graph.o nv50_graph.o \
#include <linux/pci.h>
#include <linux/acpi.h>
+#include <linux/slab.h>
#include <acpi/acpi_drivers.h>
#include <acpi/acpi_bus.h>
return type;
}
+static void
+apply_dcb_connector_quirks(struct nvbios *bios, int idx)
+{
+ struct dcb_connector_table_entry *cte = &bios->dcb.connector.entry[idx];
+ struct drm_device *dev = bios->dev;
+
+ /* Gigabyte NX85T */
+ if ((dev->pdev->device == 0x0421) &&
+ (dev->pdev->subsystem_vendor == 0x1458) &&
+ (dev->pdev->subsystem_device == 0x344c)) {
+ if (cte->type == DCB_CONNECTOR_HDMI_1)
+ cte->type = DCB_CONNECTOR_DVI_I;
+ }
+}
+
static void
parse_dcb_connector_table(struct nvbios *bios)
{
entry = conntab + conntab[1];
cte = &ct->entry[0];
for (i = 0; i < conntab[2]; i++, entry += conntab[3], cte++) {
+ cte->index = i;
if (conntab[3] == 2)
cte->entry = ROM16(entry[0]);
else
cte->entry = ROM32(entry[0]);
cte->type = (cte->entry & 0x000000ff) >> 0;
- cte->index = (cte->entry & 0x00000f00) >> 8;
+ cte->index2 = (cte->entry & 0x00000f00) >> 8;
switch (cte->entry & 0x00033000) {
case 0x00001000:
cte->gpio_tag = 0x07;
if (cte->type == 0xff)
continue;
+ apply_dcb_connector_quirks(bios, i);
+
NV_INFO(dev, " %d: 0x%08x: type 0x%02x idx %d tag 0x%02x\n",
i, cte->entry, cte->type, cte->index, cte->gpio_tag);
break;
default:
cte->type = divine_connector_type(bios, cte->index);
- NV_WARN(dev, "unknown type, using 0x%02x", cte->type);
+ NV_WARN(dev, "unknown type, using 0x%02x\n", cte->type);
break;
}
+ if (nouveau_override_conntype) {
+ int type = divine_connector_type(bios, cte->index);
+ if (type != cte->type)
+ NV_WARN(dev, " -> type 0x%02x\n", cte->type);
+ }
+
}
}
};
struct dcb_connector_table_entry {
+ uint8_t index;
uint32_t entry;
enum dcb_connector_type type;
- uint8_t index;
+ uint8_t index2;
uint8_t gpio_tag;
};
#include "nouveau_dma.h"
#include <linux/log2.h>
+#include <linux/slab.h>
static void
nouveau_bo_del_ttm(struct ttm_buffer_object *bo)
switch (bo->mem.mem_type) {
case TTM_PL_VRAM:
- nouveau_bo_placement_set(nvbo, TTM_PL_FLAG_TT |
- TTM_PL_FLAG_SYSTEM);
+ nouveau_bo_placement_set(nvbo, TTM_PL_FLAG_TT);
break;
default:
nouveau_bo_placement_set(nvbo, TTM_PL_FLAG_SYSTEM);
detect_analog:
nv_encoder = find_encoder_by_type(connector, OUTPUT_ANALOG);
- if (!nv_encoder)
+ if (!nv_encoder && !nouveau_tv_disable)
nv_encoder = find_encoder_by_type(connector, OUTPUT_TV);
if (nv_encoder) {
struct drm_encoder *encoder = to_drm_encoder(nv_encoder);
nouveau_bo_wr32(pb, ip++, upper_32_bits(offset) | length << 8);
chan->dma.ib_put = (chan->dma.ib_put + 1) & chan->dma.ib_max;
+
+ DRM_MEMORYBARRIER();
+ /* Flush writes. */
+ nouveau_bo_rd32(pb, 0);
+
nvchan_wr32(chan, 0x8c, chan->dma.ib_put);
chan->dma.ib_free--;
}
int nouveau_nofbaccel = 0;
module_param_named(nofbaccel, nouveau_nofbaccel, int, 0400);
+MODULE_PARM_DESC(override_conntype, "Ignore DCB connector type");
+int nouveau_override_conntype = 0;
+module_param_named(override_conntype, nouveau_override_conntype, int, 0400);
+
+MODULE_PARM_DESC(tv_disable, "Disable TV-out detection\n");
+int nouveau_tv_disable = 0;
+module_param_named(tv_disable, nouveau_tv_disable, int, 0400);
+
MODULE_PARM_DESC(tv_norm, "Default TV norm.\n"
"\t\tSupported: PAL, PAL-M, PAL-N, PAL-Nc, NTSC-M, NTSC-J,\n"
"\t\t\thd480i, hd480p, hd576i, hd576p, hd720p, hd1080i.\n"
if (pm_state.event == PM_EVENT_PRETHAW)
return 0;
+ NV_INFO(dev, "Disabling fbcon acceleration...\n");
fbdev_flags = dev_priv->fbdev_info->flags;
dev_priv->fbdev_info->flags |= FBINFO_HWACCEL_DISABLED;
+ NV_INFO(dev, "Unpinning framebuffer(s)...\n");
list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) {
struct nouveau_framebuffer *nouveau_fb;
extern int nouveau_vram_pushbuf;
extern int nouveau_vram_notify;
extern int nouveau_fbpercrtc;
+extern int nouveau_tv_disable;
extern char *nouveau_tv_norm;
extern int nouveau_reg_debug;
extern char *nouveau_vbios;
extern int nouveau_ignorelid;
extern int nouveau_nofbaccel;
extern int nouveau_noaccel;
+extern int nouveau_override_conntype;
extern int nouveau_pci_suspend(struct pci_dev *pdev, pm_message_t pm_state);
extern int nouveau_pci_resume(struct pci_dev *pdev);
extern void nv40_fb_set_region_tiling(struct drm_device *, int, uint32_t,
uint32_t, uint32_t);
+/* nv50_fb.c */
+extern int nv50_fb_init(struct drm_device *);
+extern void nv50_fb_takedown(struct drm_device *);
+
/* nv04_fifo.c */
extern int nv04_fifo_init(struct drm_device *);
extern void nv04_fifo_disable(struct drm_device *);
#include <linux/string.h>
#include <linux/mm.h>
#include <linux/tty.h>
-#include <linux/slab.h>
#include <linux/sysrq.h>
#include <linux/delay.h>
#include <linux/fb.h>
*/
#include <linux/firmware.h>
+#include <linux/slab.h>
#include "drmP.h"
#include "nouveau_drv.h"
#define nouveau_print_bitfield_names(val, namelist) \
nouveau_print_bitfield_names_((val), (namelist), ARRAY_SIZE(namelist))
+struct nouveau_enum_names {
+ uint32_t value;
+ const char *name;
+};
+
+static void
+nouveau_print_enum_names_(uint32_t value,
+ const struct nouveau_enum_names *namelist,
+ const int namelist_len)
+{
+ /*
+ * Caller must have already printed the KERN_* log level for us.
+ * Also the caller is responsible for adding the newline.
+ */
+ int i;
+ for (i = 0; i < namelist_len; ++i) {
+ if (value == namelist[i].value) {
+ printk("%s", namelist[i].name);
+ return;
+ }
+ }
+ printk("unknown value 0x%08x", value);
+}
+#define nouveau_print_enum_names(val, namelist) \
+ nouveau_print_enum_names_((val), (namelist), ARRAY_SIZE(namelist))
static int
nouveau_graph_chid_from_grctx(struct drm_device *dev)
struct drm_nouveau_private *dev_priv = dev->dev_private;
uint32_t nsource = trap->nsource, nstatus = trap->nstatus;
- NV_INFO(dev, "%s - nSource:", id);
- nouveau_print_bitfield_names(nsource, nsource_names);
- printk(", nStatus:");
- if (dev_priv->card_type < NV_10)
- nouveau_print_bitfield_names(nstatus, nstatus_names);
- else
- nouveau_print_bitfield_names(nstatus, nstatus_names_nv10);
- printk("\n");
+ if (dev_priv->card_type < NV_50) {
+ NV_INFO(dev, "%s - nSource:", id);
+ nouveau_print_bitfield_names(nsource, nsource_names);
+ printk(", nStatus:");
+ if (dev_priv->card_type < NV_10)
+ nouveau_print_bitfield_names(nstatus, nstatus_names);
+ else
+ nouveau_print_bitfield_names(nstatus, nstatus_names_nv10);
+ printk("\n");
+ }
NV_INFO(dev, "%s - Ch %d/%d Class 0x%04x Mthd 0x%04x "
"Data 0x%08x:0x%08x\n",
nv_wr32(dev, NV03_PMC_INTR_0, NV_PMC_INTR_0_PGRAPH_PENDING);
}
+static void
+nv50_pfb_vm_trap(struct drm_device *dev, int display, const char *name)
+{
+ struct drm_nouveau_private *dev_priv = dev->dev_private;
+ uint32_t trap[6];
+ int i, ch;
+ uint32_t idx = nv_rd32(dev, 0x100c90);
+ if (idx & 0x80000000) {
+ idx &= 0xffffff;
+ if (display) {
+ for (i = 0; i < 6; i++) {
+ nv_wr32(dev, 0x100c90, idx | i << 24);
+ trap[i] = nv_rd32(dev, 0x100c94);
+ }
+ for (ch = 0; ch < dev_priv->engine.fifo.channels; ch++) {
+ struct nouveau_channel *chan = dev_priv->fifos[ch];
+
+ if (!chan || !chan->ramin)
+ continue;
+
+ if (trap[1] == chan->ramin->instance >> 12)
+ break;
+ }
+ NV_INFO(dev, "%s - VM: Trapped %s at %02x%04x%04x status %08x %08x channel %d\n",
+ name, (trap[5]&0x100?"read":"write"),
+ trap[5]&0xff, trap[4]&0xffff,
+ trap[3]&0xffff, trap[0], trap[2], ch);
+ }
+ nv_wr32(dev, 0x100c90, idx | 0x80000000);
+ } else if (display) {
+ NV_INFO(dev, "%s - no VM fault?\n", name);
+ }
+}
+
+static struct nouveau_enum_names nv50_mp_exec_error_names[] =
+{
+ { 3, "STACK_UNDERFLOW" },
+ { 4, "QUADON_ACTIVE" },
+ { 8, "TIMEOUT" },
+ { 0x10, "INVALID_OPCODE" },
+ { 0x40, "BREAKPOINT" },
+};
+
+static void
+nv50_pgraph_mp_trap(struct drm_device *dev, int tpid, int display)
+{
+ struct drm_nouveau_private *dev_priv = dev->dev_private;
+ uint32_t units = nv_rd32(dev, 0x1540);
+ uint32_t addr, mp10, status, pc, oplow, ophigh;
+ int i;
+ int mps = 0;
+ for (i = 0; i < 4; i++) {
+ if (!(units & 1 << (i+24)))
+ continue;
+ if (dev_priv->chipset < 0xa0)
+ addr = 0x408200 + (tpid << 12) + (i << 7);
+ else
+ addr = 0x408100 + (tpid << 11) + (i << 7);
+ mp10 = nv_rd32(dev, addr + 0x10);
+ status = nv_rd32(dev, addr + 0x14);
+ if (!status)
+ continue;
+ if (display) {
+ nv_rd32(dev, addr + 0x20);
+ pc = nv_rd32(dev, addr + 0x24);
+ oplow = nv_rd32(dev, addr + 0x70);
+ ophigh= nv_rd32(dev, addr + 0x74);
+ NV_INFO(dev, "PGRAPH_TRAP_MP_EXEC - "
+ "TP %d MP %d: ", tpid, i);
+ nouveau_print_enum_names(status,
+ nv50_mp_exec_error_names);
+ printk(" at %06x warp %d, opcode %08x %08x\n",
+ pc&0xffffff, pc >> 24,
+ oplow, ophigh);
+ }
+ nv_wr32(dev, addr + 0x10, mp10);
+ nv_wr32(dev, addr + 0x14, 0);
+ mps++;
+ }
+ if (!mps && display)
+ NV_INFO(dev, "PGRAPH_TRAP_MP_EXEC - TP %d: "
+ "No MPs claiming errors?\n", tpid);
+}
+
+static void
+nv50_pgraph_tp_trap(struct drm_device *dev, int type, uint32_t ustatus_old,
+ uint32_t ustatus_new, int display, const char *name)
+{
+ struct drm_nouveau_private *dev_priv = dev->dev_private;
+ int tps = 0;
+ uint32_t units = nv_rd32(dev, 0x1540);
+ int i, r;
+ uint32_t ustatus_addr, ustatus;
+ for (i = 0; i < 16; i++) {
+ if (!(units & (1 << i)))
+ continue;
+ if (dev_priv->chipset < 0xa0)
+ ustatus_addr = ustatus_old + (i << 12);
+ else
+ ustatus_addr = ustatus_new + (i << 11);
+ ustatus = nv_rd32(dev, ustatus_addr) & 0x7fffffff;
+ if (!ustatus)
+ continue;
+ tps++;
+ switch (type) {
+ case 6: /* texture error... unknown for now */
+ nv50_pfb_vm_trap(dev, display, name);
+ if (display) {
+ NV_ERROR(dev, "magic set %d:\n", i);
+ for (r = ustatus_addr + 4; r <= ustatus_addr + 0x10; r += 4)
+ NV_ERROR(dev, "\t0x%08x: 0x%08x\n", r,
+ nv_rd32(dev, r));
+ }
+ break;
+ case 7: /* MP error */
+ if (ustatus & 0x00010000) {
+ nv50_pgraph_mp_trap(dev, i, display);
+ ustatus &= ~0x00010000;
+ }
+ break;
+ case 8: /* TPDMA error */
+ {
+ uint32_t e0c = nv_rd32(dev, ustatus_addr + 4);
+ uint32_t e10 = nv_rd32(dev, ustatus_addr + 8);
+ uint32_t e14 = nv_rd32(dev, ustatus_addr + 0xc);
+ uint32_t e18 = nv_rd32(dev, ustatus_addr + 0x10);
+ uint32_t e1c = nv_rd32(dev, ustatus_addr + 0x14);
+ uint32_t e20 = nv_rd32(dev, ustatus_addr + 0x18);
+ uint32_t e24 = nv_rd32(dev, ustatus_addr + 0x1c);
+ nv50_pfb_vm_trap(dev, display, name);
+ /* 2d engine destination */
+ if (ustatus & 0x00000010) {
+ if (display) {
+ NV_INFO(dev, "PGRAPH_TRAP_TPDMA_2D - TP %d - Unknown fault at address %02x%08x\n",
+ i, e14, e10);
+ NV_INFO(dev, "PGRAPH_TRAP_TPDMA_2D - TP %d - e0c: %08x, e18: %08x, e1c: %08x, e20: %08x, e24: %08x\n",
+ i, e0c, e18, e1c, e20, e24);
+ }
+ ustatus &= ~0x00000010;
+ }
+ /* Render target */
+ if (ustatus & 0x00000040) {
+ if (display) {
+ NV_INFO(dev, "PGRAPH_TRAP_TPDMA_RT - TP %d - Unknown fault at address %02x%08x\n",
+ i, e14, e10);
+ NV_INFO(dev, "PGRAPH_TRAP_TPDMA_RT - TP %d - e0c: %08x, e18: %08x, e1c: %08x, e20: %08x, e24: %08x\n",
+ i, e0c, e18, e1c, e20, e24);
+ }
+ ustatus &= ~0x00000040;
+ }
+ /* CUDA memory: l[], g[] or stack. */
+ if (ustatus & 0x00000080) {
+ if (display) {
+ if (e18 & 0x80000000) {
+ /* g[] read fault? */
+ NV_INFO(dev, "PGRAPH_TRAP_TPDMA - TP %d - Global read fault at address %02x%08x\n",
+ i, e14, e10 | ((e18 >> 24) & 0x1f));
+ e18 &= ~0x1f000000;
+ } else if (e18 & 0xc) {
+ /* g[] write fault? */
+ NV_INFO(dev, "PGRAPH_TRAP_TPDMA - TP %d - Global write fault at address %02x%08x\n",
+ i, e14, e10 | ((e18 >> 7) & 0x1f));
+ e18 &= ~0x00000f80;
+ } else {
+ NV_INFO(dev, "PGRAPH_TRAP_TPDMA - TP %d - Unknown CUDA fault at address %02x%08x\n",
+ i, e14, e10);
+ }
+ NV_INFO(dev, "PGRAPH_TRAP_TPDMA - TP %d - e0c: %08x, e18: %08x, e1c: %08x, e20: %08x, e24: %08x\n",
+ i, e0c, e18, e1c, e20, e24);
+ }
+ ustatus &= ~0x00000080;
+ }
+ }
+ break;
+ }
+ if (ustatus) {
+ if (display)
+ NV_INFO(dev, "%s - TP%d: Unhandled ustatus 0x%08x\n", name, i, ustatus);
+ }
+ nv_wr32(dev, ustatus_addr, 0xc0000000);
+ }
+
+ if (!tps && display)
+ NV_INFO(dev, "%s - No TPs claiming errors?\n", name);
+}
+
+static void
+nv50_pgraph_trap_handler(struct drm_device *dev)
+{
+ struct nouveau_pgraph_trap trap;
+ uint32_t status = nv_rd32(dev, 0x400108);
+ uint32_t ustatus;
+ int display = nouveau_ratelimit();
+
+
+ if (!status && display) {
+ nouveau_graph_trap_info(dev, &trap);
+ nouveau_graph_dump_trap_info(dev, "PGRAPH_TRAP", &trap);
+ NV_INFO(dev, "PGRAPH_TRAP - no units reporting traps?\n");
+ }
+
+ /* DISPATCH: Relays commands to other units and handles NOTIFY,
+ * COND, QUERY. If you get a trap from it, the command is still stuck
+ * in DISPATCH and you need to do something about it. */
+ if (status & 0x001) {
+ ustatus = nv_rd32(dev, 0x400804) & 0x7fffffff;
+ if (!ustatus && display) {
+ NV_INFO(dev, "PGRAPH_TRAP_DISPATCH - no ustatus?\n");
+ }
+
+ /* Known to be triggered by screwed up NOTIFY and COND... */
+ if (ustatus & 0x00000001) {
+ nv50_pfb_vm_trap(dev, display, "PGRAPH_TRAP_DISPATCH_FAULT");
+ nv_wr32(dev, 0x400500, 0);
+ if (nv_rd32(dev, 0x400808) & 0x80000000) {
+ if (display) {
+ if (nouveau_graph_trapped_channel(dev, &trap.channel))
+ trap.channel = -1;
+ trap.class = nv_rd32(dev, 0x400814);
+ trap.mthd = nv_rd32(dev, 0x400808) & 0x1ffc;
+ trap.subc = (nv_rd32(dev, 0x400808) >> 16) & 0x7;
+ trap.data = nv_rd32(dev, 0x40080c);
+ trap.data2 = nv_rd32(dev, 0x400810);
+ nouveau_graph_dump_trap_info(dev,
+ "PGRAPH_TRAP_DISPATCH_FAULT", &trap);
+ NV_INFO(dev, "PGRAPH_TRAP_DISPATCH_FAULT - 400808: %08x\n", nv_rd32(dev, 0x400808));
+ NV_INFO(dev, "PGRAPH_TRAP_DISPATCH_FAULT - 400848: %08x\n", nv_rd32(dev, 0x400848));
+ }
+ nv_wr32(dev, 0x400808, 0);
+ } else if (display) {
+ NV_INFO(dev, "PGRAPH_TRAP_DISPATCH_FAULT - No stuck command?\n");
+ }
+ nv_wr32(dev, 0x4008e8, nv_rd32(dev, 0x4008e8) & 3);
+ nv_wr32(dev, 0x400848, 0);
+ ustatus &= ~0x00000001;
+ }
+ if (ustatus & 0x00000002) {
+ nv50_pfb_vm_trap(dev, display, "PGRAPH_TRAP_DISPATCH_QUERY");
+ nv_wr32(dev, 0x400500, 0);
+ if (nv_rd32(dev, 0x40084c) & 0x80000000) {
+ if (display) {
+ if (nouveau_graph_trapped_channel(dev, &trap.channel))
+ trap.channel = -1;
+ trap.class = nv_rd32(dev, 0x400814);
+ trap.mthd = nv_rd32(dev, 0x40084c) & 0x1ffc;
+ trap.subc = (nv_rd32(dev, 0x40084c) >> 16) & 0x7;
+ trap.data = nv_rd32(dev, 0x40085c);
+ trap.data2 = 0;
+ nouveau_graph_dump_trap_info(dev,
+ "PGRAPH_TRAP_DISPATCH_QUERY", &trap);
+ NV_INFO(dev, "PGRAPH_TRAP_DISPATCH_QUERY - 40084c: %08x\n", nv_rd32(dev, 0x40084c));
+ }
+ nv_wr32(dev, 0x40084c, 0);
+ } else if (display) {
+ NV_INFO(dev, "PGRAPH_TRAP_DISPATCH_QUERY - No stuck command?\n");
+ }
+ ustatus &= ~0x00000002;
+ }
+ if (ustatus && display)
+ NV_INFO(dev, "PGRAPH_TRAP_DISPATCH - Unhandled ustatus 0x%08x\n", ustatus);
+ nv_wr32(dev, 0x400804, 0xc0000000);
+ nv_wr32(dev, 0x400108, 0x001);
+ status &= ~0x001;
+ }
+
+ /* TRAPs other than dispatch use the "normal" trap regs. */
+ if (status && display) {
+ nouveau_graph_trap_info(dev, &trap);
+ nouveau_graph_dump_trap_info(dev,
+ "PGRAPH_TRAP", &trap);
+ }
+
+ /* M2MF: Memory to memory copy engine. */
+ if (status & 0x002) {
+ ustatus = nv_rd32(dev, 0x406800) & 0x7fffffff;
+ if (!ustatus && display) {
+ NV_INFO(dev, "PGRAPH_TRAP_M2MF - no ustatus?\n");
+ }
+ if (ustatus & 0x00000001) {
+ nv50_pfb_vm_trap(dev, display, "PGRAPH_TRAP_M2MF_NOTIFY");
+ ustatus &= ~0x00000001;
+ }
+ if (ustatus & 0x00000002) {
+ nv50_pfb_vm_trap(dev, display, "PGRAPH_TRAP_M2MF_IN");
+ ustatus &= ~0x00000002;
+ }
+ if (ustatus & 0x00000004) {
+ nv50_pfb_vm_trap(dev, display, "PGRAPH_TRAP_M2MF_OUT");
+ ustatus &= ~0x00000004;
+ }
+ NV_INFO (dev, "PGRAPH_TRAP_M2MF - %08x %08x %08x %08x\n",
+ nv_rd32(dev, 0x406804),
+ nv_rd32(dev, 0x406808),
+ nv_rd32(dev, 0x40680c),
+ nv_rd32(dev, 0x406810));
+ if (ustatus && display)
+ NV_INFO(dev, "PGRAPH_TRAP_M2MF - Unhandled ustatus 0x%08x\n", ustatus);
+ /* No sane way found yet -- just reset the bugger. */
+ nv_wr32(dev, 0x400040, 2);
+ nv_wr32(dev, 0x400040, 0);
+ nv_wr32(dev, 0x406800, 0xc0000000);
+ nv_wr32(dev, 0x400108, 0x002);
+ status &= ~0x002;
+ }
+
+ /* VFETCH: Fetches data from vertex buffers. */
+ if (status & 0x004) {
+ ustatus = nv_rd32(dev, 0x400c04) & 0x7fffffff;
+ if (!ustatus && display) {
+ NV_INFO(dev, "PGRAPH_TRAP_VFETCH - no ustatus?\n");
+ }
+ if (ustatus & 0x00000001) {
+ nv50_pfb_vm_trap(dev, display, "PGRAPH_TRAP_VFETCH_FAULT");
+ NV_INFO (dev, "PGRAPH_TRAP_VFETCH_FAULT - %08x %08x %08x %08x\n",
+ nv_rd32(dev, 0x400c00),
+ nv_rd32(dev, 0x400c08),
+ nv_rd32(dev, 0x400c0c),
+ nv_rd32(dev, 0x400c10));
+ ustatus &= ~0x00000001;
+ }
+ if (ustatus && display)
+ NV_INFO(dev, "PGRAPH_TRAP_VFETCH - Unhandled ustatus 0x%08x\n", ustatus);
+ nv_wr32(dev, 0x400c04, 0xc0000000);
+ nv_wr32(dev, 0x400108, 0x004);
+ status &= ~0x004;
+ }
+
+ /* STRMOUT: DirectX streamout / OpenGL transform feedback. */
+ if (status & 0x008) {
+ ustatus = nv_rd32(dev, 0x401800) & 0x7fffffff;
+ if (!ustatus && display) {
+ NV_INFO(dev, "PGRAPH_TRAP_STRMOUT - no ustatus?\n");
+ }
+ if (ustatus & 0x00000001) {
+ nv50_pfb_vm_trap(dev, display, "PGRAPH_TRAP_STRMOUT_FAULT");
+ NV_INFO (dev, "PGRAPH_TRAP_STRMOUT_FAULT - %08x %08x %08x %08x\n",
+ nv_rd32(dev, 0x401804),
+ nv_rd32(dev, 0x401808),
+ nv_rd32(dev, 0x40180c),
+ nv_rd32(dev, 0x401810));
+ ustatus &= ~0x00000001;
+ }
+ if (ustatus && display)
+ NV_INFO(dev, "PGRAPH_TRAP_STRMOUT - Unhandled ustatus 0x%08x\n", ustatus);
+ /* No sane way found yet -- just reset the bugger. */
+ nv_wr32(dev, 0x400040, 0x80);
+ nv_wr32(dev, 0x400040, 0);
+ nv_wr32(dev, 0x401800, 0xc0000000);
+ nv_wr32(dev, 0x400108, 0x008);
+ status &= ~0x008;
+ }
+
+ /* CCACHE: Handles code and c[] caches and fills them. */
+ if (status & 0x010) {
+ ustatus = nv_rd32(dev, 0x405018) & 0x7fffffff;
+ if (!ustatus && display) {
+ NV_INFO(dev, "PGRAPH_TRAP_CCACHE - no ustatus?\n");
+ }
+ if (ustatus & 0x00000001) {
+ nv50_pfb_vm_trap(dev, display, "PGRAPH_TRAP_CCACHE_FAULT");
+ NV_INFO (dev, "PGRAPH_TRAP_CCACHE_FAULT - %08x %08x %08x %08x %08x %08x %08x\n",
+ nv_rd32(dev, 0x405800),
+ nv_rd32(dev, 0x405804),
+ nv_rd32(dev, 0x405808),
+ nv_rd32(dev, 0x40580c),
+ nv_rd32(dev, 0x405810),
+ nv_rd32(dev, 0x405814),
+ nv_rd32(dev, 0x40581c));
+ ustatus &= ~0x00000001;
+ }
+ if (ustatus && display)
+ NV_INFO(dev, "PGRAPH_TRAP_CCACHE - Unhandled ustatus 0x%08x\n", ustatus);
+ nv_wr32(dev, 0x405018, 0xc0000000);
+ nv_wr32(dev, 0x400108, 0x010);
+ status &= ~0x010;
+ }
+
+ /* Unknown, not seen yet... 0x402000 is the only trap status reg
+ * remaining, so try to handle it anyway. Perhaps related to that
+ * unknown DMA slot on tesla? */
+ if (status & 0x20) {
+ nv50_pfb_vm_trap(dev, display, "PGRAPH_TRAP_UNKC04");
+ ustatus = nv_rd32(dev, 0x402000) & 0x7fffffff;
+ if (display)
+ NV_INFO(dev, "PGRAPH_TRAP_UNKC04 - Unhandled ustatus 0x%08x\n", ustatus);
+ nv_wr32(dev, 0x402000, 0xc0000000);
+ /* no status modifiction on purpose */
+ }
+
+ /* TEXTURE: CUDA texturing units */
+ if (status & 0x040) {
+ nv50_pgraph_tp_trap (dev, 6, 0x408900, 0x408600, display,
+ "PGRAPH_TRAP_TEXTURE");
+ nv_wr32(dev, 0x400108, 0x040);
+ status &= ~0x040;
+ }
+
+ /* MP: CUDA execution engines. */
+ if (status & 0x080) {
+ nv50_pgraph_tp_trap (dev, 7, 0x408314, 0x40831c, display,
+ "PGRAPH_TRAP_MP");
+ nv_wr32(dev, 0x400108, 0x080);
+ status &= ~0x080;
+ }
+
+ /* TPDMA: Handles TP-initiated uncached memory accesses:
+ * l[], g[], stack, 2d surfaces, render targets. */
+ if (status & 0x100) {
+ nv50_pgraph_tp_trap (dev, 8, 0x408e08, 0x408708, display,
+ "PGRAPH_TRAP_TPDMA");
+ nv_wr32(dev, 0x400108, 0x100);
+ status &= ~0x100;
+ }
+
+ if (status) {
+ if (display)
+ NV_INFO(dev, "PGRAPH_TRAP - Unknown trap 0x%08x\n",
+ status);
+ nv_wr32(dev, 0x400108, status);
+ }
+}
+
+/* There must be a *lot* of these. Will take some time to gather them up. */
+static struct nouveau_enum_names nv50_data_error_names[] =
+{
+ { 4, "INVALID_VALUE" },
+ { 5, "INVALID_ENUM" },
+ { 8, "INVALID_OBJECT" },
+ { 0xc, "INVALID_BITFIELD" },
+ { 0x28, "MP_NO_REG_SPACE" },
+ { 0x2b, "MP_BLOCK_SIZE_MISMATCH" },
+};
+
static void
nv50_pgraph_irq_handler(struct drm_device *dev)
{
+ struct nouveau_pgraph_trap trap;
+ int unhandled = 0;
uint32_t status;
while ((status = nv_rd32(dev, NV03_PGRAPH_INTR))) {
- uint32_t nsource = nv_rd32(dev, NV03_PGRAPH_NSOURCE);
-
+ /* NOTIFY: You've set a NOTIFY an a command and it's done. */
if (status & 0x00000001) {
- nouveau_pgraph_intr_notify(dev, nsource);
+ nouveau_graph_trap_info(dev, &trap);
+ if (nouveau_ratelimit())
+ nouveau_graph_dump_trap_info(dev,
+ "PGRAPH_NOTIFY", &trap);
status &= ~0x00000001;
nv_wr32(dev, NV03_PGRAPH_INTR, 0x00000001);
}
- if (status & 0x00000010) {
- nouveau_pgraph_intr_error(dev, nsource |
- NV03_PGRAPH_NSOURCE_ILLEGAL_MTHD);
+ /* COMPUTE_QUERY: Purpose and exact cause unknown, happens
+ * when you write 0x200 to 0x50c0 method 0x31c. */
+ if (status & 0x00000002) {
+ nouveau_graph_trap_info(dev, &trap);
+ if (nouveau_ratelimit())
+ nouveau_graph_dump_trap_info(dev,
+ "PGRAPH_COMPUTE_QUERY", &trap);
+ status &= ~0x00000002;
+ nv_wr32(dev, NV03_PGRAPH_INTR, 0x00000002);
+ }
+ /* Unknown, never seen: 0x4 */
+
+ /* ILLEGAL_MTHD: You used a wrong method for this class. */
+ if (status & 0x00000010) {
+ nouveau_graph_trap_info(dev, &trap);
+ if (nouveau_pgraph_intr_swmthd(dev, &trap))
+ unhandled = 1;
+ if (unhandled && nouveau_ratelimit())
+ nouveau_graph_dump_trap_info(dev,
+ "PGRAPH_ILLEGAL_MTHD", &trap);
status &= ~0x00000010;
nv_wr32(dev, NV03_PGRAPH_INTR, 0x00000010);
}
+ /* ILLEGAL_CLASS: You used a wrong class. */
+ if (status & 0x00000020) {
+ nouveau_graph_trap_info(dev, &trap);
+ if (nouveau_ratelimit())
+ nouveau_graph_dump_trap_info(dev,
+ "PGRAPH_ILLEGAL_CLASS", &trap);
+ status &= ~0x00000020;
+ nv_wr32(dev, NV03_PGRAPH_INTR, 0x00000020);
+ }
+
+ /* DOUBLE_NOTIFY: You tried to set a NOTIFY on another NOTIFY. */
+ if (status & 0x00000040) {
+ nouveau_graph_trap_info(dev, &trap);
+ if (nouveau_ratelimit())
+ nouveau_graph_dump_trap_info(dev,
+ "PGRAPH_DOUBLE_NOTIFY", &trap);
+ status &= ~0x00000040;
+ nv_wr32(dev, NV03_PGRAPH_INTR, 0x00000040);
+ }
+
+ /* CONTEXT_SWITCH: PGRAPH needs us to load a new context */
if (status & 0x00001000) {
nv_wr32(dev, 0x400500, 0x00000000);
nv_wr32(dev, NV03_PGRAPH_INTR,
status &= ~NV_PGRAPH_INTR_CONTEXT_SWITCH;
}
- if (status & 0x00100000) {
- nouveau_pgraph_intr_error(dev, nsource |
- NV03_PGRAPH_NSOURCE_DATA_ERROR);
+ /* BUFFER_NOTIFY: Your m2mf transfer finished */
+ if (status & 0x00010000) {
+ nouveau_graph_trap_info(dev, &trap);
+ if (nouveau_ratelimit())
+ nouveau_graph_dump_trap_info(dev,
+ "PGRAPH_BUFFER_NOTIFY", &trap);
+ status &= ~0x00010000;
+ nv_wr32(dev, NV03_PGRAPH_INTR, 0x00010000);
+ }
+ /* DATA_ERROR: Invalid value for this method, or invalid
+ * state in current PGRAPH context for this operation */
+ if (status & 0x00100000) {
+ nouveau_graph_trap_info(dev, &trap);
+ if (nouveau_ratelimit()) {
+ nouveau_graph_dump_trap_info(dev,
+ "PGRAPH_DATA_ERROR", &trap);
+ NV_INFO (dev, "PGRAPH_DATA_ERROR - ");
+ nouveau_print_enum_names(nv_rd32(dev, 0x400110),
+ nv50_data_error_names);
+ printk("\n");
+ }
status &= ~0x00100000;
nv_wr32(dev, NV03_PGRAPH_INTR, 0x00100000);
}
+ /* TRAP: Something bad happened in the middle of command
+ * execution. Has a billion types, subtypes, and even
+ * subsubtypes. */
if (status & 0x00200000) {
- int r;
-
- nouveau_pgraph_intr_error(dev, nsource |
- NV03_PGRAPH_NSOURCE_PROTECTION_ERROR);
-
- NV_ERROR(dev, "magic set 1:\n");
- for (r = 0x408900; r <= 0x408910; r += 4)
- NV_ERROR(dev, "\t0x%08x: 0x%08x\n", r,
- nv_rd32(dev, r));
- nv_wr32(dev, 0x408900,
- nv_rd32(dev, 0x408904) | 0xc0000000);
- for (r = 0x408e08; r <= 0x408e24; r += 4)
- NV_ERROR(dev, "\t0x%08x: 0x%08x\n", r,
- nv_rd32(dev, r));
- nv_wr32(dev, 0x408e08,
- nv_rd32(dev, 0x408e08) | 0xc0000000);
-
- NV_ERROR(dev, "magic set 2:\n");
- for (r = 0x409900; r <= 0x409910; r += 4)
- NV_ERROR(dev, "\t0x%08x: 0x%08x\n", r,
- nv_rd32(dev, r));
- nv_wr32(dev, 0x409900,
- nv_rd32(dev, 0x409904) | 0xc0000000);
- for (r = 0x409e08; r <= 0x409e24; r += 4)
- NV_ERROR(dev, "\t0x%08x: 0x%08x\n", r,
- nv_rd32(dev, r));
- nv_wr32(dev, 0x409e08,
- nv_rd32(dev, 0x409e08) | 0xc0000000);
-
+ nv50_pgraph_trap_handler(dev);
status &= ~0x00200000;
- nv_wr32(dev, NV03_PGRAPH_NSOURCE, nsource);
nv_wr32(dev, NV03_PGRAPH_INTR, 0x00200000);
}
+ /* Unknown, never seen: 0x00400000 */
+
+ /* SINGLE_STEP: Happens on every method if you turned on
+ * single stepping in 40008c */
+ if (status & 0x01000000) {
+ nouveau_graph_trap_info(dev, &trap);
+ if (nouveau_ratelimit())
+ nouveau_graph_dump_trap_info(dev,
+ "PGRAPH_SINGLE_STEP", &trap);
+ status &= ~0x01000000;
+ nv_wr32(dev, NV03_PGRAPH_INTR, 0x01000000);
+ }
+
+ /* 0x02000000 happens when you pause a ctxprog...
+ * but the only way this can happen that I know is by
+ * poking the relevant MMIO register, and we don't
+ * do that. */
+
if (status) {
NV_INFO(dev, "Unhandled PGRAPH_INTR - 0x%08x\n",
status);
}
nv_wr32(dev, NV03_PMC_INTR_0, NV_PMC_INTR_0_PGRAPH_PENDING);
- nv_wr32(dev, 0x400824, nv_rd32(dev, 0x400824) & ~(1 << 31));
+ if (nv_rd32(dev, 0x400824) & (1 << 31))
+ nv_wr32(dev, 0x400824, nv_rd32(dev, 0x400824) & ~(1 << 31));
}
static void
#include "drmP.h"
#include "nouveau_drv.h"
#include <linux/pagemap.h>
+#include <linux/slab.h>
#define NV_CTXDMA_PAGE_SHIFT 12
#define NV_CTXDMA_PAGE_SIZE (1 << NV_CTXDMA_PAGE_SHIFT)
*/
#include <linux/swab.h>
+#include <linux/slab.h>
#include "drmP.h"
#include "drm.h"
#include "drm_sarea.h"
#include "nouveau_drm.h"
#include "nv50_display.h"
-static int nouveau_stub_init(struct drm_device *dev) { return 0; }
static void nouveau_stub_takedown(struct drm_device *dev) {}
static int nouveau_init_engine_ptrs(struct drm_device *dev)
engine->timer.init = nv04_timer_init;
engine->timer.read = nv04_timer_read;
engine->timer.takedown = nv04_timer_takedown;
- engine->fb.init = nouveau_stub_init;
- engine->fb.takedown = nouveau_stub_takedown;
+ engine->fb.init = nv50_fb_init;
+ engine->fb.takedown = nv50_fb_takedown;
engine->graph.grclass = nv50_graph_grclass;
engine->graph.init = nv50_graph_init;
engine->graph.takedown = nv50_graph_takedown;
struct drm_framebuffer *fb = crtc->fb;
/* Calculate our timings */
- int horizDisplay = (mode->crtc_hdisplay >> 3) - 1;
- int horizStart = (mode->crtc_hsync_start >> 3) - 1;
- int horizEnd = (mode->crtc_hsync_end >> 3) - 1;
+ int horizDisplay = (mode->crtc_hdisplay >> 3) - 1;
+ int horizStart = (mode->crtc_hsync_start >> 3) + 1;
+ int horizEnd = (mode->crtc_hsync_end >> 3) + 1;
int horizTotal = (mode->crtc_htotal >> 3) - 5;
int horizBlankStart = (mode->crtc_hdisplay >> 3) - 1;
int horizBlankEnd = (mode->crtc_htotal >> 3) - 1;
return;
}
- width = ALIGN(image->width, 32);
- dsize = (width * image->height) >> 5;
+ width = ALIGN(image->width, 8);
+ dsize = ALIGN(width * image->height, 32) >> 5;
if (info->fix.visual == FB_VISUAL_TRUECOLOR ||
info->fix.visual == FB_VISUAL_DIRECTCOLOR) {
((image->dx + image->width) & 0xffff));
OUT_RING(chan, bg);
OUT_RING(chan, fg);
- OUT_RING(chan, (image->height << 16) | image->width);
OUT_RING(chan, (image->height << 16) | width);
+ OUT_RING(chan, (image->height << 16) | image->width);
OUT_RING(chan, (image->dy << 16) | (image->dx & 0xffff));
while (dsize) {
}
for (i = 0 ; i < dcb->connector.entries; i++) {
- if (i != 0 && dcb->connector.entry[i].index ==
- dcb->connector.entry[i - 1].index)
+ if (i != 0 && dcb->connector.entry[i].index2 ==
+ dcb->connector.entry[i - 1].index2)
continue;
nouveau_connector_create(dev, &dcb->connector.entry[i]);
}
--- /dev/null
+#include "drmP.h"
+#include "drm.h"
+#include "nouveau_drv.h"
+#include "nouveau_drm.h"
+
+int
+nv50_fb_init(struct drm_device *dev)
+{
+ /* This is needed to get meaningful information from 100c90
+ * on traps. No idea what these values mean exactly. */
+ struct drm_nouveau_private *dev_priv = dev->dev_private;
+
+ switch (dev_priv->chipset) {
+ case 0x50:
+ nv_wr32(dev, 0x100c90, 0x0707ff);
+ break;
+ case 0xa5:
+ case 0xa8:
+ nv_wr32(dev, 0x100c90, 0x0d0fff);
+ break;
+ default:
+ nv_wr32(dev, 0x100c90, 0x1d07ff);
+ break;
+ }
+
+ return 0;
+}
+
+void
+nv50_fb_takedown(struct drm_device *dev)
+{
+}
BEGIN_RING(chan, NvSub2D, 0x0808, 3);
OUT_RING(chan, 0);
OUT_RING(chan, 0);
- OUT_RING(chan, 0);
+ OUT_RING(chan, 1);
BEGIN_RING(chan, NvSub2D, 0x081c, 1);
OUT_RING(chan, 1);
BEGIN_RING(chan, NvSub2D, 0x0840, 4);
static void
nv50_graph_init_regs__nv(struct drm_device *dev)
{
+ struct drm_nouveau_private *dev_priv = dev->dev_private;
+ uint32_t units = nv_rd32(dev, 0x1540);
+ int i;
+
NV_DEBUG(dev, "\n");
nv_wr32(dev, 0x400804, 0xc0000000);
nv_wr32(dev, 0x405018, 0xc0000000);
nv_wr32(dev, 0x402000, 0xc0000000);
+ for (i = 0; i < 16; i++) {
+ if (units & 1 << i) {
+ if (dev_priv->chipset < 0xa0) {
+ nv_wr32(dev, 0x408900 + (i << 12), 0xc0000000);
+ nv_wr32(dev, 0x408e08 + (i << 12), 0xc0000000);
+ nv_wr32(dev, 0x408314 + (i << 12), 0xc0000000);
+ } else {
+ nv_wr32(dev, 0x408600 + (i << 11), 0xc0000000);
+ nv_wr32(dev, 0x408708 + (i << 11), 0xc0000000);
+ nv_wr32(dev, 0x40831c + (i << 11), 0xc0000000);
+ }
+ }
+ }
+
nv_wr32(dev, 0x400108, 0xffffffff);
nv_wr32(dev, 0x400824, 0x00004000);
nouveau_grctx_vals_load(dev, ctx);
}
nv_wo32(dev, ctx, 0x00000/4, chan->ramin->instance >> 12);
- if ((dev_priv->chipset & 0xf0) == 0xa0)
- nv_wo32(dev, ctx, 0x00004/4, 0x00000000);
- else
- nv_wo32(dev, ctx, 0x0011c/4, 0x00000000);
dev_priv->engine.instmem.finish_access(dev);
return 0;
#define CP_FLAG_ALWAYS ((2 * 32) + 13)
#define CP_FLAG_ALWAYS_FALSE 0
#define CP_FLAG_ALWAYS_TRUE 1
+#define CP_FLAG_INTR ((2 * 32) + 15)
+#define CP_FLAG_INTR_NOT_PENDING 0
+#define CP_FLAG_INTR_PENDING 1
#define CP_CTX 0x00100000
#define CP_CTX_COUNT 0x000f0000
cp_name(ctx, cp_setup_save);
cp_set (ctx, UNK1D, SET);
cp_wait(ctx, STATUS, BUSY);
+ cp_wait(ctx, INTR, PENDING);
+ cp_bra (ctx, STATUS, BUSY, cp_setup_save);
cp_set (ctx, UNK01, SET);
cp_set (ctx, SWAP_DIRECTION, SAVE);
int offset, base;
uint32_t units = nv_rd32 (ctx->dev, 0x1540);
- /* 0800 */
+ /* 0800: DISPATCH */
cp_ctx(ctx, 0x400808, 7);
gr_def(ctx, 0x400814, 0x00000030);
cp_ctx(ctx, 0x400834, 0x32);
gr_def(ctx, 0x400b20, 0x0001629d);
}
- /* 0C00 */
+ /* 0C00: VFETCH */
cp_ctx(ctx, 0x400c08, 0x2);
gr_def(ctx, 0x400c08, 0x0000fe0c);
cp_ctx(ctx, 0x401540, 0x5);
gr_def(ctx, 0x401550, 0x00001018);
- /* 1800 */
+ /* 1800: STREAMOUT */
cp_ctx(ctx, 0x401814, 0x1);
gr_def(ctx, 0x401814, 0x000000ff);
if (dev_priv->chipset == 0x50) {
if (dev_priv->chipset == 0x50)
cp_ctx(ctx, 0x4063e0, 0x1);
- /* 6800 */
+ /* 6800: M2MF */
if (dev_priv->chipset < 0x90) {
cp_ctx(ctx, 0x406814, 0x2b);
gr_def(ctx, 0x406818, 0x00000f80);
#include <linux/firmware.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include "drmP.h"
#include "drm.h"
radeon-y := radeon_drv.o radeon_cp.o radeon_state.o radeon_mem.o \
radeon_irq.o r300_cmdbuf.o r600_cp.o
# add KMS driver
-radeon-y += radeon_device.o radeon_kms.o \
+radeon-y += radeon_device.o radeon_asic.o radeon_kms.o \
radeon_atombios.o radeon_agp.o atombios_crtc.o radeon_combios.o \
atom.o radeon_fence.o radeon_ttm.o radeon_object.o radeon_gart.o \
radeon_legacy_crtc.o radeon_legacy_encoders.o radeon_connectors.o \
#include <linux/module.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <asm/unaligned.h>
#define ATOM_DEBUG
typedef struct {
struct atom_context *ctx;
-
uint32_t *ps, *ws;
int ps_shift;
uint16_t start;
+ unsigned last_jump;
+ unsigned long last_jump_jiffies;
+ bool abort;
} atom_exec_context;
int atom_debug = 0;
-static void atom_execute_table_locked(struct atom_context *ctx, int index, uint32_t * params);
-void atom_execute_table(struct atom_context *ctx, int index, uint32_t * params);
+static int atom_execute_table_locked(struct atom_context *ctx, int index, uint32_t * params);
+int atom_execute_table(struct atom_context *ctx, int index, uint32_t * params);
static uint32_t atom_arg_mask[8] =
{ 0xFFFFFFFF, 0xFFFF, 0xFFFF00, 0xFFFF0000, 0xFF, 0xFF00, 0xFF0000,
static void atom_op_calltable(atom_exec_context *ctx, int *ptr, int arg)
{
int idx = U8((*ptr)++);
+ int r = 0;
+
if (idx < ATOM_TABLE_NAMES_CNT)
SDEBUG(" table: %d (%s)\n", idx, atom_table_names[idx]);
else
SDEBUG(" table: %d\n", idx);
if (U16(ctx->ctx->cmd_table + 4 + 2 * idx))
- atom_execute_table_locked(ctx->ctx, idx, ctx->ps + ctx->ps_shift);
+ r = atom_execute_table_locked(ctx->ctx, idx, ctx->ps + ctx->ps_shift);
+ if (r) {
+ ctx->abort = true;
+ }
}
static void atom_op_clear(atom_exec_context *ctx, int *ptr, int arg)
static void atom_op_jump(atom_exec_context *ctx, int *ptr, int arg)
{
int execute = 0, target = U16(*ptr);
+ unsigned long cjiffies;
+
(*ptr) += 2;
switch (arg) {
case ATOM_COND_ABOVE:
if (arg != ATOM_COND_ALWAYS)
SDEBUG(" taken: %s\n", execute ? "yes" : "no");
SDEBUG(" target: 0x%04X\n", target);
- if (execute)
+ if (execute) {
+ if (ctx->last_jump == (ctx->start + target)) {
+ cjiffies = jiffies;
+ if (time_after(cjiffies, ctx->last_jump_jiffies)) {
+ cjiffies -= ctx->last_jump_jiffies;
+ if ((jiffies_to_msecs(cjiffies) > 1000)) {
+ DRM_ERROR("atombios stuck in loop for more than 1sec aborting\n");
+ ctx->abort = true;
+ }
+ } else {
+ /* jiffies wrap around we will just wait a little longer */
+ ctx->last_jump_jiffies = jiffies;
+ }
+ } else {
+ ctx->last_jump = ctx->start + target;
+ ctx->last_jump_jiffies = jiffies;
+ }
*ptr = ctx->start + target;
+ }
}
static void atom_op_mask(atom_exec_context *ctx, int *ptr, int arg)
atom_op_shr, ATOM_ARG_MC}, {
atom_op_debug, 0},};
-static void atom_execute_table_locked(struct atom_context *ctx, int index, uint32_t * params)
+static int atom_execute_table_locked(struct atom_context *ctx, int index, uint32_t * params)
{
int base = CU16(ctx->cmd_table + 4 + 2 * index);
int len, ws, ps, ptr;
atom_exec_context ectx;
if (!base)
- return;
+ return -EINVAL;
len = CU16(base + ATOM_CT_SIZE_PTR);
ws = CU8(base + ATOM_CT_WS_PTR);
ectx.ps_shift = ps / 4;
ectx.start = base;
ectx.ps = params;
+ ectx.abort = false;
+ ectx.last_jump = 0;
if (ws)
ectx.ws = kzalloc(4 * ws, GFP_KERNEL);
else
SDEBUG("%s @ 0x%04X\n", atom_op_names[op], ptr - 1);
else
SDEBUG("[%d] @ 0x%04X\n", op, ptr - 1);
+ if (ectx.abort) {
+ DRM_ERROR("atombios stuck executing %04X (len %d, WS %d, PS %d) @ 0x%04X\n",
+ base, len, ws, ps, ptr - 1);
+ return -EINVAL;
+ }
if (op < ATOM_OP_CNT && op > 0)
opcode_table[op].func(&ectx, &ptr,
if (ws)
kfree(ectx.ws);
+ return 0;
}
-void atom_execute_table(struct atom_context *ctx, int index, uint32_t * params)
+int atom_execute_table(struct atom_context *ctx, int index, uint32_t * params)
{
+ int r;
+
mutex_lock(&ctx->mutex);
/* reset reg block */
ctx->reg_block = 0;
ctx->fb_base = 0;
/* reset io mode */
ctx->io_mode = ATOM_IO_MM;
- atom_execute_table_locked(ctx, index, params);
+ r = atom_execute_table_locked(ctx, index, params);
mutex_unlock(&ctx->mutex);
+ return r;
}
static int atom_iio_len[] = { 1, 2, 3, 3, 3, 3, 4, 4, 4, 3 };
if (!CU16(ctx->cmd_table + 4 + 2 * ATOM_CMD_INIT))
return 1;
- atom_execute_table(ctx, ATOM_CMD_INIT, ps);
-
- return 0;
+ return atom_execute_table(ctx, ATOM_CMD_INIT, ps);
}
void atom_destroy(struct atom_context *ctx)
kfree(ctx);
}
-void atom_parse_data_header(struct atom_context *ctx, int index,
+bool atom_parse_data_header(struct atom_context *ctx, int index,
uint16_t * size, uint8_t * frev, uint8_t * crev,
uint16_t * data_start)
{
int offset = index * 2 + 4;
int idx = CU16(ctx->data_table + offset);
+ u16 *mdt = (u16 *)(ctx->bios + ctx->data_table + 4);
+
+ if (!mdt[index])
+ return false;
if (size)
*size = CU16(idx);
if (crev)
*crev = CU8(idx + 3);
*data_start = idx;
- return;
+ return true;
}
-void atom_parse_cmd_header(struct atom_context *ctx, int index, uint8_t * frev,
+bool atom_parse_cmd_header(struct atom_context *ctx, int index, uint8_t * frev,
uint8_t * crev)
{
int offset = index * 2 + 4;
int idx = CU16(ctx->cmd_table + offset);
+ u16 *mct = (u16 *)(ctx->bios + ctx->cmd_table + 4);
+
+ if (!mct[index])
+ return false;
if (frev)
*frev = CU8(idx + 2);
if (crev)
*crev = CU8(idx + 3);
- return;
+ return true;
}
int atom_allocate_fb_scratch(struct atom_context *ctx)
{
int index = GetIndexIntoMasterTable(DATA, VRAM_UsageByFirmware);
uint16_t data_offset;
- int usage_bytes;
+ int usage_bytes = 0;
struct _ATOM_VRAM_USAGE_BY_FIRMWARE *firmware_usage;
- atom_parse_data_header(ctx, index, NULL, NULL, NULL, &data_offset);
+ if (atom_parse_data_header(ctx, index, NULL, NULL, NULL, &data_offset)) {
+ firmware_usage = (struct _ATOM_VRAM_USAGE_BY_FIRMWARE *)(ctx->bios + data_offset);
- firmware_usage = (struct _ATOM_VRAM_USAGE_BY_FIRMWARE *)(ctx->bios + data_offset);
+ DRM_DEBUG("atom firmware requested %08x %dkb\n",
+ firmware_usage->asFirmwareVramReserveInfo[0].ulStartAddrUsedByFirmware,
+ firmware_usage->asFirmwareVramReserveInfo[0].usFirmwareUseInKb);
- DRM_DEBUG("atom firmware requested %08x %dkb\n",
- firmware_usage->asFirmwareVramReserveInfo[0].ulStartAddrUsedByFirmware,
- firmware_usage->asFirmwareVramReserveInfo[0].usFirmwareUseInKb);
-
- usage_bytes = firmware_usage->asFirmwareVramReserveInfo[0].usFirmwareUseInKb * 1024;
+ usage_bytes = firmware_usage->asFirmwareVramReserveInfo[0].usFirmwareUseInKb * 1024;
+ }
if (usage_bytes == 0)
usage_bytes = 20 * 1024;
/* allocate some scratch memory */
extern int atom_debug;
struct atom_context *atom_parse(struct card_info *, void *);
-void atom_execute_table(struct atom_context *, int, uint32_t *);
+int atom_execute_table(struct atom_context *, int, uint32_t *);
int atom_asic_init(struct atom_context *);
void atom_destroy(struct atom_context *);
-void atom_parse_data_header(struct atom_context *ctx, int index, uint16_t *size, uint8_t *frev, uint8_t *crev, uint16_t *data_start);
-void atom_parse_cmd_header(struct atom_context *ctx, int index, uint8_t *frev, uint8_t *crev);
+bool atom_parse_data_header(struct atom_context *ctx, int index, uint16_t *size,
+ uint8_t *frev, uint8_t *crev, uint16_t *data_start);
+bool atom_parse_cmd_header(struct atom_context *ctx, int index,
+ uint8_t *frev, uint8_t *crev);
int atom_allocate_fb_scratch(struct atom_context *ctx);
#include "atom-types.h"
#include "atombios.h"
atom_execute_table(rdev->mode_info.atom_context, index, (uint32_t *)&args);
}
+static void atombios_disable_ss(struct drm_crtc *crtc)
+{
+ struct radeon_crtc *radeon_crtc = to_radeon_crtc(crtc);
+ struct drm_device *dev = crtc->dev;
+ struct radeon_device *rdev = dev->dev_private;
+ u32 ss_cntl;
+
+ if (ASIC_IS_DCE4(rdev)) {
+ switch (radeon_crtc->pll_id) {
+ case ATOM_PPLL1:
+ ss_cntl = RREG32(EVERGREEN_P1PLL_SS_CNTL);
+ ss_cntl &= ~EVERGREEN_PxPLL_SS_EN;
+ WREG32(EVERGREEN_P1PLL_SS_CNTL, ss_cntl);
+ break;
+ case ATOM_PPLL2:
+ ss_cntl = RREG32(EVERGREEN_P2PLL_SS_CNTL);
+ ss_cntl &= ~EVERGREEN_PxPLL_SS_EN;
+ WREG32(EVERGREEN_P2PLL_SS_CNTL, ss_cntl);
+ break;
+ case ATOM_DCPLL:
+ case ATOM_PPLL_INVALID:
+ return;
+ }
+ } else if (ASIC_IS_AVIVO(rdev)) {
+ switch (radeon_crtc->pll_id) {
+ case ATOM_PPLL1:
+ ss_cntl = RREG32(AVIVO_P1PLL_INT_SS_CNTL);
+ ss_cntl &= ~1;
+ WREG32(AVIVO_P1PLL_INT_SS_CNTL, ss_cntl);
+ break;
+ case ATOM_PPLL2:
+ ss_cntl = RREG32(AVIVO_P2PLL_INT_SS_CNTL);
+ ss_cntl &= ~1;
+ WREG32(AVIVO_P2PLL_INT_SS_CNTL, ss_cntl);
+ break;
+ case ATOM_DCPLL:
+ case ATOM_PPLL_INVALID:
+ return;
+ }
+ }
+}
+
+
union atom_enable_ss {
ENABLE_LVDS_SS_PARAMETERS legacy;
ENABLE_SPREAD_SPECTRUM_ON_PPLL_PS_ALLOCATION v1;
};
-static void atombios_set_ss(struct drm_crtc *crtc, int enable)
+static void atombios_enable_ss(struct drm_crtc *crtc)
{
struct radeon_crtc *radeon_crtc = to_radeon_crtc(crtc);
struct drm_device *dev = crtc->dev;
step = dig->ss->step;
delay = dig->ss->delay;
range = dig->ss->range;
- } else if (enable)
+ } else
return;
- } else if (enable)
+ } else
return;
break;
}
args.v1.ucSpreadSpectrumDelay = delay;
args.v1.ucSpreadSpectrumRange = range;
args.v1.ucPpll = radeon_crtc->crtc_id ? ATOM_PPLL2 : ATOM_PPLL1;
- args.v1.ucEnable = enable;
+ args.v1.ucEnable = ATOM_ENABLE;
} else {
args.legacy.usSpreadSpectrumPercentage = cpu_to_le16(percentage);
args.legacy.ucSpreadSpectrumType = type;
args.legacy.ucSpreadSpectrumStepSize_Delay = (step & 3) << 2;
args.legacy.ucSpreadSpectrumStepSize_Delay |= (delay & 7) << 4;
- args.legacy.ucEnable = enable;
+ args.legacy.ucEnable = ATOM_ENABLE;
}
atom_execute_table(rdev->mode_info.atom_context, index, (uint32_t *)&args);
}
/* DVO wants 2x pixel clock if the DVO chip is in 12 bit mode */
if (radeon_encoder->encoder_id == ENCODER_OBJECT_ID_INTERNAL_KLDSCP_DVO1)
adjusted_clock = mode->clock * 2;
- /* LVDS PLL quirks */
- if (encoder->encoder_type == DRM_MODE_ENCODER_LVDS) {
- struct radeon_encoder_atom_dig *dig = radeon_encoder->enc_priv;
- pll->algo = dig->pll_algo;
- }
} else {
if (encoder->encoder_type != DRM_MODE_ENCODER_DAC)
pll->flags |= RADEON_PLL_NO_ODD_POST_DIV;
int index;
index = GetIndexIntoMasterTable(COMMAND, AdjustDisplayPll);
- atom_parse_cmd_header(rdev->mode_info.atom_context, index, &frev,
- &crev);
+ if (!atom_parse_cmd_header(rdev->mode_info.atom_context, index, &frev,
+ &crev))
+ return adjusted_clock;
memset(&args, 0, sizeof(args));
}
} else if (radeon_encoder->devices & (ATOM_DEVICE_LCD_SUPPORT)) {
/* may want to enable SS on DP/eDP eventually */
- args.v3.sInput.ucDispPllConfig |=
- DISPPLL_CONFIG_SS_ENABLE;
- if (mode->clock > 165000)
+ /*args.v3.sInput.ucDispPllConfig |=
+ DISPPLL_CONFIG_SS_ENABLE;*/
+ if (encoder_mode == ATOM_ENCODER_MODE_DP)
args.v3.sInput.ucDispPllConfig |=
- DISPPLL_CONFIG_DUAL_LINK;
+ DISPPLL_CONFIG_COHERENT_MODE;
+ else {
+ if (mode->clock > 165000)
+ args.v3.sInput.ucDispPllConfig |=
+ DISPPLL_CONFIG_DUAL_LINK;
+ }
}
atom_execute_table(rdev->mode_info.atom_context,
index, (uint32_t *)&args);
memset(&args, 0, sizeof(args));
index = GetIndexIntoMasterTable(COMMAND, SetPixelClock);
- atom_parse_cmd_header(rdev->mode_info.atom_context, index, &frev,
- &crev);
+ if (!atom_parse_cmd_header(rdev->mode_info.atom_context, index, &frev,
+ &crev))
+ return;
switch (frev) {
case 1:
&ref_div, &post_div);
index = GetIndexIntoMasterTable(COMMAND, SetPixelClock);
- atom_parse_cmd_header(rdev->mode_info.atom_context, index, &frev,
- &crev);
+ if (!atom_parse_cmd_header(rdev->mode_info.atom_context, index, &frev,
+ &crev))
+ return;
switch (frev) {
case 1:
/* TODO color tiling */
- /* pick pll */
- radeon_crtc->pll_id = radeon_atom_pick_pll(crtc);
-
- atombios_set_ss(crtc, 0);
+ atombios_disable_ss(crtc);
/* always set DCPLL */
if (ASIC_IS_DCE4(rdev))
atombios_crtc_set_dcpll(crtc);
atombios_crtc_set_pll(crtc, adjusted_mode);
- atombios_set_ss(crtc, 1);
+ atombios_enable_ss(crtc);
if (ASIC_IS_DCE4(rdev))
atombios_set_crtc_dtd_timing(crtc, adjusted_mode);
static void atombios_crtc_prepare(struct drm_crtc *crtc)
{
+ struct radeon_crtc *radeon_crtc = to_radeon_crtc(crtc);
+
+ /* pick pll */
+ radeon_crtc->pll_id = radeon_atom_pick_pll(crtc);
+
atombios_lock_crtc(crtc, ATOM_ENABLE);
atombios_crtc_dpms(crtc, DRM_MODE_DPMS_OFF);
}
>> DP_TRAIN_PRE_EMPHASIS_SHIFT);
/* disable the training pattern on the sink */
+ dp_set_training(radeon_connector, DP_TRAINING_PATTERN_DISABLE);
+
+ /* disable the training pattern on the source */
if (ASIC_IS_DCE4(rdev))
atombios_dig_encoder_setup(encoder, ATOM_ENCODER_CMD_DP_LINK_TRAINING_COMPLETE);
else
radeon_dp_encoder_service(rdev, ATOM_DP_ACTION_TRAINING_COMPLETE,
dig_connector->dp_clock, enc_id, 0);
-
- radeon_dp_encoder_service(rdev, ATOM_DP_ACTION_TRAINING_COMPLETE,
- dig_connector->dp_clock, enc_id, 0);
}
int radeon_dp_i2c_aux_ch(struct i2c_adapter *adapter, int mode,
*/
#include <linux/firmware.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include "drmP.h"
#include "radeon.h"
+#include "radeon_asic.h"
#include "radeon_drm.h"
#include "rv770d.h"
#include "atom.h"
int evergreen_mc_init(struct radeon_device *rdev)
{
- fixed20_12 a;
u32 tmp;
int chansize, numchan;
rdev->mc.real_vram_size = rdev->mc.aper_size;
}
r600_vram_gtt_location(rdev, &rdev->mc);
- /* FIXME: we should enforce default clock in case GPU is not in
- * default setup
- */
- a.full = rfixed_const(100);
- rdev->pm.sclk.full = rfixed_const(rdev->clock.default_sclk);
- rdev->pm.sclk.full = rfixed_div(rdev->pm.sclk, a);
+ radeon_update_bandwidth_info(rdev);
+
return 0;
}
void evergreen_fini(struct radeon_device *rdev)
{
+ radeon_pm_fini(rdev);
evergreen_suspend(rdev);
#if 0
r600_blit_fini(rdev);
* Jerome Glisse
*/
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include "drmP.h"
#include "drm.h"
#include "radeon_drm.h"
#include "radeon_reg.h"
#include "radeon.h"
+#include "radeon_asic.h"
#include "r100d.h"
#include "rs100d.h"
#include "rv200d.h"
void r100_pci_gart_fini(struct radeon_device *rdev)
{
+ radeon_gart_fini(rdev);
r100_pci_gart_disable(rdev);
radeon_gart_table_ram_free(rdev);
- radeon_gart_fini(rdev);
}
int r100_irq_set(struct radeon_device *rdev)
/* Vertical blank interrupts */
if (status & RADEON_CRTC_VBLANK_STAT) {
drm_handle_vblank(rdev->ddev, 0);
+ rdev->pm.vblank_sync = true;
wake_up(&rdev->irq.vblank_queue);
}
if (status & RADEON_CRTC2_VBLANK_STAT) {
drm_handle_vblank(rdev->ddev, 1);
+ rdev->pm.vblank_sync = true;
wake_up(&rdev->irq.vblank_queue);
}
if (status & RADEON_FP_DETECT_STAT) {
udelay(10);
rdev->cp.rptr = RREG32(RADEON_CP_RB_RPTR);
rdev->cp.wptr = RREG32(RADEON_CP_RB_WPTR);
+ /* protect against crazy HW on resume */
+ rdev->cp.wptr &= rdev->cp.ptr_mask;
/* Set cp mode to bus mastering & enable cp*/
WREG32(RADEON_CP_CSQ_MODE,
REG_SET(RADEON_INDIRECT2_START, indirect2_start) |
{
struct drm_device *dev = rdev->ddev;
bool force_dac2 = false;
+ u32 tmp;
/* set these so they don't interfere with anything */
WREG32(RADEON_OV0_SCALE_CNTL, 0);
WREG32(RADEON_DISP_HW_DEBUG, disp_hw_debug);
WREG32(RADEON_DAC_CNTL2, dac2_cntl);
}
+
+ /* switch PM block to ACPI mode */
+ tmp = RREG32_PLL(RADEON_PLL_PWRMGT_CNTL);
+ tmp &= ~RADEON_PM_MODE_SEL;
+ WREG32_PLL(RADEON_PLL_PWRMGT_CNTL, tmp);
+
}
/*
radeon_vram_location(rdev, &rdev->mc, base);
if (!(rdev->flags & RADEON_IS_AGP))
radeon_gtt_location(rdev, &rdev->mc);
+ radeon_update_bandwidth_info(rdev);
}
uint32_t pixel_bytes1 = 0;
uint32_t pixel_bytes2 = 0;
+ radeon_update_display_priority(rdev);
+
if (rdev->mode_info.crtcs[0]->base.enabled) {
mode1 = &rdev->mode_info.crtcs[0]->base.mode;
pixel_bytes1 = rdev->mode_info.crtcs[0]->base.fb->bits_per_pixel / 8;
/*
* determine is there is enough bw for current mode
*/
- mclk_ff.full = rfixed_const(rdev->clock.default_mclk);
- temp_ff.full = rfixed_const(100);
- mclk_ff.full = rfixed_div(mclk_ff, temp_ff);
- sclk_ff.full = rfixed_const(rdev->clock.default_sclk);
- sclk_ff.full = rfixed_div(sclk_ff, temp_ff);
+ sclk_ff = rdev->pm.sclk;
+ mclk_ff = rdev->pm.mclk;
temp = (rdev->mc.vram_width / 8) * (rdev->mc.vram_is_ddr ? 2 : 1);
temp_ff.full = rfixed_const(temp);
void r100_fini(struct radeon_device *rdev)
{
+ radeon_pm_fini(rdev);
r100_cp_fini(rdev);
r100_wb_fini(rdev);
r100_ib_fini(rdev);
#include "radeon_drm.h"
#include "radeon_reg.h"
#include "radeon.h"
+#include "radeon_asic.h"
#include "r100d.h"
#include "r200_reg_safe.h"
* Jerome Glisse
*/
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include "drmP.h"
#include "drm.h"
#include "radeon_reg.h"
#include "radeon.h"
+#include "radeon_asic.h"
#include "radeon_drm.h"
#include "r100_track.h"
#include "r300d.h"
void rv370_pcie_gart_fini(struct radeon_device *rdev)
{
+ radeon_gart_fini(rdev);
rv370_pcie_gart_disable(rdev);
radeon_gart_table_vram_free(rdev);
- radeon_gart_fini(rdev);
}
void r300_fence_ring_emit(struct radeon_device *rdev,
radeon_vram_location(rdev, &rdev->mc, base);
if (!(rdev->flags & RADEON_IS_AGP))
radeon_gtt_location(rdev, &rdev->mc);
+ radeon_update_bandwidth_info(rdev);
}
void rv370_set_pcie_lanes(struct radeon_device *rdev, int lanes)
void r300_fini(struct radeon_device *rdev)
{
+ radeon_pm_fini(rdev);
r100_cp_fini(rdev);
r100_wb_fini(rdev);
r100_ib_fini(rdev);
* Jerome Glisse
*/
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include "drmP.h"
#include "radeon_reg.h"
#include "radeon.h"
+#include "radeon_asic.h"
#include "atom.h"
#include "r100d.h"
#include "r420d.h"
void r420_fini(struct radeon_device *rdev)
{
+ radeon_pm_fini(rdev);
r100_cp_fini(rdev);
r100_wb_fini(rdev);
r100_ib_fini(rdev);
*/
#include "drmP.h"
#include "radeon.h"
+#include "radeon_asic.h"
#include "atom.h"
#include "r520d.h"
void r520_mc_init(struct radeon_device *rdev)
{
- fixed20_12 a;
r520_vram_get_type(rdev);
r100_vram_init_sizes(rdev);
radeon_vram_location(rdev, &rdev->mc, 0);
if (!(rdev->flags & RADEON_IS_AGP))
radeon_gtt_location(rdev, &rdev->mc);
- /* FIXME: we should enforce default clock in case GPU is not in
- * default setup
- */
- a.full = rfixed_const(100);
- rdev->pm.sclk.full = rfixed_const(rdev->clock.default_sclk);
- rdev->pm.sclk.full = rfixed_div(rdev->pm.sclk, a);
+ radeon_update_bandwidth_info(rdev);
}
void r520_mc_program(struct radeon_device *rdev)
* Alex Deucher
* Jerome Glisse
*/
+#include <linux/slab.h>
#include <linux/seq_file.h>
#include <linux/firmware.h>
#include <linux/platform_device.h>
#include "drmP.h"
#include "radeon_drm.h"
#include "radeon.h"
+#include "radeon_asic.h"
#include "radeon_mode.h"
#include "r600d.h"
#include "atom.h"
void r600_pcie_gart_fini(struct radeon_device *rdev)
{
+ radeon_gart_fini(rdev);
r600_pcie_gart_disable(rdev);
radeon_gart_table_vram_free(rdev);
- radeon_gart_fini(rdev);
}
void r600_agp_enable(struct radeon_device *rdev)
int r600_mc_init(struct radeon_device *rdev)
{
- fixed20_12 a;
u32 tmp;
int chansize, numchan;
rdev->mc.real_vram_size = rdev->mc.aper_size;
}
r600_vram_gtt_location(rdev, &rdev->mc);
- /* FIXME: we should enforce default clock in case GPU is not in
- * default setup
- */
- a.full = rfixed_const(100);
- rdev->pm.sclk.full = rfixed_const(rdev->clock.default_sclk);
- rdev->pm.sclk.full = rfixed_div(rdev->pm.sclk, a);
+
if (rdev->flags & RADEON_IS_IGP)
rdev->mc.igp_sideport_enabled = radeon_atombios_sideport_present(rdev);
+ radeon_update_bandwidth_info(rdev);
return 0;
}
/* Setup pipes */
WREG32(CC_RB_BACKEND_DISABLE, cc_rb_backend_disable);
WREG32(CC_GC_SHADER_PIPE_CONFIG, cc_gc_shader_pipe_config);
+ WREG32(GC_USER_SHADER_PIPE_CONFIG, cc_gc_shader_pipe_config);
tmp = R6XX_MAX_PIPES - r600_count_pipe_bits((cc_gc_shader_pipe_config & INACTIVE_QD_PIPES_MASK) >> 8);
WREG32(VGT_OUT_DEALLOC_CNTL, (tmp * 4) & DEALLOC_DIST_MASK);
void r600_fini(struct radeon_device *rdev)
{
+ radeon_pm_fini(rdev);
r600_audio_fini(rdev);
r600_blit_fini(rdev);
r600_cp_fini(rdev);
WREG32(DC_HPD4_INT_CONTROL, tmp);
if (ASIC_IS_DCE32(rdev)) {
tmp = RREG32(DC_HPD5_INT_CONTROL) & DC_HPDx_INT_POLARITY;
- WREG32(DC_HPD5_INT_CONTROL, 0);
+ WREG32(DC_HPD5_INT_CONTROL, tmp);
tmp = RREG32(DC_HPD6_INT_CONTROL) & DC_HPDx_INT_POLARITY;
- WREG32(DC_HPD6_INT_CONTROL, 0);
+ WREG32(DC_HPD6_INT_CONTROL, tmp);
}
} else {
WREG32(DACA_AUTODETECT_INT_CONTROL, 0);
WREG32(DACB_AUTODETECT_INT_CONTROL, 0);
tmp = RREG32(DC_HOT_PLUG_DETECT1_INT_CONTROL) & DC_HOT_PLUG_DETECTx_INT_POLARITY;
- WREG32(DC_HOT_PLUG_DETECT1_INT_CONTROL, 0);
+ WREG32(DC_HOT_PLUG_DETECT1_INT_CONTROL, tmp);
tmp = RREG32(DC_HOT_PLUG_DETECT2_INT_CONTROL) & DC_HOT_PLUG_DETECTx_INT_POLARITY;
- WREG32(DC_HOT_PLUG_DETECT2_INT_CONTROL, 0);
+ WREG32(DC_HOT_PLUG_DETECT2_INT_CONTROL, tmp);
tmp = RREG32(DC_HOT_PLUG_DETECT3_INT_CONTROL) & DC_HOT_PLUG_DETECTx_INT_POLARITY;
- WREG32(DC_HOT_PLUG_DETECT3_INT_CONTROL, 0);
+ WREG32(DC_HOT_PLUG_DETECT3_INT_CONTROL, tmp);
}
}
case 0: /* D1 vblank */
if (disp_int & LB_D1_VBLANK_INTERRUPT) {
drm_handle_vblank(rdev->ddev, 0);
+ rdev->pm.vblank_sync = true;
wake_up(&rdev->irq.vblank_queue);
disp_int &= ~LB_D1_VBLANK_INTERRUPT;
DRM_DEBUG("IH: D1 vblank\n");
case 0: /* D2 vblank */
if (disp_int & LB_D2_VBLANK_INTERRUPT) {
drm_handle_vblank(rdev->ddev, 1);
+ rdev->pm.vblank_sync = true;
wake_up(&rdev->irq.vblank_queue);
disp_int &= ~LB_D2_VBLANK_INTERRUPT;
DRM_DEBUG("IH: D2 vblank\n");
break;
case 10:
if (disp_int_cont2 & DC_HPD5_INTERRUPT) {
- disp_int_cont &= ~DC_HPD5_INTERRUPT;
+ disp_int_cont2 &= ~DC_HPD5_INTERRUPT;
queue_hotplug = true;
DRM_DEBUG("IH: HPD5\n");
}
break;
case 12:
if (disp_int_cont2 & DC_HPD6_INTERRUPT) {
- disp_int_cont &= ~DC_HPD6_INTERRUPT;
+ disp_int_cont2 &= ~DC_HPD6_INTERRUPT;
queue_hotplug = true;
DRM_DEBUG("IH: HPD6\n");
}
return 0;
}
-/*
- * determin how the encoders and audio interface is wired together
- */
-int r600_audio_tmds_index(struct drm_encoder *encoder)
-{
- struct drm_device *dev = encoder->dev;
- struct radeon_encoder *radeon_encoder = to_radeon_encoder(encoder);
- struct drm_encoder *other;
-
- switch (radeon_encoder->encoder_id) {
- case ENCODER_OBJECT_ID_INTERNAL_KLDSCP_TMDS1:
- case ENCODER_OBJECT_ID_INTERNAL_UNIPHY:
- case ENCODER_OBJECT_ID_INTERNAL_UNIPHY1:
- return 0;
-
- case ENCODER_OBJECT_ID_INTERNAL_LVTM1:
- /* special case check if an TMDS1 is present */
- list_for_each_entry(other, &dev->mode_config.encoder_list, head) {
- if (to_radeon_encoder(other)->encoder_id ==
- ENCODER_OBJECT_ID_INTERNAL_TMDS1)
- return 1;
- }
- return 0;
-
- case ENCODER_OBJECT_ID_INTERNAL_UNIPHY2:
- case ENCODER_OBJECT_ID_INTERNAL_KLDSCP_LVTMA:
- return 1;
-
- default:
- DRM_ERROR("Unsupported encoder type 0x%02X\n",
- radeon_encoder->encoder_id);
- return -1;
- }
-}
-
/*
* atach the audio codec to the clock source of the encoder
*/
struct drm_device *dev = encoder->dev;
struct radeon_device *rdev = dev->dev_private;
struct radeon_encoder *radeon_encoder = to_radeon_encoder(encoder);
+ struct radeon_encoder_atom_dig *dig = radeon_encoder->enc_priv;
int base_rate = 48000;
switch (radeon_encoder->encoder_id) {
case ENCODER_OBJECT_ID_INTERNAL_LVTM1:
WREG32_P(R600_AUDIO_TIMING, 0, ~0x301);
break;
-
case ENCODER_OBJECT_ID_INTERNAL_UNIPHY:
case ENCODER_OBJECT_ID_INTERNAL_UNIPHY1:
case ENCODER_OBJECT_ID_INTERNAL_UNIPHY2:
case ENCODER_OBJECT_ID_INTERNAL_KLDSCP_LVTMA:
WREG32_P(R600_AUDIO_TIMING, 0x100, ~0x301);
break;
-
default:
DRM_ERROR("Unsupported encoder type 0x%02X\n",
radeon_encoder->encoder_id);
return;
}
- switch (r600_audio_tmds_index(encoder)) {
+ switch (dig->dig_encoder) {
case 0:
- WREG32(R600_AUDIO_PLL1_MUL, base_rate*50);
- WREG32(R600_AUDIO_PLL1_DIV, clock*100);
+ WREG32(R600_AUDIO_PLL1_MUL, base_rate * 50);
+ WREG32(R600_AUDIO_PLL1_DIV, clock * 100);
WREG32(R600_AUDIO_CLK_SRCSEL, 0);
break;
case 1:
- WREG32(R600_AUDIO_PLL2_MUL, base_rate*50);
- WREG32(R600_AUDIO_PLL2_DIV, clock*100);
+ WREG32(R600_AUDIO_PLL2_MUL, base_rate * 50);
+ WREG32(R600_AUDIO_PLL2_DIV, clock * 100);
WREG32(R600_AUDIO_CLK_SRCSEL, 1);
break;
+ default:
+ dev_err(rdev->dev, "Unsupported DIG on encoder 0x%02X\n",
+ radeon_encoder->encoder_id);
+ return;
}
}
+/*
+ * Copyright 2009 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ *
+ * Authors:
+ * Alex Deucher <alexander.deucher@amd.com>
+ */
#include <linux/types.h>
#include <linux/kernel.h>
+/*
+ * R6xx+ cards need to use the 3D engine to blit data which requires
+ * quite a bit of hw state setup. Rather than pull the whole 3D driver
+ * (which normally generates the 3D state) into the DRM, we opt to use
+ * statically generated state tables. The regsiter state and shaders
+ * were hand generated to support blitting functionality. See the 3D
+ * driver or documentation for descriptions of the registers and
+ * shader instructions.
+ */
+
const u32 r6xx_default_state[] =
{
0xc0002400,
RADEON_WRITE(R600_CC_RB_BACKEND_DISABLE, cc_rb_backend_disable);
RADEON_WRITE(R600_CC_GC_SHADER_PIPE_CONFIG, cc_gc_shader_pipe_config);
+ RADEON_WRITE(R600_GC_USER_SHADER_PIPE_CONFIG, cc_gc_shader_pipe_config);
RADEON_WRITE(R700_CC_SYS_RB_BACKEND_DISABLE, cc_rb_backend_disable);
RADEON_WRITE(R700_CGTS_SYS_TCC_DISABLE, 0);
RADEON_WRITE(R700_CGTS_TCC_DISABLE, 0);
+ RADEON_WRITE(R700_CGTS_USER_SYS_TCC_DISABLE, 0);
+ RADEON_WRITE(R700_CGTS_USER_TCC_DISABLE, 0);
num_qd_pipes =
R7XX_MAX_PIPES - r600_count_pipe_bits((cc_gc_shader_pipe_config & R600_INACTIVE_QD_PIPES_MASK) >> 8);
u32 nbanks;
u32 npipes;
/* value we track */
+ u32 sq_config;
u32 nsamples;
u32 cb_color_base_last[8];
struct radeon_bo *cb_color_bo[8];
{
int i;
+ /* assume DX9 mode */
+ track->sq_config = DX9_CONSTS;
for (i = 0; i < 8; i++) {
track->cb_color_base_last[i] = 0;
track->cb_color_size[i] = 0;
tmp =radeon_get_ib_value(p, idx);
ib[idx] = 0;
break;
+ case SQ_CONFIG:
+ track->sq_config = radeon_get_ib_value(p, idx);
+ break;
case R_028800_DB_DEPTH_CONTROL:
track->db_depth_control = radeon_get_ib_value(p, idx);
break;
case SQ_PGM_START_VS:
case SQ_PGM_START_GS:
case SQ_PGM_START_PS:
+ case SQ_ALU_CONST_CACHE_GS_0:
+ case SQ_ALU_CONST_CACHE_GS_1:
+ case SQ_ALU_CONST_CACHE_GS_2:
+ case SQ_ALU_CONST_CACHE_GS_3:
+ case SQ_ALU_CONST_CACHE_GS_4:
+ case SQ_ALU_CONST_CACHE_GS_5:
+ case SQ_ALU_CONST_CACHE_GS_6:
+ case SQ_ALU_CONST_CACHE_GS_7:
+ case SQ_ALU_CONST_CACHE_GS_8:
+ case SQ_ALU_CONST_CACHE_GS_9:
+ case SQ_ALU_CONST_CACHE_GS_10:
+ case SQ_ALU_CONST_CACHE_GS_11:
+ case SQ_ALU_CONST_CACHE_GS_12:
+ case SQ_ALU_CONST_CACHE_GS_13:
+ case SQ_ALU_CONST_CACHE_GS_14:
+ case SQ_ALU_CONST_CACHE_GS_15:
+ case SQ_ALU_CONST_CACHE_PS_0:
+ case SQ_ALU_CONST_CACHE_PS_1:
+ case SQ_ALU_CONST_CACHE_PS_2:
+ case SQ_ALU_CONST_CACHE_PS_3:
+ case SQ_ALU_CONST_CACHE_PS_4:
+ case SQ_ALU_CONST_CACHE_PS_5:
+ case SQ_ALU_CONST_CACHE_PS_6:
+ case SQ_ALU_CONST_CACHE_PS_7:
+ case SQ_ALU_CONST_CACHE_PS_8:
+ case SQ_ALU_CONST_CACHE_PS_9:
+ case SQ_ALU_CONST_CACHE_PS_10:
+ case SQ_ALU_CONST_CACHE_PS_11:
+ case SQ_ALU_CONST_CACHE_PS_12:
+ case SQ_ALU_CONST_CACHE_PS_13:
+ case SQ_ALU_CONST_CACHE_PS_14:
+ case SQ_ALU_CONST_CACHE_PS_15:
+ case SQ_ALU_CONST_CACHE_VS_0:
+ case SQ_ALU_CONST_CACHE_VS_1:
+ case SQ_ALU_CONST_CACHE_VS_2:
+ case SQ_ALU_CONST_CACHE_VS_3:
+ case SQ_ALU_CONST_CACHE_VS_4:
+ case SQ_ALU_CONST_CACHE_VS_5:
+ case SQ_ALU_CONST_CACHE_VS_6:
+ case SQ_ALU_CONST_CACHE_VS_7:
+ case SQ_ALU_CONST_CACHE_VS_8:
+ case SQ_ALU_CONST_CACHE_VS_9:
+ case SQ_ALU_CONST_CACHE_VS_10:
+ case SQ_ALU_CONST_CACHE_VS_11:
+ case SQ_ALU_CONST_CACHE_VS_12:
+ case SQ_ALU_CONST_CACHE_VS_13:
+ case SQ_ALU_CONST_CACHE_VS_14:
+ case SQ_ALU_CONST_CACHE_VS_15:
r = r600_cs_packet_next_reloc(p, &reloc);
if (r) {
dev_warn(p->dev, "bad SET_CONTEXT_REG "
}
break;
case PACKET3_SET_ALU_CONST:
- start_reg = (idx_value << 2) + PACKET3_SET_ALU_CONST_OFFSET;
- end_reg = 4 * pkt->count + start_reg - 4;
- if ((start_reg < PACKET3_SET_ALU_CONST_OFFSET) ||
- (start_reg >= PACKET3_SET_ALU_CONST_END) ||
- (end_reg >= PACKET3_SET_ALU_CONST_END)) {
- DRM_ERROR("bad SET_ALU_CONST\n");
- return -EINVAL;
+ if (track->sq_config & DX9_CONSTS) {
+ start_reg = (idx_value << 2) + PACKET3_SET_ALU_CONST_OFFSET;
+ end_reg = 4 * pkt->count + start_reg - 4;
+ if ((start_reg < PACKET3_SET_ALU_CONST_OFFSET) ||
+ (start_reg >= PACKET3_SET_ALU_CONST_END) ||
+ (end_reg >= PACKET3_SET_ALU_CONST_END)) {
+ DRM_ERROR("bad SET_ALU_CONST\n");
+ return -EINVAL;
+ }
}
break;
case PACKET3_SET_BOOL_CONST:
*/
enum r600_hdmi_iec_status_bits {
AUDIO_STATUS_DIG_ENABLE = 0x01,
- AUDIO_STATUS_V = 0x02,
- AUDIO_STATUS_VCFG = 0x04,
+ AUDIO_STATUS_V = 0x02,
+ AUDIO_STATUS_VCFG = 0x04,
AUDIO_STATUS_EMPHASIS = 0x08,
AUDIO_STATUS_COPYRIGHT = 0x10,
AUDIO_STATUS_NONAUDIO = 0x20,
AUDIO_STATUS_PROFESSIONAL = 0x40,
- AUDIO_STATUS_LEVEL = 0x80
+ AUDIO_STATUS_LEVEL = 0x80
};
struct {
static void r600_hdmi_calc_CTS(uint32_t clock, int *CTS, int N, int freq)
{
if (*CTS == 0)
- *CTS = clock*N/(128*freq)*1000;
+ *CTS = clock * N / (128 * freq) * 1000;
DRM_DEBUG("Using ACR timing N=%d CTS=%d for frequency %d\n",
N, *CTS, freq);
}
uint8_t length,
uint8_t *frame)
{
- int i;
- frame[0] = packetType + versionNumber + length;
- for (i = 1; i <= length; i++)
- frame[0] += frame[i];
- frame[0] = 0x100 - frame[0];
+ int i;
+ frame[0] = packetType + versionNumber + length;
+ for (i = 1; i <= length; i++)
+ frame[0] += frame[i];
+ frame[0] = 0x100 - frame[0];
}
/*
WREG32_P(offset+R600_HDMI_CNTL, 0x04000000, ~0x04000000);
}
-/*
- * enable/disable the HDMI engine
- */
-void r600_hdmi_enable(struct drm_encoder *encoder, int enable)
+static int r600_hdmi_find_free_block(struct drm_device *dev)
+{
+ struct radeon_device *rdev = dev->dev_private;
+ struct drm_encoder *encoder;
+ struct radeon_encoder *radeon_encoder;
+ bool free_blocks[3] = { true, true, true };
+
+ list_for_each_entry(encoder, &dev->mode_config.encoder_list, head) {
+ radeon_encoder = to_radeon_encoder(encoder);
+ switch (radeon_encoder->hdmi_offset) {
+ case R600_HDMI_BLOCK1:
+ free_blocks[0] = false;
+ break;
+ case R600_HDMI_BLOCK2:
+ free_blocks[1] = false;
+ break;
+ case R600_HDMI_BLOCK3:
+ free_blocks[2] = false;
+ break;
+ }
+ }
+
+ if (rdev->family == CHIP_RS600 || rdev->family == CHIP_RS690) {
+ return free_blocks[0] ? R600_HDMI_BLOCK1 : 0;
+ } else if (rdev->family >= CHIP_R600) {
+ if (free_blocks[0])
+ return R600_HDMI_BLOCK1;
+ else if (free_blocks[1])
+ return R600_HDMI_BLOCK2;
+ }
+ return 0;
+}
+
+static void r600_hdmi_assign_block(struct drm_encoder *encoder)
{
struct drm_device *dev = encoder->dev;
struct radeon_device *rdev = dev->dev_private;
struct radeon_encoder *radeon_encoder = to_radeon_encoder(encoder);
- uint32_t offset = to_radeon_encoder(encoder)->hdmi_offset;
+ struct radeon_encoder_atom_dig *dig = radeon_encoder->enc_priv;
- if (!offset)
+ if (!dig) {
+ dev_err(rdev->dev, "Enabling HDMI on non-dig encoder\n");
return;
+ }
- DRM_DEBUG("%s HDMI interface @ 0x%04X\n", enable ? "Enabling" : "Disabling", offset);
-
- /* some version of atombios ignore the enable HDMI flag
- * so enabling/disabling HDMI was moved here for TMDS1+2 */
- switch (radeon_encoder->encoder_id) {
- case ENCODER_OBJECT_ID_INTERNAL_KLDSCP_TMDS1:
- WREG32_P(AVIVO_TMDSA_CNTL, enable ? 0x4 : 0x0, ~0x4);
- WREG32(offset+R600_HDMI_ENABLE, enable ? 0x101 : 0x0);
- break;
-
- case ENCODER_OBJECT_ID_INTERNAL_LVTM1:
- WREG32_P(AVIVO_LVTMA_CNTL, enable ? 0x4 : 0x0, ~0x4);
- WREG32(offset+R600_HDMI_ENABLE, enable ? 0x105 : 0x0);
- break;
-
- case ENCODER_OBJECT_ID_INTERNAL_UNIPHY:
- case ENCODER_OBJECT_ID_INTERNAL_UNIPHY1:
- case ENCODER_OBJECT_ID_INTERNAL_UNIPHY2:
- case ENCODER_OBJECT_ID_INTERNAL_KLDSCP_LVTMA:
- /* This part is doubtfull in my opinion */
- WREG32(offset+R600_HDMI_ENABLE, enable ? 0x110 : 0x0);
- break;
-
- default:
- DRM_ERROR("unknown HDMI output type\n");
- break;
+ if (ASIC_IS_DCE4(rdev)) {
+ /* TODO */
+ } else if (ASIC_IS_DCE3(rdev)) {
+ radeon_encoder->hdmi_offset = dig->dig_encoder ?
+ R600_HDMI_BLOCK3 : R600_HDMI_BLOCK1;
+ if (ASIC_IS_DCE32(rdev))
+ radeon_encoder->hdmi_config_offset = dig->dig_encoder ?
+ R600_HDMI_CONFIG2 : R600_HDMI_CONFIG1;
+ } else if (rdev->family >= CHIP_R600) {
+ radeon_encoder->hdmi_offset = r600_hdmi_find_free_block(dev);
}
}
/*
- * determin at which register offset the HDMI encoder is
+ * enable the HDMI engine
*/
-void r600_hdmi_init(struct drm_encoder *encoder)
+void r600_hdmi_enable(struct drm_encoder *encoder)
{
+ struct drm_device *dev = encoder->dev;
+ struct radeon_device *rdev = dev->dev_private;
struct radeon_encoder *radeon_encoder = to_radeon_encoder(encoder);
- switch (radeon_encoder->encoder_id) {
- case ENCODER_OBJECT_ID_INTERNAL_KLDSCP_TMDS1:
- case ENCODER_OBJECT_ID_INTERNAL_UNIPHY:
- case ENCODER_OBJECT_ID_INTERNAL_UNIPHY1:
- radeon_encoder->hdmi_offset = R600_HDMI_TMDS1;
- break;
-
- case ENCODER_OBJECT_ID_INTERNAL_LVTM1:
- switch (r600_audio_tmds_index(encoder)) {
- case 0:
- radeon_encoder->hdmi_offset = R600_HDMI_TMDS1;
+ if (!radeon_encoder->hdmi_offset) {
+ r600_hdmi_assign_block(encoder);
+ if (!radeon_encoder->hdmi_offset) {
+ dev_warn(rdev->dev, "Could not find HDMI block for "
+ "0x%x encoder\n", radeon_encoder->encoder_id);
+ return;
+ }
+ }
+
+ if (ASIC_IS_DCE32(rdev) && !ASIC_IS_DCE4(rdev)) {
+ WREG32_P(radeon_encoder->hdmi_config_offset + 0x4, 0x1, ~0x1);
+ } else if (rdev->family >= CHIP_R600 && !ASIC_IS_DCE3(rdev)) {
+ int offset = radeon_encoder->hdmi_offset;
+ switch (radeon_encoder->encoder_id) {
+ case ENCODER_OBJECT_ID_INTERNAL_KLDSCP_TMDS1:
+ WREG32_P(AVIVO_TMDSA_CNTL, 0x4, ~0x4);
+ WREG32(offset + R600_HDMI_ENABLE, 0x101);
break;
- case 1:
- radeon_encoder->hdmi_offset = R600_HDMI_TMDS2;
+ case ENCODER_OBJECT_ID_INTERNAL_LVTM1:
+ WREG32_P(AVIVO_LVTMA_CNTL, 0x4, ~0x4);
+ WREG32(offset + R600_HDMI_ENABLE, 0x105);
break;
default:
- radeon_encoder->hdmi_offset = 0;
+ dev_err(rdev->dev, "Unknown HDMI output type\n");
break;
}
- case ENCODER_OBJECT_ID_INTERNAL_UNIPHY2:
- radeon_encoder->hdmi_offset = R600_HDMI_TMDS2;
- break;
+ }
- case ENCODER_OBJECT_ID_INTERNAL_KLDSCP_LVTMA:
- radeon_encoder->hdmi_offset = R600_HDMI_DIG;
- break;
+ DRM_DEBUG("Enabling HDMI interface @ 0x%04X for encoder 0x%x\n",
+ radeon_encoder->hdmi_offset, radeon_encoder->encoder_id);
+}
- default:
- radeon_encoder->hdmi_offset = 0;
- break;
+/*
+ * disable the HDMI engine
+ */
+void r600_hdmi_disable(struct drm_encoder *encoder)
+{
+ struct drm_device *dev = encoder->dev;
+ struct radeon_device *rdev = dev->dev_private;
+ struct radeon_encoder *radeon_encoder = to_radeon_encoder(encoder);
+
+ if (!radeon_encoder->hdmi_offset) {
+ dev_err(rdev->dev, "Disabling not enabled HDMI\n");
+ return;
}
- DRM_DEBUG("using HDMI engine at offset 0x%04X for encoder 0x%x\n",
- radeon_encoder->hdmi_offset, radeon_encoder->encoder_id);
+ DRM_DEBUG("Disabling HDMI interface @ 0x%04X for encoder 0x%x\n",
+ radeon_encoder->hdmi_offset, radeon_encoder->encoder_id);
+
+ if (ASIC_IS_DCE32(rdev) && !ASIC_IS_DCE4(rdev)) {
+ WREG32_P(radeon_encoder->hdmi_config_offset + 0x4, 0, ~0x1);
+ } else if (rdev->family >= CHIP_R600 && !ASIC_IS_DCE3(rdev)) {
+ int offset = radeon_encoder->hdmi_offset;
+ switch (radeon_encoder->encoder_id) {
+ case ENCODER_OBJECT_ID_INTERNAL_KLDSCP_TMDS1:
+ WREG32_P(AVIVO_TMDSA_CNTL, 0, ~0x4);
+ WREG32(offset + R600_HDMI_ENABLE, 0);
+ break;
+ case ENCODER_OBJECT_ID_INTERNAL_LVTM1:
+ WREG32_P(AVIVO_LVTMA_CNTL, 0, ~0x4);
+ WREG32(offset + R600_HDMI_ENABLE, 0);
+ break;
+ default:
+ dev_err(rdev->dev, "Unknown HDMI output type\n");
+ break;
+ }
+ }
- /* TODO: make this configureable */
- radeon_encoder->hdmi_audio_workaround = 0;
+ radeon_encoder->hdmi_offset = 0;
+ radeon_encoder->hdmi_config_offset = 0;
}
#define R600_AUDIO_STATUS_BITS 0x73d8
/* HDMI base register addresses */
-#define R600_HDMI_TMDS1 0x7400
-#define R600_HDMI_TMDS2 0x7700
-#define R600_HDMI_DIG 0x7800
+#define R600_HDMI_BLOCK1 0x7400
+#define R600_HDMI_BLOCK2 0x7700
+#define R600_HDMI_BLOCK3 0x7800
/* HDMI registers */
#define R600_HDMI_ENABLE 0x00
#define R600_HDMI_AUDIO_DEBUG_2 0xe8
#define R600_HDMI_AUDIO_DEBUG_3 0xec
+/* HDMI additional config base register addresses */
+#define R600_HDMI_CONFIG1 0x7600
+#define R600_HDMI_CONFIG2 0x7a00
+
#endif
#define CB_COLOR0_FRAG 0x280e0
#define CB_COLOR0_MASK 0x28100
+#define SQ_ALU_CONST_CACHE_PS_0 0x28940
+#define SQ_ALU_CONST_CACHE_PS_1 0x28944
+#define SQ_ALU_CONST_CACHE_PS_2 0x28948
+#define SQ_ALU_CONST_CACHE_PS_3 0x2894c
+#define SQ_ALU_CONST_CACHE_PS_4 0x28950
+#define SQ_ALU_CONST_CACHE_PS_5 0x28954
+#define SQ_ALU_CONST_CACHE_PS_6 0x28958
+#define SQ_ALU_CONST_CACHE_PS_7 0x2895c
+#define SQ_ALU_CONST_CACHE_PS_8 0x28960
+#define SQ_ALU_CONST_CACHE_PS_9 0x28964
+#define SQ_ALU_CONST_CACHE_PS_10 0x28968
+#define SQ_ALU_CONST_CACHE_PS_11 0x2896c
+#define SQ_ALU_CONST_CACHE_PS_12 0x28970
+#define SQ_ALU_CONST_CACHE_PS_13 0x28974
+#define SQ_ALU_CONST_CACHE_PS_14 0x28978
+#define SQ_ALU_CONST_CACHE_PS_15 0x2897c
+#define SQ_ALU_CONST_CACHE_VS_0 0x28980
+#define SQ_ALU_CONST_CACHE_VS_1 0x28984
+#define SQ_ALU_CONST_CACHE_VS_2 0x28988
+#define SQ_ALU_CONST_CACHE_VS_3 0x2898c
+#define SQ_ALU_CONST_CACHE_VS_4 0x28990
+#define SQ_ALU_CONST_CACHE_VS_5 0x28994
+#define SQ_ALU_CONST_CACHE_VS_6 0x28998
+#define SQ_ALU_CONST_CACHE_VS_7 0x2899c
+#define SQ_ALU_CONST_CACHE_VS_8 0x289a0
+#define SQ_ALU_CONST_CACHE_VS_9 0x289a4
+#define SQ_ALU_CONST_CACHE_VS_10 0x289a8
+#define SQ_ALU_CONST_CACHE_VS_11 0x289ac
+#define SQ_ALU_CONST_CACHE_VS_12 0x289b0
+#define SQ_ALU_CONST_CACHE_VS_13 0x289b4
+#define SQ_ALU_CONST_CACHE_VS_14 0x289b8
+#define SQ_ALU_CONST_CACHE_VS_15 0x289bc
+#define SQ_ALU_CONST_CACHE_GS_0 0x289c0
+#define SQ_ALU_CONST_CACHE_GS_1 0x289c4
+#define SQ_ALU_CONST_CACHE_GS_2 0x289c8
+#define SQ_ALU_CONST_CACHE_GS_3 0x289cc
+#define SQ_ALU_CONST_CACHE_GS_4 0x289d0
+#define SQ_ALU_CONST_CACHE_GS_5 0x289d4
+#define SQ_ALU_CONST_CACHE_GS_6 0x289d8
+#define SQ_ALU_CONST_CACHE_GS_7 0x289dc
+#define SQ_ALU_CONST_CACHE_GS_8 0x289e0
+#define SQ_ALU_CONST_CACHE_GS_9 0x289e4
+#define SQ_ALU_CONST_CACHE_GS_10 0x289e8
+#define SQ_ALU_CONST_CACHE_GS_11 0x289ec
+#define SQ_ALU_CONST_CACHE_GS_12 0x289f0
+#define SQ_ALU_CONST_CACHE_GS_13 0x289f4
+#define SQ_ALU_CONST_CACHE_GS_14 0x289f8
+#define SQ_ALU_CONST_CACHE_GS_15 0x289fc
+
#define CONFIG_MEMSIZE 0x5428
#define CONFIG_CNTL 0x5424
#define CP_STAT 0x8680
extern int radeon_new_pll;
extern int radeon_dynpm;
extern int radeon_audio;
+extern int radeon_disp_priority;
+extern int radeon_hw_i2c;
/*
* Copy from radeon_drv.h so we don't have to include both and have conflicting
* Power management
*/
int radeon_pm_init(struct radeon_device *rdev);
+void radeon_pm_fini(struct radeon_device *rdev);
void radeon_pm_compute_clocks(struct radeon_device *rdev);
void radeon_combios_get_power_modes(struct radeon_device *rdev);
void radeon_atombios_get_power_modes(struct radeon_device *rdev);
bool downclocked;
int active_crtcs;
int req_vblank;
+ bool vblank_sync;
fixed20_12 max_bandwidth;
fixed20_12 igp_sideport_mclk;
fixed20_12 igp_system_mclk;
fixed20_12 ht_bandwidth;
fixed20_12 core_bandwidth;
fixed20_12 sclk;
+ fixed20_12 mclk;
fixed20_12 needed_bandwidth;
/* XXX: use a define for num power modes */
struct radeon_power_state power_state[8];
struct radeon_power_state *requested_power_state;
struct radeon_pm_clock_info *requested_clock_mode;
struct radeon_power_state *default_power_state;
+ struct radeon_i2c_chan *i2c_bus;
};
struct drm_info_list *files,
unsigned nfiles);
int radeon_debugfs_fence_init(struct radeon_device *rdev);
-int r100_debugfs_rbbm_init(struct radeon_device *rdev);
-int r100_debugfs_cp_init(struct radeon_device *rdev);
/*
int (*set_surface_reg)(struct radeon_device *rdev, int reg,
uint32_t tiling_flags, uint32_t pitch,
uint32_t offset, uint32_t obj_size);
- int (*clear_surface_reg)(struct radeon_device *rdev, int reg);
+ void (*clear_surface_reg)(struct radeon_device *rdev, int reg);
void (*bandwidth_update)(struct radeon_device *rdev);
void (*hpd_init)(struct radeon_device *rdev);
void (*hpd_fini)(struct radeon_device *rdev);
struct rv770_asic rv770;
};
+/*
+ * asic initizalization from radeon_asic.c
+ */
+void radeon_agp_disable(struct radeon_device *rdev);
+int radeon_asic_init(struct radeon_device *rdev);
+
/*
* IOCTL.
extern int radeon_modeset_init(struct radeon_device *rdev);
extern void radeon_modeset_fini(struct radeon_device *rdev);
extern bool radeon_card_posted(struct radeon_device *rdev);
+extern void radeon_update_bandwidth_info(struct radeon_device *rdev);
+extern void radeon_update_display_priority(struct radeon_device *rdev);
extern bool radeon_boot_test_post_card(struct radeon_device *rdev);
extern int radeon_clocks_init(struct radeon_device *rdev);
extern void radeon_clocks_fini(struct radeon_device *rdev);
extern int radeon_suspend_kms(struct drm_device *dev, pm_message_t state);
/* r100,rv100,rs100,rv200,rs200,r200,rv250,rs300,rv280 */
-struct r100_mc_save {
- u32 GENMO_WT;
- u32 CRTC_EXT_CNTL;
- u32 CRTC_GEN_CNTL;
- u32 CRTC2_GEN_CNTL;
- u32 CUR_OFFSET;
- u32 CUR2_OFFSET;
-};
-extern void r100_cp_disable(struct radeon_device *rdev);
-extern int r100_cp_init(struct radeon_device *rdev, unsigned ring_size);
-extern void r100_cp_fini(struct radeon_device *rdev);
-extern void r100_pci_gart_tlb_flush(struct radeon_device *rdev);
-extern int r100_pci_gart_init(struct radeon_device *rdev);
-extern void r100_pci_gart_fini(struct radeon_device *rdev);
-extern int r100_pci_gart_enable(struct radeon_device *rdev);
-extern void r100_pci_gart_disable(struct radeon_device *rdev);
-extern int r100_pci_gart_set_page(struct radeon_device *rdev, int i, uint64_t addr);
-extern int r100_debugfs_mc_info_init(struct radeon_device *rdev);
-extern int r100_gui_wait_for_idle(struct radeon_device *rdev);
-extern void r100_ib_fini(struct radeon_device *rdev);
-extern int r100_ib_init(struct radeon_device *rdev);
-extern void r100_irq_disable(struct radeon_device *rdev);
-extern int r100_irq_set(struct radeon_device *rdev);
-extern void r100_mc_stop(struct radeon_device *rdev, struct r100_mc_save *save);
-extern void r100_mc_resume(struct radeon_device *rdev, struct r100_mc_save *save);
-extern void r100_vram_init_sizes(struct radeon_device *rdev);
-extern void r100_wb_disable(struct radeon_device *rdev);
-extern void r100_wb_fini(struct radeon_device *rdev);
-extern int r100_wb_init(struct radeon_device *rdev);
-extern void r100_hdp_reset(struct radeon_device *rdev);
-extern int r100_rb2d_reset(struct radeon_device *rdev);
-extern int r100_cp_reset(struct radeon_device *rdev);
-extern void r100_vga_render_disable(struct radeon_device *rdev);
-extern int r100_cs_track_check_pkt3_indx_buffer(struct radeon_cs_parser *p,
- struct radeon_cs_packet *pkt,
- struct radeon_bo *robj);
-extern int r100_cs_parse_packet0(struct radeon_cs_parser *p,
- struct radeon_cs_packet *pkt,
- const unsigned *auth, unsigned n,
- radeon_packet0_check_t check);
-extern int r100_cs_packet_parse(struct radeon_cs_parser *p,
- struct radeon_cs_packet *pkt,
- unsigned idx);
-extern void r100_enable_bm(struct radeon_device *rdev);
-extern void r100_set_common_regs(struct radeon_device *rdev);
/* rv200,rv250,rv280 */
extern void r200_set_safe_registers(struct radeon_device *rdev);
extern void r600_audio_set_clock(struct drm_encoder *encoder, int clock);
extern void r600_audio_fini(struct radeon_device *rdev);
extern void r600_hdmi_init(struct drm_encoder *encoder);
-extern void r600_hdmi_enable(struct drm_encoder *encoder, int enable);
+extern void r600_hdmi_enable(struct drm_encoder *encoder);
+extern void r600_hdmi_disable(struct drm_encoder *encoder);
extern void r600_hdmi_setmode(struct drm_encoder *encoder, struct drm_display_mode *mode);
extern int r600_hdmi_buffer_status_changed(struct drm_encoder *encoder);
extern void r600_hdmi_update_audio_settings(struct drm_encoder *encoder,
--- /dev/null
+/*
+ * Copyright 2008 Advanced Micro Devices, Inc.
+ * Copyright 2008 Red Hat Inc.
+ * Copyright 2009 Jerome Glisse.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: Dave Airlie
+ * Alex Deucher
+ * Jerome Glisse
+ */
+
+#include <linux/console.h>
+#include <drm/drmP.h>
+#include <drm/drm_crtc_helper.h>
+#include <drm/radeon_drm.h>
+#include <linux/vgaarb.h>
+#include <linux/vga_switcheroo.h>
+#include "radeon_reg.h"
+#include "radeon.h"
+#include "radeon_asic.h"
+#include "atom.h"
+
+/*
+ * Registers accessors functions.
+ */
+static uint32_t radeon_invalid_rreg(struct radeon_device *rdev, uint32_t reg)
+{
+ DRM_ERROR("Invalid callback to read register 0x%04X\n", reg);
+ BUG_ON(1);
+ return 0;
+}
+
+static void radeon_invalid_wreg(struct radeon_device *rdev, uint32_t reg, uint32_t v)
+{
+ DRM_ERROR("Invalid callback to write register 0x%04X with 0x%08X\n",
+ reg, v);
+ BUG_ON(1);
+}
+
+static void radeon_register_accessor_init(struct radeon_device *rdev)
+{
+ rdev->mc_rreg = &radeon_invalid_rreg;
+ rdev->mc_wreg = &radeon_invalid_wreg;
+ rdev->pll_rreg = &radeon_invalid_rreg;
+ rdev->pll_wreg = &radeon_invalid_wreg;
+ rdev->pciep_rreg = &radeon_invalid_rreg;
+ rdev->pciep_wreg = &radeon_invalid_wreg;
+
+ /* Don't change order as we are overridding accessor. */
+ if (rdev->family < CHIP_RV515) {
+ rdev->pcie_reg_mask = 0xff;
+ } else {
+ rdev->pcie_reg_mask = 0x7ff;
+ }
+ /* FIXME: not sure here */
+ if (rdev->family <= CHIP_R580) {
+ rdev->pll_rreg = &r100_pll_rreg;
+ rdev->pll_wreg = &r100_pll_wreg;
+ }
+ if (rdev->family >= CHIP_R420) {
+ rdev->mc_rreg = &r420_mc_rreg;
+ rdev->mc_wreg = &r420_mc_wreg;
+ }
+ if (rdev->family >= CHIP_RV515) {
+ rdev->mc_rreg = &rv515_mc_rreg;
+ rdev->mc_wreg = &rv515_mc_wreg;
+ }
+ if (rdev->family == CHIP_RS400 || rdev->family == CHIP_RS480) {
+ rdev->mc_rreg = &rs400_mc_rreg;
+ rdev->mc_wreg = &rs400_mc_wreg;
+ }
+ if (rdev->family == CHIP_RS690 || rdev->family == CHIP_RS740) {
+ rdev->mc_rreg = &rs690_mc_rreg;
+ rdev->mc_wreg = &rs690_mc_wreg;
+ }
+ if (rdev->family == CHIP_RS600) {
+ rdev->mc_rreg = &rs600_mc_rreg;
+ rdev->mc_wreg = &rs600_mc_wreg;
+ }
+ if ((rdev->family >= CHIP_R600) && (rdev->family <= CHIP_RV740)) {
+ rdev->pciep_rreg = &r600_pciep_rreg;
+ rdev->pciep_wreg = &r600_pciep_wreg;
+ }
+}
+
+
+/* helper to disable agp */
+void radeon_agp_disable(struct radeon_device *rdev)
+{
+ rdev->flags &= ~RADEON_IS_AGP;
+ if (rdev->family >= CHIP_R600) {
+ DRM_INFO("Forcing AGP to PCIE mode\n");
+ rdev->flags |= RADEON_IS_PCIE;
+ } else if (rdev->family >= CHIP_RV515 ||
+ rdev->family == CHIP_RV380 ||
+ rdev->family == CHIP_RV410 ||
+ rdev->family == CHIP_R423) {
+ DRM_INFO("Forcing AGP to PCIE mode\n");
+ rdev->flags |= RADEON_IS_PCIE;
+ rdev->asic->gart_tlb_flush = &rv370_pcie_gart_tlb_flush;
+ rdev->asic->gart_set_page = &rv370_pcie_gart_set_page;
+ } else {
+ DRM_INFO("Forcing AGP to PCI mode\n");
+ rdev->flags |= RADEON_IS_PCI;
+ rdev->asic->gart_tlb_flush = &r100_pci_gart_tlb_flush;
+ rdev->asic->gart_set_page = &r100_pci_gart_set_page;
+ }
+ rdev->mc.gtt_size = radeon_gart_size * 1024 * 1024;
+}
+
+/*
+ * ASIC
+ */
+static struct radeon_asic r100_asic = {
+ .init = &r100_init,
+ .fini = &r100_fini,
+ .suspend = &r100_suspend,
+ .resume = &r100_resume,
+ .vga_set_state = &r100_vga_set_state,
+ .gpu_reset = &r100_gpu_reset,
+ .gart_tlb_flush = &r100_pci_gart_tlb_flush,
+ .gart_set_page = &r100_pci_gart_set_page,
+ .cp_commit = &r100_cp_commit,
+ .ring_start = &r100_ring_start,
+ .ring_test = &r100_ring_test,
+ .ring_ib_execute = &r100_ring_ib_execute,
+ .irq_set = &r100_irq_set,
+ .irq_process = &r100_irq_process,
+ .get_vblank_counter = &r100_get_vblank_counter,
+ .fence_ring_emit = &r100_fence_ring_emit,
+ .cs_parse = &r100_cs_parse,
+ .copy_blit = &r100_copy_blit,
+ .copy_dma = NULL,
+ .copy = &r100_copy_blit,
+ .get_engine_clock = &radeon_legacy_get_engine_clock,
+ .set_engine_clock = &radeon_legacy_set_engine_clock,
+ .get_memory_clock = &radeon_legacy_get_memory_clock,
+ .set_memory_clock = NULL,
+ .get_pcie_lanes = NULL,
+ .set_pcie_lanes = NULL,
+ .set_clock_gating = &radeon_legacy_set_clock_gating,
+ .set_surface_reg = r100_set_surface_reg,
+ .clear_surface_reg = r100_clear_surface_reg,
+ .bandwidth_update = &r100_bandwidth_update,
+ .hpd_init = &r100_hpd_init,
+ .hpd_fini = &r100_hpd_fini,
+ .hpd_sense = &r100_hpd_sense,
+ .hpd_set_polarity = &r100_hpd_set_polarity,
+ .ioctl_wait_idle = NULL,
+};
+
+static struct radeon_asic r200_asic = {
+ .init = &r100_init,
+ .fini = &r100_fini,
+ .suspend = &r100_suspend,
+ .resume = &r100_resume,
+ .vga_set_state = &r100_vga_set_state,
+ .gpu_reset = &r100_gpu_reset,
+ .gart_tlb_flush = &r100_pci_gart_tlb_flush,
+ .gart_set_page = &r100_pci_gart_set_page,
+ .cp_commit = &r100_cp_commit,
+ .ring_start = &r100_ring_start,
+ .ring_test = &r100_ring_test,
+ .ring_ib_execute = &r100_ring_ib_execute,
+ .irq_set = &r100_irq_set,
+ .irq_process = &r100_irq_process,
+ .get_vblank_counter = &r100_get_vblank_counter,
+ .fence_ring_emit = &r100_fence_ring_emit,
+ .cs_parse = &r100_cs_parse,
+ .copy_blit = &r100_copy_blit,
+ .copy_dma = &r200_copy_dma,
+ .copy = &r100_copy_blit,
+ .get_engine_clock = &radeon_legacy_get_engine_clock,
+ .set_engine_clock = &radeon_legacy_set_engine_clock,
+ .get_memory_clock = &radeon_legacy_get_memory_clock,
+ .set_memory_clock = NULL,
+ .set_pcie_lanes = NULL,
+ .set_clock_gating = &radeon_legacy_set_clock_gating,
+ .set_surface_reg = r100_set_surface_reg,
+ .clear_surface_reg = r100_clear_surface_reg,
+ .bandwidth_update = &r100_bandwidth_update,
+ .hpd_init = &r100_hpd_init,
+ .hpd_fini = &r100_hpd_fini,
+ .hpd_sense = &r100_hpd_sense,
+ .hpd_set_polarity = &r100_hpd_set_polarity,
+ .ioctl_wait_idle = NULL,
+};
+
+static struct radeon_asic r300_asic = {
+ .init = &r300_init,
+ .fini = &r300_fini,
+ .suspend = &r300_suspend,
+ .resume = &r300_resume,
+ .vga_set_state = &r100_vga_set_state,
+ .gpu_reset = &r300_gpu_reset,
+ .gart_tlb_flush = &r100_pci_gart_tlb_flush,
+ .gart_set_page = &r100_pci_gart_set_page,
+ .cp_commit = &r100_cp_commit,
+ .ring_start = &r300_ring_start,
+ .ring_test = &r100_ring_test,
+ .ring_ib_execute = &r100_ring_ib_execute,
+ .irq_set = &r100_irq_set,
+ .irq_process = &r100_irq_process,
+ .get_vblank_counter = &r100_get_vblank_counter,
+ .fence_ring_emit = &r300_fence_ring_emit,
+ .cs_parse = &r300_cs_parse,
+ .copy_blit = &r100_copy_blit,
+ .copy_dma = &r200_copy_dma,
+ .copy = &r100_copy_blit,
+ .get_engine_clock = &radeon_legacy_get_engine_clock,
+ .set_engine_clock = &radeon_legacy_set_engine_clock,
+ .get_memory_clock = &radeon_legacy_get_memory_clock,
+ .set_memory_clock = NULL,
+ .get_pcie_lanes = &rv370_get_pcie_lanes,
+ .set_pcie_lanes = &rv370_set_pcie_lanes,
+ .set_clock_gating = &radeon_legacy_set_clock_gating,
+ .set_surface_reg = r100_set_surface_reg,
+ .clear_surface_reg = r100_clear_surface_reg,
+ .bandwidth_update = &r100_bandwidth_update,
+ .hpd_init = &r100_hpd_init,
+ .hpd_fini = &r100_hpd_fini,
+ .hpd_sense = &r100_hpd_sense,
+ .hpd_set_polarity = &r100_hpd_set_polarity,
+ .ioctl_wait_idle = NULL,
+};
+
+static struct radeon_asic r300_asic_pcie = {
+ .init = &r300_init,
+ .fini = &r300_fini,
+ .suspend = &r300_suspend,
+ .resume = &r300_resume,
+ .vga_set_state = &r100_vga_set_state,
+ .gpu_reset = &r300_gpu_reset,
+ .gart_tlb_flush = &rv370_pcie_gart_tlb_flush,
+ .gart_set_page = &rv370_pcie_gart_set_page,
+ .cp_commit = &r100_cp_commit,
+ .ring_start = &r300_ring_start,
+ .ring_test = &r100_ring_test,
+ .ring_ib_execute = &r100_ring_ib_execute,
+ .irq_set = &r100_irq_set,
+ .irq_process = &r100_irq_process,
+ .get_vblank_counter = &r100_get_vblank_counter,
+ .fence_ring_emit = &r300_fence_ring_emit,
+ .cs_parse = &r300_cs_parse,
+ .copy_blit = &r100_copy_blit,
+ .copy_dma = &r200_copy_dma,
+ .copy = &r100_copy_blit,
+ .get_engine_clock = &radeon_legacy_get_engine_clock,
+ .set_engine_clock = &radeon_legacy_set_engine_clock,
+ .get_memory_clock = &radeon_legacy_get_memory_clock,
+ .set_memory_clock = NULL,
+ .set_pcie_lanes = &rv370_set_pcie_lanes,
+ .set_clock_gating = &radeon_legacy_set_clock_gating,
+ .set_surface_reg = r100_set_surface_reg,
+ .clear_surface_reg = r100_clear_surface_reg,
+ .bandwidth_update = &r100_bandwidth_update,
+ .hpd_init = &r100_hpd_init,
+ .hpd_fini = &r100_hpd_fini,
+ .hpd_sense = &r100_hpd_sense,
+ .hpd_set_polarity = &r100_hpd_set_polarity,
+ .ioctl_wait_idle = NULL,
+};
+
+static struct radeon_asic r420_asic = {
+ .init = &r420_init,
+ .fini = &r420_fini,
+ .suspend = &r420_suspend,
+ .resume = &r420_resume,
+ .vga_set_state = &r100_vga_set_state,
+ .gpu_reset = &r300_gpu_reset,
+ .gart_tlb_flush = &rv370_pcie_gart_tlb_flush,
+ .gart_set_page = &rv370_pcie_gart_set_page,
+ .cp_commit = &r100_cp_commit,
+ .ring_start = &r300_ring_start,
+ .ring_test = &r100_ring_test,
+ .ring_ib_execute = &r100_ring_ib_execute,
+ .irq_set = &r100_irq_set,
+ .irq_process = &r100_irq_process,
+ .get_vblank_counter = &r100_get_vblank_counter,
+ .fence_ring_emit = &r300_fence_ring_emit,
+ .cs_parse = &r300_cs_parse,
+ .copy_blit = &r100_copy_blit,
+ .copy_dma = &r200_copy_dma,
+ .copy = &r100_copy_blit,
+ .get_engine_clock = &radeon_atom_get_engine_clock,
+ .set_engine_clock = &radeon_atom_set_engine_clock,
+ .get_memory_clock = &radeon_atom_get_memory_clock,
+ .set_memory_clock = &radeon_atom_set_memory_clock,
+ .get_pcie_lanes = &rv370_get_pcie_lanes,
+ .set_pcie_lanes = &rv370_set_pcie_lanes,
+ .set_clock_gating = &radeon_atom_set_clock_gating,
+ .set_surface_reg = r100_set_surface_reg,
+ .clear_surface_reg = r100_clear_surface_reg,
+ .bandwidth_update = &r100_bandwidth_update,
+ .hpd_init = &r100_hpd_init,
+ .hpd_fini = &r100_hpd_fini,
+ .hpd_sense = &r100_hpd_sense,
+ .hpd_set_polarity = &r100_hpd_set_polarity,
+ .ioctl_wait_idle = NULL,
+};
+
+static struct radeon_asic rs400_asic = {
+ .init = &rs400_init,
+ .fini = &rs400_fini,
+ .suspend = &rs400_suspend,
+ .resume = &rs400_resume,
+ .vga_set_state = &r100_vga_set_state,
+ .gpu_reset = &r300_gpu_reset,
+ .gart_tlb_flush = &rs400_gart_tlb_flush,
+ .gart_set_page = &rs400_gart_set_page,
+ .cp_commit = &r100_cp_commit,
+ .ring_start = &r300_ring_start,
+ .ring_test = &r100_ring_test,
+ .ring_ib_execute = &r100_ring_ib_execute,
+ .irq_set = &r100_irq_set,
+ .irq_process = &r100_irq_process,
+ .get_vblank_counter = &r100_get_vblank_counter,
+ .fence_ring_emit = &r300_fence_ring_emit,
+ .cs_parse = &r300_cs_parse,
+ .copy_blit = &r100_copy_blit,
+ .copy_dma = &r200_copy_dma,
+ .copy = &r100_copy_blit,
+ .get_engine_clock = &radeon_legacy_get_engine_clock,
+ .set_engine_clock = &radeon_legacy_set_engine_clock,
+ .get_memory_clock = &radeon_legacy_get_memory_clock,
+ .set_memory_clock = NULL,
+ .get_pcie_lanes = NULL,
+ .set_pcie_lanes = NULL,
+ .set_clock_gating = &radeon_legacy_set_clock_gating,
+ .set_surface_reg = r100_set_surface_reg,
+ .clear_surface_reg = r100_clear_surface_reg,
+ .bandwidth_update = &r100_bandwidth_update,
+ .hpd_init = &r100_hpd_init,
+ .hpd_fini = &r100_hpd_fini,
+ .hpd_sense = &r100_hpd_sense,
+ .hpd_set_polarity = &r100_hpd_set_polarity,
+ .ioctl_wait_idle = NULL,
+};
+
+static struct radeon_asic rs600_asic = {
+ .init = &rs600_init,
+ .fini = &rs600_fini,
+ .suspend = &rs600_suspend,
+ .resume = &rs600_resume,
+ .vga_set_state = &r100_vga_set_state,
+ .gpu_reset = &r300_gpu_reset,
+ .gart_tlb_flush = &rs600_gart_tlb_flush,
+ .gart_set_page = &rs600_gart_set_page,
+ .cp_commit = &r100_cp_commit,
+ .ring_start = &r300_ring_start,
+ .ring_test = &r100_ring_test,
+ .ring_ib_execute = &r100_ring_ib_execute,
+ .irq_set = &rs600_irq_set,
+ .irq_process = &rs600_irq_process,
+ .get_vblank_counter = &rs600_get_vblank_counter,
+ .fence_ring_emit = &r300_fence_ring_emit,
+ .cs_parse = &r300_cs_parse,
+ .copy_blit = &r100_copy_blit,
+ .copy_dma = &r200_copy_dma,
+ .copy = &r100_copy_blit,
+ .get_engine_clock = &radeon_atom_get_engine_clock,
+ .set_engine_clock = &radeon_atom_set_engine_clock,
+ .get_memory_clock = &radeon_atom_get_memory_clock,
+ .set_memory_clock = &radeon_atom_set_memory_clock,
+ .get_pcie_lanes = NULL,
+ .set_pcie_lanes = NULL,
+ .set_clock_gating = &radeon_atom_set_clock_gating,
+ .set_surface_reg = r100_set_surface_reg,
+ .clear_surface_reg = r100_clear_surface_reg,
+ .bandwidth_update = &rs600_bandwidth_update,
+ .hpd_init = &rs600_hpd_init,
+ .hpd_fini = &rs600_hpd_fini,
+ .hpd_sense = &rs600_hpd_sense,
+ .hpd_set_polarity = &rs600_hpd_set_polarity,
+ .ioctl_wait_idle = NULL,
+};
+
+static struct radeon_asic rs690_asic = {
+ .init = &rs690_init,
+ .fini = &rs690_fini,
+ .suspend = &rs690_suspend,
+ .resume = &rs690_resume,
+ .vga_set_state = &r100_vga_set_state,
+ .gpu_reset = &r300_gpu_reset,
+ .gart_tlb_flush = &rs400_gart_tlb_flush,
+ .gart_set_page = &rs400_gart_set_page,
+ .cp_commit = &r100_cp_commit,
+ .ring_start = &r300_ring_start,
+ .ring_test = &r100_ring_test,
+ .ring_ib_execute = &r100_ring_ib_execute,
+ .irq_set = &rs600_irq_set,
+ .irq_process = &rs600_irq_process,
+ .get_vblank_counter = &rs600_get_vblank_counter,
+ .fence_ring_emit = &r300_fence_ring_emit,
+ .cs_parse = &r300_cs_parse,
+ .copy_blit = &r100_copy_blit,
+ .copy_dma = &r200_copy_dma,
+ .copy = &r200_copy_dma,
+ .get_engine_clock = &radeon_atom_get_engine_clock,
+ .set_engine_clock = &radeon_atom_set_engine_clock,
+ .get_memory_clock = &radeon_atom_get_memory_clock,
+ .set_memory_clock = &radeon_atom_set_memory_clock,
+ .get_pcie_lanes = NULL,
+ .set_pcie_lanes = NULL,
+ .set_clock_gating = &radeon_atom_set_clock_gating,
+ .set_surface_reg = r100_set_surface_reg,
+ .clear_surface_reg = r100_clear_surface_reg,
+ .bandwidth_update = &rs690_bandwidth_update,
+ .hpd_init = &rs600_hpd_init,
+ .hpd_fini = &rs600_hpd_fini,
+ .hpd_sense = &rs600_hpd_sense,
+ .hpd_set_polarity = &rs600_hpd_set_polarity,
+ .ioctl_wait_idle = NULL,
+};
+
+static struct radeon_asic rv515_asic = {
+ .init = &rv515_init,
+ .fini = &rv515_fini,
+ .suspend = &rv515_suspend,
+ .resume = &rv515_resume,
+ .vga_set_state = &r100_vga_set_state,
+ .gpu_reset = &rv515_gpu_reset,
+ .gart_tlb_flush = &rv370_pcie_gart_tlb_flush,
+ .gart_set_page = &rv370_pcie_gart_set_page,
+ .cp_commit = &r100_cp_commit,
+ .ring_start = &rv515_ring_start,
+ .ring_test = &r100_ring_test,
+ .ring_ib_execute = &r100_ring_ib_execute,
+ .irq_set = &rs600_irq_set,
+ .irq_process = &rs600_irq_process,
+ .get_vblank_counter = &rs600_get_vblank_counter,
+ .fence_ring_emit = &r300_fence_ring_emit,
+ .cs_parse = &r300_cs_parse,
+ .copy_blit = &r100_copy_blit,
+ .copy_dma = &r200_copy_dma,
+ .copy = &r100_copy_blit,
+ .get_engine_clock = &radeon_atom_get_engine_clock,
+ .set_engine_clock = &radeon_atom_set_engine_clock,
+ .get_memory_clock = &radeon_atom_get_memory_clock,
+ .set_memory_clock = &radeon_atom_set_memory_clock,
+ .get_pcie_lanes = &rv370_get_pcie_lanes,
+ .set_pcie_lanes = &rv370_set_pcie_lanes,
+ .set_clock_gating = &radeon_atom_set_clock_gating,
+ .set_surface_reg = r100_set_surface_reg,
+ .clear_surface_reg = r100_clear_surface_reg,
+ .bandwidth_update = &rv515_bandwidth_update,
+ .hpd_init = &rs600_hpd_init,
+ .hpd_fini = &rs600_hpd_fini,
+ .hpd_sense = &rs600_hpd_sense,
+ .hpd_set_polarity = &rs600_hpd_set_polarity,
+ .ioctl_wait_idle = NULL,
+};
+
+static struct radeon_asic r520_asic = {
+ .init = &r520_init,
+ .fini = &rv515_fini,
+ .suspend = &rv515_suspend,
+ .resume = &r520_resume,
+ .vga_set_state = &r100_vga_set_state,
+ .gpu_reset = &rv515_gpu_reset,
+ .gart_tlb_flush = &rv370_pcie_gart_tlb_flush,
+ .gart_set_page = &rv370_pcie_gart_set_page,
+ .cp_commit = &r100_cp_commit,
+ .ring_start = &rv515_ring_start,
+ .ring_test = &r100_ring_test,
+ .ring_ib_execute = &r100_ring_ib_execute,
+ .irq_set = &rs600_irq_set,
+ .irq_process = &rs600_irq_process,
+ .get_vblank_counter = &rs600_get_vblank_counter,
+ .fence_ring_emit = &r300_fence_ring_emit,
+ .cs_parse = &r300_cs_parse,
+ .copy_blit = &r100_copy_blit,
+ .copy_dma = &r200_copy_dma,
+ .copy = &r100_copy_blit,
+ .get_engine_clock = &radeon_atom_get_engine_clock,
+ .set_engine_clock = &radeon_atom_set_engine_clock,
+ .get_memory_clock = &radeon_atom_get_memory_clock,
+ .set_memory_clock = &radeon_atom_set_memory_clock,
+ .get_pcie_lanes = &rv370_get_pcie_lanes,
+ .set_pcie_lanes = &rv370_set_pcie_lanes,
+ .set_clock_gating = &radeon_atom_set_clock_gating,
+ .set_surface_reg = r100_set_surface_reg,
+ .clear_surface_reg = r100_clear_surface_reg,
+ .bandwidth_update = &rv515_bandwidth_update,
+ .hpd_init = &rs600_hpd_init,
+ .hpd_fini = &rs600_hpd_fini,
+ .hpd_sense = &rs600_hpd_sense,
+ .hpd_set_polarity = &rs600_hpd_set_polarity,
+ .ioctl_wait_idle = NULL,
+};
+
+static struct radeon_asic r600_asic = {
+ .init = &r600_init,
+ .fini = &r600_fini,
+ .suspend = &r600_suspend,
+ .resume = &r600_resume,
+ .cp_commit = &r600_cp_commit,
+ .vga_set_state = &r600_vga_set_state,
+ .gpu_reset = &r600_gpu_reset,
+ .gart_tlb_flush = &r600_pcie_gart_tlb_flush,
+ .gart_set_page = &rs600_gart_set_page,
+ .ring_test = &r600_ring_test,
+ .ring_ib_execute = &r600_ring_ib_execute,
+ .irq_set = &r600_irq_set,
+ .irq_process = &r600_irq_process,
+ .get_vblank_counter = &rs600_get_vblank_counter,
+ .fence_ring_emit = &r600_fence_ring_emit,
+ .cs_parse = &r600_cs_parse,
+ .copy_blit = &r600_copy_blit,
+ .copy_dma = &r600_copy_blit,
+ .copy = &r600_copy_blit,
+ .get_engine_clock = &radeon_atom_get_engine_clock,
+ .set_engine_clock = &radeon_atom_set_engine_clock,
+ .get_memory_clock = &radeon_atom_get_memory_clock,
+ .set_memory_clock = &radeon_atom_set_memory_clock,
+ .get_pcie_lanes = &rv370_get_pcie_lanes,
+ .set_pcie_lanes = NULL,
+ .set_clock_gating = NULL,
+ .set_surface_reg = r600_set_surface_reg,
+ .clear_surface_reg = r600_clear_surface_reg,
+ .bandwidth_update = &rv515_bandwidth_update,
+ .hpd_init = &r600_hpd_init,
+ .hpd_fini = &r600_hpd_fini,
+ .hpd_sense = &r600_hpd_sense,
+ .hpd_set_polarity = &r600_hpd_set_polarity,
+ .ioctl_wait_idle = r600_ioctl_wait_idle,
+};
+
+static struct radeon_asic rs780_asic = {
+ .init = &r600_init,
+ .fini = &r600_fini,
+ .suspend = &r600_suspend,
+ .resume = &r600_resume,
+ .cp_commit = &r600_cp_commit,
+ .vga_set_state = &r600_vga_set_state,
+ .gpu_reset = &r600_gpu_reset,
+ .gart_tlb_flush = &r600_pcie_gart_tlb_flush,
+ .gart_set_page = &rs600_gart_set_page,
+ .ring_test = &r600_ring_test,
+ .ring_ib_execute = &r600_ring_ib_execute,
+ .irq_set = &r600_irq_set,
+ .irq_process = &r600_irq_process,
+ .get_vblank_counter = &rs600_get_vblank_counter,
+ .fence_ring_emit = &r600_fence_ring_emit,
+ .cs_parse = &r600_cs_parse,
+ .copy_blit = &r600_copy_blit,
+ .copy_dma = &r600_copy_blit,
+ .copy = &r600_copy_blit,
+ .get_engine_clock = &radeon_atom_get_engine_clock,
+ .set_engine_clock = &radeon_atom_set_engine_clock,
+ .get_memory_clock = NULL,
+ .set_memory_clock = NULL,
+ .get_pcie_lanes = NULL,
+ .set_pcie_lanes = NULL,
+ .set_clock_gating = NULL,
+ .set_surface_reg = r600_set_surface_reg,
+ .clear_surface_reg = r600_clear_surface_reg,
+ .bandwidth_update = &rs690_bandwidth_update,
+ .hpd_init = &r600_hpd_init,
+ .hpd_fini = &r600_hpd_fini,
+ .hpd_sense = &r600_hpd_sense,
+ .hpd_set_polarity = &r600_hpd_set_polarity,
+ .ioctl_wait_idle = r600_ioctl_wait_idle,
+};
+
+static struct radeon_asic rv770_asic = {
+ .init = &rv770_init,
+ .fini = &rv770_fini,
+ .suspend = &rv770_suspend,
+ .resume = &rv770_resume,
+ .cp_commit = &r600_cp_commit,
+ .gpu_reset = &rv770_gpu_reset,
+ .vga_set_state = &r600_vga_set_state,
+ .gart_tlb_flush = &r600_pcie_gart_tlb_flush,
+ .gart_set_page = &rs600_gart_set_page,
+ .ring_test = &r600_ring_test,
+ .ring_ib_execute = &r600_ring_ib_execute,
+ .irq_set = &r600_irq_set,
+ .irq_process = &r600_irq_process,
+ .get_vblank_counter = &rs600_get_vblank_counter,
+ .fence_ring_emit = &r600_fence_ring_emit,
+ .cs_parse = &r600_cs_parse,
+ .copy_blit = &r600_copy_blit,
+ .copy_dma = &r600_copy_blit,
+ .copy = &r600_copy_blit,
+ .get_engine_clock = &radeon_atom_get_engine_clock,
+ .set_engine_clock = &radeon_atom_set_engine_clock,
+ .get_memory_clock = &radeon_atom_get_memory_clock,
+ .set_memory_clock = &radeon_atom_set_memory_clock,
+ .get_pcie_lanes = &rv370_get_pcie_lanes,
+ .set_pcie_lanes = NULL,
+ .set_clock_gating = &radeon_atom_set_clock_gating,
+ .set_surface_reg = r600_set_surface_reg,
+ .clear_surface_reg = r600_clear_surface_reg,
+ .bandwidth_update = &rv515_bandwidth_update,
+ .hpd_init = &r600_hpd_init,
+ .hpd_fini = &r600_hpd_fini,
+ .hpd_sense = &r600_hpd_sense,
+ .hpd_set_polarity = &r600_hpd_set_polarity,
+ .ioctl_wait_idle = r600_ioctl_wait_idle,
+};
+
+static struct radeon_asic evergreen_asic = {
+ .init = &evergreen_init,
+ .fini = &evergreen_fini,
+ .suspend = &evergreen_suspend,
+ .resume = &evergreen_resume,
+ .cp_commit = NULL,
+ .gpu_reset = &evergreen_gpu_reset,
+ .vga_set_state = &r600_vga_set_state,
+ .gart_tlb_flush = &r600_pcie_gart_tlb_flush,
+ .gart_set_page = &rs600_gart_set_page,
+ .ring_test = NULL,
+ .ring_ib_execute = NULL,
+ .irq_set = NULL,
+ .irq_process = NULL,
+ .get_vblank_counter = NULL,
+ .fence_ring_emit = NULL,
+ .cs_parse = NULL,
+ .copy_blit = NULL,
+ .copy_dma = NULL,
+ .copy = NULL,
+ .get_engine_clock = &radeon_atom_get_engine_clock,
+ .set_engine_clock = &radeon_atom_set_engine_clock,
+ .get_memory_clock = &radeon_atom_get_memory_clock,
+ .set_memory_clock = &radeon_atom_set_memory_clock,
+ .set_pcie_lanes = NULL,
+ .set_clock_gating = NULL,
+ .set_surface_reg = r600_set_surface_reg,
+ .clear_surface_reg = r600_clear_surface_reg,
+ .bandwidth_update = &evergreen_bandwidth_update,
+ .hpd_init = &evergreen_hpd_init,
+ .hpd_fini = &evergreen_hpd_fini,
+ .hpd_sense = &evergreen_hpd_sense,
+ .hpd_set_polarity = &evergreen_hpd_set_polarity,
+};
+
+int radeon_asic_init(struct radeon_device *rdev)
+{
+ radeon_register_accessor_init(rdev);
+ switch (rdev->family) {
+ case CHIP_R100:
+ case CHIP_RV100:
+ case CHIP_RS100:
+ case CHIP_RV200:
+ case CHIP_RS200:
+ rdev->asic = &r100_asic;
+ break;
+ case CHIP_R200:
+ case CHIP_RV250:
+ case CHIP_RS300:
+ case CHIP_RV280:
+ rdev->asic = &r200_asic;
+ break;
+ case CHIP_R300:
+ case CHIP_R350:
+ case CHIP_RV350:
+ case CHIP_RV380:
+ if (rdev->flags & RADEON_IS_PCIE)
+ rdev->asic = &r300_asic_pcie;
+ else
+ rdev->asic = &r300_asic;
+ break;
+ case CHIP_R420:
+ case CHIP_R423:
+ case CHIP_RV410:
+ rdev->asic = &r420_asic;
+ break;
+ case CHIP_RS400:
+ case CHIP_RS480:
+ rdev->asic = &rs400_asic;
+ break;
+ case CHIP_RS600:
+ rdev->asic = &rs600_asic;
+ break;
+ case CHIP_RS690:
+ case CHIP_RS740:
+ rdev->asic = &rs690_asic;
+ break;
+ case CHIP_RV515:
+ rdev->asic = &rv515_asic;
+ break;
+ case CHIP_R520:
+ case CHIP_RV530:
+ case CHIP_RV560:
+ case CHIP_RV570:
+ case CHIP_R580:
+ rdev->asic = &r520_asic;
+ break;
+ case CHIP_R600:
+ case CHIP_RV610:
+ case CHIP_RV630:
+ case CHIP_RV620:
+ case CHIP_RV635:
+ case CHIP_RV670:
+ rdev->asic = &r600_asic;
+ break;
+ case CHIP_RS780:
+ case CHIP_RS880:
+ rdev->asic = &rs780_asic;
+ break;
+ case CHIP_RV770:
+ case CHIP_RV730:
+ case CHIP_RV710:
+ case CHIP_RV740:
+ rdev->asic = &rv770_asic;
+ break;
+ case CHIP_CEDAR:
+ case CHIP_REDWOOD:
+ case CHIP_JUNIPER:
+ case CHIP_CYPRESS:
+ case CHIP_HEMLOCK:
+ rdev->asic = &evergreen_asic;
+ break;
+ default:
+ /* FIXME: not supported yet */
+ return -EINVAL;
+ }
+
+ if (rdev->flags & RADEON_IS_IGP) {
+ rdev->asic->get_memory_clock = NULL;
+ rdev->asic->set_memory_clock = NULL;
+ }
+
+ /* set the number of crtcs */
+ if (rdev->flags & RADEON_SINGLE_CRTC)
+ rdev->num_crtc = 1;
+ else {
+ if (ASIC_IS_DCE4(rdev))
+ rdev->num_crtc = 6;
+ else
+ rdev->num_crtc = 2;
+ }
+
+ return 0;
+}
+
+/*
+ * Wrapper around modesetting bits. Move to radeon_clocks.c?
+ */
+int radeon_clocks_init(struct radeon_device *rdev)
+{
+ int r;
+
+ r = radeon_static_clocks_init(rdev->ddev);
+ if (r) {
+ return r;
+ }
+ DRM_INFO("Clocks initialized !\n");
+ return 0;
+}
+
+void radeon_clocks_fini(struct radeon_device *rdev)
+{
+}
/*
* r100,rv100,rs100,rv200,rs200
*/
-extern int r100_init(struct radeon_device *rdev);
-extern void r100_fini(struct radeon_device *rdev);
-extern int r100_suspend(struct radeon_device *rdev);
-extern int r100_resume(struct radeon_device *rdev);
+struct r100_mc_save {
+ u32 GENMO_WT;
+ u32 CRTC_EXT_CNTL;
+ u32 CRTC_GEN_CNTL;
+ u32 CRTC2_GEN_CNTL;
+ u32 CUR_OFFSET;
+ u32 CUR2_OFFSET;
+};
+int r100_init(struct radeon_device *rdev);
+void r100_fini(struct radeon_device *rdev);
+int r100_suspend(struct radeon_device *rdev);
+int r100_resume(struct radeon_device *rdev);
uint32_t r100_mm_rreg(struct radeon_device *rdev, uint32_t reg);
void r100_mm_wreg(struct radeon_device *rdev, uint32_t reg, uint32_t v);
void r100_vga_set_state(struct radeon_device *rdev, bool state);
int r100_set_surface_reg(struct radeon_device *rdev, int reg,
uint32_t tiling_flags, uint32_t pitch,
uint32_t offset, uint32_t obj_size);
-int r100_clear_surface_reg(struct radeon_device *rdev, int reg);
+void r100_clear_surface_reg(struct radeon_device *rdev, int reg);
void r100_bandwidth_update(struct radeon_device *rdev);
void r100_ring_ib_execute(struct radeon_device *rdev, struct radeon_ib *ib);
int r100_ring_test(struct radeon_device *rdev);
bool r100_hpd_sense(struct radeon_device *rdev, enum radeon_hpd_id hpd);
void r100_hpd_set_polarity(struct radeon_device *rdev,
enum radeon_hpd_id hpd);
-
-static struct radeon_asic r100_asic = {
- .init = &r100_init,
- .fini = &r100_fini,
- .suspend = &r100_suspend,
- .resume = &r100_resume,
- .vga_set_state = &r100_vga_set_state,
- .gpu_reset = &r100_gpu_reset,
- .gart_tlb_flush = &r100_pci_gart_tlb_flush,
- .gart_set_page = &r100_pci_gart_set_page,
- .cp_commit = &r100_cp_commit,
- .ring_start = &r100_ring_start,
- .ring_test = &r100_ring_test,
- .ring_ib_execute = &r100_ring_ib_execute,
- .irq_set = &r100_irq_set,
- .irq_process = &r100_irq_process,
- .get_vblank_counter = &r100_get_vblank_counter,
- .fence_ring_emit = &r100_fence_ring_emit,
- .cs_parse = &r100_cs_parse,
- .copy_blit = &r100_copy_blit,
- .copy_dma = NULL,
- .copy = &r100_copy_blit,
- .get_engine_clock = &radeon_legacy_get_engine_clock,
- .set_engine_clock = &radeon_legacy_set_engine_clock,
- .get_memory_clock = &radeon_legacy_get_memory_clock,
- .set_memory_clock = NULL,
- .get_pcie_lanes = NULL,
- .set_pcie_lanes = NULL,
- .set_clock_gating = &radeon_legacy_set_clock_gating,
- .set_surface_reg = r100_set_surface_reg,
- .clear_surface_reg = r100_clear_surface_reg,
- .bandwidth_update = &r100_bandwidth_update,
- .hpd_init = &r100_hpd_init,
- .hpd_fini = &r100_hpd_fini,
- .hpd_sense = &r100_hpd_sense,
- .hpd_set_polarity = &r100_hpd_set_polarity,
- .ioctl_wait_idle = NULL,
-};
+int r100_debugfs_rbbm_init(struct radeon_device *rdev);
+int r100_debugfs_cp_init(struct radeon_device *rdev);
+void r100_cp_disable(struct radeon_device *rdev);
+int r100_cp_init(struct radeon_device *rdev, unsigned ring_size);
+void r100_cp_fini(struct radeon_device *rdev);
+int r100_pci_gart_init(struct radeon_device *rdev);
+void r100_pci_gart_fini(struct radeon_device *rdev);
+int r100_pci_gart_enable(struct radeon_device *rdev);
+void r100_pci_gart_disable(struct radeon_device *rdev);
+int r100_debugfs_mc_info_init(struct radeon_device *rdev);
+int r100_gui_wait_for_idle(struct radeon_device *rdev);
+void r100_ib_fini(struct radeon_device *rdev);
+int r100_ib_init(struct radeon_device *rdev);
+void r100_irq_disable(struct radeon_device *rdev);
+void r100_mc_stop(struct radeon_device *rdev, struct r100_mc_save *save);
+void r100_mc_resume(struct radeon_device *rdev, struct r100_mc_save *save);
+void r100_vram_init_sizes(struct radeon_device *rdev);
+void r100_wb_disable(struct radeon_device *rdev);
+void r100_wb_fini(struct radeon_device *rdev);
+int r100_wb_init(struct radeon_device *rdev);
+void r100_hdp_reset(struct radeon_device *rdev);
+int r100_rb2d_reset(struct radeon_device *rdev);
+int r100_cp_reset(struct radeon_device *rdev);
+void r100_vga_render_disable(struct radeon_device *rdev);
+int r100_cs_track_check_pkt3_indx_buffer(struct radeon_cs_parser *p,
+ struct radeon_cs_packet *pkt,
+ struct radeon_bo *robj);
+int r100_cs_parse_packet0(struct radeon_cs_parser *p,
+ struct radeon_cs_packet *pkt,
+ const unsigned *auth, unsigned n,
+ radeon_packet0_check_t check);
+int r100_cs_packet_parse(struct radeon_cs_parser *p,
+ struct radeon_cs_packet *pkt,
+ unsigned idx);
+void r100_enable_bm(struct radeon_device *rdev);
+void r100_set_common_regs(struct radeon_device *rdev);
/*
* r200,rv250,rs300,rv280
uint64_t dst_offset,
unsigned num_pages,
struct radeon_fence *fence);
-static struct radeon_asic r200_asic = {
- .init = &r100_init,
- .fini = &r100_fini,
- .suspend = &r100_suspend,
- .resume = &r100_resume,
- .vga_set_state = &r100_vga_set_state,
- .gpu_reset = &r100_gpu_reset,
- .gart_tlb_flush = &r100_pci_gart_tlb_flush,
- .gart_set_page = &r100_pci_gart_set_page,
- .cp_commit = &r100_cp_commit,
- .ring_start = &r100_ring_start,
- .ring_test = &r100_ring_test,
- .ring_ib_execute = &r100_ring_ib_execute,
- .irq_set = &r100_irq_set,
- .irq_process = &r100_irq_process,
- .get_vblank_counter = &r100_get_vblank_counter,
- .fence_ring_emit = &r100_fence_ring_emit,
- .cs_parse = &r100_cs_parse,
- .copy_blit = &r100_copy_blit,
- .copy_dma = &r200_copy_dma,
- .copy = &r100_copy_blit,
- .get_engine_clock = &radeon_legacy_get_engine_clock,
- .set_engine_clock = &radeon_legacy_set_engine_clock,
- .get_memory_clock = &radeon_legacy_get_memory_clock,
- .set_memory_clock = NULL,
- .set_pcie_lanes = NULL,
- .set_clock_gating = &radeon_legacy_set_clock_gating,
- .set_surface_reg = r100_set_surface_reg,
- .clear_surface_reg = r100_clear_surface_reg,
- .bandwidth_update = &r100_bandwidth_update,
- .hpd_init = &r100_hpd_init,
- .hpd_fini = &r100_hpd_fini,
- .hpd_sense = &r100_hpd_sense,
- .hpd_set_polarity = &r100_hpd_set_polarity,
- .ioctl_wait_idle = NULL,
-};
-
/*
* r300,r350,rv350,rv380
extern void rv370_set_pcie_lanes(struct radeon_device *rdev, int lanes);
extern int rv370_get_pcie_lanes(struct radeon_device *rdev);
-static struct radeon_asic r300_asic = {
- .init = &r300_init,
- .fini = &r300_fini,
- .suspend = &r300_suspend,
- .resume = &r300_resume,
- .vga_set_state = &r100_vga_set_state,
- .gpu_reset = &r300_gpu_reset,
- .gart_tlb_flush = &r100_pci_gart_tlb_flush,
- .gart_set_page = &r100_pci_gart_set_page,
- .cp_commit = &r100_cp_commit,
- .ring_start = &r300_ring_start,
- .ring_test = &r100_ring_test,
- .ring_ib_execute = &r100_ring_ib_execute,
- .irq_set = &r100_irq_set,
- .irq_process = &r100_irq_process,
- .get_vblank_counter = &r100_get_vblank_counter,
- .fence_ring_emit = &r300_fence_ring_emit,
- .cs_parse = &r300_cs_parse,
- .copy_blit = &r100_copy_blit,
- .copy_dma = &r200_copy_dma,
- .copy = &r100_copy_blit,
- .get_engine_clock = &radeon_legacy_get_engine_clock,
- .set_engine_clock = &radeon_legacy_set_engine_clock,
- .get_memory_clock = &radeon_legacy_get_memory_clock,
- .set_memory_clock = NULL,
- .get_pcie_lanes = &rv370_get_pcie_lanes,
- .set_pcie_lanes = &rv370_set_pcie_lanes,
- .set_clock_gating = &radeon_legacy_set_clock_gating,
- .set_surface_reg = r100_set_surface_reg,
- .clear_surface_reg = r100_clear_surface_reg,
- .bandwidth_update = &r100_bandwidth_update,
- .hpd_init = &r100_hpd_init,
- .hpd_fini = &r100_hpd_fini,
- .hpd_sense = &r100_hpd_sense,
- .hpd_set_polarity = &r100_hpd_set_polarity,
- .ioctl_wait_idle = NULL,
-};
-
-
-static struct radeon_asic r300_asic_pcie = {
- .init = &r300_init,
- .fini = &r300_fini,
- .suspend = &r300_suspend,
- .resume = &r300_resume,
- .vga_set_state = &r100_vga_set_state,
- .gpu_reset = &r300_gpu_reset,
- .gart_tlb_flush = &rv370_pcie_gart_tlb_flush,
- .gart_set_page = &rv370_pcie_gart_set_page,
- .cp_commit = &r100_cp_commit,
- .ring_start = &r300_ring_start,
- .ring_test = &r100_ring_test,
- .ring_ib_execute = &r100_ring_ib_execute,
- .irq_set = &r100_irq_set,
- .irq_process = &r100_irq_process,
- .get_vblank_counter = &r100_get_vblank_counter,
- .fence_ring_emit = &r300_fence_ring_emit,
- .cs_parse = &r300_cs_parse,
- .copy_blit = &r100_copy_blit,
- .copy_dma = &r200_copy_dma,
- .copy = &r100_copy_blit,
- .get_engine_clock = &radeon_legacy_get_engine_clock,
- .set_engine_clock = &radeon_legacy_set_engine_clock,
- .get_memory_clock = &radeon_legacy_get_memory_clock,
- .set_memory_clock = NULL,
- .set_pcie_lanes = &rv370_set_pcie_lanes,
- .set_clock_gating = &radeon_legacy_set_clock_gating,
- .set_surface_reg = r100_set_surface_reg,
- .clear_surface_reg = r100_clear_surface_reg,
- .bandwidth_update = &r100_bandwidth_update,
- .hpd_init = &r100_hpd_init,
- .hpd_fini = &r100_hpd_fini,
- .hpd_sense = &r100_hpd_sense,
- .hpd_set_polarity = &r100_hpd_set_polarity,
- .ioctl_wait_idle = NULL,
-};
-
/*
* r420,r423,rv410
*/
extern void r420_fini(struct radeon_device *rdev);
extern int r420_suspend(struct radeon_device *rdev);
extern int r420_resume(struct radeon_device *rdev);
-static struct radeon_asic r420_asic = {
- .init = &r420_init,
- .fini = &r420_fini,
- .suspend = &r420_suspend,
- .resume = &r420_resume,
- .vga_set_state = &r100_vga_set_state,
- .gpu_reset = &r300_gpu_reset,
- .gart_tlb_flush = &rv370_pcie_gart_tlb_flush,
- .gart_set_page = &rv370_pcie_gart_set_page,
- .cp_commit = &r100_cp_commit,
- .ring_start = &r300_ring_start,
- .ring_test = &r100_ring_test,
- .ring_ib_execute = &r100_ring_ib_execute,
- .irq_set = &r100_irq_set,
- .irq_process = &r100_irq_process,
- .get_vblank_counter = &r100_get_vblank_counter,
- .fence_ring_emit = &r300_fence_ring_emit,
- .cs_parse = &r300_cs_parse,
- .copy_blit = &r100_copy_blit,
- .copy_dma = &r200_copy_dma,
- .copy = &r100_copy_blit,
- .get_engine_clock = &radeon_atom_get_engine_clock,
- .set_engine_clock = &radeon_atom_set_engine_clock,
- .get_memory_clock = &radeon_atom_get_memory_clock,
- .set_memory_clock = &radeon_atom_set_memory_clock,
- .get_pcie_lanes = &rv370_get_pcie_lanes,
- .set_pcie_lanes = &rv370_set_pcie_lanes,
- .set_clock_gating = &radeon_atom_set_clock_gating,
- .set_surface_reg = r100_set_surface_reg,
- .clear_surface_reg = r100_clear_surface_reg,
- .bandwidth_update = &r100_bandwidth_update,
- .hpd_init = &r100_hpd_init,
- .hpd_fini = &r100_hpd_fini,
- .hpd_sense = &r100_hpd_sense,
- .hpd_set_polarity = &r100_hpd_set_polarity,
- .ioctl_wait_idle = NULL,
-};
-
/*
* rs400,rs480
int rs400_gart_set_page(struct radeon_device *rdev, int i, uint64_t addr);
uint32_t rs400_mc_rreg(struct radeon_device *rdev, uint32_t reg);
void rs400_mc_wreg(struct radeon_device *rdev, uint32_t reg, uint32_t v);
-static struct radeon_asic rs400_asic = {
- .init = &rs400_init,
- .fini = &rs400_fini,
- .suspend = &rs400_suspend,
- .resume = &rs400_resume,
- .vga_set_state = &r100_vga_set_state,
- .gpu_reset = &r300_gpu_reset,
- .gart_tlb_flush = &rs400_gart_tlb_flush,
- .gart_set_page = &rs400_gart_set_page,
- .cp_commit = &r100_cp_commit,
- .ring_start = &r300_ring_start,
- .ring_test = &r100_ring_test,
- .ring_ib_execute = &r100_ring_ib_execute,
- .irq_set = &r100_irq_set,
- .irq_process = &r100_irq_process,
- .get_vblank_counter = &r100_get_vblank_counter,
- .fence_ring_emit = &r300_fence_ring_emit,
- .cs_parse = &r300_cs_parse,
- .copy_blit = &r100_copy_blit,
- .copy_dma = &r200_copy_dma,
- .copy = &r100_copy_blit,
- .get_engine_clock = &radeon_legacy_get_engine_clock,
- .set_engine_clock = &radeon_legacy_set_engine_clock,
- .get_memory_clock = &radeon_legacy_get_memory_clock,
- .set_memory_clock = NULL,
- .get_pcie_lanes = NULL,
- .set_pcie_lanes = NULL,
- .set_clock_gating = &radeon_legacy_set_clock_gating,
- .set_surface_reg = r100_set_surface_reg,
- .clear_surface_reg = r100_clear_surface_reg,
- .bandwidth_update = &r100_bandwidth_update,
- .hpd_init = &r100_hpd_init,
- .hpd_fini = &r100_hpd_fini,
- .hpd_sense = &r100_hpd_sense,
- .hpd_set_polarity = &r100_hpd_set_polarity,
- .ioctl_wait_idle = NULL,
-};
-
/*
* rs600.
void rs600_hpd_set_polarity(struct radeon_device *rdev,
enum radeon_hpd_id hpd);
-static struct radeon_asic rs600_asic = {
- .init = &rs600_init,
- .fini = &rs600_fini,
- .suspend = &rs600_suspend,
- .resume = &rs600_resume,
- .vga_set_state = &r100_vga_set_state,
- .gpu_reset = &r300_gpu_reset,
- .gart_tlb_flush = &rs600_gart_tlb_flush,
- .gart_set_page = &rs600_gart_set_page,
- .cp_commit = &r100_cp_commit,
- .ring_start = &r300_ring_start,
- .ring_test = &r100_ring_test,
- .ring_ib_execute = &r100_ring_ib_execute,
- .irq_set = &rs600_irq_set,
- .irq_process = &rs600_irq_process,
- .get_vblank_counter = &rs600_get_vblank_counter,
- .fence_ring_emit = &r300_fence_ring_emit,
- .cs_parse = &r300_cs_parse,
- .copy_blit = &r100_copy_blit,
- .copy_dma = &r200_copy_dma,
- .copy = &r100_copy_blit,
- .get_engine_clock = &radeon_atom_get_engine_clock,
- .set_engine_clock = &radeon_atom_set_engine_clock,
- .get_memory_clock = &radeon_atom_get_memory_clock,
- .set_memory_clock = &radeon_atom_set_memory_clock,
- .get_pcie_lanes = NULL,
- .set_pcie_lanes = NULL,
- .set_clock_gating = &radeon_atom_set_clock_gating,
- .set_surface_reg = r100_set_surface_reg,
- .clear_surface_reg = r100_clear_surface_reg,
- .bandwidth_update = &rs600_bandwidth_update,
- .hpd_init = &rs600_hpd_init,
- .hpd_fini = &rs600_hpd_fini,
- .hpd_sense = &rs600_hpd_sense,
- .hpd_set_polarity = &rs600_hpd_set_polarity,
- .ioctl_wait_idle = NULL,
-};
-
-
/*
* rs690,rs740
*/
uint32_t rs690_mc_rreg(struct radeon_device *rdev, uint32_t reg);
void rs690_mc_wreg(struct radeon_device *rdev, uint32_t reg, uint32_t v);
void rs690_bandwidth_update(struct radeon_device *rdev);
-static struct radeon_asic rs690_asic = {
- .init = &rs690_init,
- .fini = &rs690_fini,
- .suspend = &rs690_suspend,
- .resume = &rs690_resume,
- .vga_set_state = &r100_vga_set_state,
- .gpu_reset = &r300_gpu_reset,
- .gart_tlb_flush = &rs400_gart_tlb_flush,
- .gart_set_page = &rs400_gart_set_page,
- .cp_commit = &r100_cp_commit,
- .ring_start = &r300_ring_start,
- .ring_test = &r100_ring_test,
- .ring_ib_execute = &r100_ring_ib_execute,
- .irq_set = &rs600_irq_set,
- .irq_process = &rs600_irq_process,
- .get_vblank_counter = &rs600_get_vblank_counter,
- .fence_ring_emit = &r300_fence_ring_emit,
- .cs_parse = &r300_cs_parse,
- .copy_blit = &r100_copy_blit,
- .copy_dma = &r200_copy_dma,
- .copy = &r200_copy_dma,
- .get_engine_clock = &radeon_atom_get_engine_clock,
- .set_engine_clock = &radeon_atom_set_engine_clock,
- .get_memory_clock = &radeon_atom_get_memory_clock,
- .set_memory_clock = &radeon_atom_set_memory_clock,
- .get_pcie_lanes = NULL,
- .set_pcie_lanes = NULL,
- .set_clock_gating = &radeon_atom_set_clock_gating,
- .set_surface_reg = r100_set_surface_reg,
- .clear_surface_reg = r100_clear_surface_reg,
- .bandwidth_update = &rs690_bandwidth_update,
- .hpd_init = &rs600_hpd_init,
- .hpd_fini = &rs600_hpd_fini,
- .hpd_sense = &rs600_hpd_sense,
- .hpd_set_polarity = &rs600_hpd_set_polarity,
- .ioctl_wait_idle = NULL,
-};
-
/*
* rv515
void rv515_bandwidth_update(struct radeon_device *rdev);
int rv515_resume(struct radeon_device *rdev);
int rv515_suspend(struct radeon_device *rdev);
-static struct radeon_asic rv515_asic = {
- .init = &rv515_init,
- .fini = &rv515_fini,
- .suspend = &rv515_suspend,
- .resume = &rv515_resume,
- .vga_set_state = &r100_vga_set_state,
- .gpu_reset = &rv515_gpu_reset,
- .gart_tlb_flush = &rv370_pcie_gart_tlb_flush,
- .gart_set_page = &rv370_pcie_gart_set_page,
- .cp_commit = &r100_cp_commit,
- .ring_start = &rv515_ring_start,
- .ring_test = &r100_ring_test,
- .ring_ib_execute = &r100_ring_ib_execute,
- .irq_set = &rs600_irq_set,
- .irq_process = &rs600_irq_process,
- .get_vblank_counter = &rs600_get_vblank_counter,
- .fence_ring_emit = &r300_fence_ring_emit,
- .cs_parse = &r300_cs_parse,
- .copy_blit = &r100_copy_blit,
- .copy_dma = &r200_copy_dma,
- .copy = &r100_copy_blit,
- .get_engine_clock = &radeon_atom_get_engine_clock,
- .set_engine_clock = &radeon_atom_set_engine_clock,
- .get_memory_clock = &radeon_atom_get_memory_clock,
- .set_memory_clock = &radeon_atom_set_memory_clock,
- .get_pcie_lanes = &rv370_get_pcie_lanes,
- .set_pcie_lanes = &rv370_set_pcie_lanes,
- .set_clock_gating = &radeon_atom_set_clock_gating,
- .set_surface_reg = r100_set_surface_reg,
- .clear_surface_reg = r100_clear_surface_reg,
- .bandwidth_update = &rv515_bandwidth_update,
- .hpd_init = &rs600_hpd_init,
- .hpd_fini = &rs600_hpd_fini,
- .hpd_sense = &rs600_hpd_sense,
- .hpd_set_polarity = &rs600_hpd_set_polarity,
- .ioctl_wait_idle = NULL,
-};
-
/*
* r520,rv530,rv560,rv570,r580
*/
int r520_init(struct radeon_device *rdev);
int r520_resume(struct radeon_device *rdev);
-static struct radeon_asic r520_asic = {
- .init = &r520_init,
- .fini = &rv515_fini,
- .suspend = &rv515_suspend,
- .resume = &r520_resume,
- .vga_set_state = &r100_vga_set_state,
- .gpu_reset = &rv515_gpu_reset,
- .gart_tlb_flush = &rv370_pcie_gart_tlb_flush,
- .gart_set_page = &rv370_pcie_gart_set_page,
- .cp_commit = &r100_cp_commit,
- .ring_start = &rv515_ring_start,
- .ring_test = &r100_ring_test,
- .ring_ib_execute = &r100_ring_ib_execute,
- .irq_set = &rs600_irq_set,
- .irq_process = &rs600_irq_process,
- .get_vblank_counter = &rs600_get_vblank_counter,
- .fence_ring_emit = &r300_fence_ring_emit,
- .cs_parse = &r300_cs_parse,
- .copy_blit = &r100_copy_blit,
- .copy_dma = &r200_copy_dma,
- .copy = &r100_copy_blit,
- .get_engine_clock = &radeon_atom_get_engine_clock,
- .set_engine_clock = &radeon_atom_set_engine_clock,
- .get_memory_clock = &radeon_atom_get_memory_clock,
- .set_memory_clock = &radeon_atom_set_memory_clock,
- .get_pcie_lanes = &rv370_get_pcie_lanes,
- .set_pcie_lanes = &rv370_set_pcie_lanes,
- .set_clock_gating = &radeon_atom_set_clock_gating,
- .set_surface_reg = r100_set_surface_reg,
- .clear_surface_reg = r100_clear_surface_reg,
- .bandwidth_update = &rv515_bandwidth_update,
- .hpd_init = &rs600_hpd_init,
- .hpd_fini = &rs600_hpd_fini,
- .hpd_sense = &rs600_hpd_sense,
- .hpd_set_polarity = &rs600_hpd_set_polarity,
- .ioctl_wait_idle = NULL,
-};
/*
* r600,rv610,rv630,rv620,rv635,rv670,rs780,rs880
int r600_set_surface_reg(struct radeon_device *rdev, int reg,
uint32_t tiling_flags, uint32_t pitch,
uint32_t offset, uint32_t obj_size);
-int r600_clear_surface_reg(struct radeon_device *rdev, int reg);
+void r600_clear_surface_reg(struct radeon_device *rdev, int reg);
void r600_ring_ib_execute(struct radeon_device *rdev, struct radeon_ib *ib);
int r600_ring_test(struct radeon_device *rdev);
int r600_copy_blit(struct radeon_device *rdev,
enum radeon_hpd_id hpd);
extern void r600_ioctl_wait_idle(struct radeon_device *rdev, struct radeon_bo *bo);
-static struct radeon_asic r600_asic = {
- .init = &r600_init,
- .fini = &r600_fini,
- .suspend = &r600_suspend,
- .resume = &r600_resume,
- .cp_commit = &r600_cp_commit,
- .vga_set_state = &r600_vga_set_state,
- .gpu_reset = &r600_gpu_reset,
- .gart_tlb_flush = &r600_pcie_gart_tlb_flush,
- .gart_set_page = &rs600_gart_set_page,
- .ring_test = &r600_ring_test,
- .ring_ib_execute = &r600_ring_ib_execute,
- .irq_set = &r600_irq_set,
- .irq_process = &r600_irq_process,
- .get_vblank_counter = &rs600_get_vblank_counter,
- .fence_ring_emit = &r600_fence_ring_emit,
- .cs_parse = &r600_cs_parse,
- .copy_blit = &r600_copy_blit,
- .copy_dma = &r600_copy_blit,
- .copy = &r600_copy_blit,
- .get_engine_clock = &radeon_atom_get_engine_clock,
- .set_engine_clock = &radeon_atom_set_engine_clock,
- .get_memory_clock = &radeon_atom_get_memory_clock,
- .set_memory_clock = &radeon_atom_set_memory_clock,
- .get_pcie_lanes = &rv370_get_pcie_lanes,
- .set_pcie_lanes = NULL,
- .set_clock_gating = NULL,
- .set_surface_reg = r600_set_surface_reg,
- .clear_surface_reg = r600_clear_surface_reg,
- .bandwidth_update = &rv515_bandwidth_update,
- .hpd_init = &r600_hpd_init,
- .hpd_fini = &r600_hpd_fini,
- .hpd_sense = &r600_hpd_sense,
- .hpd_set_polarity = &r600_hpd_set_polarity,
- .ioctl_wait_idle = r600_ioctl_wait_idle,
-};
-
/*
* rv770,rv730,rv710,rv740
*/
int rv770_resume(struct radeon_device *rdev);
int rv770_gpu_reset(struct radeon_device *rdev);
-static struct radeon_asic rv770_asic = {
- .init = &rv770_init,
- .fini = &rv770_fini,
- .suspend = &rv770_suspend,
- .resume = &rv770_resume,
- .cp_commit = &r600_cp_commit,
- .gpu_reset = &rv770_gpu_reset,
- .vga_set_state = &r600_vga_set_state,
- .gart_tlb_flush = &r600_pcie_gart_tlb_flush,
- .gart_set_page = &rs600_gart_set_page,
- .ring_test = &r600_ring_test,
- .ring_ib_execute = &r600_ring_ib_execute,
- .irq_set = &r600_irq_set,
- .irq_process = &r600_irq_process,
- .get_vblank_counter = &rs600_get_vblank_counter,
- .fence_ring_emit = &r600_fence_ring_emit,
- .cs_parse = &r600_cs_parse,
- .copy_blit = &r600_copy_blit,
- .copy_dma = &r600_copy_blit,
- .copy = &r600_copy_blit,
- .get_engine_clock = &radeon_atom_get_engine_clock,
- .set_engine_clock = &radeon_atom_set_engine_clock,
- .get_memory_clock = &radeon_atom_get_memory_clock,
- .set_memory_clock = &radeon_atom_set_memory_clock,
- .get_pcie_lanes = &rv370_get_pcie_lanes,
- .set_pcie_lanes = NULL,
- .set_clock_gating = &radeon_atom_set_clock_gating,
- .set_surface_reg = r600_set_surface_reg,
- .clear_surface_reg = r600_clear_surface_reg,
- .bandwidth_update = &rv515_bandwidth_update,
- .hpd_init = &r600_hpd_init,
- .hpd_fini = &r600_hpd_fini,
- .hpd_sense = &r600_hpd_sense,
- .hpd_set_polarity = &r600_hpd_set_polarity,
- .ioctl_wait_idle = r600_ioctl_wait_idle,
-};
-
/*
* evergreen
*/
bool evergreen_hpd_sense(struct radeon_device *rdev, enum radeon_hpd_id hpd);
void evergreen_hpd_set_polarity(struct radeon_device *rdev,
enum radeon_hpd_id hpd);
-
-static struct radeon_asic evergreen_asic = {
- .init = &evergreen_init,
- .fini = &evergreen_fini,
- .suspend = &evergreen_suspend,
- .resume = &evergreen_resume,
- .cp_commit = NULL,
- .gpu_reset = &evergreen_gpu_reset,
- .vga_set_state = &r600_vga_set_state,
- .gart_tlb_flush = &r600_pcie_gart_tlb_flush,
- .gart_set_page = &rs600_gart_set_page,
- .ring_test = NULL,
- .ring_ib_execute = NULL,
- .irq_set = NULL,
- .irq_process = NULL,
- .get_vblank_counter = NULL,
- .fence_ring_emit = NULL,
- .cs_parse = NULL,
- .copy_blit = NULL,
- .copy_dma = NULL,
- .copy = NULL,
- .get_engine_clock = &radeon_atom_get_engine_clock,
- .set_engine_clock = &radeon_atom_set_engine_clock,
- .get_memory_clock = &radeon_atom_get_memory_clock,
- .set_memory_clock = &radeon_atom_set_memory_clock,
- .set_pcie_lanes = NULL,
- .set_clock_gating = NULL,
- .set_surface_reg = r600_set_surface_reg,
- .clear_surface_reg = r600_clear_surface_reg,
- .bandwidth_update = &evergreen_bandwidth_update,
- .hpd_init = &evergreen_hpd_init,
- .hpd_fini = &evergreen_hpd_fini,
- .hpd_sense = &evergreen_hpd_sense,
- .hpd_set_polarity = &evergreen_hpd_set_polarity,
-};
-
#endif
memset(&i2c, 0, sizeof(struct radeon_i2c_bus_rec));
i2c.valid = false;
- atom_parse_data_header(ctx, index, NULL, NULL, NULL, &data_offset);
-
- i2c_info = (struct _ATOM_GPIO_I2C_INFO *)(ctx->bios + data_offset);
-
-
- for (i = 0; i < ATOM_MAX_SUPPORTED_DEVICE; i++) {
- gpio = &i2c_info->asGPIO_Info[i];
-
- if (gpio->sucI2cId.ucAccess == id) {
- i2c.mask_clk_reg = le16_to_cpu(gpio->usClkMaskRegisterIndex) * 4;
- i2c.mask_data_reg = le16_to_cpu(gpio->usDataMaskRegisterIndex) * 4;
- i2c.en_clk_reg = le16_to_cpu(gpio->usClkEnRegisterIndex) * 4;
- i2c.en_data_reg = le16_to_cpu(gpio->usDataEnRegisterIndex) * 4;
- i2c.y_clk_reg = le16_to_cpu(gpio->usClkY_RegisterIndex) * 4;
- i2c.y_data_reg = le16_to_cpu(gpio->usDataY_RegisterIndex) * 4;
- i2c.a_clk_reg = le16_to_cpu(gpio->usClkA_RegisterIndex) * 4;
- i2c.a_data_reg = le16_to_cpu(gpio->usDataA_RegisterIndex) * 4;
- i2c.mask_clk_mask = (1 << gpio->ucClkMaskShift);
- i2c.mask_data_mask = (1 << gpio->ucDataMaskShift);
- i2c.en_clk_mask = (1 << gpio->ucClkEnShift);
- i2c.en_data_mask = (1 << gpio->ucDataEnShift);
- i2c.y_clk_mask = (1 << gpio->ucClkY_Shift);
- i2c.y_data_mask = (1 << gpio->ucDataY_Shift);
- i2c.a_clk_mask = (1 << gpio->ucClkA_Shift);
- i2c.a_data_mask = (1 << gpio->ucDataA_Shift);
-
- if (gpio->sucI2cId.sbfAccess.bfHW_Capable)
- i2c.hw_capable = true;
- else
- i2c.hw_capable = false;
-
- if (gpio->sucI2cId.ucAccess == 0xa0)
- i2c.mm_i2c = true;
- else
- i2c.mm_i2c = false;
-
- i2c.i2c_id = gpio->sucI2cId.ucAccess;
-
- i2c.valid = true;
- break;
+ if (atom_parse_data_header(ctx, index, NULL, NULL, NULL, &data_offset)) {
+ i2c_info = (struct _ATOM_GPIO_I2C_INFO *)(ctx->bios + data_offset);
+
+ for (i = 0; i < ATOM_MAX_SUPPORTED_DEVICE; i++) {
+ gpio = &i2c_info->asGPIO_Info[i];
+
+ if (gpio->sucI2cId.ucAccess == id) {
+ i2c.mask_clk_reg = le16_to_cpu(gpio->usClkMaskRegisterIndex) * 4;
+ i2c.mask_data_reg = le16_to_cpu(gpio->usDataMaskRegisterIndex) * 4;
+ i2c.en_clk_reg = le16_to_cpu(gpio->usClkEnRegisterIndex) * 4;
+ i2c.en_data_reg = le16_to_cpu(gpio->usDataEnRegisterIndex) * 4;
+ i2c.y_clk_reg = le16_to_cpu(gpio->usClkY_RegisterIndex) * 4;
+ i2c.y_data_reg = le16_to_cpu(gpio->usDataY_RegisterIndex) * 4;
+ i2c.a_clk_reg = le16_to_cpu(gpio->usClkA_RegisterIndex) * 4;
+ i2c.a_data_reg = le16_to_cpu(gpio->usDataA_RegisterIndex) * 4;
+ i2c.mask_clk_mask = (1 << gpio->ucClkMaskShift);
+ i2c.mask_data_mask = (1 << gpio->ucDataMaskShift);
+ i2c.en_clk_mask = (1 << gpio->ucClkEnShift);
+ i2c.en_data_mask = (1 << gpio->ucDataEnShift);
+ i2c.y_clk_mask = (1 << gpio->ucClkY_Shift);
+ i2c.y_data_mask = (1 << gpio->ucDataY_Shift);
+ i2c.a_clk_mask = (1 << gpio->ucClkA_Shift);
+ i2c.a_data_mask = (1 << gpio->ucDataA_Shift);
+
+ if (gpio->sucI2cId.sbfAccess.bfHW_Capable)
+ i2c.hw_capable = true;
+ else
+ i2c.hw_capable = false;
+
+ if (gpio->sucI2cId.ucAccess == 0xa0)
+ i2c.mm_i2c = true;
+ else
+ i2c.mm_i2c = false;
+
+ i2c.i2c_id = gpio->sucI2cId.ucAccess;
+
+ i2c.valid = true;
+ break;
+ }
}
}
memset(&gpio, 0, sizeof(struct radeon_gpio_rec));
gpio.valid = false;
- atom_parse_data_header(ctx, index, &size, NULL, NULL, &data_offset);
+ if (atom_parse_data_header(ctx, index, &size, NULL, NULL, &data_offset)) {
+ gpio_info = (struct _ATOM_GPIO_PIN_LUT *)(ctx->bios + data_offset);
- gpio_info = (struct _ATOM_GPIO_PIN_LUT *)(ctx->bios + data_offset);
+ num_indices = (size - sizeof(ATOM_COMMON_TABLE_HEADER)) /
+ sizeof(ATOM_GPIO_PIN_ASSIGNMENT);
- num_indices = (size - sizeof(ATOM_COMMON_TABLE_HEADER)) / sizeof(ATOM_GPIO_PIN_ASSIGNMENT);
-
- for (i = 0; i < num_indices; i++) {
- pin = &gpio_info->asGPIO_Pin[i];
- if (id == pin->ucGPIO_ID) {
- gpio.id = pin->ucGPIO_ID;
- gpio.reg = pin->usGpioPin_AIndex * 4;
- gpio.mask = (1 << pin->ucGpioPinBitShift);
- gpio.valid = true;
- break;
+ for (i = 0; i < num_indices; i++) {
+ pin = &gpio_info->asGPIO_Pin[i];
+ if (id == pin->ucGPIO_ID) {
+ gpio.id = pin->ucGPIO_ID;
+ gpio.reg = pin->usGpioPin_AIndex * 4;
+ gpio.mask = (1 << pin->ucGpioPinBitShift);
+ gpio.valid = true;
+ break;
+ }
}
}
if ((supported_device == ATOM_DEVICE_CRT1_SUPPORT) ||
(supported_device == ATOM_DEVICE_DFP2_SUPPORT))
return false;
+ if (supported_device == ATOM_DEVICE_CRT2_SUPPORT)
+ *line_mux = 0x90;
}
/* ASUS HD 3600 XT board lists the DVI port as HDMI */
struct radeon_gpio_rec gpio;
struct radeon_hpd hpd;
- atom_parse_data_header(ctx, index, &size, &frev, &crev, &data_offset);
-
- if (data_offset == 0)
+ if (!atom_parse_data_header(ctx, index, &size, &frev, &crev, &data_offset))
return false;
if (crev < 2)
GetIndexIntoMasterTable(DATA,
IntegratedSystemInfo);
- atom_parse_data_header(ctx, index, &size, &frev,
- &crev, &igp_offset);
-
- if (crev >= 2) {
- igp_obj =
- (ATOM_INTEGRATED_SYSTEM_INFO_V2
- *) (ctx->bios + igp_offset);
-
- if (igp_obj) {
- uint32_t slot_config, ct;
-
- if (con_obj_num == 1)
- slot_config =
- igp_obj->
- ulDDISlot1Config;
- else
- slot_config =
- igp_obj->
- ulDDISlot2Config;
-
- ct = (slot_config >> 16) & 0xff;
- connector_type =
- object_connector_convert
- [ct];
- connector_object_id = ct;
- igp_lane_info =
- slot_config & 0xffff;
+ if (atom_parse_data_header(ctx, index, &size, &frev,
+ &crev, &igp_offset)) {
+
+ if (crev >= 2) {
+ igp_obj =
+ (ATOM_INTEGRATED_SYSTEM_INFO_V2
+ *) (ctx->bios + igp_offset);
+
+ if (igp_obj) {
+ uint32_t slot_config, ct;
+
+ if (con_obj_num == 1)
+ slot_config =
+ igp_obj->
+ ulDDISlot1Config;
+ else
+ slot_config =
+ igp_obj->
+ ulDDISlot2Config;
+
+ ct = (slot_config >> 16) & 0xff;
+ connector_type =
+ object_connector_convert
+ [ct];
+ connector_object_id = ct;
+ igp_lane_info =
+ slot_config & 0xffff;
+ } else
+ continue;
} else
continue;
- } else
- continue;
+ } else {
+ igp_lane_info = 0;
+ connector_type =
+ object_connector_convert[con_obj_id];
+ connector_object_id = con_obj_id;
+ }
} else {
igp_lane_info = 0;
connector_type =
uint8_t frev, crev;
ATOM_XTMDS_INFO *xtmds;
- atom_parse_data_header(ctx, index, &size, &frev, &crev, &data_offset);
- xtmds = (ATOM_XTMDS_INFO *)(ctx->bios + data_offset);
+ if (atom_parse_data_header(ctx, index, &size, &frev, &crev, &data_offset)) {
+ xtmds = (ATOM_XTMDS_INFO *)(ctx->bios + data_offset);
- if (xtmds->ucSupportedLink & ATOM_XTMDS_SUPPORTED_DUALLINK) {
- if (connector_type == DRM_MODE_CONNECTOR_DVII)
- return CONNECTOR_OBJECT_ID_DUAL_LINK_DVI_I;
- else
- return CONNECTOR_OBJECT_ID_DUAL_LINK_DVI_D;
- } else {
- if (connector_type == DRM_MODE_CONNECTOR_DVII)
- return CONNECTOR_OBJECT_ID_SINGLE_LINK_DVI_I;
- else
- return CONNECTOR_OBJECT_ID_SINGLE_LINK_DVI_D;
- }
+ if (xtmds->ucSupportedLink & ATOM_XTMDS_SUPPORTED_DUALLINK) {
+ if (connector_type == DRM_MODE_CONNECTOR_DVII)
+ return CONNECTOR_OBJECT_ID_DUAL_LINK_DVI_I;
+ else
+ return CONNECTOR_OBJECT_ID_DUAL_LINK_DVI_D;
+ } else {
+ if (connector_type == DRM_MODE_CONNECTOR_DVII)
+ return CONNECTOR_OBJECT_ID_SINGLE_LINK_DVI_I;
+ else
+ return CONNECTOR_OBJECT_ID_SINGLE_LINK_DVI_D;
+ }
+ } else
+ return supported_devices_connector_object_id_convert
+ [connector_type];
} else {
return supported_devices_connector_object_id_convert
[connector_type];
int i, j, max_device;
struct bios_connector bios_connectors[ATOM_MAX_SUPPORTED_DEVICE];
- atom_parse_data_header(ctx, index, &size, &frev, &crev, &data_offset);
+ if (!atom_parse_data_header(ctx, index, &size, &frev, &crev, &data_offset))
+ return false;
supported_devices =
(union atom_supported_devices *)(ctx->bios + data_offset);
struct radeon_pll *mpll = &rdev->clock.mpll;
uint16_t data_offset;
- atom_parse_data_header(mode_info->atom_context, index, NULL, &frev,
- &crev, &data_offset);
-
- firmware_info =
- (union firmware_info *)(mode_info->atom_context->bios +
- data_offset);
-
- if (firmware_info) {
+ if (atom_parse_data_header(mode_info->atom_context, index, NULL,
+ &frev, &crev, &data_offset)) {
+ firmware_info =
+ (union firmware_info *)(mode_info->atom_context->bios +
+ data_offset);
/* pixel clocks */
p1pll->reference_freq =
le16_to_cpu(firmware_info->info.usReferenceClock);
p1pll->pll_out_max =
le32_to_cpu(firmware_info->info.ulMaxPixelClockPLL_Output);
+ if (crev >= 4) {
+ p1pll->lcd_pll_out_min =
+ le16_to_cpu(firmware_info->info_14.usLcdMinPixelClockPLL_Output) * 100;
+ if (p1pll->lcd_pll_out_min == 0)
+ p1pll->lcd_pll_out_min = p1pll->pll_out_min;
+ p1pll->lcd_pll_out_max =
+ le16_to_cpu(firmware_info->info_14.usLcdMaxPixelClockPLL_Output) * 100;
+ if (p1pll->lcd_pll_out_max == 0)
+ p1pll->lcd_pll_out_max = p1pll->pll_out_max;
+ } else {
+ p1pll->lcd_pll_out_min = p1pll->pll_out_min;
+ p1pll->lcd_pll_out_max = p1pll->pll_out_max;
+ }
+
if (p1pll->pll_out_min == 0) {
if (ASIC_IS_AVIVO(rdev))
p1pll->pll_out_min = 64800;
u8 frev, crev;
u16 data_offset;
- atom_parse_data_header(mode_info->atom_context, index, NULL, &frev,
- &crev, &data_offset);
-
- igp_info = (union igp_info *)(mode_info->atom_context->bios +
+ if (atom_parse_data_header(mode_info->atom_context, index, NULL,
+ &frev, &crev, &data_offset)) {
+ igp_info = (union igp_info *)(mode_info->atom_context->bios +
data_offset);
-
- if (igp_info) {
switch (crev) {
case 1:
if (igp_info->info.ucMemoryType & 0xf0)
uint16_t maxfreq;
int i;
- atom_parse_data_header(mode_info->atom_context, index, NULL, &frev,
- &crev, &data_offset);
-
- tmds_info =
- (struct _ATOM_TMDS_INFO *)(mode_info->atom_context->bios +
- data_offset);
+ if (atom_parse_data_header(mode_info->atom_context, index, NULL,
+ &frev, &crev, &data_offset)) {
+ tmds_info =
+ (struct _ATOM_TMDS_INFO *)(mode_info->atom_context->bios +
+ data_offset);
- if (tmds_info) {
maxfreq = le16_to_cpu(tmds_info->usMaxFrequency);
for (i = 0; i < 4; i++) {
tmds->tmds_pll[i].freq =
if (id > ATOM_MAX_SS_ENTRY)
return NULL;
- atom_parse_data_header(mode_info->atom_context, index, NULL, &frev,
- &crev, &data_offset);
+ if (atom_parse_data_header(mode_info->atom_context, index, NULL,
+ &frev, &crev, &data_offset)) {
+ ss_info =
+ (struct _ATOM_SPREAD_SPECTRUM_INFO *)(mode_info->atom_context->bios + data_offset);
- ss_info =
- (struct _ATOM_SPREAD_SPECTRUM_INFO *)(mode_info->atom_context->bios + data_offset);
-
- if (ss_info) {
ss =
kzalloc(sizeof(struct radeon_atom_ss), GFP_KERNEL);
return ss;
}
-static void radeon_atom_apply_lvds_quirks(struct drm_device *dev,
- struct radeon_encoder_atom_dig *lvds)
-{
-
- /* Toshiba A300-1BU laptop panel doesn't like new pll divider algo */
- if ((dev->pdev->device == 0x95c4) &&
- (dev->pdev->subsystem_vendor == 0x1179) &&
- (dev->pdev->subsystem_device == 0xff50)) {
- if ((lvds->native_mode.hdisplay == 1280) &&
- (lvds->native_mode.vdisplay == 800))
- lvds->pll_algo = PLL_ALGO_LEGACY;
- }
-
- /* Dell Studio 15 laptop panel doesn't like new pll divider algo */
- if ((dev->pdev->device == 0x95c4) &&
- (dev->pdev->subsystem_vendor == 0x1028) &&
- (dev->pdev->subsystem_device == 0x029f)) {
- if ((lvds->native_mode.hdisplay == 1280) &&
- (lvds->native_mode.vdisplay == 800))
- lvds->pll_algo = PLL_ALGO_LEGACY;
- }
-
-}
-
union lvds_info {
struct _ATOM_LVDS_INFO info;
struct _ATOM_LVDS_INFO_V12 info_12;
uint8_t frev, crev;
struct radeon_encoder_atom_dig *lvds = NULL;
- atom_parse_data_header(mode_info->atom_context, index, NULL, &frev,
- &crev, &data_offset);
-
- lvds_info =
- (union lvds_info *)(mode_info->atom_context->bios + data_offset);
-
- if (lvds_info) {
+ if (atom_parse_data_header(mode_info->atom_context, index, NULL,
+ &frev, &crev, &data_offset)) {
+ lvds_info =
+ (union lvds_info *)(mode_info->atom_context->bios + data_offset);
lvds =
kzalloc(sizeof(struct radeon_encoder_atom_dig), GFP_KERNEL);
lvds->pll_algo = PLL_ALGO_LEGACY;
}
- /* LVDS quirks */
- radeon_atom_apply_lvds_quirks(dev, lvds);
-
encoder->native_mode = lvds->native_mode;
}
return lvds;
uint8_t bg, dac;
struct radeon_encoder_primary_dac *p_dac = NULL;
- atom_parse_data_header(mode_info->atom_context, index, NULL, &frev, &crev, &data_offset);
-
- dac_info = (struct _COMPASSIONATE_DATA *)(mode_info->atom_context->bios + data_offset);
+ if (atom_parse_data_header(mode_info->atom_context, index, NULL,
+ &frev, &crev, &data_offset)) {
+ dac_info = (struct _COMPASSIONATE_DATA *)
+ (mode_info->atom_context->bios + data_offset);
- if (dac_info) {
p_dac = kzalloc(sizeof(struct radeon_encoder_primary_dac), GFP_KERNEL);
if (!p_dac)
u8 frev, crev;
u16 data_offset, misc;
- atom_parse_data_header(mode_info->atom_context, data_index, NULL, &frev, &crev, &data_offset);
+ if (!atom_parse_data_header(mode_info->atom_context, data_index, NULL,
+ &frev, &crev, &data_offset))
+ return false;
switch (crev) {
case 1:
struct _ATOM_ANALOG_TV_INFO *tv_info;
enum radeon_tv_std tv_std = TV_STD_NTSC;
- atom_parse_data_header(mode_info->atom_context, index, NULL, &frev, &crev, &data_offset);
+ if (atom_parse_data_header(mode_info->atom_context, index, NULL,
+ &frev, &crev, &data_offset)) {
- tv_info = (struct _ATOM_ANALOG_TV_INFO *)(mode_info->atom_context->bios + data_offset);
+ tv_info = (struct _ATOM_ANALOG_TV_INFO *)
+ (mode_info->atom_context->bios + data_offset);
- switch (tv_info->ucTV_BootUpDefaultStandard) {
- case ATOM_TV_NTSC:
- tv_std = TV_STD_NTSC;
- DRM_INFO("Default TV standard: NTSC\n");
- break;
- case ATOM_TV_NTSCJ:
- tv_std = TV_STD_NTSC_J;
- DRM_INFO("Default TV standard: NTSC-J\n");
- break;
- case ATOM_TV_PAL:
- tv_std = TV_STD_PAL;
- DRM_INFO("Default TV standard: PAL\n");
- break;
- case ATOM_TV_PALM:
- tv_std = TV_STD_PAL_M;
- DRM_INFO("Default TV standard: PAL-M\n");
- break;
- case ATOM_TV_PALN:
- tv_std = TV_STD_PAL_N;
- DRM_INFO("Default TV standard: PAL-N\n");
- break;
- case ATOM_TV_PALCN:
- tv_std = TV_STD_PAL_CN;
- DRM_INFO("Default TV standard: PAL-CN\n");
- break;
- case ATOM_TV_PAL60:
- tv_std = TV_STD_PAL_60;
- DRM_INFO("Default TV standard: PAL-60\n");
- break;
- case ATOM_TV_SECAM:
- tv_std = TV_STD_SECAM;
- DRM_INFO("Default TV standard: SECAM\n");
- break;
- default:
- tv_std = TV_STD_NTSC;
- DRM_INFO("Unknown TV standard; defaulting to NTSC\n");
- break;
+ switch (tv_info->ucTV_BootUpDefaultStandard) {
+ case ATOM_TV_NTSC:
+ tv_std = TV_STD_NTSC;
+ DRM_INFO("Default TV standard: NTSC\n");
+ break;
+ case ATOM_TV_NTSCJ:
+ tv_std = TV_STD_NTSC_J;
+ DRM_INFO("Default TV standard: NTSC-J\n");
+ break;
+ case ATOM_TV_PAL:
+ tv_std = TV_STD_PAL;
+ DRM_INFO("Default TV standard: PAL\n");
+ break;
+ case ATOM_TV_PALM:
+ tv_std = TV_STD_PAL_M;
+ DRM_INFO("Default TV standard: PAL-M\n");
+ break;
+ case ATOM_TV_PALN:
+ tv_std = TV_STD_PAL_N;
+ DRM_INFO("Default TV standard: PAL-N\n");
+ break;
+ case ATOM_TV_PALCN:
+ tv_std = TV_STD_PAL_CN;
+ DRM_INFO("Default TV standard: PAL-CN\n");
+ break;
+ case ATOM_TV_PAL60:
+ tv_std = TV_STD_PAL_60;
+ DRM_INFO("Default TV standard: PAL-60\n");
+ break;
+ case ATOM_TV_SECAM:
+ tv_std = TV_STD_SECAM;
+ DRM_INFO("Default TV standard: SECAM\n");
+ break;
+ default:
+ tv_std = TV_STD_NTSC;
+ DRM_INFO("Unknown TV standard; defaulting to NTSC\n");
+ break;
+ }
}
return tv_std;
}
uint8_t bg, dac;
struct radeon_encoder_tv_dac *tv_dac = NULL;
- atom_parse_data_header(mode_info->atom_context, index, NULL, &frev, &crev, &data_offset);
+ if (atom_parse_data_header(mode_info->atom_context, index, NULL,
+ &frev, &crev, &data_offset)) {
- dac_info = (struct _COMPASSIONATE_DATA *)(mode_info->atom_context->bios + data_offset);
+ dac_info = (struct _COMPASSIONATE_DATA *)
+ (mode_info->atom_context->bios + data_offset);
- if (dac_info) {
tv_dac = kzalloc(sizeof(struct radeon_encoder_tv_dac), GFP_KERNEL);
if (!tv_dac)
return tv_dac;
}
+static const char *thermal_controller_names[] = {
+ "NONE",
+ "LM63",
+ "ADM1032",
+ "ADM1030",
+ "MUA6649",
+ "LM64",
+ "F75375",
+ "ASC7512",
+};
+
+static const char *pp_lib_thermal_controller_names[] = {
+ "NONE",
+ "LM63",
+ "ADM1032",
+ "ADM1030",
+ "MUA6649",
+ "LM64",
+ "F75375",
+ "RV6xx",
+ "RV770",
+ "ADT7473",
+};
+
union power_info {
struct _ATOM_POWERPLAY_INFO info;
struct _ATOM_POWERPLAY_INFO_V2 info_2;
struct _ATOM_PPLIB_STATE *power_state;
int num_modes = 0, i, j;
int state_index = 0, mode_index = 0;
-
- atom_parse_data_header(mode_info->atom_context, index, NULL, &frev, &crev, &data_offset);
-
- power_info = (union power_info *)(mode_info->atom_context->bios + data_offset);
+ struct radeon_i2c_bus_rec i2c_bus;
rdev->pm.default_power_state = NULL;
- if (power_info) {
+ if (atom_parse_data_header(mode_info->atom_context, index, NULL,
+ &frev, &crev, &data_offset)) {
+ power_info = (union power_info *)(mode_info->atom_context->bios + data_offset);
if (frev < 4) {
+ /* add the i2c bus for thermal/fan chip */
+ if (power_info->info.ucOverdriveThermalController > 0) {
+ DRM_INFO("Possible %s thermal controller at 0x%02x\n",
+ thermal_controller_names[power_info->info.ucOverdriveThermalController],
+ power_info->info.ucOverdriveControllerAddress >> 1);
+ i2c_bus = radeon_lookup_i2c_gpio(rdev, power_info->info.ucOverdriveI2cLine);
+ rdev->pm.i2c_bus = radeon_i2c_create(rdev->ddev, &i2c_bus, "Thermal");
+ }
num_modes = power_info->info.ucNumOfPowerModeEntries;
if (num_modes > ATOM_MAX_NUMBEROF_POWER_BLOCK)
num_modes = ATOM_MAX_NUMBEROF_POWER_BLOCK;
}
}
} else if (frev == 4) {
+ /* add the i2c bus for thermal/fan chip */
+ /* no support for internal controller yet */
+ if (power_info->info_4.sThermalController.ucType > 0) {
+ if ((power_info->info_4.sThermalController.ucType == ATOM_PP_THERMALCONTROLLER_RV6xx) ||
+ (power_info->info_4.sThermalController.ucType == ATOM_PP_THERMALCONTROLLER_RV770)) {
+ DRM_INFO("Internal thermal controller %s fan control\n",
+ (power_info->info_4.sThermalController.ucFanParameters &
+ ATOM_PP_FANPARAMETERS_NOFAN) ? "without" : "with");
+ } else {
+ DRM_INFO("Possible %s thermal controller at 0x%02x %s fan control\n",
+ pp_lib_thermal_controller_names[power_info->info_4.sThermalController.ucType],
+ power_info->info_4.sThermalController.ucI2cAddress >> 1,
+ (power_info->info_4.sThermalController.ucFanParameters &
+ ATOM_PP_FANPARAMETERS_NOFAN) ? "without" : "with");
+ i2c_bus = radeon_lookup_i2c_gpio(rdev, power_info->info_4.sThermalController.ucI2cLine);
+ rdev->pm.i2c_bus = radeon_i2c_create(rdev->ddev, &i2c_bus, "Thermal");
+ }
+ }
for (i = 0; i < power_info->info_4.ucNumStates; i++) {
mode_index = 0;
power_state = (struct _ATOM_PPLIB_STATE *)
* ATPX support for both Intel/ATI
*/
#include <linux/vga_switcheroo.h>
+#include <linux/slab.h>
#include <acpi/acpi.h>
#include <acpi/acpi_bus.h>
#include <linux/pci.h>
#include "atom.h"
#include <linux/vga_switcheroo.h>
+#include <linux/slab.h>
/*
* BIOS.
*/
case CHIP_RS300:
switch (ddc_line) {
case RADEON_GPIO_DVI_DDC:
- /* in theory this should be hw capable,
- * but it doesn't seem to work
- */
- i2c.hw_capable = false;
+ i2c.hw_capable = true;
break;
default:
i2c.hw_capable = false;
p1pll->reference_div = RBIOS16(pll_info + 0x10);
p1pll->pll_out_min = RBIOS32(pll_info + 0x12);
p1pll->pll_out_max = RBIOS32(pll_info + 0x16);
+ p1pll->lcd_pll_out_min = p1pll->pll_out_min;
+ p1pll->lcd_pll_out_max = p1pll->pll_out_max;
if (rev > 9) {
p1pll->pll_in_min = RBIOS32(pll_info + 0x36);
if (radeon_connector->edid)
kfree(radeon_connector->edid);
if (radeon_dig_connector->dp_i2c_bus)
- radeon_i2c_destroy_dp(radeon_dig_connector->dp_i2c_bus);
+ radeon_i2c_destroy(radeon_dig_connector->dp_i2c_bus);
kfree(radeon_connector->con_priv);
drm_sysfs_connector_remove(connector);
drm_connector_cleanup(connector);
radeon_bo_list_fence(&parser->validated, parser->ib->fence);
}
radeon_bo_list_unreserve(&parser->validated);
- for (i = 0; i < parser->nrelocs; i++) {
- if (parser->relocs[i].gobj)
- drm_gem_object_unreference_unlocked(parser->relocs[i].gobj);
+ if (parser->relocs != NULL) {
+ for (i = 0; i < parser->nrelocs; i++) {
+ if (parser->relocs[i].gobj)
+ drm_gem_object_unreference_unlocked(parser->relocs[i].gobj);
+ }
}
kfree(parser->track);
kfree(parser->relocs);
}
r = radeon_cs_parser_relocs(&parser);
if (r) {
- DRM_ERROR("Failed to parse relocation !\n");
+ if (r != -ERESTARTSYS)
+ DRM_ERROR("Failed to parse relocation %d!\n", r);
radeon_cs_parser_fini(&parser, r);
mutex_unlock(&rdev->cs_mutex);
return r;
* Jerome Glisse
*/
#include <linux/console.h>
+#include <linux/slab.h>
#include <drm/drmP.h>
#include <drm/drm_crtc_helper.h>
#include <drm/radeon_drm.h>
#include <linux/vga_switcheroo.h>
#include "radeon_reg.h"
#include "radeon.h"
-#include "radeon_asic.h"
#include "atom.h"
/*
}
+void radeon_update_bandwidth_info(struct radeon_device *rdev)
+{
+ fixed20_12 a;
+ u32 sclk, mclk;
+
+ if (rdev->flags & RADEON_IS_IGP) {
+ sclk = radeon_get_engine_clock(rdev);
+ mclk = rdev->clock.default_mclk;
+
+ a.full = rfixed_const(100);
+ rdev->pm.sclk.full = rfixed_const(sclk);
+ rdev->pm.sclk.full = rfixed_div(rdev->pm.sclk, a);
+ rdev->pm.mclk.full = rfixed_const(mclk);
+ rdev->pm.mclk.full = rfixed_div(rdev->pm.mclk, a);
+
+ a.full = rfixed_const(16);
+ /* core_bandwidth = sclk(Mhz) * 16 */
+ rdev->pm.core_bandwidth.full = rfixed_div(rdev->pm.sclk, a);
+ } else {
+ sclk = radeon_get_engine_clock(rdev);
+ mclk = radeon_get_memory_clock(rdev);
+
+ a.full = rfixed_const(100);
+ rdev->pm.sclk.full = rfixed_const(sclk);
+ rdev->pm.sclk.full = rfixed_div(rdev->pm.sclk, a);
+ rdev->pm.mclk.full = rfixed_const(mclk);
+ rdev->pm.mclk.full = rfixed_div(rdev->pm.mclk, a);
+ }
+}
+
bool radeon_boot_test_post_card(struct radeon_device *rdev)
{
if (radeon_card_posted(rdev))
}
-/*
- * Registers accessors functions.
- */
-uint32_t radeon_invalid_rreg(struct radeon_device *rdev, uint32_t reg)
-{
- DRM_ERROR("Invalid callback to read register 0x%04X\n", reg);
- BUG_ON(1);
- return 0;
-}
-
-void radeon_invalid_wreg(struct radeon_device *rdev, uint32_t reg, uint32_t v)
-{
- DRM_ERROR("Invalid callback to write register 0x%04X with 0x%08X\n",
- reg, v);
- BUG_ON(1);
-}
-
-void radeon_register_accessor_init(struct radeon_device *rdev)
-{
- rdev->mc_rreg = &radeon_invalid_rreg;
- rdev->mc_wreg = &radeon_invalid_wreg;
- rdev->pll_rreg = &radeon_invalid_rreg;
- rdev->pll_wreg = &radeon_invalid_wreg;
- rdev->pciep_rreg = &radeon_invalid_rreg;
- rdev->pciep_wreg = &radeon_invalid_wreg;
-
- /* Don't change order as we are overridding accessor. */
- if (rdev->family < CHIP_RV515) {
- rdev->pcie_reg_mask = 0xff;
- } else {
- rdev->pcie_reg_mask = 0x7ff;
- }
- /* FIXME: not sure here */
- if (rdev->family <= CHIP_R580) {
- rdev->pll_rreg = &r100_pll_rreg;
- rdev->pll_wreg = &r100_pll_wreg;
- }
- if (rdev->family >= CHIP_R420) {
- rdev->mc_rreg = &r420_mc_rreg;
- rdev->mc_wreg = &r420_mc_wreg;
- }
- if (rdev->family >= CHIP_RV515) {
- rdev->mc_rreg = &rv515_mc_rreg;
- rdev->mc_wreg = &rv515_mc_wreg;
- }
- if (rdev->family == CHIP_RS400 || rdev->family == CHIP_RS480) {
- rdev->mc_rreg = &rs400_mc_rreg;
- rdev->mc_wreg = &rs400_mc_wreg;
- }
- if (rdev->family == CHIP_RS690 || rdev->family == CHIP_RS740) {
- rdev->mc_rreg = &rs690_mc_rreg;
- rdev->mc_wreg = &rs690_mc_wreg;
- }
- if (rdev->family == CHIP_RS600) {
- rdev->mc_rreg = &rs600_mc_rreg;
- rdev->mc_wreg = &rs600_mc_wreg;
- }
- if ((rdev->family >= CHIP_R600) && (rdev->family <= CHIP_RV740)) {
- rdev->pciep_rreg = &r600_pciep_rreg;
- rdev->pciep_wreg = &r600_pciep_wreg;
- }
-}
-
-
-/*
- * ASIC
- */
-int radeon_asic_init(struct radeon_device *rdev)
-{
- radeon_register_accessor_init(rdev);
- switch (rdev->family) {
- case CHIP_R100:
- case CHIP_RV100:
- case CHIP_RS100:
- case CHIP_RV200:
- case CHIP_RS200:
- rdev->asic = &r100_asic;
- break;
- case CHIP_R200:
- case CHIP_RV250:
- case CHIP_RS300:
- case CHIP_RV280:
- rdev->asic = &r200_asic;
- break;
- case CHIP_R300:
- case CHIP_R350:
- case CHIP_RV350:
- case CHIP_RV380:
- if (rdev->flags & RADEON_IS_PCIE)
- rdev->asic = &r300_asic_pcie;
- else
- rdev->asic = &r300_asic;
- break;
- case CHIP_R420:
- case CHIP_R423:
- case CHIP_RV410:
- rdev->asic = &r420_asic;
- break;
- case CHIP_RS400:
- case CHIP_RS480:
- rdev->asic = &rs400_asic;
- break;
- case CHIP_RS600:
- rdev->asic = &rs600_asic;
- break;
- case CHIP_RS690:
- case CHIP_RS740:
- rdev->asic = &rs690_asic;
- break;
- case CHIP_RV515:
- rdev->asic = &rv515_asic;
- break;
- case CHIP_R520:
- case CHIP_RV530:
- case CHIP_RV560:
- case CHIP_RV570:
- case CHIP_R580:
- rdev->asic = &r520_asic;
- break;
- case CHIP_R600:
- case CHIP_RV610:
- case CHIP_RV630:
- case CHIP_RV620:
- case CHIP_RV635:
- case CHIP_RV670:
- case CHIP_RS780:
- case CHIP_RS880:
- rdev->asic = &r600_asic;
- break;
- case CHIP_RV770:
- case CHIP_RV730:
- case CHIP_RV710:
- case CHIP_RV740:
- rdev->asic = &rv770_asic;
- break;
- case CHIP_CEDAR:
- case CHIP_REDWOOD:
- case CHIP_JUNIPER:
- case CHIP_CYPRESS:
- case CHIP_HEMLOCK:
- rdev->asic = &evergreen_asic;
- break;
- default:
- /* FIXME: not supported yet */
- return -EINVAL;
- }
-
- if (rdev->flags & RADEON_IS_IGP) {
- rdev->asic->get_memory_clock = NULL;
- rdev->asic->set_memory_clock = NULL;
- }
-
- return 0;
-}
-
-
-/*
- * Wrapper around modesetting bits.
- */
-int radeon_clocks_init(struct radeon_device *rdev)
-{
- int r;
-
- r = radeon_static_clocks_init(rdev->ddev);
- if (r) {
- return r;
- }
- DRM_INFO("Clocks initialized !\n");
- return 0;
-}
-
-void radeon_clocks_fini(struct radeon_device *rdev)
-{
-}
-
/* ATOM accessor methods */
static uint32_t cail_pll_read(struct card_info *info, uint32_t reg)
{
return VGA_RSRC_NORMAL_IO | VGA_RSRC_NORMAL_MEM;
}
-void radeon_agp_disable(struct radeon_device *rdev)
-{
- rdev->flags &= ~RADEON_IS_AGP;
- if (rdev->family >= CHIP_R600) {
- DRM_INFO("Forcing AGP to PCIE mode\n");
- rdev->flags |= RADEON_IS_PCIE;
- } else if (rdev->family >= CHIP_RV515 ||
- rdev->family == CHIP_RV380 ||
- rdev->family == CHIP_RV410 ||
- rdev->family == CHIP_R423) {
- DRM_INFO("Forcing AGP to PCIE mode\n");
- rdev->flags |= RADEON_IS_PCIE;
- rdev->asic->gart_tlb_flush = &rv370_pcie_gart_tlb_flush;
- rdev->asic->gart_set_page = &rv370_pcie_gart_set_page;
- } else {
- DRM_INFO("Forcing AGP to PCI mode\n");
- rdev->flags |= RADEON_IS_PCI;
- rdev->asic->gart_tlb_flush = &r100_pci_gart_tlb_flush;
- rdev->asic->gart_set_page = &r100_pci_gart_set_page;
- }
- rdev->mc.gtt_size = radeon_gart_size * 1024 * 1024;
-}
-
void radeon_check_arguments(struct radeon_device *rdev)
{
/* vramlimit must be a power of two */
return r;
radeon_check_arguments(rdev);
+ /* all of the newer IGP chips have an internal gart
+ * However some rs4xx report as AGP, so remove that here.
+ */
+ if ((rdev->family >= CHIP_RS400) &&
+ (rdev->flags & RADEON_IS_IGP)) {
+ rdev->flags &= ~RADEON_IS_AGP;
+ }
+
if (rdev->flags & RADEON_IS_AGP && radeon_agpmode == -1) {
radeon_agp_disable(rdev);
}
if (rdev->bios) {
if (rdev->is_atom_bios) {
- if (rdev->family >= CHIP_R600)
+ ret = radeon_get_atom_connector_info_from_supported_devices_table(dev);
+ if (ret == false)
ret = radeon_get_atom_connector_info_from_object_table(dev);
- else
- ret = radeon_get_atom_connector_info_from_supported_devices_table(dev);
} else {
ret = radeon_get_legacy_connector_info_from_bios(dev);
if (ret == false)
uint32_t best_error = 0xffffffff;
uint32_t best_vco_diff = 1;
uint32_t post_div;
+ u32 pll_out_min, pll_out_max;
DRM_DEBUG("PLL freq %llu %u %u\n", freq, pll->min_ref_div, pll->max_ref_div);
freq = freq * 1000;
+ if (pll->flags & RADEON_PLL_IS_LCD) {
+ pll_out_min = pll->lcd_pll_out_min;
+ pll_out_max = pll->lcd_pll_out_max;
+ } else {
+ pll_out_min = pll->pll_out_min;
+ pll_out_max = pll->pll_out_max;
+ }
+
if (pll->flags & RADEON_PLL_USE_REF_DIV)
min_ref_div = max_ref_div = pll->reference_div;
else {
tmp = (uint64_t)pll->reference_freq * feedback_div;
vco = radeon_div(tmp, ref_div);
- if (vco < pll->pll_out_min) {
+ if (vco < pll_out_min) {
min_feed_div = feedback_div + 1;
continue;
- } else if (vco > pll->pll_out_max) {
+ } else if (vco > pll_out_max) {
max_feed_div = feedback_div;
continue;
}
{
fixed20_12 ffreq, max_error, error, pll_out, a;
u32 vco;
+ u32 pll_out_min, pll_out_max;
+
+ if (pll->flags & RADEON_PLL_IS_LCD) {
+ pll_out_min = pll->lcd_pll_out_min;
+ pll_out_max = pll->lcd_pll_out_max;
+ } else {
+ pll_out_min = pll->pll_out_min;
+ pll_out_max = pll->pll_out_max;
+ }
ffreq.full = rfixed_const(freq);
/* max_error = ffreq * 0.0025; */
vco = pll->reference_freq * (((*fb_div) * 10) + (*fb_div_frac));
vco = vco / ((*ref_div) * 10);
- if ((vco < pll->pll_out_min) || (vco > pll->pll_out_max))
+ if ((vco < pll_out_min) || (vco > pll_out_max))
continue;
/* pll_out = vco / post_div; */
{
u32 fb_div = 0, fb_div_frac = 0, post_div = 0, ref_div = 0;
u32 best_freq = 0, vco_frequency;
+ u32 pll_out_min, pll_out_max;
+
+ if (pll->flags & RADEON_PLL_IS_LCD) {
+ pll_out_min = pll->lcd_pll_out_min;
+ pll_out_max = pll->lcd_pll_out_max;
+ } else {
+ pll_out_min = pll->pll_out_min;
+ pll_out_max = pll->pll_out_max;
+ }
/* freq = freq / 10; */
do_div(freq, 10);
goto done;
vco_frequency = freq * post_div;
- if ((vco_frequency < pll->pll_out_min) || (vco_frequency > pll->pll_out_max))
+ if ((vco_frequency < pll_out_min) || (vco_frequency > pll_out_max))
goto done;
if (pll->flags & RADEON_PLL_USE_REF_DIV) {
continue;
vco_frequency = freq * post_div;
- if ((vco_frequency < pll->pll_out_min) || (vco_frequency > pll->pll_out_max))
+ if ((vco_frequency < pll_out_min) || (vco_frequency > pll_out_max))
continue;
if (pll->flags & RADEON_PLL_USE_REF_DIV) {
ref_div = pll->reference_div;
return 0;
}
+void radeon_update_display_priority(struct radeon_device *rdev)
+{
+ /* adjustment options for the display watermarks */
+ if ((radeon_disp_priority == 0) || (radeon_disp_priority > 2)) {
+ /* set display priority to high for r3xx, rv515 chips
+ * this avoids flickering due to underflow to the
+ * display controllers during heavy acceleration.
+ */
+ if (ASIC_IS_R300(rdev) || (rdev->family == CHIP_RV515))
+ rdev->disp_priority = 2;
+ else
+ rdev->disp_priority = 0;
+ } else
+ rdev->disp_priority = radeon_disp_priority;
+
+}
+
int radeon_modeset_init(struct radeon_device *rdev)
{
int i;
radeon_combios_check_hardcoded_edid(rdev);
}
- if (rdev->flags & RADEON_SINGLE_CRTC)
- rdev->num_crtc = 1;
- else {
- if (ASIC_IS_DCE4(rdev))
- rdev->num_crtc = 6;
- else
- rdev->num_crtc = 2;
- }
-
/* allocate crtcs */
for (i = 0; i < rdev->num_crtc; i++) {
radeon_crtc_init(rdev->ddev, i);
* KMS wrapper.
* - 2.0.0 - initial interface
* - 2.1.0 - add square tiling interface
+ * - 2.2.0 - add r6xx/r7xx const buffer support
*/
#define KMS_DRIVER_MAJOR 2
-#define KMS_DRIVER_MINOR 1
+#define KMS_DRIVER_MINOR 2
#define KMS_DRIVER_PATCHLEVEL 0
int radeon_driver_load_kms(struct drm_device *dev, unsigned long flags);
int radeon_driver_unload_kms(struct drm_device *dev);
int radeon_new_pll = -1;
int radeon_dynpm = -1;
int radeon_audio = 1;
+int radeon_disp_priority = 0;
+int radeon_hw_i2c = 0;
MODULE_PARM_DESC(no_wb, "Disable AGP writeback for scratch registers");
module_param_named(no_wb, radeon_no_wb, int, 0444);
MODULE_PARM_DESC(audio, "Audio enable (0 = disable)");
module_param_named(audio, radeon_audio, int, 0444);
+MODULE_PARM_DESC(disp_priority, "Display Priority (0 = auto, 1 = normal, 2 = high)");
+module_param_named(disp_priority, radeon_disp_priority, int, 0444);
+
+MODULE_PARM_DESC(hw_i2c, "hw i2c engine enable (0 = disable)");
+module_param_named(hw_i2c, radeon_hw_i2c, int, 0444);
+
static int radeon_suspend(struct drm_device *dev, pm_message_t state)
{
drm_radeon_private_t *dev_priv = dev->dev_private;
* 1.30- Add support for occlusion queries
* 1.31- Add support for num Z pipes from GET_PARAM
* 1.32- fixes for rv740 setup
+ * 1.33- Add r6xx/r7xx const buffer support
*/
#define DRIVER_MAJOR 1
-#define DRIVER_MINOR 32
+#define DRIVER_MINOR 33
#define DRIVER_PATCHLEVEL 0
enum radeon_cp_microcode_version {
}
if (ASIC_IS_DCE3(rdev) &&
- (radeon_encoder->active_device & (ATOM_DEVICE_DFP_SUPPORT))) {
+ (radeon_encoder->active_device & (ATOM_DEVICE_DFP_SUPPORT | ATOM_DEVICE_LCD_SUPPORT))) {
struct drm_connector *connector = radeon_get_connector_for_encoder(encoder);
radeon_dp_set_link_config(connector, mode);
}
break;
}
- atom_parse_cmd_header(rdev->mode_info.atom_context, index, &frev, &crev);
+ if (!atom_parse_cmd_header(rdev->mode_info.atom_context, index, &frev, &crev))
+ return;
switch (frev) {
case 1:
}
atom_execute_table(rdev->mode_info.atom_context, index, (uint32_t *)&args);
- r600_hdmi_enable(encoder, hdmi_detected);
}
int
struct radeon_connector_atom_dig *dig_connector =
radeon_get_atom_connector_priv_from_encoder(encoder);
union dig_encoder_control args;
- int index = 0, num = 0;
+ int index = 0;
uint8_t frev, crev;
if (!dig || !dig_connector)
else
index = GetIndexIntoMasterTable(COMMAND, DIG1EncoderControl);
}
- num = dig->dig_encoder + 1;
- atom_parse_cmd_header(rdev->mode_info.atom_context, index, &frev, &crev);
+ if (!atom_parse_cmd_header(rdev->mode_info.atom_context, index, &frev, &crev))
+ return;
args.v1.ucAction = action;
args.v1.usPixelClock = cpu_to_le16(radeon_encoder->pixel_clock / 10);
struct drm_connector *connector;
struct radeon_connector *radeon_connector;
union dig_transmitter_control args;
- int index = 0, num = 0;
+ int index = 0;
uint8_t frev, crev;
bool is_dp = false;
int pll_id = 0;
}
}
- atom_parse_cmd_header(rdev->mode_info.atom_context, index, &frev, &crev);
+ if (!atom_parse_cmd_header(rdev->mode_info.atom_context, index, &frev, &crev))
+ return;
args.v1.ucAction = action;
if (action == ATOM_TRANSMITTER_ACTION_INIT) {
switch (radeon_encoder->encoder_id) {
case ENCODER_OBJECT_ID_INTERNAL_UNIPHY:
args.v3.acConfig.ucTransmitterSel = 0;
- num = 0;
break;
case ENCODER_OBJECT_ID_INTERNAL_UNIPHY1:
args.v3.acConfig.ucTransmitterSel = 1;
- num = 1;
break;
case ENCODER_OBJECT_ID_INTERNAL_UNIPHY2:
args.v3.acConfig.ucTransmitterSel = 2;
- num = 2;
break;
}
args.v3.acConfig.fCoherentMode = 1;
}
} else if (ASIC_IS_DCE32(rdev)) {
- if (dig->dig_encoder == 1)
- args.v2.acConfig.ucEncoderSel = 1;
+ args.v2.acConfig.ucEncoderSel = dig->dig_encoder;
if (dig_connector->linkb)
args.v2.acConfig.ucLinkSel = 1;
switch (radeon_encoder->encoder_id) {
case ENCODER_OBJECT_ID_INTERNAL_UNIPHY:
args.v2.acConfig.ucTransmitterSel = 0;
- num = 0;
break;
case ENCODER_OBJECT_ID_INTERNAL_UNIPHY1:
args.v2.acConfig.ucTransmitterSel = 1;
- num = 1;
break;
case ENCODER_OBJECT_ID_INTERNAL_UNIPHY2:
args.v2.acConfig.ucTransmitterSel = 2;
- num = 2;
break;
}
else
args.v1.ucConfig |= ATOM_TRANSMITTER_CONFIG_DIG1_ENCODER;
- switch (radeon_encoder->encoder_id) {
- case ENCODER_OBJECT_ID_INTERNAL_UNIPHY:
- if (rdev->flags & RADEON_IS_IGP) {
- if (radeon_encoder->pixel_clock > 165000) {
- if (dig_connector->igp_lane_info & 0x3)
- args.v1.ucConfig |= ATOM_TRANSMITTER_CONFIG_LANE_0_7;
- else if (dig_connector->igp_lane_info & 0xc)
- args.v1.ucConfig |= ATOM_TRANSMITTER_CONFIG_LANE_8_15;
- } else {
- if (dig_connector->igp_lane_info & 0x1)
- args.v1.ucConfig |= ATOM_TRANSMITTER_CONFIG_LANE_0_3;
- else if (dig_connector->igp_lane_info & 0x2)
- args.v1.ucConfig |= ATOM_TRANSMITTER_CONFIG_LANE_4_7;
- else if (dig_connector->igp_lane_info & 0x4)
- args.v1.ucConfig |= ATOM_TRANSMITTER_CONFIG_LANE_8_11;
- else if (dig_connector->igp_lane_info & 0x8)
- args.v1.ucConfig |= ATOM_TRANSMITTER_CONFIG_LANE_12_15;
- }
+ if ((rdev->flags & RADEON_IS_IGP) &&
+ (radeon_encoder->encoder_id == ENCODER_OBJECT_ID_INTERNAL_UNIPHY)) {
+ if (is_dp || (radeon_encoder->pixel_clock <= 165000)) {
+ if (dig_connector->igp_lane_info & 0x1)
+ args.v1.ucConfig |= ATOM_TRANSMITTER_CONFIG_LANE_0_3;
+ else if (dig_connector->igp_lane_info & 0x2)
+ args.v1.ucConfig |= ATOM_TRANSMITTER_CONFIG_LANE_4_7;
+ else if (dig_connector->igp_lane_info & 0x4)
+ args.v1.ucConfig |= ATOM_TRANSMITTER_CONFIG_LANE_8_11;
+ else if (dig_connector->igp_lane_info & 0x8)
+ args.v1.ucConfig |= ATOM_TRANSMITTER_CONFIG_LANE_12_15;
+ } else {
+ if (dig_connector->igp_lane_info & 0x3)
+ args.v1.ucConfig |= ATOM_TRANSMITTER_CONFIG_LANE_0_7;
+ else if (dig_connector->igp_lane_info & 0xc)
+ args.v1.ucConfig |= ATOM_TRANSMITTER_CONFIG_LANE_8_15;
}
- break;
}
- if (radeon_encoder->pixel_clock > 165000)
- args.v1.ucConfig |= ATOM_TRANSMITTER_CONFIG_8LANE_LINK;
-
if (dig_connector->linkb)
args.v1.ucConfig |= ATOM_TRANSMITTER_CONFIG_LINKB;
else
else if (radeon_encoder->devices & (ATOM_DEVICE_DFP_SUPPORT)) {
if (dig->coherent_mode)
args.v1.ucConfig |= ATOM_TRANSMITTER_CONFIG_COHERENT;
+ if (radeon_encoder->pixel_clock > 165000)
+ args.v1.ucConfig |= ATOM_TRANSMITTER_CONFIG_8LANE_LINK;
}
}
if (is_dig) {
switch (mode) {
case DRM_MODE_DPMS_ON:
- atombios_dig_transmitter_setup(encoder, ATOM_TRANSMITTER_ACTION_ENABLE_OUTPUT, 0, 0);
- {
+ if (atombios_get_encoder_mode(encoder) == ATOM_ENCODER_MODE_DP) {
struct drm_connector *connector = radeon_get_connector_for_encoder(encoder);
+
dp_link_train(encoder, connector);
+ if (ASIC_IS_DCE4(rdev))
+ atombios_dig_encoder_setup(encoder, ATOM_ENCODER_CMD_DP_VIDEO_ON);
}
+ if (!ASIC_IS_DCE4(rdev))
+ atombios_dig_transmitter_setup(encoder, ATOM_TRANSMITTER_ACTION_ENABLE_OUTPUT, 0, 0);
break;
case DRM_MODE_DPMS_STANDBY:
case DRM_MODE_DPMS_SUSPEND:
case DRM_MODE_DPMS_OFF:
- atombios_dig_transmitter_setup(encoder, ATOM_TRANSMITTER_ACTION_DISABLE_OUTPUT, 0, 0);
+ if (!ASIC_IS_DCE4(rdev))
+ atombios_dig_transmitter_setup(encoder, ATOM_TRANSMITTER_ACTION_DISABLE_OUTPUT, 0, 0);
+ if (atombios_get_encoder_mode(encoder) == ATOM_ENCODER_MODE_DP) {
+ if (ASIC_IS_DCE4(rdev))
+ atombios_dig_encoder_setup(encoder, ATOM_ENCODER_CMD_DP_VIDEO_OFF);
+ }
break;
}
} else {
memset(&args, 0, sizeof(args));
- atom_parse_cmd_header(rdev->mode_info.atom_context, index, &frev, &crev);
+ if (!atom_parse_cmd_header(rdev->mode_info.atom_context, index, &frev, &crev))
+ return;
switch (frev) {
case 1:
}
atom_execute_table(rdev->mode_info.atom_context, index, (uint32_t *)&args);
+
+ /* update scratch regs with new routing */
+ radeon_atombios_encoder_crtc_scratch_regs(encoder, radeon_crtc->crtc_id);
}
static void
struct drm_device *dev = encoder->dev;
struct radeon_device *rdev = dev->dev_private;
struct radeon_encoder *radeon_encoder = to_radeon_encoder(encoder);
- struct radeon_crtc *radeon_crtc = to_radeon_crtc(encoder->crtc);
- if (radeon_encoder->active_device &
- (ATOM_DEVICE_DFP_SUPPORT | ATOM_DEVICE_LCD_SUPPORT)) {
- struct radeon_encoder_atom_dig *dig = radeon_encoder->enc_priv;
- if (dig)
- dig->dig_encoder = radeon_atom_pick_dig_encoder(encoder);
- }
radeon_encoder->pixel_clock = adjusted_mode->clock;
- radeon_atombios_encoder_crtc_scratch_regs(encoder, radeon_crtc->crtc_id);
- atombios_set_encoder_crtc_source(encoder);
-
if (ASIC_IS_AVIVO(rdev)) {
if (radeon_encoder->active_device & (ATOM_DEVICE_CV_SUPPORT | ATOM_DEVICE_TV_SUPPORT))
atombios_yuv_setup(encoder, true);
}
atombios_apply_encoder_quirks(encoder, adjusted_mode);
- /* XXX */
- if (!ASIC_IS_DCE4(rdev))
+ if (atombios_get_encoder_mode(encoder) == ATOM_ENCODER_MODE_HDMI) {
+ r600_hdmi_enable(encoder);
r600_hdmi_setmode(encoder, adjusted_mode);
+ }
}
static bool
memset(&args, 0, sizeof(args));
- atom_parse_cmd_header(rdev->mode_info.atom_context, index, &frev, &crev);
+ if (!atom_parse_cmd_header(rdev->mode_info.atom_context, index, &frev, &crev))
+ return false;
args.sDacload.ucMisc = 0;
static void radeon_atom_encoder_prepare(struct drm_encoder *encoder)
{
+ struct radeon_encoder *radeon_encoder = to_radeon_encoder(encoder);
+
+ if (radeon_encoder->active_device &
+ (ATOM_DEVICE_DFP_SUPPORT | ATOM_DEVICE_LCD_SUPPORT)) {
+ struct radeon_encoder_atom_dig *dig = radeon_encoder->enc_priv;
+ if (dig)
+ dig->dig_encoder = radeon_atom_pick_dig_encoder(encoder);
+ }
+
radeon_atom_output_lock(encoder, true);
radeon_atom_encoder_dpms(encoder, DRM_MODE_DPMS_OFF);
+
+ /* this is needed for the pll/ss setup to work correctly in some cases */
+ atombios_set_encoder_crtc_source(encoder);
}
static void radeon_atom_encoder_commit(struct drm_encoder *encoder)
radeon_atom_encoder_dpms(encoder, DRM_MODE_DPMS_OFF);
if (radeon_encoder_is_digital(encoder)) {
+ if (atombios_get_encoder_mode(encoder) == ATOM_ENCODER_MODE_HDMI)
+ r600_hdmi_disable(encoder);
dig = radeon_encoder->enc_priv;
dig->dig_encoder = -1;
}
drm_encoder_helper_add(encoder, &radeon_atom_dig_helper_funcs);
break;
}
-
- r600_hdmi_init(encoder);
}
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/fb.h>
#include "drmP.h"
#include <linux/wait.h>
#include <linux/list.h>
#include <linux/kref.h>
+#include <linux/slab.h>
#include "drmP.h"
#include "drm.h"
#include "radeon_reg.h"
return false;
}
+/* bit banging i2c */
static void radeon_i2c_do_lock(struct radeon_i2c_chan *i2c, int lock_state)
{
WREG32(rec->en_data_reg, val);
}
+static int pre_xfer(struct i2c_adapter *i2c_adap)
+{
+ struct radeon_i2c_chan *i2c = i2c_get_adapdata(i2c_adap);
+
+ radeon_i2c_do_lock(i2c, 1);
+
+ return 0;
+}
+
+static void post_xfer(struct i2c_adapter *i2c_adap)
+{
+ struct radeon_i2c_chan *i2c = i2c_get_adapdata(i2c_adap);
+
+ radeon_i2c_do_lock(i2c, 0);
+}
+
+/* hw i2c */
+
static u32 radeon_get_i2c_prescale(struct radeon_device *rdev)
{
- struct radeon_pll *spll = &rdev->clock.spll;
u32 sclk = radeon_get_engine_clock(rdev);
u32 prescale = 0;
- u32 n, m;
- u8 loop;
+ u32 nm;
+ u8 n, m, loop;
int i2c_clock;
switch (rdev->family) {
case CHIP_R300:
case CHIP_R350:
case CHIP_RV350:
- n = (spll->reference_freq) / (4 * 6);
+ i2c_clock = 60;
+ nm = (sclk * 10) / (i2c_clock * 4);
for (loop = 1; loop < 255; loop++) {
- if ((loop * (loop - 1)) > n)
+ if ((nm / loop) < loop)
break;
}
- m = loop - 1;
- prescale = m | (loop << 8);
+ n = loop - 1;
+ m = loop - 2;
+ prescale = m | (n << 8);
break;
case CHIP_RV380:
case CHIP_RS400:
case CHIP_R420:
case CHIP_R423:
case CHIP_RV410:
- sclk = radeon_get_engine_clock(rdev);
prescale = (((sclk * 10)/(4 * 128 * 100) + 1) << 8) + 128;
break;
case CHIP_RS600:
case CHIP_RV570:
case CHIP_R580:
i2c_clock = 50;
- sclk = radeon_get_engine_clock(rdev);
if (rdev->family == CHIP_R520)
prescale = (127 << 8) + ((sclk * 10) / (4 * 127 * i2c_clock));
else
prescale = radeon_get_i2c_prescale(rdev);
reg = ((prescale << RADEON_I2C_PRESCALE_SHIFT) |
+ RADEON_I2C_DRIVE_EN |
RADEON_I2C_START |
RADEON_I2C_STOP |
RADEON_I2C_GO);
return ret;
}
-static int radeon_sw_i2c_xfer(struct i2c_adapter *i2c_adap,
+static int radeon_hw_i2c_xfer(struct i2c_adapter *i2c_adap,
struct i2c_msg *msgs, int num)
-{
- struct radeon_i2c_chan *i2c = i2c_get_adapdata(i2c_adap);
- int ret;
-
- radeon_i2c_do_lock(i2c, 1);
- ret = i2c_transfer(&i2c->algo.radeon.bit_adapter, msgs, num);
- radeon_i2c_do_lock(i2c, 0);
-
- return ret;
-}
-
-static int radeon_i2c_xfer(struct i2c_adapter *i2c_adap,
- struct i2c_msg *msgs, int num)
{
struct radeon_i2c_chan *i2c = i2c_get_adapdata(i2c_adap);
struct radeon_device *rdev = i2c->dev->dev_private;
struct radeon_i2c_bus_rec *rec = &i2c->rec;
- int ret;
+ int ret = 0;
switch (rdev->family) {
case CHIP_R100:
case CHIP_RV410:
case CHIP_RS400:
case CHIP_RS480:
- if (rec->hw_capable)
- ret = r100_hw_i2c_xfer(i2c_adap, msgs, num);
- else
- ret = radeon_sw_i2c_xfer(i2c_adap, msgs, num);
+ ret = r100_hw_i2c_xfer(i2c_adap, msgs, num);
break;
case CHIP_RS600:
case CHIP_RS690:
case CHIP_RS740:
/* XXX fill in hw i2c implementation */
- ret = radeon_sw_i2c_xfer(i2c_adap, msgs, num);
break;
case CHIP_RV515:
case CHIP_R520:
case CHIP_RV560:
case CHIP_RV570:
case CHIP_R580:
- if (rec->hw_capable) {
- if (rec->mm_i2c)
- ret = r100_hw_i2c_xfer(i2c_adap, msgs, num);
- else
- ret = r500_hw_i2c_xfer(i2c_adap, msgs, num);
- } else
- ret = radeon_sw_i2c_xfer(i2c_adap, msgs, num);
+ if (rec->mm_i2c)
+ ret = r100_hw_i2c_xfer(i2c_adap, msgs, num);
+ else
+ ret = r500_hw_i2c_xfer(i2c_adap, msgs, num);
break;
case CHIP_R600:
case CHIP_RV610:
case CHIP_RV630:
case CHIP_RV670:
/* XXX fill in hw i2c implementation */
- ret = radeon_sw_i2c_xfer(i2c_adap, msgs, num);
break;
case CHIP_RV620:
case CHIP_RV635:
case CHIP_RV710:
case CHIP_RV740:
/* XXX fill in hw i2c implementation */
- ret = radeon_sw_i2c_xfer(i2c_adap, msgs, num);
break;
case CHIP_CEDAR:
case CHIP_REDWOOD:
case CHIP_CYPRESS:
case CHIP_HEMLOCK:
/* XXX fill in hw i2c implementation */
- ret = radeon_sw_i2c_xfer(i2c_adap, msgs, num);
break;
default:
DRM_ERROR("i2c: unhandled radeon chip\n");
return ret;
}
-static u32 radeon_i2c_func(struct i2c_adapter *adap)
+static u32 radeon_hw_i2c_func(struct i2c_adapter *adap)
{
return I2C_FUNC_I2C | I2C_FUNC_SMBUS_EMUL;
}
static const struct i2c_algorithm radeon_i2c_algo = {
- .master_xfer = radeon_i2c_xfer,
- .functionality = radeon_i2c_func,
+ .master_xfer = radeon_hw_i2c_xfer,
+ .functionality = radeon_hw_i2c_func,
};
struct radeon_i2c_chan *radeon_i2c_create(struct drm_device *dev,
struct radeon_i2c_bus_rec *rec,
const char *name)
{
+ struct radeon_device *rdev = dev->dev_private;
struct radeon_i2c_chan *i2c;
int ret;
if (i2c == NULL)
return NULL;
- /* set the internal bit adapter */
- i2c->algo.radeon.bit_adapter.owner = THIS_MODULE;
- i2c_set_adapdata(&i2c->algo.radeon.bit_adapter, i2c);
- sprintf(i2c->algo.radeon.bit_adapter.name, "Radeon internal i2c bit bus %s", name);
- i2c->algo.radeon.bit_adapter.algo_data = &i2c->algo.radeon.bit_data;
- i2c->algo.radeon.bit_data.setsda = set_data;
- i2c->algo.radeon.bit_data.setscl = set_clock;
- i2c->algo.radeon.bit_data.getsda = get_data;
- i2c->algo.radeon.bit_data.getscl = get_clock;
- i2c->algo.radeon.bit_data.udelay = 20;
- /* vesa says 2.2 ms is enough, 1 jiffy doesn't seem to always
- * make this, 2 jiffies is a lot more reliable */
- i2c->algo.radeon.bit_data.timeout = 2;
- i2c->algo.radeon.bit_data.data = i2c;
- ret = i2c_bit_add_bus(&i2c->algo.radeon.bit_adapter);
- if (ret) {
- DRM_ERROR("Failed to register internal bit i2c %s\n", name);
- goto out_free;
- }
- /* set the radeon i2c adapter */
- i2c->dev = dev;
i2c->rec = *rec;
i2c->adapter.owner = THIS_MODULE;
+ i2c->dev = dev;
i2c_set_adapdata(&i2c->adapter, i2c);
- sprintf(i2c->adapter.name, "Radeon i2c %s", name);
- i2c->adapter.algo_data = &i2c->algo.radeon;
- i2c->adapter.algo = &radeon_i2c_algo;
- ret = i2c_add_adapter(&i2c->adapter);
- if (ret) {
- DRM_ERROR("Failed to register i2c %s\n", name);
- goto out_free;
+ if (rec->mm_i2c ||
+ (rec->hw_capable &&
+ radeon_hw_i2c &&
+ ((rdev->family <= CHIP_RS480) ||
+ ((rdev->family >= CHIP_RV515) && (rdev->family <= CHIP_R580))))) {
+ /* set the radeon hw i2c adapter */
+ sprintf(i2c->adapter.name, "Radeon i2c hw bus %s", name);
+ i2c->adapter.algo = &radeon_i2c_algo;
+ ret = i2c_add_adapter(&i2c->adapter);
+ if (ret) {
+ DRM_ERROR("Failed to register hw i2c %s\n", name);
+ goto out_free;
+ }
+ } else {
+ /* set the radeon bit adapter */
+ sprintf(i2c->adapter.name, "Radeon i2c bit bus %s", name);
+ i2c->adapter.algo_data = &i2c->algo.bit;
+ i2c->algo.bit.pre_xfer = pre_xfer;
+ i2c->algo.bit.post_xfer = post_xfer;
+ i2c->algo.bit.setsda = set_data;
+ i2c->algo.bit.setscl = set_clock;
+ i2c->algo.bit.getsda = get_data;
+ i2c->algo.bit.getscl = get_clock;
+ i2c->algo.bit.udelay = 20;
+ /* vesa says 2.2 ms is enough, 1 jiffy doesn't seem to always
+ * make this, 2 jiffies is a lot more reliable */
+ i2c->algo.bit.timeout = 2;
+ i2c->algo.bit.data = i2c;
+ ret = i2c_bit_add_bus(&i2c->adapter);
+ if (ret) {
+ DRM_ERROR("Failed to register bit i2c %s\n", name);
+ goto out_free;
+ }
}
return i2c;
{
if (!i2c)
return;
- i2c_del_adapter(&i2c->algo.radeon.bit_adapter);
- i2c_del_adapter(&i2c->adapter);
- kfree(i2c);
-}
-
-void radeon_i2c_destroy_dp(struct radeon_i2c_chan *i2c)
-{
- if (!i2c)
- return;
-
i2c_del_adapter(&i2c->adapter);
kfree(i2c);
}
/* Disable *all* interrupts */
rdev->irq.sw_int = false;
- for (i = 0; i < 2; i++) {
+ for (i = 0; i < rdev->num_crtc; i++)
rdev->irq.crtc_vblank_int[i] = false;
- }
+ for (i = 0; i < 6; i++)
+ rdev->irq.hpd[i] = false;
radeon_irq_set(rdev);
/* Clear bits */
radeon_irq_process(rdev);
}
/* Disable *all* interrupts */
rdev->irq.sw_int = false;
- for (i = 0; i < 2; i++) {
+ for (i = 0; i < rdev->num_crtc; i++)
rdev->irq.crtc_vblank_int[i] = false;
+ for (i = 0; i < 6; i++)
rdev->irq.hpd[i] = false;
- }
radeon_irq_set(rdev);
}
int radeon_irq_kms_init(struct radeon_device *rdev)
{
int r = 0;
- int num_crtc = 2;
- if (rdev->flags & RADEON_SINGLE_CRTC)
- num_crtc = 1;
spin_lock_init(&rdev->irq.sw_lock);
- r = drm_vblank_init(rdev->ddev, num_crtc);
+ r = drm_vblank_init(rdev->ddev, rdev->num_crtc);
if (r) {
return r;
}
/* enable msi */
rdev->msi_enabled = 0;
- /* MSIs don't seem to work on my rs780;
- * not sure about rs880 or other rs780s.
- * Needs more investigation.
+ /* MSIs don't seem to work reliably on all IGP
+ * chips. Disable MSI on them for now.
*/
if ((rdev->family >= CHIP_RV380) &&
- (rdev->family != CHIP_RS780) &&
- (rdev->family != CHIP_RS880)) {
+ (!(rdev->flags & RADEON_IS_IGP))) {
int ret = pci_enable_msi(rdev->pdev);
if (!ret) {
rdev->msi_enabled = 1;
#include "radeon_drm.h"
#include <linux/vga_switcheroo.h>
+#include <linux/slab.h>
int radeon_driver_unload_kms(struct drm_device *dev)
{
? RADEON_CRTC2_INTERLACE_EN
: 0));
+ /* rs4xx chips seem to like to have the crtc enabled when the timing is set */
+ if ((rdev->family == CHIP_RS400) || (rdev->family == CHIP_RS480))
+ crtc2_gen_cntl |= RADEON_CRTC2_EN;
+
disp2_merge_cntl = RREG32(RADEON_DISP2_MERGE_CNTL);
disp2_merge_cntl &= ~RADEON_DISP2_RGB_OFFSET_EN;
? RADEON_CRTC_INTERLACE_EN
: 0));
+ /* rs4xx chips seem to like to have the crtc enabled when the timing is set */
+ if ((rdev->family == CHIP_RS400) || (rdev->family == CHIP_RS480))
+ crtc_gen_cntl |= RADEON_CRTC_EN;
+
crtc_ext_cntl = RREG32(RADEON_CRTC_EXT_CNTL);
crtc_ext_cntl |= (RADEON_XCRT_CNT_EN |
RADEON_CRTC_VSYNC_DIS |
#define NTSC_TV_PLL_N_14 693
#define NTSC_TV_PLL_P_14 7
+#define PAL_TV_PLL_M_14 19
+#define PAL_TV_PLL_N_14 353
+#define PAL_TV_PLL_P_14 5
+
#define VERT_LEAD_IN_LINES 2
#define FRAC_BITS 0xe
#define FRAC_MASK 0x3fff
630627, /* defRestart */
347, /* crtcPLL_N */
14, /* crtcPLL_M */
- 8, /* crtcPLL_postDiv */
+ 8, /* crtcPLL_postDiv */
1022, /* pixToTV */
},
+ { /* PAL timing for 14 Mhz ref clk */
+ 800, /* horResolution */
+ 600, /* verResolution */
+ TV_STD_PAL, /* standard */
+ 1131, /* horTotal */
+ 742, /* verTotal */
+ 813, /* horStart */
+ 840, /* horSyncStart */
+ 633, /* verSyncStart */
+ 708369, /* defRestart */
+ 211, /* crtcPLL_N */
+ 9, /* crtcPLL_M */
+ 8, /* crtcPLL_postDiv */
+ 759, /* pixToTV */
+ },
};
#define N_AVAILABLE_MODES ARRAY_SIZE(available_tv_modes)
if (pll->reference_freq == 2700)
const_ptr = &available_tv_modes[1];
else
- const_ptr = &available_tv_modes[1]; /* FIX ME */
+ const_ptr = &available_tv_modes[3];
}
return const_ptr;
}
n = PAL_TV_PLL_N_27;
p = PAL_TV_PLL_P_27;
} else {
- m = PAL_TV_PLL_M_27;
- n = PAL_TV_PLL_N_27;
- p = PAL_TV_PLL_P_27;
+ m = PAL_TV_PLL_M_14;
+ n = PAL_TV_PLL_N_14;
+ p = PAL_TV_PLL_P_14;
}
}
#define RADEON_PLL_USE_FRAC_FB_DIV (1 << 10)
#define RADEON_PLL_PREFER_CLOSEST_LOWER (1 << 11)
#define RADEON_PLL_USE_POST_DIV (1 << 12)
+#define RADEON_PLL_IS_LCD (1 << 13)
/* pll algo */
enum radeon_pll_algo {
uint32_t pll_in_max;
uint32_t pll_out_min;
uint32_t pll_out_max;
+ uint32_t lcd_pll_out_min;
+ uint32_t lcd_pll_out_max;
uint32_t best_vco;
/* divider limits */
enum radeon_pll_algo algo;
};
-struct i2c_algo_radeon_data {
- struct i2c_adapter bit_adapter;
- struct i2c_algo_bit_data bit_data;
-};
-
struct radeon_i2c_chan {
struct i2c_adapter adapter;
struct drm_device *dev;
union {
+ struct i2c_algo_bit_data bit;
struct i2c_algo_dp_aux_data dp;
- struct i2c_algo_radeon_data radeon;
} algo;
struct radeon_i2c_bus_rec rec;
};
struct drm_display_mode native_mode;
void *enc_priv;
int hdmi_offset;
+ int hdmi_config_offset;
int hdmi_audio_workaround;
int hdmi_buffer_status;
};
struct radeon_i2c_bus_rec *rec,
const char *name);
extern void radeon_i2c_destroy(struct radeon_i2c_chan *i2c);
-extern void radeon_i2c_destroy_dp(struct radeon_i2c_chan *i2c);
extern void radeon_i2c_get_byte(struct radeon_i2c_chan *i2c_bus,
u8 slave_addr,
u8 addr,
* Dave Airlie
*/
#include <linux/list.h>
+#include <linux/slab.h>
#include <drm/drmP.h>
#include "radeon_drm.h"
#include "radeon.h"
return 0;
}
radeon_ttm_placement_from_domain(bo, domain);
- /* force to pin into visible video ram */
- bo->placement.lpfn = bo->rdev->mc.visible_vram_size >> PAGE_SHIFT;
+ if (domain == RADEON_GEM_DOMAIN_VRAM) {
+ /* force to pin into visible video ram */
+ bo->placement.lpfn = bo->rdev->mc.visible_vram_size >> PAGE_SHIFT;
+ }
for (i = 0; i < bo->placement.num_placement; i++)
bo->placements[i] |= TTM_PL_FLAG_NO_EVICT;
r = ttm_bo_validate(&bo->tbo, &bo->placement, false, false);
#define RADEON_RECLOCK_DELAY_MS 200
#define RADEON_WAIT_VBLANK_TIMEOUT 200
+static bool radeon_pm_debug_check_in_vbl(struct radeon_device *rdev, bool finish);
static void radeon_pm_set_clocks_locked(struct radeon_device *rdev);
static void radeon_pm_set_clocks(struct radeon_device *rdev);
static void radeon_pm_idle_work_handler(struct work_struct *work);
rdev->pm.requested_power_state->non_clock_info.pcie_lanes);
}
+static inline void radeon_sync_with_vblank(struct radeon_device *rdev)
+{
+ if (rdev->pm.active_crtcs) {
+ rdev->pm.vblank_sync = false;
+ wait_event_timeout(
+ rdev->irq.vblank_queue, rdev->pm.vblank_sync,
+ msecs_to_jiffies(RADEON_WAIT_VBLANK_TIMEOUT));
+ }
+}
+
static void radeon_set_power_state(struct radeon_device *rdev)
{
/* if *_clock_mode are the same, *_power_state are as well */
rdev->pm.requested_clock_mode->sclk,
rdev->pm.requested_clock_mode->mclk,
rdev->pm.requested_power_state->non_clock_info.pcie_lanes);
+
/* set pcie lanes */
+ /* TODO */
+
/* set voltage */
+ /* TODO */
+
/* set engine clock */
+ radeon_sync_with_vblank(rdev);
+ radeon_pm_debug_check_in_vbl(rdev, false);
radeon_set_engine_clock(rdev, rdev->pm.requested_clock_mode->sclk);
+ radeon_pm_debug_check_in_vbl(rdev, true);
+
+#if 0
/* set memory clock */
+ if (rdev->asic->set_memory_clock) {
+ radeon_sync_with_vblank(rdev);
+ radeon_pm_debug_check_in_vbl(rdev, false);
+ radeon_set_memory_clock(rdev, rdev->pm.requested_clock_mode->mclk);
+ radeon_pm_debug_check_in_vbl(rdev, true);
+ }
+#endif
rdev->pm.current_power_state = rdev->pm.requested_power_state;
rdev->pm.current_clock_mode = rdev->pm.requested_clock_mode;
return 0;
}
+void radeon_pm_fini(struct radeon_device *rdev)
+{
+ if (rdev->pm.i2c_bus)
+ radeon_i2c_destroy(rdev->pm.i2c_bus);
+}
+
void radeon_pm_compute_clocks(struct radeon_device *rdev)
{
struct drm_device *ddev = rdev->ddev;
list_for_each_entry(connector,
&ddev->mode_config.connector_list, head) {
if (connector->encoder &&
- connector->dpms != DRM_MODE_DPMS_OFF) {
+ connector->encoder->crtc &&
+ connector->dpms != DRM_MODE_DPMS_OFF) {
radeon_crtc = to_radeon_crtc(connector->encoder->crtc);
rdev->pm.active_crtcs |= (1 << radeon_crtc->crtc_id);
++count;
break;
}
- /* check if we are in vblank */
- radeon_pm_debug_check_in_vbl(rdev, false);
radeon_set_power_state(rdev);
- radeon_pm_debug_check_in_vbl(rdev, true);
rdev->pm.planned_action = PM_ACTION_NONE;
}
rdev->pm.req_vblank |= (1 << 1);
drm_vblank_get(rdev->ddev, 1);
}
- if (rdev->pm.active_crtcs)
- wait_event_interruptible_timeout(
- rdev->irq.vblank_queue, 0,
- msecs_to_jiffies(RADEON_WAIT_VBLANK_TIMEOUT));
+ radeon_pm_set_clocks_locked(rdev);
if (rdev->pm.req_vblank & (1 << 0)) {
rdev->pm.req_vblank &= ~(1 << 0);
drm_vblank_put(rdev->ddev, 0);
drm_vblank_put(rdev->ddev, 1);
}
- radeon_pm_set_clocks_locked(rdev);
mutex_unlock(&rdev->cp.mutex);
}
# define RADEON_TVPLL_PWRMGT_OFF (1 << 30)
# define RADEON_TVCLK_TURNOFF (1 << 31)
#define RADEON_PLL_PWRMGT_CNTL 0x0015 /* PLL */
+# define RADEON_PM_MODE_SEL (1 << 13)
# define RADEON_TCL_BYPASS_DISABLE (1 << 20)
#define RADEON_CLR_CMP_CLR_3D 0x1a24
#define RADEON_CLR_CMP_CLR_DST 0x15c8
* Jerome Glisse
*/
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include "drmP.h"
#include "radeon_drm.h"
#include "radeon_reg.h"
#include <drm/drmP.h>
#include <drm/radeon_drm.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include "radeon_reg.h"
#include "radeon.h"
0x00028408 VGT_INDX_OFFSET
0x00028AA0 VGT_INSTANCE_STEP_RATE_0
0x00028AA4 VGT_INSTANCE_STEP_RATE_1
-0x000088C0 VGT_LAST_COPY_STATE
0x00028400 VGT_MAX_VTX_INDX
-0x000088D8 VGT_MC_LAT_CNTL
0x00028404 VGT_MIN_VTX_INDX
0x00028A94 VGT_MULTI_PRIM_IB_RESET_EN
0x0002840C VGT_MULTI_PRIM_IB_RESET_INDX
0x00008970 VGT_NUM_INDICES
0x00008974 VGT_NUM_INSTANCES
0x00028A10 VGT_OUTPUT_PATH_CNTL
-0x00028C5C VGT_OUT_DEALLOC_CNTL
0x00028A84 VGT_PRIMITIVEID_EN
0x00008958 VGT_PRIMITIVE_TYPE
0x00028AB4 VGT_REUSE_OFF
-0x00028C58 VGT_VERTEX_REUSE_BLOCK_CNTL
0x00028AB8 VGT_VTX_CNT_EN
0x000088B0 VGT_VTX_VECT_EJECT_REG
0x00028810 PA_CL_CLIP_CNTL
0x00028E00 PA_SU_POLY_OFFSET_FRONT_SCALE
0x00028814 PA_SU_SC_MODE_CNTL
0x00028C08 PA_SU_VTX_CNTL
-0x00008C00 SQ_CONFIG
0x00008C04 SQ_GPR_RESOURCE_MGMT_1
0x00008C08 SQ_GPR_RESOURCE_MGMT_2
0x00008C10 SQ_STACK_RESOURCE_MGMT_1
0x000283FC SQ_VTX_SEMANTIC_31
0x000288E0 SQ_VTX_SEMANTIC_CLEAR
0x0003CFF4 SQ_VTX_START_INST_LOC
-0x0003C000 SQ_TEX_SAMPLER_WORD0_0
-0x0003C004 SQ_TEX_SAMPLER_WORD1_0
-0x0003C008 SQ_TEX_SAMPLER_WORD2_0
-0x00030000 SQ_ALU_CONSTANT0_0
-0x00030004 SQ_ALU_CONSTANT1_0
-0x00030008 SQ_ALU_CONSTANT2_0
-0x0003000C SQ_ALU_CONSTANT3_0
-0x0003E380 SQ_BOOL_CONST_0
-0x0003E384 SQ_BOOL_CONST_1
-0x0003E388 SQ_BOOL_CONST_2
-0x0003E200 SQ_LOOP_CONST_0
-0x0003E200 SQ_LOOP_CONST_DX10_0
0x000281C0 SQ_ALU_CONST_BUFFER_SIZE_GS_0
0x000281C4 SQ_ALU_CONST_BUFFER_SIZE_GS_1
0x000281C8 SQ_ALU_CONST_BUFFER_SIZE_GS_2
0x000281B4 SQ_ALU_CONST_BUFFER_SIZE_VS_13
0x000281B8 SQ_ALU_CONST_BUFFER_SIZE_VS_14
0x000281BC SQ_ALU_CONST_BUFFER_SIZE_VS_15
-0x000289C0 SQ_ALU_CONST_CACHE_GS_0
-0x000289C4 SQ_ALU_CONST_CACHE_GS_1
-0x000289C8 SQ_ALU_CONST_CACHE_GS_2
-0x000289CC SQ_ALU_CONST_CACHE_GS_3
-0x000289D0 SQ_ALU_CONST_CACHE_GS_4
-0x000289D4 SQ_ALU_CONST_CACHE_GS_5
-0x000289D8 SQ_ALU_CONST_CACHE_GS_6
-0x000289DC SQ_ALU_CONST_CACHE_GS_7
-0x000289E0 SQ_ALU_CONST_CACHE_GS_8
-0x000289E4 SQ_ALU_CONST_CACHE_GS_9
-0x000289E8 SQ_ALU_CONST_CACHE_GS_10
-0x000289EC SQ_ALU_CONST_CACHE_GS_11
-0x000289F0 SQ_ALU_CONST_CACHE_GS_12
-0x000289F4 SQ_ALU_CONST_CACHE_GS_13
-0x000289F8 SQ_ALU_CONST_CACHE_GS_14
-0x000289FC SQ_ALU_CONST_CACHE_GS_15
-0x00028940 SQ_ALU_CONST_CACHE_PS_0
-0x00028944 SQ_ALU_CONST_CACHE_PS_1
-0x00028948 SQ_ALU_CONST_CACHE_PS_2
-0x0002894C SQ_ALU_CONST_CACHE_PS_3
-0x00028950 SQ_ALU_CONST_CACHE_PS_4
-0x00028954 SQ_ALU_CONST_CACHE_PS_5
-0x00028958 SQ_ALU_CONST_CACHE_PS_6
-0x0002895C SQ_ALU_CONST_CACHE_PS_7
-0x00028960 SQ_ALU_CONST_CACHE_PS_8
-0x00028964 SQ_ALU_CONST_CACHE_PS_9
-0x00028968 SQ_ALU_CONST_CACHE_PS_10
-0x0002896C SQ_ALU_CONST_CACHE_PS_11
-0x00028970 SQ_ALU_CONST_CACHE_PS_12
-0x00028974 SQ_ALU_CONST_CACHE_PS_13
-0x00028978 SQ_ALU_CONST_CACHE_PS_14
-0x0002897C SQ_ALU_CONST_CACHE_PS_15
-0x00028980 SQ_ALU_CONST_CACHE_VS_0
-0x00028984 SQ_ALU_CONST_CACHE_VS_1
-0x00028988 SQ_ALU_CONST_CACHE_VS_2
-0x0002898C SQ_ALU_CONST_CACHE_VS_3
-0x00028990 SQ_ALU_CONST_CACHE_VS_4
-0x00028994 SQ_ALU_CONST_CACHE_VS_5
-0x00028998 SQ_ALU_CONST_CACHE_VS_6
-0x0002899C SQ_ALU_CONST_CACHE_VS_7
-0x000289A0 SQ_ALU_CONST_CACHE_VS_8
-0x000289A4 SQ_ALU_CONST_CACHE_VS_9
-0x000289A8 SQ_ALU_CONST_CACHE_VS_10
-0x000289AC SQ_ALU_CONST_CACHE_VS_11
-0x000289B0 SQ_ALU_CONST_CACHE_VS_12
-0x000289B4 SQ_ALU_CONST_CACHE_VS_13
-0x000289B8 SQ_ALU_CONST_CACHE_VS_14
-0x000289BC SQ_ALU_CONST_CACHE_VS_15
0x000288D8 SQ_PGM_CF_OFFSET_ES
0x000288DC SQ_PGM_CF_OFFSET_FS
0x000288D4 SQ_PGM_CF_OFFSET_GS
0x00028438 SX_ALPHA_REF
0x00028410 SX_ALPHA_TEST_CONTROL
0x00028350 SX_MISC
-0x0000A020 SMX_DC_CTL0
-0x0000A024 SMX_DC_CTL1
-0x0000A028 SMX_DC_CTL2
-0x00009608 TC_CNTL
0x00009604 TC_INVALIDATE
-0x00009490 TD_CNTL
0x00009400 TD_FILTER4
0x00009404 TD_FILTER4_1
0x00009408 TD_FILTER4_2
0x00028428 CB_FOG_GREEN
0x00028424 CB_FOG_RED
0x00008040 WAIT_UNTIL
-0x00008950 CC_GC_SHADER_PIPE_CONFIG
-0x00008954 GC_USER_SHADER_PIPE_CONFIG
0x00009714 VC_ENHANCE
0x00009830 DB_DEBUG
0x00009838 DB_WATERMARKS
0x00028D28 DB_SRESULTS_COMPARE_STATE0
0x00028D44 DB_ALPHA_TO_MASK
-0x00009504 TA_CNTL
0x00009700 VC_CNTL
-0x00009718 VC_CONFIG
-0x0000A02C SMX_DC_MC_INTF_CTL
* Jerome Glisse
*/
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include <drm/drmP.h>
#include "radeon.h"
+#include "radeon_asic.h"
#include "rs400d.h"
/* This files gather functions specifics to : rs400,rs480 */
void rs400_gart_fini(struct radeon_device *rdev)
{
+ radeon_gart_fini(rdev);
rs400_gart_disable(rdev);
radeon_gart_table_ram_free(rdev);
- radeon_gart_fini(rdev);
}
int rs400_gart_set_page(struct radeon_device *rdev, int i, uint64_t addr)
base = (RREG32(RADEON_NB_TOM) & 0xffff) << 16;
radeon_vram_location(rdev, &rdev->mc, base);
radeon_gtt_location(rdev, &rdev->mc);
+ radeon_update_bandwidth_info(rdev);
}
uint32_t rs400_mc_rreg(struct radeon_device *rdev, uint32_t reg)
{
int r;
+ r100_set_common_regs(rdev);
+
rs400_mc_program(rdev);
/* Resume clock */
r300_clock_startup(rdev);
void rs400_fini(struct radeon_device *rdev)
{
+ radeon_pm_fini(rdev);
r100_cp_fini(rdev);
r100_wb_fini(rdev);
r100_ib_fini(rdev);
*/
#include "drmP.h"
#include "radeon.h"
+#include "radeon_asic.h"
#include "atom.h"
#include "rs600d.h"
void rs600_gart_fini(struct radeon_device *rdev)
{
+ radeon_gart_fini(rdev);
rs600_gart_disable(rdev);
radeon_gart_table_vram_free(rdev);
- radeon_gart_fini(rdev);
}
#define R600_PTE_VALID (1 << 0)
/* Vertical blank interrupts */
if (G_007EDC_LB_D1_VBLANK_INTERRUPT(r500_disp_int)) {
drm_handle_vblank(rdev->ddev, 0);
+ rdev->pm.vblank_sync = true;
wake_up(&rdev->irq.vblank_queue);
}
if (G_007EDC_LB_D2_VBLANK_INTERRUPT(r500_disp_int)) {
drm_handle_vblank(rdev->ddev, 1);
+ rdev->pm.vblank_sync = true;
wake_up(&rdev->irq.vblank_queue);
}
if (G_007EDC_DC_HOT_PLUG_DETECT1_INTERRUPT(r500_disp_int)) {
rdev->mc.igp_sideport_enabled = radeon_atombios_sideport_present(rdev);
base = RREG32_MC(R_000004_MC_FB_LOCATION);
base = G_000004_MC_FB_START(base) << 16;
+ rdev->mc.igp_sideport_enabled = radeon_atombios_sideport_present(rdev);
radeon_vram_location(rdev, &rdev->mc, base);
radeon_gtt_location(rdev, &rdev->mc);
+ radeon_update_bandwidth_info(rdev);
}
void rs600_bandwidth_update(struct radeon_device *rdev)
{
- /* FIXME: implement, should this be like rs690 ? */
+ struct drm_display_mode *mode0 = NULL;
+ struct drm_display_mode *mode1 = NULL;
+ u32 d1mode_priority_a_cnt, d2mode_priority_a_cnt;
+ /* FIXME: implement full support */
+
+ radeon_update_display_priority(rdev);
+
+ if (rdev->mode_info.crtcs[0]->base.enabled)
+ mode0 = &rdev->mode_info.crtcs[0]->base.mode;
+ if (rdev->mode_info.crtcs[1]->base.enabled)
+ mode1 = &rdev->mode_info.crtcs[1]->base.mode;
+
+ rs690_line_buffer_adjust(rdev, mode0, mode1);
+
+ if (rdev->disp_priority == 2) {
+ d1mode_priority_a_cnt = RREG32(R_006548_D1MODE_PRIORITY_A_CNT);
+ d2mode_priority_a_cnt = RREG32(R_006D48_D2MODE_PRIORITY_A_CNT);
+ d1mode_priority_a_cnt |= S_006548_D1MODE_PRIORITY_A_ALWAYS_ON(1);
+ d2mode_priority_a_cnt |= S_006D48_D2MODE_PRIORITY_A_ALWAYS_ON(1);
+ WREG32(R_006548_D1MODE_PRIORITY_A_CNT, d1mode_priority_a_cnt);
+ WREG32(R_00654C_D1MODE_PRIORITY_B_CNT, d1mode_priority_a_cnt);
+ WREG32(R_006D48_D2MODE_PRIORITY_A_CNT, d2mode_priority_a_cnt);
+ WREG32(R_006D4C_D2MODE_PRIORITY_B_CNT, d2mode_priority_a_cnt);
+ }
}
uint32_t rs600_mc_rreg(struct radeon_device *rdev, uint32_t reg)
void rs600_fini(struct radeon_device *rdev)
{
+ radeon_pm_fini(rdev);
r100_cp_fini(rdev);
r100_wb_fini(rdev);
r100_ib_fini(rdev);
#define G_00016C_INVALIDATE_L1_TLB(x) (((x) >> 20) & 0x1)
#define C_00016C_INVALIDATE_L1_TLB 0xFFEFFFFF
+#define R_006548_D1MODE_PRIORITY_A_CNT 0x006548
+#define S_006548_D1MODE_PRIORITY_MARK_A(x) (((x) & 0x7FFF) << 0)
+#define G_006548_D1MODE_PRIORITY_MARK_A(x) (((x) >> 0) & 0x7FFF)
+#define C_006548_D1MODE_PRIORITY_MARK_A 0xFFFF8000
+#define S_006548_D1MODE_PRIORITY_A_OFF(x) (((x) & 0x1) << 16)
+#define G_006548_D1MODE_PRIORITY_A_OFF(x) (((x) >> 16) & 0x1)
+#define C_006548_D1MODE_PRIORITY_A_OFF 0xFFFEFFFF
+#define S_006548_D1MODE_PRIORITY_A_ALWAYS_ON(x) (((x) & 0x1) << 20)
+#define G_006548_D1MODE_PRIORITY_A_ALWAYS_ON(x) (((x) >> 20) & 0x1)
+#define C_006548_D1MODE_PRIORITY_A_ALWAYS_ON 0xFFEFFFFF
+#define S_006548_D1MODE_PRIORITY_A_FORCE_MASK(x) (((x) & 0x1) << 24)
+#define G_006548_D1MODE_PRIORITY_A_FORCE_MASK(x) (((x) >> 24) & 0x1)
+#define C_006548_D1MODE_PRIORITY_A_FORCE_MASK 0xFEFFFFFF
+#define R_00654C_D1MODE_PRIORITY_B_CNT 0x00654C
+#define S_00654C_D1MODE_PRIORITY_MARK_B(x) (((x) & 0x7FFF) << 0)
+#define G_00654C_D1MODE_PRIORITY_MARK_B(x) (((x) >> 0) & 0x7FFF)
+#define C_00654C_D1MODE_PRIORITY_MARK_B 0xFFFF8000
+#define S_00654C_D1MODE_PRIORITY_B_OFF(x) (((x) & 0x1) << 16)
+#define G_00654C_D1MODE_PRIORITY_B_OFF(x) (((x) >> 16) & 0x1)
+#define C_00654C_D1MODE_PRIORITY_B_OFF 0xFFFEFFFF
+#define S_00654C_D1MODE_PRIORITY_B_ALWAYS_ON(x) (((x) & 0x1) << 20)
+#define G_00654C_D1MODE_PRIORITY_B_ALWAYS_ON(x) (((x) >> 20) & 0x1)
+#define C_00654C_D1MODE_PRIORITY_B_ALWAYS_ON 0xFFEFFFFF
+#define S_00654C_D1MODE_PRIORITY_B_FORCE_MASK(x) (((x) & 0x1) << 24)
+#define G_00654C_D1MODE_PRIORITY_B_FORCE_MASK(x) (((x) >> 24) & 0x1)
+#define C_00654C_D1MODE_PRIORITY_B_FORCE_MASK 0xFEFFFFFF
+#define R_006D48_D2MODE_PRIORITY_A_CNT 0x006D48
+#define S_006D48_D2MODE_PRIORITY_MARK_A(x) (((x) & 0x7FFF) << 0)
+#define G_006D48_D2MODE_PRIORITY_MARK_A(x) (((x) >> 0) & 0x7FFF)
+#define C_006D48_D2MODE_PRIORITY_MARK_A 0xFFFF8000
+#define S_006D48_D2MODE_PRIORITY_A_OFF(x) (((x) & 0x1) << 16)
+#define G_006D48_D2MODE_PRIORITY_A_OFF(x) (((x) >> 16) & 0x1)
+#define C_006D48_D2MODE_PRIORITY_A_OFF 0xFFFEFFFF
+#define S_006D48_D2MODE_PRIORITY_A_ALWAYS_ON(x) (((x) & 0x1) << 20)
+#define G_006D48_D2MODE_PRIORITY_A_ALWAYS_ON(x) (((x) >> 20) & 0x1)
+#define C_006D48_D2MODE_PRIORITY_A_ALWAYS_ON 0xFFEFFFFF
+#define S_006D48_D2MODE_PRIORITY_A_FORCE_MASK(x) (((x) & 0x1) << 24)
+#define G_006D48_D2MODE_PRIORITY_A_FORCE_MASK(x) (((x) >> 24) & 0x1)
+#define C_006D48_D2MODE_PRIORITY_A_FORCE_MASK 0xFEFFFFFF
+#define R_006D4C_D2MODE_PRIORITY_B_CNT 0x006D4C
+#define S_006D4C_D2MODE_PRIORITY_MARK_B(x) (((x) & 0x7FFF) << 0)
+#define G_006D4C_D2MODE_PRIORITY_MARK_B(x) (((x) >> 0) & 0x7FFF)
+#define C_006D4C_D2MODE_PRIORITY_MARK_B 0xFFFF8000
+#define S_006D4C_D2MODE_PRIORITY_B_OFF(x) (((x) & 0x1) << 16)
+#define G_006D4C_D2MODE_PRIORITY_B_OFF(x) (((x) >> 16) & 0x1)
+#define C_006D4C_D2MODE_PRIORITY_B_OFF 0xFFFEFFFF
+#define S_006D4C_D2MODE_PRIORITY_B_ALWAYS_ON(x) (((x) & 0x1) << 20)
+#define G_006D4C_D2MODE_PRIORITY_B_ALWAYS_ON(x) (((x) >> 20) & 0x1)
+#define C_006D4C_D2MODE_PRIORITY_B_ALWAYS_ON 0xFFEFFFFF
+#define S_006D4C_D2MODE_PRIORITY_B_FORCE_MASK(x) (((x) & 0x1) << 24)
+#define G_006D4C_D2MODE_PRIORITY_B_FORCE_MASK(x) (((x) >> 24) & 0x1)
+#define C_006D4C_D2MODE_PRIORITY_B_FORCE_MASK 0xFEFFFFFF
+
#endif
*/
#include "drmP.h"
#include "radeon.h"
+#include "radeon_asic.h"
#include "atom.h"
#include "rs690d.h"
}
}
+union igp_info {
+ struct _ATOM_INTEGRATED_SYSTEM_INFO info;
+ struct _ATOM_INTEGRATED_SYSTEM_INFO_V2 info_v2;
+};
+
void rs690_pm_info(struct radeon_device *rdev)
{
int index = GetIndexIntoMasterTable(DATA, IntegratedSystemInfo);
- struct _ATOM_INTEGRATED_SYSTEM_INFO *info;
- struct _ATOM_INTEGRATED_SYSTEM_INFO_V2 *info_v2;
- void *ptr;
+ union igp_info *info;
uint16_t data_offset;
uint8_t frev, crev;
fixed20_12 tmp;
- atom_parse_data_header(rdev->mode_info.atom_context, index, NULL,
- &frev, &crev, &data_offset);
- ptr = rdev->mode_info.atom_context->bios + data_offset;
- info = (struct _ATOM_INTEGRATED_SYSTEM_INFO *)ptr;
- info_v2 = (struct _ATOM_INTEGRATED_SYSTEM_INFO_V2 *)ptr;
- /* Get various system informations from bios */
- switch (crev) {
- case 1:
- tmp.full = rfixed_const(100);
- rdev->pm.igp_sideport_mclk.full = rfixed_const(info->ulBootUpMemoryClock);
- rdev->pm.igp_sideport_mclk.full = rfixed_div(rdev->pm.igp_sideport_mclk, tmp);
- rdev->pm.igp_system_mclk.full = rfixed_const(le16_to_cpu(info->usK8MemoryClock));
- rdev->pm.igp_ht_link_clk.full = rfixed_const(le16_to_cpu(info->usFSBClock));
- rdev->pm.igp_ht_link_width.full = rfixed_const(info->ucHTLinkWidth);
- break;
- case 2:
- tmp.full = rfixed_const(100);
- rdev->pm.igp_sideport_mclk.full = rfixed_const(info_v2->ulBootUpSidePortClock);
- rdev->pm.igp_sideport_mclk.full = rfixed_div(rdev->pm.igp_sideport_mclk, tmp);
- rdev->pm.igp_system_mclk.full = rfixed_const(info_v2->ulBootUpUMAClock);
- rdev->pm.igp_system_mclk.full = rfixed_div(rdev->pm.igp_system_mclk, tmp);
- rdev->pm.igp_ht_link_clk.full = rfixed_const(info_v2->ulHTLinkFreq);
- rdev->pm.igp_ht_link_clk.full = rfixed_div(rdev->pm.igp_ht_link_clk, tmp);
- rdev->pm.igp_ht_link_width.full = rfixed_const(le16_to_cpu(info_v2->usMinHTLinkWidth));
- break;
- default:
+ if (atom_parse_data_header(rdev->mode_info.atom_context, index, NULL,
+ &frev, &crev, &data_offset)) {
+ info = (union igp_info *)(rdev->mode_info.atom_context->bios + data_offset);
+
+ /* Get various system informations from bios */
+ switch (crev) {
+ case 1:
+ tmp.full = rfixed_const(100);
+ rdev->pm.igp_sideport_mclk.full = rfixed_const(info->info.ulBootUpMemoryClock);
+ rdev->pm.igp_sideport_mclk.full = rfixed_div(rdev->pm.igp_sideport_mclk, tmp);
+ rdev->pm.igp_system_mclk.full = rfixed_const(le16_to_cpu(info->info.usK8MemoryClock));
+ rdev->pm.igp_ht_link_clk.full = rfixed_const(le16_to_cpu(info->info.usFSBClock));
+ rdev->pm.igp_ht_link_width.full = rfixed_const(info->info.ucHTLinkWidth);
+ break;
+ case 2:
+ tmp.full = rfixed_const(100);
+ rdev->pm.igp_sideport_mclk.full = rfixed_const(info->info_v2.ulBootUpSidePortClock);
+ rdev->pm.igp_sideport_mclk.full = rfixed_div(rdev->pm.igp_sideport_mclk, tmp);
+ rdev->pm.igp_system_mclk.full = rfixed_const(info->info_v2.ulBootUpUMAClock);
+ rdev->pm.igp_system_mclk.full = rfixed_div(rdev->pm.igp_system_mclk, tmp);
+ rdev->pm.igp_ht_link_clk.full = rfixed_const(info->info_v2.ulHTLinkFreq);
+ rdev->pm.igp_ht_link_clk.full = rfixed_div(rdev->pm.igp_ht_link_clk, tmp);
+ rdev->pm.igp_ht_link_width.full = rfixed_const(le16_to_cpu(info->info_v2.usMinHTLinkWidth));
+ break;
+ default:
+ tmp.full = rfixed_const(100);
+ /* We assume the slower possible clock ie worst case */
+ /* DDR 333Mhz */
+ rdev->pm.igp_sideport_mclk.full = rfixed_const(333);
+ /* FIXME: system clock ? */
+ rdev->pm.igp_system_mclk.full = rfixed_const(100);
+ rdev->pm.igp_system_mclk.full = rfixed_div(rdev->pm.igp_system_mclk, tmp);
+ rdev->pm.igp_ht_link_clk.full = rfixed_const(200);
+ rdev->pm.igp_ht_link_width.full = rfixed_const(8);
+ DRM_ERROR("No integrated system info for your GPU, using safe default\n");
+ break;
+ }
+ } else {
tmp.full = rfixed_const(100);
/* We assume the slower possible clock ie worst case */
/* DDR 333Mhz */
rdev->pm.igp_ht_link_clk.full = rfixed_const(200);
rdev->pm.igp_ht_link_width.full = rfixed_const(8);
DRM_ERROR("No integrated system info for your GPU, using safe default\n");
- break;
}
/* Compute various bandwidth */
/* k8_bandwidth = (memory_clk / 2) * 2 * 8 * 0.5 = memory_clk * 4 */
void rs690_mc_init(struct radeon_device *rdev)
{
- fixed20_12 a;
u64 base;
rs400_gart_adjust_size(rdev);
base = RREG32_MC(R_000100_MCCFG_FB_LOCATION);
base = G_000100_MC_FB_START(base) << 16;
rs690_pm_info(rdev);
- /* FIXME: we should enforce default clock in case GPU is not in
- * default setup
- */
- a.full = rfixed_const(100);
- rdev->pm.sclk.full = rfixed_const(rdev->clock.default_sclk);
- rdev->pm.sclk.full = rfixed_div(rdev->pm.sclk, a);
- a.full = rfixed_const(16);
- /* core_bandwidth = sclk(Mhz) * 16 */
- rdev->pm.core_bandwidth.full = rfixed_div(rdev->pm.sclk, a);
rdev->mc.igp_sideport_enabled = radeon_atombios_sideport_present(rdev);
radeon_vram_location(rdev, &rdev->mc, base);
radeon_gtt_location(rdev, &rdev->mc);
+ radeon_update_bandwidth_info(rdev);
}
void rs690_line_buffer_adjust(struct radeon_device *rdev,
struct drm_display_mode *mode1 = NULL;
struct rs690_watermark wm0;
struct rs690_watermark wm1;
- u32 tmp;
+ u32 tmp, d1mode_priority_a_cnt, d2mode_priority_a_cnt;
fixed20_12 priority_mark02, priority_mark12, fill_rate;
fixed20_12 a, b;
+ radeon_update_display_priority(rdev);
+
if (rdev->mode_info.crtcs[0]->base.enabled)
mode0 = &rdev->mode_info.crtcs[0]->base.mode;
if (rdev->mode_info.crtcs[1]->base.enabled)
* modes if the user specifies HIGH for displaypriority
* option.
*/
- if (rdev->disp_priority == 2) {
+ if ((rdev->disp_priority == 2) &&
+ ((rdev->family == CHIP_RS690) || (rdev->family == CHIP_RS740))) {
tmp = RREG32_MC(R_000104_MC_INIT_MISC_LAT_TIMER);
tmp &= C_000104_MC_DISP0R_INIT_LAT;
tmp &= C_000104_MC_DISP1R_INIT_LAT;
priority_mark12.full = 0;
if (wm1.priority_mark_max.full > priority_mark12.full)
priority_mark12.full = wm1.priority_mark_max.full;
- WREG32(R_006548_D1MODE_PRIORITY_A_CNT, rfixed_trunc(priority_mark02));
- WREG32(R_00654C_D1MODE_PRIORITY_B_CNT, rfixed_trunc(priority_mark02));
- WREG32(R_006D48_D2MODE_PRIORITY_A_CNT, rfixed_trunc(priority_mark12));
- WREG32(R_006D4C_D2MODE_PRIORITY_B_CNT, rfixed_trunc(priority_mark12));
+ d1mode_priority_a_cnt = rfixed_trunc(priority_mark02);
+ d2mode_priority_a_cnt = rfixed_trunc(priority_mark12);
+ if (rdev->disp_priority == 2) {
+ d1mode_priority_a_cnt |= S_006548_D1MODE_PRIORITY_A_ALWAYS_ON(1);
+ d2mode_priority_a_cnt |= S_006D48_D2MODE_PRIORITY_A_ALWAYS_ON(1);
+ }
+ WREG32(R_006548_D1MODE_PRIORITY_A_CNT, d1mode_priority_a_cnt);
+ WREG32(R_00654C_D1MODE_PRIORITY_B_CNT, d1mode_priority_a_cnt);
+ WREG32(R_006D48_D2MODE_PRIORITY_A_CNT, d2mode_priority_a_cnt);
+ WREG32(R_006D4C_D2MODE_PRIORITY_B_CNT, d2mode_priority_a_cnt);
} else if (mode0) {
if (rfixed_trunc(wm0.dbpp) > 64)
a.full = rfixed_mul(wm0.dbpp, wm0.num_line_pair);
priority_mark02.full = 0;
if (wm0.priority_mark_max.full > priority_mark02.full)
priority_mark02.full = wm0.priority_mark_max.full;
- WREG32(R_006548_D1MODE_PRIORITY_A_CNT, rfixed_trunc(priority_mark02));
- WREG32(R_00654C_D1MODE_PRIORITY_B_CNT, rfixed_trunc(priority_mark02));
+ d1mode_priority_a_cnt = rfixed_trunc(priority_mark02);
+ if (rdev->disp_priority == 2)
+ d1mode_priority_a_cnt |= S_006548_D1MODE_PRIORITY_A_ALWAYS_ON(1);
+ WREG32(R_006548_D1MODE_PRIORITY_A_CNT, d1mode_priority_a_cnt);
+ WREG32(R_00654C_D1MODE_PRIORITY_B_CNT, d1mode_priority_a_cnt);
WREG32(R_006D48_D2MODE_PRIORITY_A_CNT,
S_006D48_D2MODE_PRIORITY_A_OFF(1));
WREG32(R_006D4C_D2MODE_PRIORITY_B_CNT,
priority_mark12.full = 0;
if (wm1.priority_mark_max.full > priority_mark12.full)
priority_mark12.full = wm1.priority_mark_max.full;
+ d2mode_priority_a_cnt = rfixed_trunc(priority_mark12);
+ if (rdev->disp_priority == 2)
+ d2mode_priority_a_cnt |= S_006D48_D2MODE_PRIORITY_A_ALWAYS_ON(1);
WREG32(R_006548_D1MODE_PRIORITY_A_CNT,
S_006548_D1MODE_PRIORITY_A_OFF(1));
WREG32(R_00654C_D1MODE_PRIORITY_B_CNT,
S_00654C_D1MODE_PRIORITY_B_OFF(1));
- WREG32(R_006D48_D2MODE_PRIORITY_A_CNT, rfixed_trunc(priority_mark12));
- WREG32(R_006D4C_D2MODE_PRIORITY_B_CNT, rfixed_trunc(priority_mark12));
+ WREG32(R_006D48_D2MODE_PRIORITY_A_CNT, d2mode_priority_a_cnt);
+ WREG32(R_006D4C_D2MODE_PRIORITY_B_CNT, d2mode_priority_a_cnt);
}
}
void rs690_fini(struct radeon_device *rdev)
{
+ radeon_pm_fini(rdev);
r100_cp_fini(rdev);
r100_wb_fini(rdev);
r100_ib_fini(rdev);
#define S_006548_D1MODE_PRIORITY_A_OFF(x) (((x) & 0x1) << 16)
#define G_006548_D1MODE_PRIORITY_A_OFF(x) (((x) >> 16) & 0x1)
#define C_006548_D1MODE_PRIORITY_A_OFF 0xFFFEFFFF
+#define S_006548_D1MODE_PRIORITY_A_ALWAYS_ON(x) (((x) & 0x1) << 20)
+#define G_006548_D1MODE_PRIORITY_A_ALWAYS_ON(x) (((x) >> 20) & 0x1)
+#define C_006548_D1MODE_PRIORITY_A_ALWAYS_ON 0xFFEFFFFF
#define S_006548_D1MODE_PRIORITY_A_FORCE_MASK(x) (((x) & 0x1) << 24)
#define G_006548_D1MODE_PRIORITY_A_FORCE_MASK(x) (((x) >> 24) & 0x1)
#define C_006548_D1MODE_PRIORITY_A_FORCE_MASK 0xFEFFFFFF
* Jerome Glisse
*/
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include "drmP.h"
#include "rv515d.h"
#include "radeon.h"
+#include "radeon_asic.h"
#include "atom.h"
#include "rv515_reg_safe.h"
void rv515_mc_init(struct radeon_device *rdev)
{
- fixed20_12 a;
rv515_vram_get_type(rdev);
r100_vram_init_sizes(rdev);
radeon_vram_location(rdev, &rdev->mc, 0);
if (!(rdev->flags & RADEON_IS_AGP))
radeon_gtt_location(rdev, &rdev->mc);
- /* FIXME: we should enforce default clock in case GPU is not in
- * default setup
- */
- a.full = rfixed_const(100);
- rdev->pm.sclk.full = rfixed_const(rdev->clock.default_sclk);
- rdev->pm.sclk.full = rfixed_div(rdev->pm.sclk, a);
+ radeon_update_bandwidth_info(rdev);
}
uint32_t rv515_mc_rreg(struct radeon_device *rdev, uint32_t reg)
void rv515_fini(struct radeon_device *rdev)
{
+ radeon_pm_fini(rdev);
r100_cp_fini(rdev);
r100_wb_fini(rdev);
r100_ib_fini(rdev);
struct drm_display_mode *mode1 = NULL;
struct rv515_watermark wm0;
struct rv515_watermark wm1;
- u32 tmp;
+ u32 tmp, d1mode_priority_a_cnt, d2mode_priority_a_cnt;
fixed20_12 priority_mark02, priority_mark12, fill_rate;
fixed20_12 a, b;
priority_mark12.full = 0;
if (wm1.priority_mark_max.full > priority_mark12.full)
priority_mark12.full = wm1.priority_mark_max.full;
- WREG32(D1MODE_PRIORITY_A_CNT, rfixed_trunc(priority_mark02));
- WREG32(D1MODE_PRIORITY_B_CNT, rfixed_trunc(priority_mark02));
- WREG32(D2MODE_PRIORITY_A_CNT, rfixed_trunc(priority_mark12));
- WREG32(D2MODE_PRIORITY_B_CNT, rfixed_trunc(priority_mark12));
+ d1mode_priority_a_cnt = rfixed_trunc(priority_mark02);
+ d2mode_priority_a_cnt = rfixed_trunc(priority_mark12);
+ if (rdev->disp_priority == 2) {
+ d1mode_priority_a_cnt |= MODE_PRIORITY_ALWAYS_ON;
+ d2mode_priority_a_cnt |= MODE_PRIORITY_ALWAYS_ON;
+ }
+ WREG32(D1MODE_PRIORITY_A_CNT, d1mode_priority_a_cnt);
+ WREG32(D1MODE_PRIORITY_B_CNT, d1mode_priority_a_cnt);
+ WREG32(D2MODE_PRIORITY_A_CNT, d2mode_priority_a_cnt);
+ WREG32(D2MODE_PRIORITY_B_CNT, d2mode_priority_a_cnt);
} else if (mode0) {
if (rfixed_trunc(wm0.dbpp) > 64)
a.full = rfixed_div(wm0.dbpp, wm0.num_line_pair);
priority_mark02.full = 0;
if (wm0.priority_mark_max.full > priority_mark02.full)
priority_mark02.full = wm0.priority_mark_max.full;
- WREG32(D1MODE_PRIORITY_A_CNT, rfixed_trunc(priority_mark02));
- WREG32(D1MODE_PRIORITY_B_CNT, rfixed_trunc(priority_mark02));
+ d1mode_priority_a_cnt = rfixed_trunc(priority_mark02);
+ if (rdev->disp_priority == 2)
+ d1mode_priority_a_cnt |= MODE_PRIORITY_ALWAYS_ON;
+ WREG32(D1MODE_PRIORITY_A_CNT, d1mode_priority_a_cnt);
+ WREG32(D1MODE_PRIORITY_B_CNT, d1mode_priority_a_cnt);
WREG32(D2MODE_PRIORITY_A_CNT, MODE_PRIORITY_OFF);
WREG32(D2MODE_PRIORITY_B_CNT, MODE_PRIORITY_OFF);
} else {
priority_mark12.full = 0;
if (wm1.priority_mark_max.full > priority_mark12.full)
priority_mark12.full = wm1.priority_mark_max.full;
+ d2mode_priority_a_cnt = rfixed_trunc(priority_mark12);
+ if (rdev->disp_priority == 2)
+ d2mode_priority_a_cnt |= MODE_PRIORITY_ALWAYS_ON;
WREG32(D1MODE_PRIORITY_A_CNT, MODE_PRIORITY_OFF);
WREG32(D1MODE_PRIORITY_B_CNT, MODE_PRIORITY_OFF);
- WREG32(D2MODE_PRIORITY_A_CNT, rfixed_trunc(priority_mark12));
- WREG32(D2MODE_PRIORITY_B_CNT, rfixed_trunc(priority_mark12));
+ WREG32(D2MODE_PRIORITY_A_CNT, d2mode_priority_a_cnt);
+ WREG32(D2MODE_PRIORITY_B_CNT, d2mode_priority_a_cnt);
}
}
struct drm_display_mode *mode0 = NULL;
struct drm_display_mode *mode1 = NULL;
+ radeon_update_display_priority(rdev);
+
if (rdev->mode_info.crtcs[0]->base.enabled)
mode0 = &rdev->mode_info.crtcs[0]->base.mode;
if (rdev->mode_info.crtcs[1]->base.enabled)
* modes if the user specifies HIGH for displaypriority
* option.
*/
- if (rdev->disp_priority == 2) {
+ if ((rdev->disp_priority == 2) &&
+ (rdev->family == CHIP_RV515)) {
tmp = RREG32_MC(MC_MISC_LAT_TIMER);
tmp &= ~MC_DISP1R_INIT_LAT_MASK;
tmp &= ~MC_DISP0R_INIT_LAT_MASK;
*/
#include <linux/firmware.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include "drmP.h"
#include "radeon.h"
+#include "radeon_asic.h"
#include "radeon_drm.h"
#include "rv770d.h"
#include "atom.h"
void rv770_pcie_gart_fini(struct radeon_device *rdev)
{
+ radeon_gart_fini(rdev);
rv770_pcie_gart_disable(rdev);
radeon_gart_table_vram_free(rdev);
- radeon_gart_fini(rdev);
}
WREG32(CC_RB_BACKEND_DISABLE, cc_rb_backend_disable);
WREG32(CC_GC_SHADER_PIPE_CONFIG, cc_gc_shader_pipe_config);
+ WREG32(GC_USER_SHADER_PIPE_CONFIG, cc_gc_shader_pipe_config);
WREG32(CC_SYS_RB_BACKEND_DISABLE, cc_rb_backend_disable);
WREG32(CGTS_SYS_TCC_DISABLE, 0);
WREG32(CGTS_TCC_DISABLE, 0);
+ WREG32(CGTS_USER_SYS_TCC_DISABLE, 0);
+ WREG32(CGTS_USER_TCC_DISABLE, 0);
num_qd_pipes =
R7XX_MAX_PIPES - r600_count_pipe_bits((cc_gc_shader_pipe_config & INACTIVE_QD_PIPES_MASK) >> 8);
int rv770_mc_init(struct radeon_device *rdev)
{
- fixed20_12 a;
u32 tmp;
int chansize, numchan;
rdev->mc.real_vram_size = rdev->mc.aper_size;
}
r600_vram_gtt_location(rdev, &rdev->mc);
- /* FIXME: we should enforce default clock in case GPU is not in
- * default setup
- */
- a.full = rfixed_const(100);
- rdev->pm.sclk.full = rfixed_const(rdev->clock.default_sclk);
- rdev->pm.sclk.full = rfixed_div(rdev->pm.sclk, a);
+ radeon_update_bandwidth_info(rdev);
+
return 0;
}
DRM_ERROR("radeon: failled testing IB (%d).\n", r);
return r;
}
+
+ r = r600_audio_init(rdev);
+ if (r) {
+ dev_err(rdev->dev, "radeon: audio init failed\n");
+ return r;
+ }
+
return r;
}
{
int r;
+ r600_audio_fini(rdev);
/* FIXME: we should wait for ring to be empty */
r700_cp_stop(rdev);
rdev->cp.ready = false;
}
}
}
+
+ r = r600_audio_init(rdev);
+ if (r) {
+ dev_err(rdev->dev, "radeon: audio init failed\n");
+ return r;
+ }
+
return 0;
}
void rv770_fini(struct radeon_device *rdev)
{
+ radeon_pm_fini(rdev);
r600_blit_fini(rdev);
r600_cp_fini(rdev);
r600_wb_fini(rdev);
#include "ttm/ttm_placement.h"
#include <linux/agp_backend.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/io.h>
#include <asm/agp.h>
atomic_set(&glob->bo_count, 0);
- kobject_init(&glob->kobj, &ttm_bo_glob_kobj_type);
- ret = kobject_add(&glob->kobj, ttm_get_kobj(), "buffer_objects");
+ ret = kobject_init_and_add(
+ &glob->kobj, &ttm_bo_glob_kobj_type, ttm_get_kobj(), "buffer_objects");
if (unlikely(ret != 0))
kobject_put(&glob->kobj);
return ret;
#include <linux/io.h>
#include <linux/highmem.h>
#include <linux/wait.h>
+#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/module.h>
#include <linux/wait.h>
#include <linux/mm.h>
#include <linux/module.h>
+#include <linux/slab.h>
#define TTM_MEMORY_ALLOC_RETRIES 4
zone->used_mem = 0;
zone->glob = glob;
glob->zone_kernel = zone;
- kobject_init(&zone->kobj, &ttm_mem_zone_kobj_type);
- ret = kobject_add(&zone->kobj, &glob->kobj, zone->name);
+ ret = kobject_init_and_add(
+ &zone->kobj, &ttm_mem_zone_kobj_type, &glob->kobj, zone->name);
if (unlikely(ret != 0)) {
kobject_put(&zone->kobj);
return ret;
zone->used_mem = 0;
zone->glob = glob;
glob->zone_highmem = zone;
- kobject_init(&zone->kobj, &ttm_mem_zone_kobj_type);
- ret = kobject_add(&zone->kobj, &glob->kobj, zone->name);
+ ret = kobject_init_and_add(
+ &zone->kobj, &ttm_mem_zone_kobj_type, &glob->kobj, zone->name);
if (unlikely(ret != 0)) {
kobject_put(&zone->kobj);
return ret;
zone->used_mem = 0;
zone->glob = glob;
glob->zone_dma32 = zone;
- kobject_init(&zone->kobj, &ttm_mem_zone_kobj_type);
- ret = kobject_add(&zone->kobj, &glob->kobj, zone->name);
+ ret = kobject_init_and_add(
+ &zone->kobj, &ttm_mem_zone_kobj_type, &glob->kobj, zone->name);
if (unlikely(ret != 0)) {
kobject_put(&zone->kobj);
return ret;
glob->swap_queue = create_singlethread_workqueue("ttm_swap");
INIT_WORK(&glob->work, ttm_shrink_work);
init_waitqueue_head(&glob->queue);
- kobject_init(&glob->kobj, &ttm_mem_glob_kobj_type);
- ret = kobject_add(&glob->kobj,
- ttm_get_kobj(),
- "memory_accounting");
+ ret = kobject_init_and_add(
+ &glob->kobj, &ttm_mem_glob_kobj_type, ttm_get_kobj(), "memory_accounting");
if (unlikely(ret != 0)) {
kobject_put(&glob->kobj);
return ret;
* Authors: Thomas Hellstrom <thellstrom-at-vmware-dot-com>
*/
-#include <linux/vmalloc.h>
#include <linux/sched.h>
#include <linux/highmem.h>
#include <linux/pagemap.h>
#include <linux/file.h>
#include <linux/swap.h>
+#include <linux/slab.h>
#include "drm_cache.h"
+#include "drm_mem_util.h"
#include "ttm/ttm_module.h"
#include "ttm/ttm_bo_driver.h"
#include "ttm/ttm_placement.h"
/**
* Allocates storage for pointers to the pages that back the ttm.
- *
- * Uses kmalloc if possible. Otherwise falls back to vmalloc.
*/
static void ttm_tt_alloc_page_directory(struct ttm_tt *ttm)
{
- unsigned long size = ttm->num_pages * sizeof(*ttm->pages);
- ttm->pages = NULL;
-
- if (size <= PAGE_SIZE)
- ttm->pages = kzalloc(size, GFP_KERNEL);
-
- if (!ttm->pages) {
- ttm->pages = vmalloc_user(size);
- if (ttm->pages)
- ttm->page_flags |= TTM_PAGE_FLAG_VMALLOC;
- }
+ ttm->pages = drm_calloc_large(ttm->num_pages, sizeof(*ttm->pages));
}
static void ttm_tt_free_page_directory(struct ttm_tt *ttm)
{
- if (ttm->page_flags & TTM_PAGE_FLAG_VMALLOC) {
- vfree(ttm->pages);
- ttm->page_flags &= ~TTM_PAGE_FLAG_VMALLOC;
- } else {
- kfree(ttm->pages);
- }
+ drm_free_large(ttm->pages);
ttm->pages = NULL;
}
#include "via_dmablit.h"
#include <linux/pagemap.h>
+#include <linux/slab.h>
#define VIA_PGDN(x) (((unsigned long)(x)) & PAGE_MASK)
#define VIA_PGOFF(x) (((unsigned long)(x)) & ~PAGE_MASK)
config DRM_VMWGFX
tristate "DRM driver for VMware Virtual GPU"
- depends on DRM && PCI
+ depends on DRM && PCI && FB
select FB_DEFERRED_IO
select FB_CFB_FILLRECT
select FB_CFB_COPYAREA
#include <linux/spinlock.h>
#include <linux/poll.h>
#include <linux/miscdevice.h>
+#include <linux/slab.h>
#include <linux/uaccess.h>
#include <linux/device.h>
#include <linux/hid.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/usb.h>
MODULE_AUTHOR("Stephane Chatty <chatty@enac.fr>");
#include <linux/input.h>
#include <linux/hid.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include "hid-ids.h"
#include <linux/device.h>
#include <linux/hid.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include "hid-ids.h"
#include <linux/debugfs.h>
#include <linux/seq_file.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/uaccess.h>
#include <linux/poll.h>
*/
#include <linux/input.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include <linux/hid.h>
*/
#include <linux/input.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include <linux/hid.h>
#include "hid-ids.h"
static int gyration_event(struct hid_device *hdev, struct hid_field *field,
struct hid_usage *usage, __s32 value)
{
- struct input_dev *input = field->hidinput->input;
+
+ if (!(hdev->claimed & HID_CLAIMED_INPUT) || !field->hidinput)
+ return 0;
if ((usage->hid & HID_USAGE_PAGE) == HID_UP_GENDESK &&
(usage->hid & 0xff) == 0x82) {
+ struct input_dev *input = field->hidinput->input;
input_event(input, usage->type, usage->code, 1);
input_sync(input);
input_event(input, usage->type, usage->code, 0);
#include <linux/input.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include <linux/hid.h>
#include <linux/device.h>
#include <linux/hid.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include "hid-ids.h"
#include <linux/device.h>
#include <linux/hid.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include "usbhid/usbhid.h"
#include <linux/device.h>
#include <linux/hid.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include "hid-ids.h"
#define debug(format, arg...) pr_debug("hid-plff: " format "\n" , ## arg)
#include <linux/input.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include <linux/hid.h>
#include <linux/device.h>
#include <linux/hid.h>
#include <linux/module.h>
+#include <linux/slab.h>
MODULE_AUTHOR("Stephane Chatty <chatty@enac.fr>");
MODULE_DESCRIPTION("Quanta dual-touch panel");
/* #define DEBUG */
#include <linux/input.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include <linux/hid.h>
#include "hid-ids.h"
#include <linux/device.h>
#include <linux/hid.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include "hid-ids.h"
#include <linux/device.h>
#include <linux/hid.h>
#include <linux/module.h>
+#include <linux/slab.h>
MODULE_AUTHOR("Stephane Chatty <chatty@enac.fr>");
MODULE_DESCRIPTION("Stantum HID multitouch panels");
#include <linux/hid.h>
#include <linux/input.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include "hid-ids.h"
#include <linux/device.h>
#include <linux/hid.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include "hid-ids.h"
#include <linux/hid.h>
#include <linux/input.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include "hid-ids.h"
#include <linux/poll.h>
#include <linux/device.h>
#include <linux/major.h>
+#include <linux/slab.h>
#include <linux/hid.h>
#include <linux/mutex.h>
#include <linux/sched.h>
{
set_bit(HID_REPORTED_IDLE, &usbhid->iofl);
spin_unlock_irq(&usbhid->lock);
+ if (hid->driver && hid->driver->suspend) {
+ status = hid->driver->suspend(hid, message);
+ if (status < 0)
+ return status;
+ }
} else {
usbhid_mark_busy(usbhid);
spin_unlock_irq(&usbhid->lock);
}
} else {
+ if (hid->driver && hid->driver->suspend) {
+ status = hid->driver->suspend(hid, message);
+ if (status < 0)
+ return status;
+ }
spin_lock_irq(&usbhid->lock);
set_bit(HID_REPORTED_IDLE, &usbhid->iofl);
spin_unlock_irq(&usbhid->lock);
hid_io_error(hid);
usbhid_restart_queues(usbhid);
+ if (status >= 0 && hid->driver && hid->driver->resume) {
+ int ret = hid->driver->resume(hid);
+ if (ret < 0)
+ status = ret;
+ }
dev_dbg(&intf->dev, "resume status %d\n", status);
return 0;
}
{
struct hid_device *hid = usb_get_intfdata(intf);
struct usbhid_device *usbhid = hid->driver_data;
+ int status;
clear_bit(HID_REPORTED_IDLE, &usbhid->iofl);
- return hid_post_reset(intf);
+ status = hid_post_reset(intf);
+ if (status >= 0 && hid->driver && hid->driver->reset_resume) {
+ int ret = hid->driver->reset_resume(hid);
+ if (ret < 0)
+ status = ret;
+ }
+ return status;
}
#endif /* CONFIG_PM */
#define debug(format, arg...) pr_debug("hid-pidff: " format "\n" , ## arg)
#include <linux/input.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include <linux/hid.h>
*/
#include <linux/hid.h>
+#include <linux/slab.h>
#include "../hid-ids.h"
{ USB_VENDOR_ID_DMI, USB_DEVICE_ID_DMI_ENC, HID_QUIRK_NOGET },
{ USB_VENDOR_ID_ELO, USB_DEVICE_ID_ELO_TS2700, HID_QUIRK_NOGET },
{ USB_VENDOR_ID_PRODIGE, USB_DEVICE_ID_PRODIGE_CORDLESS, HID_QUIRK_NOGET },
+ { USB_VENDOR_ID_QUANTA, USB_DEVICE_ID_PIXART_IMAGING_INC_OPTICAL_TOUCH_SCREEN, HID_QUIRK_NOGET },
{ USB_VENDOR_ID_SUN, USB_DEVICE_ID_RARITAN_KVM_DONGLE, HID_QUIRK_NOGET },
{ USB_VENDOR_ID_TURBOX, USB_DEVICE_ID_TURBOX_KEYBOARD, HID_QUIRK_NOGET },
{ USB_VENDOR_ID_UCLOGIC, USB_DEVICE_ID_UCLOGIC_TABLET_PF1209, HID_QUIRK_MULTI_INPUT },
depends on HWMON && I2C
help
If you say yes here you get support for the aSC7621
- family of SMBus sensors chip found on most Intel X48, X38, 975,
- 965 and 945 desktop boards. Currently supported chips:
+ family of SMBus sensors chip found on most Intel X38, X48, X58,
+ 945, 965 and 975 desktop boards. Currently supported chips:
aSC7621
aSC7621a
#include <linux/err.h>
#include <linux/mutex.h>
#include <linux/sysfs.h>
+#include <linux/slab.h>
/* AD7414 registers */
#include <linux/err.h>
#include <linux/mutex.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include "lm75.h"
#include <linux/init.h>
#include <linux/module.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/device.h>
#include <linux/err.h>
#include <linux/sysfs.h>
#include <linux/i2c.h>
#include <linux/hwmon.h>
#include <linux/hwmon-sysfs.h>
+#include <linux/slab.h>
#define ADT7411_REG_INT_TEMP_VDD_LSB 0x03
#define ADT7411_REG_EXT_TEMP_AIN14_LSB 0x04
#include <linux/mutex.h>
#include <linux/delay.h>
#include <linux/log2.h>
+#include <linux/slab.h>
/* Addresses to scan */
static const unsigned short normal_i2c[] = { 0x58, 0x5C, I2C_CLIENT_END };
#include <linux/delay.h>
#include <linux/log2.h>
#include <linux/kthread.h>
+#include <linux/slab.h>
/* Addresses to scan */
static const unsigned short normal_i2c[] = { 0x2C, 0x2E, 0x2F, I2C_CLIENT_END };
#include <linux/hwmon.h>
#include <linux/list.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <acpi/acpi.h>
#include <acpi/acpixf.h>
#include <linux/err.h>
#include <linux/mutex.h>
#include <linux/sysfs.h>
+#include <linux/slab.h>
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("System voltages control via Attansic ATXP1");
if (err) {
dev_warn(dev,
"Unable to access MSR 0xEE, for Tjmax, left"
- " at default");
+ " at default\n");
} else if (eax & 0x40000000) {
tjmax = tjmax_ee;
}
family 6 CPU */
if ((c->x86 == 0x6) && (c->x86_model > 0xf))
printk(KERN_WARNING DRVNAME ": Unknown CPU "
- "model %x\n", c->x86_model);
+ "model 0x%x\n", c->x86_model);
continue;
}
#include <linux/err.h>
#include <linux/mutex.h>
#include <linux/f75375s.h>
+#include <linux/slab.h>
/* Addresses to scan */
static const unsigned short normal_i2c[] = { 0x2d, 0x2e, I2C_CLIENT_END };
#include <linux/log2.h>
#include <linux/pci.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#define DRVNAME "i5k_amb"
#include <linux/kdev_t.h>
#include <linux/spinlock.h>
#include <linux/idr.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#include <linux/platform_device.h>
#include <linux/math64.h>
#include <linux/hwmon-sysfs.h>
#include <linux/jiffies.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#define REFRESH_INTERVAL (2 * HZ)
#define DRVNAME "ibmpex"
#include <linux/mutex.h>
#include <linux/mod_devicetable.h>
#include <linux/spi/spi.h>
+#include <linux/slab.h>
#define DRVNAME "lm70"
#include <linux/module.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/i2c.h>
#include <linux/hwmon.h>
#include <linux/hwmon-sysfs.h>
#include <linux/hwmon.h>
#include <linux/hwmon-sysfs.h>
#include <linux/spi/spi.h>
+#include <linux/slab.h>
#define MAX1111_TX_BUF_SIZE 1
#define MAX1111_RX_BUF_SIZE 2
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/hwmon.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/err.h>
#include <linux/err.h>
#include <linux/sht15.h>
#include <linux/regulator/consumer.h>
+#include <linux/slab.h>
#include <asm/atomic.h>
#define SHT15_MEASURE_TEMP 3
static ssize_t watchdog_write(struct file *filp, const char __user *buf,
size_t count, loff_t *offset)
{
- size_t ret;
+ ssize_t ret;
struct w83793_data *data = filp->private_data;
if (count) {
#include <linux/err.h>
#include <linux/hwmon.h>
#include <linux/hwmon-sysfs.h>
+#include <linux/slab.h>
#include <linux/mfd/wm831x/core.h>
#include <linux/mfd/wm831x/auxadc.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/delay.h>
-#include <linux/slab.h>
#include <linux/init.h>
#include <linux/errno.h>
#include <linux/sched.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/delay.h>
-#include <linux/slab.h>
#include <linux/init.h>
#include <linux/errno.h>
#include <linux/i2c.h>
#include <linux/i2c.h>
#include <linux/delay.h>
#include <linux/acpi.h>
+#include <linux/slab.h>
#include <asm/io.h>
MODULE_LICENSE("GPL");
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/i2c.h>
+#include <linux/slab.h>
#include <linux/io.h>
#include <linux/mm.h>
#include <linux/timer.h>
#include <linux/interrupt.h>
#include <linux/platform_device.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <mach/hardware.h>
#include <linux/interrupt.h>
#include <linux/platform_device.h>
#include <linux/io.h>
+#include <linux/slab.h>
/*
* Registers offset
#include <linux/ioport.h>
#include <linux/module.h>
#include <linux/delay.h>
-#include <linux/slab.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
#include <linux/i2c-gpio.h>
#include <linux/init.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/platform_device.h>
#include <asm/gpio.h>
#include <linux/completion.h>
#include <linux/io.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#define SMCR 0x00
#define SMCR_START (1 << 0)
#include <linux/sched.h>
#include <linux/platform_device.h>
#include <linux/clk.h>
+#include <linux/slab.h>
#include <mach/irqs.h>
#include <mach/hardware.h>
#include <linux/module.h>
#include <linux/i2c.h>
#include <linux/i2c-algo-bit.h>
+#include <linux/slab.h>
#include <mach/hardware.h> /* Pick up IXP2000-specific bits */
#include <mach/gpio.h>
#include <linux/init.h>
#include <linux/of_platform.h>
#include <linux/of_i2c.h>
+#include <linux/slab.h>
#include <linux/io.h>
#include <linux/fsl_devices.h>
* or implied.
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/spinlock.h>
#include <linux/i2c.h>
#include <linux/delay.h>
#include <linux/dmi.h>
#include <linux/acpi.h>
+#include <linux/slab.h>
#include <asm/io.h>
MODULE_LICENSE("GPL");
#include <linux/module.h>
#include <linux/platform_device.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/i2c.h>
#include <linux/err.h>
#include <linux/interrupt.h>
#include <linux/wait.h>
#include <linux/i2c-ocores.h>
+#include <linux/slab.h>
#include <asm/io.h>
struct ocores_i2c {
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/io.h>
#include <linux/platform_device.h>
#include <linux/clk.h>
#include <linux/io.h>
+#include <linux/slab.h>
/* I2C controller revisions */
#define OMAP_I2C_REV_2 0x20
#include <linux/i2c.h>
#include <linux/i2c-algo-bit.h>
#include <linux/i2c-smbus.h>
+#include <linux/slab.h>
#include "i2c-parport.h"
/* ----- Device list ------------------------------------------------------ */
#include <linux/sched.h>
#include <linux/i2c.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <asm/io.h>
static struct pci_driver pasemi_smb_driver;
#include <linux/io.h>
#include <linux/err.h>
#include <linux/clk.h>
+#include <linux/slab.h>
#include <mach/hardware.h>
#include <mach/i2c.h>
#include <linux/platform_device.h>
#include <linux/err.h>
#include <linux/clk.h>
+#include <linux/slab.h>
#include <asm/irq.h>
#include <asm/io.h>
#include <linux/platform_device.h>
#include <linux/clk.h>
#include <linux/cpufreq.h>
+#include <linux/slab.h>
#include <asm/irq.h>
#include <asm/io.h>
u8 cap_info:1;
u8 cap_read:1;
u8 cap_write:1;
+ struct smbus_methods_t *methods;
};
static const struct smbus_methods_t smbus_methods = {
.mt_sbw = "_SBW",
};
+/* Some IBM BIOSes omit the leading underscore */
+static const struct smbus_methods_t ibm_smbus_methods = {
+ .mt_info = "SBI_",
+ .mt_sbr = "SBR_",
+ .mt_sbw = "SBW_",
+};
+
static const struct acpi_device_id acpi_smbus_cmi_ids[] = {
- {"SMBUS01", 0},
+ {"SMBUS01", (kernel_ulong_t)&smbus_methods},
+ {ACPI_SMBUS_IBM_HID, (kernel_ulong_t)&ibm_smbus_methods},
{"", 0}
};
+MODULE_DEVICE_TABLE(acpi, acpi_smbus_cmi_ids);
#define ACPI_SMBUS_STATUS_OK 0x00
#define ACPI_SMBUS_STATUS_FAIL 0x07
if (read_write == I2C_SMBUS_READ) {
protocol |= ACPI_SMBUS_PRTCL_READ;
- method = smbus_methods.mt_sbr;
+ method = smbus_cmi->methods->mt_sbr;
input.count = 3;
} else {
protocol |= ACPI_SMBUS_PRTCL_WRITE;
- method = smbus_methods.mt_sbw;
+ method = smbus_cmi->methods->mt_sbw;
input.count = 5;
}
union acpi_object *obj;
acpi_status status;
- if (!strcmp(name, smbus_methods.mt_info)) {
+ if (!strcmp(name, smbus_cmi->methods->mt_info)) {
status = acpi_evaluate_object(smbus_cmi->handle,
- smbus_methods.mt_info,
+ smbus_cmi->methods->mt_info,
NULL, &buffer);
if (ACPI_FAILURE(status)) {
ACPI_ERROR((AE_INFO, "Evaluating %s: %i",
- smbus_methods.mt_info, status));
+ smbus_cmi->methods->mt_info, status));
return -EIO;
}
kfree(buffer.pointer);
smbus_cmi->cap_info = 1;
- } else if (!strcmp(name, smbus_methods.mt_sbr))
+ } else if (!strcmp(name, smbus_cmi->methods->mt_sbr))
smbus_cmi->cap_read = 1;
- else if (!strcmp(name, smbus_methods.mt_sbw))
+ else if (!strcmp(name, smbus_cmi->methods->mt_sbw))
smbus_cmi->cap_write = 1;
else
ACPI_DEBUG_PRINT((ACPI_DB_INFO, "Unsupported CMI method: %s\n",
static int acpi_smbus_cmi_add(struct acpi_device *device)
{
struct acpi_smbus_cmi *smbus_cmi;
+ const struct acpi_device_id *id;
smbus_cmi = kzalloc(sizeof(struct acpi_smbus_cmi), GFP_KERNEL);
if (!smbus_cmi)
smbus_cmi->cap_read = 0;
smbus_cmi->cap_write = 0;
+ for (id = acpi_smbus_cmi_ids; id->id[0]; id++)
+ if (!strcmp(id->id, acpi_device_hid(device)))
+ smbus_cmi->methods =
+ (struct smbus_methods_t *) id->driver_data;
+
acpi_walk_namespace(ACPI_TYPE_METHOD, smbus_cmi->handle, 1,
acpi_smbus_cmi_query_methods, NULL, smbus_cmi, NULL);
#include <linux/pm_runtime.h>
#include <linux/clk.h>
#include <linux/io.h>
+#include <linux/slab.h>
/* Transmit operation: */
/* */
#include <linux/init.h>
#include <linux/delay.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/i2c.h>
#include <linux/i2c-algo-bit.h>
#include <linux/interrupt.h>
#include <linux/clk.h>
#include <linux/io.h>
+#include <linux/slab.h>
/* the name of this kernel module */
#define NAME "stu300"
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/types.h>
/* include interfaces to usb layer */
#include <linux/i2c-algo-bit.h>
#include <linux/init.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <linux/wait.h>
#include <linux/i2c-xiic.h>
#include <linux/io.h>
+#include <linux/slab.h>
#define DRIVER_NAME "xiic-i2c"
#include <linux/pci.h>
#include <linux/delay.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <linux/scx200.h>
#include <linux/kernel.h>
#include <linux/i2c.h>
+#include <linux/slab.h>
#include <linux/rwsem.h>
#include "i2c-core.h"
#include <linux/workqueue.h>
#include <linux/i2c.h>
#include <linux/i2c-smbus.h>
+#include <linux/slab.h>
struct i2c_smbus_alert {
unsigned int alert_edge_triggered:1;
#include <linux/pci.h>
#include <linux/init.h>
#include <linux/ide.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/io.h>
#include <linux/device.h>
#include <linux/errno.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <acpi/acpi.h>
#include <linux/ide.h>
#include <linux/pci.h>
#include <linux/delay.h>
#include <linux/ide.h>
#include <linux/scatterlist.h>
+#include <linux/gfp.h>
#include <scsi/scsi.h>
#include <linux/kernel.h>
#include <linux/cdrom.h>
+#include <linux/gfp.h>
#include <linux/ide.h>
#include <scsi/scsi.h>
#include <linux/kernel.h>
+#include <linux/gfp.h>
#include <linux/ide.h>
DEFINE_MUTEX(ide_setting_mtx);
#include <linux/kernel.h>
#include <linux/ide.h>
+#include <linux/slab.h>
#include <linux/seq_file.h>
#include "ide-disk.h"
*/
#include <linux/types.h>
+#include <linux/gfp.h>
#include <linux/kernel.h>
#include <linux/ide.h>
#include <linux/scatterlist.h>
#include <linux/major.h>
#include <linux/errno.h>
#include <linux/genhd.h>
-#include <linux/slab.h>
#include <linux/cdrom.h>
#include <linux/ide.h>
#include <linux/hdreg.h>
#include <linux/ide.h>
#include <linux/hdreg.h>
#include <linux/dmi.h>
+#include <linux/slab.h>
#if !defined(CONFIG_DEBUG_BLOCK_EXT_DEVT)
#define IDE_DISK_MINORS (1 << PARTN_BITS)
#include <linux/hdreg.h>
#include <linux/ide.h>
+#include <linux/slab.h>
static const struct ide_ioctl_devset ide_ioctl_settings[] = {
{ HDIO_GET_32BIT, HDIO_SET_32BIT, &ide_devset_io_32bit },
#include <linux/kernel.h>
+#include <linux/gfp.h>
#include <linux/ide.h>
#include <linux/jiffies.h>
#include <linux/blkdev.h>
#include <linux/kernel.h>
+#include <linux/gfp.h>
#include <linux/ide.h>
int generic_ide_suspend(struct device *dev, pm_message_t mesg)
if (irqd)
disable_irq(hwif->irq);
- rc = ide_port_wait_ready(hwif);
- if (rc == -ENODEV) {
- printk(KERN_INFO "%s: no devices on the port\n", hwif->name);
- goto out;
- } else if (rc == -EBUSY)
- printk(KERN_ERR "%s: not ready before the probe\n", hwif->name);
- else
- rc = -ENODEV;
+ if (ide_port_wait_ready(hwif) == -EBUSY)
+ printk(KERN_DEBUG "%s: Wait for ready failed before probe !\n", hwif->name);
/*
* Second drive should only exist if first drive was found,
if (drive->dev_flags & IDE_DFLAG_PRESENT)
rc = 0;
}
-out:
+
/*
* Use cached IRQ number. It might be (and is...) changed by probe
* code above
#include <linux/ctype.h>
#include <linux/ide.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <linux/major.h>
#include <linux/errno.h>
#include <linux/genhd.h>
-#include <linux/slab.h>
#include <linux/init.h>
#include <linux/pci.h>
#include <linux/ide.h>
#include <linux/types.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/ide.h>
#include <linux/init.h>
#include <linux/adb.h>
#include <linux/pmu.h>
#include <linux/scatterlist.h>
+#include <linux/slab.h>
#include <asm/prom.h>
#include <asm/io.h>
*/
#include <linux/module.h>
-#include <linux/slab.h>
#include <linux/blkdev.h>
#include <linux/errno.h>
#include <linux/ide.h>
#include <linux/module.h>
#include <linux/types.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/init.h>
#include <linux/ide.h>
#include <linux/module.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/init.h>
#include <linux/ide.h>
{
struct via_isa_bridge *via_config;
unsigned int via_80w;
- u8 cached_device[2];
};
/**
.cable_detect = via82cxxx_cable_detect,
};
-static void via_write_devctl(ide_hwif_t *hwif, u8 ctl)
-{
- struct via82cxxx_dev *vdev = hwif->host->host_priv;
-
- outb(ctl, hwif->io_ports.ctl_addr);
- outb(vdev->cached_device[hwif->channel], hwif->io_ports.device_addr);
-}
-
-static void __via_dev_select(ide_drive_t *drive, u8 select)
-{
- ide_hwif_t *hwif = drive->hwif;
- struct via82cxxx_dev *vdev = hwif->host->host_priv;
-
- outb(select, hwif->io_ports.device_addr);
- vdev->cached_device[hwif->channel] = select;
-}
-
-static void via_dev_select(ide_drive_t *drive)
-{
- __via_dev_select(drive, drive->select | ATA_DEVICE_OBS);
-}
-
-static void via_tf_load(ide_drive_t *drive, struct ide_taskfile *tf, u8 valid)
-{
- ide_hwif_t *hwif = drive->hwif;
- struct ide_io_ports *io_ports = &hwif->io_ports;
-
- if (valid & IDE_VALID_FEATURE)
- outb(tf->feature, io_ports->feature_addr);
- if (valid & IDE_VALID_NSECT)
- outb(tf->nsect, io_ports->nsect_addr);
- if (valid & IDE_VALID_LBAL)
- outb(tf->lbal, io_ports->lbal_addr);
- if (valid & IDE_VALID_LBAM)
- outb(tf->lbam, io_ports->lbam_addr);
- if (valid & IDE_VALID_LBAH)
- outb(tf->lbah, io_ports->lbah_addr);
- if (valid & IDE_VALID_DEVICE)
- __via_dev_select(drive, tf->device);
-}
-
-const struct ide_tp_ops via_tp_ops = {
- .exec_command = ide_exec_command,
- .read_status = ide_read_status,
- .read_altstatus = ide_read_altstatus,
- .write_devctl = via_write_devctl,
-
- .dev_select = via_dev_select,
- .tf_load = via_tf_load,
- .tf_read = ide_tf_read,
-
- .input_data = ide_input_data,
- .output_data = ide_output_data,
-};
-
static const struct ide_port_info via82cxxx_chipset __devinitdata = {
.name = DRV_NAME,
.init_chipset = init_chipset_via82cxxx,
.enablebits = { { 0x40, 0x02, 0x02 }, { 0x40, 0x01, 0x01 } },
- .tp_ops = &via_tp_ops,
.port_ops = &via_port_ops,
.host_flags = IDE_HFLAG_PIO_NO_BLACKLIST |
IDE_HFLAG_POST_SET_MODE |
#include <linux/module.h>
#include <linux/pci.h>
+#include <linux/gfp.h>
#include <linux/sched.h>
#include <linux/notifier.h>
#include <linux/cpumask.h>
#include <linux/mm.h>
#include <linux/module.h>
#include <linux/pci.h>
-#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/scatterlist.h>
#include <linux/delay.h>
#include <linux/device.h>
#include <linux/dma-mapping.h>
-#include <linux/gfp.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/list.h>
#include <linux/mutex.h>
#include <linux/inetdevice.h>
+#include <linux/slab.h>
#include <linux/workqueue.h>
#include <net/arp.h>
#include <net/neighbour.h>
#include <linux/random.h>
#include <linux/rbtree.h>
#include <linux/spinlock.h>
+#include <linux/slab.h>
#include <linux/sysfs.h>
#include <linux/workqueue.h>
#include <linux/kdev_t.h>
#include <linux/random.h>
#include <linux/idr.h>
#include <linux/inetdevice.h>
+#include <linux/slab.h>
#include <net/tcp.h>
#include <net/ipv6.h>
#include <linux/spinlock.h>
#include <linux/workqueue.h>
#include <linux/completion.h>
+#include <linux/slab.h>
#include <rdma/iw_cm.h>
#include <rdma/ib_addr.h>
*
*/
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <rdma/ib_cache.h>
#include "mad_priv.h"
* SOFTWARE.
*/
+#include <linux/slab.h>
+
#include "mad_priv.h"
#include "mad_rmpp.h"
#include <linux/dma-mapping.h>
#include <linux/err.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <linux/bitops.h>
#include <linux/random.h>
element->attr.attr.mode = S_IRUGO;
element->attr.show = show;
element->index = i;
+ sysfs_attr_init(&element->attr.attr);
tab_attr[i] = &element->attr.attr;
}
#include <linux/cdev.h>
#include <linux/idr.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <linux/in.h>
#include <linux/in6.h>
#include <linux/miscdevice.h>
+#include <linux/slab.h>
#include <rdma/rdma_user_cm.h>
#include <rdma/ib_marshall.h>
#include <linux/sched.h>
#include <linux/hugetlb.h>
#include <linux/dma-attrs.h>
+#include <linux/slab.h>
#include "uverbs.h"
#include <linux/compat.h>
#include <linux/sched.h>
#include <linux/semaphore.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <linux/file.h>
#include <linux/fs.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <linux/file.h>
#include <linux/cdev.h>
#include <linux/anon_inodes.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <linux/tcp.h>
#include <linux/init.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/irq.h>
*/
#include <linux/errno.h>
-#include <linux/slab.h>
#include <linux/bitmap.h>
#include "c2.h"
* SOFTWARE.
*
*/
+#include <linux/slab.h>
+
#include "c2.h"
#include "c2_wr.h"
#include "c2_vq.h"
* SOFTWARE.
*
*/
+#include <linux/gfp.h>
+
#include "c2.h"
#include "c2_vq.h"
#include "c2_status.h"
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
+#include <linux/slab.h>
+
#include "c2.h"
#include "c2_vq.h"
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/errno.h>
#include "c2.h"
#include <linux/dma-mapping.h>
#include <linux/if_arp.h>
#include <linux/vmalloc.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/irq.h>
*/
#include <linux/delay.h>
+#include <linux/gfp.h>
#include "c2.h"
#include "c2_vq.h"
#include <linux/mm.h>
#include <linux/inet.h>
#include <linux/vmalloc.h>
+#include <linux/slab.h>
#include <linux/route.h>
*/
#ifdef DEBUG
#include <linux/types.h>
+#include <linux/slab.h>
#include "common.h"
#include "cxgb3_ioctl.h"
#include "cxio_hal.h"
#include <linux/spinlock.h>
#include <linux/pci.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <net/net_namespace.h>
#include "cxio_resource.h"
*/
#include <linux/module.h>
#include <linux/list.h>
+#include <linux/slab.h>
#include <linux/workqueue.h>
#include <linux/skbuff.h>
#include <linux/timer.h>
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
-#include <linux/slab.h>
+#include <linux/gfp.h>
#include <linux/mman.h>
#include <net/sock.h>
#include "iwch_provider.h"
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
+#include <linux/slab.h>
#include <asm/byteorder.h>
#include <rdma/iw_cm.h>
#include <linux/ethtool.h>
#include <linux/rtnetlink.h>
#include <linux/inetdevice.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/irq.h>
* SOFTWARE.
*/
#include <linux/sched.h>
+#include <linux/gfp.h>
#include "iwch_provider.h"
#include "iwch.h"
#include "iwch_cm.h"
* POSSIBILITY OF SUCH DAMAGE.
*/
+#include <linux/slab.h>
+
#include "ehca_tools.h"
#include "ehca_iverbs.h"
#include "hcp_if.h"
* POSSIBILITY OF SUCH DAMAGE.
*/
+#include <linux/slab.h>
+
#include "ehca_iverbs.h"
#include "ehca_classes.h"
#include "ehca_irq.h"
* POSSIBILITY OF SUCH DAMAGE.
*/
+#include <linux/gfp.h>
+
#include "ehca_tools.h"
#include "ehca_iverbs.h"
#include "hcp_if.h"
* POSSIBILITY OF SUCH DAMAGE.
*/
+#include <linux/slab.h>
+
#include "ehca_classes.h"
#include "ehca_irq.h"
#include "ehca_iverbs.h"
* POSSIBILITY OF SUCH DAMAGE.
*/
+#include <linux/slab.h>
#include <rdma/ib_umem.h>
#include "ehca_iverbs.h"
* POSSIBILITY OF SUCH DAMAGE.
*/
+#include <linux/slab.h>
+
#include "ehca_tools.h"
#include "ehca_iverbs.h"
* POSSIBILITY OF SUCH DAMAGE.
*/
+#include <linux/slab.h>
+
#include "ehca_classes.h"
#include "ehca_tools.h"
#include "ehca_qes.h"
* POSSIBILITY OF SUCH DAMAGE.
*/
+#include <linux/slab.h>
+
#include "ehca_classes.h"
#include "ehca_iverbs.h"
#include "ehca_mrmw.h"
* POSSIBILITY OF SUCH DAMAGE.
*/
+#include <linux/slab.h>
+
#include "ehca_tools.h"
#include "ipz_pt_fn.h"
#include "ehca_classes.h"
*/
#include <linux/err.h>
+#include <linux/slab.h>
#include <linux/vmalloc.h>
#include "ipath_verbs.h"
*/
#include <linux/scatterlist.h>
+#include <linux/gfp.h>
#include <rdma/ib_verbs.h>
#include "ipath_verbs.h"
#include <linux/netdevice.h>
#include <linux/vmalloc.h>
#include <linux/bitmap.h>
+#include <linux/slab.h>
#include "ipath_kernel.h"
#include "ipath_verbs.h"
#include <linux/cdev.h>
#include <linux/swap.h>
#include <linux/vmalloc.h>
+#include <linux/slab.h>
#include <linux/highmem.h>
#include <linux/io.h>
#include <linux/jiffies.h>
#include <linux/pagemap.h>
#include <linux/init.h>
#include <linux/namei.h>
+#include <linux/slab.h>
#include "ipath_kernel.h"
#include <linux/pci.h>
#include <linux/netdevice.h>
+#include <linux/slab.h>
#include <linux/vmalloc.h>
#include "ipath_kernel.h"
#include <linux/module.h>
#include <linux/vmalloc.h>
+#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/errno.h>
#include <asm/pgtable.h>
* SOFTWARE.
*/
+#include <linux/slab.h>
+
#include <rdma/ib_umem.h>
#include <rdma/ib_pack.h>
#include <rdma/ib_smi.h>
#include <linux/err.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/vmalloc.h>
#include "ipath_verbs.h"
*/
#include <linux/spinlock.h>
+#include <linux/gfp.h>
#include "ipath_kernel.h"
#include "ipath_verbs.h"
*/
#include <linux/err.h>
+#include <linux/slab.h>
#include <linux/vmalloc.h>
#include "ipath_verbs.h"
#include <linux/mm.h>
#include <linux/device.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#include "ipath_kernel.h"
#include <rdma/ib_mad.h>
#include <rdma/ib_user_verbs.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <linux/utsname.h>
#include <linux/rculist.h>
#include <linux/rculist.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include "ipath_verbs.h"
* SOFTWARE.
*/
+#include <linux/slab.h>
+
#include "mlx4_ib.h"
struct ib_ah *mlx4_ib_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr)
#include <linux/mlx4/cq.h>
#include <linux/mlx4/qp.h>
+#include <linux/slab.h>
#include "mlx4_ib.h"
#include "user.h"
#include <rdma/ib_smi.h>
#include <linux/mlx4/cmd.h>
+#include <linux/gfp.h>
#include "mlx4_ib.h"
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/errno.h>
#include <rdma/ib_smi.h>
* SOFTWARE.
*/
+#include <linux/slab.h>
+
#include "mlx4_ib.h"
static u32 convert_access(int acc)
*/
#include <linux/log2.h>
+#include <linux/slab.h>
#include <rdma/ib_cache.h>
#include <rdma/ib_pack.h>
#include <linux/mlx4/qp.h>
#include <linux/mlx4/srq.h>
+#include <linux/slab.h>
#include "mlx4_ib.h"
#include "user.h"
#include <linux/pci.h>
#include <linux/errno.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <rdma/ib_mad.h>
* SOFTWARE.
*/
+#include <linux/gfp.h>
#include <linux/hardirq.h>
#include <linux/sched.h>
#include <linux/errno.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include "mthca_dev.h"
#include "mthca_cmd.h"
#include <linux/errno.h>
#include <linux/pci.h>
#include <linux/interrupt.h>
+#include <linux/gfp.h>
#include "mthca_dev.h"
#include "mthca_config_reg.h"
*/
#include <linux/string.h>
-#include <linux/slab.h>
+#include <linux/gfp.h>
#include "mthca_dev.h"
#include "mthca_cmd.h"
#include <linux/mm.h>
#include <linux/scatterlist.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <asm/page.h>
#include <rdma/ib_user_verbs.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/mm.h>
#include "mthca_dev.h"
#include <linux/init.h>
#include <linux/if_arp.h>
#include <linux/highmem.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/irq.h>
#include <asm/byteorder.h>
#include <linux/list.h>
#include <linux/threads.h>
#include <linux/highmem.h>
+#include <linux/slab.h>
#include <net/arp.h>
#include <net/neighbour.h>
#include <net/route.h>
#include <linux/tcp.h>
#include <linux/if_vlan.h>
#include <linux/inet_lro.h>
+#include <linux/slab.h>
#include "nes.h"
#include <linux/if_arp.h>
#include <linux/if_vlan.h>
#include <linux/ethtool.h>
+#include <linux/slab.h>
#include <net/tcp.h>
#include <net/inet_common.h>
#include <linux/ethtool.h>
#include <linux/mii.h>
#include <linux/if_vlan.h>
+#include <linux/slab.h>
#include <linux/crc32.h>
#include <linux/in.h>
#include <linux/ip.h>
#include <linux/moduleparam.h>
#include <linux/random.h>
#include <linux/highmem.h>
+#include <linux/slab.h>
#include <asm/byteorder.h>
#include <rdma/ib_verbs.h>
#include <net/icmp.h>
#include <linux/icmpv6.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/vmalloc.h>
#include "ipoib.h"
#include <linux/err.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
struct file_operations;
#include <linux/delay.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <linux/ip.h>
#include <linux/tcp.h>
#include <linux/inetdevice.h>
#include <linux/delay.h>
#include <linux/completion.h>
+#include <linux/slab.h>
#include <net/dst.h>
* SOFTWARE.
*/
+#include <linux/slab.h>
+
#include "ipoib.h"
int ipoib_mcast_attach(struct net_device *dev, u16 mlid, union ib_gid *mgid, int set_qkey)
#include <linux/module.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/seq_file.h>
#include <asm/uaccess.h>
#include <linux/net.h>
#include <linux/scatterlist.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <net/sock.h>
*/
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include "iscsi_iser.h"
#include <linux/module.h>
#include <linux/mutex.h>
#include <linux/sched.h>
+#include <linux/slab.h>
/*
* Check that the effect_id is a valid effect and whether the user
#define debug(format, arg...) pr_debug("ff-memless: " format "\n", ## arg)
+#include <linux/slab.h>
#include <linux/input.h>
#include <linux/module.h>
#include <linux/mutex.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/gameport.h>
-#include <linux/slab.h>
#define L4_PORT 0x201
#define L4_SELECT_ANALOG 0xa4
*/
#include <linux/jiffies.h>
+#include <linux/slab.h>
#include <linux/mutex.h>
#include <linux/input-polldev.h>
#include <linux/types.h>
#include <linux/input.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/random.h>
#include <linux/major.h>
#include <linux/proc_fs.h>
#include <linux/parport.h>
#include <linux/input.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
MODULE_AUTHOR("Vojtech Pavlik <vojtech@ucw.cz>");
MODULE_DESCRIPTION("Atari, Amstrad, Commodore, Amiga, Sega, etc. joystick driver");
#include <linux/parport.h>
#include <linux/input.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
MODULE_AUTHOR("Vojtech Pavlik <vojtech@ucw.cz>");
MODULE_DESCRIPTION("NES, SNES, N64, MultiSystem, PSX gamepad driver");
#include <linux/module.h>
#include <linux/init.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
MODULE_AUTHOR("Vojtech Pavlik <vojtech@ucw.cz>");
MODULE_DESCRIPTION("TurboGraFX parallel port interface driver");
#include <linux/platform_device.h>
#include <linux/input.h>
#include <linux/mfd/adp5520.h>
+#include <linux/slab.h>
struct adp5520_keys {
struct input_dev *input;
#include <linux/platform_device.h>
#include <linux/input.h>
#include <linux/i2c.h>
+#include <linux/slab.h>
#include <linux/i2c/adp5588.h>
#include <linux/fs.h>
#include <linux/interrupt.h>
#include <linux/irq.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#include <linux/pm.h>
#include <linux/sysctl.h>
#include <linux/delay.h>
#include <linux/platform_device.h>
#include <linux/errno.h>
+#include <linux/slab.h>
#include <asm/irq.h>
#include <linux/clk.h>
#include <linux/io.h>
#include <linux/input/matrix_keypad.h>
+#include <linux/slab.h>
#include <mach/hardware.h>
#include <mach/ep93xx_keypad.h>
#include <linux/irq.h>
#include <linux/sched.h>
#include <linux/pm.h>
+#include <linux/slab.h>
#include <linux/sysctl.h>
#include <linux/proc_fs.h>
#include <linux/delay.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/timer.h>
/*
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <asm/delay.h>
#include <asm/io.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <mach/jornada720.h>
#include <mach/hardware.h>
#include <linux/input.h>
#include <linux/leds.h>
#include <linux/i2c/lm8323.h>
+#include <linux/slab.h>
/* Commands to send to the chip. */
#define LM8323_CMD_READ_ID 0x80 /* Read chip ID. */
#include <linux/module.h>
#include <linux/gpio.h>
#include <linux/input/matrix_keypad.h>
+#include <linux/slab.h>
struct matrix_keypad {
const struct matrix_keypad_platform_data *pdata;
#include <linux/module.h>
#include <linux/i2c.h>
+#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/input.h>
#include <linux/input/matrix_keypad.h>
#include <linux/platform_device.h>
#include <linux/mutex.h>
#include <linux/errno.h>
+#include <linux/slab.h>
#include <mach/gpio.h>
#include <plat/keypad.h>
#include <plat/menelaus.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
struct opencores_kbd {
struct input_dev *input;
#include <linux/clk.h>
#include <linux/err.h>
#include <linux/input/matrix_keypad.h>
+#include <linux/slab.h>
#include <asm/mach/arch.h>
#include <asm/mach/map.h>
#include <linux/input.h>
#include <linux/platform_device.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <mach/pxa930_rotary.h>
#include <linux/bitmap.h>
#include <linux/clk.h>
#include <linux/io.h>
+#include <linux/slab.h>
static const struct {
unsigned char kymd, keyout, keyin;
#include <linux/input.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <mach/gpio.h>
#include <mach/tosa.h>
#include <linux/input.h>
#include <linux/platform_device.h>
#include <linux/i2c/twl.h>
+#include <linux/slab.h>
/*
#include <linux/clk.h>
#include <linux/err.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <mach/w90p910_keypad.h>
#include <linux/input.h>
#include <linux/interrupt.h>
#include <linux/mfd/88pm860x.h>
+#include <linux/slab.h>
#define PM8607_WAKEUP 0x0b
*/
#include <linux/usb/input.h>
+#include <linux/slab.h>
#define DRIVER_DESC "ATI/Philips USB RF remote driver"
#define DRIVER_VERSION "0.3"
#include <linux/pm.h>
#include <linux/platform_device.h>
#include <linux/input.h>
+#include <linux/slab.h>
#include <asm/portmux.h>
#include <asm/bfin_rotary.h>
#include <linux/ioport.h>
#include <linux/module.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#define BUTTONS_POLL_INTERVAL 30 /* msec */
#define BUTTONS_COUNT_THRESHOLD 3
*/
#include <linux/kernel.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/input.h>
#include <linux/input/sparse-keymap.h>
#include <linux/platform_device.h>
#include <linux/platform_device.h>
#include <linux/input.h>
#include <linux/mfd/ezx-pcap.h>
+#include <linux/slab.h>
struct pcap_keys {
struct pcap_chip *pcap;
#include <linux/device.h>
#include <linux/platform_device.h>
#include <linux/input.h>
+#include <linux/slab.h>
#include <linux/mfd/pcf50633/core.h>
#include <linux/platform_device.h>
#include <linux/gpio.h>
#include <linux/rotary_encoder.h>
+#include <linux/slab.h>
#define DRV_NAME "rotary-encoder"
#include <linux/ioport.h>
#include <linux/module.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#ifdef CONFIG_SGI_IP22
#include <asm/sgi/ioc.h>
#include <linux/init.h>
#include <linux/input.h>
#include <linux/of_device.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <linux/i2c/twl.h>
#include <linux/mfd/twl4030-codec.h>
#include <linux/input.h>
+#include <linux/slab.h>
/* MODULE ID2 */
#define LEDEN 0x00
#include <linux/io.h>
#include <linux/bitrev.h>
#include <linux/bitops.h>
+#include <linux/slab.h>
#define DRVNAME "winbond-cir"
#include <linux/module.h>
#include <linux/preempt.h>
#include <linux/string.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/platform_device.h>
#include <linux/leds.h>
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/input.h>
* the Free Software Foundation.
*/
+#include <linux/slab.h>
#include <linux/input.h>
#include <linux/serio.h>
#include <linux/libps2.h>
*/
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/input.h>
#include <linux/serio.h>
*/
#define DEBUG
+#include <linux/slab.h>
#include <linux/input.h>
#include <linux/serio.h>
#include <linux/libps2.h>
#include <linux/serio.h>
#include <linux/libps2.h>
#include <linux/dmi.h>
+#include <linux/slab.h>
#include "psmouse.h"
#include "lifebook.h"
#include <linux/platform_device.h>
#include <linux/delay.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <mach/hardware.h>
#include <mach/pxa930_trkball.h>
#include <linux/libps2.h>
#include <linux/serio.h>
#include <linux/jiffies.h>
+#include <linux/slab.h>
#include "psmouse.h"
#include "sentelic.h"
#include <linux/input.h>
#include <linux/serio.h>
#include <linux/libps2.h>
+#include <linux/slab.h>
#include "psmouse.h"
#include "synaptics.h"
#include <linux/input.h>
#include <linux/delay.h>
#include <linux/workqueue.h>
+#include <linux/slab.h>
#define DRIVER_NAME "synaptics_i2c"
/* maximum product id is 15 characters */
*/
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/input.h>
#include <linux/serio.h>
* Trademarks are the property of their respective owners.
*/
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/serio.h>
#include <linux/module.h>
#include <linux/interrupt.h>
#include <linux/platform_device.h>
#include <linux/io.h>
+#include <linux/slab.h>
#define DRV_NAME "altera_ps2"
#include <linux/io.h>
#include <linux/clk.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
/* PSIF register offsets */
#define PSIF_CR 0x00
#include <linux/errno.h>
#include <linux/err.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <linux/init.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/serio.h>
#include <linux/input.h>
#include <linux/interrupt.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <linux/timer.h>
#include <linux/list.h>
#include <linux/rcupdate.h>
#include <linux/platform_device.h>
#include <linux/i8042.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <linux/delay.h>
#include <linux/module.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/input.h>
#include <linux/serio.h>
#include <linux/module.h>
#include <linux/parport.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/serio.h>
#include <linux/ioport.h>
#include <linux/input.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/serio.h>
#include <linux/delay.h>
#include <linux/err.h>
#include <linux/bitops.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/uaccess.h>
#include <linux/err.h>
#include <linux/platform_device.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <asm/irq.h>
#include <mach/hardware.h>
#include <linux/serio.h>
#include <linux/interrupt.h>
#include <linux/errno.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/list.h>
#include <linux/io.h>
#include <linux/input.h>
#include <linux/input/sparse-keymap.h>
+#include <linux/slab.h>
MODULE_AUTHOR("Dmitry Torokhov <dtor@mail.ru>");
MODULE_DESCRIPTION("Generic support for sparse keymaps");
#include <linux/i2c.h>
#include <linux/input.h>
#include <linux/mfd/88pm860x.h>
+#include <linux/slab.h>
#define MEAS_LEN (8)
#define ACCURATE_BIT (12)
#include <linux/timer.h>
#include <linux/gpio.h>
#include <linux/io.h>
+#include <linux/slab.h>
#define AC97C_ICA 0x10
#define AC97C_CBRHR 0x30
#include <linux/input.h>
#include <linux/workqueue.h>
#include <linux/mfd/da903x.h>
+#include <linux/slab.h>
#define DA9034_MANUAL_CTRL 0x50
#define DA9034_LDO_ADC_EN (1 << 4)
#include <linux/timer.h>
#include <linux/gpio.h>
#include <linux/input/eeti_ts.h>
+#include <linux/slab.h>
static int flip_x;
module_param(flip_x, bool, 0644);
#include <linux/input.h>
#include <linux/interrupt.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <mach/hardware.h>
#include <mach/jornada720.h>
#include <linux/module.h>
#include <linux/input.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/init.h>
#define MC13783_TS_NAME "mc13783-ts"
#include <linux/interrupt.h>
#include <linux/input.h>
#include <linux/irq.h>
+#include <linux/slab.h>
/* Registers */
#define MCS5000_TS_STATUS 0x00
#include <linux/kernel.h>
#include <linux/input.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <linux/i2c.h>
#include <linux/timer.h>
#include <linux/init.h>
#include <linux/fs.h>
#include <linux/string.h>
+#include <linux/slab.h>
#include <linux/pm.h>
#include <linux/timer.h>
#include <linux/interrupt.h>
#include <linux/errno.h>
#include <linux/kernel.h>
#include <linux/module.h>
-#include <linux/slab.h>
#include <linux/gpio.h>
#include <linux/input.h>
#include <linux/init.h>
#include <linux/device.h>
#include <linux/interrupt.h>
#include <linux/suspend.h>
-#include <linux/slab.h>
#include <linux/kthread.h>
#include <linux/freezer.h>
#include <linux/ucb1400.h>
#include <linux/clk.h>
#include <linux/input.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
/* ADC controller bit defines */
#define ADC_DELAY 0xf00
#include <linux/wm97xx.h>
#include <linux/uaccess.h>
#include <linux/io.h>
+#include <linux/slab.h>
#define TS_NAME "wm97xx"
#define WM_CORE_VERSION "1.00"
#include <linux/errno.h>
#include <linux/module.h>
#include <linux/input.h>
+#include <linux/slab.h>
#include <asm/xen/hypervisor.h>
#include "act2000_isa.h"
#include "capi.h"
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
static unsigned short act2000_isa_ports[] =
#include <linux/fs.h>
#include <linux/mount.h>
+#include <linux/slab.h>
#include <linux/namei.h>
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/isdn/capilli.h>
#include <linux/mm.h>
#include <linux/init.h>
#include <linux/isdn/capiutil.h>
+#include <linux/slab.h>
/* from CAPI2.0 DDK AVM Berlin GmbH */
#include <linux/init.h>
#include <linux/moduleparam.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <linux/isdn/capicmd.h>
#include <linux/isdn/capiutil.h>
#include <linux/module.h>
#include <linux/poll.h>
+#include <linux/slab.h>
#ifdef CONFIG_PROC_FS
#include <linux/proc_fs.h>
#else
*/
#include <linux/proc_fs.h>
+#include <linux/slab.h>
#include <linux/timer.h>
#include <linux/jiffies.h>
*/
#include "gigaset.h"
+#include <linux/slab.h>
#include <linux/ctype.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
#include <linux/ctype.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
+#include <linux/slab.h>
/* Version Information */
#define DRIVER_AUTHOR "Hansjoerg Lipp <hjlipp@web.de>, Tilman Schmidt <tilman@imap.cc>, Stefan Eilers"
#include <linux/kernel.h>
#include <linux/compiler.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/usb.h>
#include <linux/skbuff.h>
#include "gigaset.h"
#include <linux/isdnif.h>
+#include <linux/slab.h>
#define HW_HDR_LEN 2 /* Header size used to store ack info */
#include <linux/platform_device.h>
#include <linux/tty.h>
#include <linux/completion.h>
+#include <linux/slab.h>
/* Version Information */
#define DRIVER_AUTHOR "Tilman Schmidt"
#include <linux/ioport.h>
#include <linux/capi.h>
#include <linux/kernelcapi.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <linux/init.h>
#include <asm/uaccess.h>
#include <linux/ioport.h>
#include <linux/capi.h>
#include <linux/kernelcapi.h>
+#include <linux/gfp.h>
#include <asm/io.h>
#include <linux/init.h>
#include <asm/uaccess.h>
#include <linux/capi.h>
#include <linux/kernelcapi.h>
#include <linux/init.h>
+#include <linux/gfp.h>
#include <asm/io.h>
#include <asm/uaccess.h>
#include <linux/netdevice.h>
#include <linux/kernelcapi.h>
#include <linux/init.h>
#include <linux/pci.h>
+#include <linux/gfp.h>
#include <asm/io.h>
#include <linux/isdn/capicmd.h>
#include <linux/isdn/capiutil.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <asm/uaccess.h>
#include <linux/seq_file.h>
#include <linux/pci.h>
#include <linux/delay.h>
#include <linux/mISDNhw.h>
+#include <linux/slab.h>
#include <asm/unaligned.h>
#include "ipac.h"
#define HFC_MULTI_VERSION "2.03"
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/delay.h>
#include <linux/mISDNhw.h>
#include <linux/pci.h>
#include <linux/delay.h>
#include <linux/mISDNhw.h>
+#include <linux/slab.h>
#include "hfc_pci.h"
#include <linux/delay.h>
#include <linux/usb.h>
#include <linux/mISDNhw.h>
+#include <linux/slab.h>
#include "hfcsusb.h"
static const char *hfcsusb_rev = "Revision: 0.3.3 (socket), 2008-11-05";
#include <linux/pci.h>
#include <linux/delay.h>
#include <linux/mISDNhw.h>
+#include <linux/slab.h>
#include "ipac.h"
#define INFINEON_REV "1.0"
*
*/
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/mISDNhw.h>
#include "ipac.h"
*/
/* #define DEBUG */
+#include <linux/gfp.h>
#include <linux/delay.h>
#include <linux/vmalloc.h>
#include <linux/mISDNhw.h>
#include <linux/pci.h>
#include <linux/delay.h>
#include <linux/mISDNhw.h>
+#include <linux/slab.h>
#include "ipac.h"
#include "iohelper.h"
#include "netjet.h"
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/delay.h>
#include <linux/mISDNhw.h>
#include <linux/pci.h>
#include <linux/delay.h>
#include <linux/mISDNhw.h>
+#include <linux/slab.h>
#include "w6692.h"
#define W6692_REV "2.0"
#include "amd7930_fn.h"
#include <linux/interrupt.h>
#include <linux/init.h>
+#include <linux/gfp.h>
static void Amd7930_new_ph(struct IsdnCardState *cs);
#include "isac.h"
#include "isdnl1.h"
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/isapnp.h>
#include <linux/interrupt.h>
handler.
*/
-static int avma1cs_config(struct pcmcia_device *link);
+static int avma1cs_config(struct pcmcia_device *link) __devinit ;
static void avma1cs_release(struct pcmcia_device *link);
/*
needed to manage one actual PCMCIA card.
*/
-static void avma1cs_detach(struct pcmcia_device *p_dev);
+static void avma1cs_detach(struct pcmcia_device *p_dev) __devexit ;
/*
======================================================================*/
-static int avma1cs_probe(struct pcmcia_device *p_dev)
+static int __devinit avma1cs_probe(struct pcmcia_device *p_dev)
{
local_info_t *local;
======================================================================*/
-static void avma1cs_detach(struct pcmcia_device *link)
+static void __devexit avma1cs_detach(struct pcmcia_device *link)
{
dev_dbg(&link->dev, "avma1cs_detach(0x%p)\n", link);
avma1cs_release(link);
}
-static int avma1cs_config(struct pcmcia_device *link)
+static int __devinit avma1cs_config(struct pcmcia_device *link)
{
local_info_t *dev;
int i;
.name = "avma1_cs",
},
.probe = avma1cs_probe,
- .remove = avma1cs_detach,
+ .remove = __devexit_p(avma1cs_detach),
.id_table = avma1cs_ids,
};
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include "hisax.h"
#include <linux/isdn/capicmd.h>
#include <linux/kernel_stat.h>
#include <linux/workqueue.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#define HISAX_STATUS_BUFSIZE 4096
/*
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include "hisax.h"
#include "arcofi.h"
#include "isac.h"
handler.
*/
-static int elsa_cs_config(struct pcmcia_device *link);
+static int elsa_cs_config(struct pcmcia_device *link) __devinit ;
static void elsa_cs_release(struct pcmcia_device *link);
/*
needed to manage one actual PCMCIA card.
*/
-static void elsa_cs_detach(struct pcmcia_device *p_dev);
+static void elsa_cs_detach(struct pcmcia_device *p_dev) __devexit;
/*
A driver needs to provide a dev_node_t structure for each device
======================================================================*/
-static int elsa_cs_probe(struct pcmcia_device *link)
+static int __devinit elsa_cs_probe(struct pcmcia_device *link)
{
local_info_t *local;
======================================================================*/
-static void elsa_cs_detach(struct pcmcia_device *link)
+static void __devexit elsa_cs_detach(struct pcmcia_device *link)
{
local_info_t *info = link->priv;
return -ENODEV;
}
-static int elsa_cs_config(struct pcmcia_device *link)
+static int __devinit elsa_cs_config(struct pcmcia_device *link)
{
local_info_t *dev;
int i;
.name = "elsa_cs",
},
.probe = elsa_cs_probe,
- .remove = elsa_cs_detach,
+ .remove = __devexit_p(elsa_cs_detach),
.id_table = elsa_ids,
.suspend = elsa_suspend,
.resume = elsa_resume,
#include <linux/serial.h>
#include <linux/serial_reg.h>
+#include <linux/slab.h>
#define MAX_MODEM_BUF 256
#define WAKEUP_CHARS (MAX_MODEM_BUF/2)
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include "hisax.h"
#include <linux/pci.h>
#include <linux/interrupt.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/timer.h>
#include <linux/skbuff.h>
#include <linux/wait.h>
#include <linux/init.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include "hisax.h"
#include "hfc_2bds0.h"
#include "isdnl1.h"
#include "isac.h"
#include "isdnl1.h"
#include <linux/interrupt.h>
+#include <linux/slab.h>
static inline int
WaitForBusy(struct IsdnCardState *cs)
#include "isdnl1.h"
#include <linux/interrupt.h>
#include <linux/isapnp.h>
+#include <linux/slab.h>
static const char *hfcsx_revision = "$Revision: 1.12.2.5 $";
#include <linux/kernel.h>
#include <linux/sched.h>
#include <linux/moduleparam.h>
+#include <linux/slab.h>
#include "hisax.h"
#include "hisax_if.h"
#include "hfc_usb.h"
*/
#include <linux/module.h>
+#include <linux/gfp.h>
#include <linux/init.h>
#include <linux/netdevice.h>
#include "hisax_isac.h"
#include "isac.h"
#include "isdnl1.h"
#include <linux/interrupt.h>
+#include <linux/slab.h>
static char *HSCXVer[] =
{"A1", "?1", "A2", "?3", "A3", "V2.1", "?6", "?7",
// #include "arcofi.h"
#include "isdnl1.h"
#include <linux/interrupt.h>
+#include <linux/slab.h>
#define DBUSY_TIMER_VALUE 80
#define ARCOFI_USE 0
*
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include "hisax_if.h"
#include "hisax.h"
#include "arcofi.h"
#include "isdnl1.h"
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <linux/init.h>
#define DBUSY_TIMER_VALUE 80
#include "isar.h"
#include "isdnl1.h"
#include <linux/interrupt.h>
+#include <linux/slab.h>
#define DBG_LOADFIRM 0
#define DUMP_MBOXFRAME 2
*/
#include <linux/init.h>
+#include <linux/gfp.h>
#include "hisax.h"
#include "isdnl1.h"
*/
#include <linux/init.h>
+#include <linux/gfp.h>
#include "hisax.h"
#include "isdnl2.h"
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include "hisax.h"
#include "isdnl3.h"
#include "jade.h"
#include "isdnl1.h"
#include <linux/interrupt.h>
+#include <linux/slab.h>
int
#include "isdnl3.h"
#include "l3dss1.h"
#include <linux/ctype.h>
+#include <linux/slab.h>
extern char *HiSax_getrev(const char *revision);
static const char *dss1_revision = "$Revision: 2.32.2.3 $";
#include "isdnl3.h"
#include "l3ni1.h"
#include <linux/ctype.h>
+#include <linux/slab.h>
extern char *HiSax_getrev(const char *revision);
static const char *ni1_revision = "$Revision: 2.8.2.3 $";
#include "isdnl1.h"
#include <linux/interrupt.h>
#include <linux/ppp_defs.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include "netjet.h"
event handler.
*/
-static int sedlbauer_config(struct pcmcia_device *link);
+static int sedlbauer_config(struct pcmcia_device *link) __devinit ;
static void sedlbauer_release(struct pcmcia_device *link);
/*
needed to manage one actual PCMCIA card.
*/
-static void sedlbauer_detach(struct pcmcia_device *p_dev);
+static void sedlbauer_detach(struct pcmcia_device *p_dev) __devexit;
/*
You'll also need to prototype all the functions that will actually
======================================================================*/
-static int sedlbauer_probe(struct pcmcia_device *link)
+static int __devinit sedlbauer_probe(struct pcmcia_device *link)
{
local_info_t *local;
======================================================================*/
-static void sedlbauer_detach(struct pcmcia_device *link)
+static void __devexit sedlbauer_detach(struct pcmcia_device *link)
{
dev_dbg(&link->dev, "sedlbauer_detach(0x%p)\n", link);
-static int sedlbauer_config(struct pcmcia_device *link)
+static int __devinit sedlbauer_config(struct pcmcia_device *link)
{
local_info_t *dev = link->priv;
win_req_t *req;
.name = "sedlbauer_cs",
},
.probe = sedlbauer_probe,
- .remove = sedlbauer_detach,
+ .remove = __devexit_p(sedlbauer_detach),
.id_table = sedlbauer_ids,
.suspend = sedlbauer_suspend,
.resume = sedlbauer_resume,
*/
#include <linux/init.h>
+#include <linux/gfp.h>
#include <linux/usb.h>
-#include <linux/slab.h>
#include <linux/netdevice.h>
#include <linux/bitrev.h>
#include "st5481.h"
*/
#include <linux/init.h>
+#include <linux/gfp.h>
#include <linux/usb.h>
-#include <linux/slab.h>
#include <linux/netdevice.h>
#include "st5481.h"
#include "hisax.h"
#include "isdnl2.h"
+#include <linux/gfp.h>
#include <linux/init.h>
#include <linux/random.h>
handler.
*/
-static int teles_cs_config(struct pcmcia_device *link);
+static int teles_cs_config(struct pcmcia_device *link) __devinit ;
static void teles_cs_release(struct pcmcia_device *link);
/*
needed to manage one actual PCMCIA card.
*/
-static void teles_detach(struct pcmcia_device *p_dev);
+static void teles_detach(struct pcmcia_device *p_dev) __devexit ;
/*
A linked list of "instances" of the teles_cs device. Each actual
======================================================================*/
-static int teles_probe(struct pcmcia_device *link)
+static int __devinit teles_probe(struct pcmcia_device *link)
{
local_info_t *local;
======================================================================*/
-static void teles_detach(struct pcmcia_device *link)
+static void __devexit teles_detach(struct pcmcia_device *link)
{
local_info_t *info = link->priv;
return -ENODEV;
}
-static int teles_cs_config(struct pcmcia_device *link)
+static int __devinit teles_cs_config(struct pcmcia_device *link)
{
local_info_t *dev;
int i;
.name = "teles_cs",
},
.probe = teles_probe,
- .remove = teles_detach,
+ .remove = __devexit_p(teles_detach),
.id_table = teles_ids,
.suspend = teles_suspend,
.resume = teles_resume,
#include "isdnl1.h"
#include <linux/interrupt.h>
#include <linux/pci.h>
+#include <linux/slab.h>
/* table entry in the PCI devices list */
typedef struct {
#include <linux/kernel.h>
#include <linux/skbuff.h>
#include <linux/netdevice.h>
+#include <linux/slab.h>
#define VER_DRIVER 0
#define VER_CARDTYPE 1
#include <linux/poll.h>
#include <linux/proc_fs.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/smp_lock.h>
#include <net/net_namespace.h>
#include <linux/poll.h>
#include <linux/proc_fs.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/smp_lock.h>
#include "hysdn_defs.h"
*/
#include <linux/isdn.h>
+#include <linux/slab.h>
#include "isdn_audio.h"
#include "isdn_common.h"
#include <linux/module.h>
#include <linux/init.h>
#include <linux/poll.h>
+#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/isdn.h>
#include <linux/smp_lock.h>
*/
#include <linux/isdn.h>
+#include <linux/slab.h>
#include <net/arp.h>
#include <net/dst.h>
#include <net/pkt_sched.h>
#include <linux/isdn.h>
#include <linux/poll.h>
#include <linux/ppp-comp.h>
+#include <linux/slab.h>
#ifdef CONFIG_IPPP_FILTER
#include <linux/filter.h>
#endif
#undef ISDN_TTY_STAT_DEBUG
#include <linux/isdn.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/smp_lock.h>
#include "isdn_common.h"
/* #include <linux/isdn.h> */
#include <linux/netdevice.h>
#include <linux/concap.h>
+#include <linux/slab.h>
#include <linux/wanrouter.h>
#include <net/x25device.h>
#include "isdn_x25iface.h"
#include "icn.h"
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/sched.h>
static int portbase = ICN_BASEADDR;
#include <linux/module.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/sched.h>
#include "isdnloop.h"
*
*/
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/stddef.h>
#include <linux/spinlock.h>
*
*/
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/stddef.h>
#include <linux/module.h>
/* delay.h is required for hw_lock.h */
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/mISDNif.h>
#include <linux/mISDNdsp.h>
*/
#include <linux/delay.h>
+#include <linux/gfp.h>
#include <linux/mISDNif.h>
#include <linux/mISDNdsp.h>
#include <linux/module.h>
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/list.h>
#include <linux/string.h>
#include <linux/mISDNif.h>
*
*/
+#include <linux/gfp.h>
#include <linux/mISDNif.h>
#include <linux/mISDNdsp.h>
#include "core.h"
*
*/
+#include <linux/gfp.h>
#include <linux/module.h>
#include <linux/mISDNhw.h>
#include <linux/inet.h>
#include <linux/workqueue.h>
#include <linux/kthread.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include "core.h"
#include "l1oip.h"
*/
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/mISDNhw.h>
#include "core.h"
*/
#include <linux/mISDNif.h>
+#include <linux/slab.h>
#include "core.h"
#include "fsm.h"
#include "layer2.h"
*/
#include <linux/mISDNif.h>
+#include <linux/slab.h>
#include "core.h"
static u_int *debug;
*
*/
+#include <linux/slab.h>
#include <linux/mISDNif.h>
#include <linux/kthread.h>
#include <linux/smp_lock.h>
*/
#include "layer2.h"
#include <linux/random.h>
+#include <linux/slab.h>
#include "core.h"
#define ID_REQUEST 1
#include <linux/poll.h>
#include <linux/vmalloc.h>
+#include <linux/slab.h>
#include <linux/timer.h>
#include <linux/miscdevice.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/types.h>
-#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/skbuff.h>
#include <linux/kernel.h>
#include <linux/types.h>
-#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/skbuff.h>
#include <linux/interrupt.h>
#include <linux/delay.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include "includes.h"
#include "hardware.h"
#include "card.h"
#include <linux/acpi.h>
#include <linux/leds.h>
+#include <linux/slab.h>
MODULE_AUTHOR("Louis Davis/Jim Dailey");
MODULE_DESCRIPTION("Dell LED Control Driver");
#include <linux/timer.h>
#include <linux/rwsem.h>
#include <linux/leds.h>
+#include <linux/slab.h>
#include "leds.h"
/*
#include <linux/platform_device.h>
#include <linux/i2c.h>
#include <linux/leds.h>
+#include <linux/slab.h>
#include <linux/workqueue.h>
#include <linux/mfd/88pm860x.h>
#include <linux/leds.h>
#include <linux/workqueue.h>
#include <linux/mfd/adp5520.h>
+#include <linux/slab.h>
struct adp5520_led {
struct led_classdev cdev;
#include <linux/leds.h>
#include <linux/io.h>
#include <linux/atmel_pwm.h>
+#include <linux/slab.h>
struct pwmled {
#include <linux/delay.h>
#include <linux/leds.h>
#include <linux/leds-bd2802.h>
+#include <linux/slab.h>
#define LED_CTL(rgb2en, rgb1en) ((rgb2en) << 4 | ((rgb1en) << 0))
#include <linux/leds.h>
#include <linux/workqueue.h>
#include <linux/mfd/da903x.h>
+#include <linux/slab.h>
#define DA9030_LED1_CONTROL 0x20
#define DA9030_LED2_CONTROL 0x21
* LED driver for the DAC124S085 SPI DAC
*/
-#include <linux/gfp.h>
#include <linux/leds.h>
#include <linux/module.h>
#include <linux/mutex.h>
#include <linux/init.h>
#include <linux/platform_device.h>
#include <linux/leds.h>
+#include <linux/slab.h>
#include <linux/workqueue.h>
#include <asm/gpio.h>
#include <linux/module.h>
#include <linux/i2c.h>
+#include <linux/slab.h>
#include <linux/leds.h>
#include <linux/mutex.h>
#include <linux/workqueue.h>
#include <linux/workqueue.h>
#include <linux/delay.h>
#include <linux/gpio.h>
+#include <linux/slab.h>
struct lt3593_led_data {
struct led_classdev cdev;
#include <linux/module.h>
#include <linux/i2c.h>
+#include <linux/slab.h>
#include <linux/leds.h>
#include <linux/input.h>
#include <linux/mutex.h>
#include <linux/err.h>
#include <linux/i2c.h>
#include <linux/workqueue.h>
+#include <linux/slab.h>
/* LED select registers determine the source that drives LED outputs */
#define PCA955X_LS_LED_ON 0x0 /* Output LOW */
#include <linux/err.h>
#include <linux/pwm.h>
#include <linux/leds_pwm.h>
+#include <linux/slab.h>
struct led_pwm_data {
struct led_classdev cdev;
#include <linux/module.h>
#include <linux/err.h>
+#include <linux/slab.h>
#include <linux/workqueue.h>
#include <linux/leds.h>
#include <linux/leds-regulator.h>
#include <linux/platform_device.h>
#include <linux/leds.h>
#include <linux/gpio.h>
+#include <linux/slab.h>
#include <mach/hardware.h>
#include <mach/regs-gpio.h>
#include <linux/leds.h>
#include <linux/io.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <asm/fhc.h>
#include <asm/upa.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/leds.h>
#include <linux/err.h>
#include <linux/mfd/wm831x/core.h>
#include <linux/err.h>
#include <linux/mfd/wm8350/pmic.h>
#include <linux/regulator/consumer.h>
+#include <linux/slab.h>
/* Microamps */
static const int isink_cur[] = {
#include <linux/module.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/fb.h>
#include <linux/leds.h>
#include <linux/interrupt.h>
#include <linux/workqueue.h>
#include <linux/leds.h>
+#include <linux/slab.h>
#include "leds.h"
struct gpio_trig_data {
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/timer.h>
#include <linux/sched.h>
#include <linux/leds.h>
#include <linux/timer.h>
#include <linux/ctype.h>
#include <linux/leds.h>
+#include <linux/slab.h>
#include "leds.h"
struct timer_trig_data {
#include <linux/cpu.h>
#include <linux/freezer.h>
#include <linux/highmem.h>
+#include <linux/slab.h>
#include <asm/paravirt.h>
#include <asm/pgtable.h>
#include <asm/uaccess.h>
#include <linux/wait.h>
#include <linux/hrtimer.h>
#include <linux/err.h>
+#include <linux/slab.h>
#include <asm/lguest.h>
#include <linux/interrupt.h>
#include <linux/virtio_ring.h>
#include <linux/err.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/paravirt.h>
#include <asm/lguest_hcall.h>
#include <linux/sched.h>
#include <linux/eventfd.h>
#include <linux/file.h>
+#include <linux/slab.h>
#include "lg.h"
/*L:056
/* Copyright (C) Rusty Russell IBM Corporation 2006.
* GPL v2 and any later version */
#include <linux/mm.h>
+#include <linux/gfp.h>
#include <linux/types.h>
#include <linux/spinlock.h>
#include <linux/random.h>
#include <linux/sysctl.h>
#include <linux/input.h>
#include <linux/module.h>
+#include <linux/slab.h>
MODULE_LICENSE("GPL");
#include <linux/types.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/device.h>
#include <linux/interrupt.h>
#include <linux/module.h>
#include <linux/mutex.h>
#include <linux/of_device.h>
#include <linux/of_platform.h>
+#include <linux/slab.h>
#include <asm/byteorder.h>
#include <asm/io.h>
#include <linux/kernel.h>
#include <linux/delay.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/init.h>
#include <linux/spinlock.h>
#include <linux/wait.h>
#include <linux/delay.h>
#include <linux/sched.h>
#include <linux/i2c.h>
-#include <linux/slab.h>
#include <linux/init.h>
#include <linux/kthread.h>
#include <linux/of_platform.h>
#include <linux/miscdevice.h>
#include <linux/blkdev.h>
#include <linux/pci.h>
-#include <linux/slab.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/types.h>
#include <linux/errno.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/spinlock.h>
#include <linux/kthread.h>
*/
#include <linux/bio.h>
+#include <linux/slab.h>
#include <linux/dm-dirty-log.h>
#include <linux/device-mapper.h>
#include <linux/dm-log-userspace.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <linux/workqueue.h>
#include <linux/connector.h>
#include <linux/ctype.h>
#include <linux/init.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/vmalloc.h>
#include "dm.h"
#include "dm.h"
#include "dm-path-selector.h"
+#include <linux/slab.h>
+
#define DM_MSG_PREFIX "multipath service-time"
#define ST_MIN_IO 1
#define ST_MAX_RELATIVE_THROUGHPUT 100
#include <linux/init.h>
#include <linux/kmod.h>
#include <linux/bio.h>
-#include <linux/slab.h>
#define DM_MSG_PREFIX "target"
#define MaxFault 50
#include <linux/blkdev.h>
#include <linux/raid/md_u.h>
+#include <linux/slab.h>
#include "md.h"
#include <linux/seq_file.h>
#include <linux/blkdev.h>
#include <linux/raid/md_u.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include "md.h"
#include "linear.h"
#include <linux/delay.h>
#include <linux/raid/md_p.h>
#include <linux/raid/md_u.h>
+#include <linux/slab.h>
#include "md.h"
#include "bitmap.h"
#include <linux/blkdev.h>
#include <linux/raid/md_u.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include "md.h"
#include "multipath.h"
#include <linux/blkdev.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include "md.h"
#include "raid0.h"
* Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/blkdev.h>
#include <linux/seq_file.h>
* Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/blkdev.h>
#include <linux/seq_file.h>
#include <linux/async.h>
#include <linux/seq_file.h>
#include <linux/cpu.h>
+#include <linux/slab.h>
#include "md.h"
#include "raid5.h"
#include "bitmap.h"
*/
#include <linux/raid/pq.h>
+#include <linux/gfp.h>
#ifndef __KERNEL__
#include <sys/mman.h>
#include <stdio.h>
#include <linux/input.h>
+#include <linux/slab.h>
#include <media/ir-common.h>
#define IR_TAB_MIN_SIZE 32
* GNU General Public License for more details.
*/
+#include <linux/slab.h>
#include <linux/input.h>
#include <linux/device.h>
#include <media/ir-core.h>
#include <linux/delay.h>
#include <linux/dvb/frontend.h>
#include <linux/i2c.h>
+#include <linux/slab.h>
#include "dvb_frontend.h"
#include <linux/delay.h>
#include <linux/dvb/frontend.h>
#include <linux/i2c.h>
+#include <linux/slab.h>
#include "dvb_frontend.h"
#include <linux/delay.h>
#include <linux/dvb/frontend.h>
#include <linux/i2c.h>
+#include <linux/slab.h>
#include "dvb_frontend.h"
*/
#include <linux/delay.h>
#include <linux/i2c.h>
+#include <linux/slab.h>
#include <linux/videodev2.h>
#include "tuner-i2c.h"
#include "mt20xx.h"
#include <linux/delay.h>
#include <linux/dvb/frontend.h>
#include <linux/i2c.h>
+#include <linux/slab.h>
#include "dvb_frontend.h"
#include <linux/delay.h>
#include <linux/dvb/frontend.h>
#include <linux/i2c.h>
+#include <linux/slab.h>
#include "dvb_frontend.h"
#include "mt2266.h"
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <asm/types.h>
#include <linux/dvb/frontend.h>
#include <linux/videodev2.h>
*/
#include <linux/i2c.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/videodev2.h>
#include "tuner-i2c.h"
#include <linux/types.h>
#include <linux/init.h>
#include <linux/errno.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/videodev2.h>
#include <media/v4l2-common.h>
*/
#include <linux/i2c.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/videodev2.h>
#include <media/tuner.h>
*/
#include <linux/i2c.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/videodev2.h>
#include "tuner-i2c.h"
#define __TUNER_I2C_H__
#include <linux/i2c.h>
+#include <linux/slab.h>
struct tuner_i2c_props {
u8 addr;
#include <linux/delay.h>
#include <media/tuner.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <asm/unaligned.h>
#include "tuner-i2c.h"
#include "tuner-xc2028.h"
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/smp_lock.h>
#include <linux/string.h>
#include <linux/pci.h>
#include <linux/dma-mapping.h>
#include <linux/input.h>
+#include <linux/slab.h>
#include <media/ir-common.h>
#include "demux.h"
#include <linux/fs.h>
#include <linux/string.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <linux/dvb/dmx.h>
#include <linux/errno.h>
#include <linux/delay.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <linux/dvb/frontend.h>
*/
#include <linux/hash.h>
+#include <linux/slab.h>
#include "af9015.h"
#include "af9013.h"
*/
#include <media/tuner.h>
#include <linux/vmalloc.h>
+#include <linux/slab.h>
#include "cxusb.h"
#include <linux/errno.h>
#include <linux/kernel.h>
#include <linux/list.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/types.h>
#include <linux/bitops.h>
#include <linux/input.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include <linux/types.h>
#include <linux/workqueue.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/string.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include "dvb_frontend.h"
#include "au8522.h"
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/i2c.h>
#include "dvb_frontend.h"
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/i2c.h>
#include "dvb_frontend.h"
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/i2c.h>
#include "dvb_frontend.h"
* published by the Free Software Foundation, version 2.
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/i2c.h>
#include "dvb_frontend.h"
* published by the Free Software Foundation, version 2.
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/i2c.h>
#include "dvb_math.h"
* published by the Free Software Foundation, version 2.
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/i2c.h>
#include "dvb_math.h"
#include <linux/delay.h>
#include <linux/string.h>
#include <linux/firmware.h>
+#include <linux/slab.h>
#include <asm/div64.h>
#include "dvb_frontend.h"
* Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/dvb/frontend.h>
#include <asm/types.h>
#include <linux/delay.h>
#include <linux/dvb/frontend.h>
#include <linux/i2c.h>
+#include <linux/slab.h>
#include "dvb_frontend.h"
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include "dvb_frontend.h"
#include "lgdt3304.h"
#include <asm/div64.h>
#include <linux/dvb/frontend.h>
+#include <linux/slab.h>
#include "dvb_math.h"
#include "lgdt3305.h"
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
+#include <linux/slab.h>
#include "dvb_frontend.h"
#include "mb86a16.h"
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include "dvb_frontend.h"
#include "s921_module.h"
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include <linux/dvb/frontend.h>
*/
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/dvb/frontend.h>
#include <asm/types.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include "dvb_frontend.h"
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/string.h>
+#include <linux/slab.h>
#include <linux/mutex.h>
#include <linux/dvb/frontend.h>
* Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/dvb/frontend.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include "dvb_frontend.h"
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include "dvb_frontend.h"
#include "tda665x.h"
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include "dvb_frontend.h"
#include "tda8261.h"
*/
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/dvb/frontend.h>
#include <asm/types.h>
* Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/dvb/frontend.h>
#include <asm/types.h>
#include <linux/module.h>
#include <linux/dvb/frontend.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include "zl10036.h"
#include <linux/moduleparam.h>
#include <linux/kernel.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <asm/irq.h>
#include <linux/interrupt.h>
*/
#include <linux/signal.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#include <linux/interrupt.h>
#include <linux/moduleparam.h>
#include <linux/kernel.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <asm/irq.h>
#include <linux/interrupt.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/delay.h>
-#include <linux/slab.h>
#include <linux/poll.h>
#include <linux/io.h>
#include <asm/div64.h>
#include <linux/module.h>
#include <linux/pci.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include "demux.h"
#include "dmxdev.h"
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/pci.h>
#include <linux/kthread.h>
#include <linux/dma-mapping.h>
#include <linux/delay.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <linux/firmware.h>
#include <linux/wait.h>
****************************************************************/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include "dmxdev.h"
*/
#include <linux/moduleparam.h>
+#include <linux/slab.h>
#include <linux/firmware.h>
#include <linux/delay.h>
#include <linux/mmc/card.h>
#include <linux/init.h>
#include <linux/usb.h>
#include <linux/firmware.h>
+#include <linux/slab.h>
#include "smscoreapi.h"
#include "sms-cards.h"
#include <linux/crc32.h>
#include <linux/i2c.h>
#include <linux/kthread.h>
+#include <linux/slab.h>
#include <asm/unaligned.h>
#include <asm/byteorder.h>
#include <linux/fs.h>
#include <linux/timer.h>
#include <linux/poll.h>
+#include <linux/gfp.h>
#include "av7110.h"
#include "av7110_hw.h"
#include <linux/errno.h>
#include <linux/version.h> /* for KERNEL_VERSION MACRO */
#include <linux/io.h>
+#include <linux/slab.h>
#include <media/v4l2-device.h>
#include <media/v4l2-ioctl.h>
#include <linux/pci.h>
#include <linux/videodev2.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <media/v4l2-device.h>
#include <media/v4l2-ioctl.h>
#include <linux/videodev2.h>
#include <linux/version.h> /* for KERNEL_VERSION MACRO */
#include <linux/io.h>
+#include <linux/slab.h>
#include <media/v4l2-device.h>
#include <media/v4l2-ioctl.h>
#include <linux/platform_device.h>
#include <linux/i2c.h>
#include <linux/videodev2.h>
+#include <linux/slab.h>
#include <media/v4l2-device.h>
#include <media/v4l2-common.h>
#include <media/v4l2-ioctl.h>
* add RDS support
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/init.h> /* Initdata */
#include <linux/videodev2.h> /* kernel radio structs */
#include <media/v4l2-device.h>
#include <linux/platform_device.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <linux/i2c.h>
#include <media/timb_radio.h>
#include <linux/kernel.h>
#include <linux/interrupt.h>
#include <linux/i2c.h>
+#include <linux/slab.h>
#include <media/v4l2-device.h>
#include <media/v4l2-chip-ident.h>
/* kernel includes */
#include <linux/i2c.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
/* kernel includes */
#include <linux/usb.h>
#include <linux/hid.h>
+#include <linux/slab.h>
#include "radio-si470x.h"
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/i2c.h>
+#include <linux/slab.h>
#include <media/v4l2-device.h>
#include <media/v4l2-ioctl.h>
#include <media/v4l2-common.h>
#include <linux/interrupt.h>
#include <linux/i2c.h>
#include <linux/i2c-id.h>
+#include <linux/slab.h>
#include <media/v4l2-ioctl.h>
#include <media/v4l2-device.h>
#include <media/v4l2-chip-ident.h>
#include <linux/module.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/ioctl.h>
#include <asm/uaccess.h>
#include <linux/i2c.h>
#include <linux/module.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/ioctl.h>
#include <asm/uaccess.h>
#include <linux/i2c.h>
#include <linux/interrupt.h>
#include <linux/i2c.h>
#include <linux/i2c-id.h>
+#include <linux/slab.h>
#include <media/v4l2-ioctl.h>
#include <linux/videodev2.h>
#include <media/v4l2-device.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/ctype.h>
+#include <linux/slab.h>
#include <linux/i2c.h>
#include <linux/device.h>
#include <linux/delay.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/videodev2.h>
#include <media/v4l2-common.h>
#include <linux/mutex.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/device.h>
#include <linux/suspend.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/device.h>
#include <linux/suspend.h>
#include <linux/i2c.h>
#include <linux/i2c-id.h>
#include <linux/videodev2.h>
+#include <linux/slab.h>
#include <media/v4l2-device.h>
#include <media/v4l2-chip-ident.h>
#include <media/v4l2-i2c-drv.h>
#include <linux/module.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/ioctl.h>
#include <asm/uaccess.h>
#include <linux/i2c.h>
#include <linux/module.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/ioctl.h>
#include <asm/uaccess.h>
#include <linux/i2c.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/fs.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/delay.h>
#include <linux/device.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include "bttvp.h"
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/input.h>
+#include <linux/slab.h>
#include "bttv.h"
#include "bttvp.h"
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/vmalloc.h>
#include <linux/interrupt.h>
#include <linux/interrupt.h>
#include <linux/spinlock.h>
#include <linux/videodev2.h>
+#include <linux/slab.h>
#include <media/v4l2-device.h>
#include <media/v4l2-ioctl.h>
#include <media/v4l2-chip-ident.h>
#include <linux/delay.h>
#include <linux/workqueue.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/kmod.h>
#include <linux/kernel.h>
#include <linux/i2c.h>
#include <linux/videodev2.h>
+#include <linux/slab.h>
#include <media/v4l2-device.h>
#include <media/v4l2-chip-ident.h>
#include <media/v4l2-i2c-drv.h>
#include <linux/module.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/ioctl.h>
#include <asm/uaccess.h>
#include <linux/i2c.h>
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/device.h>
* 02111-1307 USA
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include "cx18-driver.h"
#include "cx18-cards.h"
#include <linux/pagemap.h>
#include <linux/workqueue.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <asm/byteorder.h>
#include <linux/dvb/video.h>
#include <linux/init.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/i2c.h>
#include <linux/usb.h>
#include <linux/init.h>
#include <linux/list.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include <linux/vmalloc.h>
#include <media/v4l2-common.h>
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include "cx231xx.h"
#include <linux/interrupt.h>
#include <linux/input.h>
#include <linux/usb.h>
+#include <linux/slab.h>
#include "cx231xx.h"
#include <linux/i2c.h>
#include <linux/mm.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <media/v4l2-common.h>
#include <media/v4l2-ioctl.h>
#include <linux/version.h>
#include <linux/mm.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <media/v4l2-common.h>
#include <media/v4l2-ioctl.h>
#include <linux/device.h>
#include <linux/firmware.h>
#include <linux/smp_lock.h>
+#include <linux/slab.h>
#include <media/v4l2-common.h>
#include <media/v4l2-ioctl.h>
#include <media/cx2341x.h>
*/
#include <linux/input.h>
+#include <linux/slab.h>
#include <media/ir-common.h>
#include <media/v4l2-subdev.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include "cx23885.h"
#include <linux/i2c.h>
#include <linux/i2c-algo-bit.h>
#include <linux/kdev_t.h>
+#include <linux/slab.h>
#include <media/v4l2-device.h>
#include <media/tuner.h>
*/
#include <linux/kfifo.h>
+#include <linux/slab.h>
#include <media/v4l2-device.h>
#include <media/v4l2-chip-ident.h>
#include <linux/vmalloc.h>
#include <linux/dma-mapping.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <asm/delay.h>
#include <sound/core.h>
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/fs.h>
#include <linux/delay.h>
#include <linux/device.h>
#include <linux/module.h>
#include <linux/pci.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include "cx88.h"
#include "tea5767.h"
* Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/jiffies.h>
#include <linux/hrtimer.h>
#include <linux/input.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include "cx88.h"
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/device.h>
#include <linux/dma-mapping.h>
#include <linux/errno.h>
#include <linux/freezer.h>
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/poll.h>
#include <linux/signal.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include "cx88.h"
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <asm/io.h>
#include <linux/platform_device.h>
#include <linux/uaccess.h>
#include <linux/videodev2.h>
+#include <linux/gfp.h>
#include <linux/clk.h>
#include <linux/err.h>
* - Support for control ioctls
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/platform_device.h>
#include <linux/interrupt.h>
#include <linux/platform_device.h>
#include <linux/io.h>
#include <linux/version.h>
+#include <linux/slab.h>
#include <media/v4l2-device.h>
#include <media/v4l2-ioctl.h>
#include <linux/platform_device.h>
#include <linux/io.h>
#include <linux/version.h>
+#include <linux/slab.h>
#include <asm/irq.h>
#include <asm/page.h>
#include <linux/init.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/i2c.h>
#include <linux/usb.h>
#include <linux/init.h>
#include <linux/list.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include <linux/vmalloc.h>
#include <media/v4l2-common.h>
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include "em28xx.h"
#include <linux/interrupt.h>
#include <linux/input.h>
#include <linux/usb.h>
+#include <linux/slab.h>
#include "em28xx.h"
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include "em28xx.h"
#include <linux/version.h>
#include <linux/mm.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include "em28xx.h"
#include <media/v4l2-common.h>
#include <linux/videodev2.h>
#include <media/v4l2-common.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
/* compilation option */
#define GSPCA_DEBUG 1
#define MODULE_NAME "jeilinj"
#include <linux/workqueue.h>
+#include <linux/slab.h>
#include "gspca.h"
#include "jpeg.h"
*/
#include <linux/kthread.h>
+#include <linux/slab.h>
#include "m5602_s5k83a.h"
static int s5k83a_set_gain(struct gspca_dev *gspca_dev, __s32 val);
#include <linux/freezer.h>
#include <linux/usb/input.h>
#include <linux/input.h>
+#include <linux/slab.h>
#endif
#include "gspca.h"
#define MODULE_NAME "sonixj"
#include <linux/input.h>
+#include <linux/slab.h>
#include "gspca.h"
#include "jpeg.h"
#define MODULE_NAME "sq905"
#include <linux/workqueue.h>
+#include <linux/slab.h>
#include "gspca.h"
MODULE_AUTHOR("Adam Baker <linux@baker-net.org.uk>, "
#define MODULE_NAME "sq905c"
#include <linux/workqueue.h>
+#include <linux/slab.h>
#include "gspca.h"
MODULE_AUTHOR("Theodore Kilgore <kilgota@auburn.edu>");
#define MODULE_NAME "zc3xx"
#include <linux/input.h>
+#include <linux/slab.h>
#include "gspca.h"
#include "jpeg.h"
*/
#include <linux/i2c.h>
+#include <linux/slab.h>
#include "hdpvr.h"
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include "ivtv-driver.h"
#include "ivtv-cards.h"
#include <linux/scatterlist.h>
#include <linux/workqueue.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/system.h>
#include <asm/byteorder.h>
#include <linux/kernel.h>
#include <linux/fb.h>
#include <linux/ivtvfb.h>
+#include <linux/slab.h>
#ifdef CONFIG_MTRR
#include <asm/mtrr.h>
#include <linux/kernel.h>
#include <linux/i2c.h>
#include <linux/videodev2.h>
+#include <linux/slab.h>
#include <media/v4l2-device.h>
#include <media/v4l2-chip-ident.h>
#include <media/v4l2-i2c-drv.h>
#include <linux/module.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/ioctl.h>
#include <asm/uaccess.h>
#include <linux/i2c.h>
#include <linux/sched.h>
#include <linux/init.h>
#include <linux/videodev.h>
+#include <linux/gfp.h>
#include <media/v4l2-common.h>
#include <media/v4l2-ioctl.h>
#include <asm/uaccess.h>
#include <linux/kernel.h>
#include <linux/module.h>
-#include <linux/slab.h>
#include <linux/i2c.h>
#include <linux/freezer.h>
#include <linux/videodev2.h>
*/
#include <linux/i2c.h>
+#include <linux/slab.h>
#include <linux/videodev2.h>
#include <linux/delay.h>
#include <asm/div64.h>
#include <linux/mutex.h>
#include <linux/platform_device.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/time.h>
#include <linux/version.h>
#include <linux/videodev2.h>
#include <linux/platform_device.h>
#include <linux/clk.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <media/v4l2-common.h>
#include <media/v4l2-ioctl.h>
*/
#include <linux/init.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/i2c.h>
#include <linux/delay.h>
#include <linux/videodev2.h>
#include <linux/errno.h>
#include <linux/fs.h>
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/ioport.h>
#include <linux/init.h>
#include <linux/videodev2.h>
#include <media/v4l2-common.h>
#include <linux/errno.h>
-#include <linux/slab.h>
struct routing_scheme {
const int *def;
#include <linux/videodev2.h>
#include <media/v4l2-common.h>
#include <linux/errno.h>
-#include <linux/slab.h>
struct routing_scheme_item {
*/
#include <linux/string.h>
-#include <linux/slab.h>
#include "pvrusb2-debugifc.h"
#include "pvrusb2-hdw.h"
#include "pvrusb2-debug.h"
#include <linux/kthread.h>
#include <linux/freezer.h>
+#include <linux/slab.h>
#include <linux/mm.h>
#include "dvbdev.h"
#include "pvrusb2-debug.h"
*
*/
+#include <linux/slab.h>
#include "pvrusb2-eeprom.h"
#include "pvrusb2-hdw-internal.h"
#include "pvrusb2-debug.h"
#include <linux/kernel.h>
#include <linux/errno.h>
-#include <linux/slab.h>
#include <linux/module.h>
#include <linux/usb.h>
#include <linux/videodev2.h>
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/version.h>
#include "pvrusb2-context.h"
#include "pvrusb2-hdw.h"
#include <media/v4l2-common.h>
#include <media/saa7115.h>
#include <linux/errno.h>
-#include <linux/slab.h>
struct routing_scheme {
const int *def;
#include <linux/videodev2.h>
#include <media/v4l2-common.h>
#include <linux/errno.h>
-#include <linux/slab.h>
void pvr2_wm8775_subdev_update(struct pvr2_hdw *hdw, struct v4l2_subdev *sd)
{
#include <media/pwc-ioctl.h>
#include <linux/string.h>
+#include <linux/slab.h>
/*
* USE_LOOKUP_TABLE_TO_CLAMP
#include <linux/mm.h>
#include <linux/module.h>
#include <linux/poll.h>
-#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <asm/io.h>
#include <linux/version.h>
#include <linux/mutex.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include <asm/errno.h>
#include <linux/videodev.h>
#include <media/v4l2-common.h>
#include <linux/platform_device.h>
#include <linux/clk.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <media/v4l2-common.h>
#include <media/v4l2-dev.h>
#include <linux/firmware.h>
#include <linux/kernel.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <linux/videodev2.h>
#include <linux/version.h>
#include <linux/mm.h>
#include <linux/mm.h>
#include <linux/init.h>
#include <linux/i2c.h>
+#include <linux/slab.h>
#include <linux/mutex.h>
#include <linux/videotext.h>
#include <linux/videodev2.h>
#include <linux/delay.h>
#include <linux/videotext.h>
#include <linux/videodev2.h>
+#include <linux/slab.h>
#include <media/v4l2-device.h>
#include <media/v4l2-chip-ident.h>
#include <media/v4l2-ioctl.h>
#include <linux/list.h>
#include <linux/module.h>
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/kthread.h>
#include <linux/suspend.h>
#include <linux/list.h>
#include <linux/module.h>
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/smp_lock.h>
#include <linux/delay.h>
#include <linux/list.h>
#include <linux/module.h>
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include "saa7134-reg.h"
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/input.h>
+#include <linux/slab.h>
#include "saa7134-reg.h"
#include "saa7134.h"
#include <linux/list.h>
#include <linux/module.h>
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include "saa7134-reg.h"
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/kthread.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/freezer.h>
#include <asm/div64.h>
#include <linux/list.h>
#include <linux/module.h>
#include <linux/kernel.h>
-#include <linux/slab.h>
#include "saa7134-reg.h"
#include "saa7134.h"
*/
#include <linux/wait.h>
+#include <linux/slab.h>
#include "saa7164.h"
* Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
+#include <linux/slab.h>
+
#include "saa7164.h"
/* The PCI address space for buffer handling looks like this:
*/
#include <linux/firmware.h>
+#include <linux/slab.h>
#include "saa7164.h"
#include <linux/module.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#include <linux/videodev2.h>
#include <linux/module.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/ioctl.h>
#include <asm/uaccess.h>
#include <linux/i2c.h>
#include <linux/moduleparam.h>
#include <linux/time.h>
#include <linux/version.h>
+#include <linux/slab.h>
#include <linux/device.h>
#include <linux/platform_device.h>
#include <linux/videodev2.h>
#include <linux/mutex.h>
#include <linux/module.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <media/soc_camera.h>
#include <linux/module.h>
#include <linux/ioctl.h>
+#include <linux/slab.h>
#include <linux/i2c.h>
#include <media/v4l2-device.h>
#include <media/v4l2-chip-ident.h>
#include <linux/module.h>
#include <linux/ioctl.h>
+#include <linux/slab.h>
#include <linux/i2c.h>
#include <media/v4l2-device.h>
#include <media/v4l2-chip-ident.h>
#include <linux/module.h>
#include <linux/ioctl.h>
+#include <linux/slab.h>
#include <linux/i2c.h>
#include <media/v4l2-device.h>
#include <media/v4l2-chip-ident.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/ctype.h>
+#include <linux/slab.h>
#include <linux/i2c.h>
#include <linux/device.h>
#include <linux/delay.h>
#include <linux/sound.h>
#include <linux/spinlock.h>
#include <linux/soundcard.h>
-#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/proc_fs.h>
#include <linux/module.h>
+#include <linux/gfp.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/usb.h>
#include <linux/dvb/dmx.h>
#include <linux/delay.h>
+#include <linux/gfp.h>
#include "vendorcmds.h"
#include <linux/sched.h>
#include <linux/usb.h>
#include <linux/mm.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <media/v4l2-ioctl.h>
#include <media/v4l2-dev.h>
#include <linux/module.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/ioctl.h>
#include <asm/uaccess.h>
#include <linux/i2c.h>
*/
#include <linux/i2c.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/videodev2.h>
*/
#include <linux/i2c.h>
+#include <linux/slab.h>
#include <linux/videodev2.h>
#include <linux/delay.h>
#include <media/v4l2-device.h>
*/
#include <linux/delay.h>
#include <linux/i2c.h>
+#include <linux/slab.h>
#include <linux/videodev2.h>
#include <media/tvp7002.h>
#include <media/v4l2-device.h>
#include <linux/kernel.h>
#include <linux/i2c.h>
#include <linux/videodev2.h>
+#include <linux/slab.h>
#include <media/v4l2-device.h>
#include <media/v4l2-chip-ident.h>
#include <media/v4l2-i2c-drv.h>
#include <linux/kernel.h>
#include <linux/i2c.h>
#include <linux/videodev2.h>
+#include <linux/slab.h>
#include <media/v4l2-device.h>
#include <media/v4l2-chip-ident.h>
#include <media/v4l2-i2c-drv.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/usb/input.h>
+#include <linux/gfp.h>
#include "usbvideo.h"
#include <linux/init.h>
#include <linux/input.h>
#include <linux/usb/input.h>
+#include <linux/slab.h>
#include "usbvideo.h"
#include "quickcam_messenger.h"
#include <linux/kernel.h>
#include <linux/list.h>
#include <linux/timer.h>
-#include <linux/slab.h>
+#include <linux/gfp.h>
#include <linux/mm.h>
#include <linux/highmem.h>
#include <linux/vmalloc.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/delay.h>
-#include <linux/slab.h>
#include <linux/init.h>
#include <asm/uaccess.h>
#include <linux/ioport.h>
#include <linux/kernel.h>
#include <linux/list.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/uaccess.h>
#include <linux/usb.h>
#include <linux/videodev2.h>
#include <linux/kernel.h>
#include <linux/list.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include <linux/videodev2.h>
#include <linux/vmalloc.h>
#include <linux/kernel.h>
#include <linux/input.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include <linux/usb/input.h>
#include <linux/version.h>
#include <linux/list.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include <linux/videodev2.h>
#include <linux/vmalloc.h>
#include <linux/kernel.h>
#include <linux/list.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include <linux/videodev2.h>
#include <linux/vmalloc.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/pagemap.h>
#include <linux/dma-mapping.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <media/videobuf-dma-contig.h>
struct videobuf_dma_contig_memory {
#include <linux/fs.h>
#include <linux/kthread.h>
#include <linux/file.h>
+#include <linux/slab.h>
#include <linux/freezer.h>
#include <linux/fs.h>
#include <linux/interrupt.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/time.h>
#include <linux/version.h>
#include <linux/module.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/ioctl.h>
#include <asm/uaccess.h>
#include <linux/i2c.h>
#include <linux/init.h>
#include <linux/delay.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <linux/i2c.h>
#include <linux/videodev2.h>
#include <linux/init.h>
#include <linux/delay.h>
#include <linux/videodev.h>
+#include <linux/slab.h>
#include <media/v4l2-common.h>
#include <media/v4l2-ioctl.h>
#include <linux/parport.h>
#include <linux/module.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/ioctl.h>
#include <asm/uaccess.h>
#include <linux/i2c.h>
#include <linux/module.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/ioctl.h>
#include <asm/uaccess.h>
#include <linux/i2c.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/vmalloc.h>
+#include <linux/slab.h>
#include <linux/proc_fs.h>
#include <linux/i2c.h>
#include <linux/idr.h>
#include <linux/fs.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#define DRIVER_NAME "memstick"
#include <linux/hdreg.h>
#include <linux/kthread.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/memstick.h>
#define DRIVER_NAME "mspro_block"
#include <linux/delay.h>
#include <linux/highmem.h>
#include <linux/memstick.h>
+#include <linux/slab.h>
#define DRIVER_NAME "jmb38x_ms"
#include <linux/reboot.h> /* notifier code */
#include <linux/workqueue.h>
#include <linux/sort.h>
+#include <linux/slab.h>
#include <scsi/scsi.h>
#include <scsi/scsi_cmnd.h>
#include <linux/module.h>
#include <linux/fs.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#define my_VERSION MPT_LINUX_VERSION_COMMON
#define MYNAM "mptlan"
#include <linux/module.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/errno.h>
#include <linux/jiffies.h>
#include <linux/module.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/errno.h>
#include <linux/kdev_t.h>
#include <linux/module.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/errno.h>
#include <linux/kdev_t.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/i2o.h>
#include <linux/mempool.h>
#include <linux/miscdevice.h>
#include <linux/smp_lock.h>
#include <linux/compat.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <linux/kernel.h>
#include <linux/pci.h>
#include <linux/i2o.h>
+#include <linux/slab.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
#include <linux/init.h>
#include <linux/i2o.h>
#include <linux/delay.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include "core.h"
#define OSM_NAME "i2o"
#include <linux/pci.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <linux/i2o.h>
#include "core.h"
#include <linux/platform_device.h>
#include <linux/i2c.h>
#include <linux/mfd/88pm860x.h>
+#include <linux/slab.h>
static inline int pm860x_read_device(struct i2c_client *i2c,
int reg, int bytes, void *dest)
#include <linux/mutex.h>
#include <linux/list.h>
#include <linux/notifier.h>
+#include <linux/slab.h>
#include <linux/err.h>
#include <linux/platform_device.h>
#include <linux/device.h>
#include <linux/module.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/platform_device.h>
#include <linux/mfd/ab3100.h>
* Interrupt management to be added - TODO.
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/platform_device.h>
#include <linux/module.h>
#include <linux/platform_device.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/irq.h>
#include <linux/err.h>
#include <linux/irq.h>
#include <linux/gpio.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/platform_device.h>
#include <linux/platform_device.h>
#include <linux/i2c.h>
#include <linux/mfd/da903x.h>
+#include <linux/slab.h>
#define DA9030_CHIP_ID 0x00
#define DA9030_EVENT_A 0x01
#include <linux/mfd/ezx-pcap.h>
#include <linux/spi/spi.h>
#include <linux/gpio.h>
+#include <linux/slab.h>
#define PCAP_ADC_MAXQ 8
struct pcap_adc_request {
#include <linux/io.h>
#include <linux/spinlock.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/mfd/htc-egpio.h>
#include <linux/spinlock.h>
#include <linux/htcpld.h>
#include <linux/gpio.h>
+#include <linux/slab.h>
struct htcpld_chip {
spinlock_t lock;
#include <linux/mfd/core.h>
#include <linux/mfd/ds1wm.h>
#include <linux/mfd/htc-pasic3.h>
+#include <linux/slab.h>
struct pasic3_data {
void __iomem *mapping;
#include <linux/platform_device.h>
#include <linux/i2c.h>
#include <linux/mfd/max8925.h>
+#include <linux/slab.h>
#define RTC_I2C_ADDR 0x68
#define ADC_I2C_ADDR 0x47
* the terms of the GNU General Public License version 2 as published by the
* Free Software Foundation.
*/
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/spi/spi.h>
#include <linux/mfd/core.h>
#include <linux/kernel.h>
#include <linux/delay.h>
#include <linux/spinlock.h>
-#include <linux/slab.h>
#include <linux/platform_device.h>
#include <linux/mfd/mcp.h>
#include <linux/delay.h>
#include <linux/rtc.h>
#include <linux/bcd.h>
+#include <linux/slab.h>
#include <asm/mach/irq.h>
#include <linux/platform_device.h>
#include <linux/acpi.h>
#include <linux/mfd/core.h>
+#include <linux/slab.h>
static int mfd_add_device(struct device *parent, int id,
const struct mfd_cell *cell,
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/device.h>
#include <linux/platform_device.h>
#include <linux/i2c.h>
#include <linux/irq.h>
+#include <linux/slab.h>
#include <linux/mfd/pcf50633/core.h>
#include <linux/kernel.h>
#include <linux/clk.h>
+#include <linux/slab.h>
#include <linux/platform_device.h>
#include <linux/mmc/host.h>
#include <linux/mfd/core.h>
#include <linux/platform_device.h>
#include <linux/pci.h>
#include <linux/i2c-gpio.h>
+#include <linux/slab.h>
#include <linux/sm501.h>
#include <linux/sm501-regs.h>
#include <linux/module.h>
#include <linux/err.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <linux/irq.h>
#include <linux/clk.h>
#include <linux/platform_device.h>
#include <linux/mfd/core.h>
#include <linux/mfd/tmio.h>
#include <linux/mfd/tc6387xb.h>
+#include <linux/slab.h>
enum {
TC6387XB_CELL_MMC,
#include <linux/mfd/tmio.h>
#include <linux/mfd/tc6393xb.h>
#include <linux/gpio.h>
+#include <linux/slab.h>
#define SCR_REVID 0x08 /* b Revision ID */
#define SCR_ISR 0x50 /* b Interrupt Status */
#include <linux/pci.h>
#include <linux/msi.h>
#include <linux/mfd/core.h>
+#include <linux/slab.h>
#include <linux/timb_gpio.h>
#include <linux/module.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/fs.h>
#include <linux/platform_device.h>
#include <linux/interrupt.h>
#include <linux/irq.h>
#include <linux/kthread.h>
+#include <linux/slab.h>
#include <linux/i2c/twl.h>
#include <linux/module.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/ucb1400.h>
unsigned int ucb1400_adc_read(struct snd_ac97 *ac97, u16 adc_channel,
#include <linux/bcd.h>
#include <linux/delay.h>
#include <linux/mfd/core.h>
+#include <linux/slab.h>
#include <linux/mfd/wm831x/core.h>
#include <linux/mfd/wm831x/pdata.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/bug.h>
#include <linux/device.h>
#include <linux/delay.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
#include <linux/mfd/wm8350/core.h>
+#include <linux/slab.h>
static int wm8350_i2c_read_device(struct wm8350 *wm8350, char reg,
int bytes, void *dest)
#include <linux/mfd/core.h>
#include <linux/mfd/wm8400-private.h>
#include <linux/mfd/wm8400-audio.h>
+#include <linux/slab.h>
static struct {
u16 readable; /* Mask of readable bits */
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/i2c.h>
#include <linux/delay.h>
#include <linux/mfd/core.h>
#include <linux/io.h>
#include <linux/spinlock.h>
#include <linux/atmel-ssc.h>
+#include <linux/slab.h>
/* Serialize access to ssc_list and user count */
static DEFINE_SPINLOCK(user_lock);
#include <linux/module.h>
#include <linux/clk.h>
#include <linux/err.h>
+#include <linux/slab.h>
#include <linux/io.h>
#include <linux/interrupt.h>
#include <linux/platform_device.h>
#include <linux/ioport.h>
#include <linux/kernel.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
/* Number of bytes to reserve for the iomem resource */
#define ATMEL_TC_IOMEM_SIZE 256
#include <linux/delay.h>
#include <linux/idr.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/c2port.h>
c2dev->dev = device_create(c2port_class, NULL, 0, c2dev,
"c2port%d", id);
- if (unlikely(!c2dev->dev)) {
- ret = -ENOMEM;
+ if (unlikely(IS_ERR(c2dev->dev))) {
+ ret = PTR_ERR(c2dev->dev);
goto error_device_create;
}
dev_set_drvdata(c2dev->dev, c2dev);
*/
#include <linux/kernel.h>
#include <linux/module.h>
-#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/spinlock.h>
#include <linux/idr.h>
#include <linux/cb710.h>
+#include <linux/gfp.h>
static DEFINE_IDA(cb710_ida);
static DEFINE_SPINLOCK(cb710_ida_lock);
#include <linux/cb710.h>
#include <linux/kernel.h>
#include <linux/module.h>
-#include <linux/slab.h>
#define CB710_REG_COUNT 0x80
#include <linux/module.h>
#include <linux/pci.h>
#include <linux/cs5535.h>
+#include <linux/slab.h>
#define DRV_NAME "cs5535-mfgpt"
#define MFGPT_BAR 2
#include <linux/module.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/i2c.h>
#include <linux/string.h>
#include <linux/list.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
static LIST_HEAD(container_list);
static DEFINE_MUTEX(container_list_lock);
#include <linux/module.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/clk.h>
#include <linux/err.h>
#include <linux/io.h>
#include <linux/io.h>
#include <linux/wait.h>
#include <linux/poll.h>
+#include <linux/slab.h>
#include "hpilo.h"
static struct class *ilo_class;
*/
#include <linux/sched.h>
+#include <linux/slab.h>
#include "ibmasm.h"
#include "lowlevel.h"
*/
#include <linux/sched.h>
+#include <linux/slab.h>
#include "ibmasm.h"
#include "lowlevel.h"
#include <linux/fs.h>
#include <linux/pagemap.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/io.h>
#include "ibmasm.h"
#include <linux/pci.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include "ibmasm.h"
#include "lowlevel.h"
#include "remote.h"
#include <linux/mutex.h>
#include <linux/delay.h>
#include <linux/log2.h>
+#include <linux/slab.h>
/* Addresses to scan */
static const unsigned short normal_i2c[] = { 0x69, I2C_CLIENT_END };
#include <linux/pci.h>
#include <linux/ioc4.h>
#include <linux/ktime.h>
+#include <linux/slab.h>
#include <linux/mutex.h>
#include <linux/time.h>
#include <asm/io.h>
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include <linux/ctype.h>
#include <linux/mmc/sdio_func.h>
#include <linux/firmware.h>
#include <linux/mmc/sdio_func.h>
+#include <linux/slab.h>
#include <asm/unaligned.h>
#include "iwmc3200top.h"
#include <linux/kernel.h>
#include <linux/mmc/sdio_func.h>
+#include <linux/slab.h>
#include <linux/ctype.h>
#include "fw-msg.h"
#include "iwmc3200top.h"
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/debugfs.h>
/* On x86 a breakpoint stop requires it to be decremented */
if (addr + 1 == kgdbts_regs.ip)
offset = -1;
+#elif defined(CONFIG_SUPERH)
+ /* On SUPERH a breakpoint stop requires it to be decremented */
+ if (addr + 2 == kgdbts_regs.pc)
+ offset = -2;
#endif
if (strcmp(arg, "silent") &&
instruction_pointer(&kgdbts_regs) + offset != addr) {
#ifdef CONFIG_X86
/* On x86 adjust the instruction pointer if needed */
kgdbts_regs.ip += offset;
+#elif defined(CONFIG_SUPERH)
+ kgdbts_regs.pc += offset;
#endif
return 0;
}
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/hrtimer.h>
+#include <linux/slab.h>
#include <scsi/scsi_cmnd.h>
#include <linux/debugfs.h>
#include <linux/poll.h>
#include <linux/interrupt.h>
#include <linux/cdev.h>
+#include <linux/slab.h>
#include <linux/phantom.h>
#include <linux/sched.h>
#include <linux/smp_lock.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/sysctl.h>
#include <linux/device.h>
#include <linux/delay.h>
#include <linux/device.h>
#include <linux/hardirq.h>
+#include <linux/slab.h>
#include "xpc.h"
#include <asm/uv/uv_hub.h>
*/
#include <linux/delay.h>
+#include <linux/slab.h>
#include <asm/uncached.h>
#include <asm/sn/mspec.h>
#include <asm/sn/sn_sal.h>
#include <linux/delay.h>
#include <linux/device.h>
#include <linux/err.h>
+#include <linux/slab.h>
#include <asm/uv/uv_hub.h>
#if defined CONFIG_X86_64
#include <asm/uv/bios.h>
*
*/
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/netdevice.h>
#include <linux/etherdevice.h>
*/
#include <linux/tifm.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/idr.h>
#include <linux/kernel.h>
#include <linux/fs.h>
+#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/hdreg.h>
#include <linux/kdev_t.h>
#include <linux/mmc/card.h>
#include <linux/mmc/host.h>
#include <linux/mmc/mmc.h>
+#include <linux/slab.h>
#include <linux/scatterlist.h>
* published by the Free Software Foundation.
*
*/
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/blkdev.h>
#include <linux/freezer.h>
#include <linux/seq_file.h>
#include <linux/serial_reg.h>
#include <linux/circ_buf.h>
-#include <linux/gfp.h>
#include <linux/tty.h>
#include <linux/tty_flip.h>
#include <linux/kfifo.h>
+#include <linux/slab.h>
#include <linux/mmc/core.h>
#include <linux/mmc/card.h>
#include <linux/device.h>
#include <linux/err.h>
+#include <linux/slab.h>
#include <linux/mmc/card.h>
#include <linux/mmc/host.h>
#include <linux/debugfs.h>
#include <linux/fs.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include <linux/stat.h>
#include <linux/mmc/card.h>
#include <linux/idr.h>
#include <linux/pagemap.h>
#include <linux/leds.h>
+#include <linux/slab.h>
#include <linux/mmc/host.h>
*/
#include <linux/err.h>
+#include <linux/slab.h>
#include <linux/mmc/host.h>
#include <linux/mmc/card.h>
mmc_card_set_blockaddr(card);
}
- switch (ext_csd[EXT_CSD_CARD_TYPE]) {
+ switch (ext_csd[EXT_CSD_CARD_TYPE] & EXT_CSD_CARD_TYPE_MASK) {
case EXT_CSD_CARD_TYPE_52 | EXT_CSD_CARD_TYPE_26:
card->ext_csd.hs_max_dtr = 52000000;
break;
printk(KERN_WARNING "%s: card is mmc v4 but doesn't "
"support any high-speed modes.\n",
mmc_hostname(card->host));
- goto out;
}
if (card->ext_csd.rev >= 3) {
* your option) any later version.
*/
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/scatterlist.h>
*/
#include <linux/err.h>
+#include <linux/slab.h>
#include <linux/mmc/host.h>
#include <linux/mmc/card.h>
#include <linux/device.h>
#include <linux/err.h>
+#include <linux/slab.h>
#include <linux/mmc/card.h>
#include <linux/mmc/sdio_func.h>
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/mmc/host.h>
#include <linux/mmc/card.h>
#include <linux/dma-mapping.h>
#include <linux/clk.h>
#include <linux/atmel_pdc.h>
+#include <linux/gfp.h>
#include <linux/mmc/host.h>
#include <linux/platform_device.h>
#include <linux/scatterlist.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include <linux/stat.h>
#include <linux/mmc/host.h>
#include <linux/scatterlist.h>
#include <linux/leds.h>
#include <linux/mmc/host.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/mach-au1x00/au1000.h>
#include <linux/dma-mapping.h>
#include <linux/mmc/host.h>
#include <linux/proc_fs.h>
+#include <linux/gfp.h>
#include <asm/cacheflush.h>
#include <asm/dma.h>
*/
#include <linux/kernel.h>
#include <linux/module.h>
-#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/delay.h>
#include "cb710-mmc.h"
*/
#include <linux/sched.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/bio.h>
#include <linux/dma-mapping.h>
#include <linux/crc7.h>
#include <linux/debugfs.h>
#include <linux/io.h>
#include <linux/memory.h>
+#include <linux/gfp.h>
#include <asm/cacheflush.h>
#include <asm/div64.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/device.h>
+#include <linux/slab.h>
#include <linux/gpio.h>
#include <linux/of.h>
#include <linux/of_gpio.h>
#include <linux/clk.h>
#include <linux/scatterlist.h>
#include <linux/i2c/tps65010.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/irq.h>
#include <linux/io.h>
#include <linux/regulator/consumer.h>
#include <linux/gpio.h>
+#include <linux/gfp.h>
#include <asm/sizes.h>
#include <linux/highmem.h>
#include <linux/pci.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <linux/mmc/host.h>
#include <linux/delay.h>
#include <linux/dma-mapping.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/clk.h>
#include <linux/io.h>
#include <linux/highmem.h>
#include <linux/io.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <linux/scatterlist.h>
#include <linux/leds.h>
#include <linux/highmem.h>
#include <linux/mmc/host.h>
#include <linux/scatterlist.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/dma.h>
#include <linux/buffer_head.h>
#include <linux/mutex.h>
#include <linux/mount.h>
+#include <linux/slab.h>
#define ERROR(fmt, args...) printk(KERN_ERR "block2mtd: " fmt "\n" , ## args)
#define INFO(fmt, args...) printk(KERN_INFO "block2mtd: " fmt "\n" , ## args)
#include <linux/interrupt.h>
#include <linux/mutex.h>
#include <linux/math64.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#include <linux/mod_devicetable.h>
#include <linux/device.h>
#include <linux/mutex.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#include <linux/mtd/mtd.h>
*/
#include <linux/mtd/pfow.h>
#include <linux/mtd/qinfo.h>
+#include <linux/slab.h>
static int lpddr_read(struct mtd_info *mtd, loff_t adr, size_t len,
size_t *retlen, u_char *buf);
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <linux/mtd/mtd.h>
#include <linux/mtd/map.h>
#include <linux/mtd/partitions.h>
#include <linux/mtd/physmap.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <asm/blackfin.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <linux/mtd/mtd.h>
#include <linux/mtd/map.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <linux/mtd/mtd.h>
#include <linux/mtd/map.h>
#include <linux/mtd/partitions.h>
#include <linux/mtd/physmap.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/types.h>
#define pr_devinit(fmt, args...) ({ static const __devinitconst char __fmt[] = fmt; printk(__fmt, ## args); })
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <linux/mtd/mtd.h>
#include <linux/mtd/map.h>
#include <linux/module.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/init.h>
#include <linux/mtd/mtd.h>
##################################################################### */
#include <linux/module.h>
-#include <linux/slab.h>
#include <linux/ioport.h>
#include <linux/init.h>
#include <asm/io.h>
#include <linux/mtd/concat.h>
#include <linux/of.h>
#include <linux/of_platform.h>
+#include <linux/slab.h>
struct of_flash_list {
struct mtd_info *mtd;
#include <linux/init.h>
#include <linux/module.h>
#include <linux/i2c.h>
+#include <linux/slab.h>
#include <linux/platform_device.h>
#include <linux/spinlock.h>
#include <linux/mutex.h>
* 675 Mass Ave, Cambridge, MA 02139, USA.
*/
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/platform_device.h>
// Includes
#include <linux/module.h>
-#include <linux/slab.h>
#include <linux/ioport.h>
#include <linux/init.h>
#include <asm/io.h>
#include <linux/ioport.h>
#include <linux/of.h>
#include <linux/of_device.h>
+#include <linux/slab.h>
#include <asm/prom.h>
#include <asm/uaccess.h>
#include <asm/io.h>
##################################################################### */
#include <linux/module.h>
-#include <linux/slab.h>
#include <linux/ioport.h>
#include <linux/init.h>
#include <linux/spinlock.h>
* GNU General Public Licence
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#include <linux/delay.h>
#include <linux/maple.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/timer.h>
#include <linux/major.h>
#include <linux/types.h>
#include <linux/init.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include <linux/ioport.h>
#include <linux/device.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <asm/io.h>
#define CAFE_NAND_CTRL1 0x00
#include <linux/mtd/nand.h>
#include <linux/mtd/partitions.h>
+#include <linux/slab.h>
#include <linux/gpio.h>
#include <asm/io.h>
#include <linux/io.h>
#include <linux/mtd/nand.h>
#include <linux/mtd/partitions.h>
+#include <linux/slab.h>
#include <mach/nand.h>
#include <linux/delay.h>
#include <linux/rslib.h>
#include <linux/moduleparam.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <linux/mtd/mtd.h>
#include <linux/of_platform.h>
#include <linux/of_gpio.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <asm/fsl_lbc.h>
#define FSL_UPM_WAIT_RUN_PATTERN 0x1
#include <linux/mtd/nand_ecc.h>
#include <linux/mtd/partitions.h>
#include <linux/mtd/ndfc.h>
+#include <linux/slab.h>
#include <linux/mtd/mtd.h>
#include <linux/of_platform.h>
#include <asm/io.h>
#include <linux/platform_device.h>
#include <linux/mtd/partitions.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <mach/nand.h>
#include <mach/fsmc.h>
#include <linux/mtd/nand.h>
#include <linux/mtd/partitions.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <plat/dma.h>
#include <plat/gpmc.h>
#include <linux/mtd/partitions.h>
#include <linux/io.h>
#include <linux/irq.h>
+#include <linux/slab.h>
#include <mach/dma.h>
#include <plat/pxa3xx_nand.h>
#include <linux/delay.h>
#include <linux/io.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/mtd/mtd.h>
#include <linux/mtd/nand.h>
#include <linux/mtd/nand.h>
#include <linux/mtd/nand_ecc.h>
#include <linux/mtd/partitions.h>
+#include <linux/slab.h>
/*--------------------------------------------------------------------------*/
#include <linux/init.h>
#include <linux/of.h>
#include <linux/mtd/mtd.h>
+#include <linux/slab.h>
#include <linux/mtd/partitions.h>
int __devinit of_mtd_parse_partitions(struct device *dev,
#include <linux/delay.h>
#include <linux/dma-mapping.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <asm/mach/flash.h>
#include <plat/gpmc.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/sched.h>
#include <linux/delay.h>
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/vmalloc.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/list.h>
-#include <linux/slab.h>
#include <linux/random.h>
#include <linux/string.h>
#include <linux/bitops.h>
#include <linux/moduleparam.h>
#include <linux/err.h>
#include <linux/mtd/mtd.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#define PRINT_PREF KERN_INFO "mtd_oobtest: "
#include <linux/moduleparam.h>
#include <linux/err.h>
#include <linux/mtd/mtd.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#define PRINT_PREF KERN_INFO "mtd_pagetest: "
#include <linux/moduleparam.h>
#include <linux/err.h>
#include <linux/mtd/mtd.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#define PRINT_PREF KERN_INFO "mtd_readtest: "
#include <linux/moduleparam.h>
#include <linux/err.h>
#include <linux/mtd/mtd.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#define PRINT_PREF KERN_INFO "mtd_speedtest: "
#include <linux/moduleparam.h>
#include <linux/err.h>
#include <linux/mtd/mtd.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#include <linux/vmalloc.h>
#include <linux/moduleparam.h>
#include <linux/err.h>
#include <linux/mtd/mtd.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#define PRINT_PREF KERN_INFO "mtd_subpagetest: "
#include <linux/moduleparam.h>
#include <linux/err.h>
#include <linux/mtd/mtd.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#define PRINT_PREF KERN_INFO "mtd_torturetest: "
#include <linux/kthread.h>
#include <linux/reboot.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include "ubi.h"
/* Maximum length of the 'mtd=' parameter */
#include <linux/module.h>
#include <linux/stat.h>
+#include <linux/slab.h>
#include <linux/ioctl.h>
#include <linux/capability.h>
#include <linux/uaccess.h>
#include <linux/err.h>
#include <linux/list.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#include <linux/math64.h>
#include <linux/module.h>
#include <linux/crc32.h>
#include <linux/err.h>
+#include <linux/slab.h>
#include "ubi.h"
#ifdef CONFIG_MTD_UBI_DEBUG_PARANOID
#include <linux/module.h>
#include <linux/err.h>
+#include <linux/slab.h>
#include <linux/namei.h>
#include <linux/fs.h>
#include <asm/div64.h>
*/
#include <linux/err.h>
+#include <linux/slab.h>
#include <linux/crc32.h>
#include <linux/math64.h>
#include "ubi.h"
#include <linux/fs.h>
#include <linux/cdev.h>
#include <linux/device.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include <linux/vmalloc.h>
#include <linux/notifier.h>
#include <linux/err.h>
#include <linux/math64.h>
+#include <linux/slab.h>
#include "ubi.h"
#ifdef CONFIG_MTD_UBI_DEBUG_PARANOID
#include <linux/crc32.h>
#include <linux/err.h>
+#include <linux/slab.h>
#include <asm/div64.h>
#include "ubi.h"
#include <linux/fcntl.h>
#include <linux/ioport.h>
#include <linux/interrupt.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/spinlock.h>
#include <linux/interrupt.h>
#include <linux/errno.h>
#include <linux/in.h>
-#include <linux/slab.h>
#include <linux/ioport.h>
#include <linux/spinlock.h>
#include <linux/ethtool.h>
#include <linux/delay.h>
#include <linux/bitops.h>
+#include <linux/gfp.h>
#include <asm/uaccess.h>
#include <asm/io.h>
#include <linux/etherdevice.h>
#include <linux/if_ether.h>
#include <linux/skbuff.h>
-#include <linux/slab.h>
#include <linux/init.h>
#include <linux/bitops.h>
#include <linux/interrupt.h>
#include <linux/errno.h>
#include <linux/in.h>
-#include <linux/slab.h>
#include <linux/ioport.h>
#include <linux/init.h>
#include <linux/netdevice.h>
#include <linux/errno.h>
#include <linux/in.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/skbuff.h>
#include <linux/etherdevice.h>
#include <linux/interrupt.h>
#include <linux/errno.h>
#include <linux/ioport.h>
#include <linux/skbuff.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/delay.h>
#include <linux/mca-legacy.h>
#include <linux/errno.h>
#include <linux/in.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
#include <linux/mii.h>
#include <linux/eisa.h>
#include <linux/bitops.h>
#include <linux/jiffies.h>
+#include <linux/gfp.h>
#include <asm/irq.h> /* For nr_irqs only. */
#include <asm/io.h>
#include <asm/uaccess.h>
#include <linux/ioport.h>
#include <linux/in.h>
#include <linux/route.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/skbuff.h>
#include <asm/irq.h>
#include <linux/dma-mapping.h>
#include <linux/delay.h>
#include <linux/ethtool.h>
+#include <linux/gfp.h>
#include <linux/mii.h>
#include <linux/if_vlan.h>
#include <linux/crc32.h>
#include <linux/crc32.h>
#include <linux/io.h>
#include <linux/uaccess.h>
+#include <linux/gfp.h>
#include <asm/irq.h>
#define RTL8139_DRIVER_NAME DRV_NAME " Fast Ethernet driver " DRV_VERSION
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/delay.h>
#include <linux/netdevice.h>
#include <linux/skbuff.h>
#include <linux/init.h>
#include <linux/bitops.h>
+#include <linux/gfp.h>
#include <asm/io.h>
#include <asm/dma.h>
To compile this driver as a module, choose M here: the module
will be called cxgb3.
+config CHELSIO_T4_DEPENDS
+ tristate
+ depends on PCI && INET
+ default y
+
+config CHELSIO_T4
+ tristate "Chelsio Communications T4 Ethernet support"
+ depends on CHELSIO_T4_DEPENDS
+ select FW_LOADER
+ select MDIO
+ help
+ This driver supports Chelsio T4-based gigabit and 10Gb Ethernet
+ adapters.
+
+ For general information about Chelsio and our products, visit
+ our website at <http://www.chelsio.com>.
+
+ For customer support, please visit our customer support page at
+ <http://www.chelsio.com/support.htm>.
+
+ Please send feedback to <linux-bugs@chelsio.com>.
+
+ To compile this driver as a module choose M here; the module
+ will be called cxgb4.
+
config EHEA
tristate "eHEA Ethernet support"
depends on IBMEBUS && INET && SPARSEMEM
obj-$(CONFIG_IP1000) += ipg.o
obj-$(CONFIG_CHELSIO_T1) += chelsio/
obj-$(CONFIG_CHELSIO_T3) += cxgb3/
+obj-$(CONFIG_CHELSIO_T4) += cxgb4/
obj-$(CONFIG_EHEA) += ehea/
obj-$(CONFIG_CAN) += can/
obj-$(CONFIG_BONDING) += bonding/
#include <linux/interrupt.h>
#include <linux/ioport.h>
#include <linux/skbuff.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/init.h>
#include <linux/crc32.h>
#include <linux/highmem.h>
#include <linux/sockios.h>
#include <linux/firmware.h>
+#include <linux/slab.h>
#if defined(CONFIG_VLAN_8021Q) || defined(CONFIG_VLAN_8021Q_MODULE)
#include <linux/if_vlan.h>
#include <linux/kernel.h>
#include <linux/types.h>
#include <linux/compiler.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/ioport.h>
#include <linux/ptrace.h>
#include <linux/ioport.h>
#include <linux/in.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/init.h>
#include <linux/ip.h>
#include <linux/atalk.h>
#include <linux/if_arp.h>
+#include <linux/slab.h>
#include <net/route.h>
#include <asm/uaccess.h>
#include <linux/ioport.h>
#include <linux/spinlock.h>
#include <linux/in.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/init.h>
#include <linux/timer.h>
#include <linux/atalk.h>
#include <linux/bitops.h>
+#include <linux/gfp.h>
#include <asm/system.h>
#include <asm/dma.h>
*/
#include <linux/module.h>
+#include <linux/gfp.h>
#include <linux/init.h>
#include <linux/if_arp.h>
#include <net/arp.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/netdevice.h>
#include <linux/bootmem.h>
*/
#include <linux/module.h>
+#include <linux/gfp.h>
#include <linux/init.h>
#include <linux/if_arp.h>
#include <net/arp.h>
#include <linux/kernel.h>
#include <linux/types.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/delay.h>
#include <linux/netdevice.h>
#include <linux/kernel.h>
#include <linux/types.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/netdevice.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/types.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/delay.h>
#include <linux/netdevice.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/netdevice.h>
#include <linux/bootmem.h>
#include <linux/ioport.h>
#include <linux/delay.h>
#include <linux/netdevice.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <linux/arcdevice.h>
* **********************
*/
#include <linux/module.h>
+#include <linux/gfp.h>
#include <linux/init.h>
#include <linux/if_arp.h>
#include <net/arp.h>
*
* **********************
*/
+#include <linux/gfp.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/if_arp.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/netdevice.h>
#include <linux/etherdevice.h>
#include <linux/interrupt.h>
#include <linux/ethtool.h>
#include <linux/platform_device.h>
#include <linux/clk.h>
+#include <linux/gfp.h>
#include <asm/io.h>
#include <asm/uaccess.h>
#include <linux/platform_device.h>
#include <linux/delay.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <mach/hardware.h>
#include <linux/interrupt.h>
#include <linux/ioport.h>
#include <linux/in.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/netdevice.h>
#include <linux/kernel.h>
#include <linux/phy.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <mach/npe.h>
#include <mach/qmgr.h>
#include <linux/platform_device.h>
#include <linux/irq.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <asm/irq.h>
}
/**
- * ks8695_rx - Receive packets called by NAPI poll method
+ * ks8695_rx - Receive packets called by NAPI poll method
* @ksp: Private data for the KS8695 Ethernet
- * @budget: The max packets would be receive
+ * @budget: Number of packets allowed to process
*/
-
static int ks8695_rx(struct ks8695_priv *ksp, int budget)
{
struct net_device *ndev = ksp->ndev;
int buff_n;
u32 flags;
int pktlen;
- int last_rx_processed = -1;
int received = 0;
buff_n = ksp->next_rx_desc_read;
cpu_to_le32(RDES_OWN)))) {
rmb();
flags = le32_to_cpu(ksp->rx_ring[buff_n].status);
+
/* Found an SKB which we own, this means we
* received a packet
*/
ksp->rx_ring[buff_n].status = cpu_to_le32(RDES_OWN);
rx_finished:
received++;
- /* And note this as processed so we can start
- * from here next time
- */
- last_rx_processed = buff_n;
buff_n = (buff_n + 1) & MAX_RX_DESC_MASK;
- /*And note which RX descriptor we last did */
- if (likely(last_rx_processed != -1))
- ksp->next_rx_desc_read =
- (last_rx_processed + 1) &
- MAX_RX_DESC_MASK;
}
+
+ /* And note which RX descriptor we last did */
+ ksp->next_rx_desc_read = buff_n;
+
/* And refill the buffers */
ks8695_refill_rxbuffers(ksp);
- /* Kick the RX DMA engine, in case it became
- * suspended */
+ /* Kick the RX DMA engine, in case it became suspended */
ks8695_writereg(ksp, KS8695_DRSC, 0);
+
return received;
}
#include <linux/ethtool.h>
#include <linux/platform_device.h>
#include <linux/clk.h>
+#include <linux/gfp.h>
#define DRV_MODULE_NAME "w90p910-emc"
#define DRV_MODULE_VERSION "0.1"
#include <linux/ioport.h>
#include <linux/in.h>
#include <linux/skbuff.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/init.h>
#include <linux/crc32.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/skbuff.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/init.h>
#include <linux/bitops.h>
#include <linux/netdevice.h>
#include <linux/ethtool.h>
+#include <linux/slab.h>
#include "atl1c.h"
#include <linux/netdevice.h>
#include <linux/ethtool.h>
+#include <linux/slab.h>
#include "atl1e.h"
#define ATLX_DRIVER_VERSION "2.1.3"
MODULE_AUTHOR("Xiong Huang <xiong.huang@atheros.com>, \
- Chris Snook <csnook@redhat.com>, Jay Cliburn <jcliburn@gmail.com>");
+Chris Snook <csnook@redhat.com>, Jay Cliburn <jcliburn@gmail.com>");
MODULE_LICENSE("GPL");
MODULE_VERSION(ATLX_DRIVER_VERSION);
#include <linux/pci_ids.h>
#include <linux/pm.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/string.h>
#include <linux/tcp.h>
#include <linux/interrupt.h>
#include <linux/ioport.h>
#include <linux/in.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/init.h>
#include <linux/ethtool.h>
#include <linux/mii.h>
#include <linux/eeprom_93cx6.h>
+#include <linux/slab.h>
#include <net/ax88796.h>
#include <linux/init.h>
#include <linux/dma-mapping.h>
#include <linux/ssb/ssb.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/io.h>
#include <linux/module.h>
#include <linux/clk.h>
#include <linux/etherdevice.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/ethtool.h>
#include <linux/crc32.h>
#include <linux/workqueue.h>
#include <linux/interrupt.h>
#include <linux/firmware.h>
+#include <linux/slab.h>
#include "be_hw.h"
req->params.op_type = cpu_to_le32(IMG_TYPE_REDBOOT);
req->params.op_code = cpu_to_le32(FLASHROM_OPER_REPORT);
- req->params.offset = offset;
- req->params.data_buf_size = 0x4;
+ req->params.offset = cpu_to_le32(offset);
+ req->params.data_buf_size = cpu_to_le32(0x4);
status = be_mcc_notify_wait(adapter);
if (!status)
{
int ret, i;
struct be_dma_mem ddrdma_cmd;
- u64 pattern[2] = {0x5a5a5a5a5a5a5a5a, 0xa5a5a5a5a5a5a5a5};
+ u64 pattern[2] = {0x5a5a5a5a5a5a5a5aULL, 0xa5a5a5a5a5a5a5a5ULL};
ddrdma_cmd.size = sizeof(struct be_cmd_req_ddrdma_test);
ddrdma_cmd.va = pci_alloc_consistent(adapter->pdev, ddrdma_cmd.size,
return;
}
vid = AMAP_GET_BITS(struct amap_eth_rx_compl, vlan_tag, rxcp);
- vid = be16_to_cpu(vid);
+ vid = swab16(vid);
vlan_hwaccel_receive_skb(skb, adapter->vlan_grp, vid);
} else {
netif_receive_skb(skb);
napi_gro_frags(&eq_obj->napi);
} else {
vid = AMAP_GET_BITS(struct amap_eth_rx_compl, vlan_tag, rxcp);
- vid = be16_to_cpu(vid);
+ vid = swab16(vid);
if (!adapter->vlan_grp || adapter->vlans_added == 0)
return;
p += crc_offset;
status = be_cmd_get_flash_crc(adapter, flashed_crc,
- (img_start + image_size - 4));
+ (image_size - 4));
if (status) {
dev_err(&adapter->pdev->dev,
"could not get crc from flash, not flashing redboot\n");
struct flash_file_hdr_g3 *fhdr3;
struct image_hdr *img_hdr_ptr = NULL;
struct be_dma_mem flash_cmd;
- int status, i = 0;
+ int status, i = 0, num_imgs = 0;
const u8 *p;
strcpy(fw_file, func);
if ((adapter->generation == BE_GEN3) &&
(get_ufigen_type(fhdr) == BE_GEN3)) {
fhdr3 = (struct flash_file_hdr_g3 *) fw->data;
- for (i = 0; i < fhdr3->num_imgs; i++) {
+ num_imgs = le32_to_cpu(fhdr3->num_imgs);
+ for (i = 0; i < num_imgs; i++) {
img_hdr_ptr = (struct image_hdr *) (fw->data +
(sizeof(struct flash_file_hdr_g3) +
- i * sizeof(struct image_hdr)));
- if (img_hdr_ptr->imageid == 1) {
- status = be_flash_data(adapter, fw,
- &flash_cmd, fhdr3->num_imgs);
- }
-
+ i * sizeof(struct image_hdr)));
+ if (le32_to_cpu(img_hdr_ptr->imageid) == 1)
+ status = be_flash_data(adapter, fw, &flash_cmd,
+ num_imgs);
}
} else if ((adapter->generation == BE_GEN2) &&
(get_ufigen_type(fhdr) == BE_GEN2)) {
#include <linux/crc32.h>
#include <linux/bitrev.h>
#include <linux/ethtool.h>
+#include <linux/slab.h>
#include <asm/prom.h>
#include <asm/dbdma.h>
#include <asm/io.h>
MODULE_DEVICE_TABLE(pci, bnx2_pci_tbl);
+static void bnx2_init_napi(struct bnx2 *bp);
+
static inline u32 bnx2_tx_avail(struct bnx2 *bp, struct bnx2_tx_ring_info *txr)
{
u32 diff;
bnx2_disable_int(bp);
bnx2_setup_int_mode(bp, disable_msi);
+ bnx2_init_napi(bp);
bnx2_napi_enable(bp);
rc = bnx2_alloc_mem(bp);
if (rc)
int i;
for (i = 0; i < bp->irq_nvecs; i++) {
- disable_irq(bp->irq_tbl[i].vector);
- bnx2_interrupt(bp->irq_tbl[i].vector, &bp->bnx2_napi[i]);
- enable_irq(bp->irq_tbl[i].vector);
+ struct bnx2_irq *irq = &bp->irq_tbl[i];
+
+ disable_irq(irq->vector);
+ irq->handler(irq->vector, &bp->bnx2_napi[i]);
+ enable_irq(irq->vector);
}
}
#endif
{
int i;
- for (i = 0; i < BNX2_MAX_MSIX_VEC; i++) {
+ for (i = 0; i < bp->irq_nvecs; i++) {
struct bnx2_napi *bnapi = &bp->bnx2_napi[i];
int (*poll)(struct napi_struct *, int);
dev->ethtool_ops = &bnx2_ethtool_ops;
bp = netdev_priv(dev);
- bnx2_init_napi(bp);
pci_set_drvdata(pdev, dev);
write_lock_bh(&bond->curr_slave_lock);
}
}
+
+ /* resend IGMP joins since all were sent on curr_active_slave */
+ if (bond->params.mode == BOND_MODE_ROUNDROBIN) {
+ bond_resend_igmp_join_requests(bond);
+ }
}
/**
struct bonding *bond = netdev_priv(bond_dev);
struct slave *slave, *start_at;
int i, slave_no, res = 1;
+ struct iphdr *iph = ip_hdr(skb);
read_lock(&bond->lock);
if (!BOND_IS_OK(bond))
goto out;
-
/*
- * Concurrent TX may collide on rr_tx_counter; we accept that
- * as being rare enough not to justify using an atomic op here
+ * Start with the curr_active_slave that joined the bond as the
+ * default for sending IGMP traffic. For failover purposes one
+ * needs to maintain some consistency for the interface that will
+ * send the join/membership reports. The curr_active_slave found
+ * will send all of this type of traffic.
*/
- slave_no = bond->rr_tx_counter++ % bond->slave_cnt;
+ if ((iph->protocol == IPPROTO_IGMP) &&
+ (skb->protocol == htons(ETH_P_IP))) {
- bond_for_each_slave(bond, slave, i) {
- slave_no--;
- if (slave_no < 0)
- break;
+ read_lock(&bond->curr_slave_lock);
+ slave = bond->curr_active_slave;
+ read_unlock(&bond->curr_slave_lock);
+
+ if (!slave)
+ goto out;
+ } else {
+ /*
+ * Concurrent TX may collide on rr_tx_counter; we accept
+ * that as being rare enough not to justify using an
+ * atomic op here.
+ */
+ slave_no = bond->rr_tx_counter++ % bond->slave_cnt;
+
+ bond_for_each_slave(bond, slave, i) {
+ slave_no--;
+ if (slave_no < 0)
+ break;
+ }
}
start_at = slave;
.ndo_vlan_rx_kill_vid = bond_vlan_rx_kill_vid,
};
+static void bond_destructor(struct net_device *bond_dev)
+{
+ struct bonding *bond = netdev_priv(bond_dev);
+ if (bond->wq)
+ destroy_workqueue(bond->wq);
+ free_netdev(bond_dev);
+}
+
static void bond_setup(struct net_device *bond_dev)
{
struct bonding *bond = netdev_priv(bond_dev);
bond_dev->ethtool_ops = &bond_ethtool_ops;
bond_set_mode_ops(bond, bond->params.mode);
- bond_dev->destructor = free_netdev;
+ bond_dev->destructor = bond_destructor;
/* Initialize the device options */
bond_dev->tx_queue_len = 0;
bond_remove_proc_entry(bond);
- if (bond->wq)
- destroy_workqueue(bond->wq);
-
netif_addr_lock_bh(bond_dev);
bond_mc_list_destroy(bond);
netif_addr_unlock_bh(bond_dev);
bond_setup);
if (!bond_dev) {
pr_err("%s: eek! can't alloc netdev!\n", name);
- res = -ENOMEM;
- goto out;
+ rtnl_unlock();
+ return -ENOMEM;
}
dev_net_set(bond_dev, net);
if (!name) {
res = dev_alloc_name(bond_dev, "bond%d");
if (res < 0)
- goto out_netdev;
+ goto out;
}
res = register_netdevice(bond_dev);
- if (res < 0)
- goto out_netdev;
out:
rtnl_unlock();
+ if (res < 0)
+ bond_destructor(bond_dev);
return res;
-out_netdev:
- free_netdev(bond_dev);
- goto out;
}
static int __net_init bond_net_init(struct net *net)
#include <linux/can/dev.h>
#include <linux/can/error.h>
+#include <asm/bfin_can.h>
#include <asm/portmux.h>
#define DRV_NAME "bfin_can"
#define BFIN_CAN_TIMEOUT 100
#define TX_ECHO_SKB_MAX 1
-/*
- * transmit and receive channels
- */
-#define TRANSMIT_CHL 24
-#define RECEIVE_STD_CHL 0
-#define RECEIVE_EXT_CHL 4
-#define RECEIVE_RTR_CHL 8
-#define RECEIVE_EXT_RTR_CHL 12
-#define MAX_CHL_NUMBER 32
-
-/*
- * bfin can registers layout
- */
-struct bfin_can_mask_regs {
- u16 aml;
- u16 dummy1;
- u16 amh;
- u16 dummy2;
-};
-
-struct bfin_can_channel_regs {
- u16 data[8];
- u16 dlc;
- u16 dummy1;
- u16 tsv;
- u16 dummy2;
- u16 id0;
- u16 dummy3;
- u16 id1;
- u16 dummy4;
-};
-
-struct bfin_can_regs {
- /*
- * global control and status registers
- */
- u16 mc1; /* offset 0 */
- u16 dummy1;
- u16 md1; /* offset 4 */
- u16 rsv1[13];
- u16 mbtif1; /* offset 0x20 */
- u16 dummy2;
- u16 mbrif1; /* offset 0x24 */
- u16 dummy3;
- u16 mbim1; /* offset 0x28 */
- u16 rsv2[11];
- u16 mc2; /* offset 0x40 */
- u16 dummy4;
- u16 md2; /* offset 0x44 */
- u16 dummy5;
- u16 trs2; /* offset 0x48 */
- u16 rsv3[11];
- u16 mbtif2; /* offset 0x60 */
- u16 dummy6;
- u16 mbrif2; /* offset 0x64 */
- u16 dummy7;
- u16 mbim2; /* offset 0x68 */
- u16 rsv4[11];
- u16 clk; /* offset 0x80 */
- u16 dummy8;
- u16 timing; /* offset 0x84 */
- u16 rsv5[3];
- u16 status; /* offset 0x8c */
- u16 dummy9;
- u16 cec; /* offset 0x90 */
- u16 dummy10;
- u16 gis; /* offset 0x94 */
- u16 dummy11;
- u16 gim; /* offset 0x98 */
- u16 rsv6[3];
- u16 ctrl; /* offset 0xa0 */
- u16 dummy12;
- u16 intr; /* offset 0xa4 */
- u16 rsv7[7];
- u16 esr; /* offset 0xb4 */
- u16 rsv8[37];
-
- /*
- * channel(mailbox) mask and message registers
- */
- struct bfin_can_mask_regs msk[MAX_CHL_NUMBER]; /* offset 0x100 */
- struct bfin_can_channel_regs chl[MAX_CHL_NUMBER]; /* offset 0x200 */
-};
-
/*
* bfin can private data
*/
if (priv->can.ctrlmode & CAN_CTRLMODE_3_SAMPLES)
timing |= SAM;
- bfin_write16(®->clk, clk);
+ bfin_write16(®->clock, clk);
bfin_write16(®->timing, timing);
dev_info(dev->dev.parent, "setting CLOCK=0x%04x TIMING=0x%04x\n",
bfin_write16(®->gim, 0);
/* reset can and enter configuration mode */
- bfin_write16(®->ctrl, SRS | CCR);
+ bfin_write16(®->control, SRS | CCR);
SSYNC();
- bfin_write16(®->ctrl, CCR);
+ bfin_write16(®->control, CCR);
SSYNC();
- while (!(bfin_read16(®->ctrl) & CCA)) {
+ while (!(bfin_read16(®->control) & CCA)) {
udelay(10);
if (--timeout == 0) {
dev_err(dev->dev.parent,
/*
* leave configuration mode
*/
- bfin_write16(®->ctrl, bfin_read16(®->ctrl) & ~CCR);
+ bfin_write16(®->control, bfin_read16(®->control) & ~CCR);
while (bfin_read16(®->status) & CCA) {
udelay(10);
if (netif_running(dev)) {
/* enter sleep mode */
- bfin_write16(®->ctrl, bfin_read16(®->ctrl) | SMR);
+ bfin_write16(®->control, bfin_read16(®->control) | SMR);
SSYNC();
while (!(bfin_read16(®->intr) & SMACK)) {
udelay(10);
#include <linux/module.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/netdevice.h>
#include <linux/if_arp.h>
#include <linux/can.h>
#include <linux/module.h>
#include <linux/netdevice.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/spi/spi.h>
#include <linux/uaccess.h>
#include <linux/interrupt.h>
#include <linux/netdevice.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/can.h>
#include <linux/can/dev.h>
#include <linux/interrupt.h>
#include <linux/netdevice.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/can.h>
#include <linux/can/dev.h>
#include <linux/if_ether.h>
#include <linux/can.h>
#include <linux/can/dev.h>
+#include <linux/slab.h>
#include <net/rtnetlink.h>
static __initdata const char banner[] =
#include <linux/mdio.h>
#include <linux/crc32.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <linux/pci_ids.h>
#include "suni1x10gexp_regs.h"
#include <linux/crc32.h>
+#include <linux/slab.h>
#define OFFSET(REG_ADDR) ((REG_ADDR) << 2)
#include <linux/ip.h>
#include <linux/in.h>
#include <linux/if_arp.h>
+#include <linux/slab.h>
#include "cpl5_cmd.h"
#include "sge.h"
#include <linux/ptrace.h>
#include <linux/ioport.h>
#include <linux/in.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/spinlock.h>
#include <linux/errno.h>
#include <linux/ioport.h>
#include <linux/in.h>
#include <linux/skbuff.h>
-#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/string.h>
#include <linux/init.h>
#include <linux/bitops.h>
#include <linux/delay.h>
+#include <linux/gfp.h>
#include <asm/system.h>
#include <asm/io.h>
#include <linux/log2.h>
#include <linux/stringify.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include "common.h"
*/
#include <linux/list.h>
+#include <linux/slab.h>
#include <net/neighbour.h>
#include <linux/notifier.h>
#include <asm/atomic.h>
#include <linux/if.h>
#include <linux/if_vlan.h>
#include <linux/jhash.h>
+#include <linux/slab.h>
#include <net/neighbour.h>
#include "common.h"
#include "t3cdev.h"
#include <linux/ip.h>
#include <linux/tcp.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <net/arp.h>
#include "common.h"
#include "regs.h"
--- /dev/null
+#
+# Chelsio T4 driver
+#
+
+obj-$(CONFIG_CHELSIO_T4) += cxgb4.o
+
+cxgb4-objs := cxgb4_main.o l2t.o t4_hw.o sge.o
--- /dev/null
+/*
+ * This file is part of the Chelsio T4 Ethernet driver for Linux.
+ *
+ * Copyright (c) 2003-2010 Chelsio Communications, Inc. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef __CXGB4_H__
+#define __CXGB4_H__
+
+#include <linux/bitops.h>
+#include <linux/cache.h>
+#include <linux/interrupt.h>
+#include <linux/list.h>
+#include <linux/netdevice.h>
+#include <linux/pci.h>
+#include <linux/spinlock.h>
+#include <linux/timer.h>
+#include <asm/io.h>
+#include "cxgb4_uld.h"
+#include "t4_hw.h"
+
+#define FW_VERSION_MAJOR 1
+#define FW_VERSION_MINOR 1
+#define FW_VERSION_MICRO 0
+
+enum {
+ MAX_NPORTS = 4, /* max # of ports */
+ SERNUM_LEN = 16, /* Serial # length */
+ EC_LEN = 16, /* E/C length */
+ ID_LEN = 16, /* ID length */
+};
+
+enum {
+ MEM_EDC0,
+ MEM_EDC1,
+ MEM_MC
+};
+
+enum dev_master {
+ MASTER_CANT,
+ MASTER_MAY,
+ MASTER_MUST
+};
+
+enum dev_state {
+ DEV_STATE_UNINIT,
+ DEV_STATE_INIT,
+ DEV_STATE_ERR
+};
+
+enum {
+ PAUSE_RX = 1 << 0,
+ PAUSE_TX = 1 << 1,
+ PAUSE_AUTONEG = 1 << 2
+};
+
+struct port_stats {
+ u64 tx_octets; /* total # of octets in good frames */
+ u64 tx_frames; /* all good frames */
+ u64 tx_bcast_frames; /* all broadcast frames */
+ u64 tx_mcast_frames; /* all multicast frames */
+ u64 tx_ucast_frames; /* all unicast frames */
+ u64 tx_error_frames; /* all error frames */
+
+ u64 tx_frames_64; /* # of Tx frames in a particular range */
+ u64 tx_frames_65_127;
+ u64 tx_frames_128_255;
+ u64 tx_frames_256_511;
+ u64 tx_frames_512_1023;
+ u64 tx_frames_1024_1518;
+ u64 tx_frames_1519_max;
+
+ u64 tx_drop; /* # of dropped Tx frames */
+ u64 tx_pause; /* # of transmitted pause frames */
+ u64 tx_ppp0; /* # of transmitted PPP prio 0 frames */
+ u64 tx_ppp1; /* # of transmitted PPP prio 1 frames */
+ u64 tx_ppp2; /* # of transmitted PPP prio 2 frames */
+ u64 tx_ppp3; /* # of transmitted PPP prio 3 frames */
+ u64 tx_ppp4; /* # of transmitted PPP prio 4 frames */
+ u64 tx_ppp5; /* # of transmitted PPP prio 5 frames */
+ u64 tx_ppp6; /* # of transmitted PPP prio 6 frames */
+ u64 tx_ppp7; /* # of transmitted PPP prio 7 frames */
+
+ u64 rx_octets; /* total # of octets in good frames */
+ u64 rx_frames; /* all good frames */
+ u64 rx_bcast_frames; /* all broadcast frames */
+ u64 rx_mcast_frames; /* all multicast frames */
+ u64 rx_ucast_frames; /* all unicast frames */
+ u64 rx_too_long; /* # of frames exceeding MTU */
+ u64 rx_jabber; /* # of jabber frames */
+ u64 rx_fcs_err; /* # of received frames with bad FCS */
+ u64 rx_len_err; /* # of received frames with length error */
+ u64 rx_symbol_err; /* symbol errors */
+ u64 rx_runt; /* # of short frames */
+
+ u64 rx_frames_64; /* # of Rx frames in a particular range */
+ u64 rx_frames_65_127;
+ u64 rx_frames_128_255;
+ u64 rx_frames_256_511;
+ u64 rx_frames_512_1023;
+ u64 rx_frames_1024_1518;
+ u64 rx_frames_1519_max;
+
+ u64 rx_pause; /* # of received pause frames */
+ u64 rx_ppp0; /* # of received PPP prio 0 frames */
+ u64 rx_ppp1; /* # of received PPP prio 1 frames */
+ u64 rx_ppp2; /* # of received PPP prio 2 frames */
+ u64 rx_ppp3; /* # of received PPP prio 3 frames */
+ u64 rx_ppp4; /* # of received PPP prio 4 frames */
+ u64 rx_ppp5; /* # of received PPP prio 5 frames */
+ u64 rx_ppp6; /* # of received PPP prio 6 frames */
+ u64 rx_ppp7; /* # of received PPP prio 7 frames */
+
+ u64 rx_ovflow0; /* drops due to buffer-group 0 overflows */
+ u64 rx_ovflow1; /* drops due to buffer-group 1 overflows */
+ u64 rx_ovflow2; /* drops due to buffer-group 2 overflows */
+ u64 rx_ovflow3; /* drops due to buffer-group 3 overflows */
+ u64 rx_trunc0; /* buffer-group 0 truncated packets */
+ u64 rx_trunc1; /* buffer-group 1 truncated packets */
+ u64 rx_trunc2; /* buffer-group 2 truncated packets */
+ u64 rx_trunc3; /* buffer-group 3 truncated packets */
+};
+
+struct lb_port_stats {
+ u64 octets;
+ u64 frames;
+ u64 bcast_frames;
+ u64 mcast_frames;
+ u64 ucast_frames;
+ u64 error_frames;
+
+ u64 frames_64;
+ u64 frames_65_127;
+ u64 frames_128_255;
+ u64 frames_256_511;
+ u64 frames_512_1023;
+ u64 frames_1024_1518;
+ u64 frames_1519_max;
+
+ u64 drop;
+
+ u64 ovflow0;
+ u64 ovflow1;
+ u64 ovflow2;
+ u64 ovflow3;
+ u64 trunc0;
+ u64 trunc1;
+ u64 trunc2;
+ u64 trunc3;
+};
+
+struct tp_tcp_stats {
+ u32 tcpOutRsts;
+ u64 tcpInSegs;
+ u64 tcpOutSegs;
+ u64 tcpRetransSegs;
+};
+
+struct tp_err_stats {
+ u32 macInErrs[4];
+ u32 hdrInErrs[4];
+ u32 tcpInErrs[4];
+ u32 tnlCongDrops[4];
+ u32 ofldChanDrops[4];
+ u32 tnlTxDrops[4];
+ u32 ofldVlanDrops[4];
+ u32 tcp6InErrs[4];
+ u32 ofldNoNeigh;
+ u32 ofldCongDefer;
+};
+
+struct tp_params {
+ unsigned int ntxchan; /* # of Tx channels */
+ unsigned int tre; /* log2 of core clocks per TP tick */
+};
+
+struct vpd_params {
+ unsigned int cclk;
+ u8 ec[EC_LEN + 1];
+ u8 sn[SERNUM_LEN + 1];
+ u8 id[ID_LEN + 1];
+};
+
+struct pci_params {
+ unsigned char speed;
+ unsigned char width;
+};
+
+struct adapter_params {
+ struct tp_params tp;
+ struct vpd_params vpd;
+ struct pci_params pci;
+
+ unsigned int fw_vers;
+ unsigned int tp_vers;
+ u8 api_vers[7];
+
+ unsigned short mtus[NMTUS];
+ unsigned short a_wnd[NCCTRL_WIN];
+ unsigned short b_wnd[NCCTRL_WIN];
+
+ unsigned char nports; /* # of ethernet ports */
+ unsigned char portvec;
+ unsigned char rev; /* chip revision */
+ unsigned char offload;
+
+ unsigned int ofldq_wr_cred;
+};
+
+struct trace_params {
+ u32 data[TRACE_LEN / 4];
+ u32 mask[TRACE_LEN / 4];
+ unsigned short snap_len;
+ unsigned short min_len;
+ unsigned char skip_ofst;
+ unsigned char skip_len;
+ unsigned char invert;
+ unsigned char port;
+};
+
+struct link_config {
+ unsigned short supported; /* link capabilities */
+ unsigned short advertising; /* advertised capabilities */
+ unsigned short requested_speed; /* speed user has requested */
+ unsigned short speed; /* actual link speed */
+ unsigned char requested_fc; /* flow control user has requested */
+ unsigned char fc; /* actual link flow control */
+ unsigned char autoneg; /* autonegotiating? */
+ unsigned char link_ok; /* link up? */
+};
+
+#define FW_LEN16(fw_struct) FW_CMD_LEN16(sizeof(fw_struct) / 16)
+
+enum {
+ MAX_ETH_QSETS = 32, /* # of Ethernet Tx/Rx queue sets */
+ MAX_OFLD_QSETS = 16, /* # of offload Tx/Rx queue sets */
+ MAX_CTRL_QUEUES = NCHAN, /* # of control Tx queues */
+ MAX_RDMA_QUEUES = NCHAN, /* # of streaming RDMA Rx queues */
+};
+
+enum {
+ MAX_EGRQ = 128, /* max # of egress queues, including FLs */
+ MAX_INGQ = 64 /* max # of interrupt-capable ingress queues */
+};
+
+struct adapter;
+struct vlan_group;
+struct sge_rspq;
+
+struct port_info {
+ struct adapter *adapter;
+ struct vlan_group *vlan_grp;
+ u16 viid;
+ s16 xact_addr_filt; /* index of exact MAC address filter */
+ u16 rss_size; /* size of VI's RSS table slice */
+ s8 mdio_addr;
+ u8 port_type;
+ u8 mod_type;
+ u8 port_id;
+ u8 tx_chan;
+ u8 lport; /* associated offload logical port */
+ u8 rx_offload; /* CSO, etc */
+ u8 nqsets; /* # of qsets */
+ u8 first_qset; /* index of first qset */
+ struct link_config link_cfg;
+};
+
+/* port_info.rx_offload flags */
+enum {
+ RX_CSO = 1 << 0,
+};
+
+struct dentry;
+struct work_struct;
+
+enum { /* adapter flags */
+ FULL_INIT_DONE = (1 << 0),
+ USING_MSI = (1 << 1),
+ USING_MSIX = (1 << 2),
+ QUEUES_BOUND = (1 << 3),
+ FW_OK = (1 << 4),
+};
+
+struct rx_sw_desc;
+
+struct sge_fl { /* SGE free-buffer queue state */
+ unsigned int avail; /* # of available Rx buffers */
+ unsigned int pend_cred; /* new buffers since last FL DB ring */
+ unsigned int cidx; /* consumer index */
+ unsigned int pidx; /* producer index */
+ unsigned long alloc_failed; /* # of times buffer allocation failed */
+ unsigned long large_alloc_failed;
+ unsigned long starving;
+ /* RO fields */
+ unsigned int cntxt_id; /* SGE context id for the free list */
+ unsigned int size; /* capacity of free list */
+ struct rx_sw_desc *sdesc; /* address of SW Rx descriptor ring */
+ __be64 *desc; /* address of HW Rx descriptor ring */
+ dma_addr_t addr; /* bus address of HW ring start */
+};
+
+/* A packet gather list */
+struct pkt_gl {
+ skb_frag_t frags[MAX_SKB_FRAGS];
+ void *va; /* virtual address of first byte */
+ unsigned int nfrags; /* # of fragments */
+ unsigned int tot_len; /* total length of fragments */
+};
+
+typedef int (*rspq_handler_t)(struct sge_rspq *q, const __be64 *rsp,
+ const struct pkt_gl *gl);
+
+struct sge_rspq { /* state for an SGE response queue */
+ struct napi_struct napi;
+ const __be64 *cur_desc; /* current descriptor in queue */
+ unsigned int cidx; /* consumer index */
+ u8 gen; /* current generation bit */
+ u8 intr_params; /* interrupt holdoff parameters */
+ u8 next_intr_params; /* holdoff params for next interrupt */
+ u8 pktcnt_idx; /* interrupt packet threshold */
+ u8 uld; /* ULD handling this queue */
+ u8 idx; /* queue index within its group */
+ int offset; /* offset into current Rx buffer */
+ u16 cntxt_id; /* SGE context id for the response q */
+ u16 abs_id; /* absolute SGE id for the response q */
+ __be64 *desc; /* address of HW response ring */
+ dma_addr_t phys_addr; /* physical address of the ring */
+ unsigned int iqe_len; /* entry size */
+ unsigned int size; /* capacity of response queue */
+ struct adapter *adap;
+ struct net_device *netdev; /* associated net device */
+ rspq_handler_t handler;
+};
+
+struct sge_eth_stats { /* Ethernet queue statistics */
+ unsigned long pkts; /* # of ethernet packets */
+ unsigned long lro_pkts; /* # of LRO super packets */
+ unsigned long lro_merged; /* # of wire packets merged by LRO */
+ unsigned long rx_cso; /* # of Rx checksum offloads */
+ unsigned long vlan_ex; /* # of Rx VLAN extractions */
+ unsigned long rx_drops; /* # of packets dropped due to no mem */
+};
+
+struct sge_eth_rxq { /* SW Ethernet Rx queue */
+ struct sge_rspq rspq;
+ struct sge_fl fl;
+ struct sge_eth_stats stats;
+} ____cacheline_aligned_in_smp;
+
+struct sge_ofld_stats { /* offload queue statistics */
+ unsigned long pkts; /* # of packets */
+ unsigned long imm; /* # of immediate-data packets */
+ unsigned long an; /* # of asynchronous notifications */
+ unsigned long nomem; /* # of responses deferred due to no mem */
+};
+
+struct sge_ofld_rxq { /* SW offload Rx queue */
+ struct sge_rspq rspq;
+ struct sge_fl fl;
+ struct sge_ofld_stats stats;
+} ____cacheline_aligned_in_smp;
+
+struct tx_desc {
+ __be64 flit[8];
+};
+
+struct tx_sw_desc;
+
+struct sge_txq {
+ unsigned int in_use; /* # of in-use Tx descriptors */
+ unsigned int size; /* # of descriptors */
+ unsigned int cidx; /* SW consumer index */
+ unsigned int pidx; /* producer index */
+ unsigned long stops; /* # of times q has been stopped */
+ unsigned long restarts; /* # of queue restarts */
+ unsigned int cntxt_id; /* SGE context id for the Tx q */
+ struct tx_desc *desc; /* address of HW Tx descriptor ring */
+ struct tx_sw_desc *sdesc; /* address of SW Tx descriptor ring */
+ struct sge_qstat *stat; /* queue status entry */
+ dma_addr_t phys_addr; /* physical address of the ring */
+};
+
+struct sge_eth_txq { /* state for an SGE Ethernet Tx queue */
+ struct sge_txq q;
+ struct netdev_queue *txq; /* associated netdev TX queue */
+ unsigned long tso; /* # of TSO requests */
+ unsigned long tx_cso; /* # of Tx checksum offloads */
+ unsigned long vlan_ins; /* # of Tx VLAN insertions */
+ unsigned long mapping_err; /* # of I/O MMU packet mapping errors */
+} ____cacheline_aligned_in_smp;
+
+struct sge_ofld_txq { /* state for an SGE offload Tx queue */
+ struct sge_txq q;
+ struct adapter *adap;
+ struct sk_buff_head sendq; /* list of backpressured packets */
+ struct tasklet_struct qresume_tsk; /* restarts the queue */
+ u8 full; /* the Tx ring is full */
+ unsigned long mapping_err; /* # of I/O MMU packet mapping errors */
+} ____cacheline_aligned_in_smp;
+
+struct sge_ctrl_txq { /* state for an SGE control Tx queue */
+ struct sge_txq q;
+ struct adapter *adap;
+ struct sk_buff_head sendq; /* list of backpressured packets */
+ struct tasklet_struct qresume_tsk; /* restarts the queue */
+ u8 full; /* the Tx ring is full */
+} ____cacheline_aligned_in_smp;
+
+struct sge {
+ struct sge_eth_txq ethtxq[MAX_ETH_QSETS];
+ struct sge_ofld_txq ofldtxq[MAX_OFLD_QSETS];
+ struct sge_ctrl_txq ctrlq[MAX_CTRL_QUEUES];
+
+ struct sge_eth_rxq ethrxq[MAX_ETH_QSETS];
+ struct sge_ofld_rxq ofldrxq[MAX_OFLD_QSETS];
+ struct sge_ofld_rxq rdmarxq[MAX_RDMA_QUEUES];
+ struct sge_rspq fw_evtq ____cacheline_aligned_in_smp;
+
+ struct sge_rspq intrq ____cacheline_aligned_in_smp;
+ spinlock_t intrq_lock;
+
+ u16 max_ethqsets; /* # of available Ethernet queue sets */
+ u16 ethqsets; /* # of active Ethernet queue sets */
+ u16 ethtxq_rover; /* Tx queue to clean up next */
+ u16 ofldqsets; /* # of active offload queue sets */
+ u16 rdmaqs; /* # of available RDMA Rx queues */
+ u16 ofld_rxq[MAX_OFLD_QSETS];
+ u16 rdma_rxq[NCHAN];
+ u16 timer_val[SGE_NTIMERS];
+ u8 counter_val[SGE_NCOUNTERS];
+ unsigned int starve_thres;
+ u8 idma_state[2];
+ void *egr_map[MAX_EGRQ]; /* qid->queue egress queue map */
+ struct sge_rspq *ingr_map[MAX_INGQ]; /* qid->queue ingress queue map */
+ DECLARE_BITMAP(starving_fl, MAX_EGRQ);
+ DECLARE_BITMAP(txq_maperr, MAX_EGRQ);
+ struct timer_list rx_timer; /* refills starving FLs */
+ struct timer_list tx_timer; /* checks Tx queues */
+};
+
+#define for_each_ethrxq(sge, i) for (i = 0; i < (sge)->ethqsets; i++)
+#define for_each_ofldrxq(sge, i) for (i = 0; i < (sge)->ofldqsets; i++)
+#define for_each_rdmarxq(sge, i) for (i = 0; i < (sge)->rdmaqs; i++)
+
+struct l2t_data;
+
+struct adapter {
+ void __iomem *regs;
+ struct pci_dev *pdev;
+ struct device *pdev_dev;
+ unsigned long registered_device_map;
+ unsigned long open_device_map;
+ unsigned long flags;
+
+ const char *name;
+ int msg_enable;
+
+ struct adapter_params params;
+ struct cxgb4_virt_res vres;
+ unsigned int swintr;
+
+ unsigned int wol;
+
+ struct {
+ unsigned short vec;
+ char desc[14];
+ } msix_info[MAX_INGQ + 1];
+
+ struct sge sge;
+
+ struct net_device *port[MAX_NPORTS];
+ u8 chan_map[NCHAN]; /* channel -> port map */
+
+ struct l2t_data *l2t;
+ void *uld_handle[CXGB4_ULD_MAX];
+ struct list_head list_node;
+
+ struct tid_info tids;
+ void **tid_release_head;
+ spinlock_t tid_release_lock;
+ struct work_struct tid_release_task;
+ bool tid_release_task_busy;
+
+ struct dentry *debugfs_root;
+
+ spinlock_t stats_lock;
+};
+
+static inline u32 t4_read_reg(struct adapter *adap, u32 reg_addr)
+{
+ return readl(adap->regs + reg_addr);
+}
+
+static inline void t4_write_reg(struct adapter *adap, u32 reg_addr, u32 val)
+{
+ writel(val, adap->regs + reg_addr);
+}
+
+#ifndef readq
+static inline u64 readq(const volatile void __iomem *addr)
+{
+ return readl(addr) + ((u64)readl(addr + 4) << 32);
+}
+
+static inline void writeq(u64 val, volatile void __iomem *addr)
+{
+ writel(val, addr);
+ writel(val >> 32, addr + 4);
+}
+#endif
+
+static inline u64 t4_read_reg64(struct adapter *adap, u32 reg_addr)
+{
+ return readq(adap->regs + reg_addr);
+}
+
+static inline void t4_write_reg64(struct adapter *adap, u32 reg_addr, u64 val)
+{
+ writeq(val, adap->regs + reg_addr);
+}
+
+/**
+ * netdev2pinfo - return the port_info structure associated with a net_device
+ * @dev: the netdev
+ *
+ * Return the struct port_info associated with a net_device
+ */
+static inline struct port_info *netdev2pinfo(const struct net_device *dev)
+{
+ return netdev_priv(dev);
+}
+
+/**
+ * adap2pinfo - return the port_info of a port
+ * @adap: the adapter
+ * @idx: the port index
+ *
+ * Return the port_info structure for the port of the given index.
+ */
+static inline struct port_info *adap2pinfo(struct adapter *adap, int idx)
+{
+ return netdev_priv(adap->port[idx]);
+}
+
+/**
+ * netdev2adap - return the adapter structure associated with a net_device
+ * @dev: the netdev
+ *
+ * Return the struct adapter associated with a net_device
+ */
+static inline struct adapter *netdev2adap(const struct net_device *dev)
+{
+ return netdev2pinfo(dev)->adapter;
+}
+
+void t4_os_portmod_changed(const struct adapter *adap, int port_id);
+void t4_os_link_changed(struct adapter *adap, int port_id, int link_stat);
+
+void *t4_alloc_mem(size_t size);
+void t4_free_mem(void *addr);
+
+void t4_free_sge_resources(struct adapter *adap);
+irq_handler_t t4_intr_handler(struct adapter *adap);
+netdev_tx_t t4_eth_xmit(struct sk_buff *skb, struct net_device *dev);
+int t4_ethrx_handler(struct sge_rspq *q, const __be64 *rsp,
+ const struct pkt_gl *gl);
+int t4_mgmt_tx(struct adapter *adap, struct sk_buff *skb);
+int t4_ofld_send(struct adapter *adap, struct sk_buff *skb);
+int t4_sge_alloc_rxq(struct adapter *adap, struct sge_rspq *iq, bool fwevtq,
+ struct net_device *dev, int intr_idx,
+ struct sge_fl *fl, rspq_handler_t hnd);
+int t4_sge_alloc_eth_txq(struct adapter *adap, struct sge_eth_txq *txq,
+ struct net_device *dev, struct netdev_queue *netdevq,
+ unsigned int iqid);
+int t4_sge_alloc_ctrl_txq(struct adapter *adap, struct sge_ctrl_txq *txq,
+ struct net_device *dev, unsigned int iqid,
+ unsigned int cmplqid);
+int t4_sge_alloc_ofld_txq(struct adapter *adap, struct sge_ofld_txq *txq,
+ struct net_device *dev, unsigned int iqid);
+irqreturn_t t4_sge_intr_msix(int irq, void *cookie);
+void t4_sge_init(struct adapter *adap);
+void t4_sge_start(struct adapter *adap);
+void t4_sge_stop(struct adapter *adap);
+
+#define for_each_port(adapter, iter) \
+ for (iter = 0; iter < (adapter)->params.nports; ++iter)
+
+static inline unsigned int core_ticks_per_usec(const struct adapter *adap)
+{
+ return adap->params.vpd.cclk / 1000;
+}
+
+static inline unsigned int us_to_core_ticks(const struct adapter *adap,
+ unsigned int us)
+{
+ return (us * adap->params.vpd.cclk) / 1000;
+}
+
+void t4_set_reg_field(struct adapter *adap, unsigned int addr, u32 mask,
+ u32 val);
+
+int t4_wr_mbox_meat(struct adapter *adap, int mbox, const void *cmd, int size,
+ void *rpl, bool sleep_ok);
+
+static inline int t4_wr_mbox(struct adapter *adap, int mbox, const void *cmd,
+ int size, void *rpl)
+{
+ return t4_wr_mbox_meat(adap, mbox, cmd, size, rpl, true);
+}
+
+static inline int t4_wr_mbox_ns(struct adapter *adap, int mbox, const void *cmd,
+ int size, void *rpl)
+{
+ return t4_wr_mbox_meat(adap, mbox, cmd, size, rpl, false);
+}
+
+void t4_intr_enable(struct adapter *adapter);
+void t4_intr_disable(struct adapter *adapter);
+void t4_intr_clear(struct adapter *adapter);
+int t4_slow_intr_handler(struct adapter *adapter);
+
+int t4_link_start(struct adapter *adap, unsigned int mbox, unsigned int port,
+ struct link_config *lc);
+int t4_restart_aneg(struct adapter *adap, unsigned int mbox, unsigned int port);
+int t4_seeprom_wp(struct adapter *adapter, bool enable);
+int t4_read_flash(struct adapter *adapter, unsigned int addr,
+ unsigned int nwords, u32 *data, int byte_oriented);
+int t4_load_fw(struct adapter *adapter, const u8 *fw_data, unsigned int size);
+int t4_check_fw_version(struct adapter *adapter);
+int t4_prep_adapter(struct adapter *adapter);
+int t4_port_init(struct adapter *adap, int mbox, int pf, int vf);
+void t4_fatal_err(struct adapter *adapter);
+void t4_set_vlan_accel(struct adapter *adapter, unsigned int ports, int on);
+int t4_set_trace_filter(struct adapter *adapter, const struct trace_params *tp,
+ int filter_index, int enable);
+void t4_get_trace_filter(struct adapter *adapter, struct trace_params *tp,
+ int filter_index, int *enabled);
+int t4_config_rss_range(struct adapter *adapter, int mbox, unsigned int viid,
+ int start, int n, const u16 *rspq, unsigned int nrspq);
+int t4_config_glbl_rss(struct adapter *adapter, int mbox, unsigned int mode,
+ unsigned int flags);
+int t4_read_rss(struct adapter *adapter, u16 *entries);
+int t4_mc_read(struct adapter *adap, u32 addr, __be32 *data, u64 *parity);
+int t4_edc_read(struct adapter *adap, int idx, u32 addr, __be32 *data,
+ u64 *parity);
+
+void t4_get_port_stats(struct adapter *adap, int idx, struct port_stats *p);
+void t4_get_lb_stats(struct adapter *adap, int idx, struct lb_port_stats *p);
+
+void t4_read_mtu_tbl(struct adapter *adap, u16 *mtus, u8 *mtu_log);
+void t4_tp_get_err_stats(struct adapter *adap, struct tp_err_stats *st);
+void t4_tp_get_tcp_stats(struct adapter *adap, struct tp_tcp_stats *v4,
+ struct tp_tcp_stats *v6);
+void t4_load_mtus(struct adapter *adap, const unsigned short *mtus,
+ const unsigned short *alpha, const unsigned short *beta);
+
+void t4_wol_magic_enable(struct adapter *adap, unsigned int port,
+ const u8 *addr);
+int t4_wol_pat_enable(struct adapter *adap, unsigned int port, unsigned int map,
+ u64 mask0, u64 mask1, unsigned int crc, bool enable);
+
+int t4_fw_hello(struct adapter *adap, unsigned int mbox, unsigned int evt_mbox,
+ enum dev_master master, enum dev_state *state);
+int t4_fw_bye(struct adapter *adap, unsigned int mbox);
+int t4_early_init(struct adapter *adap, unsigned int mbox);
+int t4_fw_reset(struct adapter *adap, unsigned int mbox, int reset);
+int t4_query_params(struct adapter *adap, unsigned int mbox, unsigned int pf,
+ unsigned int vf, unsigned int nparams, const u32 *params,
+ u32 *val);
+int t4_set_params(struct adapter *adap, unsigned int mbox, unsigned int pf,
+ unsigned int vf, unsigned int nparams, const u32 *params,
+ const u32 *val);
+int t4_cfg_pfvf(struct adapter *adap, unsigned int mbox, unsigned int pf,
+ unsigned int vf, unsigned int txq, unsigned int txq_eth_ctrl,
+ unsigned int rxqi, unsigned int rxq, unsigned int tc,
+ unsigned int vi, unsigned int cmask, unsigned int pmask,
+ unsigned int nexact, unsigned int rcaps, unsigned int wxcaps);
+int t4_alloc_vi(struct adapter *adap, unsigned int mbox, unsigned int port,
+ unsigned int pf, unsigned int vf, unsigned int nmac, u8 *mac,
+ unsigned int *rss_size);
+int t4_free_vi(struct adapter *adap, unsigned int mbox, unsigned int pf,
+ unsigned int vf, unsigned int viid);
+int t4_set_rxmode(struct adapter *adap, unsigned int mbox, unsigned int viid,
+ int mtu, int promisc, int all_multi, int bcast, bool sleep_ok);
+int t4_alloc_mac_filt(struct adapter *adap, unsigned int mbox,
+ unsigned int viid, bool free, unsigned int naddr,
+ const u8 **addr, u16 *idx, u64 *hash, bool sleep_ok);
+int t4_change_mac(struct adapter *adap, unsigned int mbox, unsigned int viid,
+ int idx, const u8 *addr, bool persist, bool add_smt);
+int t4_set_addr_hash(struct adapter *adap, unsigned int mbox, unsigned int viid,
+ bool ucast, u64 vec, bool sleep_ok);
+int t4_enable_vi(struct adapter *adap, unsigned int mbox, unsigned int viid,
+ bool rx_en, bool tx_en);
+int t4_identify_port(struct adapter *adap, unsigned int mbox, unsigned int viid,
+ unsigned int nblinks);
+int t4_mdio_rd(struct adapter *adap, unsigned int mbox, unsigned int phy_addr,
+ unsigned int mmd, unsigned int reg, u16 *valp);
+int t4_mdio_wr(struct adapter *adap, unsigned int mbox, unsigned int phy_addr,
+ unsigned int mmd, unsigned int reg, u16 val);
+int t4_iq_start_stop(struct adapter *adap, unsigned int mbox, bool start,
+ unsigned int pf, unsigned int vf, unsigned int iqid,
+ unsigned int fl0id, unsigned int fl1id);
+int t4_iq_free(struct adapter *adap, unsigned int mbox, unsigned int pf,
+ unsigned int vf, unsigned int iqtype, unsigned int iqid,
+ unsigned int fl0id, unsigned int fl1id);
+int t4_eth_eq_free(struct adapter *adap, unsigned int mbox, unsigned int pf,
+ unsigned int vf, unsigned int eqid);
+int t4_ctrl_eq_free(struct adapter *adap, unsigned int mbox, unsigned int pf,
+ unsigned int vf, unsigned int eqid);
+int t4_ofld_eq_free(struct adapter *adap, unsigned int mbox, unsigned int pf,
+ unsigned int vf, unsigned int eqid);
+int t4_handle_fw_rpl(struct adapter *adap, const __be64 *rpl);
+#endif /* __CXGB4_H__ */
--- /dev/null
+/*
+ * This file is part of the Chelsio T4 Ethernet driver for Linux.
+ *
+ * Copyright (c) 2003-2010 Chelsio Communications, Inc. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/bitmap.h>
+#include <linux/crc32.h>
+#include <linux/ctype.h>
+#include <linux/debugfs.h>
+#include <linux/err.h>
+#include <linux/etherdevice.h>
+#include <linux/firmware.h>
+#include <linux/if_vlan.h>
+#include <linux/init.h>
+#include <linux/log2.h>
+#include <linux/mdio.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/mutex.h>
+#include <linux/netdevice.h>
+#include <linux/pci.h>
+#include <linux/aer.h>
+#include <linux/rtnetlink.h>
+#include <linux/sched.h>
+#include <linux/seq_file.h>
+#include <linux/sockios.h>
+#include <linux/vmalloc.h>
+#include <linux/workqueue.h>
+#include <net/neighbour.h>
+#include <net/netevent.h>
+#include <asm/uaccess.h>
+
+#include "cxgb4.h"
+#include "t4_regs.h"
+#include "t4_msg.h"
+#include "t4fw_api.h"
+#include "l2t.h"
+
+#define DRV_VERSION "1.0.0-ko"
+#define DRV_DESC "Chelsio T4 Network Driver"
+
+/*
+ * Max interrupt hold-off timer value in us. Queues fall back to this value
+ * under extreme memory pressure so it's largish to give the system time to
+ * recover.
+ */
+#define MAX_SGE_TIMERVAL 200U
+
+enum {
+ MEMWIN0_APERTURE = 65536,
+ MEMWIN0_BASE = 0x30000,
+ MEMWIN1_APERTURE = 32768,
+ MEMWIN1_BASE = 0x28000,
+ MEMWIN2_APERTURE = 2048,
+ MEMWIN2_BASE = 0x1b800,
+};
+
+enum {
+ MAX_TXQ_ENTRIES = 16384,
+ MAX_CTRL_TXQ_ENTRIES = 1024,
+ MAX_RSPQ_ENTRIES = 16384,
+ MAX_RX_BUFFERS = 16384,
+ MIN_TXQ_ENTRIES = 32,
+ MIN_CTRL_TXQ_ENTRIES = 32,
+ MIN_RSPQ_ENTRIES = 128,
+ MIN_FL_ENTRIES = 16
+};
+
+#define DFLT_MSG_ENABLE (NETIF_MSG_DRV | NETIF_MSG_PROBE | NETIF_MSG_LINK | \
+ NETIF_MSG_TIMER | NETIF_MSG_IFDOWN | NETIF_MSG_IFUP |\
+ NETIF_MSG_RX_ERR | NETIF_MSG_TX_ERR)
+
+#define CH_DEVICE(devid) { PCI_VDEVICE(CHELSIO, devid), 0 }
+
+static DEFINE_PCI_DEVICE_TABLE(cxgb4_pci_tbl) = {
+ CH_DEVICE(0xa000), /* PE10K */
+ { 0, }
+};
+
+#define FW_FNAME "cxgb4/t4fw.bin"
+
+MODULE_DESCRIPTION(DRV_DESC);
+MODULE_AUTHOR("Chelsio Communications");
+MODULE_LICENSE("Dual BSD/GPL");
+MODULE_VERSION(DRV_VERSION);
+MODULE_DEVICE_TABLE(pci, cxgb4_pci_tbl);
+MODULE_FIRMWARE(FW_FNAME);
+
+static int dflt_msg_enable = DFLT_MSG_ENABLE;
+
+module_param(dflt_msg_enable, int, 0644);
+MODULE_PARM_DESC(dflt_msg_enable, "Chelsio T4 default message enable bitmap");
+
+/*
+ * The driver uses the best interrupt scheme available on a platform in the
+ * order MSI-X, MSI, legacy INTx interrupts. This parameter determines which
+ * of these schemes the driver may consider as follows:
+ *
+ * msi = 2: choose from among all three options
+ * msi = 1: only consider MSI and INTx interrupts
+ * msi = 0: force INTx interrupts
+ */
+static int msi = 2;
+
+module_param(msi, int, 0644);
+MODULE_PARM_DESC(msi, "whether to use INTx (0), MSI (1) or MSI-X (2)");
+
+/*
+ * Queue interrupt hold-off timer values. Queues default to the first of these
+ * upon creation.
+ */
+static unsigned int intr_holdoff[SGE_NTIMERS - 1] = { 5, 10, 20, 50, 100 };
+
+module_param_array(intr_holdoff, uint, NULL, 0644);
+MODULE_PARM_DESC(intr_holdoff, "values for queue interrupt hold-off timers "
+ "0..4 in microseconds");
+
+static unsigned int intr_cnt[SGE_NCOUNTERS - 1] = { 4, 8, 16 };
+
+module_param_array(intr_cnt, uint, NULL, 0644);
+MODULE_PARM_DESC(intr_cnt,
+ "thresholds 1..3 for queue interrupt packet counters");
+
+static int vf_acls;
+
+#ifdef CONFIG_PCI_IOV
+module_param(vf_acls, bool, 0644);
+MODULE_PARM_DESC(vf_acls, "if set enable virtualization L2 ACL enforcement");
+
+static unsigned int num_vf[4];
+
+module_param_array(num_vf, uint, NULL, 0644);
+MODULE_PARM_DESC(num_vf, "number of VFs for each of PFs 0-3");
+#endif
+
+static struct dentry *cxgb4_debugfs_root;
+
+static LIST_HEAD(adapter_list);
+static DEFINE_MUTEX(uld_mutex);
+static struct cxgb4_uld_info ulds[CXGB4_ULD_MAX];
+static const char *uld_str[] = { "RDMA", "iSCSI" };
+
+static void link_report(struct net_device *dev)
+{
+ if (!netif_carrier_ok(dev))
+ netdev_info(dev, "link down\n");
+ else {
+ static const char *fc[] = { "no", "Rx", "Tx", "Tx/Rx" };
+
+ const char *s = "10Mbps";
+ const struct port_info *p = netdev_priv(dev);
+
+ switch (p->link_cfg.speed) {
+ case SPEED_10000:
+ s = "10Gbps";
+ break;
+ case SPEED_1000:
+ s = "1000Mbps";
+ break;
+ case SPEED_100:
+ s = "100Mbps";
+ break;
+ }
+
+ netdev_info(dev, "link up, %s, full-duplex, %s PAUSE\n", s,
+ fc[p->link_cfg.fc]);
+ }
+}
+
+void t4_os_link_changed(struct adapter *adapter, int port_id, int link_stat)
+{
+ struct net_device *dev = adapter->port[port_id];
+
+ /* Skip changes from disabled ports. */
+ if (netif_running(dev) && link_stat != netif_carrier_ok(dev)) {
+ if (link_stat)
+ netif_carrier_on(dev);
+ else
+ netif_carrier_off(dev);
+
+ link_report(dev);
+ }
+}
+
+void t4_os_portmod_changed(const struct adapter *adap, int port_id)
+{
+ static const char *mod_str[] = {
+ NULL, "LR", "SR", "ER", "passive DA", "active DA"
+ };
+
+ const struct net_device *dev = adap->port[port_id];
+ const struct port_info *pi = netdev_priv(dev);
+
+ if (pi->mod_type == FW_PORT_MOD_TYPE_NONE)
+ netdev_info(dev, "port module unplugged\n");
+ else
+ netdev_info(dev, "%s module inserted\n", mod_str[pi->mod_type]);
+}
+
+/*
+ * Configure the exact and hash address filters to handle a port's multicast
+ * and secondary unicast MAC addresses.
+ */
+static int set_addr_filters(const struct net_device *dev, bool sleep)
+{
+ u64 mhash = 0;
+ u64 uhash = 0;
+ bool free = true;
+ u16 filt_idx[7];
+ const u8 *addr[7];
+ int ret, naddr = 0;
+ const struct dev_addr_list *d;
+ const struct netdev_hw_addr *ha;
+ int uc_cnt = netdev_uc_count(dev);
+ const struct port_info *pi = netdev_priv(dev);
+
+ /* first do the secondary unicast addresses */
+ netdev_for_each_uc_addr(ha, dev) {
+ addr[naddr++] = ha->addr;
+ if (--uc_cnt == 0 || naddr >= ARRAY_SIZE(addr)) {
+ ret = t4_alloc_mac_filt(pi->adapter, 0, pi->viid, free,
+ naddr, addr, filt_idx, &uhash, sleep);
+ if (ret < 0)
+ return ret;
+
+ free = false;
+ naddr = 0;
+ }
+ }
+
+ /* next set up the multicast addresses */
+ netdev_for_each_mc_addr(d, dev) {
+ addr[naddr++] = d->dmi_addr;
+ if (naddr >= ARRAY_SIZE(addr) || d->next == NULL) {
+ ret = t4_alloc_mac_filt(pi->adapter, 0, pi->viid, free,
+ naddr, addr, filt_idx, &mhash, sleep);
+ if (ret < 0)
+ return ret;
+
+ free = false;
+ naddr = 0;
+ }
+ }
+
+ return t4_set_addr_hash(pi->adapter, 0, pi->viid, uhash != 0,
+ uhash | mhash, sleep);
+}
+
+/*
+ * Set Rx properties of a port, such as promiscruity, address filters, and MTU.
+ * If @mtu is -1 it is left unchanged.
+ */
+static int set_rxmode(struct net_device *dev, int mtu, bool sleep_ok)
+{
+ int ret;
+ struct port_info *pi = netdev_priv(dev);
+
+ ret = set_addr_filters(dev, sleep_ok);
+ if (ret == 0)
+ ret = t4_set_rxmode(pi->adapter, 0, pi->viid, mtu,
+ (dev->flags & IFF_PROMISC) ? 1 : 0,
+ (dev->flags & IFF_ALLMULTI) ? 1 : 0, 1,
+ sleep_ok);
+ return ret;
+}
+
+/**
+ * link_start - enable a port
+ * @dev: the port to enable
+ *
+ * Performs the MAC and PHY actions needed to enable a port.
+ */
+static int link_start(struct net_device *dev)
+{
+ int ret;
+ struct port_info *pi = netdev_priv(dev);
+
+ /*
+ * We do not set address filters and promiscuity here, the stack does
+ * that step explicitly.
+ */
+ ret = t4_set_rxmode(pi->adapter, 0, pi->viid, dev->mtu, -1, -1, -1,
+ true);
+ if (ret == 0) {
+ ret = t4_change_mac(pi->adapter, 0, pi->viid,
+ pi->xact_addr_filt, dev->dev_addr, true,
+ false);
+ if (ret >= 0) {
+ pi->xact_addr_filt = ret;
+ ret = 0;
+ }
+ }
+ if (ret == 0)
+ ret = t4_link_start(pi->adapter, 0, pi->tx_chan, &pi->link_cfg);
+ if (ret == 0)
+ ret = t4_enable_vi(pi->adapter, 0, pi->viid, true, true);
+ return ret;
+}
+
+/*
+ * Response queue handler for the FW event queue.
+ */
+static int fwevtq_handler(struct sge_rspq *q, const __be64 *rsp,
+ const struct pkt_gl *gl)
+{
+ u8 opcode = ((const struct rss_header *)rsp)->opcode;
+
+ rsp++; /* skip RSS header */
+ if (likely(opcode == CPL_SGE_EGR_UPDATE)) {
+ const struct cpl_sge_egr_update *p = (void *)rsp;
+ unsigned int qid = EGR_QID(ntohl(p->opcode_qid));
+ struct sge_txq *txq = q->adap->sge.egr_map[qid];
+
+ txq->restarts++;
+ if ((u8 *)txq < (u8 *)q->adap->sge.ethrxq) {
+ struct sge_eth_txq *eq;
+
+ eq = container_of(txq, struct sge_eth_txq, q);
+ netif_tx_wake_queue(eq->txq);
+ } else {
+ struct sge_ofld_txq *oq;
+
+ oq = container_of(txq, struct sge_ofld_txq, q);
+ tasklet_schedule(&oq->qresume_tsk);
+ }
+ } else if (opcode == CPL_FW6_MSG || opcode == CPL_FW4_MSG) {
+ const struct cpl_fw6_msg *p = (void *)rsp;
+
+ if (p->type == 0)
+ t4_handle_fw_rpl(q->adap, p->data);
+ } else if (opcode == CPL_L2T_WRITE_RPL) {
+ const struct cpl_l2t_write_rpl *p = (void *)rsp;
+
+ do_l2t_write_rpl(q->adap, p);
+ } else
+ dev_err(q->adap->pdev_dev,
+ "unexpected CPL %#x on FW event queue\n", opcode);
+ return 0;
+}
+
+/**
+ * uldrx_handler - response queue handler for ULD queues
+ * @q: the response queue that received the packet
+ * @rsp: the response queue descriptor holding the offload message
+ * @gl: the gather list of packet fragments
+ *
+ * Deliver an ingress offload packet to a ULD. All processing is done by
+ * the ULD, we just maintain statistics.
+ */
+static int uldrx_handler(struct sge_rspq *q, const __be64 *rsp,
+ const struct pkt_gl *gl)
+{
+ struct sge_ofld_rxq *rxq = container_of(q, struct sge_ofld_rxq, rspq);
+
+ if (ulds[q->uld].rx_handler(q->adap->uld_handle[q->uld], rsp, gl)) {
+ rxq->stats.nomem++;
+ return -1;
+ }
+ if (gl == NULL)
+ rxq->stats.imm++;
+ else if (gl == CXGB4_MSG_AN)
+ rxq->stats.an++;
+ else
+ rxq->stats.pkts++;
+ return 0;
+}
+
+static void disable_msi(struct adapter *adapter)
+{
+ if (adapter->flags & USING_MSIX) {
+ pci_disable_msix(adapter->pdev);
+ adapter->flags &= ~USING_MSIX;
+ } else if (adapter->flags & USING_MSI) {
+ pci_disable_msi(adapter->pdev);
+ adapter->flags &= ~USING_MSI;
+ }
+}
+
+/*
+ * Interrupt handler for non-data events used with MSI-X.
+ */
+static irqreturn_t t4_nondata_intr(int irq, void *cookie)
+{
+ struct adapter *adap = cookie;
+
+ u32 v = t4_read_reg(adap, MYPF_REG(PL_PF_INT_CAUSE));
+ if (v & PFSW) {
+ adap->swintr = 1;
+ t4_write_reg(adap, MYPF_REG(PL_PF_INT_CAUSE), v);
+ }
+ t4_slow_intr_handler(adap);
+ return IRQ_HANDLED;
+}
+
+/*
+ * Name the MSI-X interrupts.
+ */
+static void name_msix_vecs(struct adapter *adap)
+{
+ int i, j, msi_idx = 2, n = sizeof(adap->msix_info[0].desc) - 1;
+
+ /* non-data interrupts */
+ snprintf(adap->msix_info[0].desc, n, "%s", adap->name);
+ adap->msix_info[0].desc[n] = 0;
+
+ /* FW events */
+ snprintf(adap->msix_info[1].desc, n, "%s-FWeventq", adap->name);
+ adap->msix_info[1].desc[n] = 0;
+
+ /* Ethernet queues */
+ for_each_port(adap, j) {
+ struct net_device *d = adap->port[j];
+ const struct port_info *pi = netdev_priv(d);
+
+ for (i = 0; i < pi->nqsets; i++, msi_idx++) {
+ snprintf(adap->msix_info[msi_idx].desc, n, "%s-Rx%d",
+ d->name, i);
+ adap->msix_info[msi_idx].desc[n] = 0;
+ }
+ }
+
+ /* offload queues */
+ for_each_ofldrxq(&adap->sge, i) {
+ snprintf(adap->msix_info[msi_idx].desc, n, "%s-ofld%d",
+ adap->name, i);
+ adap->msix_info[msi_idx++].desc[n] = 0;
+ }
+ for_each_rdmarxq(&adap->sge, i) {
+ snprintf(adap->msix_info[msi_idx].desc, n, "%s-rdma%d",
+ adap->name, i);
+ adap->msix_info[msi_idx++].desc[n] = 0;
+ }
+}
+
+static int request_msix_queue_irqs(struct adapter *adap)
+{
+ struct sge *s = &adap->sge;
+ int err, ethqidx, ofldqidx = 0, rdmaqidx = 0, msi = 2;
+
+ err = request_irq(adap->msix_info[1].vec, t4_sge_intr_msix, 0,
+ adap->msix_info[1].desc, &s->fw_evtq);
+ if (err)
+ return err;
+
+ for_each_ethrxq(s, ethqidx) {
+ err = request_irq(adap->msix_info[msi].vec, t4_sge_intr_msix, 0,
+ adap->msix_info[msi].desc,
+ &s->ethrxq[ethqidx].rspq);
+ if (err)
+ goto unwind;
+ msi++;
+ }
+ for_each_ofldrxq(s, ofldqidx) {
+ err = request_irq(adap->msix_info[msi].vec, t4_sge_intr_msix, 0,
+ adap->msix_info[msi].desc,
+ &s->ofldrxq[ofldqidx].rspq);
+ if (err)
+ goto unwind;
+ msi++;
+ }
+ for_each_rdmarxq(s, rdmaqidx) {
+ err = request_irq(adap->msix_info[msi].vec, t4_sge_intr_msix, 0,
+ adap->msix_info[msi].desc,
+ &s->rdmarxq[rdmaqidx].rspq);
+ if (err)
+ goto unwind;
+ msi++;
+ }
+ return 0;
+
+unwind:
+ while (--rdmaqidx >= 0)
+ free_irq(adap->msix_info[--msi].vec,
+ &s->rdmarxq[rdmaqidx].rspq);
+ while (--ofldqidx >= 0)
+ free_irq(adap->msix_info[--msi].vec,
+ &s->ofldrxq[ofldqidx].rspq);
+ while (--ethqidx >= 0)
+ free_irq(adap->msix_info[--msi].vec, &s->ethrxq[ethqidx].rspq);
+ free_irq(adap->msix_info[1].vec, &s->fw_evtq);
+ return err;
+}
+
+static void free_msix_queue_irqs(struct adapter *adap)
+{
+ int i, msi = 2;
+ struct sge *s = &adap->sge;
+
+ free_irq(adap->msix_info[1].vec, &s->fw_evtq);
+ for_each_ethrxq(s, i)
+ free_irq(adap->msix_info[msi++].vec, &s->ethrxq[i].rspq);
+ for_each_ofldrxq(s, i)
+ free_irq(adap->msix_info[msi++].vec, &s->ofldrxq[i].rspq);
+ for_each_rdmarxq(s, i)
+ free_irq(adap->msix_info[msi++].vec, &s->rdmarxq[i].rspq);
+}
+
+/**
+ * setup_rss - configure RSS
+ * @adap: the adapter
+ *
+ * Sets up RSS to distribute packets to multiple receive queues. We
+ * configure the RSS CPU lookup table to distribute to the number of HW
+ * receive queues, and the response queue lookup table to narrow that
+ * down to the response queues actually configured for each port.
+ * We always configure the RSS mapping for all ports since the mapping
+ * table has plenty of entries.
+ */
+static int setup_rss(struct adapter *adap)
+{
+ int i, j, err;
+ u16 rss[MAX_ETH_QSETS];
+
+ for_each_port(adap, i) {
+ const struct port_info *pi = adap2pinfo(adap, i);
+ const struct sge_eth_rxq *q = &adap->sge.ethrxq[pi->first_qset];
+
+ for (j = 0; j < pi->nqsets; j++)
+ rss[j] = q[j].rspq.abs_id;
+
+ err = t4_config_rss_range(adap, 0, pi->viid, 0, pi->rss_size,
+ rss, pi->nqsets);
+ if (err)
+ return err;
+ }
+ return 0;
+}
+
+/*
+ * Wait until all NAPI handlers are descheduled.
+ */
+static void quiesce_rx(struct adapter *adap)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(adap->sge.ingr_map); i++) {
+ struct sge_rspq *q = adap->sge.ingr_map[i];
+
+ if (q && q->handler)
+ napi_disable(&q->napi);
+ }
+}
+
+/*
+ * Enable NAPI scheduling and interrupt generation for all Rx queues.
+ */
+static void enable_rx(struct adapter *adap)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(adap->sge.ingr_map); i++) {
+ struct sge_rspq *q = adap->sge.ingr_map[i];
+
+ if (!q)
+ continue;
+ if (q->handler)
+ napi_enable(&q->napi);
+ /* 0-increment GTS to start the timer and enable interrupts */
+ t4_write_reg(adap, MYPF_REG(SGE_PF_GTS),
+ SEINTARM(q->intr_params) |
+ INGRESSQID(q->cntxt_id));
+ }
+}
+
+/**
+ * setup_sge_queues - configure SGE Tx/Rx/response queues
+ * @adap: the adapter
+ *
+ * Determines how many sets of SGE queues to use and initializes them.
+ * We support multiple queue sets per port if we have MSI-X, otherwise
+ * just one queue set per port.
+ */
+static int setup_sge_queues(struct adapter *adap)
+{
+ int err, msi_idx, i, j;
+ struct sge *s = &adap->sge;
+
+ bitmap_zero(s->starving_fl, MAX_EGRQ);
+ bitmap_zero(s->txq_maperr, MAX_EGRQ);
+
+ if (adap->flags & USING_MSIX)
+ msi_idx = 1; /* vector 0 is for non-queue interrupts */
+ else {
+ err = t4_sge_alloc_rxq(adap, &s->intrq, false, adap->port[0], 0,
+ NULL, NULL);
+ if (err)
+ return err;
+ msi_idx = -((int)s->intrq.abs_id + 1);
+ }
+
+ err = t4_sge_alloc_rxq(adap, &s->fw_evtq, true, adap->port[0],
+ msi_idx, NULL, fwevtq_handler);
+ if (err) {
+freeout: t4_free_sge_resources(adap);
+ return err;
+ }
+
+ for_each_port(adap, i) {
+ struct net_device *dev = adap->port[i];
+ struct port_info *pi = netdev_priv(dev);
+ struct sge_eth_rxq *q = &s->ethrxq[pi->first_qset];
+ struct sge_eth_txq *t = &s->ethtxq[pi->first_qset];
+
+ for (j = 0; j < pi->nqsets; j++, q++) {
+ if (msi_idx > 0)
+ msi_idx++;
+ err = t4_sge_alloc_rxq(adap, &q->rspq, false, dev,
+ msi_idx, &q->fl,
+ t4_ethrx_handler);
+ if (err)
+ goto freeout;
+ q->rspq.idx = j;
+ memset(&q->stats, 0, sizeof(q->stats));
+ }
+ for (j = 0; j < pi->nqsets; j++, t++) {
+ err = t4_sge_alloc_eth_txq(adap, t, dev,
+ netdev_get_tx_queue(dev, j),
+ s->fw_evtq.cntxt_id);
+ if (err)
+ goto freeout;
+ }
+ }
+
+ j = s->ofldqsets / adap->params.nports; /* ofld queues per channel */
+ for_each_ofldrxq(s, i) {
+ struct sge_ofld_rxq *q = &s->ofldrxq[i];
+ struct net_device *dev = adap->port[i / j];
+
+ if (msi_idx > 0)
+ msi_idx++;
+ err = t4_sge_alloc_rxq(adap, &q->rspq, false, dev, msi_idx,
+ &q->fl, uldrx_handler);
+ if (err)
+ goto freeout;
+ memset(&q->stats, 0, sizeof(q->stats));
+ s->ofld_rxq[i] = q->rspq.abs_id;
+ err = t4_sge_alloc_ofld_txq(adap, &s->ofldtxq[i], dev,
+ s->fw_evtq.cntxt_id);
+ if (err)
+ goto freeout;
+ }
+
+ for_each_rdmarxq(s, i) {
+ struct sge_ofld_rxq *q = &s->rdmarxq[i];
+
+ if (msi_idx > 0)
+ msi_idx++;
+ err = t4_sge_alloc_rxq(adap, &q->rspq, false, adap->port[i],
+ msi_idx, &q->fl, uldrx_handler);
+ if (err)
+ goto freeout;
+ memset(&q->stats, 0, sizeof(q->stats));
+ s->rdma_rxq[i] = q->rspq.abs_id;
+ }
+
+ for_each_port(adap, i) {
+ /*
+ * Note that ->rdmarxq[i].rspq.cntxt_id below is 0 if we don't
+ * have RDMA queues, and that's the right value.
+ */
+ err = t4_sge_alloc_ctrl_txq(adap, &s->ctrlq[i], adap->port[i],
+ s->fw_evtq.cntxt_id,
+ s->rdmarxq[i].rspq.cntxt_id);
+ if (err)
+ goto freeout;
+ }
+
+ t4_write_reg(adap, MPS_TRC_RSS_CONTROL,
+ RSSCONTROL(netdev2pinfo(adap->port[0])->tx_chan) |
+ QUEUENUMBER(s->ethrxq[0].rspq.abs_id));
+ return 0;
+}
+
+/*
+ * Returns 0 if new FW was successfully loaded, a positive errno if a load was
+ * started but failed, and a negative errno if flash load couldn't start.
+ */
+static int upgrade_fw(struct adapter *adap)
+{
+ int ret;
+ u32 vers;
+ const struct fw_hdr *hdr;
+ const struct firmware *fw;
+ struct device *dev = adap->pdev_dev;
+
+ ret = request_firmware(&fw, FW_FNAME, dev);
+ if (ret < 0) {
+ dev_err(dev, "unable to load firmware image " FW_FNAME
+ ", error %d\n", ret);
+ return ret;
+ }
+
+ hdr = (const struct fw_hdr *)fw->data;
+ vers = ntohl(hdr->fw_ver);
+ if (FW_HDR_FW_VER_MAJOR_GET(vers) != FW_VERSION_MAJOR) {
+ ret = -EINVAL; /* wrong major version, won't do */
+ goto out;
+ }
+
+ /*
+ * If the flash FW is unusable or we found something newer, load it.
+ */
+ if (FW_HDR_FW_VER_MAJOR_GET(adap->params.fw_vers) != FW_VERSION_MAJOR ||
+ vers > adap->params.fw_vers) {
+ ret = -t4_load_fw(adap, fw->data, fw->size);
+ if (!ret)
+ dev_info(dev, "firmware upgraded to version %pI4 from "
+ FW_FNAME "\n", &hdr->fw_ver);
+ }
+out: release_firmware(fw);
+ return ret;
+}
+
+/*
+ * Allocate a chunk of memory using kmalloc or, if that fails, vmalloc.
+ * The allocated memory is cleared.
+ */
+void *t4_alloc_mem(size_t size)
+{
+ void *p = kmalloc(size, GFP_KERNEL);
+
+ if (!p)
+ p = vmalloc(size);
+ if (p)
+ memset(p, 0, size);
+ return p;
+}
+
+/*
+ * Free memory allocated through alloc_mem().
+ */
+void t4_free_mem(void *addr)
+{
+ if (is_vmalloc_addr(addr))
+ vfree(addr);
+ else
+ kfree(addr);
+}
+
+static inline int is_offload(const struct adapter *adap)
+{
+ return adap->params.offload;
+}
+
+/*
+ * Implementation of ethtool operations.
+ */
+
+static u32 get_msglevel(struct net_device *dev)
+{
+ return netdev2adap(dev)->msg_enable;
+}
+
+static void set_msglevel(struct net_device *dev, u32 val)
+{
+ netdev2adap(dev)->msg_enable = val;
+}
+
+static char stats_strings[][ETH_GSTRING_LEN] = {
+ "TxOctetsOK ",
+ "TxFramesOK ",
+ "TxBroadcastFrames ",
+ "TxMulticastFrames ",
+ "TxUnicastFrames ",
+ "TxErrorFrames ",
+
+ "TxFrames64 ",
+ "TxFrames65To127 ",
+ "TxFrames128To255 ",
+ "TxFrames256To511 ",
+ "TxFrames512To1023 ",
+ "TxFrames1024To1518 ",
+ "TxFrames1519ToMax ",
+
+ "TxFramesDropped ",
+ "TxPauseFrames ",
+ "TxPPP0Frames ",
+ "TxPPP1Frames ",
+ "TxPPP2Frames ",
+ "TxPPP3Frames ",
+ "TxPPP4Frames ",
+ "TxPPP5Frames ",
+ "TxPPP6Frames ",
+ "TxPPP7Frames ",
+
+ "RxOctetsOK ",
+ "RxFramesOK ",
+ "RxBroadcastFrames ",
+ "RxMulticastFrames ",
+ "RxUnicastFrames ",
+
+ "RxFramesTooLong ",
+ "RxJabberErrors ",
+ "RxFCSErrors ",
+ "RxLengthErrors ",
+ "RxSymbolErrors ",
+ "RxRuntFrames ",
+
+ "RxFrames64 ",
+ "RxFrames65To127 ",
+ "RxFrames128To255 ",
+ "RxFrames256To511 ",
+ "RxFrames512To1023 ",
+ "RxFrames1024To1518 ",
+ "RxFrames1519ToMax ",
+
+ "RxPauseFrames ",
+ "RxPPP0Frames ",
+ "RxPPP1Frames ",
+ "RxPPP2Frames ",
+ "RxPPP3Frames ",
+ "RxPPP4Frames ",
+ "RxPPP5Frames ",
+ "RxPPP6Frames ",
+ "RxPPP7Frames ",
+
+ "RxBG0FramesDropped ",
+ "RxBG1FramesDropped ",
+ "RxBG2FramesDropped ",
+ "RxBG3FramesDropped ",
+ "RxBG0FramesTrunc ",
+ "RxBG1FramesTrunc ",
+ "RxBG2FramesTrunc ",
+ "RxBG3FramesTrunc ",
+
+ "TSO ",
+ "TxCsumOffload ",
+ "RxCsumGood ",
+ "VLANextractions ",
+ "VLANinsertions ",
+};
+
+static int get_sset_count(struct net_device *dev, int sset)
+{
+ switch (sset) {
+ case ETH_SS_STATS:
+ return ARRAY_SIZE(stats_strings);
+ default:
+ return -EOPNOTSUPP;
+ }
+}
+
+#define T4_REGMAP_SIZE (160 * 1024)
+
+static int get_regs_len(struct net_device *dev)
+{
+ return T4_REGMAP_SIZE;
+}
+
+static int get_eeprom_len(struct net_device *dev)
+{
+ return EEPROMSIZE;
+}
+
+static void get_drvinfo(struct net_device *dev, struct ethtool_drvinfo *info)
+{
+ struct adapter *adapter = netdev2adap(dev);
+
+ strcpy(info->driver, KBUILD_MODNAME);
+ strcpy(info->version, DRV_VERSION);
+ strcpy(info->bus_info, pci_name(adapter->pdev));
+
+ if (!adapter->params.fw_vers)
+ strcpy(info->fw_version, "N/A");
+ else
+ snprintf(info->fw_version, sizeof(info->fw_version),
+ "%u.%u.%u.%u, TP %u.%u.%u.%u",
+ FW_HDR_FW_VER_MAJOR_GET(adapter->params.fw_vers),
+ FW_HDR_FW_VER_MINOR_GET(adapter->params.fw_vers),
+ FW_HDR_FW_VER_MICRO_GET(adapter->params.fw_vers),
+ FW_HDR_FW_VER_BUILD_GET(adapter->params.fw_vers),
+ FW_HDR_FW_VER_MAJOR_GET(adapter->params.tp_vers),
+ FW_HDR_FW_VER_MINOR_GET(adapter->params.tp_vers),
+ FW_HDR_FW_VER_MICRO_GET(adapter->params.tp_vers),
+ FW_HDR_FW_VER_BUILD_GET(adapter->params.tp_vers));
+}
+
+static void get_strings(struct net_device *dev, u32 stringset, u8 *data)
+{
+ if (stringset == ETH_SS_STATS)
+ memcpy(data, stats_strings, sizeof(stats_strings));
+}
+
+/*
+ * port stats maintained per queue of the port. They should be in the same
+ * order as in stats_strings above.
+ */
+struct queue_port_stats {
+ u64 tso;
+ u64 tx_csum;
+ u64 rx_csum;
+ u64 vlan_ex;
+ u64 vlan_ins;
+};
+
+static void collect_sge_port_stats(const struct adapter *adap,
+ const struct port_info *p, struct queue_port_stats *s)
+{
+ int i;
+ const struct sge_eth_txq *tx = &adap->sge.ethtxq[p->first_qset];
+ const struct sge_eth_rxq *rx = &adap->sge.ethrxq[p->first_qset];
+
+ memset(s, 0, sizeof(*s));
+ for (i = 0; i < p->nqsets; i++, rx++, tx++) {
+ s->tso += tx->tso;
+ s->tx_csum += tx->tx_cso;
+ s->rx_csum += rx->stats.rx_cso;
+ s->vlan_ex += rx->stats.vlan_ex;
+ s->vlan_ins += tx->vlan_ins;
+ }
+}
+
+static void get_stats(struct net_device *dev, struct ethtool_stats *stats,
+ u64 *data)
+{
+ struct port_info *pi = netdev_priv(dev);
+ struct adapter *adapter = pi->adapter;
+
+ t4_get_port_stats(adapter, pi->tx_chan, (struct port_stats *)data);
+
+ data += sizeof(struct port_stats) / sizeof(u64);
+ collect_sge_port_stats(adapter, pi, (struct queue_port_stats *)data);
+}
+
+/*
+ * Return a version number to identify the type of adapter. The scheme is:
+ * - bits 0..9: chip version
+ * - bits 10..15: chip revision
+ */
+static inline unsigned int mk_adap_vers(const struct adapter *ap)
+{
+ return 4 | (ap->params.rev << 10);
+}
+
+static void reg_block_dump(struct adapter *ap, void *buf, unsigned int start,
+ unsigned int end)
+{
+ u32 *p = buf + start;
+
+ for ( ; start <= end; start += sizeof(u32))
+ *p++ = t4_read_reg(ap, start);
+}
+
+static void get_regs(struct net_device *dev, struct ethtool_regs *regs,
+ void *buf)
+{
+ static const unsigned int reg_ranges[] = {
+ 0x1008, 0x1108,
+ 0x1180, 0x11b4,
+ 0x11fc, 0x123c,
+ 0x1300, 0x173c,
+ 0x1800, 0x18fc,
+ 0x3000, 0x30d8,
+ 0x30e0, 0x5924,
+ 0x5960, 0x59d4,
+ 0x5a00, 0x5af8,
+ 0x6000, 0x6098,
+ 0x6100, 0x6150,
+ 0x6200, 0x6208,
+ 0x6240, 0x6248,
+ 0x6280, 0x6338,
+ 0x6370, 0x638c,
+ 0x6400, 0x643c,
+ 0x6500, 0x6524,
+ 0x6a00, 0x6a38,
+ 0x6a60, 0x6a78,
+ 0x6b00, 0x6b84,
+ 0x6bf0, 0x6c84,
+ 0x6cf0, 0x6d84,
+ 0x6df0, 0x6e84,
+ 0x6ef0, 0x6f84,
+ 0x6ff0, 0x7084,
+ 0x70f0, 0x7184,
+ 0x71f0, 0x7284,
+ 0x72f0, 0x7384,
+ 0x73f0, 0x7450,
+ 0x7500, 0x7530,
+ 0x7600, 0x761c,
+ 0x7680, 0x76cc,
+ 0x7700, 0x7798,
+ 0x77c0, 0x77fc,
+ 0x7900, 0x79fc,
+ 0x7b00, 0x7c38,
+ 0x7d00, 0x7efc,
+ 0x8dc0, 0x8e1c,
+ 0x8e30, 0x8e78,
+ 0x8ea0, 0x8f6c,
+ 0x8fc0, 0x9074,
+ 0x90fc, 0x90fc,
+ 0x9400, 0x9458,
+ 0x9600, 0x96bc,
+ 0x9800, 0x9808,
+ 0x9820, 0x983c,
+ 0x9850, 0x9864,
+ 0x9c00, 0x9c6c,
+ 0x9c80, 0x9cec,
+ 0x9d00, 0x9d6c,
+ 0x9d80, 0x9dec,
+ 0x9e00, 0x9e6c,
+ 0x9e80, 0x9eec,
+ 0x9f00, 0x9f6c,
+ 0x9f80, 0x9fec,
+ 0xd004, 0xd03c,
+ 0xdfc0, 0xdfe0,
+ 0xe000, 0xea7c,
+ 0xf000, 0x11190,
+ 0x19040, 0x19124,
+ 0x19150, 0x191b0,
+ 0x191d0, 0x191e8,
+ 0x19238, 0x1924c,
+ 0x193f8, 0x19474,
+ 0x19490, 0x194f8,
+ 0x19800, 0x19f30,
+ 0x1a000, 0x1a06c,
+ 0x1a0b0, 0x1a120,
+ 0x1a128, 0x1a138,
+ 0x1a190, 0x1a1c4,
+ 0x1a1fc, 0x1a1fc,
+ 0x1e040, 0x1e04c,
+ 0x1e240, 0x1e28c,
+ 0x1e2c0, 0x1e2c0,
+ 0x1e2e0, 0x1e2e0,
+ 0x1e300, 0x1e384,
+ 0x1e3c0, 0x1e3c8,
+ 0x1e440, 0x1e44c,
+ 0x1e640, 0x1e68c,
+ 0x1e6c0, 0x1e6c0,
+ 0x1e6e0, 0x1e6e0,
+ 0x1e700, 0x1e784,
+ 0x1e7c0, 0x1e7c8,
+ 0x1e840, 0x1e84c,
+ 0x1ea40, 0x1ea8c,
+ 0x1eac0, 0x1eac0,
+ 0x1eae0, 0x1eae0,
+ 0x1eb00, 0x1eb84,
+ 0x1ebc0, 0x1ebc8,
+ 0x1ec40, 0x1ec4c,
+ 0x1ee40, 0x1ee8c,
+ 0x1eec0, 0x1eec0,
+ 0x1eee0, 0x1eee0,
+ 0x1ef00, 0x1ef84,
+ 0x1efc0, 0x1efc8,
+ 0x1f040, 0x1f04c,
+ 0x1f240, 0x1f28c,
+ 0x1f2c0, 0x1f2c0,
+ 0x1f2e0, 0x1f2e0,
+ 0x1f300, 0x1f384,
+ 0x1f3c0, 0x1f3c8,
+ 0x1f440, 0x1f44c,
+ 0x1f640, 0x1f68c,
+ 0x1f6c0, 0x1f6c0,
+ 0x1f6e0, 0x1f6e0,
+ 0x1f700, 0x1f784,
+ 0x1f7c0, 0x1f7c8,
+ 0x1f840, 0x1f84c,
+ 0x1fa40, 0x1fa8c,
+ 0x1fac0, 0x1fac0,
+ 0x1fae0, 0x1fae0,
+ 0x1fb00, 0x1fb84,
+ 0x1fbc0, 0x1fbc8,
+ 0x1fc40, 0x1fc4c,
+ 0x1fe40, 0x1fe8c,
+ 0x1fec0, 0x1fec0,
+ 0x1fee0, 0x1fee0,
+ 0x1ff00, 0x1ff84,
+ 0x1ffc0, 0x1ffc8,
+ 0x20000, 0x2002c,
+ 0x20100, 0x2013c,
+ 0x20190, 0x201c8,
+ 0x20200, 0x20318,
+ 0x20400, 0x20528,
+ 0x20540, 0x20614,
+ 0x21000, 0x21040,
+ 0x2104c, 0x21060,
+ 0x210c0, 0x210ec,
+ 0x21200, 0x21268,
+ 0x21270, 0x21284,
+ 0x212fc, 0x21388,
+ 0x21400, 0x21404,
+ 0x21500, 0x21518,
+ 0x2152c, 0x2153c,
+ 0x21550, 0x21554,
+ 0x21600, 0x21600,
+ 0x21608, 0x21628,
+ 0x21630, 0x2163c,
+ 0x21700, 0x2171c,
+ 0x21780, 0x2178c,
+ 0x21800, 0x21c38,
+ 0x21c80, 0x21d7c,
+ 0x21e00, 0x21e04,
+ 0x22000, 0x2202c,
+ 0x22100, 0x2213c,
+ 0x22190, 0x221c8,
+ 0x22200, 0x22318,
+ 0x22400, 0x22528,
+ 0x22540, 0x22614,
+ 0x23000, 0x23040,
+ 0x2304c, 0x23060,
+ 0x230c0, 0x230ec,
+ 0x23200, 0x23268,
+ 0x23270, 0x23284,
+ 0x232fc, 0x23388,
+ 0x23400, 0x23404,
+ 0x23500, 0x23518,
+ 0x2352c, 0x2353c,
+ 0x23550, 0x23554,
+ 0x23600, 0x23600,
+ 0x23608, 0x23628,
+ 0x23630, 0x2363c,
+ 0x23700, 0x2371c,
+ 0x23780, 0x2378c,
+ 0x23800, 0x23c38,
+ 0x23c80, 0x23d7c,
+ 0x23e00, 0x23e04,
+ 0x24000, 0x2402c,
+ 0x24100, 0x2413c,
+ 0x24190, 0x241c8,
+ 0x24200, 0x24318,
+ 0x24400, 0x24528,
+ 0x24540, 0x24614,
+ 0x25000, 0x25040,
+ 0x2504c, 0x25060,
+ 0x250c0, 0x250ec,
+ 0x25200, 0x25268,
+ 0x25270, 0x25284,
+ 0x252fc, 0x25388,
+ 0x25400, 0x25404,
+ 0x25500, 0x25518,
+ 0x2552c, 0x2553c,
+ 0x25550, 0x25554,
+ 0x25600, 0x25600,
+ 0x25608, 0x25628,
+ 0x25630, 0x2563c,
+ 0x25700, 0x2571c,
+ 0x25780, 0x2578c,
+ 0x25800, 0x25c38,
+ 0x25c80, 0x25d7c,
+ 0x25e00, 0x25e04,
+ 0x26000, 0x2602c,
+ 0x26100, 0x2613c,
+ 0x26190, 0x261c8,
+ 0x26200, 0x26318,
+ 0x26400, 0x26528,
+ 0x26540, 0x26614,
+ 0x27000, 0x27040,
+ 0x2704c, 0x27060,
+ 0x270c0, 0x270ec,
+ 0x27200, 0x27268,
+ 0x27270, 0x27284,
+ 0x272fc, 0x27388,
+ 0x27400, 0x27404,
+ 0x27500, 0x27518,
+ 0x2752c, 0x2753c,
+ 0x27550, 0x27554,
+ 0x27600, 0x27600,
+ 0x27608, 0x27628,
+ 0x27630, 0x2763c,
+ 0x27700, 0x2771c,
+ 0x27780, 0x2778c,
+ 0x27800, 0x27c38,
+ 0x27c80, 0x27d7c,
+ 0x27e00, 0x27e04
+ };
+
+ int i;
+ struct adapter *ap = netdev2adap(dev);
+
+ regs->version = mk_adap_vers(ap);
+
+ memset(buf, 0, T4_REGMAP_SIZE);
+ for (i = 0; i < ARRAY_SIZE(reg_ranges); i += 2)
+ reg_block_dump(ap, buf, reg_ranges[i], reg_ranges[i + 1]);
+}
+
+static int restart_autoneg(struct net_device *dev)
+{
+ struct port_info *p = netdev_priv(dev);
+
+ if (!netif_running(dev))
+ return -EAGAIN;
+ if (p->link_cfg.autoneg != AUTONEG_ENABLE)
+ return -EINVAL;
+ t4_restart_aneg(p->adapter, 0, p->tx_chan);
+ return 0;
+}
+
+static int identify_port(struct net_device *dev, u32 data)
+{
+ if (data == 0)
+ data = 2; /* default to 2 seconds */
+
+ return t4_identify_port(netdev2adap(dev), 0, netdev2pinfo(dev)->viid,
+ data * 5);
+}
+
+static unsigned int from_fw_linkcaps(unsigned int type, unsigned int caps)
+{
+ unsigned int v = 0;
+
+ if (type == FW_PORT_TYPE_BT_SGMII || type == FW_PORT_TYPE_BT_XAUI) {
+ v |= SUPPORTED_TP;
+ if (caps & FW_PORT_CAP_SPEED_100M)
+ v |= SUPPORTED_100baseT_Full;
+ if (caps & FW_PORT_CAP_SPEED_1G)
+ v |= SUPPORTED_1000baseT_Full;
+ if (caps & FW_PORT_CAP_SPEED_10G)
+ v |= SUPPORTED_10000baseT_Full;
+ } else if (type == FW_PORT_TYPE_KX4 || type == FW_PORT_TYPE_KX) {
+ v |= SUPPORTED_Backplane;
+ if (caps & FW_PORT_CAP_SPEED_1G)
+ v |= SUPPORTED_1000baseKX_Full;
+ if (caps & FW_PORT_CAP_SPEED_10G)
+ v |= SUPPORTED_10000baseKX4_Full;
+ } else if (type == FW_PORT_TYPE_KR)
+ v |= SUPPORTED_Backplane | SUPPORTED_10000baseKR_Full;
+ else if (type == FW_PORT_TYPE_FIBER)
+ v |= SUPPORTED_FIBRE;
+
+ if (caps & FW_PORT_CAP_ANEG)
+ v |= SUPPORTED_Autoneg;
+ return v;
+}
+
+static unsigned int to_fw_linkcaps(unsigned int caps)
+{
+ unsigned int v = 0;
+
+ if (caps & ADVERTISED_100baseT_Full)
+ v |= FW_PORT_CAP_SPEED_100M;
+ if (caps & ADVERTISED_1000baseT_Full)
+ v |= FW_PORT_CAP_SPEED_1G;
+ if (caps & ADVERTISED_10000baseT_Full)
+ v |= FW_PORT_CAP_SPEED_10G;
+ return v;
+}
+
+static int get_settings(struct net_device *dev, struct ethtool_cmd *cmd)
+{
+ const struct port_info *p = netdev_priv(dev);
+
+ if (p->port_type == FW_PORT_TYPE_BT_SGMII ||
+ p->port_type == FW_PORT_TYPE_BT_XAUI)
+ cmd->port = PORT_TP;
+ else if (p->port_type == FW_PORT_TYPE_FIBER)
+ cmd->port = PORT_FIBRE;
+ else if (p->port_type == FW_PORT_TYPE_TWINAX)
+ cmd->port = PORT_DA;
+ else
+ cmd->port = PORT_OTHER;
+
+ if (p->mdio_addr >= 0) {
+ cmd->phy_address = p->mdio_addr;
+ cmd->transceiver = XCVR_EXTERNAL;
+ cmd->mdio_support = p->port_type == FW_PORT_TYPE_BT_SGMII ?
+ MDIO_SUPPORTS_C22 : MDIO_SUPPORTS_C45;
+ } else {
+ cmd->phy_address = 0; /* not really, but no better option */
+ cmd->transceiver = XCVR_INTERNAL;
+ cmd->mdio_support = 0;
+ }
+
+ cmd->supported = from_fw_linkcaps(p->port_type, p->link_cfg.supported);
+ cmd->advertising = from_fw_linkcaps(p->port_type,
+ p->link_cfg.advertising);
+ cmd->speed = netif_carrier_ok(dev) ? p->link_cfg.speed : 0;
+ cmd->duplex = DUPLEX_FULL;
+ cmd->autoneg = p->link_cfg.autoneg;
+ cmd->maxtxpkt = 0;
+ cmd->maxrxpkt = 0;
+ return 0;
+}
+
+static unsigned int speed_to_caps(int speed)
+{
+ if (speed == SPEED_100)
+ return FW_PORT_CAP_SPEED_100M;
+ if (speed == SPEED_1000)
+ return FW_PORT_CAP_SPEED_1G;
+ if (speed == SPEED_10000)
+ return FW_PORT_CAP_SPEED_10G;
+ return 0;
+}
+
+static int set_settings(struct net_device *dev, struct ethtool_cmd *cmd)
+{
+ unsigned int cap;
+ struct port_info *p = netdev_priv(dev);
+ struct link_config *lc = &p->link_cfg;
+
+ if (cmd->duplex != DUPLEX_FULL) /* only full-duplex supported */
+ return -EINVAL;
+
+ if (!(lc->supported & FW_PORT_CAP_ANEG)) {
+ /*
+ * PHY offers a single speed. See if that's what's
+ * being requested.
+ */
+ if (cmd->autoneg == AUTONEG_DISABLE &&
+ (lc->supported & speed_to_caps(cmd->speed)))
+ return 0;
+ return -EINVAL;
+ }
+
+ if (cmd->autoneg == AUTONEG_DISABLE) {
+ cap = speed_to_caps(cmd->speed);
+
+ if (!(lc->supported & cap) || cmd->speed == SPEED_1000 ||
+ cmd->speed == SPEED_10000)
+ return -EINVAL;
+ lc->requested_speed = cap;
+ lc->advertising = 0;
+ } else {
+ cap = to_fw_linkcaps(cmd->advertising);
+ if (!(lc->supported & cap))
+ return -EINVAL;
+ lc->requested_speed = 0;
+ lc->advertising = cap | FW_PORT_CAP_ANEG;
+ }
+ lc->autoneg = cmd->autoneg;
+
+ if (netif_running(dev))
+ return t4_link_start(p->adapter, 0, p->tx_chan, lc);
+ return 0;
+}
+
+static void get_pauseparam(struct net_device *dev,
+ struct ethtool_pauseparam *epause)
+{
+ struct port_info *p = netdev_priv(dev);
+
+ epause->autoneg = (p->link_cfg.requested_fc & PAUSE_AUTONEG) != 0;
+ epause->rx_pause = (p->link_cfg.fc & PAUSE_RX) != 0;
+ epause->tx_pause = (p->link_cfg.fc & PAUSE_TX) != 0;
+}
+
+static int set_pauseparam(struct net_device *dev,
+ struct ethtool_pauseparam *epause)
+{
+ struct port_info *p = netdev_priv(dev);
+ struct link_config *lc = &p->link_cfg;
+
+ if (epause->autoneg == AUTONEG_DISABLE)
+ lc->requested_fc = 0;
+ else if (lc->supported & FW_PORT_CAP_ANEG)
+ lc->requested_fc = PAUSE_AUTONEG;
+ else
+ return -EINVAL;
+
+ if (epause->rx_pause)
+ lc->requested_fc |= PAUSE_RX;
+ if (epause->tx_pause)
+ lc->requested_fc |= PAUSE_TX;
+ if (netif_running(dev))
+ return t4_link_start(p->adapter, 0, p->tx_chan, lc);
+ return 0;
+}
+
+static u32 get_rx_csum(struct net_device *dev)
+{
+ struct port_info *p = netdev_priv(dev);
+
+ return p->rx_offload & RX_CSO;
+}
+
+static int set_rx_csum(struct net_device *dev, u32 data)
+{
+ struct port_info *p = netdev_priv(dev);
+
+ if (data)
+ p->rx_offload |= RX_CSO;
+ else
+ p->rx_offload &= ~RX_CSO;
+ return 0;
+}
+
+static void get_sge_param(struct net_device *dev, struct ethtool_ringparam *e)
+{
+ const struct port_info *pi = netdev_priv(dev);
+ const struct sge *s = &pi->adapter->sge;
+
+ e->rx_max_pending = MAX_RX_BUFFERS;
+ e->rx_mini_max_pending = MAX_RSPQ_ENTRIES;
+ e->rx_jumbo_max_pending = 0;
+ e->tx_max_pending = MAX_TXQ_ENTRIES;
+
+ e->rx_pending = s->ethrxq[pi->first_qset].fl.size - 8;
+ e->rx_mini_pending = s->ethrxq[pi->first_qset].rspq.size;
+ e->rx_jumbo_pending = 0;
+ e->tx_pending = s->ethtxq[pi->first_qset].q.size;
+}
+
+static int set_sge_param(struct net_device *dev, struct ethtool_ringparam *e)
+{
+ int i;
+ const struct port_info *pi = netdev_priv(dev);
+ struct adapter *adapter = pi->adapter;
+ struct sge *s = &adapter->sge;
+
+ if (e->rx_pending > MAX_RX_BUFFERS || e->rx_jumbo_pending ||
+ e->tx_pending > MAX_TXQ_ENTRIES ||
+ e->rx_mini_pending > MAX_RSPQ_ENTRIES ||
+ e->rx_mini_pending < MIN_RSPQ_ENTRIES ||
+ e->rx_pending < MIN_FL_ENTRIES || e->tx_pending < MIN_TXQ_ENTRIES)
+ return -EINVAL;
+
+ if (adapter->flags & FULL_INIT_DONE)
+ return -EBUSY;
+
+ for (i = 0; i < pi->nqsets; ++i) {
+ s->ethtxq[pi->first_qset + i].q.size = e->tx_pending;
+ s->ethrxq[pi->first_qset + i].fl.size = e->rx_pending + 8;
+ s->ethrxq[pi->first_qset + i].rspq.size = e->rx_mini_pending;
+ }
+ return 0;
+}
+
+static int closest_timer(const struct sge *s, int time)
+{
+ int i, delta, match = 0, min_delta = INT_MAX;
+
+ for (i = 0; i < ARRAY_SIZE(s->timer_val); i++) {
+ delta = time - s->timer_val[i];
+ if (delta < 0)
+ delta = -delta;
+ if (delta < min_delta) {
+ min_delta = delta;
+ match = i;
+ }
+ }
+ return match;
+}
+
+static int closest_thres(const struct sge *s, int thres)
+{
+ int i, delta, match = 0, min_delta = INT_MAX;
+
+ for (i = 0; i < ARRAY_SIZE(s->counter_val); i++) {
+ delta = thres - s->counter_val[i];
+ if (delta < 0)
+ delta = -delta;
+ if (delta < min_delta) {
+ min_delta = delta;
+ match = i;
+ }
+ }
+ return match;
+}
+
+/*
+ * Return a queue's interrupt hold-off time in us. 0 means no timer.
+ */
+static unsigned int qtimer_val(const struct adapter *adap,
+ const struct sge_rspq *q)
+{
+ unsigned int idx = q->intr_params >> 1;
+
+ return idx < SGE_NTIMERS ? adap->sge.timer_val[idx] : 0;
+}
+
+/**
+ * set_rxq_intr_params - set a queue's interrupt holdoff parameters
+ * @adap: the adapter
+ * @q: the Rx queue
+ * @us: the hold-off time in us, or 0 to disable timer
+ * @cnt: the hold-off packet count, or 0 to disable counter
+ *
+ * Sets an Rx queue's interrupt hold-off time and packet count. At least
+ * one of the two needs to be enabled for the queue to generate interrupts.
+ */
+static int set_rxq_intr_params(struct adapter *adap, struct sge_rspq *q,
+ unsigned int us, unsigned int cnt)
+{
+ if ((us | cnt) == 0)
+ cnt = 1;
+
+ if (cnt) {
+ int err;
+ u32 v, new_idx;
+
+ new_idx = closest_thres(&adap->sge, cnt);
+ if (q->desc && q->pktcnt_idx != new_idx) {
+ /* the queue has already been created, update it */
+ v = FW_PARAMS_MNEM(FW_PARAMS_MNEM_DMAQ) |
+ FW_PARAMS_PARAM_X(FW_PARAMS_PARAM_DMAQ_IQ_INTCNTTHRESH) |
+ FW_PARAMS_PARAM_YZ(q->cntxt_id);
+ err = t4_set_params(adap, 0, 0, 0, 1, &v, &new_idx);
+ if (err)
+ return err;
+ }
+ q->pktcnt_idx = new_idx;
+ }
+
+ us = us == 0 ? 6 : closest_timer(&adap->sge, us);
+ q->intr_params = QINTR_TIMER_IDX(us) | (cnt > 0 ? QINTR_CNT_EN : 0);
+ return 0;
+}
+
+static int set_coalesce(struct net_device *dev, struct ethtool_coalesce *c)
+{
+ const struct port_info *pi = netdev_priv(dev);
+ struct adapter *adap = pi->adapter;
+
+ return set_rxq_intr_params(adap, &adap->sge.ethrxq[pi->first_qset].rspq,
+ c->rx_coalesce_usecs, c->rx_max_coalesced_frames);
+}
+
+static int get_coalesce(struct net_device *dev, struct ethtool_coalesce *c)
+{
+ const struct port_info *pi = netdev_priv(dev);
+ const struct adapter *adap = pi->adapter;
+ const struct sge_rspq *rq = &adap->sge.ethrxq[pi->first_qset].rspq;
+
+ c->rx_coalesce_usecs = qtimer_val(adap, rq);
+ c->rx_max_coalesced_frames = (rq->intr_params & QINTR_CNT_EN) ?
+ adap->sge.counter_val[rq->pktcnt_idx] : 0;
+ return 0;
+}
+
+/*
+ * Translate a physical EEPROM address to virtual. The first 1K is accessed
+ * through virtual addresses starting at 31K, the rest is accessed through
+ * virtual addresses starting at 0. This mapping is correct only for PF0.
+ */
+static int eeprom_ptov(unsigned int phys_addr)
+{
+ if (phys_addr < 1024)
+ return phys_addr + (31 << 10);
+ if (phys_addr < EEPROMSIZE)
+ return phys_addr - 1024;
+ return -EINVAL;
+}
+
+/*
+ * The next two routines implement eeprom read/write from physical addresses.
+ * The physical->virtual translation is correct only for PF0.
+ */
+static int eeprom_rd_phys(struct adapter *adap, unsigned int phys_addr, u32 *v)
+{
+ int vaddr = eeprom_ptov(phys_addr);
+
+ if (vaddr >= 0)
+ vaddr = pci_read_vpd(adap->pdev, vaddr, sizeof(u32), v);
+ return vaddr < 0 ? vaddr : 0;
+}
+
+static int eeprom_wr_phys(struct adapter *adap, unsigned int phys_addr, u32 v)
+{
+ int vaddr = eeprom_ptov(phys_addr);
+
+ if (vaddr >= 0)
+ vaddr = pci_write_vpd(adap->pdev, vaddr, sizeof(u32), &v);
+ return vaddr < 0 ? vaddr : 0;
+}
+
+#define EEPROM_MAGIC 0x38E2F10C
+
+static int get_eeprom(struct net_device *dev, struct ethtool_eeprom *e,
+ u8 *data)
+{
+ int i, err = 0;
+ struct adapter *adapter = netdev2adap(dev);
+
+ u8 *buf = kmalloc(EEPROMSIZE, GFP_KERNEL);
+ if (!buf)
+ return -ENOMEM;
+
+ e->magic = EEPROM_MAGIC;
+ for (i = e->offset & ~3; !err && i < e->offset + e->len; i += 4)
+ err = eeprom_rd_phys(adapter, i, (u32 *)&buf[i]);
+
+ if (!err)
+ memcpy(data, buf + e->offset, e->len);
+ kfree(buf);
+ return err;
+}
+
+static int set_eeprom(struct net_device *dev, struct ethtool_eeprom *eeprom,
+ u8 *data)
+{
+ u8 *buf;
+ int err = 0;
+ u32 aligned_offset, aligned_len, *p;
+ struct adapter *adapter = netdev2adap(dev);
+
+ if (eeprom->magic != EEPROM_MAGIC)
+ return -EINVAL;
+
+ aligned_offset = eeprom->offset & ~3;
+ aligned_len = (eeprom->len + (eeprom->offset & 3) + 3) & ~3;
+
+ if (aligned_offset != eeprom->offset || aligned_len != eeprom->len) {
+ /*
+ * RMW possibly needed for first or last words.
+ */
+ buf = kmalloc(aligned_len, GFP_KERNEL);
+ if (!buf)
+ return -ENOMEM;
+ err = eeprom_rd_phys(adapter, aligned_offset, (u32 *)buf);
+ if (!err && aligned_len > 4)
+ err = eeprom_rd_phys(adapter,
+ aligned_offset + aligned_len - 4,
+ (u32 *)&buf[aligned_len - 4]);
+ if (err)
+ goto out;
+ memcpy(buf + (eeprom->offset & 3), data, eeprom->len);
+ } else
+ buf = data;
+
+ err = t4_seeprom_wp(adapter, false);
+ if (err)
+ goto out;
+
+ for (p = (u32 *)buf; !err && aligned_len; aligned_len -= 4, p++) {
+ err = eeprom_wr_phys(adapter, aligned_offset, *p);
+ aligned_offset += 4;
+ }
+
+ if (!err)
+ err = t4_seeprom_wp(adapter, true);
+out:
+ if (buf != data)
+ kfree(buf);
+ return err;
+}
+
+static int set_flash(struct net_device *netdev, struct ethtool_flash *ef)
+{
+ int ret;
+ const struct firmware *fw;
+ struct adapter *adap = netdev2adap(netdev);
+
+ ef->data[sizeof(ef->data) - 1] = '\0';
+ ret = request_firmware(&fw, ef->data, adap->pdev_dev);
+ if (ret < 0)
+ return ret;
+
+ ret = t4_load_fw(adap, fw->data, fw->size);
+ release_firmware(fw);
+ if (!ret)
+ dev_info(adap->pdev_dev, "loaded firmware %s\n", ef->data);
+ return ret;
+}
+
+#define WOL_SUPPORTED (WAKE_BCAST | WAKE_MAGIC)
+#define BCAST_CRC 0xa0ccc1a6
+
+static void get_wol(struct net_device *dev, struct ethtool_wolinfo *wol)
+{
+ wol->supported = WAKE_BCAST | WAKE_MAGIC;
+ wol->wolopts = netdev2adap(dev)->wol;
+ memset(&wol->sopass, 0, sizeof(wol->sopass));
+}
+
+static int set_wol(struct net_device *dev, struct ethtool_wolinfo *wol)
+{
+ int err = 0;
+ struct port_info *pi = netdev_priv(dev);
+
+ if (wol->wolopts & ~WOL_SUPPORTED)
+ return -EINVAL;
+ t4_wol_magic_enable(pi->adapter, pi->tx_chan,
+ (wol->wolopts & WAKE_MAGIC) ? dev->dev_addr : NULL);
+ if (wol->wolopts & WAKE_BCAST) {
+ err = t4_wol_pat_enable(pi->adapter, pi->tx_chan, 0xfe, ~0ULL,
+ ~0ULL, 0, false);
+ if (!err)
+ err = t4_wol_pat_enable(pi->adapter, pi->tx_chan, 1,
+ ~6ULL, ~0ULL, BCAST_CRC, true);
+ } else
+ t4_wol_pat_enable(pi->adapter, pi->tx_chan, 0, 0, 0, 0, false);
+ return err;
+}
+
+static int set_tso(struct net_device *dev, u32 value)
+{
+ if (value)
+ dev->features |= NETIF_F_TSO | NETIF_F_TSO6;
+ else
+ dev->features &= ~(NETIF_F_TSO | NETIF_F_TSO6);
+ return 0;
+}
+
+static struct ethtool_ops cxgb_ethtool_ops = {
+ .get_settings = get_settings,
+ .set_settings = set_settings,
+ .get_drvinfo = get_drvinfo,
+ .get_msglevel = get_msglevel,
+ .set_msglevel = set_msglevel,
+ .get_ringparam = get_sge_param,
+ .set_ringparam = set_sge_param,
+ .get_coalesce = get_coalesce,
+ .set_coalesce = set_coalesce,
+ .get_eeprom_len = get_eeprom_len,
+ .get_eeprom = get_eeprom,
+ .set_eeprom = set_eeprom,
+ .get_pauseparam = get_pauseparam,
+ .set_pauseparam = set_pauseparam,
+ .get_rx_csum = get_rx_csum,
+ .set_rx_csum = set_rx_csum,
+ .set_tx_csum = ethtool_op_set_tx_ipv6_csum,
+ .set_sg = ethtool_op_set_sg,
+ .get_link = ethtool_op_get_link,
+ .get_strings = get_strings,
+ .phys_id = identify_port,
+ .nway_reset = restart_autoneg,
+ .get_sset_count = get_sset_count,
+ .get_ethtool_stats = get_stats,
+ .get_regs_len = get_regs_len,
+ .get_regs = get_regs,
+ .get_wol = get_wol,
+ .set_wol = set_wol,
+ .set_tso = set_tso,
+ .flash_device = set_flash,
+};
+
+/*
+ * debugfs support
+ */
+
+static int mem_open(struct inode *inode, struct file *file)
+{
+ file->private_data = inode->i_private;
+ return 0;
+}
+
+static ssize_t mem_read(struct file *file, char __user *buf, size_t count,
+ loff_t *ppos)
+{
+ loff_t pos = *ppos;
+ loff_t avail = file->f_path.dentry->d_inode->i_size;
+ unsigned int mem = (uintptr_t)file->private_data & 3;
+ struct adapter *adap = file->private_data - mem;
+
+ if (pos < 0)
+ return -EINVAL;
+ if (pos >= avail)
+ return 0;
+ if (count > avail - pos)
+ count = avail - pos;
+
+ while (count) {
+ size_t len;
+ int ret, ofst;
+ __be32 data[16];
+
+ if (mem == MEM_MC)
+ ret = t4_mc_read(adap, pos, data, NULL);
+ else
+ ret = t4_edc_read(adap, mem, pos, data, NULL);
+ if (ret)
+ return ret;
+
+ ofst = pos % sizeof(data);
+ len = min(count, sizeof(data) - ofst);
+ if (copy_to_user(buf, (u8 *)data + ofst, len))
+ return -EFAULT;
+
+ buf += len;
+ pos += len;
+ count -= len;
+ }
+ count = pos - *ppos;
+ *ppos = pos;
+ return count;
+}
+
+static const struct file_operations mem_debugfs_fops = {
+ .owner = THIS_MODULE,
+ .open = mem_open,
+ .read = mem_read,
+};
+
+static void __devinit add_debugfs_mem(struct adapter *adap, const char *name,
+ unsigned int idx, unsigned int size_mb)
+{
+ struct dentry *de;
+
+ de = debugfs_create_file(name, S_IRUSR, adap->debugfs_root,
+ (void *)adap + idx, &mem_debugfs_fops);
+ if (de && de->d_inode)
+ de->d_inode->i_size = size_mb << 20;
+}
+
+static int __devinit setup_debugfs(struct adapter *adap)
+{
+ int i;
+
+ if (IS_ERR_OR_NULL(adap->debugfs_root))
+ return -1;
+
+ i = t4_read_reg(adap, MA_TARGET_MEM_ENABLE);
+ if (i & EDRAM0_ENABLE)
+ add_debugfs_mem(adap, "edc0", MEM_EDC0, 5);
+ if (i & EDRAM1_ENABLE)
+ add_debugfs_mem(adap, "edc1", MEM_EDC1, 5);
+ if (i & EXT_MEM_ENABLE)
+ add_debugfs_mem(adap, "mc", MEM_MC,
+ EXT_MEM_SIZE_GET(t4_read_reg(adap, MA_EXT_MEMORY_BAR)));
+ if (adap->l2t)
+ debugfs_create_file("l2t", S_IRUSR, adap->debugfs_root, adap,
+ &t4_l2t_fops);
+ return 0;
+}
+
+/*
+ * upper-layer driver support
+ */
+
+/*
+ * Allocate an active-open TID and set it to the supplied value.
+ */
+int cxgb4_alloc_atid(struct tid_info *t, void *data)
+{
+ int atid = -1;
+
+ spin_lock_bh(&t->atid_lock);
+ if (t->afree) {
+ union aopen_entry *p = t->afree;
+
+ atid = p - t->atid_tab;
+ t->afree = p->next;
+ p->data = data;
+ t->atids_in_use++;
+ }
+ spin_unlock_bh(&t->atid_lock);
+ return atid;
+}
+EXPORT_SYMBOL(cxgb4_alloc_atid);
+
+/*
+ * Release an active-open TID.
+ */
+void cxgb4_free_atid(struct tid_info *t, unsigned int atid)
+{
+ union aopen_entry *p = &t->atid_tab[atid];
+
+ spin_lock_bh(&t->atid_lock);
+ p->next = t->afree;
+ t->afree = p;
+ t->atids_in_use--;
+ spin_unlock_bh(&t->atid_lock);
+}
+EXPORT_SYMBOL(cxgb4_free_atid);
+
+/*
+ * Allocate a server TID and set it to the supplied value.
+ */
+int cxgb4_alloc_stid(struct tid_info *t, int family, void *data)
+{
+ int stid;
+
+ spin_lock_bh(&t->stid_lock);
+ if (family == PF_INET) {
+ stid = find_first_zero_bit(t->stid_bmap, t->nstids);
+ if (stid < t->nstids)
+ __set_bit(stid, t->stid_bmap);
+ else
+ stid = -1;
+ } else {
+ stid = bitmap_find_free_region(t->stid_bmap, t->nstids, 2);
+ if (stid < 0)
+ stid = -1;
+ }
+ if (stid >= 0) {
+ t->stid_tab[stid].data = data;
+ stid += t->stid_base;
+ t->stids_in_use++;
+ }
+ spin_unlock_bh(&t->stid_lock);
+ return stid;
+}
+EXPORT_SYMBOL(cxgb4_alloc_stid);
+
+/*
+ * Release a server TID.
+ */
+void cxgb4_free_stid(struct tid_info *t, unsigned int stid, int family)
+{
+ stid -= t->stid_base;
+ spin_lock_bh(&t->stid_lock);
+ if (family == PF_INET)
+ __clear_bit(stid, t->stid_bmap);
+ else
+ bitmap_release_region(t->stid_bmap, stid, 2);
+ t->stid_tab[stid].data = NULL;
+ t->stids_in_use--;
+ spin_unlock_bh(&t->stid_lock);
+}
+EXPORT_SYMBOL(cxgb4_free_stid);
+
+/*
+ * Populate a TID_RELEASE WR. Caller must properly size the skb.
+ */
+static void mk_tid_release(struct sk_buff *skb, unsigned int chan,
+ unsigned int tid)
+{
+ struct cpl_tid_release *req;
+
+ set_wr_txq(skb, CPL_PRIORITY_SETUP, chan);
+ req = (struct cpl_tid_release *)__skb_put(skb, sizeof(*req));
+ INIT_TP_WR(req, tid);
+ OPCODE_TID(req) = htonl(MK_OPCODE_TID(CPL_TID_RELEASE, tid));
+}
+
+/*
+ * Queue a TID release request and if necessary schedule a work queue to
+ * process it.
+ */
+void cxgb4_queue_tid_release(struct tid_info *t, unsigned int chan,
+ unsigned int tid)
+{
+ void **p = &t->tid_tab[tid];
+ struct adapter *adap = container_of(t, struct adapter, tids);
+
+ spin_lock_bh(&adap->tid_release_lock);
+ *p = adap->tid_release_head;
+ /* Low 2 bits encode the Tx channel number */
+ adap->tid_release_head = (void **)((uintptr_t)p | chan);
+ if (!adap->tid_release_task_busy) {
+ adap->tid_release_task_busy = true;
+ schedule_work(&adap->tid_release_task);
+ }
+ spin_unlock_bh(&adap->tid_release_lock);
+}
+EXPORT_SYMBOL(cxgb4_queue_tid_release);
+
+/*
+ * Process the list of pending TID release requests.
+ */
+static void process_tid_release_list(struct work_struct *work)
+{
+ struct sk_buff *skb;
+ struct adapter *adap;
+
+ adap = container_of(work, struct adapter, tid_release_task);
+
+ spin_lock_bh(&adap->tid_release_lock);
+ while (adap->tid_release_head) {
+ void **p = adap->tid_release_head;
+ unsigned int chan = (uintptr_t)p & 3;
+ p = (void *)p - chan;
+
+ adap->tid_release_head = *p;
+ *p = NULL;
+ spin_unlock_bh(&adap->tid_release_lock);
+
+ while (!(skb = alloc_skb(sizeof(struct cpl_tid_release),
+ GFP_KERNEL)))
+ schedule_timeout_uninterruptible(1);
+
+ mk_tid_release(skb, chan, p - adap->tids.tid_tab);
+ t4_ofld_send(adap, skb);
+ spin_lock_bh(&adap->tid_release_lock);
+ }
+ adap->tid_release_task_busy = false;
+ spin_unlock_bh(&adap->tid_release_lock);
+}
+
+/*
+ * Release a TID and inform HW. If we are unable to allocate the release
+ * message we defer to a work queue.
+ */
+void cxgb4_remove_tid(struct tid_info *t, unsigned int chan, unsigned int tid)
+{
+ void *old;
+ struct sk_buff *skb;
+ struct adapter *adap = container_of(t, struct adapter, tids);
+
+ old = t->tid_tab[tid];
+ skb = alloc_skb(sizeof(struct cpl_tid_release), GFP_ATOMIC);
+ if (likely(skb)) {
+ t->tid_tab[tid] = NULL;
+ mk_tid_release(skb, chan, tid);
+ t4_ofld_send(adap, skb);
+ } else
+ cxgb4_queue_tid_release(t, chan, tid);
+ if (old)
+ atomic_dec(&t->tids_in_use);
+}
+EXPORT_SYMBOL(cxgb4_remove_tid);
+
+/*
+ * Allocate and initialize the TID tables. Returns 0 on success.
+ */
+static int tid_init(struct tid_info *t)
+{
+ size_t size;
+ unsigned int natids = t->natids;
+
+ size = t->ntids * sizeof(*t->tid_tab) + natids * sizeof(*t->atid_tab) +
+ t->nstids * sizeof(*t->stid_tab) +
+ BITS_TO_LONGS(t->nstids) * sizeof(long);
+ t->tid_tab = t4_alloc_mem(size);
+ if (!t->tid_tab)
+ return -ENOMEM;
+
+ t->atid_tab = (union aopen_entry *)&t->tid_tab[t->ntids];
+ t->stid_tab = (struct serv_entry *)&t->atid_tab[natids];
+ t->stid_bmap = (unsigned long *)&t->stid_tab[t->nstids];
+ spin_lock_init(&t->stid_lock);
+ spin_lock_init(&t->atid_lock);
+
+ t->stids_in_use = 0;
+ t->afree = NULL;
+ t->atids_in_use = 0;
+ atomic_set(&t->tids_in_use, 0);
+
+ /* Setup the free list for atid_tab and clear the stid bitmap. */
+ if (natids) {
+ while (--natids)
+ t->atid_tab[natids - 1].next = &t->atid_tab[natids];
+ t->afree = t->atid_tab;
+ }
+ bitmap_zero(t->stid_bmap, t->nstids);
+ return 0;
+}
+
+/**
+ * cxgb4_create_server - create an IP server
+ * @dev: the device
+ * @stid: the server TID
+ * @sip: local IP address to bind server to
+ * @sport: the server's TCP port
+ * @queue: queue to direct messages from this server to
+ *
+ * Create an IP server for the given port and address.
+ * Returns <0 on error and one of the %NET_XMIT_* values on success.
+ */
+int cxgb4_create_server(const struct net_device *dev, unsigned int stid,
+ __be32 sip, __be16 sport, unsigned int queue)
+{
+ unsigned int chan;
+ struct sk_buff *skb;
+ struct adapter *adap;
+ struct cpl_pass_open_req *req;
+
+ skb = alloc_skb(sizeof(*req), GFP_KERNEL);
+ if (!skb)
+ return -ENOMEM;
+
+ adap = netdev2adap(dev);
+ req = (struct cpl_pass_open_req *)__skb_put(skb, sizeof(*req));
+ INIT_TP_WR(req, 0);
+ OPCODE_TID(req) = htonl(MK_OPCODE_TID(CPL_PASS_OPEN_REQ, stid));
+ req->local_port = sport;
+ req->peer_port = htons(0);
+ req->local_ip = sip;
+ req->peer_ip = htonl(0);
+ chan = netdev2pinfo(adap->sge.ingr_map[queue]->netdev)->tx_chan;
+ req->opt0 = cpu_to_be64(TX_CHAN(chan));
+ req->opt1 = cpu_to_be64(CONN_POLICY_ASK |
+ SYN_RSS_ENABLE | SYN_RSS_QUEUE(queue));
+ return t4_mgmt_tx(adap, skb);
+}
+EXPORT_SYMBOL(cxgb4_create_server);
+
+/**
+ * cxgb4_create_server6 - create an IPv6 server
+ * @dev: the device
+ * @stid: the server TID
+ * @sip: local IPv6 address to bind server to
+ * @sport: the server's TCP port
+ * @queue: queue to direct messages from this server to
+ *
+ * Create an IPv6 server for the given port and address.
+ * Returns <0 on error and one of the %NET_XMIT_* values on success.
+ */
+int cxgb4_create_server6(const struct net_device *dev, unsigned int stid,
+ const struct in6_addr *sip, __be16 sport,
+ unsigned int queue)
+{
+ unsigned int chan;
+ struct sk_buff *skb;
+ struct adapter *adap;
+ struct cpl_pass_open_req6 *req;
+
+ skb = alloc_skb(sizeof(*req), GFP_KERNEL);
+ if (!skb)
+ return -ENOMEM;
+
+ adap = netdev2adap(dev);
+ req = (struct cpl_pass_open_req6 *)__skb_put(skb, sizeof(*req));
+ INIT_TP_WR(req, 0);
+ OPCODE_TID(req) = htonl(MK_OPCODE_TID(CPL_PASS_OPEN_REQ6, stid));
+ req->local_port = sport;
+ req->peer_port = htons(0);
+ req->local_ip_hi = *(__be64 *)(sip->s6_addr);
+ req->local_ip_lo = *(__be64 *)(sip->s6_addr + 8);
+ req->peer_ip_hi = cpu_to_be64(0);
+ req->peer_ip_lo = cpu_to_be64(0);
+ chan = netdev2pinfo(adap->sge.ingr_map[queue]->netdev)->tx_chan;
+ req->opt0 = cpu_to_be64(TX_CHAN(chan));
+ req->opt1 = cpu_to_be64(CONN_POLICY_ASK |
+ SYN_RSS_ENABLE | SYN_RSS_QUEUE(queue));
+ return t4_mgmt_tx(adap, skb);
+}
+EXPORT_SYMBOL(cxgb4_create_server6);
+
+/**
+ * cxgb4_best_mtu - find the entry in the MTU table closest to an MTU
+ * @mtus: the HW MTU table
+ * @mtu: the target MTU
+ * @idx: index of selected entry in the MTU table
+ *
+ * Returns the index and the value in the HW MTU table that is closest to
+ * but does not exceed @mtu, unless @mtu is smaller than any value in the
+ * table, in which case that smallest available value is selected.
+ */
+unsigned int cxgb4_best_mtu(const unsigned short *mtus, unsigned short mtu,
+ unsigned int *idx)
+{
+ unsigned int i = 0;
+
+ while (i < NMTUS - 1 && mtus[i + 1] <= mtu)
+ ++i;
+ if (idx)
+ *idx = i;
+ return mtus[i];
+}
+EXPORT_SYMBOL(cxgb4_best_mtu);
+
+/**
+ * cxgb4_port_chan - get the HW channel of a port
+ * @dev: the net device for the port
+ *
+ * Return the HW Tx channel of the given port.
+ */
+unsigned int cxgb4_port_chan(const struct net_device *dev)
+{
+ return netdev2pinfo(dev)->tx_chan;
+}
+EXPORT_SYMBOL(cxgb4_port_chan);
+
+/**
+ * cxgb4_port_viid - get the VI id of a port
+ * @dev: the net device for the port
+ *
+ * Return the VI id of the given port.
+ */
+unsigned int cxgb4_port_viid(const struct net_device *dev)
+{
+ return netdev2pinfo(dev)->viid;
+}
+EXPORT_SYMBOL(cxgb4_port_viid);
+
+/**
+ * cxgb4_port_idx - get the index of a port
+ * @dev: the net device for the port
+ *
+ * Return the index of the given port.
+ */
+unsigned int cxgb4_port_idx(const struct net_device *dev)
+{
+ return netdev2pinfo(dev)->port_id;
+}
+EXPORT_SYMBOL(cxgb4_port_idx);
+
+/**
+ * cxgb4_netdev_by_hwid - return the net device of a HW port
+ * @pdev: identifies the adapter
+ * @id: the HW port id
+ *
+ * Return the net device associated with the interface with the given HW
+ * id.
+ */
+struct net_device *cxgb4_netdev_by_hwid(struct pci_dev *pdev, unsigned int id)
+{
+ const struct adapter *adap = pci_get_drvdata(pdev);
+
+ if (!adap || id >= NCHAN)
+ return NULL;
+ id = adap->chan_map[id];
+ return id < MAX_NPORTS ? adap->port[id] : NULL;
+}
+EXPORT_SYMBOL(cxgb4_netdev_by_hwid);
+
+void cxgb4_get_tcp_stats(struct pci_dev *pdev, struct tp_tcp_stats *v4,
+ struct tp_tcp_stats *v6)
+{
+ struct adapter *adap = pci_get_drvdata(pdev);
+
+ spin_lock(&adap->stats_lock);
+ t4_tp_get_tcp_stats(adap, v4, v6);
+ spin_unlock(&adap->stats_lock);
+}
+EXPORT_SYMBOL(cxgb4_get_tcp_stats);
+
+void cxgb4_iscsi_init(struct net_device *dev, unsigned int tag_mask,
+ const unsigned int *pgsz_order)
+{
+ struct adapter *adap = netdev2adap(dev);
+
+ t4_write_reg(adap, ULP_RX_ISCSI_TAGMASK, tag_mask);
+ t4_write_reg(adap, ULP_RX_ISCSI_PSZ, HPZ0(pgsz_order[0]) |
+ HPZ1(pgsz_order[1]) | HPZ2(pgsz_order[2]) |
+ HPZ3(pgsz_order[3]));
+}
+EXPORT_SYMBOL(cxgb4_iscsi_init);
+
+static struct pci_driver cxgb4_driver;
+
+static void check_neigh_update(struct neighbour *neigh)
+{
+ const struct device *parent;
+ const struct net_device *netdev = neigh->dev;
+
+ if (netdev->priv_flags & IFF_802_1Q_VLAN)
+ netdev = vlan_dev_real_dev(netdev);
+ parent = netdev->dev.parent;
+ if (parent && parent->driver == &cxgb4_driver.driver)
+ t4_l2t_update(dev_get_drvdata(parent), neigh);
+}
+
+static int netevent_cb(struct notifier_block *nb, unsigned long event,
+ void *data)
+{
+ switch (event) {
+ case NETEVENT_NEIGH_UPDATE:
+ check_neigh_update(data);
+ break;
+ case NETEVENT_PMTU_UPDATE:
+ case NETEVENT_REDIRECT:
+ default:
+ break;
+ }
+ return 0;
+}
+
+static bool netevent_registered;
+static struct notifier_block cxgb4_netevent_nb = {
+ .notifier_call = netevent_cb
+};
+
+static void uld_attach(struct adapter *adap, unsigned int uld)
+{
+ void *handle;
+ struct cxgb4_lld_info lli;
+
+ lli.pdev = adap->pdev;
+ lli.l2t = adap->l2t;
+ lli.tids = &adap->tids;
+ lli.ports = adap->port;
+ lli.vr = &adap->vres;
+ lli.mtus = adap->params.mtus;
+ if (uld == CXGB4_ULD_RDMA) {
+ lli.rxq_ids = adap->sge.rdma_rxq;
+ lli.nrxq = adap->sge.rdmaqs;
+ } else if (uld == CXGB4_ULD_ISCSI) {
+ lli.rxq_ids = adap->sge.ofld_rxq;
+ lli.nrxq = adap->sge.ofldqsets;
+ }
+ lli.ntxq = adap->sge.ofldqsets;
+ lli.nchan = adap->params.nports;
+ lli.nports = adap->params.nports;
+ lli.wr_cred = adap->params.ofldq_wr_cred;
+ lli.adapter_type = adap->params.rev;
+ lli.iscsi_iolen = MAXRXDATA_GET(t4_read_reg(adap, TP_PARA_REG2));
+ lli.udb_density = 1 << QUEUESPERPAGEPF0_GET(
+ t4_read_reg(adap, SGE_EGRESS_QUEUES_PER_PAGE_PF));
+ lli.ucq_density = 1 << QUEUESPERPAGEPF0_GET(
+ t4_read_reg(adap, SGE_INGRESS_QUEUES_PER_PAGE_PF));
+ lli.gts_reg = adap->regs + MYPF_REG(SGE_PF_GTS);
+ lli.db_reg = adap->regs + MYPF_REG(SGE_PF_KDOORBELL);
+ lli.fw_vers = adap->params.fw_vers;
+
+ handle = ulds[uld].add(&lli);
+ if (IS_ERR(handle)) {
+ dev_warn(adap->pdev_dev,
+ "could not attach to the %s driver, error %ld\n",
+ uld_str[uld], PTR_ERR(handle));
+ return;
+ }
+
+ adap->uld_handle[uld] = handle;
+
+ if (!netevent_registered) {
+ register_netevent_notifier(&cxgb4_netevent_nb);
+ netevent_registered = true;
+ }
+}
+
+static void attach_ulds(struct adapter *adap)
+{
+ unsigned int i;
+
+ mutex_lock(&uld_mutex);
+ list_add_tail(&adap->list_node, &adapter_list);
+ for (i = 0; i < CXGB4_ULD_MAX; i++)
+ if (ulds[i].add)
+ uld_attach(adap, i);
+ mutex_unlock(&uld_mutex);
+}
+
+static void detach_ulds(struct adapter *adap)
+{
+ unsigned int i;
+
+ mutex_lock(&uld_mutex);
+ list_del(&adap->list_node);
+ for (i = 0; i < CXGB4_ULD_MAX; i++)
+ if (adap->uld_handle[i]) {
+ ulds[i].state_change(adap->uld_handle[i],
+ CXGB4_STATE_DETACH);
+ adap->uld_handle[i] = NULL;
+ }
+ if (netevent_registered && list_empty(&adapter_list)) {
+ unregister_netevent_notifier(&cxgb4_netevent_nb);
+ netevent_registered = false;
+ }
+ mutex_unlock(&uld_mutex);
+}
+
+static void notify_ulds(struct adapter *adap, enum cxgb4_state new_state)
+{
+ unsigned int i;
+
+ mutex_lock(&uld_mutex);
+ for (i = 0; i < CXGB4_ULD_MAX; i++)
+ if (adap->uld_handle[i])
+ ulds[i].state_change(adap->uld_handle[i], new_state);
+ mutex_unlock(&uld_mutex);
+}
+
+/**
+ * cxgb4_register_uld - register an upper-layer driver
+ * @type: the ULD type
+ * @p: the ULD methods
+ *
+ * Registers an upper-layer driver with this driver and notifies the ULD
+ * about any presently available devices that support its type. Returns
+ * %-EBUSY if a ULD of the same type is already registered.
+ */
+int cxgb4_register_uld(enum cxgb4_uld type, const struct cxgb4_uld_info *p)
+{
+ int ret = 0;
+ struct adapter *adap;
+
+ if (type >= CXGB4_ULD_MAX)
+ return -EINVAL;
+ mutex_lock(&uld_mutex);
+ if (ulds[type].add) {
+ ret = -EBUSY;
+ goto out;
+ }
+ ulds[type] = *p;
+ list_for_each_entry(adap, &adapter_list, list_node)
+ uld_attach(adap, type);
+out: mutex_unlock(&uld_mutex);
+ return ret;
+}
+EXPORT_SYMBOL(cxgb4_register_uld);
+
+/**
+ * cxgb4_unregister_uld - unregister an upper-layer driver
+ * @type: the ULD type
+ *
+ * Unregisters an existing upper-layer driver.
+ */
+int cxgb4_unregister_uld(enum cxgb4_uld type)
+{
+ struct adapter *adap;
+
+ if (type >= CXGB4_ULD_MAX)
+ return -EINVAL;
+ mutex_lock(&uld_mutex);
+ list_for_each_entry(adap, &adapter_list, list_node)
+ adap->uld_handle[type] = NULL;
+ ulds[type].add = NULL;
+ mutex_unlock(&uld_mutex);
+ return 0;
+}
+EXPORT_SYMBOL(cxgb4_unregister_uld);
+
+/**
+ * cxgb_up - enable the adapter
+ * @adap: adapter being enabled
+ *
+ * Called when the first port is enabled, this function performs the
+ * actions necessary to make an adapter operational, such as completing
+ * the initialization of HW modules, and enabling interrupts.
+ *
+ * Must be called with the rtnl lock held.
+ */
+static int cxgb_up(struct adapter *adap)
+{
+ int err = 0;
+
+ if (!(adap->flags & FULL_INIT_DONE)) {
+ err = setup_sge_queues(adap);
+ if (err)
+ goto out;
+ err = setup_rss(adap);
+ if (err) {
+ t4_free_sge_resources(adap);
+ goto out;
+ }
+ if (adap->flags & USING_MSIX)
+ name_msix_vecs(adap);
+ adap->flags |= FULL_INIT_DONE;
+ }
+
+ if (adap->flags & USING_MSIX) {
+ err = request_irq(adap->msix_info[0].vec, t4_nondata_intr, 0,
+ adap->msix_info[0].desc, adap);
+ if (err)
+ goto irq_err;
+
+ err = request_msix_queue_irqs(adap);
+ if (err) {
+ free_irq(adap->msix_info[0].vec, adap);
+ goto irq_err;
+ }
+ } else {
+ err = request_irq(adap->pdev->irq, t4_intr_handler(adap),
+ (adap->flags & USING_MSI) ? 0 : IRQF_SHARED,
+ adap->name, adap);
+ if (err)
+ goto irq_err;
+ }
+ enable_rx(adap);
+ t4_sge_start(adap);
+ t4_intr_enable(adap);
+ notify_ulds(adap, CXGB4_STATE_UP);
+ out:
+ return err;
+ irq_err:
+ dev_err(adap->pdev_dev, "request_irq failed, err %d\n", err);
+ goto out;
+}
+
+static void cxgb_down(struct adapter *adapter)
+{
+ t4_intr_disable(adapter);
+ cancel_work_sync(&adapter->tid_release_task);
+ adapter->tid_release_task_busy = false;
+
+ if (adapter->flags & USING_MSIX) {
+ free_msix_queue_irqs(adapter);
+ free_irq(adapter->msix_info[0].vec, adapter);
+ } else
+ free_irq(adapter->pdev->irq, adapter);
+ quiesce_rx(adapter);
+}
+
+/*
+ * net_device operations
+ */
+static int cxgb_open(struct net_device *dev)
+{
+ int err;
+ struct port_info *pi = netdev_priv(dev);
+ struct adapter *adapter = pi->adapter;
+
+ if (!adapter->open_device_map && (err = cxgb_up(adapter)) < 0)
+ return err;
+
+ dev->real_num_tx_queues = pi->nqsets;
+ set_bit(pi->tx_chan, &adapter->open_device_map);
+ link_start(dev);
+ netif_tx_start_all_queues(dev);
+ return 0;
+}
+
+static int cxgb_close(struct net_device *dev)
+{
+ int ret;
+ struct port_info *pi = netdev_priv(dev);
+ struct adapter *adapter = pi->adapter;
+
+ netif_tx_stop_all_queues(dev);
+ netif_carrier_off(dev);
+ ret = t4_enable_vi(adapter, 0, pi->viid, false, false);
+
+ clear_bit(pi->tx_chan, &adapter->open_device_map);
+
+ if (!adapter->open_device_map)
+ cxgb_down(adapter);
+ return 0;
+}
+
+static struct net_device_stats *cxgb_get_stats(struct net_device *dev)
+{
+ struct port_stats stats;
+ struct port_info *p = netdev_priv(dev);
+ struct adapter *adapter = p->adapter;
+ struct net_device_stats *ns = &dev->stats;
+
+ spin_lock(&adapter->stats_lock);
+ t4_get_port_stats(adapter, p->tx_chan, &stats);
+ spin_unlock(&adapter->stats_lock);
+
+ ns->tx_bytes = stats.tx_octets;
+ ns->tx_packets = stats.tx_frames;
+ ns->rx_bytes = stats.rx_octets;
+ ns->rx_packets = stats.rx_frames;
+ ns->multicast = stats.rx_mcast_frames;
+
+ /* detailed rx_errors */
+ ns->rx_length_errors = stats.rx_jabber + stats.rx_too_long +
+ stats.rx_runt;
+ ns->rx_over_errors = 0;
+ ns->rx_crc_errors = stats.rx_fcs_err;
+ ns->rx_frame_errors = stats.rx_symbol_err;
+ ns->rx_fifo_errors = stats.rx_ovflow0 + stats.rx_ovflow1 +
+ stats.rx_ovflow2 + stats.rx_ovflow3 +
+ stats.rx_trunc0 + stats.rx_trunc1 +
+ stats.rx_trunc2 + stats.rx_trunc3;
+ ns->rx_missed_errors = 0;
+
+ /* detailed tx_errors */
+ ns->tx_aborted_errors = 0;
+ ns->tx_carrier_errors = 0;
+ ns->tx_fifo_errors = 0;
+ ns->tx_heartbeat_errors = 0;
+ ns->tx_window_errors = 0;
+
+ ns->tx_errors = stats.tx_error_frames;
+ ns->rx_errors = stats.rx_symbol_err + stats.rx_fcs_err +
+ ns->rx_length_errors + stats.rx_len_err + ns->rx_fifo_errors;
+ return ns;
+}
+
+static int cxgb_ioctl(struct net_device *dev, struct ifreq *req, int cmd)
+{
+ int ret = 0, prtad, devad;
+ struct port_info *pi = netdev_priv(dev);
+ struct mii_ioctl_data *data = (struct mii_ioctl_data *)&req->ifr_data;
+
+ switch (cmd) {
+ case SIOCGMIIPHY:
+ if (pi->mdio_addr < 0)
+ return -EOPNOTSUPP;
+ data->phy_id = pi->mdio_addr;
+ break;
+ case SIOCGMIIREG:
+ case SIOCSMIIREG:
+ if (mdio_phy_id_is_c45(data->phy_id)) {
+ prtad = mdio_phy_id_prtad(data->phy_id);
+ devad = mdio_phy_id_devad(data->phy_id);
+ } else if (data->phy_id < 32) {
+ prtad = data->phy_id;
+ devad = 0;
+ data->reg_num &= 0x1f;
+ } else
+ return -EINVAL;
+
+ if (cmd == SIOCGMIIREG)
+ ret = t4_mdio_rd(pi->adapter, 0, prtad, devad,
+ data->reg_num, &data->val_out);
+ else
+ ret = t4_mdio_wr(pi->adapter, 0, prtad, devad,
+ data->reg_num, data->val_in);
+ break;
+ default:
+ return -EOPNOTSUPP;
+ }
+ return ret;
+}
+
+static void cxgb_set_rxmode(struct net_device *dev)
+{
+ /* unfortunately we can't return errors to the stack */
+ set_rxmode(dev, -1, false);
+}
+
+static int cxgb_change_mtu(struct net_device *dev, int new_mtu)
+{
+ int ret;
+ struct port_info *pi = netdev_priv(dev);
+
+ if (new_mtu < 81 || new_mtu > MAX_MTU) /* accommodate SACK */
+ return -EINVAL;
+ ret = t4_set_rxmode(pi->adapter, 0, pi->viid, new_mtu, -1, -1, -1,
+ true);
+ if (!ret)
+ dev->mtu = new_mtu;
+ return ret;
+}
+
+static int cxgb_set_mac_addr(struct net_device *dev, void *p)
+{
+ int ret;
+ struct sockaddr *addr = p;
+ struct port_info *pi = netdev_priv(dev);
+
+ if (!is_valid_ether_addr(addr->sa_data))
+ return -EINVAL;
+
+ ret = t4_change_mac(pi->adapter, 0, pi->viid, pi->xact_addr_filt,
+ addr->sa_data, true, true);
+ if (ret < 0)
+ return ret;
+
+ memcpy(dev->dev_addr, addr->sa_data, dev->addr_len);
+ pi->xact_addr_filt = ret;
+ return 0;
+}
+
+static void vlan_rx_register(struct net_device *dev, struct vlan_group *grp)
+{
+ struct port_info *pi = netdev_priv(dev);
+
+ pi->vlan_grp = grp;
+ t4_set_vlan_accel(pi->adapter, 1 << pi->tx_chan, grp != NULL);
+}
+
+#ifdef CONFIG_NET_POLL_CONTROLLER
+static void cxgb_netpoll(struct net_device *dev)
+{
+ struct port_info *pi = netdev_priv(dev);
+ struct adapter *adap = pi->adapter;
+
+ if (adap->flags & USING_MSIX) {
+ int i;
+ struct sge_eth_rxq *rx = &adap->sge.ethrxq[pi->first_qset];
+
+ for (i = pi->nqsets; i; i--, rx++)
+ t4_sge_intr_msix(0, &rx->rspq);
+ } else
+ t4_intr_handler(adap)(0, adap);
+}
+#endif
+
+static const struct net_device_ops cxgb4_netdev_ops = {
+ .ndo_open = cxgb_open,
+ .ndo_stop = cxgb_close,
+ .ndo_start_xmit = t4_eth_xmit,
+ .ndo_get_stats = cxgb_get_stats,
+ .ndo_set_rx_mode = cxgb_set_rxmode,
+ .ndo_set_mac_address = cxgb_set_mac_addr,
+ .ndo_validate_addr = eth_validate_addr,
+ .ndo_do_ioctl = cxgb_ioctl,
+ .ndo_change_mtu = cxgb_change_mtu,
+ .ndo_vlan_rx_register = vlan_rx_register,
+#ifdef CONFIG_NET_POLL_CONTROLLER
+ .ndo_poll_controller = cxgb_netpoll,
+#endif
+};
+
+void t4_fatal_err(struct adapter *adap)
+{
+ t4_set_reg_field(adap, SGE_CONTROL, GLOBALENABLE, 0);
+ t4_intr_disable(adap);
+ dev_alert(adap->pdev_dev, "encountered fatal error, adapter stopped\n");
+}
+
+static void setup_memwin(struct adapter *adap)
+{
+ u32 bar0;
+
+ bar0 = pci_resource_start(adap->pdev, 0); /* truncation intentional */
+ t4_write_reg(adap, PCIE_MEM_ACCESS_REG(PCIE_MEM_ACCESS_BASE_WIN, 0),
+ (bar0 + MEMWIN0_BASE) | BIR(0) |
+ WINDOW(ilog2(MEMWIN0_APERTURE) - 10));
+ t4_write_reg(adap, PCIE_MEM_ACCESS_REG(PCIE_MEM_ACCESS_BASE_WIN, 1),
+ (bar0 + MEMWIN1_BASE) | BIR(0) |
+ WINDOW(ilog2(MEMWIN1_APERTURE) - 10));
+ t4_write_reg(adap, PCIE_MEM_ACCESS_REG(PCIE_MEM_ACCESS_BASE_WIN, 2),
+ (bar0 + MEMWIN2_BASE) | BIR(0) |
+ WINDOW(ilog2(MEMWIN2_APERTURE) - 10));
+}
+
+/*
+ * Max # of ATIDs. The absolute HW max is 16K but we keep it lower.
+ */
+#define MAX_ATIDS 8192U
+
+/*
+ * Phase 0 of initialization: contact FW, obtain config, perform basic init.
+ */
+static int adap_init0(struct adapter *adap)
+{
+ int ret;
+ u32 v, port_vec;
+ enum dev_state state;
+ u32 params[7], val[7];
+ struct fw_caps_config_cmd c;
+
+ ret = t4_check_fw_version(adap);
+ if (ret == -EINVAL || ret > 0) {
+ if (upgrade_fw(adap) >= 0) /* recache FW version */
+ ret = t4_check_fw_version(adap);
+ }
+ if (ret < 0)
+ return ret;
+
+ /* contact FW, request master */
+ ret = t4_fw_hello(adap, 0, 0, MASTER_MUST, &state);
+ if (ret < 0) {
+ dev_err(adap->pdev_dev, "could not connect to FW, error %d\n",
+ ret);
+ return ret;
+ }
+
+ /* reset device */
+ ret = t4_fw_reset(adap, 0, PIORSTMODE | PIORST);
+ if (ret < 0)
+ goto bye;
+
+ /* get device capabilities */
+ memset(&c, 0, sizeof(c));
+ c.op_to_write = htonl(FW_CMD_OP(FW_CAPS_CONFIG_CMD) |
+ FW_CMD_REQUEST | FW_CMD_READ);
+ c.retval_len16 = htonl(FW_LEN16(c));
+ ret = t4_wr_mbox(adap, 0, &c, sizeof(c), &c);
+ if (ret < 0)
+ goto bye;
+
+ /* select capabilities we'll be using */
+ if (c.niccaps & htons(FW_CAPS_CONFIG_NIC_VM)) {
+ if (!vf_acls)
+ c.niccaps ^= htons(FW_CAPS_CONFIG_NIC_VM);
+ else
+ c.niccaps = htons(FW_CAPS_CONFIG_NIC_VM);
+ } else if (vf_acls) {
+ dev_err(adap->pdev_dev, "virtualization ACLs not supported");
+ goto bye;
+ }
+ c.op_to_write = htonl(FW_CMD_OP(FW_CAPS_CONFIG_CMD) |
+ FW_CMD_REQUEST | FW_CMD_WRITE);
+ ret = t4_wr_mbox(adap, 0, &c, sizeof(c), NULL);
+ if (ret < 0)
+ goto bye;
+
+ ret = t4_config_glbl_rss(adap, 0,
+ FW_RSS_GLB_CONFIG_CMD_MODE_BASICVIRTUAL,
+ FW_RSS_GLB_CONFIG_CMD_TNLMAPEN |
+ FW_RSS_GLB_CONFIG_CMD_TNLALLLKP);
+ if (ret < 0)
+ goto bye;
+
+ ret = t4_cfg_pfvf(adap, 0, 0, 0, 64, 64, 64, 0, 0, 4, 0xf, 0xf, 16,
+ FW_CMD_CAP_PF, FW_CMD_CAP_PF);
+ if (ret < 0)
+ goto bye;
+
+ for (v = 0; v < SGE_NTIMERS - 1; v++)
+ adap->sge.timer_val[v] = min(intr_holdoff[v], MAX_SGE_TIMERVAL);
+ adap->sge.timer_val[SGE_NTIMERS - 1] = MAX_SGE_TIMERVAL;
+ adap->sge.counter_val[0] = 1;
+ for (v = 1; v < SGE_NCOUNTERS; v++)
+ adap->sge.counter_val[v] = min(intr_cnt[v - 1],
+ THRESHOLD_3_MASK);
+ t4_sge_init(adap);
+
+ /* get basic stuff going */
+ ret = t4_early_init(adap, 0);
+ if (ret < 0)
+ goto bye;
+
+#define FW_PARAM_DEV(param) \
+ (FW_PARAMS_MNEM(FW_PARAMS_MNEM_DEV) | \
+ FW_PARAMS_PARAM_X(FW_PARAMS_PARAM_DEV_##param))
+
+#define FW_PARAM_PFVF(param) \
+ (FW_PARAMS_MNEM(FW_PARAMS_MNEM_PFVF) | \
+ FW_PARAMS_PARAM_X(FW_PARAMS_PARAM_PFVF_##param))
+
+ params[0] = FW_PARAM_DEV(PORTVEC);
+ params[1] = FW_PARAM_PFVF(L2T_START);
+ params[2] = FW_PARAM_PFVF(L2T_END);
+ params[3] = FW_PARAM_PFVF(FILTER_START);
+ params[4] = FW_PARAM_PFVF(FILTER_END);
+ ret = t4_query_params(adap, 0, 0, 0, 5, params, val);
+ if (ret < 0)
+ goto bye;
+ port_vec = val[0];
+ adap->tids.ftid_base = val[3];
+ adap->tids.nftids = val[4] - val[3] + 1;
+
+ if (c.ofldcaps) {
+ /* query offload-related parameters */
+ params[0] = FW_PARAM_DEV(NTID);
+ params[1] = FW_PARAM_PFVF(SERVER_START);
+ params[2] = FW_PARAM_PFVF(SERVER_END);
+ params[3] = FW_PARAM_PFVF(TDDP_START);
+ params[4] = FW_PARAM_PFVF(TDDP_END);
+ params[5] = FW_PARAM_DEV(FLOWC_BUFFIFO_SZ);
+ ret = t4_query_params(adap, 0, 0, 0, 6, params, val);
+ if (ret < 0)
+ goto bye;
+ adap->tids.ntids = val[0];
+ adap->tids.natids = min(adap->tids.ntids / 2, MAX_ATIDS);
+ adap->tids.stid_base = val[1];
+ adap->tids.nstids = val[2] - val[1] + 1;
+ adap->vres.ddp.start = val[3];
+ adap->vres.ddp.size = val[4] - val[3] + 1;
+ adap->params.ofldq_wr_cred = val[5];
+ adap->params.offload = 1;
+ }
+ if (c.rdmacaps) {
+ params[0] = FW_PARAM_PFVF(STAG_START);
+ params[1] = FW_PARAM_PFVF(STAG_END);
+ params[2] = FW_PARAM_PFVF(RQ_START);
+ params[3] = FW_PARAM_PFVF(RQ_END);
+ params[4] = FW_PARAM_PFVF(PBL_START);
+ params[5] = FW_PARAM_PFVF(PBL_END);
+ ret = t4_query_params(adap, 0, 0, 0, 6, params, val);
+ if (ret < 0)
+ goto bye;
+ adap->vres.stag.start = val[0];
+ adap->vres.stag.size = val[1] - val[0] + 1;
+ adap->vres.rq.start = val[2];
+ adap->vres.rq.size = val[3] - val[2] + 1;
+ adap->vres.pbl.start = val[4];
+ adap->vres.pbl.size = val[5] - val[4] + 1;
+ }
+ if (c.iscsicaps) {
+ params[0] = FW_PARAM_PFVF(ISCSI_START);
+ params[1] = FW_PARAM_PFVF(ISCSI_END);
+ ret = t4_query_params(adap, 0, 0, 0, 2, params, val);
+ if (ret < 0)
+ goto bye;
+ adap->vres.iscsi.start = val[0];
+ adap->vres.iscsi.size = val[1] - val[0] + 1;
+ }
+#undef FW_PARAM_PFVF
+#undef FW_PARAM_DEV
+
+ adap->params.nports = hweight32(port_vec);
+ adap->params.portvec = port_vec;
+ adap->flags |= FW_OK;
+
+ /* These are finalized by FW initialization, load their values now */
+ v = t4_read_reg(adap, TP_TIMER_RESOLUTION);
+ adap->params.tp.tre = TIMERRESOLUTION_GET(v);
+ t4_read_mtu_tbl(adap, adap->params.mtus, NULL);
+ t4_load_mtus(adap, adap->params.mtus, adap->params.a_wnd,
+ adap->params.b_wnd);
+
+ /* tweak some settings */
+ t4_write_reg(adap, TP_SHIFT_CNT, 0x64f8849);
+ t4_write_reg(adap, ULP_RX_TDDP_PSZ, HPZ0(PAGE_SHIFT - 12));
+ t4_write_reg(adap, TP_PIO_ADDR, TP_INGRESS_CONFIG);
+ v = t4_read_reg(adap, TP_PIO_DATA);
+ t4_write_reg(adap, TP_PIO_DATA, v & ~CSUM_HAS_PSEUDO_HDR);
+ setup_memwin(adap);
+ return 0;
+
+ /*
+ * If a command timed out or failed with EIO FW does not operate within
+ * its spec or something catastrophic happened to HW/FW, stop issuing
+ * commands.
+ */
+bye: if (ret != -ETIMEDOUT && ret != -EIO)
+ t4_fw_bye(adap, 0);
+ return ret;
+}
+
+static inline bool is_10g_port(const struct link_config *lc)
+{
+ return (lc->supported & FW_PORT_CAP_SPEED_10G) != 0;
+}
+
+static inline void init_rspq(struct sge_rspq *q, u8 timer_idx, u8 pkt_cnt_idx,
+ unsigned int size, unsigned int iqe_size)
+{
+ q->intr_params = QINTR_TIMER_IDX(timer_idx) |
+ (pkt_cnt_idx < SGE_NCOUNTERS ? QINTR_CNT_EN : 0);
+ q->pktcnt_idx = pkt_cnt_idx < SGE_NCOUNTERS ? pkt_cnt_idx : 0;
+ q->iqe_len = iqe_size;
+ q->size = size;
+}
+
+/*
+ * Perform default configuration of DMA queues depending on the number and type
+ * of ports we found and the number of available CPUs. Most settings can be
+ * modified by the admin prior to actual use.
+ */
+static void __devinit cfg_queues(struct adapter *adap)
+{
+ struct sge *s = &adap->sge;
+ int i, q10g = 0, n10g = 0, qidx = 0;
+
+ for_each_port(adap, i)
+ n10g += is_10g_port(&adap2pinfo(adap, i)->link_cfg);
+
+ /*
+ * We default to 1 queue per non-10G port and up to # of cores queues
+ * per 10G port.
+ */
+ if (n10g)
+ q10g = (MAX_ETH_QSETS - (adap->params.nports - n10g)) / n10g;
+ if (q10g > num_online_cpus())
+ q10g = num_online_cpus();
+
+ for_each_port(adap, i) {
+ struct port_info *pi = adap2pinfo(adap, i);
+
+ pi->first_qset = qidx;
+ pi->nqsets = is_10g_port(&pi->link_cfg) ? q10g : 1;
+ qidx += pi->nqsets;
+ }
+
+ s->ethqsets = qidx;
+ s->max_ethqsets = qidx; /* MSI-X may lower it later */
+
+ if (is_offload(adap)) {
+ /*
+ * For offload we use 1 queue/channel if all ports are up to 1G,
+ * otherwise we divide all available queues amongst the channels
+ * capped by the number of available cores.
+ */
+ if (n10g) {
+ i = min_t(int, ARRAY_SIZE(s->ofldrxq),
+ num_online_cpus());
+ s->ofldqsets = roundup(i, adap->params.nports);
+ } else
+ s->ofldqsets = adap->params.nports;
+ /* For RDMA one Rx queue per channel suffices */
+ s->rdmaqs = adap->params.nports;
+ }
+
+ for (i = 0; i < ARRAY_SIZE(s->ethrxq); i++) {
+ struct sge_eth_rxq *r = &s->ethrxq[i];
+
+ init_rspq(&r->rspq, 0, 0, 1024, 64);
+ r->fl.size = 72;
+ }
+
+ for (i = 0; i < ARRAY_SIZE(s->ethtxq); i++)
+ s->ethtxq[i].q.size = 1024;
+
+ for (i = 0; i < ARRAY_SIZE(s->ctrlq); i++)
+ s->ctrlq[i].q.size = 512;
+
+ for (i = 0; i < ARRAY_SIZE(s->ofldtxq); i++)
+ s->ofldtxq[i].q.size = 1024;
+
+ for (i = 0; i < ARRAY_SIZE(s->ofldrxq); i++) {
+ struct sge_ofld_rxq *r = &s->ofldrxq[i];
+
+ init_rspq(&r->rspq, 0, 0, 1024, 64);
+ r->rspq.uld = CXGB4_ULD_ISCSI;
+ r->fl.size = 72;
+ }
+
+ for (i = 0; i < ARRAY_SIZE(s->rdmarxq); i++) {
+ struct sge_ofld_rxq *r = &s->rdmarxq[i];
+
+ init_rspq(&r->rspq, 0, 0, 511, 64);
+ r->rspq.uld = CXGB4_ULD_RDMA;
+ r->fl.size = 72;
+ }
+
+ init_rspq(&s->fw_evtq, 6, 0, 512, 64);
+ init_rspq(&s->intrq, 6, 0, 2 * MAX_INGQ, 64);
+}
+
+/*
+ * Reduce the number of Ethernet queues across all ports to at most n.
+ * n provides at least one queue per port.
+ */
+static void __devinit reduce_ethqs(struct adapter *adap, int n)
+{
+ int i;
+ struct port_info *pi;
+
+ while (n < adap->sge.ethqsets)
+ for_each_port(adap, i) {
+ pi = adap2pinfo(adap, i);
+ if (pi->nqsets > 1) {
+ pi->nqsets--;
+ adap->sge.ethqsets--;
+ if (adap->sge.ethqsets <= n)
+ break;
+ }
+ }
+
+ n = 0;
+ for_each_port(adap, i) {
+ pi = adap2pinfo(adap, i);
+ pi->first_qset = n;
+ n += pi->nqsets;
+ }
+}
+
+/* 2 MSI-X vectors needed for the FW queue and non-data interrupts */
+#define EXTRA_VECS 2
+
+static int __devinit enable_msix(struct adapter *adap)
+{
+ int ofld_need = 0;
+ int i, err, want, need;
+ struct sge *s = &adap->sge;
+ unsigned int nchan = adap->params.nports;
+ struct msix_entry entries[MAX_INGQ + 1];
+
+ for (i = 0; i < ARRAY_SIZE(entries); ++i)
+ entries[i].entry = i;
+
+ want = s->max_ethqsets + EXTRA_VECS;
+ if (is_offload(adap)) {
+ want += s->rdmaqs + s->ofldqsets;
+ /* need nchan for each possible ULD */
+ ofld_need = 2 * nchan;
+ }
+ need = adap->params.nports + EXTRA_VECS + ofld_need;
+
+ while ((err = pci_enable_msix(adap->pdev, entries, want)) >= need)
+ want = err;
+
+ if (!err) {
+ /*
+ * Distribute available vectors to the various queue groups.
+ * Every group gets its minimum requirement and NIC gets top
+ * priority for leftovers.
+ */
+ i = want - EXTRA_VECS - ofld_need;
+ if (i < s->max_ethqsets) {
+ s->max_ethqsets = i;
+ if (i < s->ethqsets)
+ reduce_ethqs(adap, i);
+ }
+ if (is_offload(adap)) {
+ i = want - EXTRA_VECS - s->max_ethqsets;
+ i -= ofld_need - nchan;
+ s->ofldqsets = (i / nchan) * nchan; /* round down */
+ }
+ for (i = 0; i < want; ++i)
+ adap->msix_info[i].vec = entries[i].vector;
+ } else if (err > 0)
+ dev_info(adap->pdev_dev,
+ "only %d MSI-X vectors left, not using MSI-X\n", err);
+ return err;
+}
+
+#undef EXTRA_VECS
+
+static void __devinit print_port_info(struct adapter *adap)
+{
+ static const char *base[] = {
+ "R", "KX4", "T", "KX", "T", "KR", "CX4"
+ };
+
+ int i;
+ char buf[80];
+
+ for_each_port(adap, i) {
+ struct net_device *dev = adap->port[i];
+ const struct port_info *pi = netdev_priv(dev);
+ char *bufp = buf;
+
+ if (!test_bit(i, &adap->registered_device_map))
+ continue;
+
+ if (pi->link_cfg.supported & FW_PORT_CAP_SPEED_100M)
+ bufp += sprintf(bufp, "100/");
+ if (pi->link_cfg.supported & FW_PORT_CAP_SPEED_1G)
+ bufp += sprintf(bufp, "1000/");
+ if (pi->link_cfg.supported & FW_PORT_CAP_SPEED_10G)
+ bufp += sprintf(bufp, "10G/");
+ if (bufp != buf)
+ --bufp;
+ sprintf(bufp, "BASE-%s", base[pi->port_type]);
+
+ netdev_info(dev, "Chelsio %s rev %d %s %sNIC PCIe x%d%s\n",
+ adap->params.vpd.id, adap->params.rev,
+ buf, is_offload(adap) ? "R" : "",
+ adap->params.pci.width,
+ (adap->flags & USING_MSIX) ? " MSI-X" :
+ (adap->flags & USING_MSI) ? " MSI" : "");
+ if (adap->name == dev->name)
+ netdev_info(dev, "S/N: %s, E/C: %s\n",
+ adap->params.vpd.sn, adap->params.vpd.ec);
+ }
+}
+
+#define VLAN_FEAT (NETIF_F_SG | NETIF_F_IP_CSUM | NETIF_F_TSO | NETIF_F_TSO6 |\
+ NETIF_F_IPV6_CSUM | NETIF_F_HIGHDMA)
+
+static int __devinit init_one(struct pci_dev *pdev,
+ const struct pci_device_id *ent)
+{
+ int func, i, err;
+ struct port_info *pi;
+ unsigned int highdma = 0;
+ struct adapter *adapter = NULL;
+
+ printk_once(KERN_INFO "%s - version %s\n", DRV_DESC, DRV_VERSION);
+
+ err = pci_request_regions(pdev, KBUILD_MODNAME);
+ if (err) {
+ /* Just info, some other driver may have claimed the device. */
+ dev_info(&pdev->dev, "cannot obtain PCI resources\n");
+ return err;
+ }
+
+ /* We control everything through PF 0 */
+ func = PCI_FUNC(pdev->devfn);
+ if (func > 0)
+ goto sriov;
+
+ err = pci_enable_device(pdev);
+ if (err) {
+ dev_err(&pdev->dev, "cannot enable PCI device\n");
+ goto out_release_regions;
+ }
+
+ if (!pci_set_dma_mask(pdev, DMA_BIT_MASK(64))) {
+ highdma = NETIF_F_HIGHDMA;
+ err = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64));
+ if (err) {
+ dev_err(&pdev->dev, "unable to obtain 64-bit DMA for "
+ "coherent allocations\n");
+ goto out_disable_device;
+ }
+ } else {
+ err = pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
+ if (err) {
+ dev_err(&pdev->dev, "no usable DMA configuration\n");
+ goto out_disable_device;
+ }
+ }
+
+ pci_enable_pcie_error_reporting(pdev);
+ pci_set_master(pdev);
+ pci_save_state(pdev);
+
+ adapter = kzalloc(sizeof(*adapter), GFP_KERNEL);
+ if (!adapter) {
+ err = -ENOMEM;
+ goto out_disable_device;
+ }
+
+ adapter->regs = pci_ioremap_bar(pdev, 0);
+ if (!adapter->regs) {
+ dev_err(&pdev->dev, "cannot map device registers\n");
+ err = -ENOMEM;
+ goto out_free_adapter;
+ }
+
+ adapter->pdev = pdev;
+ adapter->pdev_dev = &pdev->dev;
+ adapter->name = pci_name(pdev);
+ adapter->msg_enable = dflt_msg_enable;
+ memset(adapter->chan_map, 0xff, sizeof(adapter->chan_map));
+
+ spin_lock_init(&adapter->stats_lock);
+ spin_lock_init(&adapter->tid_release_lock);
+
+ INIT_WORK(&adapter->tid_release_task, process_tid_release_list);
+
+ err = t4_prep_adapter(adapter);
+ if (err)
+ goto out_unmap_bar;
+ err = adap_init0(adapter);
+ if (err)
+ goto out_unmap_bar;
+
+ for_each_port(adapter, i) {
+ struct net_device *netdev;
+
+ netdev = alloc_etherdev_mq(sizeof(struct port_info),
+ MAX_ETH_QSETS);
+ if (!netdev) {
+ err = -ENOMEM;
+ goto out_free_dev;
+ }
+
+ SET_NETDEV_DEV(netdev, &pdev->dev);
+
+ adapter->port[i] = netdev;
+ pi = netdev_priv(netdev);
+ pi->adapter = adapter;
+ pi->xact_addr_filt = -1;
+ pi->rx_offload = RX_CSO;
+ pi->port_id = i;
+ netif_carrier_off(netdev);
+ netif_tx_stop_all_queues(netdev);
+ netdev->irq = pdev->irq;
+
+ netdev->features |= NETIF_F_SG | NETIF_F_TSO | NETIF_F_TSO6;
+ netdev->features |= NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM;
+ netdev->features |= NETIF_F_GRO | highdma;
+ netdev->features |= NETIF_F_HW_VLAN_TX | NETIF_F_HW_VLAN_RX;
+ netdev->vlan_features = netdev->features & VLAN_FEAT;
+
+ netdev->netdev_ops = &cxgb4_netdev_ops;
+ SET_ETHTOOL_OPS(netdev, &cxgb_ethtool_ops);
+ }
+
+ pci_set_drvdata(pdev, adapter);
+
+ if (adapter->flags & FW_OK) {
+ err = t4_port_init(adapter, 0, 0, 0);
+ if (err)
+ goto out_free_dev;
+ }
+
+ /*
+ * Configure queues and allocate tables now, they can be needed as
+ * soon as the first register_netdev completes.
+ */
+ cfg_queues(adapter);
+
+ adapter->l2t = t4_init_l2t();
+ if (!adapter->l2t) {
+ /* We tolerate a lack of L2T, giving up some functionality */
+ dev_warn(&pdev->dev, "could not allocate L2T, continuing\n");
+ adapter->params.offload = 0;
+ }
+
+ if (is_offload(adapter) && tid_init(&adapter->tids) < 0) {
+ dev_warn(&pdev->dev, "could not allocate TID table, "
+ "continuing\n");
+ adapter->params.offload = 0;
+ }
+
+ /*
+ * The card is now ready to go. If any errors occur during device
+ * registration we do not fail the whole card but rather proceed only
+ * with the ports we manage to register successfully. However we must
+ * register at least one net device.
+ */
+ for_each_port(adapter, i) {
+ err = register_netdev(adapter->port[i]);
+ if (err)
+ dev_warn(&pdev->dev,
+ "cannot register net device %s, skipping\n",
+ adapter->port[i]->name);
+ else {
+ /*
+ * Change the name we use for messages to the name of
+ * the first successfully registered interface.
+ */
+ if (!adapter->registered_device_map)
+ adapter->name = adapter->port[i]->name;
+
+ __set_bit(i, &adapter->registered_device_map);
+ adapter->chan_map[adap2pinfo(adapter, i)->tx_chan] = i;
+ }
+ }
+ if (!adapter->registered_device_map) {
+ dev_err(&pdev->dev, "could not register any net devices\n");
+ goto out_free_dev;
+ }
+
+ if (cxgb4_debugfs_root) {
+ adapter->debugfs_root = debugfs_create_dir(pci_name(pdev),
+ cxgb4_debugfs_root);
+ setup_debugfs(adapter);
+ }
+
+ /* See what interrupts we'll be using */
+ if (msi > 1 && enable_msix(adapter) == 0)
+ adapter->flags |= USING_MSIX;
+ else if (msi > 0 && pci_enable_msi(pdev) == 0)
+ adapter->flags |= USING_MSI;
+
+ if (is_offload(adapter))
+ attach_ulds(adapter);
+
+ print_port_info(adapter);
+
+sriov:
+#ifdef CONFIG_PCI_IOV
+ if (func < ARRAY_SIZE(num_vf) && num_vf[func] > 0)
+ if (pci_enable_sriov(pdev, num_vf[func]) == 0)
+ dev_info(&pdev->dev,
+ "instantiated %u virtual functions\n",
+ num_vf[func]);
+#endif
+ return 0;
+
+ out_free_dev:
+ t4_free_mem(adapter->tids.tid_tab);
+ t4_free_mem(adapter->l2t);
+ for_each_port(adapter, i)
+ if (adapter->port[i])
+ free_netdev(adapter->port[i]);
+ if (adapter->flags & FW_OK)
+ t4_fw_bye(adapter, 0);
+ out_unmap_bar:
+ iounmap(adapter->regs);
+ out_free_adapter:
+ kfree(adapter);
+ out_disable_device:
+ pci_disable_pcie_error_reporting(pdev);
+ pci_disable_device(pdev);
+ out_release_regions:
+ pci_release_regions(pdev);
+ pci_set_drvdata(pdev, NULL);
+ return err;
+}
+
+static void __devexit remove_one(struct pci_dev *pdev)
+{
+ struct adapter *adapter = pci_get_drvdata(pdev);
+
+ pci_disable_sriov(pdev);
+
+ if (adapter) {
+ int i;
+
+ if (is_offload(adapter))
+ detach_ulds(adapter);
+
+ for_each_port(adapter, i)
+ if (test_bit(i, &adapter->registered_device_map))
+ unregister_netdev(adapter->port[i]);
+
+ if (adapter->debugfs_root)
+ debugfs_remove_recursive(adapter->debugfs_root);
+
+ t4_sge_stop(adapter);
+ t4_free_sge_resources(adapter);
+ t4_free_mem(adapter->l2t);
+ t4_free_mem(adapter->tids.tid_tab);
+ disable_msi(adapter);
+
+ for_each_port(adapter, i)
+ if (adapter->port[i])
+ free_netdev(adapter->port[i]);
+
+ if (adapter->flags & FW_OK)
+ t4_fw_bye(adapter, 0);
+ iounmap(adapter->regs);
+ kfree(adapter);
+ pci_disable_pcie_error_reporting(pdev);
+ pci_disable_device(pdev);
+ pci_release_regions(pdev);
+ pci_set_drvdata(pdev, NULL);
+ } else if (PCI_FUNC(pdev->devfn) > 0)
+ pci_release_regions(pdev);
+}
+
+static struct pci_driver cxgb4_driver = {
+ .name = KBUILD_MODNAME,
+ .id_table = cxgb4_pci_tbl,
+ .probe = init_one,
+ .remove = __devexit_p(remove_one),
+};
+
+static int __init cxgb4_init_module(void)
+{
+ int ret;
+
+ /* Debugfs support is optional, just warn if this fails */
+ cxgb4_debugfs_root = debugfs_create_dir(KBUILD_MODNAME, NULL);
+ if (!cxgb4_debugfs_root)
+ pr_warning("could not create debugfs entry, continuing\n");
+
+ ret = pci_register_driver(&cxgb4_driver);
+ if (ret < 0)
+ debugfs_remove(cxgb4_debugfs_root);
+ return ret;
+}
+
+static void __exit cxgb4_cleanup_module(void)
+{
+ pci_unregister_driver(&cxgb4_driver);
+ debugfs_remove(cxgb4_debugfs_root); /* NULL ok */
+}
+
+module_init(cxgb4_init_module);
+module_exit(cxgb4_cleanup_module);
--- /dev/null
+/*
+ * This file is part of the Chelsio T4 Ethernet driver for Linux.
+ *
+ * Copyright (c) 2003-2010 Chelsio Communications, Inc. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef __CXGB4_OFLD_H
+#define __CXGB4_OFLD_H
+
+#include <linux/cache.h>
+#include <linux/spinlock.h>
+#include <linux/skbuff.h>
+#include <asm/atomic.h>
+
+/* CPL message priority levels */
+enum {
+ CPL_PRIORITY_DATA = 0, /* data messages */
+ CPL_PRIORITY_SETUP = 1, /* connection setup messages */
+ CPL_PRIORITY_TEARDOWN = 0, /* connection teardown messages */
+ CPL_PRIORITY_LISTEN = 1, /* listen start/stop messages */
+ CPL_PRIORITY_ACK = 1, /* RX ACK messages */
+ CPL_PRIORITY_CONTROL = 1 /* control messages */
+};
+
+#define INIT_TP_WR(w, tid) do { \
+ (w)->wr.wr_hi = htonl(FW_WR_OP(FW_TP_WR) | \
+ FW_WR_IMMDLEN(sizeof(*w) - sizeof(w->wr))); \
+ (w)->wr.wr_mid = htonl(FW_WR_LEN16(DIV_ROUND_UP(sizeof(*w), 16)) | \
+ FW_WR_FLOWID(tid)); \
+ (w)->wr.wr_lo = cpu_to_be64(0); \
+} while (0)
+
+#define INIT_TP_WR_CPL(w, cpl, tid) do { \
+ INIT_TP_WR(w, tid); \
+ OPCODE_TID(w) = htonl(MK_OPCODE_TID(cpl, tid)); \
+} while (0)
+
+#define INIT_ULPTX_WR(w, wrlen, atomic, tid) do { \
+ (w)->wr.wr_hi = htonl(FW_WR_OP(FW_ULPTX_WR) | FW_WR_ATOMIC(atomic)); \
+ (w)->wr.wr_mid = htonl(FW_WR_LEN16(DIV_ROUND_UP(wrlen, 16)) | \
+ FW_WR_FLOWID(tid)); \
+ (w)->wr.wr_lo = cpu_to_be64(0); \
+} while (0)
+
+/* Special asynchronous notification message */
+#define CXGB4_MSG_AN ((void *)1)
+
+struct serv_entry {
+ void *data;
+};
+
+union aopen_entry {
+ void *data;
+ union aopen_entry *next;
+};
+
+/*
+ * Holds the size, base address, free list start, etc of the TID, server TID,
+ * and active-open TID tables. The tables themselves are allocated dynamically.
+ */
+struct tid_info {
+ void **tid_tab;
+ unsigned int ntids;
+
+ struct serv_entry *stid_tab;
+ unsigned long *stid_bmap;
+ unsigned int nstids;
+ unsigned int stid_base;
+
+ union aopen_entry *atid_tab;
+ unsigned int natids;
+
+ unsigned int nftids;
+ unsigned int ftid_base;
+
+ spinlock_t atid_lock ____cacheline_aligned_in_smp;
+ union aopen_entry *afree;
+ unsigned int atids_in_use;
+
+ spinlock_t stid_lock;
+ unsigned int stids_in_use;
+
+ atomic_t tids_in_use;
+};
+
+static inline void *lookup_tid(const struct tid_info *t, unsigned int tid)
+{
+ return tid < t->ntids ? t->tid_tab[tid] : NULL;
+}
+
+static inline void *lookup_atid(const struct tid_info *t, unsigned int atid)
+{
+ return atid < t->natids ? t->atid_tab[atid].data : NULL;
+}
+
+static inline void *lookup_stid(const struct tid_info *t, unsigned int stid)
+{
+ stid -= t->stid_base;
+ return stid < t->nstids ? t->stid_tab[stid].data : NULL;
+}
+
+static inline void cxgb4_insert_tid(struct tid_info *t, void *data,
+ unsigned int tid)
+{
+ t->tid_tab[tid] = data;
+ atomic_inc(&t->tids_in_use);
+}
+
+int cxgb4_alloc_atid(struct tid_info *t, void *data);
+int cxgb4_alloc_stid(struct tid_info *t, int family, void *data);
+void cxgb4_free_atid(struct tid_info *t, unsigned int atid);
+void cxgb4_free_stid(struct tid_info *t, unsigned int stid, int family);
+void cxgb4_remove_tid(struct tid_info *t, unsigned int qid, unsigned int tid);
+void cxgb4_queue_tid_release(struct tid_info *t, unsigned int chan,
+ unsigned int tid);
+
+struct in6_addr;
+
+int cxgb4_create_server(const struct net_device *dev, unsigned int stid,
+ __be32 sip, __be16 sport, unsigned int queue);
+int cxgb4_create_server6(const struct net_device *dev, unsigned int stid,
+ const struct in6_addr *sip, __be16 sport,
+ unsigned int queue);
+
+static inline void set_wr_txq(struct sk_buff *skb, int prio, int queue)
+{
+ skb_set_queue_mapping(skb, (queue << 1) | prio);
+}
+
+enum cxgb4_uld {
+ CXGB4_ULD_RDMA,
+ CXGB4_ULD_ISCSI,
+ CXGB4_ULD_MAX
+};
+
+enum cxgb4_state {
+ CXGB4_STATE_UP,
+ CXGB4_STATE_START_RECOVERY,
+ CXGB4_STATE_DOWN,
+ CXGB4_STATE_DETACH
+};
+
+struct pci_dev;
+struct l2t_data;
+struct net_device;
+struct pkt_gl;
+struct tp_tcp_stats;
+
+struct cxgb4_range {
+ unsigned int start;
+ unsigned int size;
+};
+
+struct cxgb4_virt_res { /* virtualized HW resources */
+ struct cxgb4_range ddp;
+ struct cxgb4_range iscsi;
+ struct cxgb4_range stag;
+ struct cxgb4_range rq;
+ struct cxgb4_range pbl;
+};
+
+/*
+ * Block of information the LLD provides to ULDs attaching to a device.
+ */
+struct cxgb4_lld_info {
+ struct pci_dev *pdev; /* associated PCI device */
+ struct l2t_data *l2t; /* L2 table */
+ struct tid_info *tids; /* TID table */
+ struct net_device **ports; /* device ports */
+ const struct cxgb4_virt_res *vr; /* assorted HW resources */
+ const unsigned short *mtus; /* MTU table */
+ const unsigned short *rxq_ids; /* the ULD's Rx queue ids */
+ unsigned short nrxq; /* # of Rx queues */
+ unsigned short ntxq; /* # of Tx queues */
+ unsigned char nchan:4; /* # of channels */
+ unsigned char nports:4; /* # of ports */
+ unsigned char wr_cred; /* WR 16-byte credits */
+ unsigned char adapter_type; /* type of adapter */
+ unsigned char fw_api_ver; /* FW API version */
+ unsigned int fw_vers; /* FW version */
+ unsigned int iscsi_iolen; /* iSCSI max I/O length */
+ unsigned short udb_density; /* # of user DB/page */
+ unsigned short ucq_density; /* # of user CQs/page */
+ void __iomem *gts_reg; /* address of GTS register */
+ void __iomem *db_reg; /* address of kernel doorbell */
+};
+
+struct cxgb4_uld_info {
+ const char *name;
+ void *(*add)(const struct cxgb4_lld_info *p);
+ int (*rx_handler)(void *handle, const __be64 *rsp,
+ const struct pkt_gl *gl);
+ int (*state_change)(void *handle, enum cxgb4_state new_state);
+};
+
+int cxgb4_register_uld(enum cxgb4_uld type, const struct cxgb4_uld_info *p);
+int cxgb4_unregister_uld(enum cxgb4_uld type);
+int cxgb4_ofld_send(struct net_device *dev, struct sk_buff *skb);
+unsigned int cxgb4_port_chan(const struct net_device *dev);
+unsigned int cxgb4_port_viid(const struct net_device *dev);
+unsigned int cxgb4_port_idx(const struct net_device *dev);
+struct net_device *cxgb4_netdev_by_hwid(struct pci_dev *pdev, unsigned int id);
+unsigned int cxgb4_best_mtu(const unsigned short *mtus, unsigned short mtu,
+ unsigned int *idx);
+void cxgb4_get_tcp_stats(struct pci_dev *pdev, struct tp_tcp_stats *v4,
+ struct tp_tcp_stats *v6);
+void cxgb4_iscsi_init(struct net_device *dev, unsigned int tag_mask,
+ const unsigned int *pgsz_order);
+struct sk_buff *cxgb4_pktgl_to_skb(const struct pkt_gl *gl,
+ unsigned int skb_len, unsigned int pull_len);
+#endif /* !__CXGB4_OFLD_H */
--- /dev/null
+/*
+ * This file is part of the Chelsio T4 Ethernet driver for Linux.
+ *
+ * Copyright (c) 2003-2010 Chelsio Communications, Inc. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include <linux/skbuff.h>
+#include <linux/netdevice.h>
+#include <linux/if.h>
+#include <linux/if_vlan.h>
+#include <linux/jhash.h>
+#include <net/neighbour.h>
+#include "cxgb4.h"
+#include "l2t.h"
+#include "t4_msg.h"
+#include "t4fw_api.h"
+
+#define VLAN_NONE 0xfff
+
+/* identifies sync vs async L2T_WRITE_REQs */
+#define F_SYNC_WR (1 << 12)
+
+enum {
+ L2T_STATE_VALID, /* entry is up to date */
+ L2T_STATE_STALE, /* entry may be used but needs revalidation */
+ L2T_STATE_RESOLVING, /* entry needs address resolution */
+ L2T_STATE_SYNC_WRITE, /* synchronous write of entry underway */
+
+ /* when state is one of the below the entry is not hashed */
+ L2T_STATE_SWITCHING, /* entry is being used by a switching filter */
+ L2T_STATE_UNUSED /* entry not in use */
+};
+
+struct l2t_data {
+ rwlock_t lock;
+ atomic_t nfree; /* number of free entries */
+ struct l2t_entry *rover; /* starting point for next allocation */
+ struct l2t_entry l2tab[L2T_SIZE];
+};
+
+static inline unsigned int vlan_prio(const struct l2t_entry *e)
+{
+ return e->vlan >> 13;
+}
+
+static inline void l2t_hold(struct l2t_data *d, struct l2t_entry *e)
+{
+ if (atomic_add_return(1, &e->refcnt) == 1) /* 0 -> 1 transition */
+ atomic_dec(&d->nfree);
+}
+
+/*
+ * To avoid having to check address families we do not allow v4 and v6
+ * neighbors to be on the same hash chain. We keep v4 entries in the first
+ * half of available hash buckets and v6 in the second.
+ */
+enum {
+ L2T_SZ_HALF = L2T_SIZE / 2,
+ L2T_HASH_MASK = L2T_SZ_HALF - 1
+};
+
+static inline unsigned int arp_hash(const u32 *key, int ifindex)
+{
+ return jhash_2words(*key, ifindex, 0) & L2T_HASH_MASK;
+}
+
+static inline unsigned int ipv6_hash(const u32 *key, int ifindex)
+{
+ u32 xor = key[0] ^ key[1] ^ key[2] ^ key[3];
+
+ return L2T_SZ_HALF + (jhash_2words(xor, ifindex, 0) & L2T_HASH_MASK);
+}
+
+static unsigned int addr_hash(const u32 *addr, int addr_len, int ifindex)
+{
+ return addr_len == 4 ? arp_hash(addr, ifindex) :
+ ipv6_hash(addr, ifindex);
+}
+
+/*
+ * Checks if an L2T entry is for the given IP/IPv6 address. It does not check
+ * whether the L2T entry and the address are of the same address family.
+ * Callers ensure an address is only checked against L2T entries of the same
+ * family, something made trivial by the separation of IP and IPv6 hash chains
+ * mentioned above. Returns 0 if there's a match,
+ */
+static int addreq(const struct l2t_entry *e, const u32 *addr)
+{
+ if (e->v6)
+ return (e->addr[0] ^ addr[0]) | (e->addr[1] ^ addr[1]) |
+ (e->addr[2] ^ addr[2]) | (e->addr[3] ^ addr[3]);
+ return e->addr[0] ^ addr[0];
+}
+
+static void neigh_replace(struct l2t_entry *e, struct neighbour *n)
+{
+ neigh_hold(n);
+ if (e->neigh)
+ neigh_release(e->neigh);
+ e->neigh = n;
+}
+
+/*
+ * Write an L2T entry. Must be called with the entry locked.
+ * The write may be synchronous or asynchronous.
+ */
+static int write_l2e(struct adapter *adap, struct l2t_entry *e, int sync)
+{
+ struct sk_buff *skb;
+ struct cpl_l2t_write_req *req;
+
+ skb = alloc_skb(sizeof(*req), GFP_ATOMIC);
+ if (!skb)
+ return -ENOMEM;
+
+ req = (struct cpl_l2t_write_req *)__skb_put(skb, sizeof(*req));
+ INIT_TP_WR(req, 0);
+
+ OPCODE_TID(req) = htonl(MK_OPCODE_TID(CPL_L2T_WRITE_REQ,
+ e->idx | (sync ? F_SYNC_WR : 0) |
+ TID_QID(adap->sge.fw_evtq.abs_id)));
+ req->params = htons(L2T_W_PORT(e->lport) | L2T_W_NOREPLY(!sync));
+ req->l2t_idx = htons(e->idx);
+ req->vlan = htons(e->vlan);
+ if (e->neigh)
+ memcpy(e->dmac, e->neigh->ha, sizeof(e->dmac));
+ memcpy(req->dst_mac, e->dmac, sizeof(req->dst_mac));
+
+ set_wr_txq(skb, CPL_PRIORITY_CONTROL, 0);
+ t4_ofld_send(adap, skb);
+
+ if (sync && e->state != L2T_STATE_SWITCHING)
+ e->state = L2T_STATE_SYNC_WRITE;
+ return 0;
+}
+
+/*
+ * Send packets waiting in an L2T entry's ARP queue. Must be called with the
+ * entry locked.
+ */
+static void send_pending(struct adapter *adap, struct l2t_entry *e)
+{
+ while (e->arpq_head) {
+ struct sk_buff *skb = e->arpq_head;
+
+ e->arpq_head = skb->next;
+ skb->next = NULL;
+ t4_ofld_send(adap, skb);
+ }
+ e->arpq_tail = NULL;
+}
+
+/*
+ * Process a CPL_L2T_WRITE_RPL. Wake up the ARP queue if it completes a
+ * synchronous L2T_WRITE. Note that the TID in the reply is really the L2T
+ * index it refers to.
+ */
+void do_l2t_write_rpl(struct adapter *adap, const struct cpl_l2t_write_rpl *rpl)
+{
+ unsigned int tid = GET_TID(rpl);
+ unsigned int idx = tid & (L2T_SIZE - 1);
+
+ if (unlikely(rpl->status != CPL_ERR_NONE)) {
+ dev_err(adap->pdev_dev,
+ "Unexpected L2T_WRITE_RPL status %u for entry %u\n",
+ rpl->status, idx);
+ return;
+ }
+
+ if (tid & F_SYNC_WR) {
+ struct l2t_entry *e = &adap->l2t->l2tab[idx];
+
+ spin_lock(&e->lock);
+ if (e->state != L2T_STATE_SWITCHING) {
+ send_pending(adap, e);
+ e->state = (e->neigh->nud_state & NUD_STALE) ?
+ L2T_STATE_STALE : L2T_STATE_VALID;
+ }
+ spin_unlock(&e->lock);
+ }
+}
+
+/*
+ * Add a packet to an L2T entry's queue of packets awaiting resolution.
+ * Must be called with the entry's lock held.
+ */
+static inline void arpq_enqueue(struct l2t_entry *e, struct sk_buff *skb)
+{
+ skb->next = NULL;
+ if (e->arpq_head)
+ e->arpq_tail->next = skb;
+ else
+ e->arpq_head = skb;
+ e->arpq_tail = skb;
+}
+
+int cxgb4_l2t_send(struct net_device *dev, struct sk_buff *skb,
+ struct l2t_entry *e)
+{
+ struct adapter *adap = netdev2adap(dev);
+
+again:
+ switch (e->state) {
+ case L2T_STATE_STALE: /* entry is stale, kick off revalidation */
+ neigh_event_send(e->neigh, NULL);
+ spin_lock_bh(&e->lock);
+ if (e->state == L2T_STATE_STALE)
+ e->state = L2T_STATE_VALID;
+ spin_unlock_bh(&e->lock);
+ case L2T_STATE_VALID: /* fast-path, send the packet on */
+ return t4_ofld_send(adap, skb);
+ case L2T_STATE_RESOLVING:
+ case L2T_STATE_SYNC_WRITE:
+ spin_lock_bh(&e->lock);
+ if (e->state != L2T_STATE_SYNC_WRITE &&
+ e->state != L2T_STATE_RESOLVING) {
+ spin_unlock_bh(&e->lock);
+ goto again;
+ }
+ arpq_enqueue(e, skb);
+ spin_unlock_bh(&e->lock);
+
+ if (e->state == L2T_STATE_RESOLVING &&
+ !neigh_event_send(e->neigh, NULL)) {
+ spin_lock_bh(&e->lock);
+ if (e->state == L2T_STATE_RESOLVING && e->arpq_head)
+ write_l2e(adap, e, 1);
+ spin_unlock_bh(&e->lock);
+ }
+ }
+ return 0;
+}
+EXPORT_SYMBOL(cxgb4_l2t_send);
+
+/*
+ * Allocate a free L2T entry. Must be called with l2t_data.lock held.
+ */
+static struct l2t_entry *alloc_l2e(struct l2t_data *d)
+{
+ struct l2t_entry *end, *e, **p;
+
+ if (!atomic_read(&d->nfree))
+ return NULL;
+
+ /* there's definitely a free entry */
+ for (e = d->rover, end = &d->l2tab[L2T_SIZE]; e != end; ++e)
+ if (atomic_read(&e->refcnt) == 0)
+ goto found;
+
+ for (e = d->l2tab; atomic_read(&e->refcnt); ++e)
+ ;
+found:
+ d->rover = e + 1;
+ atomic_dec(&d->nfree);
+
+ /*
+ * The entry we found may be an inactive entry that is
+ * presently in the hash table. We need to remove it.
+ */
+ if (e->state < L2T_STATE_SWITCHING)
+ for (p = &d->l2tab[e->hash].first; *p; p = &(*p)->next)
+ if (*p == e) {
+ *p = e->next;
+ e->next = NULL;
+ break;
+ }
+
+ e->state = L2T_STATE_UNUSED;
+ return e;
+}
+
+/*
+ * Called when an L2T entry has no more users.
+ */
+static void t4_l2e_free(struct l2t_entry *e)
+{
+ struct l2t_data *d;
+
+ spin_lock_bh(&e->lock);
+ if (atomic_read(&e->refcnt) == 0) { /* hasn't been recycled */
+ if (e->neigh) {
+ neigh_release(e->neigh);
+ e->neigh = NULL;
+ }
+ }
+ spin_unlock_bh(&e->lock);
+
+ d = container_of(e, struct l2t_data, l2tab[e->idx]);
+ atomic_inc(&d->nfree);
+}
+
+void cxgb4_l2t_release(struct l2t_entry *e)
+{
+ if (atomic_dec_and_test(&e->refcnt))
+ t4_l2e_free(e);
+}
+EXPORT_SYMBOL(cxgb4_l2t_release);
+
+/*
+ * Update an L2T entry that was previously used for the same next hop as neigh.
+ * Must be called with softirqs disabled.
+ */
+static void reuse_entry(struct l2t_entry *e, struct neighbour *neigh)
+{
+ unsigned int nud_state;
+
+ spin_lock(&e->lock); /* avoid race with t4_l2t_free */
+ if (neigh != e->neigh)
+ neigh_replace(e, neigh);
+ nud_state = neigh->nud_state;
+ if (memcmp(e->dmac, neigh->ha, sizeof(e->dmac)) ||
+ !(nud_state & NUD_VALID))
+ e->state = L2T_STATE_RESOLVING;
+ else if (nud_state & NUD_CONNECTED)
+ e->state = L2T_STATE_VALID;
+ else
+ e->state = L2T_STATE_STALE;
+ spin_unlock(&e->lock);
+}
+
+struct l2t_entry *cxgb4_l2t_get(struct l2t_data *d, struct neighbour *neigh,
+ const struct net_device *physdev,
+ unsigned int priority)
+{
+ u8 lport;
+ u16 vlan;
+ struct l2t_entry *e;
+ int addr_len = neigh->tbl->key_len;
+ u32 *addr = (u32 *)neigh->primary_key;
+ int ifidx = neigh->dev->ifindex;
+ int hash = addr_hash(addr, addr_len, ifidx);
+
+ if (neigh->dev->flags & IFF_LOOPBACK)
+ lport = netdev2pinfo(physdev)->tx_chan + 4;
+ else
+ lport = netdev2pinfo(physdev)->lport;
+
+ if (neigh->dev->priv_flags & IFF_802_1Q_VLAN)
+ vlan = vlan_dev_vlan_id(neigh->dev);
+ else
+ vlan = VLAN_NONE;
+
+ write_lock_bh(&d->lock);
+ for (e = d->l2tab[hash].first; e; e = e->next)
+ if (!addreq(e, addr) && e->ifindex == ifidx &&
+ e->vlan == vlan && e->lport == lport) {
+ l2t_hold(d, e);
+ if (atomic_read(&e->refcnt) == 1)
+ reuse_entry(e, neigh);
+ goto done;
+ }
+
+ /* Need to allocate a new entry */
+ e = alloc_l2e(d);
+ if (e) {
+ spin_lock(&e->lock); /* avoid race with t4_l2t_free */
+ e->state = L2T_STATE_RESOLVING;
+ memcpy(e->addr, addr, addr_len);
+ e->ifindex = ifidx;
+ e->hash = hash;
+ e->lport = lport;
+ e->v6 = addr_len == 16;
+ atomic_set(&e->refcnt, 1);
+ neigh_replace(e, neigh);
+ e->vlan = vlan;
+ e->next = d->l2tab[hash].first;
+ d->l2tab[hash].first = e;
+ spin_unlock(&e->lock);
+ }
+done:
+ write_unlock_bh(&d->lock);
+ return e;
+}
+EXPORT_SYMBOL(cxgb4_l2t_get);
+
+/*
+ * Called when address resolution fails for an L2T entry to handle packets
+ * on the arpq head. If a packet specifies a failure handler it is invoked,
+ * otherwise the packet is sent to the device.
+ */
+static void handle_failed_resolution(struct adapter *adap, struct sk_buff *arpq)
+{
+ while (arpq) {
+ struct sk_buff *skb = arpq;
+ const struct l2t_skb_cb *cb = L2T_SKB_CB(skb);
+
+ arpq = skb->next;
+ skb->next = NULL;
+ if (cb->arp_err_handler)
+ cb->arp_err_handler(cb->handle, skb);
+ else
+ t4_ofld_send(adap, skb);
+ }
+}
+
+/*
+ * Called when the host's neighbor layer makes a change to some entry that is
+ * loaded into the HW L2 table.
+ */
+void t4_l2t_update(struct adapter *adap, struct neighbour *neigh)
+{
+ struct l2t_entry *e;
+ struct sk_buff *arpq = NULL;
+ struct l2t_data *d = adap->l2t;
+ int addr_len = neigh->tbl->key_len;
+ u32 *addr = (u32 *) neigh->primary_key;
+ int ifidx = neigh->dev->ifindex;
+ int hash = addr_hash(addr, addr_len, ifidx);
+
+ read_lock_bh(&d->lock);
+ for (e = d->l2tab[hash].first; e; e = e->next)
+ if (!addreq(e, addr) && e->ifindex == ifidx) {
+ spin_lock(&e->lock);
+ if (atomic_read(&e->refcnt))
+ goto found;
+ spin_unlock(&e->lock);
+ break;
+ }
+ read_unlock_bh(&d->lock);
+ return;
+
+ found:
+ read_unlock(&d->lock);
+
+ if (neigh != e->neigh)
+ neigh_replace(e, neigh);
+
+ if (e->state == L2T_STATE_RESOLVING) {
+ if (neigh->nud_state & NUD_FAILED) {
+ arpq = e->arpq_head;
+ e->arpq_head = e->arpq_tail = NULL;
+ } else if ((neigh->nud_state & (NUD_CONNECTED | NUD_STALE)) &&
+ e->arpq_head) {
+ write_l2e(adap, e, 1);
+ }
+ } else {
+ e->state = neigh->nud_state & NUD_CONNECTED ?
+ L2T_STATE_VALID : L2T_STATE_STALE;
+ if (memcmp(e->dmac, neigh->ha, sizeof(e->dmac)))
+ write_l2e(adap, e, 0);
+ }
+
+ spin_unlock_bh(&e->lock);
+
+ if (arpq)
+ handle_failed_resolution(adap, arpq);
+}
+
+/*
+ * Allocate an L2T entry for use by a switching rule. Such entries need to be
+ * explicitly freed and while busy they are not on any hash chain, so normal
+ * address resolution updates do not see them.
+ */
+struct l2t_entry *t4_l2t_alloc_switching(struct l2t_data *d)
+{
+ struct l2t_entry *e;
+
+ write_lock_bh(&d->lock);
+ e = alloc_l2e(d);
+ if (e) {
+ spin_lock(&e->lock); /* avoid race with t4_l2t_free */
+ e->state = L2T_STATE_SWITCHING;
+ atomic_set(&e->refcnt, 1);
+ spin_unlock(&e->lock);
+ }
+ write_unlock_bh(&d->lock);
+ return e;
+}
+
+/*
+ * Sets/updates the contents of a switching L2T entry that has been allocated
+ * with an earlier call to @t4_l2t_alloc_switching.
+ */
+int t4_l2t_set_switching(struct adapter *adap, struct l2t_entry *e, u16 vlan,
+ u8 port, u8 *eth_addr)
+{
+ e->vlan = vlan;
+ e->lport = port;
+ memcpy(e->dmac, eth_addr, ETH_ALEN);
+ return write_l2e(adap, e, 0);
+}
+
+struct l2t_data *t4_init_l2t(void)
+{
+ int i;
+ struct l2t_data *d;
+
+ d = t4_alloc_mem(sizeof(*d));
+ if (!d)
+ return NULL;
+
+ d->rover = d->l2tab;
+ atomic_set(&d->nfree, L2T_SIZE);
+ rwlock_init(&d->lock);
+
+ for (i = 0; i < L2T_SIZE; ++i) {
+ d->l2tab[i].idx = i;
+ d->l2tab[i].state = L2T_STATE_UNUSED;
+ spin_lock_init(&d->l2tab[i].lock);
+ atomic_set(&d->l2tab[i].refcnt, 0);
+ }
+ return d;
+}
+
+#include <linux/module.h>
+#include <linux/debugfs.h>
+#include <linux/seq_file.h>
+
+static inline void *l2t_get_idx(struct seq_file *seq, loff_t pos)
+{
+ struct l2t_entry *l2tab = seq->private;
+
+ return pos >= L2T_SIZE ? NULL : &l2tab[pos];
+}
+
+static void *l2t_seq_start(struct seq_file *seq, loff_t *pos)
+{
+ return *pos ? l2t_get_idx(seq, *pos - 1) : SEQ_START_TOKEN;
+}
+
+static void *l2t_seq_next(struct seq_file *seq, void *v, loff_t *pos)
+{
+ v = l2t_get_idx(seq, *pos);
+ if (v)
+ ++*pos;
+ return v;
+}
+
+static void l2t_seq_stop(struct seq_file *seq, void *v)
+{
+}
+
+static char l2e_state(const struct l2t_entry *e)
+{
+ switch (e->state) {
+ case L2T_STATE_VALID: return 'V';
+ case L2T_STATE_STALE: return 'S';
+ case L2T_STATE_SYNC_WRITE: return 'W';
+ case L2T_STATE_RESOLVING: return e->arpq_head ? 'A' : 'R';
+ case L2T_STATE_SWITCHING: return 'X';
+ default:
+ return 'U';
+ }
+}
+
+static int l2t_seq_show(struct seq_file *seq, void *v)
+{
+ if (v == SEQ_START_TOKEN)
+ seq_puts(seq, " Idx IP address "
+ "Ethernet address VLAN/P LP State Users Port\n");
+ else {
+ char ip[60];
+ struct l2t_entry *e = v;
+
+ spin_lock_bh(&e->lock);
+ if (e->state == L2T_STATE_SWITCHING)
+ ip[0] = '\0';
+ else
+ sprintf(ip, e->v6 ? "%pI6c" : "%pI4", e->addr);
+ seq_printf(seq, "%4u %-25s %17pM %4d %u %2u %c %5u %s\n",
+ e->idx, ip, e->dmac,
+ e->vlan & VLAN_VID_MASK, vlan_prio(e), e->lport,
+ l2e_state(e), atomic_read(&e->refcnt),
+ e->neigh ? e->neigh->dev->name : "");
+ spin_unlock_bh(&e->lock);
+ }
+ return 0;
+}
+
+static const struct seq_operations l2t_seq_ops = {
+ .start = l2t_seq_start,
+ .next = l2t_seq_next,
+ .stop = l2t_seq_stop,
+ .show = l2t_seq_show
+};
+
+static int l2t_seq_open(struct inode *inode, struct file *file)
+{
+ int rc = seq_open(file, &l2t_seq_ops);
+
+ if (!rc) {
+ struct adapter *adap = inode->i_private;
+ struct seq_file *seq = file->private_data;
+
+ seq->private = adap->l2t->l2tab;
+ }
+ return rc;
+}
+
+const struct file_operations t4_l2t_fops = {
+ .owner = THIS_MODULE,
+ .open = l2t_seq_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = seq_release,
+};
--- /dev/null
+/*
+ * This file is part of the Chelsio T4 Ethernet driver for Linux.
+ *
+ * Copyright (c) 2003-2010 Chelsio Communications, Inc. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef __CXGB4_L2T_H
+#define __CXGB4_L2T_H
+
+#include <linux/spinlock.h>
+#include <linux/if_ether.h>
+#include <asm/atomic.h>
+
+struct adapter;
+struct l2t_data;
+struct neighbour;
+struct net_device;
+struct file_operations;
+struct cpl_l2t_write_rpl;
+
+/*
+ * Each L2T entry plays multiple roles. First of all, it keeps state for the
+ * corresponding entry of the HW L2 table and maintains a queue of offload
+ * packets awaiting address resolution. Second, it is a node of a hash table
+ * chain, where the nodes of the chain are linked together through their next
+ * pointer. Finally, each node is a bucket of a hash table, pointing to the
+ * first element in its chain through its first pointer.
+ */
+struct l2t_entry {
+ u16 state; /* entry state */
+ u16 idx; /* entry index */
+ u32 addr[4]; /* next hop IP or IPv6 address */
+ int ifindex; /* neighbor's net_device's ifindex */
+ struct neighbour *neigh; /* associated neighbour */
+ struct l2t_entry *first; /* start of hash chain */
+ struct l2t_entry *next; /* next l2t_entry on chain */
+ struct sk_buff *arpq_head; /* queue of packets awaiting resolution */
+ struct sk_buff *arpq_tail;
+ spinlock_t lock;
+ atomic_t refcnt; /* entry reference count */
+ u16 hash; /* hash bucket the entry is on */
+ u16 vlan; /* VLAN TCI (id: bits 0-11, prio: 13-15 */
+ u8 v6; /* whether entry is for IPv6 */
+ u8 lport; /* associated offload logical interface */
+ u8 dmac[ETH_ALEN]; /* neighbour's MAC address */
+};
+
+typedef void (*arp_err_handler_t)(void *handle, struct sk_buff *skb);
+
+/*
+ * Callback stored in an skb to handle address resolution failure.
+ */
+struct l2t_skb_cb {
+ void *handle;
+ arp_err_handler_t arp_err_handler;
+};
+
+#define L2T_SKB_CB(skb) ((struct l2t_skb_cb *)(skb)->cb)
+
+static inline void t4_set_arp_err_handler(struct sk_buff *skb, void *handle,
+ arp_err_handler_t handler)
+{
+ L2T_SKB_CB(skb)->handle = handle;
+ L2T_SKB_CB(skb)->arp_err_handler = handler;
+}
+
+void cxgb4_l2t_release(struct l2t_entry *e);
+int cxgb4_l2t_send(struct net_device *dev, struct sk_buff *skb,
+ struct l2t_entry *e);
+struct l2t_entry *cxgb4_l2t_get(struct l2t_data *d, struct neighbour *neigh,
+ const struct net_device *physdev,
+ unsigned int priority);
+
+void t4_l2t_update(struct adapter *adap, struct neighbour *neigh);
+struct l2t_entry *t4_l2t_alloc_switching(struct l2t_data *d);
+int t4_l2t_set_switching(struct adapter *adap, struct l2t_entry *e, u16 vlan,
+ u8 port, u8 *eth_addr);
+struct l2t_data *t4_init_l2t(void);
+void do_l2t_write_rpl(struct adapter *p, const struct cpl_l2t_write_rpl *rpl);
+
+extern const struct file_operations t4_l2t_fops;
+#endif /* __CXGB4_L2T_H */
--- /dev/null
+/*
+ * This file is part of the Chelsio T4 Ethernet driver for Linux.
+ *
+ * Copyright (c) 2003-2010 Chelsio Communications, Inc. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include <linux/skbuff.h>
+#include <linux/netdevice.h>
+#include <linux/etherdevice.h>
+#include <linux/if_vlan.h>
+#include <linux/ip.h>
+#include <linux/dma-mapping.h>
+#include <linux/jiffies.h>
+#include <net/ipv6.h>
+#include <net/tcp.h>
+#include "cxgb4.h"
+#include "t4_regs.h"
+#include "t4_msg.h"
+#include "t4fw_api.h"
+
+/*
+ * Rx buffer size. We use largish buffers if possible but settle for single
+ * pages under memory shortage.
+ */
+#if PAGE_SHIFT >= 16
+# define FL_PG_ORDER 0
+#else
+# define FL_PG_ORDER (16 - PAGE_SHIFT)
+#endif
+
+/* RX_PULL_LEN should be <= RX_COPY_THRES */
+#define RX_COPY_THRES 256
+#define RX_PULL_LEN 128
+
+/*
+ * Main body length for sk_buffs used for Rx Ethernet packets with fragments.
+ * Should be >= RX_PULL_LEN but possibly bigger to give pskb_may_pull some room.
+ */
+#define RX_PKT_SKB_LEN 512
+
+/* Ethernet header padding prepended to RX_PKTs */
+#define RX_PKT_PAD 2
+
+/*
+ * Max number of Tx descriptors we clean up at a time. Should be modest as
+ * freeing skbs isn't cheap and it happens while holding locks. We just need
+ * to free packets faster than they arrive, we eventually catch up and keep
+ * the amortized cost reasonable. Must be >= 2 * TXQ_STOP_THRES.
+ */
+#define MAX_TX_RECLAIM 16
+
+/*
+ * Max number of Rx buffers we replenish at a time. Again keep this modest,
+ * allocating buffers isn't cheap either.
+ */
+#define MAX_RX_REFILL 16U
+
+/*
+ * Period of the Rx queue check timer. This timer is infrequent as it has
+ * something to do only when the system experiences severe memory shortage.
+ */
+#define RX_QCHECK_PERIOD (HZ / 2)
+
+/*
+ * Period of the Tx queue check timer.
+ */
+#define TX_QCHECK_PERIOD (HZ / 2)
+
+/*
+ * Max number of Tx descriptors to be reclaimed by the Tx timer.
+ */
+#define MAX_TIMER_TX_RECLAIM 100
+
+/*
+ * Timer index used when backing off due to memory shortage.
+ */
+#define NOMEM_TMR_IDX (SGE_NTIMERS - 1)
+
+/*
+ * An FL with <= FL_STARVE_THRES buffers is starving and a periodic timer will
+ * attempt to refill it.
+ */
+#define FL_STARVE_THRES 4
+
+/*
+ * Suspend an Ethernet Tx queue with fewer available descriptors than this.
+ * This is the same as calc_tx_descs() for a TSO packet with
+ * nr_frags == MAX_SKB_FRAGS.
+ */
+#define ETHTXQ_STOP_THRES \
+ (1 + DIV_ROUND_UP((3 * MAX_SKB_FRAGS) / 2 + (MAX_SKB_FRAGS & 1), 8))
+
+/*
+ * Suspension threshold for non-Ethernet Tx queues. We require enough room
+ * for a full sized WR.
+ */
+#define TXQ_STOP_THRES (SGE_MAX_WR_LEN / sizeof(struct tx_desc))
+
+/*
+ * Max Tx descriptor space we allow for an Ethernet packet to be inlined
+ * into a WR.
+ */
+#define MAX_IMM_TX_PKT_LEN 128
+
+/*
+ * Max size of a WR sent through a control Tx queue.
+ */
+#define MAX_CTRL_WR_LEN SGE_MAX_WR_LEN
+
+enum {
+ /* packet alignment in FL buffers */
+ FL_ALIGN = L1_CACHE_BYTES < 32 ? 32 : L1_CACHE_BYTES,
+ /* egress status entry size */
+ STAT_LEN = L1_CACHE_BYTES > 64 ? 128 : 64
+};
+
+struct tx_sw_desc { /* SW state per Tx descriptor */
+ struct sk_buff *skb;
+ struct ulptx_sgl *sgl;
+};
+
+struct rx_sw_desc { /* SW state per Rx descriptor */
+ struct page *page;
+ dma_addr_t dma_addr;
+};
+
+/*
+ * The low bits of rx_sw_desc.dma_addr have special meaning.
+ */
+enum {
+ RX_LARGE_BUF = 1 << 0, /* buffer is larger than PAGE_SIZE */
+ RX_UNMAPPED_BUF = 1 << 1, /* buffer is not mapped */
+};
+
+static inline dma_addr_t get_buf_addr(const struct rx_sw_desc *d)
+{
+ return d->dma_addr & ~(dma_addr_t)(RX_LARGE_BUF | RX_UNMAPPED_BUF);
+}
+
+static inline bool is_buf_mapped(const struct rx_sw_desc *d)
+{
+ return !(d->dma_addr & RX_UNMAPPED_BUF);
+}
+
+/**
+ * txq_avail - return the number of available slots in a Tx queue
+ * @q: the Tx queue
+ *
+ * Returns the number of descriptors in a Tx queue available to write new
+ * packets.
+ */
+static inline unsigned int txq_avail(const struct sge_txq *q)
+{
+ return q->size - 1 - q->in_use;
+}
+
+/**
+ * fl_cap - return the capacity of a free-buffer list
+ * @fl: the FL
+ *
+ * Returns the capacity of a free-buffer list. The capacity is less than
+ * the size because one descriptor needs to be left unpopulated, otherwise
+ * HW will think the FL is empty.
+ */
+static inline unsigned int fl_cap(const struct sge_fl *fl)
+{
+ return fl->size - 8; /* 1 descriptor = 8 buffers */
+}
+
+static inline bool fl_starving(const struct sge_fl *fl)
+{
+ return fl->avail - fl->pend_cred <= FL_STARVE_THRES;
+}
+
+static int map_skb(struct device *dev, const struct sk_buff *skb,
+ dma_addr_t *addr)
+{
+ const skb_frag_t *fp, *end;
+ const struct skb_shared_info *si;
+
+ *addr = dma_map_single(dev, skb->data, skb_headlen(skb), DMA_TO_DEVICE);
+ if (dma_mapping_error(dev, *addr))
+ goto out_err;
+
+ si = skb_shinfo(skb);
+ end = &si->frags[si->nr_frags];
+
+ for (fp = si->frags; fp < end; fp++) {
+ *++addr = dma_map_page(dev, fp->page, fp->page_offset, fp->size,
+ DMA_TO_DEVICE);
+ if (dma_mapping_error(dev, *addr))
+ goto unwind;
+ }
+ return 0;
+
+unwind:
+ while (fp-- > si->frags)
+ dma_unmap_page(dev, *--addr, fp->size, DMA_TO_DEVICE);
+
+ dma_unmap_single(dev, addr[-1], skb_headlen(skb), DMA_TO_DEVICE);
+out_err:
+ return -ENOMEM;
+}
+
+#ifdef CONFIG_NEED_DMA_MAP_STATE
+static void unmap_skb(struct device *dev, const struct sk_buff *skb,
+ const dma_addr_t *addr)
+{
+ const skb_frag_t *fp, *end;
+ const struct skb_shared_info *si;
+
+ dma_unmap_single(dev, *addr++, skb_headlen(skb), DMA_TO_DEVICE);
+
+ si = skb_shinfo(skb);
+ end = &si->frags[si->nr_frags];
+ for (fp = si->frags; fp < end; fp++)
+ dma_unmap_page(dev, *addr++, fp->size, DMA_TO_DEVICE);
+}
+
+/**
+ * deferred_unmap_destructor - unmap a packet when it is freed
+ * @skb: the packet
+ *
+ * This is the packet destructor used for Tx packets that need to remain
+ * mapped until they are freed rather than until their Tx descriptors are
+ * freed.
+ */
+static void deferred_unmap_destructor(struct sk_buff *skb)
+{
+ unmap_skb(skb->dev->dev.parent, skb, (dma_addr_t *)skb->head);
+}
+#endif
+
+static void unmap_sgl(struct device *dev, const struct sk_buff *skb,
+ const struct ulptx_sgl *sgl, const struct sge_txq *q)
+{
+ const struct ulptx_sge_pair *p;
+ unsigned int nfrags = skb_shinfo(skb)->nr_frags;
+
+ if (likely(skb_headlen(skb)))
+ dma_unmap_single(dev, be64_to_cpu(sgl->addr0), ntohl(sgl->len0),
+ DMA_TO_DEVICE);
+ else {
+ dma_unmap_page(dev, be64_to_cpu(sgl->addr0), ntohl(sgl->len0),
+ DMA_TO_DEVICE);
+ nfrags--;
+ }
+
+ /*
+ * the complexity below is because of the possibility of a wrap-around
+ * in the middle of an SGL
+ */
+ for (p = sgl->sge; nfrags >= 2; nfrags -= 2) {
+ if (likely((u8 *)(p + 1) <= (u8 *)q->stat)) {
+unmap: dma_unmap_page(dev, be64_to_cpu(p->addr[0]),
+ ntohl(p->len[0]), DMA_TO_DEVICE);
+ dma_unmap_page(dev, be64_to_cpu(p->addr[1]),
+ ntohl(p->len[1]), DMA_TO_DEVICE);
+ p++;
+ } else if ((u8 *)p == (u8 *)q->stat) {
+ p = (const struct ulptx_sge_pair *)q->desc;
+ goto unmap;
+ } else if ((u8 *)p + 8 == (u8 *)q->stat) {
+ const __be64 *addr = (const __be64 *)q->desc;
+
+ dma_unmap_page(dev, be64_to_cpu(addr[0]),
+ ntohl(p->len[0]), DMA_TO_DEVICE);
+ dma_unmap_page(dev, be64_to_cpu(addr[1]),
+ ntohl(p->len[1]), DMA_TO_DEVICE);
+ p = (const struct ulptx_sge_pair *)&addr[2];
+ } else {
+ const __be64 *addr = (const __be64 *)q->desc;
+
+ dma_unmap_page(dev, be64_to_cpu(p->addr[0]),
+ ntohl(p->len[0]), DMA_TO_DEVICE);
+ dma_unmap_page(dev, be64_to_cpu(addr[0]),
+ ntohl(p->len[1]), DMA_TO_DEVICE);
+ p = (const struct ulptx_sge_pair *)&addr[1];
+ }
+ }
+ if (nfrags) {
+ __be64 addr;
+
+ if ((u8 *)p == (u8 *)q->stat)
+ p = (const struct ulptx_sge_pair *)q->desc;
+ addr = (u8 *)p + 16 <= (u8 *)q->stat ? p->addr[0] :
+ *(const __be64 *)q->desc;
+ dma_unmap_page(dev, be64_to_cpu(addr), ntohl(p->len[0]),
+ DMA_TO_DEVICE);
+ }
+}
+
+/**
+ * free_tx_desc - reclaims Tx descriptors and their buffers
+ * @adapter: the adapter
+ * @q: the Tx queue to reclaim descriptors from
+ * @n: the number of descriptors to reclaim
+ * @unmap: whether the buffers should be unmapped for DMA
+ *
+ * Reclaims Tx descriptors from an SGE Tx queue and frees the associated
+ * Tx buffers. Called with the Tx queue lock held.
+ */
+static void free_tx_desc(struct adapter *adap, struct sge_txq *q,
+ unsigned int n, bool unmap)
+{
+ struct tx_sw_desc *d;
+ unsigned int cidx = q->cidx;
+ struct device *dev = adap->pdev_dev;
+
+ d = &q->sdesc[cidx];
+ while (n--) {
+ if (d->skb) { /* an SGL is present */
+ if (unmap)
+ unmap_sgl(dev, d->skb, d->sgl, q);
+ kfree_skb(d->skb);
+ d->skb = NULL;
+ }
+ ++d;
+ if (++cidx == q->size) {
+ cidx = 0;
+ d = q->sdesc;
+ }
+ }
+ q->cidx = cidx;
+}
+
+/*
+ * Return the number of reclaimable descriptors in a Tx queue.
+ */
+static inline int reclaimable(const struct sge_txq *q)
+{
+ int hw_cidx = ntohs(q->stat->cidx);
+ hw_cidx -= q->cidx;
+ return hw_cidx < 0 ? hw_cidx + q->size : hw_cidx;
+}
+
+/**
+ * reclaim_completed_tx - reclaims completed Tx descriptors
+ * @adap: the adapter
+ * @q: the Tx queue to reclaim completed descriptors from
+ * @unmap: whether the buffers should be unmapped for DMA
+ *
+ * Reclaims Tx descriptors that the SGE has indicated it has processed,
+ * and frees the associated buffers if possible. Called with the Tx
+ * queue locked.
+ */
+static inline void reclaim_completed_tx(struct adapter *adap, struct sge_txq *q,
+ bool unmap)
+{
+ int avail = reclaimable(q);
+
+ if (avail) {
+ /*
+ * Limit the amount of clean up work we do at a time to keep
+ * the Tx lock hold time O(1).
+ */
+ if (avail > MAX_TX_RECLAIM)
+ avail = MAX_TX_RECLAIM;
+
+ free_tx_desc(adap, q, avail, unmap);
+ q->in_use -= avail;
+ }
+}
+
+static inline int get_buf_size(const struct rx_sw_desc *d)
+{
+#if FL_PG_ORDER > 0
+ return (d->dma_addr & RX_LARGE_BUF) ? (PAGE_SIZE << FL_PG_ORDER) :
+ PAGE_SIZE;
+#else
+ return PAGE_SIZE;
+#endif
+}
+
+/**
+ * free_rx_bufs - free the Rx buffers on an SGE free list
+ * @adap: the adapter
+ * @q: the SGE free list to free buffers from
+ * @n: how many buffers to free
+ *
+ * Release the next @n buffers on an SGE free-buffer Rx queue. The
+ * buffers must be made inaccessible to HW before calling this function.
+ */
+static void free_rx_bufs(struct adapter *adap, struct sge_fl *q, int n)
+{
+ while (n--) {
+ struct rx_sw_desc *d = &q->sdesc[q->cidx];
+
+ if (is_buf_mapped(d))
+ dma_unmap_page(adap->pdev_dev, get_buf_addr(d),
+ get_buf_size(d), PCI_DMA_FROMDEVICE);
+ put_page(d->page);
+ d->page = NULL;
+ if (++q->cidx == q->size)
+ q->cidx = 0;
+ q->avail--;
+ }
+}
+
+/**
+ * unmap_rx_buf - unmap the current Rx buffer on an SGE free list
+ * @adap: the adapter
+ * @q: the SGE free list
+ *
+ * Unmap the current buffer on an SGE free-buffer Rx queue. The
+ * buffer must be made inaccessible to HW before calling this function.
+ *
+ * This is similar to @free_rx_bufs above but does not free the buffer.
+ * Do note that the FL still loses any further access to the buffer.
+ */
+static void unmap_rx_buf(struct adapter *adap, struct sge_fl *q)
+{
+ struct rx_sw_desc *d = &q->sdesc[q->cidx];
+
+ if (is_buf_mapped(d))
+ dma_unmap_page(adap->pdev_dev, get_buf_addr(d),
+ get_buf_size(d), PCI_DMA_FROMDEVICE);
+ d->page = NULL;
+ if (++q->cidx == q->size)
+ q->cidx = 0;
+ q->avail--;
+}
+
+static inline void ring_fl_db(struct adapter *adap, struct sge_fl *q)
+{
+ if (q->pend_cred >= 8) {
+ wmb();
+ t4_write_reg(adap, MYPF_REG(SGE_PF_KDOORBELL), DBPRIO |
+ QID(q->cntxt_id) | PIDX(q->pend_cred / 8));
+ q->pend_cred &= 7;
+ }
+}
+
+static inline void set_rx_sw_desc(struct rx_sw_desc *sd, struct page *pg,
+ dma_addr_t mapping)
+{
+ sd->page = pg;
+ sd->dma_addr = mapping; /* includes size low bits */
+}
+
+/**
+ * refill_fl - refill an SGE Rx buffer ring
+ * @adap: the adapter
+ * @q: the ring to refill
+ * @n: the number of new buffers to allocate
+ * @gfp: the gfp flags for the allocations
+ *
+ * (Re)populate an SGE free-buffer queue with up to @n new packet buffers,
+ * allocated with the supplied gfp flags. The caller must assure that
+ * @n does not exceed the queue's capacity. If afterwards the queue is
+ * found critically low mark it as starving in the bitmap of starving FLs.
+ *
+ * Returns the number of buffers allocated.
+ */
+static unsigned int refill_fl(struct adapter *adap, struct sge_fl *q, int n,
+ gfp_t gfp)
+{
+ struct page *pg;
+ dma_addr_t mapping;
+ unsigned int cred = q->avail;
+ __be64 *d = &q->desc[q->pidx];
+ struct rx_sw_desc *sd = &q->sdesc[q->pidx];
+
+ gfp |= __GFP_NOWARN; /* failures are expected */
+
+#if FL_PG_ORDER > 0
+ /*
+ * Prefer large buffers
+ */
+ while (n) {
+ pg = alloc_pages(gfp | __GFP_COMP, FL_PG_ORDER);
+ if (unlikely(!pg)) {
+ q->large_alloc_failed++;
+ break; /* fall back to single pages */
+ }
+
+ mapping = dma_map_page(adap->pdev_dev, pg, 0,
+ PAGE_SIZE << FL_PG_ORDER,
+ PCI_DMA_FROMDEVICE);
+ if (unlikely(dma_mapping_error(adap->pdev_dev, mapping))) {
+ __free_pages(pg, FL_PG_ORDER);
+ goto out; /* do not try small pages for this error */
+ }
+ mapping |= RX_LARGE_BUF;
+ *d++ = cpu_to_be64(mapping);
+
+ set_rx_sw_desc(sd, pg, mapping);
+ sd++;
+
+ q->avail++;
+ if (++q->pidx == q->size) {
+ q->pidx = 0;
+ sd = q->sdesc;
+ d = q->desc;
+ }
+ n--;
+ }
+#endif
+
+ while (n--) {
+ pg = __netdev_alloc_page(adap->port[0], gfp);
+ if (unlikely(!pg)) {
+ q->alloc_failed++;
+ break;
+ }
+
+ mapping = dma_map_page(adap->pdev_dev, pg, 0, PAGE_SIZE,
+ PCI_DMA_FROMDEVICE);
+ if (unlikely(dma_mapping_error(adap->pdev_dev, mapping))) {
+ netdev_free_page(adap->port[0], pg);
+ goto out;
+ }
+ *d++ = cpu_to_be64(mapping);
+
+ set_rx_sw_desc(sd, pg, mapping);
+ sd++;
+
+ q->avail++;
+ if (++q->pidx == q->size) {
+ q->pidx = 0;
+ sd = q->sdesc;
+ d = q->desc;
+ }
+ }
+
+out: cred = q->avail - cred;
+ q->pend_cred += cred;
+ ring_fl_db(adap, q);
+
+ if (unlikely(fl_starving(q))) {
+ smp_wmb();
+ set_bit(q->cntxt_id, adap->sge.starving_fl);
+ }
+
+ return cred;
+}
+
+static inline void __refill_fl(struct adapter *adap, struct sge_fl *fl)
+{
+ refill_fl(adap, fl, min(MAX_RX_REFILL, fl_cap(fl) - fl->avail),
+ GFP_ATOMIC);
+}
+
+/**
+ * alloc_ring - allocate resources for an SGE descriptor ring
+ * @dev: the PCI device's core device
+ * @nelem: the number of descriptors
+ * @elem_size: the size of each descriptor
+ * @sw_size: the size of the SW state associated with each ring element
+ * @phys: the physical address of the allocated ring
+ * @metadata: address of the array holding the SW state for the ring
+ * @stat_size: extra space in HW ring for status information
+ *
+ * Allocates resources for an SGE descriptor ring, such as Tx queues,
+ * free buffer lists, or response queues. Each SGE ring requires
+ * space for its HW descriptors plus, optionally, space for the SW state
+ * associated with each HW entry (the metadata). The function returns
+ * three values: the virtual address for the HW ring (the return value
+ * of the function), the bus address of the HW ring, and the address
+ * of the SW ring.
+ */
+static void *alloc_ring(struct device *dev, size_t nelem, size_t elem_size,
+ size_t sw_size, dma_addr_t *phys, void *metadata,
+ size_t stat_size)
+{
+ size_t len = nelem * elem_size + stat_size;
+ void *s = NULL;
+ void *p = dma_alloc_coherent(dev, len, phys, GFP_KERNEL);
+
+ if (!p)
+ return NULL;
+ if (sw_size) {
+ s = kcalloc(nelem, sw_size, GFP_KERNEL);
+
+ if (!s) {
+ dma_free_coherent(dev, len, p, *phys);
+ return NULL;
+ }
+ }
+ if (metadata)
+ *(void **)metadata = s;
+ memset(p, 0, len);
+ return p;
+}
+
+/**
+ * sgl_len - calculates the size of an SGL of the given capacity
+ * @n: the number of SGL entries
+ *
+ * Calculates the number of flits needed for a scatter/gather list that
+ * can hold the given number of entries.
+ */
+static inline unsigned int sgl_len(unsigned int n)
+{
+ n--;
+ return (3 * n) / 2 + (n & 1) + 2;
+}
+
+/**
+ * flits_to_desc - returns the num of Tx descriptors for the given flits
+ * @n: the number of flits
+ *
+ * Returns the number of Tx descriptors needed for the supplied number
+ * of flits.
+ */
+static inline unsigned int flits_to_desc(unsigned int n)
+{
+ BUG_ON(n > SGE_MAX_WR_LEN / 8);
+ return DIV_ROUND_UP(n, 8);
+}
+
+/**
+ * is_eth_imm - can an Ethernet packet be sent as immediate data?
+ * @skb: the packet
+ *
+ * Returns whether an Ethernet packet is small enough to fit as
+ * immediate data.
+ */
+static inline int is_eth_imm(const struct sk_buff *skb)
+{
+ return skb->len <= MAX_IMM_TX_PKT_LEN - sizeof(struct cpl_tx_pkt);
+}
+
+/**
+ * calc_tx_flits - calculate the number of flits for a packet Tx WR
+ * @skb: the packet
+ *
+ * Returns the number of flits needed for a Tx WR for the given Ethernet
+ * packet, including the needed WR and CPL headers.
+ */
+static inline unsigned int calc_tx_flits(const struct sk_buff *skb)
+{
+ unsigned int flits;
+
+ if (is_eth_imm(skb))
+ return DIV_ROUND_UP(skb->len + sizeof(struct cpl_tx_pkt), 8);
+
+ flits = sgl_len(skb_shinfo(skb)->nr_frags + 1) + 4;
+ if (skb_shinfo(skb)->gso_size)
+ flits += 2;
+ return flits;
+}
+
+/**
+ * calc_tx_descs - calculate the number of Tx descriptors for a packet
+ * @skb: the packet
+ *
+ * Returns the number of Tx descriptors needed for the given Ethernet
+ * packet, including the needed WR and CPL headers.
+ */
+static inline unsigned int calc_tx_descs(const struct sk_buff *skb)
+{
+ return flits_to_desc(calc_tx_flits(skb));
+}
+
+/**
+ * write_sgl - populate a scatter/gather list for a packet
+ * @skb: the packet
+ * @q: the Tx queue we are writing into
+ * @sgl: starting location for writing the SGL
+ * @end: points right after the end of the SGL
+ * @start: start offset into skb main-body data to include in the SGL
+ * @addr: the list of bus addresses for the SGL elements
+ *
+ * Generates a gather list for the buffers that make up a packet.
+ * The caller must provide adequate space for the SGL that will be written.
+ * The SGL includes all of the packet's page fragments and the data in its
+ * main body except for the first @start bytes. @sgl must be 16-byte
+ * aligned and within a Tx descriptor with available space. @end points
+ * right after the end of the SGL but does not account for any potential
+ * wrap around, i.e., @end > @sgl.
+ */
+static void write_sgl(const struct sk_buff *skb, struct sge_txq *q,
+ struct ulptx_sgl *sgl, u64 *end, unsigned int start,
+ const dma_addr_t *addr)
+{
+ unsigned int i, len;
+ struct ulptx_sge_pair *to;
+ const struct skb_shared_info *si = skb_shinfo(skb);
+ unsigned int nfrags = si->nr_frags;
+ struct ulptx_sge_pair buf[MAX_SKB_FRAGS / 2 + 1];
+
+ len = skb_headlen(skb) - start;
+ if (likely(len)) {
+ sgl->len0 = htonl(len);
+ sgl->addr0 = cpu_to_be64(addr[0] + start);
+ nfrags++;
+ } else {
+ sgl->len0 = htonl(si->frags[0].size);
+ sgl->addr0 = cpu_to_be64(addr[1]);
+ }
+
+ sgl->cmd_nsge = htonl(ULPTX_CMD(ULP_TX_SC_DSGL) | ULPTX_NSGE(nfrags));
+ if (likely(--nfrags == 0))
+ return;
+ /*
+ * Most of the complexity below deals with the possibility we hit the
+ * end of the queue in the middle of writing the SGL. For this case
+ * only we create the SGL in a temporary buffer and then copy it.
+ */
+ to = (u8 *)end > (u8 *)q->stat ? buf : sgl->sge;
+
+ for (i = (nfrags != si->nr_frags); nfrags >= 2; nfrags -= 2, to++) {
+ to->len[0] = cpu_to_be32(si->frags[i].size);
+ to->len[1] = cpu_to_be32(si->frags[++i].size);
+ to->addr[0] = cpu_to_be64(addr[i]);
+ to->addr[1] = cpu_to_be64(addr[++i]);
+ }
+ if (nfrags) {
+ to->len[0] = cpu_to_be32(si->frags[i].size);
+ to->len[1] = cpu_to_be32(0);
+ to->addr[0] = cpu_to_be64(addr[i + 1]);
+ }
+ if (unlikely((u8 *)end > (u8 *)q->stat)) {
+ unsigned int part0 = (u8 *)q->stat - (u8 *)sgl->sge, part1;
+
+ if (likely(part0))
+ memcpy(sgl->sge, buf, part0);
+ part1 = (u8 *)end - (u8 *)q->stat;
+ memcpy(q->desc, (u8 *)buf + part0, part1);
+ end = (void *)q->desc + part1;
+ }
+ if ((uintptr_t)end & 8) /* 0-pad to multiple of 16 */
+ *(u64 *)end = 0;
+}
+
+/**
+ * ring_tx_db - check and potentially ring a Tx queue's doorbell
+ * @adap: the adapter
+ * @q: the Tx queue
+ * @n: number of new descriptors to give to HW
+ *
+ * Ring the doorbel for a Tx queue.
+ */
+static inline void ring_tx_db(struct adapter *adap, struct sge_txq *q, int n)
+{
+ wmb(); /* write descriptors before telling HW */
+ t4_write_reg(adap, MYPF_REG(SGE_PF_KDOORBELL),
+ QID(q->cntxt_id) | PIDX(n));
+}
+
+/**
+ * inline_tx_skb - inline a packet's data into Tx descriptors
+ * @skb: the packet
+ * @q: the Tx queue where the packet will be inlined
+ * @pos: starting position in the Tx queue where to inline the packet
+ *
+ * Inline a packet's contents directly into Tx descriptors, starting at
+ * the given position within the Tx DMA ring.
+ * Most of the complexity of this operation is dealing with wrap arounds
+ * in the middle of the packet we want to inline.
+ */
+static void inline_tx_skb(const struct sk_buff *skb, const struct sge_txq *q,
+ void *pos)
+{
+ u64 *p;
+ int left = (void *)q->stat - pos;
+
+ if (likely(skb->len <= left)) {
+ if (likely(!skb->data_len))
+ skb_copy_from_linear_data(skb, pos, skb->len);
+ else
+ skb_copy_bits(skb, 0, pos, skb->len);
+ pos += skb->len;
+ } else {
+ skb_copy_bits(skb, 0, pos, left);
+ skb_copy_bits(skb, left, q->desc, skb->len - left);
+ pos = (void *)q->desc + (skb->len - left);
+ }
+
+ /* 0-pad to multiple of 16 */
+ p = PTR_ALIGN(pos, 8);
+ if ((uintptr_t)p & 8)
+ *p = 0;
+}
+
+/*
+ * Figure out what HW csum a packet wants and return the appropriate control
+ * bits.
+ */
+static u64 hwcsum(const struct sk_buff *skb)
+{
+ int csum_type;
+ const struct iphdr *iph = ip_hdr(skb);
+
+ if (iph->version == 4) {
+ if (iph->protocol == IPPROTO_TCP)
+ csum_type = TX_CSUM_TCPIP;
+ else if (iph->protocol == IPPROTO_UDP)
+ csum_type = TX_CSUM_UDPIP;
+ else {
+nocsum: /*
+ * unknown protocol, disable HW csum
+ * and hope a bad packet is detected
+ */
+ return TXPKT_L4CSUM_DIS;
+ }
+ } else {
+ /*
+ * this doesn't work with extension headers
+ */
+ const struct ipv6hdr *ip6h = (const struct ipv6hdr *)iph;
+
+ if (ip6h->nexthdr == IPPROTO_TCP)
+ csum_type = TX_CSUM_TCPIP6;
+ else if (ip6h->nexthdr == IPPROTO_UDP)
+ csum_type = TX_CSUM_UDPIP6;
+ else
+ goto nocsum;
+ }
+
+ if (likely(csum_type >= TX_CSUM_TCPIP))
+ return TXPKT_CSUM_TYPE(csum_type) |
+ TXPKT_IPHDR_LEN(skb_network_header_len(skb)) |
+ TXPKT_ETHHDR_LEN(skb_network_offset(skb) - ETH_HLEN);
+ else {
+ int start = skb_transport_offset(skb);
+
+ return TXPKT_CSUM_TYPE(csum_type) | TXPKT_CSUM_START(start) |
+ TXPKT_CSUM_LOC(start + skb->csum_offset);
+ }
+}
+
+static void eth_txq_stop(struct sge_eth_txq *q)
+{
+ netif_tx_stop_queue(q->txq);
+ q->q.stops++;
+}
+
+static inline void txq_advance(struct sge_txq *q, unsigned int n)
+{
+ q->in_use += n;
+ q->pidx += n;
+ if (q->pidx >= q->size)
+ q->pidx -= q->size;
+}
+
+/**
+ * t4_eth_xmit - add a packet to an Ethernet Tx queue
+ * @skb: the packet
+ * @dev: the egress net device
+ *
+ * Add a packet to an SGE Ethernet Tx queue. Runs with softirqs disabled.
+ */
+netdev_tx_t t4_eth_xmit(struct sk_buff *skb, struct net_device *dev)
+{
+ u32 wr_mid;
+ u64 cntrl, *end;
+ int qidx, credits;
+ unsigned int flits, ndesc;
+ struct adapter *adap;
+ struct sge_eth_txq *q;
+ const struct port_info *pi;
+ struct fw_eth_tx_pkt_wr *wr;
+ struct cpl_tx_pkt_core *cpl;
+ const struct skb_shared_info *ssi;
+ dma_addr_t addr[MAX_SKB_FRAGS + 1];
+
+ /*
+ * The chip min packet length is 10 octets but play safe and reject
+ * anything shorter than an Ethernet header.
+ */
+ if (unlikely(skb->len < ETH_HLEN)) {
+out_free: dev_kfree_skb(skb);
+ return NETDEV_TX_OK;
+ }
+
+ pi = netdev_priv(dev);
+ adap = pi->adapter;
+ qidx = skb_get_queue_mapping(skb);
+ q = &adap->sge.ethtxq[qidx + pi->first_qset];
+
+ reclaim_completed_tx(adap, &q->q, true);
+
+ flits = calc_tx_flits(skb);
+ ndesc = flits_to_desc(flits);
+ credits = txq_avail(&q->q) - ndesc;
+
+ if (unlikely(credits < 0)) {
+ eth_txq_stop(q);
+ dev_err(adap->pdev_dev,
+ "%s: Tx ring %u full while queue awake!\n",
+ dev->name, qidx);
+ return NETDEV_TX_BUSY;
+ }
+
+ if (!is_eth_imm(skb) &&
+ unlikely(map_skb(adap->pdev_dev, skb, addr) < 0)) {
+ q->mapping_err++;
+ goto out_free;
+ }
+
+ wr_mid = FW_WR_LEN16(DIV_ROUND_UP(flits, 2));
+ if (unlikely(credits < ETHTXQ_STOP_THRES)) {
+ eth_txq_stop(q);
+ wr_mid |= FW_WR_EQUEQ | FW_WR_EQUIQ;
+ }
+
+ wr = (void *)&q->q.desc[q->q.pidx];
+ wr->equiq_to_len16 = htonl(wr_mid);
+ wr->r3 = cpu_to_be64(0);
+ end = (u64 *)wr + flits;
+
+ ssi = skb_shinfo(skb);
+ if (ssi->gso_size) {
+ struct cpl_tx_pkt_lso *lso = (void *)wr;
+ bool v6 = (ssi->gso_type & SKB_GSO_TCPV6) != 0;
+ int l3hdr_len = skb_network_header_len(skb);
+ int eth_xtra_len = skb_network_offset(skb) - ETH_HLEN;
+
+ wr->op_immdlen = htonl(FW_WR_OP(FW_ETH_TX_PKT_WR) |
+ FW_WR_IMMDLEN(sizeof(*lso)));
+ lso->lso_ctrl = htonl(LSO_OPCODE(CPL_TX_PKT_LSO) |
+ LSO_FIRST_SLICE | LSO_LAST_SLICE |
+ LSO_IPV6(v6) |
+ LSO_ETHHDR_LEN(eth_xtra_len / 4) |
+ LSO_IPHDR_LEN(l3hdr_len / 4) |
+ LSO_TCPHDR_LEN(tcp_hdr(skb)->doff));
+ lso->ipid_ofst = htons(0);
+ lso->mss = htons(ssi->gso_size);
+ lso->seqno_offset = htonl(0);
+ lso->len = htonl(skb->len);
+ cpl = (void *)(lso + 1);
+ cntrl = TXPKT_CSUM_TYPE(v6 ? TX_CSUM_TCPIP6 : TX_CSUM_TCPIP) |
+ TXPKT_IPHDR_LEN(l3hdr_len) |
+ TXPKT_ETHHDR_LEN(eth_xtra_len);
+ q->tso++;
+ q->tx_cso += ssi->gso_segs;
+ } else {
+ int len;
+
+ len = is_eth_imm(skb) ? skb->len + sizeof(*cpl) : sizeof(*cpl);
+ wr->op_immdlen = htonl(FW_WR_OP(FW_ETH_TX_PKT_WR) |
+ FW_WR_IMMDLEN(len));
+ cpl = (void *)(wr + 1);
+ if (skb->ip_summed == CHECKSUM_PARTIAL) {
+ cntrl = hwcsum(skb) | TXPKT_IPCSUM_DIS;
+ q->tx_cso++;
+ } else
+ cntrl = TXPKT_L4CSUM_DIS | TXPKT_IPCSUM_DIS;
+ }
+
+ if (vlan_tx_tag_present(skb)) {
+ q->vlan_ins++;
+ cntrl |= TXPKT_VLAN_VLD | TXPKT_VLAN(vlan_tx_tag_get(skb));
+ }
+
+ cpl->ctrl0 = htonl(TXPKT_OPCODE(CPL_TX_PKT_XT) |
+ TXPKT_INTF(pi->tx_chan) | TXPKT_PF(0));
+ cpl->pack = htons(0);
+ cpl->len = htons(skb->len);
+ cpl->ctrl1 = cpu_to_be64(cntrl);
+
+ if (is_eth_imm(skb)) {
+ inline_tx_skb(skb, &q->q, cpl + 1);
+ dev_kfree_skb(skb);
+ } else {
+ int last_desc;
+
+ write_sgl(skb, &q->q, (struct ulptx_sgl *)(cpl + 1), end, 0,
+ addr);
+ skb_orphan(skb);
+
+ last_desc = q->q.pidx + ndesc - 1;
+ if (last_desc >= q->q.size)
+ last_desc -= q->q.size;
+ q->q.sdesc[last_desc].skb = skb;
+ q->q.sdesc[last_desc].sgl = (struct ulptx_sgl *)(cpl + 1);
+ }
+
+ txq_advance(&q->q, ndesc);
+
+ ring_tx_db(adap, &q->q, ndesc);
+ return NETDEV_TX_OK;
+}
+
+/**
+ * reclaim_completed_tx_imm - reclaim completed control-queue Tx descs
+ * @q: the SGE control Tx queue
+ *
+ * This is a variant of reclaim_completed_tx() that is used for Tx queues
+ * that send only immediate data (presently just the control queues) and
+ * thus do not have any sk_buffs to release.
+ */
+static inline void reclaim_completed_tx_imm(struct sge_txq *q)
+{
+ int hw_cidx = ntohs(q->stat->cidx);
+ int reclaim = hw_cidx - q->cidx;
+
+ if (reclaim < 0)
+ reclaim += q->size;
+
+ q->in_use -= reclaim;
+ q->cidx = hw_cidx;
+}
+
+/**
+ * is_imm - check whether a packet can be sent as immediate data
+ * @skb: the packet
+ *
+ * Returns true if a packet can be sent as a WR with immediate data.
+ */
+static inline int is_imm(const struct sk_buff *skb)
+{
+ return skb->len <= MAX_CTRL_WR_LEN;
+}
+
+/**
+ * ctrlq_check_stop - check if a control queue is full and should stop
+ * @q: the queue
+ * @wr: most recent WR written to the queue
+ *
+ * Check if a control queue has become full and should be stopped.
+ * We clean up control queue descriptors very lazily, only when we are out.
+ * If the queue is still full after reclaiming any completed descriptors
+ * we suspend it and have the last WR wake it up.
+ */
+static void ctrlq_check_stop(struct sge_ctrl_txq *q, struct fw_wr_hdr *wr)
+{
+ reclaim_completed_tx_imm(&q->q);
+ if (unlikely(txq_avail(&q->q) < TXQ_STOP_THRES)) {
+ wr->lo |= htonl(FW_WR_EQUEQ | FW_WR_EQUIQ);
+ q->q.stops++;
+ q->full = 1;
+ }
+}
+
+/**
+ * ctrl_xmit - send a packet through an SGE control Tx queue
+ * @q: the control queue
+ * @skb: the packet
+ *
+ * Send a packet through an SGE control Tx queue. Packets sent through
+ * a control queue must fit entirely as immediate data.
+ */
+static int ctrl_xmit(struct sge_ctrl_txq *q, struct sk_buff *skb)
+{
+ unsigned int ndesc;
+ struct fw_wr_hdr *wr;
+
+ if (unlikely(!is_imm(skb))) {
+ WARN_ON(1);
+ dev_kfree_skb(skb);
+ return NET_XMIT_DROP;
+ }
+
+ ndesc = DIV_ROUND_UP(skb->len, sizeof(struct tx_desc));
+ spin_lock(&q->sendq.lock);
+
+ if (unlikely(q->full)) {
+ skb->priority = ndesc; /* save for restart */
+ __skb_queue_tail(&q->sendq, skb);
+ spin_unlock(&q->sendq.lock);
+ return NET_XMIT_CN;
+ }
+
+ wr = (struct fw_wr_hdr *)&q->q.desc[q->q.pidx];
+ inline_tx_skb(skb, &q->q, wr);
+
+ txq_advance(&q->q, ndesc);
+ if (unlikely(txq_avail(&q->q) < TXQ_STOP_THRES))
+ ctrlq_check_stop(q, wr);
+
+ ring_tx_db(q->adap, &q->q, ndesc);
+ spin_unlock(&q->sendq.lock);
+
+ kfree_skb(skb);
+ return NET_XMIT_SUCCESS;
+}
+
+/**
+ * restart_ctrlq - restart a suspended control queue
+ * @data: the control queue to restart
+ *
+ * Resumes transmission on a suspended Tx control queue.
+ */
+static void restart_ctrlq(unsigned long data)
+{
+ struct sk_buff *skb;
+ unsigned int written = 0;
+ struct sge_ctrl_txq *q = (struct sge_ctrl_txq *)data;
+
+ spin_lock(&q->sendq.lock);
+ reclaim_completed_tx_imm(&q->q);
+ BUG_ON(txq_avail(&q->q) < TXQ_STOP_THRES); /* q should be empty */
+
+ while ((skb = __skb_dequeue(&q->sendq)) != NULL) {
+ struct fw_wr_hdr *wr;
+ unsigned int ndesc = skb->priority; /* previously saved */
+
+ /*
+ * Write descriptors and free skbs outside the lock to limit
+ * wait times. q->full is still set so new skbs will be queued.
+ */
+ spin_unlock(&q->sendq.lock);
+
+ wr = (struct fw_wr_hdr *)&q->q.desc[q->q.pidx];
+ inline_tx_skb(skb, &q->q, wr);
+ kfree_skb(skb);
+
+ written += ndesc;
+ txq_advance(&q->q, ndesc);
+ if (unlikely(txq_avail(&q->q) < TXQ_STOP_THRES)) {
+ unsigned long old = q->q.stops;
+
+ ctrlq_check_stop(q, wr);
+ if (q->q.stops != old) { /* suspended anew */
+ spin_lock(&q->sendq.lock);
+ goto ringdb;
+ }
+ }
+ if (written > 16) {
+ ring_tx_db(q->adap, &q->q, written);
+ written = 0;
+ }
+ spin_lock(&q->sendq.lock);
+ }
+ q->full = 0;
+ringdb: if (written)
+ ring_tx_db(q->adap, &q->q, written);
+ spin_unlock(&q->sendq.lock);
+}
+
+/**
+ * t4_mgmt_tx - send a management message
+ * @adap: the adapter
+ * @skb: the packet containing the management message
+ *
+ * Send a management message through control queue 0.
+ */
+int t4_mgmt_tx(struct adapter *adap, struct sk_buff *skb)
+{
+ int ret;
+
+ local_bh_disable();
+ ret = ctrl_xmit(&adap->sge.ctrlq[0], skb);
+ local_bh_enable();
+ return ret;
+}
+
+/**
+ * is_ofld_imm - check whether a packet can be sent as immediate data
+ * @skb: the packet
+ *
+ * Returns true if a packet can be sent as an offload WR with immediate
+ * data. We currently use the same limit as for Ethernet packets.
+ */
+static inline int is_ofld_imm(const struct sk_buff *skb)
+{
+ return skb->len <= MAX_IMM_TX_PKT_LEN;
+}
+
+/**
+ * calc_tx_flits_ofld - calculate # of flits for an offload packet
+ * @skb: the packet
+ *
+ * Returns the number of flits needed for the given offload packet.
+ * These packets are already fully constructed and no additional headers
+ * will be added.
+ */
+static inline unsigned int calc_tx_flits_ofld(const struct sk_buff *skb)
+{
+ unsigned int flits, cnt;
+
+ if (is_ofld_imm(skb))
+ return DIV_ROUND_UP(skb->len, 8);
+
+ flits = skb_transport_offset(skb) / 8U; /* headers */
+ cnt = skb_shinfo(skb)->nr_frags;
+ if (skb->tail != skb->transport_header)
+ cnt++;
+ return flits + sgl_len(cnt);
+}
+
+/**
+ * txq_stop_maperr - stop a Tx queue due to I/O MMU exhaustion
+ * @adap: the adapter
+ * @q: the queue to stop
+ *
+ * Mark a Tx queue stopped due to I/O MMU exhaustion and resulting
+ * inability to map packets. A periodic timer attempts to restart
+ * queues so marked.
+ */
+static void txq_stop_maperr(struct sge_ofld_txq *q)
+{
+ q->mapping_err++;
+ q->q.stops++;
+ set_bit(q->q.cntxt_id, q->adap->sge.txq_maperr);
+}
+
+/**
+ * ofldtxq_stop - stop an offload Tx queue that has become full
+ * @q: the queue to stop
+ * @skb: the packet causing the queue to become full
+ *
+ * Stops an offload Tx queue that has become full and modifies the packet
+ * being written to request a wakeup.
+ */
+static void ofldtxq_stop(struct sge_ofld_txq *q, struct sk_buff *skb)
+{
+ struct fw_wr_hdr *wr = (struct fw_wr_hdr *)skb->data;
+
+ wr->lo |= htonl(FW_WR_EQUEQ | FW_WR_EQUIQ);
+ q->q.stops++;
+ q->full = 1;
+}
+
+/**
+ * service_ofldq - restart a suspended offload queue
+ * @q: the offload queue
+ *
+ * Services an offload Tx queue by moving packets from its packet queue
+ * to the HW Tx ring. The function starts and ends with the queue locked.
+ */
+static void service_ofldq(struct sge_ofld_txq *q)
+{
+ u64 *pos;
+ int credits;
+ struct sk_buff *skb;
+ unsigned int written = 0;
+ unsigned int flits, ndesc;
+
+ while ((skb = skb_peek(&q->sendq)) != NULL && !q->full) {
+ /*
+ * We drop the lock but leave skb on sendq, thus retaining
+ * exclusive access to the state of the queue.
+ */
+ spin_unlock(&q->sendq.lock);
+
+ reclaim_completed_tx(q->adap, &q->q, false);
+
+ flits = skb->priority; /* previously saved */
+ ndesc = flits_to_desc(flits);
+ credits = txq_avail(&q->q) - ndesc;
+ BUG_ON(credits < 0);
+ if (unlikely(credits < TXQ_STOP_THRES))
+ ofldtxq_stop(q, skb);
+
+ pos = (u64 *)&q->q.desc[q->q.pidx];
+ if (is_ofld_imm(skb))
+ inline_tx_skb(skb, &q->q, pos);
+ else if (map_skb(q->adap->pdev_dev, skb,
+ (dma_addr_t *)skb->head)) {
+ txq_stop_maperr(q);
+ spin_lock(&q->sendq.lock);
+ break;
+ } else {
+ int last_desc, hdr_len = skb_transport_offset(skb);
+
+ memcpy(pos, skb->data, hdr_len);
+ write_sgl(skb, &q->q, (void *)pos + hdr_len,
+ pos + flits, hdr_len,
+ (dma_addr_t *)skb->head);
+#ifdef CONFIG_NEED_DMA_MAP_STATE
+ skb->dev = q->adap->port[0];
+ skb->destructor = deferred_unmap_destructor;
+#endif
+ last_desc = q->q.pidx + ndesc - 1;
+ if (last_desc >= q->q.size)
+ last_desc -= q->q.size;
+ q->q.sdesc[last_desc].skb = skb;
+ }
+
+ txq_advance(&q->q, ndesc);
+ written += ndesc;
+ if (unlikely(written > 32)) {
+ ring_tx_db(q->adap, &q->q, written);
+ written = 0;
+ }
+
+ spin_lock(&q->sendq.lock);
+ __skb_unlink(skb, &q->sendq);
+ if (is_ofld_imm(skb))
+ kfree_skb(skb);
+ }
+ if (likely(written))
+ ring_tx_db(q->adap, &q->q, written);
+}
+
+/**
+ * ofld_xmit - send a packet through an offload queue
+ * @q: the Tx offload queue
+ * @skb: the packet
+ *
+ * Send an offload packet through an SGE offload queue.
+ */
+static int ofld_xmit(struct sge_ofld_txq *q, struct sk_buff *skb)
+{
+ skb->priority = calc_tx_flits_ofld(skb); /* save for restart */
+ spin_lock(&q->sendq.lock);
+ __skb_queue_tail(&q->sendq, skb);
+ if (q->sendq.qlen == 1)
+ service_ofldq(q);
+ spin_unlock(&q->sendq.lock);
+ return NET_XMIT_SUCCESS;
+}
+
+/**
+ * restart_ofldq - restart a suspended offload queue
+ * @data: the offload queue to restart
+ *
+ * Resumes transmission on a suspended Tx offload queue.
+ */
+static void restart_ofldq(unsigned long data)
+{
+ struct sge_ofld_txq *q = (struct sge_ofld_txq *)data;
+
+ spin_lock(&q->sendq.lock);
+ q->full = 0; /* the queue actually is completely empty now */
+ service_ofldq(q);
+ spin_unlock(&q->sendq.lock);
+}
+
+/**
+ * skb_txq - return the Tx queue an offload packet should use
+ * @skb: the packet
+ *
+ * Returns the Tx queue an offload packet should use as indicated by bits
+ * 1-15 in the packet's queue_mapping.
+ */
+static inline unsigned int skb_txq(const struct sk_buff *skb)
+{
+ return skb->queue_mapping >> 1;
+}
+
+/**
+ * is_ctrl_pkt - return whether an offload packet is a control packet
+ * @skb: the packet
+ *
+ * Returns whether an offload packet should use an OFLD or a CTRL
+ * Tx queue as indicated by bit 0 in the packet's queue_mapping.
+ */
+static inline unsigned int is_ctrl_pkt(const struct sk_buff *skb)
+{
+ return skb->queue_mapping & 1;
+}
+
+static inline int ofld_send(struct adapter *adap, struct sk_buff *skb)
+{
+ unsigned int idx = skb_txq(skb);
+
+ if (unlikely(is_ctrl_pkt(skb)))
+ return ctrl_xmit(&adap->sge.ctrlq[idx], skb);
+ return ofld_xmit(&adap->sge.ofldtxq[idx], skb);
+}
+
+/**
+ * t4_ofld_send - send an offload packet
+ * @adap: the adapter
+ * @skb: the packet
+ *
+ * Sends an offload packet. We use the packet queue_mapping to select the
+ * appropriate Tx queue as follows: bit 0 indicates whether the packet
+ * should be sent as regular or control, bits 1-15 select the queue.
+ */
+int t4_ofld_send(struct adapter *adap, struct sk_buff *skb)
+{
+ int ret;
+
+ local_bh_disable();
+ ret = ofld_send(adap, skb);
+ local_bh_enable();
+ return ret;
+}
+
+/**
+ * cxgb4_ofld_send - send an offload packet
+ * @dev: the net device
+ * @skb: the packet
+ *
+ * Sends an offload packet. This is an exported version of @t4_ofld_send,
+ * intended for ULDs.
+ */
+int cxgb4_ofld_send(struct net_device *dev, struct sk_buff *skb)
+{
+ return t4_ofld_send(netdev2adap(dev), skb);
+}
+EXPORT_SYMBOL(cxgb4_ofld_send);
+
+static inline void copy_frags(struct skb_shared_info *ssi,
+ const struct pkt_gl *gl, unsigned int offset)
+{
+ unsigned int n;
+
+ /* usually there's just one frag */
+ ssi->frags[0].page = gl->frags[0].page;
+ ssi->frags[0].page_offset = gl->frags[0].page_offset + offset;
+ ssi->frags[0].size = gl->frags[0].size - offset;
+ ssi->nr_frags = gl->nfrags;
+ n = gl->nfrags - 1;
+ if (n)
+ memcpy(&ssi->frags[1], &gl->frags[1], n * sizeof(skb_frag_t));
+
+ /* get a reference to the last page, we don't own it */
+ get_page(gl->frags[n].page);
+}
+
+/**
+ * cxgb4_pktgl_to_skb - build an sk_buff from a packet gather list
+ * @gl: the gather list
+ * @skb_len: size of sk_buff main body if it carries fragments
+ * @pull_len: amount of data to move to the sk_buff's main body
+ *
+ * Builds an sk_buff from the given packet gather list. Returns the
+ * sk_buff or %NULL if sk_buff allocation failed.
+ */
+struct sk_buff *cxgb4_pktgl_to_skb(const struct pkt_gl *gl,
+ unsigned int skb_len, unsigned int pull_len)
+{
+ struct sk_buff *skb;
+
+ /*
+ * Below we rely on RX_COPY_THRES being less than the smallest Rx buffer
+ * size, which is expected since buffers are at least PAGE_SIZEd.
+ * In this case packets up to RX_COPY_THRES have only one fragment.
+ */
+ if (gl->tot_len <= RX_COPY_THRES) {
+ skb = dev_alloc_skb(gl->tot_len);
+ if (unlikely(!skb))
+ goto out;
+ __skb_put(skb, gl->tot_len);
+ skb_copy_to_linear_data(skb, gl->va, gl->tot_len);
+ } else {
+ skb = dev_alloc_skb(skb_len);
+ if (unlikely(!skb))
+ goto out;
+ __skb_put(skb, pull_len);
+ skb_copy_to_linear_data(skb, gl->va, pull_len);
+
+ copy_frags(skb_shinfo(skb), gl, pull_len);
+ skb->len = gl->tot_len;
+ skb->data_len = skb->len - pull_len;
+ skb->truesize += skb->data_len;
+ }
+out: return skb;
+}
+EXPORT_SYMBOL(cxgb4_pktgl_to_skb);
+
+/**
+ * t4_pktgl_free - free a packet gather list
+ * @gl: the gather list
+ *
+ * Releases the pages of a packet gather list. We do not own the last
+ * page on the list and do not free it.
+ */
+void t4_pktgl_free(const struct pkt_gl *gl)
+{
+ int n;
+ const skb_frag_t *p;
+
+ for (p = gl->frags, n = gl->nfrags - 1; n--; p++)
+ put_page(p->page);
+}
+
+/*
+ * Process an MPS trace packet. Give it an unused protocol number so it won't
+ * be delivered to anyone and send it to the stack for capture.
+ */
+static noinline int handle_trace_pkt(struct adapter *adap,
+ const struct pkt_gl *gl)
+{
+ struct sk_buff *skb;
+ struct cpl_trace_pkt *p;
+
+ skb = cxgb4_pktgl_to_skb(gl, RX_PULL_LEN, RX_PULL_LEN);
+ if (unlikely(!skb)) {
+ t4_pktgl_free(gl);
+ return 0;
+ }
+
+ p = (struct cpl_trace_pkt *)skb->data;
+ __skb_pull(skb, sizeof(*p));
+ skb_reset_mac_header(skb);
+ skb->protocol = htons(0xffff);
+ skb->dev = adap->port[0];
+ netif_receive_skb(skb);
+ return 0;
+}
+
+static void do_gro(struct sge_eth_rxq *rxq, const struct pkt_gl *gl,
+ const struct cpl_rx_pkt *pkt)
+{
+ int ret;
+ struct sk_buff *skb;
+
+ skb = napi_get_frags(&rxq->rspq.napi);
+ if (unlikely(!skb)) {
+ t4_pktgl_free(gl);
+ rxq->stats.rx_drops++;
+ return;
+ }
+
+ copy_frags(skb_shinfo(skb), gl, RX_PKT_PAD);
+ skb->len = gl->tot_len - RX_PKT_PAD;
+ skb->data_len = skb->len;
+ skb->truesize += skb->data_len;
+ skb->ip_summed = CHECKSUM_UNNECESSARY;
+ skb_record_rx_queue(skb, rxq->rspq.idx);
+
+ if (unlikely(pkt->vlan_ex)) {
+ struct port_info *pi = netdev_priv(rxq->rspq.netdev);
+ struct vlan_group *grp = pi->vlan_grp;
+
+ rxq->stats.vlan_ex++;
+ if (likely(grp)) {
+ ret = vlan_gro_frags(&rxq->rspq.napi, grp,
+ ntohs(pkt->vlan));
+ goto stats;
+ }
+ }
+ ret = napi_gro_frags(&rxq->rspq.napi);
+stats: if (ret == GRO_HELD)
+ rxq->stats.lro_pkts++;
+ else if (ret == GRO_MERGED || ret == GRO_MERGED_FREE)
+ rxq->stats.lro_merged++;
+ rxq->stats.pkts++;
+ rxq->stats.rx_cso++;
+}
+
+/**
+ * t4_ethrx_handler - process an ingress ethernet packet
+ * @q: the response queue that received the packet
+ * @rsp: the response queue descriptor holding the RX_PKT message
+ * @si: the gather list of packet fragments
+ *
+ * Process an ingress ethernet packet and deliver it to the stack.
+ */
+int t4_ethrx_handler(struct sge_rspq *q, const __be64 *rsp,
+ const struct pkt_gl *si)
+{
+ bool csum_ok;
+ struct sk_buff *skb;
+ struct port_info *pi;
+ const struct cpl_rx_pkt *pkt;
+ struct sge_eth_rxq *rxq = container_of(q, struct sge_eth_rxq, rspq);
+
+ if (unlikely(*(u8 *)rsp == CPL_TRACE_PKT))
+ return handle_trace_pkt(q->adap, si);
+
+ pkt = (void *)&rsp[1];
+ csum_ok = pkt->csum_calc && !pkt->err_vec;
+ if ((pkt->l2info & htonl(RXF_TCP)) &&
+ (q->netdev->features & NETIF_F_GRO) && csum_ok && !pkt->ip_frag) {
+ do_gro(rxq, si, pkt);
+ return 0;
+ }
+
+ skb = cxgb4_pktgl_to_skb(si, RX_PKT_SKB_LEN, RX_PULL_LEN);
+ if (unlikely(!skb)) {
+ t4_pktgl_free(si);
+ rxq->stats.rx_drops++;
+ return 0;
+ }
+
+ __skb_pull(skb, RX_PKT_PAD); /* remove ethernet header padding */
+ skb->protocol = eth_type_trans(skb, q->netdev);
+ skb_record_rx_queue(skb, q->idx);
+ pi = netdev_priv(skb->dev);
+ rxq->stats.pkts++;
+
+ if (csum_ok && (pi->rx_offload & RX_CSO) &&
+ (pkt->l2info & htonl(RXF_UDP | RXF_TCP))) {
+ if (!pkt->ip_frag)
+ skb->ip_summed = CHECKSUM_UNNECESSARY;
+ else {
+ __sum16 c = (__force __sum16)pkt->csum;
+ skb->csum = csum_unfold(c);
+ skb->ip_summed = CHECKSUM_COMPLETE;
+ }
+ rxq->stats.rx_cso++;
+ } else
+ skb->ip_summed = CHECKSUM_NONE;
+
+ if (unlikely(pkt->vlan_ex)) {
+ struct vlan_group *grp = pi->vlan_grp;
+
+ rxq->stats.vlan_ex++;
+ if (likely(grp))
+ vlan_hwaccel_receive_skb(skb, grp, ntohs(pkt->vlan));
+ else
+ dev_kfree_skb_any(skb);
+ } else
+ netif_receive_skb(skb);
+
+ return 0;
+}
+
+/**
+ * restore_rx_bufs - put back a packet's Rx buffers
+ * @si: the packet gather list
+ * @q: the SGE free list
+ * @frags: number of FL buffers to restore
+ *
+ * Puts back on an FL the Rx buffers associated with @si. The buffers
+ * have already been unmapped and are left unmapped, we mark them so to
+ * prevent further unmapping attempts.
+ *
+ * This function undoes a series of @unmap_rx_buf calls when we find out
+ * that the current packet can't be processed right away afterall and we
+ * need to come back to it later. This is a very rare event and there's
+ * no effort to make this particularly efficient.
+ */
+static void restore_rx_bufs(const struct pkt_gl *si, struct sge_fl *q,
+ int frags)
+{
+ struct rx_sw_desc *d;
+
+ while (frags--) {
+ if (q->cidx == 0)
+ q->cidx = q->size - 1;
+ else
+ q->cidx--;
+ d = &q->sdesc[q->cidx];
+ d->page = si->frags[frags].page;
+ d->dma_addr |= RX_UNMAPPED_BUF;
+ q->avail++;
+ }
+}
+
+/**
+ * is_new_response - check if a response is newly written
+ * @r: the response descriptor
+ * @q: the response queue
+ *
+ * Returns true if a response descriptor contains a yet unprocessed
+ * response.
+ */
+static inline bool is_new_response(const struct rsp_ctrl *r,
+ const struct sge_rspq *q)
+{
+ return RSPD_GEN(r->type_gen) == q->gen;
+}
+
+/**
+ * rspq_next - advance to the next entry in a response queue
+ * @q: the queue
+ *
+ * Updates the state of a response queue to advance it to the next entry.
+ */
+static inline void rspq_next(struct sge_rspq *q)
+{
+ q->cur_desc = (void *)q->cur_desc + q->iqe_len;
+ if (unlikely(++q->cidx == q->size)) {
+ q->cidx = 0;
+ q->gen ^= 1;
+ q->cur_desc = q->desc;
+ }
+}
+
+/**
+ * process_responses - process responses from an SGE response queue
+ * @q: the ingress queue to process
+ * @budget: how many responses can be processed in this round
+ *
+ * Process responses from an SGE response queue up to the supplied budget.
+ * Responses include received packets as well as control messages from FW
+ * or HW.
+ *
+ * Additionally choose the interrupt holdoff time for the next interrupt
+ * on this queue. If the system is under memory shortage use a fairly
+ * long delay to help recovery.
+ */
+static int process_responses(struct sge_rspq *q, int budget)
+{
+ int ret, rsp_type;
+ int budget_left = budget;
+ const struct rsp_ctrl *rc;
+ struct sge_eth_rxq *rxq = container_of(q, struct sge_eth_rxq, rspq);
+
+ while (likely(budget_left)) {
+ rc = (void *)q->cur_desc + (q->iqe_len - sizeof(*rc));
+ if (!is_new_response(rc, q))
+ break;
+
+ rmb();
+ rsp_type = RSPD_TYPE(rc->type_gen);
+ if (likely(rsp_type == RSP_TYPE_FLBUF)) {
+ skb_frag_t *fp;
+ struct pkt_gl si;
+ const struct rx_sw_desc *rsd;
+ u32 len = ntohl(rc->pldbuflen_qid), bufsz, frags;
+
+ if (len & RSPD_NEWBUF) {
+ if (likely(q->offset > 0)) {
+ free_rx_bufs(q->adap, &rxq->fl, 1);
+ q->offset = 0;
+ }
+ len &= RSPD_LEN;
+ }
+ si.tot_len = len;
+
+ /* gather packet fragments */
+ for (frags = 0, fp = si.frags; ; frags++, fp++) {
+ rsd = &rxq->fl.sdesc[rxq->fl.cidx];
+ bufsz = get_buf_size(rsd);
+ fp->page = rsd->page;
+ fp->page_offset = q->offset;
+ fp->size = min(bufsz, len);
+ len -= fp->size;
+ if (!len)
+ break;
+ unmap_rx_buf(q->adap, &rxq->fl);
+ }
+
+ /*
+ * Last buffer remains mapped so explicitly make it
+ * coherent for CPU access.
+ */
+ dma_sync_single_for_cpu(q->adap->pdev_dev,
+ get_buf_addr(rsd),
+ fp->size, DMA_FROM_DEVICE);
+
+ si.va = page_address(si.frags[0].page) +
+ si.frags[0].page_offset;
+ prefetch(si.va);
+
+ si.nfrags = frags + 1;
+ ret = q->handler(q, q->cur_desc, &si);
+ if (likely(ret == 0))
+ q->offset += ALIGN(fp->size, FL_ALIGN);
+ else
+ restore_rx_bufs(&si, &rxq->fl, frags);
+ } else if (likely(rsp_type == RSP_TYPE_CPL)) {
+ ret = q->handler(q, q->cur_desc, NULL);
+ } else {
+ ret = q->handler(q, (const __be64 *)rc, CXGB4_MSG_AN);
+ }
+
+ if (unlikely(ret)) {
+ /* couldn't process descriptor, back off for recovery */
+ q->next_intr_params = QINTR_TIMER_IDX(NOMEM_TMR_IDX);
+ break;
+ }
+
+ rspq_next(q);
+ budget_left--;
+ }
+
+ if (q->offset >= 0 && rxq->fl.size - rxq->fl.avail >= 16)
+ __refill_fl(q->adap, &rxq->fl);
+ return budget - budget_left;
+}
+
+/**
+ * napi_rx_handler - the NAPI handler for Rx processing
+ * @napi: the napi instance
+ * @budget: how many packets we can process in this round
+ *
+ * Handler for new data events when using NAPI. This does not need any
+ * locking or protection from interrupts as data interrupts are off at
+ * this point and other adapter interrupts do not interfere (the latter
+ * in not a concern at all with MSI-X as non-data interrupts then have
+ * a separate handler).
+ */
+static int napi_rx_handler(struct napi_struct *napi, int budget)
+{
+ unsigned int params;
+ struct sge_rspq *q = container_of(napi, struct sge_rspq, napi);
+ int work_done = process_responses(q, budget);
+
+ if (likely(work_done < budget)) {
+ napi_complete(napi);
+ params = q->next_intr_params;
+ q->next_intr_params = q->intr_params;
+ } else
+ params = QINTR_TIMER_IDX(7);
+
+ t4_write_reg(q->adap, MYPF_REG(SGE_PF_GTS), CIDXINC(work_done) |
+ INGRESSQID((u32)q->cntxt_id) | SEINTARM(params));
+ return work_done;
+}
+
+/*
+ * The MSI-X interrupt handler for an SGE response queue.
+ */
+irqreturn_t t4_sge_intr_msix(int irq, void *cookie)
+{
+ struct sge_rspq *q = cookie;
+
+ napi_schedule(&q->napi);
+ return IRQ_HANDLED;
+}
+
+/*
+ * Process the indirect interrupt entries in the interrupt queue and kick off
+ * NAPI for each queue that has generated an entry.
+ */
+static unsigned int process_intrq(struct adapter *adap)
+{
+ unsigned int credits;
+ const struct rsp_ctrl *rc;
+ struct sge_rspq *q = &adap->sge.intrq;
+
+ spin_lock(&adap->sge.intrq_lock);
+ for (credits = 0; ; credits++) {
+ rc = (void *)q->cur_desc + (q->iqe_len - sizeof(*rc));
+ if (!is_new_response(rc, q))
+ break;
+
+ rmb();
+ if (RSPD_TYPE(rc->type_gen) == RSP_TYPE_INTR) {
+ unsigned int qid = ntohl(rc->pldbuflen_qid);
+
+ napi_schedule(&adap->sge.ingr_map[qid]->napi);
+ }
+
+ rspq_next(q);
+ }
+
+ t4_write_reg(adap, MYPF_REG(SGE_PF_GTS), CIDXINC(credits) |
+ INGRESSQID(q->cntxt_id) | SEINTARM(q->intr_params));
+ spin_unlock(&adap->sge.intrq_lock);
+ return credits;
+}
+
+/*
+ * The MSI interrupt handler, which handles data events from SGE response queues
+ * as well as error and other async events as they all use the same MSI vector.
+ */
+static irqreturn_t t4_intr_msi(int irq, void *cookie)
+{
+ struct adapter *adap = cookie;
+
+ t4_slow_intr_handler(adap);
+ process_intrq(adap);
+ return IRQ_HANDLED;
+}
+
+/*
+ * Interrupt handler for legacy INTx interrupts.
+ * Handles data events from SGE response queues as well as error and other
+ * async events as they all use the same interrupt line.
+ */
+static irqreturn_t t4_intr_intx(int irq, void *cookie)
+{
+ struct adapter *adap = cookie;
+
+ t4_write_reg(adap, MYPF_REG(PCIE_PF_CLI), 0);
+ if (t4_slow_intr_handler(adap) | process_intrq(adap))
+ return IRQ_HANDLED;
+ return IRQ_NONE; /* probably shared interrupt */
+}
+
+/**
+ * t4_intr_handler - select the top-level interrupt handler
+ * @adap: the adapter
+ *
+ * Selects the top-level interrupt handler based on the type of interrupts
+ * (MSI-X, MSI, or INTx).
+ */
+irq_handler_t t4_intr_handler(struct adapter *adap)
+{
+ if (adap->flags & USING_MSIX)
+ return t4_sge_intr_msix;
+ if (adap->flags & USING_MSI)
+ return t4_intr_msi;
+ return t4_intr_intx;
+}
+
+static void sge_rx_timer_cb(unsigned long data)
+{
+ unsigned long m;
+ unsigned int i, cnt[2];
+ struct adapter *adap = (struct adapter *)data;
+ struct sge *s = &adap->sge;
+
+ for (i = 0; i < ARRAY_SIZE(s->starving_fl); i++)
+ for (m = s->starving_fl[i]; m; m &= m - 1) {
+ struct sge_eth_rxq *rxq;
+ unsigned int id = __ffs(m) + i * BITS_PER_LONG;
+ struct sge_fl *fl = s->egr_map[id];
+
+ clear_bit(id, s->starving_fl);
+ smp_mb__after_clear_bit();
+
+ if (fl_starving(fl)) {
+ rxq = container_of(fl, struct sge_eth_rxq, fl);
+ if (napi_reschedule(&rxq->rspq.napi))
+ fl->starving++;
+ else
+ set_bit(id, s->starving_fl);
+ }
+ }
+
+ t4_write_reg(adap, SGE_DEBUG_INDEX, 13);
+ cnt[0] = t4_read_reg(adap, SGE_DEBUG_DATA_HIGH);
+ cnt[1] = t4_read_reg(adap, SGE_DEBUG_DATA_LOW);
+
+ for (i = 0; i < 2; i++)
+ if (cnt[i] >= s->starve_thres) {
+ if (s->idma_state[i] || cnt[i] == 0xffffffff)
+ continue;
+ s->idma_state[i] = 1;
+ t4_write_reg(adap, SGE_DEBUG_INDEX, 11);
+ m = t4_read_reg(adap, SGE_DEBUG_DATA_LOW) >> (i * 16);
+ dev_warn(adap->pdev_dev,
+ "SGE idma%u starvation detected for "
+ "queue %lu\n", i, m & 0xffff);
+ } else if (s->idma_state[i])
+ s->idma_state[i] = 0;
+
+ mod_timer(&s->rx_timer, jiffies + RX_QCHECK_PERIOD);
+}
+
+static void sge_tx_timer_cb(unsigned long data)
+{
+ unsigned long m;
+ unsigned int i, budget;
+ struct adapter *adap = (struct adapter *)data;
+ struct sge *s = &adap->sge;
+
+ for (i = 0; i < ARRAY_SIZE(s->txq_maperr); i++)
+ for (m = s->txq_maperr[i]; m; m &= m - 1) {
+ unsigned long id = __ffs(m) + i * BITS_PER_LONG;
+ struct sge_ofld_txq *txq = s->egr_map[id];
+
+ clear_bit(id, s->txq_maperr);
+ tasklet_schedule(&txq->qresume_tsk);
+ }
+
+ budget = MAX_TIMER_TX_RECLAIM;
+ i = s->ethtxq_rover;
+ do {
+ struct sge_eth_txq *q = &s->ethtxq[i];
+
+ if (q->q.in_use &&
+ time_after_eq(jiffies, q->txq->trans_start + HZ / 100) &&
+ __netif_tx_trylock(q->txq)) {
+ int avail = reclaimable(&q->q);
+
+ if (avail) {
+ if (avail > budget)
+ avail = budget;
+
+ free_tx_desc(adap, &q->q, avail, true);
+ q->q.in_use -= avail;
+ budget -= avail;
+ }
+ __netif_tx_unlock(q->txq);
+ }
+
+ if (++i >= s->ethqsets)
+ i = 0;
+ } while (budget && i != s->ethtxq_rover);
+ s->ethtxq_rover = i;
+ mod_timer(&s->tx_timer, jiffies + (budget ? TX_QCHECK_PERIOD : 2));
+}
+
+int t4_sge_alloc_rxq(struct adapter *adap, struct sge_rspq *iq, bool fwevtq,
+ struct net_device *dev, int intr_idx,
+ struct sge_fl *fl, rspq_handler_t hnd)
+{
+ int ret, flsz = 0;
+ struct fw_iq_cmd c;
+ struct port_info *pi = netdev_priv(dev);
+
+ /* Size needs to be multiple of 16, including status entry. */
+ iq->size = roundup(iq->size, 16);
+
+ iq->desc = alloc_ring(adap->pdev_dev, iq->size, iq->iqe_len, 0,
+ &iq->phys_addr, NULL, 0);
+ if (!iq->desc)
+ return -ENOMEM;
+
+ memset(&c, 0, sizeof(c));
+ c.op_to_vfn = htonl(FW_CMD_OP(FW_IQ_CMD) | FW_CMD_REQUEST |
+ FW_CMD_WRITE | FW_CMD_EXEC |
+ FW_IQ_CMD_PFN(0) | FW_IQ_CMD_VFN(0));
+ c.alloc_to_len16 = htonl(FW_IQ_CMD_ALLOC | FW_IQ_CMD_IQSTART(1) |
+ FW_LEN16(c));
+ c.type_to_iqandstindex = htonl(FW_IQ_CMD_TYPE(FW_IQ_TYPE_FL_INT_CAP) |
+ FW_IQ_CMD_IQASYNCH(fwevtq) | FW_IQ_CMD_VIID(pi->viid) |
+ FW_IQ_CMD_IQANDST(intr_idx < 0) | FW_IQ_CMD_IQANUD(1) |
+ FW_IQ_CMD_IQANDSTINDEX(intr_idx >= 0 ? intr_idx :
+ -intr_idx - 1));
+ c.iqdroprss_to_iqesize = htons(FW_IQ_CMD_IQPCIECH(pi->tx_chan) |
+ FW_IQ_CMD_IQGTSMODE |
+ FW_IQ_CMD_IQINTCNTTHRESH(iq->pktcnt_idx) |
+ FW_IQ_CMD_IQESIZE(ilog2(iq->iqe_len) - 4));
+ c.iqsize = htons(iq->size);
+ c.iqaddr = cpu_to_be64(iq->phys_addr);
+
+ if (fl) {
+ fl->size = roundup(fl->size, 8);
+ fl->desc = alloc_ring(adap->pdev_dev, fl->size, sizeof(__be64),
+ sizeof(struct rx_sw_desc), &fl->addr,
+ &fl->sdesc, STAT_LEN);
+ if (!fl->desc)
+ goto fl_nomem;
+
+ flsz = fl->size / 8 + STAT_LEN / sizeof(struct tx_desc);
+ c.iqns_to_fl0congen = htonl(FW_IQ_CMD_FL0PACKEN |
+ FW_IQ_CMD_FL0PADEN);
+ c.fl0dcaen_to_fl0cidxfthresh = htons(FW_IQ_CMD_FL0FBMIN(2) |
+ FW_IQ_CMD_FL0FBMAX(3));
+ c.fl0size = htons(flsz);
+ c.fl0addr = cpu_to_be64(fl->addr);
+ }
+
+ ret = t4_wr_mbox(adap, 0, &c, sizeof(c), &c);
+ if (ret)
+ goto err;
+
+ netif_napi_add(dev, &iq->napi, napi_rx_handler, 64);
+ iq->cur_desc = iq->desc;
+ iq->cidx = 0;
+ iq->gen = 1;
+ iq->next_intr_params = iq->intr_params;
+ iq->cntxt_id = ntohs(c.iqid);
+ iq->abs_id = ntohs(c.physiqid);
+ iq->size--; /* subtract status entry */
+ iq->adap = adap;
+ iq->netdev = dev;
+ iq->handler = hnd;
+
+ /* set offset to -1 to distinguish ingress queues without FL */
+ iq->offset = fl ? 0 : -1;
+
+ adap->sge.ingr_map[iq->cntxt_id] = iq;
+
+ if (fl) {
+ fl->cntxt_id = htons(c.fl0id);
+ fl->avail = fl->pend_cred = 0;
+ fl->pidx = fl->cidx = 0;
+ fl->alloc_failed = fl->large_alloc_failed = fl->starving = 0;
+ adap->sge.egr_map[fl->cntxt_id] = fl;
+ refill_fl(adap, fl, fl_cap(fl), GFP_KERNEL);
+ }
+ return 0;
+
+fl_nomem:
+ ret = -ENOMEM;
+err:
+ if (iq->desc) {
+ dma_free_coherent(adap->pdev_dev, iq->size * iq->iqe_len,
+ iq->desc, iq->phys_addr);
+ iq->desc = NULL;
+ }
+ if (fl && fl->desc) {
+ kfree(fl->sdesc);
+ fl->sdesc = NULL;
+ dma_free_coherent(adap->pdev_dev, flsz * sizeof(struct tx_desc),
+ fl->desc, fl->addr);
+ fl->desc = NULL;
+ }
+ return ret;
+}
+
+static void init_txq(struct adapter *adap, struct sge_txq *q, unsigned int id)
+{
+ q->in_use = 0;
+ q->cidx = q->pidx = 0;
+ q->stops = q->restarts = 0;
+ q->stat = (void *)&q->desc[q->size];
+ q->cntxt_id = id;
+ adap->sge.egr_map[id] = q;
+}
+
+int t4_sge_alloc_eth_txq(struct adapter *adap, struct sge_eth_txq *txq,
+ struct net_device *dev, struct netdev_queue *netdevq,
+ unsigned int iqid)
+{
+ int ret, nentries;
+ struct fw_eq_eth_cmd c;
+ struct port_info *pi = netdev_priv(dev);
+
+ /* Add status entries */
+ nentries = txq->q.size + STAT_LEN / sizeof(struct tx_desc);
+
+ txq->q.desc = alloc_ring(adap->pdev_dev, txq->q.size,
+ sizeof(struct tx_desc), sizeof(struct tx_sw_desc),
+ &txq->q.phys_addr, &txq->q.sdesc, STAT_LEN);
+ if (!txq->q.desc)
+ return -ENOMEM;
+
+ memset(&c, 0, sizeof(c));
+ c.op_to_vfn = htonl(FW_CMD_OP(FW_EQ_ETH_CMD) | FW_CMD_REQUEST |
+ FW_CMD_WRITE | FW_CMD_EXEC |
+ FW_EQ_ETH_CMD_PFN(0) | FW_EQ_ETH_CMD_VFN(0));
+ c.alloc_to_len16 = htonl(FW_EQ_ETH_CMD_ALLOC |
+ FW_EQ_ETH_CMD_EQSTART | FW_LEN16(c));
+ c.viid_pkd = htonl(FW_EQ_ETH_CMD_VIID(pi->viid));
+ c.fetchszm_to_iqid = htonl(FW_EQ_ETH_CMD_HOSTFCMODE(2) |
+ FW_EQ_ETH_CMD_PCIECHN(pi->tx_chan) |
+ FW_EQ_ETH_CMD_IQID(iqid));
+ c.dcaen_to_eqsize = htonl(FW_EQ_ETH_CMD_FBMIN(2) |
+ FW_EQ_ETH_CMD_FBMAX(3) |
+ FW_EQ_ETH_CMD_CIDXFTHRESH(5) |
+ FW_EQ_ETH_CMD_EQSIZE(nentries));
+ c.eqaddr = cpu_to_be64(txq->q.phys_addr);
+
+ ret = t4_wr_mbox(adap, 0, &c, sizeof(c), &c);
+ if (ret) {
+ kfree(txq->q.sdesc);
+ txq->q.sdesc = NULL;
+ dma_free_coherent(adap->pdev_dev,
+ nentries * sizeof(struct tx_desc),
+ txq->q.desc, txq->q.phys_addr);
+ txq->q.desc = NULL;
+ return ret;
+ }
+
+ init_txq(adap, &txq->q, FW_EQ_ETH_CMD_EQID_GET(ntohl(c.eqid_pkd)));
+ txq->txq = netdevq;
+ txq->tso = txq->tx_cso = txq->vlan_ins = 0;
+ txq->mapping_err = 0;
+ return 0;
+}
+
+int t4_sge_alloc_ctrl_txq(struct adapter *adap, struct sge_ctrl_txq *txq,
+ struct net_device *dev, unsigned int iqid,
+ unsigned int cmplqid)
+{
+ int ret, nentries;
+ struct fw_eq_ctrl_cmd c;
+ struct port_info *pi = netdev_priv(dev);
+
+ /* Add status entries */
+ nentries = txq->q.size + STAT_LEN / sizeof(struct tx_desc);
+
+ txq->q.desc = alloc_ring(adap->pdev_dev, nentries,
+ sizeof(struct tx_desc), 0, &txq->q.phys_addr,
+ NULL, 0);
+ if (!txq->q.desc)
+ return -ENOMEM;
+
+ c.op_to_vfn = htonl(FW_CMD_OP(FW_EQ_CTRL_CMD) | FW_CMD_REQUEST |
+ FW_CMD_WRITE | FW_CMD_EXEC |
+ FW_EQ_CTRL_CMD_PFN(0) | FW_EQ_CTRL_CMD_VFN(0));
+ c.alloc_to_len16 = htonl(FW_EQ_CTRL_CMD_ALLOC |
+ FW_EQ_CTRL_CMD_EQSTART | FW_LEN16(c));
+ c.cmpliqid_eqid = htonl(FW_EQ_CTRL_CMD_CMPLIQID(cmplqid));
+ c.physeqid_pkd = htonl(0);
+ c.fetchszm_to_iqid = htonl(FW_EQ_CTRL_CMD_HOSTFCMODE(2) |
+ FW_EQ_CTRL_CMD_PCIECHN(pi->tx_chan) |
+ FW_EQ_CTRL_CMD_IQID(iqid));
+ c.dcaen_to_eqsize = htonl(FW_EQ_CTRL_CMD_FBMIN(2) |
+ FW_EQ_CTRL_CMD_FBMAX(3) |
+ FW_EQ_CTRL_CMD_CIDXFTHRESH(5) |
+ FW_EQ_CTRL_CMD_EQSIZE(nentries));
+ c.eqaddr = cpu_to_be64(txq->q.phys_addr);
+
+ ret = t4_wr_mbox(adap, 0, &c, sizeof(c), &c);
+ if (ret) {
+ dma_free_coherent(adap->pdev_dev,
+ nentries * sizeof(struct tx_desc),
+ txq->q.desc, txq->q.phys_addr);
+ txq->q.desc = NULL;
+ return ret;
+ }
+
+ init_txq(adap, &txq->q, FW_EQ_CTRL_CMD_EQID_GET(ntohl(c.cmpliqid_eqid)));
+ txq->adap = adap;
+ skb_queue_head_init(&txq->sendq);
+ tasklet_init(&txq->qresume_tsk, restart_ctrlq, (unsigned long)txq);
+ txq->full = 0;
+ return 0;
+}
+
+int t4_sge_alloc_ofld_txq(struct adapter *adap, struct sge_ofld_txq *txq,
+ struct net_device *dev, unsigned int iqid)
+{
+ int ret, nentries;
+ struct fw_eq_ofld_cmd c;
+ struct port_info *pi = netdev_priv(dev);
+
+ /* Add status entries */
+ nentries = txq->q.size + STAT_LEN / sizeof(struct tx_desc);
+
+ txq->q.desc = alloc_ring(adap->pdev_dev, txq->q.size,
+ sizeof(struct tx_desc), sizeof(struct tx_sw_desc),
+ &txq->q.phys_addr, &txq->q.sdesc, STAT_LEN);
+ if (!txq->q.desc)
+ return -ENOMEM;
+
+ memset(&c, 0, sizeof(c));
+ c.op_to_vfn = htonl(FW_CMD_OP(FW_EQ_OFLD_CMD) | FW_CMD_REQUEST |
+ FW_CMD_WRITE | FW_CMD_EXEC |
+ FW_EQ_OFLD_CMD_PFN(0) | FW_EQ_OFLD_CMD_VFN(0));
+ c.alloc_to_len16 = htonl(FW_EQ_OFLD_CMD_ALLOC |
+ FW_EQ_OFLD_CMD_EQSTART | FW_LEN16(c));
+ c.fetchszm_to_iqid = htonl(FW_EQ_OFLD_CMD_HOSTFCMODE(2) |
+ FW_EQ_OFLD_CMD_PCIECHN(pi->tx_chan) |
+ FW_EQ_OFLD_CMD_IQID(iqid));
+ c.dcaen_to_eqsize = htonl(FW_EQ_OFLD_CMD_FBMIN(2) |
+ FW_EQ_OFLD_CMD_FBMAX(3) |
+ FW_EQ_OFLD_CMD_CIDXFTHRESH(5) |
+ FW_EQ_OFLD_CMD_EQSIZE(nentries));
+ c.eqaddr = cpu_to_be64(txq->q.phys_addr);
+
+ ret = t4_wr_mbox(adap, 0, &c, sizeof(c), &c);
+ if (ret) {
+ kfree(txq->q.sdesc);
+ txq->q.sdesc = NULL;
+ dma_free_coherent(adap->pdev_dev,
+ nentries * sizeof(struct tx_desc),
+ txq->q.desc, txq->q.phys_addr);
+ txq->q.desc = NULL;
+ return ret;
+ }
+
+ init_txq(adap, &txq->q, FW_EQ_OFLD_CMD_EQID_GET(ntohl(c.eqid_pkd)));
+ txq->adap = adap;
+ skb_queue_head_init(&txq->sendq);
+ tasklet_init(&txq->qresume_tsk, restart_ofldq, (unsigned long)txq);
+ txq->full = 0;
+ txq->mapping_err = 0;
+ return 0;
+}
+
+static void free_txq(struct adapter *adap, struct sge_txq *q)
+{
+ dma_free_coherent(adap->pdev_dev,
+ q->size * sizeof(struct tx_desc) + STAT_LEN,
+ q->desc, q->phys_addr);
+ q->cntxt_id = 0;
+ q->sdesc = NULL;
+ q->desc = NULL;
+}
+
+static void free_rspq_fl(struct adapter *adap, struct sge_rspq *rq,
+ struct sge_fl *fl)
+{
+ unsigned int fl_id = fl ? fl->cntxt_id : 0xffff;
+
+ adap->sge.ingr_map[rq->cntxt_id] = NULL;
+ t4_iq_free(adap, 0, 0, 0, FW_IQ_TYPE_FL_INT_CAP, rq->cntxt_id, fl_id,
+ 0xffff);
+ dma_free_coherent(adap->pdev_dev, (rq->size + 1) * rq->iqe_len,
+ rq->desc, rq->phys_addr);
+ netif_napi_del(&rq->napi);
+ rq->netdev = NULL;
+ rq->cntxt_id = rq->abs_id = 0;
+ rq->desc = NULL;
+
+ if (fl) {
+ free_rx_bufs(adap, fl, fl->avail);
+ dma_free_coherent(adap->pdev_dev, fl->size * 8 + STAT_LEN,
+ fl->desc, fl->addr);
+ kfree(fl->sdesc);
+ fl->sdesc = NULL;
+ fl->cntxt_id = 0;
+ fl->desc = NULL;
+ }
+}
+
+/**
+ * t4_free_sge_resources - free SGE resources
+ * @adap: the adapter
+ *
+ * Frees resources used by the SGE queue sets.
+ */
+void t4_free_sge_resources(struct adapter *adap)
+{
+ int i;
+ struct sge_eth_rxq *eq = adap->sge.ethrxq;
+ struct sge_eth_txq *etq = adap->sge.ethtxq;
+ struct sge_ofld_rxq *oq = adap->sge.ofldrxq;
+
+ /* clean up Ethernet Tx/Rx queues */
+ for (i = 0; i < adap->sge.ethqsets; i++, eq++, etq++) {
+ if (eq->rspq.desc)
+ free_rspq_fl(adap, &eq->rspq, &eq->fl);
+ if (etq->q.desc) {
+ t4_eth_eq_free(adap, 0, 0, 0, etq->q.cntxt_id);
+ free_tx_desc(adap, &etq->q, etq->q.in_use, true);
+ kfree(etq->q.sdesc);
+ free_txq(adap, &etq->q);
+ }
+ }
+
+ /* clean up RDMA and iSCSI Rx queues */
+ for (i = 0; i < adap->sge.ofldqsets; i++, oq++) {
+ if (oq->rspq.desc)
+ free_rspq_fl(adap, &oq->rspq, &oq->fl);
+ }
+ for (i = 0, oq = adap->sge.rdmarxq; i < adap->sge.rdmaqs; i++, oq++) {
+ if (oq->rspq.desc)
+ free_rspq_fl(adap, &oq->rspq, &oq->fl);
+ }
+
+ /* clean up offload Tx queues */
+ for (i = 0; i < ARRAY_SIZE(adap->sge.ofldtxq); i++) {
+ struct sge_ofld_txq *q = &adap->sge.ofldtxq[i];
+
+ if (q->q.desc) {
+ tasklet_kill(&q->qresume_tsk);
+ t4_ofld_eq_free(adap, 0, 0, 0, q->q.cntxt_id);
+ free_tx_desc(adap, &q->q, q->q.in_use, false);
+ kfree(q->q.sdesc);
+ __skb_queue_purge(&q->sendq);
+ free_txq(adap, &q->q);
+ }
+ }
+
+ /* clean up control Tx queues */
+ for (i = 0; i < ARRAY_SIZE(adap->sge.ctrlq); i++) {
+ struct sge_ctrl_txq *cq = &adap->sge.ctrlq[i];
+
+ if (cq->q.desc) {
+ tasklet_kill(&cq->qresume_tsk);
+ t4_ctrl_eq_free(adap, 0, 0, 0, cq->q.cntxt_id);
+ __skb_queue_purge(&cq->sendq);
+ free_txq(adap, &cq->q);
+ }
+ }
+
+ if (adap->sge.fw_evtq.desc)
+ free_rspq_fl(adap, &adap->sge.fw_evtq, NULL);
+
+ if (adap->sge.intrq.desc)
+ free_rspq_fl(adap, &adap->sge.intrq, NULL);
+
+ /* clear the reverse egress queue map */
+ memset(adap->sge.egr_map, 0, sizeof(adap->sge.egr_map));
+}
+
+void t4_sge_start(struct adapter *adap)
+{
+ adap->sge.ethtxq_rover = 0;
+ mod_timer(&adap->sge.rx_timer, jiffies + RX_QCHECK_PERIOD);
+ mod_timer(&adap->sge.tx_timer, jiffies + TX_QCHECK_PERIOD);
+}
+
+/**
+ * t4_sge_stop - disable SGE operation
+ * @adap: the adapter
+ *
+ * Stop tasklets and timers associated with the DMA engine. Note that
+ * this is effective only if measures have been taken to disable any HW
+ * events that may restart them.
+ */
+void t4_sge_stop(struct adapter *adap)
+{
+ int i;
+ struct sge *s = &adap->sge;
+
+ if (in_interrupt()) /* actions below require waiting */
+ return;
+
+ if (s->rx_timer.function)
+ del_timer_sync(&s->rx_timer);
+ if (s->tx_timer.function)
+ del_timer_sync(&s->tx_timer);
+
+ for (i = 0; i < ARRAY_SIZE(s->ofldtxq); i++) {
+ struct sge_ofld_txq *q = &s->ofldtxq[i];
+
+ if (q->q.desc)
+ tasklet_kill(&q->qresume_tsk);
+ }
+ for (i = 0; i < ARRAY_SIZE(s->ctrlq); i++) {
+ struct sge_ctrl_txq *cq = &s->ctrlq[i];
+
+ if (cq->q.desc)
+ tasklet_kill(&cq->qresume_tsk);
+ }
+}
+
+/**
+ * t4_sge_init - initialize SGE
+ * @adap: the adapter
+ *
+ * Performs SGE initialization needed every time after a chip reset.
+ * We do not initialize any of the queues here, instead the driver
+ * top-level must request them individually.
+ */
+void t4_sge_init(struct adapter *adap)
+{
+ struct sge *s = &adap->sge;
+ unsigned int fl_align_log = ilog2(FL_ALIGN);
+
+ t4_set_reg_field(adap, SGE_CONTROL, PKTSHIFT_MASK |
+ INGPADBOUNDARY_MASK | EGRSTATUSPAGESIZE,
+ INGPADBOUNDARY(fl_align_log - 5) | PKTSHIFT(2) |
+ RXPKTCPLMODE |
+ (STAT_LEN == 128 ? EGRSTATUSPAGESIZE : 0));
+ t4_set_reg_field(adap, SGE_HOST_PAGE_SIZE, HOSTPAGESIZEPF0_MASK,
+ HOSTPAGESIZEPF0(PAGE_SHIFT - 10));
+ t4_write_reg(adap, SGE_FL_BUFFER_SIZE0, PAGE_SIZE);
+#if FL_PG_ORDER > 0
+ t4_write_reg(adap, SGE_FL_BUFFER_SIZE1, PAGE_SIZE << FL_PG_ORDER);
+#endif
+ t4_write_reg(adap, SGE_INGRESS_RX_THRESHOLD,
+ THRESHOLD_0(s->counter_val[0]) |
+ THRESHOLD_1(s->counter_val[1]) |
+ THRESHOLD_2(s->counter_val[2]) |
+ THRESHOLD_3(s->counter_val[3]));
+ t4_write_reg(adap, SGE_TIMER_VALUE_0_AND_1,
+ TIMERVALUE0(us_to_core_ticks(adap, s->timer_val[0])) |
+ TIMERVALUE1(us_to_core_ticks(adap, s->timer_val[1])));
+ t4_write_reg(adap, SGE_TIMER_VALUE_2_AND_3,
+ TIMERVALUE0(us_to_core_ticks(adap, s->timer_val[2])) |
+ TIMERVALUE1(us_to_core_ticks(adap, s->timer_val[3])));
+ t4_write_reg(adap, SGE_TIMER_VALUE_4_AND_5,
+ TIMERVALUE0(us_to_core_ticks(adap, s->timer_val[4])) |
+ TIMERVALUE1(us_to_core_ticks(adap, s->timer_val[5])));
+ setup_timer(&s->rx_timer, sge_rx_timer_cb, (unsigned long)adap);
+ setup_timer(&s->tx_timer, sge_tx_timer_cb, (unsigned long)adap);
+ s->starve_thres = core_ticks_per_usec(adap) * 1000000; /* 1 s */
+ s->idma_state[0] = s->idma_state[1] = 0;
+ spin_lock_init(&s->intrq_lock);
+}
--- /dev/null
+/*
+ * This file is part of the Chelsio T4 Ethernet driver for Linux.
+ *
+ * Copyright (c) 2003-2010 Chelsio Communications, Inc. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include <linux/init.h>
+#include <linux/delay.h>
+#include "cxgb4.h"
+#include "t4_regs.h"
+#include "t4fw_api.h"
+
+/**
+ * t4_wait_op_done_val - wait until an operation is completed
+ * @adapter: the adapter performing the operation
+ * @reg: the register to check for completion
+ * @mask: a single-bit field within @reg that indicates completion
+ * @polarity: the value of the field when the operation is completed
+ * @attempts: number of check iterations
+ * @delay: delay in usecs between iterations
+ * @valp: where to store the value of the register at completion time
+ *
+ * Wait until an operation is completed by checking a bit in a register
+ * up to @attempts times. If @valp is not NULL the value of the register
+ * at the time it indicated completion is stored there. Returns 0 if the
+ * operation completes and -EAGAIN otherwise.
+ */
+int t4_wait_op_done_val(struct adapter *adapter, int reg, u32 mask,
+ int polarity, int attempts, int delay, u32 *valp)
+{
+ while (1) {
+ u32 val = t4_read_reg(adapter, reg);
+
+ if (!!(val & mask) == polarity) {
+ if (valp)
+ *valp = val;
+ return 0;
+ }
+ if (--attempts == 0)
+ return -EAGAIN;
+ if (delay)
+ udelay(delay);
+ }
+}
+
+static inline int t4_wait_op_done(struct adapter *adapter, int reg, u32 mask,
+ int polarity, int attempts, int delay)
+{
+ return t4_wait_op_done_val(adapter, reg, mask, polarity, attempts,
+ delay, NULL);
+}
+
+/**
+ * t4_set_reg_field - set a register field to a value
+ * @adapter: the adapter to program
+ * @addr: the register address
+ * @mask: specifies the portion of the register to modify
+ * @val: the new value for the register field
+ *
+ * Sets a register field specified by the supplied mask to the
+ * given value.
+ */
+void t4_set_reg_field(struct adapter *adapter, unsigned int addr, u32 mask,
+ u32 val)
+{
+ u32 v = t4_read_reg(adapter, addr) & ~mask;
+
+ t4_write_reg(adapter, addr, v | val);
+ (void) t4_read_reg(adapter, addr); /* flush */
+}
+
+/**
+ * t4_read_indirect - read indirectly addressed registers
+ * @adap: the adapter
+ * @addr_reg: register holding the indirect address
+ * @data_reg: register holding the value of the indirect register
+ * @vals: where the read register values are stored
+ * @nregs: how many indirect registers to read
+ * @start_idx: index of first indirect register to read
+ *
+ * Reads registers that are accessed indirectly through an address/data
+ * register pair.
+ */
+void t4_read_indirect(struct adapter *adap, unsigned int addr_reg,
+ unsigned int data_reg, u32 *vals, unsigned int nregs,
+ unsigned int start_idx)
+{
+ while (nregs--) {
+ t4_write_reg(adap, addr_reg, start_idx);
+ *vals++ = t4_read_reg(adap, data_reg);
+ start_idx++;
+ }
+}
+
+/**
+ * t4_write_indirect - write indirectly addressed registers
+ * @adap: the adapter
+ * @addr_reg: register holding the indirect addresses
+ * @data_reg: register holding the value for the indirect registers
+ * @vals: values to write
+ * @nregs: how many indirect registers to write
+ * @start_idx: address of first indirect register to write
+ *
+ * Writes a sequential block of registers that are accessed indirectly
+ * through an address/data register pair.
+ */
+void t4_write_indirect(struct adapter *adap, unsigned int addr_reg,
+ unsigned int data_reg, const u32 *vals,
+ unsigned int nregs, unsigned int start_idx)
+{
+ while (nregs--) {
+ t4_write_reg(adap, addr_reg, start_idx++);
+ t4_write_reg(adap, data_reg, *vals++);
+ }
+}
+
+/*
+ * Get the reply to a mailbox command and store it in @rpl in big-endian order.
+ */
+static void get_mbox_rpl(struct adapter *adap, __be64 *rpl, int nflit,
+ u32 mbox_addr)
+{
+ for ( ; nflit; nflit--, mbox_addr += 8)
+ *rpl++ = cpu_to_be64(t4_read_reg64(adap, mbox_addr));
+}
+
+/*
+ * Handle a FW assertion reported in a mailbox.
+ */
+static void fw_asrt(struct adapter *adap, u32 mbox_addr)
+{
+ struct fw_debug_cmd asrt;
+
+ get_mbox_rpl(adap, (__be64 *)&asrt, sizeof(asrt) / 8, mbox_addr);
+ dev_alert(adap->pdev_dev,
+ "FW assertion at %.16s:%u, val0 %#x, val1 %#x\n",
+ asrt.u.assert.filename_0_7, ntohl(asrt.u.assert.line),
+ ntohl(asrt.u.assert.x), ntohl(asrt.u.assert.y));
+}
+
+static void dump_mbox(struct adapter *adap, int mbox, u32 data_reg)
+{
+ dev_err(adap->pdev_dev,
+ "mbox %d: %llx %llx %llx %llx %llx %llx %llx %llx\n", mbox,
+ (unsigned long long)t4_read_reg64(adap, data_reg),
+ (unsigned long long)t4_read_reg64(adap, data_reg + 8),
+ (unsigned long long)t4_read_reg64(adap, data_reg + 16),
+ (unsigned long long)t4_read_reg64(adap, data_reg + 24),
+ (unsigned long long)t4_read_reg64(adap, data_reg + 32),
+ (unsigned long long)t4_read_reg64(adap, data_reg + 40),
+ (unsigned long long)t4_read_reg64(adap, data_reg + 48),
+ (unsigned long long)t4_read_reg64(adap, data_reg + 56));
+}
+
+/**
+ * t4_wr_mbox_meat - send a command to FW through the given mailbox
+ * @adap: the adapter
+ * @mbox: index of the mailbox to use
+ * @cmd: the command to write
+ * @size: command length in bytes
+ * @rpl: where to optionally store the reply
+ * @sleep_ok: if true we may sleep while awaiting command completion
+ *
+ * Sends the given command to FW through the selected mailbox and waits
+ * for the FW to execute the command. If @rpl is not %NULL it is used to
+ * store the FW's reply to the command. The command and its optional
+ * reply are of the same length. FW can take up to %FW_CMD_MAX_TIMEOUT ms
+ * to respond. @sleep_ok determines whether we may sleep while awaiting
+ * the response. If sleeping is allowed we use progressive backoff
+ * otherwise we spin.
+ *
+ * The return value is 0 on success or a negative errno on failure. A
+ * failure can happen either because we are not able to execute the
+ * command or FW executes it but signals an error. In the latter case
+ * the return value is the error code indicated by FW (negated).
+ */
+int t4_wr_mbox_meat(struct adapter *adap, int mbox, const void *cmd, int size,
+ void *rpl, bool sleep_ok)
+{
+ static int delay[] = {
+ 1, 1, 3, 5, 10, 10, 20, 50, 100, 200
+ };
+
+ u32 v;
+ u64 res;
+ int i, ms, delay_idx;
+ const __be64 *p = cmd;
+ u32 data_reg = PF_REG(mbox, CIM_PF_MAILBOX_DATA);
+ u32 ctl_reg = PF_REG(mbox, CIM_PF_MAILBOX_CTRL);
+
+ if ((size & 15) || size > MBOX_LEN)
+ return -EINVAL;
+
+ v = MBOWNER_GET(t4_read_reg(adap, ctl_reg));
+ for (i = 0; v == MBOX_OWNER_NONE && i < 3; i++)
+ v = MBOWNER_GET(t4_read_reg(adap, ctl_reg));
+
+ if (v != MBOX_OWNER_DRV)
+ return v ? -EBUSY : -ETIMEDOUT;
+
+ for (i = 0; i < size; i += 8)
+ t4_write_reg64(adap, data_reg + i, be64_to_cpu(*p++));
+
+ t4_write_reg(adap, ctl_reg, MBMSGVALID | MBOWNER(MBOX_OWNER_FW));
+ t4_read_reg(adap, ctl_reg); /* flush write */
+
+ delay_idx = 0;
+ ms = delay[0];
+
+ for (i = 0; i < FW_CMD_MAX_TIMEOUT; i += ms) {
+ if (sleep_ok) {
+ ms = delay[delay_idx]; /* last element may repeat */
+ if (delay_idx < ARRAY_SIZE(delay) - 1)
+ delay_idx++;
+ msleep(ms);
+ } else
+ mdelay(ms);
+
+ v = t4_read_reg(adap, ctl_reg);
+ if (MBOWNER_GET(v) == MBOX_OWNER_DRV) {
+ if (!(v & MBMSGVALID)) {
+ t4_write_reg(adap, ctl_reg, 0);
+ continue;
+ }
+
+ res = t4_read_reg64(adap, data_reg);
+ if (FW_CMD_OP_GET(res >> 32) == FW_DEBUG_CMD) {
+ fw_asrt(adap, data_reg);
+ res = FW_CMD_RETVAL(EIO);
+ } else if (rpl)
+ get_mbox_rpl(adap, rpl, size / 8, data_reg);
+
+ if (FW_CMD_RETVAL_GET((int)res))
+ dump_mbox(adap, mbox, data_reg);
+ t4_write_reg(adap, ctl_reg, 0);
+ return -FW_CMD_RETVAL_GET((int)res);
+ }
+ }
+
+ dump_mbox(adap, mbox, data_reg);
+ dev_err(adap->pdev_dev, "command %#x in mailbox %d timed out\n",
+ *(const u8 *)cmd, mbox);
+ return -ETIMEDOUT;
+}
+
+/**
+ * t4_mc_read - read from MC through backdoor accesses
+ * @adap: the adapter
+ * @addr: address of first byte requested
+ * @data: 64 bytes of data containing the requested address
+ * @ecc: where to store the corresponding 64-bit ECC word
+ *
+ * Read 64 bytes of data from MC starting at a 64-byte-aligned address
+ * that covers the requested address @addr. If @parity is not %NULL it
+ * is assigned the 64-bit ECC word for the read data.
+ */
+int t4_mc_read(struct adapter *adap, u32 addr, __be32 *data, u64 *ecc)
+{
+ int i;
+
+ if (t4_read_reg(adap, MC_BIST_CMD) & START_BIST)
+ return -EBUSY;
+ t4_write_reg(adap, MC_BIST_CMD_ADDR, addr & ~0x3fU);
+ t4_write_reg(adap, MC_BIST_CMD_LEN, 64);
+ t4_write_reg(adap, MC_BIST_DATA_PATTERN, 0xc);
+ t4_write_reg(adap, MC_BIST_CMD, BIST_OPCODE(1) | START_BIST |
+ BIST_CMD_GAP(1));
+ i = t4_wait_op_done(adap, MC_BIST_CMD, START_BIST, 0, 10, 1);
+ if (i)
+ return i;
+
+#define MC_DATA(i) MC_BIST_STATUS_REG(MC_BIST_STATUS_RDATA, i)
+
+ for (i = 15; i >= 0; i--)
+ *data++ = htonl(t4_read_reg(adap, MC_DATA(i)));
+ if (ecc)
+ *ecc = t4_read_reg64(adap, MC_DATA(16));
+#undef MC_DATA
+ return 0;
+}
+
+/**
+ * t4_edc_read - read from EDC through backdoor accesses
+ * @adap: the adapter
+ * @idx: which EDC to access
+ * @addr: address of first byte requested
+ * @data: 64 bytes of data containing the requested address
+ * @ecc: where to store the corresponding 64-bit ECC word
+ *
+ * Read 64 bytes of data from EDC starting at a 64-byte-aligned address
+ * that covers the requested address @addr. If @parity is not %NULL it
+ * is assigned the 64-bit ECC word for the read data.
+ */
+int t4_edc_read(struct adapter *adap, int idx, u32 addr, __be32 *data, u64 *ecc)
+{
+ int i;
+
+ idx *= EDC_STRIDE;
+ if (t4_read_reg(adap, EDC_BIST_CMD + idx) & START_BIST)
+ return -EBUSY;
+ t4_write_reg(adap, EDC_BIST_CMD_ADDR + idx, addr & ~0x3fU);
+ t4_write_reg(adap, EDC_BIST_CMD_LEN + idx, 64);
+ t4_write_reg(adap, EDC_BIST_DATA_PATTERN + idx, 0xc);
+ t4_write_reg(adap, EDC_BIST_CMD + idx,
+ BIST_OPCODE(1) | BIST_CMD_GAP(1) | START_BIST);
+ i = t4_wait_op_done(adap, EDC_BIST_CMD + idx, START_BIST, 0, 10, 1);
+ if (i)
+ return i;
+
+#define EDC_DATA(i) (EDC_BIST_STATUS_REG(EDC_BIST_STATUS_RDATA, i) + idx)
+
+ for (i = 15; i >= 0; i--)
+ *data++ = htonl(t4_read_reg(adap, EDC_DATA(i)));
+ if (ecc)
+ *ecc = t4_read_reg64(adap, EDC_DATA(16));
+#undef EDC_DATA
+ return 0;
+}
+
+#define VPD_ENTRY(name, len) \
+ u8 name##_kword[2]; u8 name##_len; u8 name##_data[len]
+
+/*
+ * Partial EEPROM Vital Product Data structure. Includes only the ID and
+ * VPD-R sections.
+ */
+struct t4_vpd {
+ u8 id_tag;
+ u8 id_len[2];
+ u8 id_data[ID_LEN];
+ u8 vpdr_tag;
+ u8 vpdr_len[2];
+ VPD_ENTRY(pn, 16); /* part number */
+ VPD_ENTRY(ec, EC_LEN); /* EC level */
+ VPD_ENTRY(sn, SERNUM_LEN); /* serial number */
+ VPD_ENTRY(na, 12); /* MAC address base */
+ VPD_ENTRY(port_type, 8); /* port types */
+ VPD_ENTRY(gpio, 14); /* GPIO usage */
+ VPD_ENTRY(cclk, 6); /* core clock */
+ VPD_ENTRY(port_addr, 8); /* port MDIO addresses */
+ VPD_ENTRY(rv, 1); /* csum */
+ u32 pad; /* for multiple-of-4 sizing and alignment */
+};
+
+#define EEPROM_STAT_ADDR 0x7bfc
+#define VPD_BASE 0
+
+/**
+ * t4_seeprom_wp - enable/disable EEPROM write protection
+ * @adapter: the adapter
+ * @enable: whether to enable or disable write protection
+ *
+ * Enables or disables write protection on the serial EEPROM.
+ */
+int t4_seeprom_wp(struct adapter *adapter, bool enable)
+{
+ unsigned int v = enable ? 0xc : 0;
+ int ret = pci_write_vpd(adapter->pdev, EEPROM_STAT_ADDR, 4, &v);
+ return ret < 0 ? ret : 0;
+}
+
+/**
+ * get_vpd_params - read VPD parameters from VPD EEPROM
+ * @adapter: adapter to read
+ * @p: where to store the parameters
+ *
+ * Reads card parameters stored in VPD EEPROM.
+ */
+static int get_vpd_params(struct adapter *adapter, struct vpd_params *p)
+{
+ int ret;
+ struct t4_vpd vpd;
+ u8 *q = (u8 *)&vpd, csum;
+
+ ret = pci_read_vpd(adapter->pdev, VPD_BASE, sizeof(vpd), &vpd);
+ if (ret < 0)
+ return ret;
+
+ for (csum = 0; q <= vpd.rv_data; q++)
+ csum += *q;
+
+ if (csum) {
+ dev_err(adapter->pdev_dev,
+ "corrupted VPD EEPROM, actual csum %u\n", csum);
+ return -EINVAL;
+ }
+
+ p->cclk = simple_strtoul(vpd.cclk_data, NULL, 10);
+ memcpy(p->id, vpd.id_data, sizeof(vpd.id_data));
+ strim(p->id);
+ memcpy(p->ec, vpd.ec_data, sizeof(vpd.ec_data));
+ strim(p->ec);
+ memcpy(p->sn, vpd.sn_data, sizeof(vpd.sn_data));
+ strim(p->sn);
+ return 0;
+}
+
+/* serial flash and firmware constants */
+enum {
+ SF_ATTEMPTS = 10, /* max retries for SF operations */
+
+ /* flash command opcodes */
+ SF_PROG_PAGE = 2, /* program page */
+ SF_WR_DISABLE = 4, /* disable writes */
+ SF_RD_STATUS = 5, /* read status register */
+ SF_WR_ENABLE = 6, /* enable writes */
+ SF_RD_DATA_FAST = 0xb, /* read flash */
+ SF_ERASE_SECTOR = 0xd8, /* erase sector */
+
+ FW_START_SEC = 8, /* first flash sector for FW */
+ FW_END_SEC = 15, /* last flash sector for FW */
+ FW_IMG_START = FW_START_SEC * SF_SEC_SIZE,
+ FW_MAX_SIZE = (FW_END_SEC - FW_START_SEC + 1) * SF_SEC_SIZE,
+};
+
+/**
+ * sf1_read - read data from the serial flash
+ * @adapter: the adapter
+ * @byte_cnt: number of bytes to read
+ * @cont: whether another operation will be chained
+ * @lock: whether to lock SF for PL access only
+ * @valp: where to store the read data
+ *
+ * Reads up to 4 bytes of data from the serial flash. The location of
+ * the read needs to be specified prior to calling this by issuing the
+ * appropriate commands to the serial flash.
+ */
+static int sf1_read(struct adapter *adapter, unsigned int byte_cnt, int cont,
+ int lock, u32 *valp)
+{
+ int ret;
+
+ if (!byte_cnt || byte_cnt > 4)
+ return -EINVAL;
+ if (t4_read_reg(adapter, SF_OP) & BUSY)
+ return -EBUSY;
+ cont = cont ? SF_CONT : 0;
+ lock = lock ? SF_LOCK : 0;
+ t4_write_reg(adapter, SF_OP, lock | cont | BYTECNT(byte_cnt - 1));
+ ret = t4_wait_op_done(adapter, SF_OP, BUSY, 0, SF_ATTEMPTS, 5);
+ if (!ret)
+ *valp = t4_read_reg(adapter, SF_DATA);
+ return ret;
+}
+
+/**
+ * sf1_write - write data to the serial flash
+ * @adapter: the adapter
+ * @byte_cnt: number of bytes to write
+ * @cont: whether another operation will be chained
+ * @lock: whether to lock SF for PL access only
+ * @val: value to write
+ *
+ * Writes up to 4 bytes of data to the serial flash. The location of
+ * the write needs to be specified prior to calling this by issuing the
+ * appropriate commands to the serial flash.
+ */
+static int sf1_write(struct adapter *adapter, unsigned int byte_cnt, int cont,
+ int lock, u32 val)
+{
+ if (!byte_cnt || byte_cnt > 4)
+ return -EINVAL;
+ if (t4_read_reg(adapter, SF_OP) & BUSY)
+ return -EBUSY;
+ cont = cont ? SF_CONT : 0;
+ lock = lock ? SF_LOCK : 0;
+ t4_write_reg(adapter, SF_DATA, val);
+ t4_write_reg(adapter, SF_OP, lock |
+ cont | BYTECNT(byte_cnt - 1) | OP_WR);
+ return t4_wait_op_done(adapter, SF_OP, BUSY, 0, SF_ATTEMPTS, 5);
+}
+
+/**
+ * flash_wait_op - wait for a flash operation to complete
+ * @adapter: the adapter
+ * @attempts: max number of polls of the status register
+ * @delay: delay between polls in ms
+ *
+ * Wait for a flash operation to complete by polling the status register.
+ */
+static int flash_wait_op(struct adapter *adapter, int attempts, int delay)
+{
+ int ret;
+ u32 status;
+
+ while (1) {
+ if ((ret = sf1_write(adapter, 1, 1, 1, SF_RD_STATUS)) != 0 ||
+ (ret = sf1_read(adapter, 1, 0, 1, &status)) != 0)
+ return ret;
+ if (!(status & 1))
+ return 0;
+ if (--attempts == 0)
+ return -EAGAIN;
+ if (delay)
+ msleep(delay);
+ }
+}
+
+/**
+ * t4_read_flash - read words from serial flash
+ * @adapter: the adapter
+ * @addr: the start address for the read
+ * @nwords: how many 32-bit words to read
+ * @data: where to store the read data
+ * @byte_oriented: whether to store data as bytes or as words
+ *
+ * Read the specified number of 32-bit words from the serial flash.
+ * If @byte_oriented is set the read data is stored as a byte array
+ * (i.e., big-endian), otherwise as 32-bit words in the platform's
+ * natural endianess.
+ */
+int t4_read_flash(struct adapter *adapter, unsigned int addr,
+ unsigned int nwords, u32 *data, int byte_oriented)
+{
+ int ret;
+
+ if (addr + nwords * sizeof(u32) > SF_SIZE || (addr & 3))
+ return -EINVAL;
+
+ addr = swab32(addr) | SF_RD_DATA_FAST;
+
+ if ((ret = sf1_write(adapter, 4, 1, 0, addr)) != 0 ||
+ (ret = sf1_read(adapter, 1, 1, 0, data)) != 0)
+ return ret;
+
+ for ( ; nwords; nwords--, data++) {
+ ret = sf1_read(adapter, 4, nwords > 1, nwords == 1, data);
+ if (nwords == 1)
+ t4_write_reg(adapter, SF_OP, 0); /* unlock SF */
+ if (ret)
+ return ret;
+ if (byte_oriented)
+ *data = htonl(*data);
+ }
+ return 0;
+}
+
+/**
+ * t4_write_flash - write up to a page of data to the serial flash
+ * @adapter: the adapter
+ * @addr: the start address to write
+ * @n: length of data to write in bytes
+ * @data: the data to write
+ *
+ * Writes up to a page of data (256 bytes) to the serial flash starting
+ * at the given address. All the data must be written to the same page.
+ */
+static int t4_write_flash(struct adapter *adapter, unsigned int addr,
+ unsigned int n, const u8 *data)
+{
+ int ret;
+ u32 buf[64];
+ unsigned int i, c, left, val, offset = addr & 0xff;
+
+ if (addr >= SF_SIZE || offset + n > SF_PAGE_SIZE)
+ return -EINVAL;
+
+ val = swab32(addr) | SF_PROG_PAGE;
+
+ if ((ret = sf1_write(adapter, 1, 0, 1, SF_WR_ENABLE)) != 0 ||
+ (ret = sf1_write(adapter, 4, 1, 1, val)) != 0)
+ goto unlock;
+
+ for (left = n; left; left -= c) {
+ c = min(left, 4U);
+ for (val = 0, i = 0; i < c; ++i)
+ val = (val << 8) + *data++;
+
+ ret = sf1_write(adapter, c, c != left, 1, val);
+ if (ret)
+ goto unlock;
+ }
+ ret = flash_wait_op(adapter, 5, 1);
+ if (ret)
+ goto unlock;
+
+ t4_write_reg(adapter, SF_OP, 0); /* unlock SF */
+
+ /* Read the page to verify the write succeeded */
+ ret = t4_read_flash(adapter, addr & ~0xff, ARRAY_SIZE(buf), buf, 1);
+ if (ret)
+ return ret;
+
+ if (memcmp(data - n, (u8 *)buf + offset, n)) {
+ dev_err(adapter->pdev_dev,
+ "failed to correctly write the flash page at %#x\n",
+ addr);
+ return -EIO;
+ }
+ return 0;
+
+unlock:
+ t4_write_reg(adapter, SF_OP, 0); /* unlock SF */
+ return ret;
+}
+
+/**
+ * get_fw_version - read the firmware version
+ * @adapter: the adapter
+ * @vers: where to place the version
+ *
+ * Reads the FW version from flash.
+ */
+static int get_fw_version(struct adapter *adapter, u32 *vers)
+{
+ return t4_read_flash(adapter,
+ FW_IMG_START + offsetof(struct fw_hdr, fw_ver), 1,
+ vers, 0);
+}
+
+/**
+ * get_tp_version - read the TP microcode version
+ * @adapter: the adapter
+ * @vers: where to place the version
+ *
+ * Reads the TP microcode version from flash.
+ */
+static int get_tp_version(struct adapter *adapter, u32 *vers)
+{
+ return t4_read_flash(adapter, FW_IMG_START + offsetof(struct fw_hdr,
+ tp_microcode_ver),
+ 1, vers, 0);
+}
+
+/**
+ * t4_check_fw_version - check if the FW is compatible with this driver
+ * @adapter: the adapter
+ *
+ * Checks if an adapter's FW is compatible with the driver. Returns 0
+ * if there's exact match, a negative error if the version could not be
+ * read or there's a major version mismatch, and a positive value if the
+ * expected major version is found but there's a minor version mismatch.
+ */
+int t4_check_fw_version(struct adapter *adapter)
+{
+ u32 api_vers[2];
+ int ret, major, minor, micro;
+
+ ret = get_fw_version(adapter, &adapter->params.fw_vers);
+ if (!ret)
+ ret = get_tp_version(adapter, &adapter->params.tp_vers);
+ if (!ret)
+ ret = t4_read_flash(adapter,
+ FW_IMG_START + offsetof(struct fw_hdr, intfver_nic),
+ 2, api_vers, 1);
+ if (ret)
+ return ret;
+
+ major = FW_HDR_FW_VER_MAJOR_GET(adapter->params.fw_vers);
+ minor = FW_HDR_FW_VER_MINOR_GET(adapter->params.fw_vers);
+ micro = FW_HDR_FW_VER_MICRO_GET(adapter->params.fw_vers);
+ memcpy(adapter->params.api_vers, api_vers,
+ sizeof(adapter->params.api_vers));
+
+ if (major != FW_VERSION_MAJOR) { /* major mismatch - fail */
+ dev_err(adapter->pdev_dev,
+ "card FW has major version %u, driver wants %u\n",
+ major, FW_VERSION_MAJOR);
+ return -EINVAL;
+ }
+
+ if (minor == FW_VERSION_MINOR && micro == FW_VERSION_MICRO)
+ return 0; /* perfect match */
+
+ /* Minor/micro version mismatch. Report it but often it's OK. */
+ return 1;
+}
+
+/**
+ * t4_flash_erase_sectors - erase a range of flash sectors
+ * @adapter: the adapter
+ * @start: the first sector to erase
+ * @end: the last sector to erase
+ *
+ * Erases the sectors in the given inclusive range.
+ */
+static int t4_flash_erase_sectors(struct adapter *adapter, int start, int end)
+{
+ int ret = 0;
+
+ while (start <= end) {
+ if ((ret = sf1_write(adapter, 1, 0, 1, SF_WR_ENABLE)) != 0 ||
+ (ret = sf1_write(adapter, 4, 0, 1,
+ SF_ERASE_SECTOR | (start << 8))) != 0 ||
+ (ret = flash_wait_op(adapter, 5, 500)) != 0) {
+ dev_err(adapter->pdev_dev,
+ "erase of flash sector %d failed, error %d\n",
+ start, ret);
+ break;
+ }
+ start++;
+ }
+ t4_write_reg(adapter, SF_OP, 0); /* unlock SF */
+ return ret;
+}
+
+/**
+ * t4_load_fw - download firmware
+ * @adap: the adapter
+ * @fw_data: the firmware image to write
+ * @size: image size
+ *
+ * Write the supplied firmware image to the card's serial flash.
+ */
+int t4_load_fw(struct adapter *adap, const u8 *fw_data, unsigned int size)
+{
+ u32 csum;
+ int ret, addr;
+ unsigned int i;
+ u8 first_page[SF_PAGE_SIZE];
+ const u32 *p = (const u32 *)fw_data;
+ const struct fw_hdr *hdr = (const struct fw_hdr *)fw_data;
+
+ if (!size) {
+ dev_err(adap->pdev_dev, "FW image has no data\n");
+ return -EINVAL;
+ }
+ if (size & 511) {
+ dev_err(adap->pdev_dev,
+ "FW image size not multiple of 512 bytes\n");
+ return -EINVAL;
+ }
+ if (ntohs(hdr->len512) * 512 != size) {
+ dev_err(adap->pdev_dev,
+ "FW image size differs from size in FW header\n");
+ return -EINVAL;
+ }
+ if (size > FW_MAX_SIZE) {
+ dev_err(adap->pdev_dev, "FW image too large, max is %u bytes\n",
+ FW_MAX_SIZE);
+ return -EFBIG;
+ }
+
+ for (csum = 0, i = 0; i < size / sizeof(csum); i++)
+ csum += ntohl(p[i]);
+
+ if (csum != 0xffffffff) {
+ dev_err(adap->pdev_dev,
+ "corrupted firmware image, checksum %#x\n", csum);
+ return -EINVAL;
+ }
+
+ i = DIV_ROUND_UP(size, SF_SEC_SIZE); /* # of sectors spanned */
+ ret = t4_flash_erase_sectors(adap, FW_START_SEC, FW_START_SEC + i - 1);
+ if (ret)
+ goto out;
+
+ /*
+ * We write the correct version at the end so the driver can see a bad
+ * version if the FW write fails. Start by writing a copy of the
+ * first page with a bad version.
+ */
+ memcpy(first_page, fw_data, SF_PAGE_SIZE);
+ ((struct fw_hdr *)first_page)->fw_ver = htonl(0xffffffff);
+ ret = t4_write_flash(adap, FW_IMG_START, SF_PAGE_SIZE, first_page);
+ if (ret)
+ goto out;
+
+ addr = FW_IMG_START;
+ for (size -= SF_PAGE_SIZE; size; size -= SF_PAGE_SIZE) {
+ addr += SF_PAGE_SIZE;
+ fw_data += SF_PAGE_SIZE;
+ ret = t4_write_flash(adap, addr, SF_PAGE_SIZE, fw_data);
+ if (ret)
+ goto out;
+ }
+
+ ret = t4_write_flash(adap,
+ FW_IMG_START + offsetof(struct fw_hdr, fw_ver),
+ sizeof(hdr->fw_ver), (const u8 *)&hdr->fw_ver);
+out:
+ if (ret)
+ dev_err(adap->pdev_dev, "firmware download failed, error %d\n",
+ ret);
+ return ret;
+}
+
+#define ADVERT_MASK (FW_PORT_CAP_SPEED_100M | FW_PORT_CAP_SPEED_1G |\
+ FW_PORT_CAP_SPEED_10G | FW_PORT_CAP_ANEG)
+
+/**
+ * t4_link_start - apply link configuration to MAC/PHY
+ * @phy: the PHY to setup
+ * @mac: the MAC to setup
+ * @lc: the requested link configuration
+ *
+ * Set up a port's MAC and PHY according to a desired link configuration.
+ * - If the PHY can auto-negotiate first decide what to advertise, then
+ * enable/disable auto-negotiation as desired, and reset.
+ * - If the PHY does not auto-negotiate just reset it.
+ * - If auto-negotiation is off set the MAC to the proper speed/duplex/FC,
+ * otherwise do it later based on the outcome of auto-negotiation.
+ */
+int t4_link_start(struct adapter *adap, unsigned int mbox, unsigned int port,
+ struct link_config *lc)
+{
+ struct fw_port_cmd c;
+ unsigned int fc = 0, mdi = FW_PORT_MDI(FW_PORT_MDI_AUTO);
+
+ lc->link_ok = 0;
+ if (lc->requested_fc & PAUSE_RX)
+ fc |= FW_PORT_CAP_FC_RX;
+ if (lc->requested_fc & PAUSE_TX)
+ fc |= FW_PORT_CAP_FC_TX;
+
+ memset(&c, 0, sizeof(c));
+ c.op_to_portid = htonl(FW_CMD_OP(FW_PORT_CMD) | FW_CMD_REQUEST |
+ FW_CMD_EXEC | FW_PORT_CMD_PORTID(port));
+ c.action_to_len16 = htonl(FW_PORT_CMD_ACTION(FW_PORT_ACTION_L1_CFG) |
+ FW_LEN16(c));
+
+ if (!(lc->supported & FW_PORT_CAP_ANEG)) {
+ c.u.l1cfg.rcap = htonl((lc->supported & ADVERT_MASK) | fc);
+ lc->fc = lc->requested_fc & (PAUSE_RX | PAUSE_TX);
+ } else if (lc->autoneg == AUTONEG_DISABLE) {
+ c.u.l1cfg.rcap = htonl(lc->requested_speed | fc | mdi);
+ lc->fc = lc->requested_fc & (PAUSE_RX | PAUSE_TX);
+ } else
+ c.u.l1cfg.rcap = htonl(lc->advertising | fc | mdi);
+
+ return t4_wr_mbox(adap, mbox, &c, sizeof(c), NULL);
+}
+
+/**
+ * t4_restart_aneg - restart autonegotiation
+ * @adap: the adapter
+ * @mbox: mbox to use for the FW command
+ * @port: the port id
+ *
+ * Restarts autonegotiation for the selected port.
+ */
+int t4_restart_aneg(struct adapter *adap, unsigned int mbox, unsigned int port)
+{
+ struct fw_port_cmd c;
+
+ memset(&c, 0, sizeof(c));
+ c.op_to_portid = htonl(FW_CMD_OP(FW_PORT_CMD) | FW_CMD_REQUEST |
+ FW_CMD_EXEC | FW_PORT_CMD_PORTID(port));
+ c.action_to_len16 = htonl(FW_PORT_CMD_ACTION(FW_PORT_ACTION_L1_CFG) |
+ FW_LEN16(c));
+ c.u.l1cfg.rcap = htonl(FW_PORT_CAP_ANEG);
+ return t4_wr_mbox(adap, mbox, &c, sizeof(c), NULL);
+}
+
+/**
+ * t4_set_vlan_accel - configure HW VLAN extraction
+ * @adap: the adapter
+ * @ports: bitmap of adapter ports to operate on
+ * @on: enable (1) or disable (0) HW VLAN extraction
+ *
+ * Enables or disables HW extraction of VLAN tags for the ports specified
+ * by @ports. @ports is a bitmap with the ith bit designating the port
+ * associated with the ith adapter channel.
+ */
+void t4_set_vlan_accel(struct adapter *adap, unsigned int ports, int on)
+{
+ ports <<= VLANEXTENABLE_SHIFT;
+ t4_set_reg_field(adap, TP_OUT_CONFIG, ports, on ? ports : 0);
+}
+
+struct intr_info {
+ unsigned int mask; /* bits to check in interrupt status */
+ const char *msg; /* message to print or NULL */
+ short stat_idx; /* stat counter to increment or -1 */
+ unsigned short fatal; /* whether the condition reported is fatal */
+};
+
+/**
+ * t4_handle_intr_status - table driven interrupt handler
+ * @adapter: the adapter that generated the interrupt
+ * @reg: the interrupt status register to process
+ * @acts: table of interrupt actions
+ *
+ * A table driven interrupt handler that applies a set of masks to an
+ * interrupt status word and performs the corresponding actions if the
+ * interrupts described by the mask have occured. The actions include
+ * optionally emitting a warning or alert message. The table is terminated
+ * by an entry specifying mask 0. Returns the number of fatal interrupt
+ * conditions.
+ */
+static int t4_handle_intr_status(struct adapter *adapter, unsigned int reg,
+ const struct intr_info *acts)
+{
+ int fatal = 0;
+ unsigned int mask = 0;
+ unsigned int status = t4_read_reg(adapter, reg);
+
+ for ( ; acts->mask; ++acts) {
+ if (!(status & acts->mask))
+ continue;
+ if (acts->fatal) {
+ fatal++;
+ dev_alert(adapter->pdev_dev, "%s (0x%x)\n", acts->msg,
+ status & acts->mask);
+ } else if (acts->msg && printk_ratelimit())
+ dev_warn(adapter->pdev_dev, "%s (0x%x)\n", acts->msg,
+ status & acts->mask);
+ mask |= acts->mask;
+ }
+ status &= mask;
+ if (status) /* clear processed interrupts */
+ t4_write_reg(adapter, reg, status);
+ return fatal;
+}
+
+/*
+ * Interrupt handler for the PCIE module.
+ */
+static void pcie_intr_handler(struct adapter *adapter)
+{
+ static struct intr_info sysbus_intr_info[] = {
+ { RNPP, "RXNP array parity error", -1, 1 },
+ { RPCP, "RXPC array parity error", -1, 1 },
+ { RCIP, "RXCIF array parity error", -1, 1 },
+ { RCCP, "Rx completions control array parity error", -1, 1 },
+ { RFTP, "RXFT array parity error", -1, 1 },
+ { 0 }
+ };
+ static struct intr_info pcie_port_intr_info[] = {
+ { TPCP, "TXPC array parity error", -1, 1 },
+ { TNPP, "TXNP array parity error", -1, 1 },
+ { TFTP, "TXFT array parity error", -1, 1 },
+ { TCAP, "TXCA array parity error", -1, 1 },
+ { TCIP, "TXCIF array parity error", -1, 1 },
+ { RCAP, "RXCA array parity error", -1, 1 },
+ { OTDD, "outbound request TLP discarded", -1, 1 },
+ { RDPE, "Rx data parity error", -1, 1 },
+ { TDUE, "Tx uncorrectable data error", -1, 1 },
+ { 0 }
+ };
+ static struct intr_info pcie_intr_info[] = {
+ { MSIADDRLPERR, "MSI AddrL parity error", -1, 1 },
+ { MSIADDRHPERR, "MSI AddrH parity error", -1, 1 },
+ { MSIDATAPERR, "MSI data parity error", -1, 1 },
+ { MSIXADDRLPERR, "MSI-X AddrL parity error", -1, 1 },
+ { MSIXADDRHPERR, "MSI-X AddrH parity error", -1, 1 },
+ { MSIXDATAPERR, "MSI-X data parity error", -1, 1 },
+ { MSIXDIPERR, "MSI-X DI parity error", -1, 1 },
+ { PIOCPLPERR, "PCI PIO completion FIFO parity error", -1, 1 },
+ { PIOREQPERR, "PCI PIO request FIFO parity error", -1, 1 },
+ { TARTAGPERR, "PCI PCI target tag FIFO parity error", -1, 1 },
+ { CCNTPERR, "PCI CMD channel count parity error", -1, 1 },
+ { CREQPERR, "PCI CMD channel request parity error", -1, 1 },
+ { CRSPPERR, "PCI CMD channel response parity error", -1, 1 },
+ { DCNTPERR, "PCI DMA channel count parity error", -1, 1 },
+ { DREQPERR, "PCI DMA channel request parity error", -1, 1 },
+ { DRSPPERR, "PCI DMA channel response parity error", -1, 1 },
+ { HCNTPERR, "PCI HMA channel count parity error", -1, 1 },
+ { HREQPERR, "PCI HMA channel request parity error", -1, 1 },
+ { HRSPPERR, "PCI HMA channel response parity error", -1, 1 },
+ { CFGSNPPERR, "PCI config snoop FIFO parity error", -1, 1 },
+ { FIDPERR, "PCI FID parity error", -1, 1 },
+ { INTXCLRPERR, "PCI INTx clear parity error", -1, 1 },
+ { MATAGPERR, "PCI MA tag parity error", -1, 1 },
+ { PIOTAGPERR, "PCI PIO tag parity error", -1, 1 },
+ { RXCPLPERR, "PCI Rx completion parity error", -1, 1 },
+ { RXWRPERR, "PCI Rx write parity error", -1, 1 },
+ { RPLPERR, "PCI replay buffer parity error", -1, 1 },
+ { PCIESINT, "PCI core secondary fault", -1, 1 },
+ { PCIEPINT, "PCI core primary fault", -1, 1 },
+ { UNXSPLCPLERR, "PCI unexpected split completion error", -1, 0 },
+ { 0 }
+ };
+
+ int fat;
+
+ fat = t4_handle_intr_status(adapter,
+ PCIE_CORE_UTL_SYSTEM_BUS_AGENT_STATUS,
+ sysbus_intr_info) +
+ t4_handle_intr_status(adapter,
+ PCIE_CORE_UTL_PCI_EXPRESS_PORT_STATUS,
+ pcie_port_intr_info) +
+ t4_handle_intr_status(adapter, PCIE_INT_CAUSE, pcie_intr_info);
+ if (fat)
+ t4_fatal_err(adapter);
+}
+
+/*
+ * TP interrupt handler.
+ */
+static void tp_intr_handler(struct adapter *adapter)
+{
+ static struct intr_info tp_intr_info[] = {
+ { 0x3fffffff, "TP parity error", -1, 1 },
+ { FLMTXFLSTEMPTY, "TP out of Tx pages", -1, 1 },
+ { 0 }
+ };
+
+ if (t4_handle_intr_status(adapter, TP_INT_CAUSE, tp_intr_info))
+ t4_fatal_err(adapter);
+}
+
+/*
+ * SGE interrupt handler.
+ */
+static void sge_intr_handler(struct adapter *adapter)
+{
+ u64 v;
+
+ static struct intr_info sge_intr_info[] = {
+ { ERR_CPL_EXCEED_IQE_SIZE,
+ "SGE received CPL exceeding IQE size", -1, 1 },
+ { ERR_INVALID_CIDX_INC,
+ "SGE GTS CIDX increment too large", -1, 0 },
+ { ERR_CPL_OPCODE_0, "SGE received 0-length CPL", -1, 0 },
+ { ERR_DROPPED_DB, "SGE doorbell dropped", -1, 0 },
+ { ERR_DATA_CPL_ON_HIGH_QID1 | ERR_DATA_CPL_ON_HIGH_QID0,
+ "SGE IQID > 1023 received CPL for FL", -1, 0 },
+ { ERR_BAD_DB_PIDX3, "SGE DBP 3 pidx increment too large", -1,
+ 0 },
+ { ERR_BAD_DB_PIDX2, "SGE DBP 2 pidx increment too large", -1,
+ 0 },
+ { ERR_BAD_DB_PIDX1, "SGE DBP 1 pidx increment too large", -1,
+ 0 },
+ { ERR_BAD_DB_PIDX0, "SGE DBP 0 pidx increment too large", -1,
+ 0 },
+ { ERR_ING_CTXT_PRIO,
+ "SGE too many priority ingress contexts", -1, 0 },
+ { ERR_EGR_CTXT_PRIO,
+ "SGE too many priority egress contexts", -1, 0 },
+ { INGRESS_SIZE_ERR, "SGE illegal ingress QID", -1, 0 },
+ { EGRESS_SIZE_ERR, "SGE illegal egress QID", -1, 0 },
+ { 0 }
+ };
+
+ v = (u64)t4_read_reg(adapter, SGE_INT_CAUSE1) |
+ ((u64)t4_read_reg(adapter, SGE_INT_CAUSE2) << 32);
+ if (v) {
+ dev_alert(adapter->pdev_dev, "SGE parity error (%#llx)\n",
+ (unsigned long long)v);
+ t4_write_reg(adapter, SGE_INT_CAUSE1, v);
+ t4_write_reg(adapter, SGE_INT_CAUSE2, v >> 32);
+ }
+
+ if (t4_handle_intr_status(adapter, SGE_INT_CAUSE3, sge_intr_info) ||
+ v != 0)
+ t4_fatal_err(adapter);
+}
+
+/*
+ * CIM interrupt handler.
+ */
+static void cim_intr_handler(struct adapter *adapter)
+{
+ static struct intr_info cim_intr_info[] = {
+ { PREFDROPINT, "CIM control register prefetch drop", -1, 1 },
+ { OBQPARERR, "CIM OBQ parity error", -1, 1 },
+ { IBQPARERR, "CIM IBQ parity error", -1, 1 },
+ { MBUPPARERR, "CIM mailbox uP parity error", -1, 1 },
+ { MBHOSTPARERR, "CIM mailbox host parity error", -1, 1 },
+ { TIEQINPARERRINT, "CIM TIEQ outgoing parity error", -1, 1 },
+ { TIEQOUTPARERRINT, "CIM TIEQ incoming parity error", -1, 1 },
+ { 0 }
+ };
+ static struct intr_info cim_upintr_info[] = {
+ { RSVDSPACEINT, "CIM reserved space access", -1, 1 },
+ { ILLTRANSINT, "CIM illegal transaction", -1, 1 },
+ { ILLWRINT, "CIM illegal write", -1, 1 },
+ { ILLRDINT, "CIM illegal read", -1, 1 },
+ { ILLRDBEINT, "CIM illegal read BE", -1, 1 },
+ { ILLWRBEINT, "CIM illegal write BE", -1, 1 },
+ { SGLRDBOOTINT, "CIM single read from boot space", -1, 1 },
+ { SGLWRBOOTINT, "CIM single write to boot space", -1, 1 },
+ { BLKWRBOOTINT, "CIM block write to boot space", -1, 1 },
+ { SGLRDFLASHINT, "CIM single read from flash space", -1, 1 },
+ { SGLWRFLASHINT, "CIM single write to flash space", -1, 1 },
+ { BLKWRFLASHINT, "CIM block write to flash space", -1, 1 },
+ { SGLRDEEPROMINT, "CIM single EEPROM read", -1, 1 },
+ { SGLWREEPROMINT, "CIM single EEPROM write", -1, 1 },
+ { BLKRDEEPROMINT, "CIM block EEPROM read", -1, 1 },
+ { BLKWREEPROMINT, "CIM block EEPROM write", -1, 1 },
+ { SGLRDCTLINT , "CIM single read from CTL space", -1, 1 },
+ { SGLWRCTLINT , "CIM single write to CTL space", -1, 1 },
+ { BLKRDCTLINT , "CIM block read from CTL space", -1, 1 },
+ { BLKWRCTLINT , "CIM block write to CTL space", -1, 1 },
+ { SGLRDPLINT , "CIM single read from PL space", -1, 1 },
+ { SGLWRPLINT , "CIM single write to PL space", -1, 1 },
+ { BLKRDPLINT , "CIM block read from PL space", -1, 1 },
+ { BLKWRPLINT , "CIM block write to PL space", -1, 1 },
+ { REQOVRLOOKUPINT , "CIM request FIFO overwrite", -1, 1 },
+ { RSPOVRLOOKUPINT , "CIM response FIFO overwrite", -1, 1 },
+ { TIMEOUTINT , "CIM PIF timeout", -1, 1 },
+ { TIMEOUTMAINT , "CIM PIF MA timeout", -1, 1 },
+ { 0 }
+ };
+
+ int fat;
+
+ fat = t4_handle_intr_status(adapter, CIM_HOST_INT_CAUSE,
+ cim_intr_info) +
+ t4_handle_intr_status(adapter, CIM_HOST_UPACC_INT_CAUSE,
+ cim_upintr_info);
+ if (fat)
+ t4_fatal_err(adapter);
+}
+
+/*
+ * ULP RX interrupt handler.
+ */
+static void ulprx_intr_handler(struct adapter *adapter)
+{
+ static struct intr_info ulprx_intr_info[] = {
+ { 0x7fffff, "ULPRX parity error", -1, 1 },
+ { 0 }
+ };
+
+ if (t4_handle_intr_status(adapter, ULP_RX_INT_CAUSE, ulprx_intr_info))
+ t4_fatal_err(adapter);
+}
+
+/*
+ * ULP TX interrupt handler.
+ */
+static void ulptx_intr_handler(struct adapter *adapter)
+{
+ static struct intr_info ulptx_intr_info[] = {
+ { PBL_BOUND_ERR_CH3, "ULPTX channel 3 PBL out of bounds", -1,
+ 0 },
+ { PBL_BOUND_ERR_CH2, "ULPTX channel 2 PBL out of bounds", -1,
+ 0 },
+ { PBL_BOUND_ERR_CH1, "ULPTX channel 1 PBL out of bounds", -1,
+ 0 },
+ { PBL_BOUND_ERR_CH0, "ULPTX channel 0 PBL out of bounds", -1,
+ 0 },
+ { 0xfffffff, "ULPTX parity error", -1, 1 },
+ { 0 }
+ };
+
+ if (t4_handle_intr_status(adapter, ULP_TX_INT_CAUSE, ulptx_intr_info))
+ t4_fatal_err(adapter);
+}
+
+/*
+ * PM TX interrupt handler.
+ */
+static void pmtx_intr_handler(struct adapter *adapter)
+{
+ static struct intr_info pmtx_intr_info[] = {
+ { PCMD_LEN_OVFL0, "PMTX channel 0 pcmd too large", -1, 1 },
+ { PCMD_LEN_OVFL1, "PMTX channel 1 pcmd too large", -1, 1 },
+ { PCMD_LEN_OVFL2, "PMTX channel 2 pcmd too large", -1, 1 },
+ { ZERO_C_CMD_ERROR, "PMTX 0-length pcmd", -1, 1 },
+ { PMTX_FRAMING_ERROR, "PMTX framing error", -1, 1 },
+ { OESPI_PAR_ERROR, "PMTX oespi parity error", -1, 1 },
+ { DB_OPTIONS_PAR_ERROR, "PMTX db_options parity error", -1, 1 },
+ { ICSPI_PAR_ERROR, "PMTX icspi parity error", -1, 1 },
+ { C_PCMD_PAR_ERROR, "PMTX c_pcmd parity error", -1, 1},
+ { 0 }
+ };
+
+ if (t4_handle_intr_status(adapter, PM_TX_INT_CAUSE, pmtx_intr_info))
+ t4_fatal_err(adapter);
+}
+
+/*
+ * PM RX interrupt handler.
+ */
+static void pmrx_intr_handler(struct adapter *adapter)
+{
+ static struct intr_info pmrx_intr_info[] = {
+ { ZERO_E_CMD_ERROR, "PMRX 0-length pcmd", -1, 1 },
+ { PMRX_FRAMING_ERROR, "PMRX framing error", -1, 1 },
+ { OCSPI_PAR_ERROR, "PMRX ocspi parity error", -1, 1 },
+ { DB_OPTIONS_PAR_ERROR, "PMRX db_options parity error", -1, 1 },
+ { IESPI_PAR_ERROR, "PMRX iespi parity error", -1, 1 },
+ { E_PCMD_PAR_ERROR, "PMRX e_pcmd parity error", -1, 1},
+ { 0 }
+ };
+
+ if (t4_handle_intr_status(adapter, PM_RX_INT_CAUSE, pmrx_intr_info))
+ t4_fatal_err(adapter);
+}
+
+/*
+ * CPL switch interrupt handler.
+ */
+static void cplsw_intr_handler(struct adapter *adapter)
+{
+ static struct intr_info cplsw_intr_info[] = {
+ { CIM_OP_MAP_PERR, "CPLSW CIM op_map parity error", -1, 1 },
+ { CIM_OVFL_ERROR, "CPLSW CIM overflow", -1, 1 },
+ { TP_FRAMING_ERROR, "CPLSW TP framing error", -1, 1 },
+ { SGE_FRAMING_ERROR, "CPLSW SGE framing error", -1, 1 },
+ { CIM_FRAMING_ERROR, "CPLSW CIM framing error", -1, 1 },
+ { ZERO_SWITCH_ERROR, "CPLSW no-switch error", -1, 1 },
+ { 0 }
+ };
+
+ if (t4_handle_intr_status(adapter, CPL_INTR_CAUSE, cplsw_intr_info))
+ t4_fatal_err(adapter);
+}
+
+/*
+ * LE interrupt handler.
+ */
+static void le_intr_handler(struct adapter *adap)
+{
+ static struct intr_info le_intr_info[] = {
+ { LIPMISS, "LE LIP miss", -1, 0 },
+ { LIP0, "LE 0 LIP error", -1, 0 },
+ { PARITYERR, "LE parity error", -1, 1 },
+ { UNKNOWNCMD, "LE unknown command", -1, 1 },
+ { REQQPARERR, "LE request queue parity error", -1, 1 },
+ { 0 }
+ };
+
+ if (t4_handle_intr_status(adap, LE_DB_INT_CAUSE, le_intr_info))
+ t4_fatal_err(adap);
+}
+
+/*
+ * MPS interrupt handler.
+ */
+static void mps_intr_handler(struct adapter *adapter)
+{
+ static struct intr_info mps_rx_intr_info[] = {
+ { 0xffffff, "MPS Rx parity error", -1, 1 },
+ { 0 }
+ };
+ static struct intr_info mps_tx_intr_info[] = {
+ { TPFIFO, "MPS Tx TP FIFO parity error", -1, 1 },
+ { NCSIFIFO, "MPS Tx NC-SI FIFO parity error", -1, 1 },
+ { TXDATAFIFO, "MPS Tx data FIFO parity error", -1, 1 },
+ { TXDESCFIFO, "MPS Tx desc FIFO parity error", -1, 1 },
+ { BUBBLE, "MPS Tx underflow", -1, 1 },
+ { SECNTERR, "MPS Tx SOP/EOP error", -1, 1 },
+ { FRMERR, "MPS Tx framing error", -1, 1 },
+ { 0 }
+ };
+ static struct intr_info mps_trc_intr_info[] = {
+ { FILTMEM, "MPS TRC filter parity error", -1, 1 },
+ { PKTFIFO, "MPS TRC packet FIFO parity error", -1, 1 },
+ { MISCPERR, "MPS TRC misc parity error", -1, 1 },
+ { 0 }
+ };
+ static struct intr_info mps_stat_sram_intr_info[] = {
+ { 0x1fffff, "MPS statistics SRAM parity error", -1, 1 },
+ { 0 }
+ };
+ static struct intr_info mps_stat_tx_intr_info[] = {
+ { 0xfffff, "MPS statistics Tx FIFO parity error", -1, 1 },
+ { 0 }
+ };
+ static struct intr_info mps_stat_rx_intr_info[] = {
+ { 0xffffff, "MPS statistics Rx FIFO parity error", -1, 1 },
+ { 0 }
+ };
+ static struct intr_info mps_cls_intr_info[] = {
+ { MATCHSRAM, "MPS match SRAM parity error", -1, 1 },
+ { MATCHTCAM, "MPS match TCAM parity error", -1, 1 },
+ { HASHSRAM, "MPS hash SRAM parity error", -1, 1 },
+ { 0 }
+ };
+
+ int fat;
+
+ fat = t4_handle_intr_status(adapter, MPS_RX_PERR_INT_CAUSE,
+ mps_rx_intr_info) +
+ t4_handle_intr_status(adapter, MPS_TX_INT_CAUSE,
+ mps_tx_intr_info) +
+ t4_handle_intr_status(adapter, MPS_TRC_INT_CAUSE,
+ mps_trc_intr_info) +
+ t4_handle_intr_status(adapter, MPS_STAT_PERR_INT_CAUSE_SRAM,
+ mps_stat_sram_intr_info) +
+ t4_handle_intr_status(adapter, MPS_STAT_PERR_INT_CAUSE_TX_FIFO,
+ mps_stat_tx_intr_info) +
+ t4_handle_intr_status(adapter, MPS_STAT_PERR_INT_CAUSE_RX_FIFO,
+ mps_stat_rx_intr_info) +
+ t4_handle_intr_status(adapter, MPS_CLS_INT_CAUSE,
+ mps_cls_intr_info);
+
+ t4_write_reg(adapter, MPS_INT_CAUSE, CLSINT | TRCINT |
+ RXINT | TXINT | STATINT);
+ t4_read_reg(adapter, MPS_INT_CAUSE); /* flush */
+ if (fat)
+ t4_fatal_err(adapter);
+}
+
+#define MEM_INT_MASK (PERR_INT_CAUSE | ECC_CE_INT_CAUSE | ECC_UE_INT_CAUSE)
+
+/*
+ * EDC/MC interrupt handler.
+ */
+static void mem_intr_handler(struct adapter *adapter, int idx)
+{
+ static const char name[3][5] = { "EDC0", "EDC1", "MC" };
+
+ unsigned int addr, cnt_addr, v;
+
+ if (idx <= MEM_EDC1) {
+ addr = EDC_REG(EDC_INT_CAUSE, idx);
+ cnt_addr = EDC_REG(EDC_ECC_STATUS, idx);
+ } else {
+ addr = MC_INT_CAUSE;
+ cnt_addr = MC_ECC_STATUS;
+ }
+
+ v = t4_read_reg(adapter, addr) & MEM_INT_MASK;
+ if (v & PERR_INT_CAUSE)
+ dev_alert(adapter->pdev_dev, "%s FIFO parity error\n",
+ name[idx]);
+ if (v & ECC_CE_INT_CAUSE) {
+ u32 cnt = ECC_CECNT_GET(t4_read_reg(adapter, cnt_addr));
+
+ t4_write_reg(adapter, cnt_addr, ECC_CECNT_MASK);
+ if (printk_ratelimit())
+ dev_warn(adapter->pdev_dev,
+ "%u %s correctable ECC data error%s\n",
+ cnt, name[idx], cnt > 1 ? "s" : "");
+ }
+ if (v & ECC_UE_INT_CAUSE)
+ dev_alert(adapter->pdev_dev,
+ "%s uncorrectable ECC data error\n", name[idx]);
+
+ t4_write_reg(adapter, addr, v);
+ if (v & (PERR_INT_CAUSE | ECC_UE_INT_CAUSE))
+ t4_fatal_err(adapter);
+}
+
+/*
+ * MA interrupt handler.
+ */
+static void ma_intr_handler(struct adapter *adap)
+{
+ u32 v, status = t4_read_reg(adap, MA_INT_CAUSE);
+
+ if (status & MEM_PERR_INT_CAUSE)
+ dev_alert(adap->pdev_dev,
+ "MA parity error, parity status %#x\n",
+ t4_read_reg(adap, MA_PARITY_ERROR_STATUS));
+ if (status & MEM_WRAP_INT_CAUSE) {
+ v = t4_read_reg(adap, MA_INT_WRAP_STATUS);
+ dev_alert(adap->pdev_dev, "MA address wrap-around error by "
+ "client %u to address %#x\n",
+ MEM_WRAP_CLIENT_NUM_GET(v),
+ MEM_WRAP_ADDRESS_GET(v) << 4);
+ }
+ t4_write_reg(adap, MA_INT_CAUSE, status);
+ t4_fatal_err(adap);
+}
+
+/*
+ * SMB interrupt handler.
+ */
+static void smb_intr_handler(struct adapter *adap)
+{
+ static struct intr_info smb_intr_info[] = {
+ { MSTTXFIFOPARINT, "SMB master Tx FIFO parity error", -1, 1 },
+ { MSTRXFIFOPARINT, "SMB master Rx FIFO parity error", -1, 1 },
+ { SLVFIFOPARINT, "SMB slave FIFO parity error", -1, 1 },
+ { 0 }
+ };
+
+ if (t4_handle_intr_status(adap, SMB_INT_CAUSE, smb_intr_info))
+ t4_fatal_err(adap);
+}
+
+/*
+ * NC-SI interrupt handler.
+ */
+static void ncsi_intr_handler(struct adapter *adap)
+{
+ static struct intr_info ncsi_intr_info[] = {
+ { CIM_DM_PRTY_ERR, "NC-SI CIM parity error", -1, 1 },
+ { MPS_DM_PRTY_ERR, "NC-SI MPS parity error", -1, 1 },
+ { TXFIFO_PRTY_ERR, "NC-SI Tx FIFO parity error", -1, 1 },
+ { RXFIFO_PRTY_ERR, "NC-SI Rx FIFO parity error", -1, 1 },
+ { 0 }
+ };
+
+ if (t4_handle_intr_status(adap, NCSI_INT_CAUSE, ncsi_intr_info))
+ t4_fatal_err(adap);
+}
+
+/*
+ * XGMAC interrupt handler.
+ */
+static void xgmac_intr_handler(struct adapter *adap, int port)
+{
+ u32 v = t4_read_reg(adap, PORT_REG(port, XGMAC_PORT_INT_CAUSE));
+
+ v &= TXFIFO_PRTY_ERR | RXFIFO_PRTY_ERR;
+ if (!v)
+ return;
+
+ if (v & TXFIFO_PRTY_ERR)
+ dev_alert(adap->pdev_dev, "XGMAC %d Tx FIFO parity error\n",
+ port);
+ if (v & RXFIFO_PRTY_ERR)
+ dev_alert(adap->pdev_dev, "XGMAC %d Rx FIFO parity error\n",
+ port);
+ t4_write_reg(adap, PORT_REG(port, XGMAC_PORT_INT_CAUSE), v);
+ t4_fatal_err(adap);
+}
+
+/*
+ * PL interrupt handler.
+ */
+static void pl_intr_handler(struct adapter *adap)
+{
+ static struct intr_info pl_intr_info[] = {
+ { FATALPERR, "T4 fatal parity error", -1, 1 },
+ { PERRVFID, "PL VFID_MAP parity error", -1, 1 },
+ { 0 }
+ };
+
+ if (t4_handle_intr_status(adap, PL_PL_INT_CAUSE, pl_intr_info))
+ t4_fatal_err(adap);
+}
+
+#define PF_INTR_MASK (PFSW | PFCIM)
+#define GLBL_INTR_MASK (CIM | MPS | PL | PCIE | MC | EDC0 | \
+ EDC1 | LE | TP | MA | PM_TX | PM_RX | ULP_RX | \
+ CPL_SWITCH | SGE | ULP_TX)
+
+/**
+ * t4_slow_intr_handler - control path interrupt handler
+ * @adapter: the adapter
+ *
+ * T4 interrupt handler for non-data global interrupt events, e.g., errors.
+ * The designation 'slow' is because it involves register reads, while
+ * data interrupts typically don't involve any MMIOs.
+ */
+int t4_slow_intr_handler(struct adapter *adapter)
+{
+ u32 cause = t4_read_reg(adapter, PL_INT_CAUSE);
+
+ if (!(cause & GLBL_INTR_MASK))
+ return 0;
+ if (cause & CIM)
+ cim_intr_handler(adapter);
+ if (cause & MPS)
+ mps_intr_handler(adapter);
+ if (cause & NCSI)
+ ncsi_intr_handler(adapter);
+ if (cause & PL)
+ pl_intr_handler(adapter);
+ if (cause & SMB)
+ smb_intr_handler(adapter);
+ if (cause & XGMAC0)
+ xgmac_intr_handler(adapter, 0);
+ if (cause & XGMAC1)
+ xgmac_intr_handler(adapter, 1);
+ if (cause & XGMAC_KR0)
+ xgmac_intr_handler(adapter, 2);
+ if (cause & XGMAC_KR1)
+ xgmac_intr_handler(adapter, 3);
+ if (cause & PCIE)
+ pcie_intr_handler(adapter);
+ if (cause & MC)
+ mem_intr_handler(adapter, MEM_MC);
+ if (cause & EDC0)
+ mem_intr_handler(adapter, MEM_EDC0);
+ if (cause & EDC1)
+ mem_intr_handler(adapter, MEM_EDC1);
+ if (cause & LE)
+ le_intr_handler(adapter);
+ if (cause & TP)
+ tp_intr_handler(adapter);
+ if (cause & MA)
+ ma_intr_handler(adapter);
+ if (cause & PM_TX)
+ pmtx_intr_handler(adapter);
+ if (cause & PM_RX)
+ pmrx_intr_handler(adapter);
+ if (cause & ULP_RX)
+ ulprx_intr_handler(adapter);
+ if (cause & CPL_SWITCH)
+ cplsw_intr_handler(adapter);
+ if (cause & SGE)
+ sge_intr_handler(adapter);
+ if (cause & ULP_TX)
+ ulptx_intr_handler(adapter);
+
+ /* Clear the interrupts just processed for which we are the master. */
+ t4_write_reg(adapter, PL_INT_CAUSE, cause & GLBL_INTR_MASK);
+ (void) t4_read_reg(adapter, PL_INT_CAUSE); /* flush */
+ return 1;
+}
+
+/**
+ * t4_intr_enable - enable interrupts
+ * @adapter: the adapter whose interrupts should be enabled
+ *
+ * Enable PF-specific interrupts for the calling function and the top-level
+ * interrupt concentrator for global interrupts. Interrupts are already
+ * enabled at each module, here we just enable the roots of the interrupt
+ * hierarchies.
+ *
+ * Note: this function should be called only when the driver manages
+ * non PF-specific interrupts from the various HW modules. Only one PCI
+ * function at a time should be doing this.
+ */
+void t4_intr_enable(struct adapter *adapter)
+{
+ u32 pf = SOURCEPF_GET(t4_read_reg(adapter, PL_WHOAMI));
+
+ t4_write_reg(adapter, SGE_INT_ENABLE3, ERR_CPL_EXCEED_IQE_SIZE |
+ ERR_INVALID_CIDX_INC | ERR_CPL_OPCODE_0 |
+ ERR_DROPPED_DB | ERR_DATA_CPL_ON_HIGH_QID1 |
+ ERR_DATA_CPL_ON_HIGH_QID0 | ERR_BAD_DB_PIDX3 |
+ ERR_BAD_DB_PIDX2 | ERR_BAD_DB_PIDX1 |
+ ERR_BAD_DB_PIDX0 | ERR_ING_CTXT_PRIO |
+ ERR_EGR_CTXT_PRIO | INGRESS_SIZE_ERR |
+ EGRESS_SIZE_ERR);
+ t4_write_reg(adapter, MYPF_REG(PL_PF_INT_ENABLE), PF_INTR_MASK);
+ t4_set_reg_field(adapter, PL_INT_MAP0, 0, 1 << pf);
+}
+
+/**
+ * t4_intr_disable - disable interrupts
+ * @adapter: the adapter whose interrupts should be disabled
+ *
+ * Disable interrupts. We only disable the top-level interrupt
+ * concentrators. The caller must be a PCI function managing global
+ * interrupts.
+ */
+void t4_intr_disable(struct adapter *adapter)
+{
+ u32 pf = SOURCEPF_GET(t4_read_reg(adapter, PL_WHOAMI));
+
+ t4_write_reg(adapter, MYPF_REG(PL_PF_INT_ENABLE), 0);
+ t4_set_reg_field(adapter, PL_INT_MAP0, 1 << pf, 0);
+}
+
+/**
+ * t4_intr_clear - clear all interrupts
+ * @adapter: the adapter whose interrupts should be cleared
+ *
+ * Clears all interrupts. The caller must be a PCI function managing
+ * global interrupts.
+ */
+void t4_intr_clear(struct adapter *adapter)
+{
+ static const unsigned int cause_reg[] = {
+ SGE_INT_CAUSE1, SGE_INT_CAUSE2, SGE_INT_CAUSE3,
+ PCIE_CORE_UTL_SYSTEM_BUS_AGENT_STATUS,
+ PCIE_CORE_UTL_PCI_EXPRESS_PORT_STATUS,
+ PCIE_NONFAT_ERR, PCIE_INT_CAUSE,
+ MC_INT_CAUSE,
+ MA_INT_WRAP_STATUS, MA_PARITY_ERROR_STATUS, MA_INT_CAUSE,
+ EDC_INT_CAUSE, EDC_REG(EDC_INT_CAUSE, 1),
+ CIM_HOST_INT_CAUSE, CIM_HOST_UPACC_INT_CAUSE,
+ MYPF_REG(CIM_PF_HOST_INT_CAUSE),
+ TP_INT_CAUSE,
+ ULP_RX_INT_CAUSE, ULP_TX_INT_CAUSE,
+ PM_RX_INT_CAUSE, PM_TX_INT_CAUSE,
+ MPS_RX_PERR_INT_CAUSE,
+ CPL_INTR_CAUSE,
+ MYPF_REG(PL_PF_INT_CAUSE),
+ PL_PL_INT_CAUSE,
+ LE_DB_INT_CAUSE,
+ };
+
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(cause_reg); ++i)
+ t4_write_reg(adapter, cause_reg[i], 0xffffffff);
+
+ t4_write_reg(adapter, PL_INT_CAUSE, GLBL_INTR_MASK);
+ (void) t4_read_reg(adapter, PL_INT_CAUSE); /* flush */
+}
+
+/**
+ * hash_mac_addr - return the hash value of a MAC address
+ * @addr: the 48-bit Ethernet MAC address
+ *
+ * Hashes a MAC address according to the hash function used by HW inexact
+ * (hash) address matching.
+ */
+static int hash_mac_addr(const u8 *addr)
+{
+ u32 a = ((u32)addr[0] << 16) | ((u32)addr[1] << 8) | addr[2];
+ u32 b = ((u32)addr[3] << 16) | ((u32)addr[4] << 8) | addr[5];
+ a ^= b;
+ a ^= (a >> 12);
+ a ^= (a >> 6);
+ return a & 0x3f;
+}
+
+/**
+ * t4_config_rss_range - configure a portion of the RSS mapping table
+ * @adapter: the adapter
+ * @mbox: mbox to use for the FW command
+ * @viid: virtual interface whose RSS subtable is to be written
+ * @start: start entry in the table to write
+ * @n: how many table entries to write
+ * @rspq: values for the response queue lookup table
+ * @nrspq: number of values in @rspq
+ *
+ * Programs the selected part of the VI's RSS mapping table with the
+ * provided values. If @nrspq < @n the supplied values are used repeatedly
+ * until the full table range is populated.
+ *
+ * The caller must ensure the values in @rspq are in the range allowed for
+ * @viid.
+ */
+int t4_config_rss_range(struct adapter *adapter, int mbox, unsigned int viid,
+ int start, int n, const u16 *rspq, unsigned int nrspq)
+{
+ int ret;
+ const u16 *rsp = rspq;
+ const u16 *rsp_end = rspq + nrspq;
+ struct fw_rss_ind_tbl_cmd cmd;
+
+ memset(&cmd, 0, sizeof(cmd));
+ cmd.op_to_viid = htonl(FW_CMD_OP(FW_RSS_IND_TBL_CMD) |
+ FW_CMD_REQUEST | FW_CMD_WRITE |
+ FW_RSS_IND_TBL_CMD_VIID(viid));
+ cmd.retval_len16 = htonl(FW_LEN16(cmd));
+
+ /* each fw_rss_ind_tbl_cmd takes up to 32 entries */
+ while (n > 0) {
+ int nq = min(n, 32);
+ __be32 *qp = &cmd.iq0_to_iq2;
+
+ cmd.niqid = htons(nq);
+ cmd.startidx = htons(start);
+
+ start += nq;
+ n -= nq;
+
+ while (nq > 0) {
+ unsigned int v;
+
+ v = FW_RSS_IND_TBL_CMD_IQ0(*rsp);
+ if (++rsp >= rsp_end)
+ rsp = rspq;
+ v |= FW_RSS_IND_TBL_CMD_IQ1(*rsp);
+ if (++rsp >= rsp_end)
+ rsp = rspq;
+ v |= FW_RSS_IND_TBL_CMD_IQ2(*rsp);
+ if (++rsp >= rsp_end)
+ rsp = rspq;
+
+ *qp++ = htonl(v);
+ nq -= 3;
+ }
+
+ ret = t4_wr_mbox(adapter, mbox, &cmd, sizeof(cmd), NULL);
+ if (ret)
+ return ret;
+ }
+ return 0;
+}
+
+/**
+ * t4_config_glbl_rss - configure the global RSS mode
+ * @adapter: the adapter
+ * @mbox: mbox to use for the FW command
+ * @mode: global RSS mode
+ * @flags: mode-specific flags
+ *
+ * Sets the global RSS mode.
+ */
+int t4_config_glbl_rss(struct adapter *adapter, int mbox, unsigned int mode,
+ unsigned int flags)
+{
+ struct fw_rss_glb_config_cmd c;
+
+ memset(&c, 0, sizeof(c));
+ c.op_to_write = htonl(FW_CMD_OP(FW_RSS_GLB_CONFIG_CMD) |
+ FW_CMD_REQUEST | FW_CMD_WRITE);
+ c.retval_len16 = htonl(FW_LEN16(c));
+ if (mode == FW_RSS_GLB_CONFIG_CMD_MODE_MANUAL) {
+ c.u.manual.mode_pkd = htonl(FW_RSS_GLB_CONFIG_CMD_MODE(mode));
+ } else if (mode == FW_RSS_GLB_CONFIG_CMD_MODE_BASICVIRTUAL) {
+ c.u.basicvirtual.mode_pkd =
+ htonl(FW_RSS_GLB_CONFIG_CMD_MODE(mode));
+ c.u.basicvirtual.synmapen_to_hashtoeplitz = htonl(flags);
+ } else
+ return -EINVAL;
+ return t4_wr_mbox(adapter, mbox, &c, sizeof(c), NULL);
+}
+
+/* Read an RSS table row */
+static int rd_rss_row(struct adapter *adap, int row, u32 *val)
+{
+ t4_write_reg(adap, TP_RSS_LKP_TABLE, 0xfff00000 | row);
+ return t4_wait_op_done_val(adap, TP_RSS_LKP_TABLE, LKPTBLROWVLD, 1,
+ 5, 0, val);
+}
+
+/**
+ * t4_read_rss - read the contents of the RSS mapping table
+ * @adapter: the adapter
+ * @map: holds the contents of the RSS mapping table
+ *
+ * Reads the contents of the RSS hash->queue mapping table.
+ */
+int t4_read_rss(struct adapter *adapter, u16 *map)
+{
+ u32 val;
+ int i, ret;
+
+ for (i = 0; i < RSS_NENTRIES / 2; ++i) {
+ ret = rd_rss_row(adapter, i, &val);
+ if (ret)
+ return ret;
+ *map++ = LKPTBLQUEUE0_GET(val);
+ *map++ = LKPTBLQUEUE1_GET(val);
+ }
+ return 0;
+}
+
+/**
+ * t4_tp_get_tcp_stats - read TP's TCP MIB counters
+ * @adap: the adapter
+ * @v4: holds the TCP/IP counter values
+ * @v6: holds the TCP/IPv6 counter values
+ *
+ * Returns the values of TP's TCP/IP and TCP/IPv6 MIB counters.
+ * Either @v4 or @v6 may be %NULL to skip the corresponding stats.
+ */
+void t4_tp_get_tcp_stats(struct adapter *adap, struct tp_tcp_stats *v4,
+ struct tp_tcp_stats *v6)
+{
+ u32 val[TP_MIB_TCP_RXT_SEG_LO - TP_MIB_TCP_OUT_RST + 1];
+
+#define STAT_IDX(x) ((TP_MIB_TCP_##x) - TP_MIB_TCP_OUT_RST)
+#define STAT(x) val[STAT_IDX(x)]
+#define STAT64(x) (((u64)STAT(x##_HI) << 32) | STAT(x##_LO))
+
+ if (v4) {
+ t4_read_indirect(adap, TP_MIB_INDEX, TP_MIB_DATA, val,
+ ARRAY_SIZE(val), TP_MIB_TCP_OUT_RST);
+ v4->tcpOutRsts = STAT(OUT_RST);
+ v4->tcpInSegs = STAT64(IN_SEG);
+ v4->tcpOutSegs = STAT64(OUT_SEG);
+ v4->tcpRetransSegs = STAT64(RXT_SEG);
+ }
+ if (v6) {
+ t4_read_indirect(adap, TP_MIB_INDEX, TP_MIB_DATA, val,
+ ARRAY_SIZE(val), TP_MIB_TCP_V6OUT_RST);
+ v6->tcpOutRsts = STAT(OUT_RST);
+ v6->tcpInSegs = STAT64(IN_SEG);
+ v6->tcpOutSegs = STAT64(OUT_SEG);
+ v6->tcpRetransSegs = STAT64(RXT_SEG);
+ }
+#undef STAT64
+#undef STAT
+#undef STAT_IDX
+}
+
+/**
+ * t4_tp_get_err_stats - read TP's error MIB counters
+ * @adap: the adapter
+ * @st: holds the counter values
+ *
+ * Returns the values of TP's error counters.
+ */
+void t4_tp_get_err_stats(struct adapter *adap, struct tp_err_stats *st)
+{
+ t4_read_indirect(adap, TP_MIB_INDEX, TP_MIB_DATA, st->macInErrs,
+ 12, TP_MIB_MAC_IN_ERR_0);
+ t4_read_indirect(adap, TP_MIB_INDEX, TP_MIB_DATA, st->tnlCongDrops,
+ 8, TP_MIB_TNL_CNG_DROP_0);
+ t4_read_indirect(adap, TP_MIB_INDEX, TP_MIB_DATA, st->tnlTxDrops,
+ 4, TP_MIB_TNL_DROP_0);
+ t4_read_indirect(adap, TP_MIB_INDEX, TP_MIB_DATA, st->ofldVlanDrops,
+ 4, TP_MIB_OFD_VLN_DROP_0);
+ t4_read_indirect(adap, TP_MIB_INDEX, TP_MIB_DATA, st->tcp6InErrs,
+ 4, TP_MIB_TCP_V6IN_ERR_0);
+ t4_read_indirect(adap, TP_MIB_INDEX, TP_MIB_DATA, &st->ofldNoNeigh,
+ 2, TP_MIB_OFD_ARP_DROP);
+}
+
+/**
+ * t4_read_mtu_tbl - returns the values in the HW path MTU table
+ * @adap: the adapter
+ * @mtus: where to store the MTU values
+ * @mtu_log: where to store the MTU base-2 log (may be %NULL)
+ *
+ * Reads the HW path MTU table.
+ */
+void t4_read_mtu_tbl(struct adapter *adap, u16 *mtus, u8 *mtu_log)
+{
+ u32 v;
+ int i;
+
+ for (i = 0; i < NMTUS; ++i) {
+ t4_write_reg(adap, TP_MTU_TABLE,
+ MTUINDEX(0xff) | MTUVALUE(i));
+ v = t4_read_reg(adap, TP_MTU_TABLE);
+ mtus[i] = MTUVALUE_GET(v);
+ if (mtu_log)
+ mtu_log[i] = MTUWIDTH_GET(v);
+ }
+}
+
+/**
+ * init_cong_ctrl - initialize congestion control parameters
+ * @a: the alpha values for congestion control
+ * @b: the beta values for congestion control
+ *
+ * Initialize the congestion control parameters.
+ */
+static void __devinit init_cong_ctrl(unsigned short *a, unsigned short *b)
+{
+ a[0] = a[1] = a[2] = a[3] = a[4] = a[5] = a[6] = a[7] = a[8] = 1;
+ a[9] = 2;
+ a[10] = 3;
+ a[11] = 4;
+ a[12] = 5;
+ a[13] = 6;
+ a[14] = 7;
+ a[15] = 8;
+ a[16] = 9;
+ a[17] = 10;
+ a[18] = 14;
+ a[19] = 17;
+ a[20] = 21;
+ a[21] = 25;
+ a[22] = 30;
+ a[23] = 35;
+ a[24] = 45;
+ a[25] = 60;
+ a[26] = 80;
+ a[27] = 100;
+ a[28] = 200;
+ a[29] = 300;
+ a[30] = 400;
+ a[31] = 500;
+
+ b[0] = b[1] = b[2] = b[3] = b[4] = b[5] = b[6] = b[7] = b[8] = 0;
+ b[9] = b[10] = 1;
+ b[11] = b[12] = 2;
+ b[13] = b[14] = b[15] = b[16] = 3;
+ b[17] = b[18] = b[19] = b[20] = b[21] = 4;
+ b[22] = b[23] = b[24] = b[25] = b[26] = b[27] = 5;
+ b[28] = b[29] = 6;
+ b[30] = b[31] = 7;
+}
+
+/* The minimum additive increment value for the congestion control table */
+#define CC_MIN_INCR 2U
+
+/**
+ * t4_load_mtus - write the MTU and congestion control HW tables
+ * @adap: the adapter
+ * @mtus: the values for the MTU table
+ * @alpha: the values for the congestion control alpha parameter
+ * @beta: the values for the congestion control beta parameter
+ *
+ * Write the HW MTU table with the supplied MTUs and the high-speed
+ * congestion control table with the supplied alpha, beta, and MTUs.
+ * We write the two tables together because the additive increments
+ * depend on the MTUs.
+ */
+void t4_load_mtus(struct adapter *adap, const unsigned short *mtus,
+ const unsigned short *alpha, const unsigned short *beta)
+{
+ static const unsigned int avg_pkts[NCCTRL_WIN] = {
+ 2, 6, 10, 14, 20, 28, 40, 56, 80, 112, 160, 224, 320, 448, 640,
+ 896, 1281, 1792, 2560, 3584, 5120, 7168, 10240, 14336, 20480,
+ 28672, 40960, 57344, 81920, 114688, 163840, 229376
+ };
+
+ unsigned int i, w;
+
+ for (i = 0; i < NMTUS; ++i) {
+ unsigned int mtu = mtus[i];
+ unsigned int log2 = fls(mtu);
+
+ if (!(mtu & ((1 << log2) >> 2))) /* round */
+ log2--;
+ t4_write_reg(adap, TP_MTU_TABLE, MTUINDEX(i) |
+ MTUWIDTH(log2) | MTUVALUE(mtu));
+
+ for (w = 0; w < NCCTRL_WIN; ++w) {
+ unsigned int inc;
+
+ inc = max(((mtu - 40) * alpha[w]) / avg_pkts[w],
+ CC_MIN_INCR);
+
+ t4_write_reg(adap, TP_CCTRL_TABLE, (i << 21) |
+ (w << 16) | (beta[w] << 13) | inc);
+ }
+ }
+}
+
+/**
+ * t4_set_trace_filter - configure one of the tracing filters
+ * @adap: the adapter
+ * @tp: the desired trace filter parameters
+ * @idx: which filter to configure
+ * @enable: whether to enable or disable the filter
+ *
+ * Configures one of the tracing filters available in HW. If @enable is
+ * %0 @tp is not examined and may be %NULL.
+ */
+int t4_set_trace_filter(struct adapter *adap, const struct trace_params *tp,
+ int idx, int enable)
+{
+ int i, ofst = idx * 4;
+ u32 data_reg, mask_reg, cfg;
+ u32 multitrc = TRCMULTIFILTER;
+
+ if (!enable) {
+ t4_write_reg(adap, MPS_TRC_FILTER_MATCH_CTL_A + ofst, 0);
+ goto out;
+ }
+
+ if (tp->port > 11 || tp->invert > 1 || tp->skip_len > 0x1f ||
+ tp->skip_ofst > 0x1f || tp->min_len > 0x1ff ||
+ tp->snap_len > 9600 || (idx && tp->snap_len > 256))
+ return -EINVAL;
+
+ if (tp->snap_len > 256) { /* must be tracer 0 */
+ if ((t4_read_reg(adap, MPS_TRC_FILTER_MATCH_CTL_A + 4) |
+ t4_read_reg(adap, MPS_TRC_FILTER_MATCH_CTL_A + 8) |
+ t4_read_reg(adap, MPS_TRC_FILTER_MATCH_CTL_A + 12)) & TFEN)
+ return -EINVAL; /* other tracers are enabled */
+ multitrc = 0;
+ } else if (idx) {
+ i = t4_read_reg(adap, MPS_TRC_FILTER_MATCH_CTL_B);
+ if (TFCAPTUREMAX_GET(i) > 256 &&
+ (t4_read_reg(adap, MPS_TRC_FILTER_MATCH_CTL_A) & TFEN))
+ return -EINVAL;
+ }
+
+ /* stop the tracer we'll be changing */
+ t4_write_reg(adap, MPS_TRC_FILTER_MATCH_CTL_A + ofst, 0);
+
+ /* disable tracing globally if running in the wrong single/multi mode */
+ cfg = t4_read_reg(adap, MPS_TRC_CFG);
+ if ((cfg & TRCEN) && multitrc != (cfg & TRCMULTIFILTER)) {
+ t4_write_reg(adap, MPS_TRC_CFG, cfg ^ TRCEN);
+ t4_read_reg(adap, MPS_TRC_CFG); /* flush */
+ msleep(1);
+ if (!(t4_read_reg(adap, MPS_TRC_CFG) & TRCFIFOEMPTY))
+ return -ETIMEDOUT;
+ }
+ /*
+ * At this point either the tracing is enabled and in the right mode or
+ * disabled.
+ */
+
+ idx *= (MPS_TRC_FILTER1_MATCH - MPS_TRC_FILTER0_MATCH);
+ data_reg = MPS_TRC_FILTER0_MATCH + idx;
+ mask_reg = MPS_TRC_FILTER0_DONT_CARE + idx;
+
+ for (i = 0; i < TRACE_LEN / 4; i++, data_reg += 4, mask_reg += 4) {
+ t4_write_reg(adap, data_reg, tp->data[i]);
+ t4_write_reg(adap, mask_reg, ~tp->mask[i]);
+ }
+ t4_write_reg(adap, MPS_TRC_FILTER_MATCH_CTL_B + ofst,
+ TFCAPTUREMAX(tp->snap_len) |
+ TFMINPKTSIZE(tp->min_len));
+ t4_write_reg(adap, MPS_TRC_FILTER_MATCH_CTL_A + ofst,
+ TFOFFSET(tp->skip_ofst) | TFLENGTH(tp->skip_len) |
+ TFPORT(tp->port) | TFEN |
+ (tp->invert ? TFINVERTMATCH : 0));
+
+ cfg &= ~TRCMULTIFILTER;
+ t4_write_reg(adap, MPS_TRC_CFG, cfg | TRCEN | multitrc);
+out: t4_read_reg(adap, MPS_TRC_CFG); /* flush */
+ return 0;
+}
+
+/**
+ * t4_get_trace_filter - query one of the tracing filters
+ * @adap: the adapter
+ * @tp: the current trace filter parameters
+ * @idx: which trace filter to query
+ * @enabled: non-zero if the filter is enabled
+ *
+ * Returns the current settings of one of the HW tracing filters.
+ */
+void t4_get_trace_filter(struct adapter *adap, struct trace_params *tp, int idx,
+ int *enabled)
+{
+ u32 ctla, ctlb;
+ int i, ofst = idx * 4;
+ u32 data_reg, mask_reg;
+
+ ctla = t4_read_reg(adap, MPS_TRC_FILTER_MATCH_CTL_A + ofst);
+ ctlb = t4_read_reg(adap, MPS_TRC_FILTER_MATCH_CTL_B + ofst);
+
+ *enabled = !!(ctla & TFEN);
+ tp->snap_len = TFCAPTUREMAX_GET(ctlb);
+ tp->min_len = TFMINPKTSIZE_GET(ctlb);
+ tp->skip_ofst = TFOFFSET_GET(ctla);
+ tp->skip_len = TFLENGTH_GET(ctla);
+ tp->invert = !!(ctla & TFINVERTMATCH);
+ tp->port = TFPORT_GET(ctla);
+
+ ofst = (MPS_TRC_FILTER1_MATCH - MPS_TRC_FILTER0_MATCH) * idx;
+ data_reg = MPS_TRC_FILTER0_MATCH + ofst;
+ mask_reg = MPS_TRC_FILTER0_DONT_CARE + ofst;
+
+ for (i = 0; i < TRACE_LEN / 4; i++, data_reg += 4, mask_reg += 4) {
+ tp->mask[i] = ~t4_read_reg(adap, mask_reg);
+ tp->data[i] = t4_read_reg(adap, data_reg) & tp->mask[i];
+ }
+}
+
+/**
+ * get_mps_bg_map - return the buffer groups associated with a port
+ * @adap: the adapter
+ * @idx: the port index
+ *
+ * Returns a bitmap indicating which MPS buffer groups are associated
+ * with the given port. Bit i is set if buffer group i is used by the
+ * port.
+ */
+static unsigned int get_mps_bg_map(struct adapter *adap, int idx)
+{
+ u32 n = NUMPORTS_GET(t4_read_reg(adap, MPS_CMN_CTL));
+
+ if (n == 0)
+ return idx == 0 ? 0xf : 0;
+ if (n == 1)
+ return idx < 2 ? (3 << (2 * idx)) : 0;
+ return 1 << idx;
+}
+
+/**
+ * t4_get_port_stats - collect port statistics
+ * @adap: the adapter
+ * @idx: the port index
+ * @p: the stats structure to fill
+ *
+ * Collect statistics related to the given port from HW.
+ */
+void t4_get_port_stats(struct adapter *adap, int idx, struct port_stats *p)
+{
+ u32 bgmap = get_mps_bg_map(adap, idx);
+
+#define GET_STAT(name) \
+ t4_read_reg64(adap, PORT_REG(idx, MPS_PORT_STAT_##name##_L))
+#define GET_STAT_COM(name) t4_read_reg64(adap, MPS_STAT_##name##_L)
+
+ p->tx_octets = GET_STAT(TX_PORT_BYTES);
+ p->tx_frames = GET_STAT(TX_PORT_FRAMES);
+ p->tx_bcast_frames = GET_STAT(TX_PORT_BCAST);
+ p->tx_mcast_frames = GET_STAT(TX_PORT_MCAST);
+ p->tx_ucast_frames = GET_STAT(TX_PORT_UCAST);
+ p->tx_error_frames = GET_STAT(TX_PORT_ERROR);
+ p->tx_frames_64 = GET_STAT(TX_PORT_64B);
+ p->tx_frames_65_127 = GET_STAT(TX_PORT_65B_127B);
+ p->tx_frames_128_255 = GET_STAT(TX_PORT_128B_255B);
+ p->tx_frames_256_511 = GET_STAT(TX_PORT_256B_511B);
+ p->tx_frames_512_1023 = GET_STAT(TX_PORT_512B_1023B);
+ p->tx_frames_1024_1518 = GET_STAT(TX_PORT_1024B_1518B);
+ p->tx_frames_1519_max = GET_STAT(TX_PORT_1519B_MAX);
+ p->tx_drop = GET_STAT(TX_PORT_DROP);
+ p->tx_pause = GET_STAT(TX_PORT_PAUSE);
+ p->tx_ppp0 = GET_STAT(TX_PORT_PPP0);
+ p->tx_ppp1 = GET_STAT(TX_PORT_PPP1);
+ p->tx_ppp2 = GET_STAT(TX_PORT_PPP2);
+ p->tx_ppp3 = GET_STAT(TX_PORT_PPP3);
+ p->tx_ppp4 = GET_STAT(TX_PORT_PPP4);
+ p->tx_ppp5 = GET_STAT(TX_PORT_PPP5);
+ p->tx_ppp6 = GET_STAT(TX_PORT_PPP6);
+ p->tx_ppp7 = GET_STAT(TX_PORT_PPP7);
+
+ p->rx_octets = GET_STAT(RX_PORT_BYTES);
+ p->rx_frames = GET_STAT(RX_PORT_FRAMES);
+ p->rx_bcast_frames = GET_STAT(RX_PORT_BCAST);
+ p->rx_mcast_frames = GET_STAT(RX_PORT_MCAST);
+ p->rx_ucast_frames = GET_STAT(RX_PORT_UCAST);
+ p->rx_too_long = GET_STAT(RX_PORT_MTU_ERROR);
+ p->rx_jabber = GET_STAT(RX_PORT_MTU_CRC_ERROR);
+ p->rx_fcs_err = GET_STAT(RX_PORT_CRC_ERROR);
+ p->rx_len_err = GET_STAT(RX_PORT_LEN_ERROR);
+ p->rx_symbol_err = GET_STAT(RX_PORT_SYM_ERROR);
+ p->rx_runt = GET_STAT(RX_PORT_LESS_64B);
+ p->rx_frames_64 = GET_STAT(RX_PORT_64B);
+ p->rx_frames_65_127 = GET_STAT(RX_PORT_65B_127B);
+ p->rx_frames_128_255 = GET_STAT(RX_PORT_128B_255B);
+ p->rx_frames_256_511 = GET_STAT(RX_PORT_256B_511B);
+ p->rx_frames_512_1023 = GET_STAT(RX_PORT_512B_1023B);
+ p->rx_frames_1024_1518 = GET_STAT(RX_PORT_1024B_1518B);
+ p->rx_frames_1519_max = GET_STAT(RX_PORT_1519B_MAX);
+ p->rx_pause = GET_STAT(RX_PORT_PAUSE);
+ p->rx_ppp0 = GET_STAT(RX_PORT_PPP0);
+ p->rx_ppp1 = GET_STAT(RX_PORT_PPP1);
+ p->rx_ppp2 = GET_STAT(RX_PORT_PPP2);
+ p->rx_ppp3 = GET_STAT(RX_PORT_PPP3);
+ p->rx_ppp4 = GET_STAT(RX_PORT_PPP4);
+ p->rx_ppp5 = GET_STAT(RX_PORT_PPP5);
+ p->rx_ppp6 = GET_STAT(RX_PORT_PPP6);
+ p->rx_ppp7 = GET_STAT(RX_PORT_PPP7);
+
+ p->rx_ovflow0 = (bgmap & 1) ? GET_STAT_COM(RX_BG_0_MAC_DROP_FRAME) : 0;
+ p->rx_ovflow1 = (bgmap & 2) ? GET_STAT_COM(RX_BG_1_MAC_DROP_FRAME) : 0;
+ p->rx_ovflow2 = (bgmap & 4) ? GET_STAT_COM(RX_BG_2_MAC_DROP_FRAME) : 0;
+ p->rx_ovflow3 = (bgmap & 8) ? GET_STAT_COM(RX_BG_3_MAC_DROP_FRAME) : 0;
+ p->rx_trunc0 = (bgmap & 1) ? GET_STAT_COM(RX_BG_0_MAC_TRUNC_FRAME) : 0;
+ p->rx_trunc1 = (bgmap & 2) ? GET_STAT_COM(RX_BG_1_MAC_TRUNC_FRAME) : 0;
+ p->rx_trunc2 = (bgmap & 4) ? GET_STAT_COM(RX_BG_2_MAC_TRUNC_FRAME) : 0;
+ p->rx_trunc3 = (bgmap & 8) ? GET_STAT_COM(RX_BG_3_MAC_TRUNC_FRAME) : 0;
+
+#undef GET_STAT
+#undef GET_STAT_COM
+}
+
+/**
+ * t4_get_lb_stats - collect loopback port statistics
+ * @adap: the adapter
+ * @idx: the loopback port index
+ * @p: the stats structure to fill
+ *
+ * Return HW statistics for the given loopback port.
+ */
+void t4_get_lb_stats(struct adapter *adap, int idx, struct lb_port_stats *p)
+{
+ u32 bgmap = get_mps_bg_map(adap, idx);
+
+#define GET_STAT(name) \
+ t4_read_reg64(adap, PORT_REG(idx, MPS_PORT_STAT_LB_PORT_##name##_L))
+#define GET_STAT_COM(name) t4_read_reg64(adap, MPS_STAT_##name##_L)
+
+ p->octets = GET_STAT(BYTES);
+ p->frames = GET_STAT(FRAMES);
+ p->bcast_frames = GET_STAT(BCAST);
+ p->mcast_frames = GET_STAT(MCAST);
+ p->ucast_frames = GET_STAT(UCAST);
+ p->error_frames = GET_STAT(ERROR);
+
+ p->frames_64 = GET_STAT(64B);
+ p->frames_65_127 = GET_STAT(65B_127B);
+ p->frames_128_255 = GET_STAT(128B_255B);
+ p->frames_256_511 = GET_STAT(256B_511B);
+ p->frames_512_1023 = GET_STAT(512B_1023B);
+ p->frames_1024_1518 = GET_STAT(1024B_1518B);
+ p->frames_1519_max = GET_STAT(1519B_MAX);
+ p->drop = t4_read_reg(adap, PORT_REG(idx,
+ MPS_PORT_STAT_LB_PORT_DROP_FRAMES));
+
+ p->ovflow0 = (bgmap & 1) ? GET_STAT_COM(RX_BG_0_LB_DROP_FRAME) : 0;
+ p->ovflow1 = (bgmap & 2) ? GET_STAT_COM(RX_BG_1_LB_DROP_FRAME) : 0;
+ p->ovflow2 = (bgmap & 4) ? GET_STAT_COM(RX_BG_2_LB_DROP_FRAME) : 0;
+ p->ovflow3 = (bgmap & 8) ? GET_STAT_COM(RX_BG_3_LB_DROP_FRAME) : 0;
+ p->trunc0 = (bgmap & 1) ? GET_STAT_COM(RX_BG_0_LB_TRUNC_FRAME) : 0;
+ p->trunc1 = (bgmap & 2) ? GET_STAT_COM(RX_BG_1_LB_TRUNC_FRAME) : 0;
+ p->trunc2 = (bgmap & 4) ? GET_STAT_COM(RX_BG_2_LB_TRUNC_FRAME) : 0;
+ p->trunc3 = (bgmap & 8) ? GET_STAT_COM(RX_BG_3_LB_TRUNC_FRAME) : 0;
+
+#undef GET_STAT
+#undef GET_STAT_COM
+}
+
+/**
+ * t4_wol_magic_enable - enable/disable magic packet WoL
+ * @adap: the adapter
+ * @port: the physical port index
+ * @addr: MAC address expected in magic packets, %NULL to disable
+ *
+ * Enables/disables magic packet wake-on-LAN for the selected port.
+ */
+void t4_wol_magic_enable(struct adapter *adap, unsigned int port,
+ const u8 *addr)
+{
+ if (addr) {
+ t4_write_reg(adap, PORT_REG(port, XGMAC_PORT_MAGIC_MACID_LO),
+ (addr[2] << 24) | (addr[3] << 16) |
+ (addr[4] << 8) | addr[5]);
+ t4_write_reg(adap, PORT_REG(port, XGMAC_PORT_MAGIC_MACID_HI),
+ (addr[0] << 8) | addr[1]);
+ }
+ t4_set_reg_field(adap, PORT_REG(port, XGMAC_PORT_CFG2), MAGICEN,
+ addr ? MAGICEN : 0);
+}
+
+/**
+ * t4_wol_pat_enable - enable/disable pattern-based WoL
+ * @adap: the adapter
+ * @port: the physical port index
+ * @map: bitmap of which HW pattern filters to set
+ * @mask0: byte mask for bytes 0-63 of a packet
+ * @mask1: byte mask for bytes 64-127 of a packet
+ * @crc: Ethernet CRC for selected bytes
+ * @enable: enable/disable switch
+ *
+ * Sets the pattern filters indicated in @map to mask out the bytes
+ * specified in @mask0/@mask1 in received packets and compare the CRC of
+ * the resulting packet against @crc. If @enable is %true pattern-based
+ * WoL is enabled, otherwise disabled.
+ */
+int t4_wol_pat_enable(struct adapter *adap, unsigned int port, unsigned int map,
+ u64 mask0, u64 mask1, unsigned int crc, bool enable)
+{
+ int i;
+
+ if (!enable) {
+ t4_set_reg_field(adap, PORT_REG(port, XGMAC_PORT_CFG2),
+ PATEN, 0);
+ return 0;
+ }
+ if (map > 0xff)
+ return -EINVAL;
+
+#define EPIO_REG(name) PORT_REG(port, XGMAC_PORT_EPIO_##name)
+
+ t4_write_reg(adap, EPIO_REG(DATA1), mask0 >> 32);
+ t4_write_reg(adap, EPIO_REG(DATA2), mask1);
+ t4_write_reg(adap, EPIO_REG(DATA3), mask1 >> 32);
+
+ for (i = 0; i < NWOL_PAT; i++, map >>= 1) {
+ if (!(map & 1))
+ continue;
+
+ /* write byte masks */
+ t4_write_reg(adap, EPIO_REG(DATA0), mask0);
+ t4_write_reg(adap, EPIO_REG(OP), ADDRESS(i) | EPIOWR);
+ t4_read_reg(adap, EPIO_REG(OP)); /* flush */
+ if (t4_read_reg(adap, EPIO_REG(OP)) & BUSY)
+ return -ETIMEDOUT;
+
+ /* write CRC */
+ t4_write_reg(adap, EPIO_REG(DATA0), crc);
+ t4_write_reg(adap, EPIO_REG(OP), ADDRESS(i + 32) | EPIOWR);
+ t4_read_reg(adap, EPIO_REG(OP)); /* flush */
+ if (t4_read_reg(adap, EPIO_REG(OP)) & BUSY)
+ return -ETIMEDOUT;
+ }
+#undef EPIO_REG
+
+ t4_set_reg_field(adap, PORT_REG(port, XGMAC_PORT_CFG2), 0, PATEN);
+ return 0;
+}
+
+#define INIT_CMD(var, cmd, rd_wr) do { \
+ (var).op_to_write = htonl(FW_CMD_OP(FW_##cmd##_CMD) | \
+ FW_CMD_REQUEST | FW_CMD_##rd_wr); \
+ (var).retval_len16 = htonl(FW_LEN16(var)); \
+} while (0)
+
+/**
+ * t4_mdio_rd - read a PHY register through MDIO
+ * @adap: the adapter
+ * @mbox: mailbox to use for the FW command
+ * @phy_addr: the PHY address
+ * @mmd: the PHY MMD to access (0 for clause 22 PHYs)
+ * @reg: the register to read
+ * @valp: where to store the value
+ *
+ * Issues a FW command through the given mailbox to read a PHY register.
+ */
+int t4_mdio_rd(struct adapter *adap, unsigned int mbox, unsigned int phy_addr,
+ unsigned int mmd, unsigned int reg, u16 *valp)
+{
+ int ret;
+ struct fw_ldst_cmd c;
+
+ memset(&c, 0, sizeof(c));
+ c.op_to_addrspace = htonl(FW_CMD_OP(FW_LDST_CMD) | FW_CMD_REQUEST |
+ FW_CMD_READ | FW_LDST_CMD_ADDRSPACE(FW_LDST_ADDRSPC_MDIO));
+ c.cycles_to_len16 = htonl(FW_LEN16(c));
+ c.u.mdio.paddr_mmd = htons(FW_LDST_CMD_PADDR(phy_addr) |
+ FW_LDST_CMD_MMD(mmd));
+ c.u.mdio.raddr = htons(reg);
+
+ ret = t4_wr_mbox(adap, mbox, &c, sizeof(c), &c);
+ if (ret == 0)
+ *valp = ntohs(c.u.mdio.rval);
+ return ret;
+}
+
+/**
+ * t4_mdio_wr - write a PHY register through MDIO
+ * @adap: the adapter
+ * @mbox: mailbox to use for the FW command
+ * @phy_addr: the PHY address
+ * @mmd: the PHY MMD to access (0 for clause 22 PHYs)
+ * @reg: the register to write
+ * @valp: value to write
+ *
+ * Issues a FW command through the given mailbox to write a PHY register.
+ */
+int t4_mdio_wr(struct adapter *adap, unsigned int mbox, unsigned int phy_addr,
+ unsigned int mmd, unsigned int reg, u16 val)
+{
+ struct fw_ldst_cmd c;
+
+ memset(&c, 0, sizeof(c));
+ c.op_to_addrspace = htonl(FW_CMD_OP(FW_LDST_CMD) | FW_CMD_REQUEST |
+ FW_CMD_WRITE | FW_LDST_CMD_ADDRSPACE(FW_LDST_ADDRSPC_MDIO));
+ c.cycles_to_len16 = htonl(FW_LEN16(c));
+ c.u.mdio.paddr_mmd = htons(FW_LDST_CMD_PADDR(phy_addr) |
+ FW_LDST_CMD_MMD(mmd));
+ c.u.mdio.raddr = htons(reg);
+ c.u.mdio.rval = htons(val);
+
+ return t4_wr_mbox(adap, mbox, &c, sizeof(c), NULL);
+}
+
+/**
+ * t4_fw_hello - establish communication with FW
+ * @adap: the adapter
+ * @mbox: mailbox to use for the FW command
+ * @evt_mbox: mailbox to receive async FW events
+ * @master: specifies the caller's willingness to be the device master
+ * @state: returns the current device state
+ *
+ * Issues a command to establish communication with FW.
+ */
+int t4_fw_hello(struct adapter *adap, unsigned int mbox, unsigned int evt_mbox,
+ enum dev_master master, enum dev_state *state)
+{
+ int ret;
+ struct fw_hello_cmd c;
+
+ INIT_CMD(c, HELLO, WRITE);
+ c.err_to_mbasyncnot = htonl(
+ FW_HELLO_CMD_MASTERDIS(master == MASTER_CANT) |
+ FW_HELLO_CMD_MASTERFORCE(master == MASTER_MUST) |
+ FW_HELLO_CMD_MBMASTER(master == MASTER_MUST ? mbox : 0xff) |
+ FW_HELLO_CMD_MBASYNCNOT(evt_mbox));
+
+ ret = t4_wr_mbox(adap, mbox, &c, sizeof(c), &c);
+ if (ret == 0 && state) {
+ u32 v = ntohl(c.err_to_mbasyncnot);
+ if (v & FW_HELLO_CMD_INIT)
+ *state = DEV_STATE_INIT;
+ else if (v & FW_HELLO_CMD_ERR)
+ *state = DEV_STATE_ERR;
+ else
+ *state = DEV_STATE_UNINIT;
+ }
+ return ret;
+}
+
+/**
+ * t4_fw_bye - end communication with FW
+ * @adap: the adapter
+ * @mbox: mailbox to use for the FW command
+ *
+ * Issues a command to terminate communication with FW.
+ */
+int t4_fw_bye(struct adapter *adap, unsigned int mbox)
+{
+ struct fw_bye_cmd c;
+
+ INIT_CMD(c, BYE, WRITE);
+ return t4_wr_mbox(adap, mbox, &c, sizeof(c), NULL);
+}
+
+/**
+ * t4_init_cmd - ask FW to initialize the device
+ * @adap: the adapter
+ * @mbox: mailbox to use for the FW command
+ *
+ * Issues a command to FW to partially initialize the device. This
+ * performs initialization that generally doesn't depend on user input.
+ */
+int t4_early_init(struct adapter *adap, unsigned int mbox)
+{
+ struct fw_initialize_cmd c;
+
+ INIT_CMD(c, INITIALIZE, WRITE);
+ return t4_wr_mbox(adap, mbox, &c, sizeof(c), NULL);
+}
+
+/**
+ * t4_fw_reset - issue a reset to FW
+ * @adap: the adapter
+ * @mbox: mailbox to use for the FW command
+ * @reset: specifies the type of reset to perform
+ *
+ * Issues a reset command of the specified type to FW.
+ */
+int t4_fw_reset(struct adapter *adap, unsigned int mbox, int reset)
+{
+ struct fw_reset_cmd c;
+
+ INIT_CMD(c, RESET, WRITE);
+ c.val = htonl(reset);
+ return t4_wr_mbox(adap, mbox, &c, sizeof(c), NULL);
+}
+
+/**
+ * t4_query_params - query FW or device parameters
+ * @adap: the adapter
+ * @mbox: mailbox to use for the FW command
+ * @pf: the PF
+ * @vf: the VF
+ * @nparams: the number of parameters
+ * @params: the parameter names
+ * @val: the parameter values
+ *
+ * Reads the value of FW or device parameters. Up to 7 parameters can be
+ * queried at once.
+ */
+int t4_query_params(struct adapter *adap, unsigned int mbox, unsigned int pf,
+ unsigned int vf, unsigned int nparams, const u32 *params,
+ u32 *val)
+{
+ int i, ret;
+ struct fw_params_cmd c;
+ __be32 *p = &c.param[0].mnem;
+
+ if (nparams > 7)
+ return -EINVAL;
+
+ memset(&c, 0, sizeof(c));
+ c.op_to_vfn = htonl(FW_CMD_OP(FW_PARAMS_CMD) | FW_CMD_REQUEST |
+ FW_CMD_READ | FW_PARAMS_CMD_PFN(pf) |
+ FW_PARAMS_CMD_VFN(vf));
+ c.retval_len16 = htonl(FW_LEN16(c));
+ for (i = 0; i < nparams; i++, p += 2)
+ *p = htonl(*params++);
+
+ ret = t4_wr_mbox(adap, mbox, &c, sizeof(c), &c);
+ if (ret == 0)
+ for (i = 0, p = &c.param[0].val; i < nparams; i++, p += 2)
+ *val++ = ntohl(*p);
+ return ret;
+}
+
+/**
+ * t4_set_params - sets FW or device parameters
+ * @adap: the adapter
+ * @mbox: mailbox to use for the FW command
+ * @pf: the PF
+ * @vf: the VF
+ * @nparams: the number of parameters
+ * @params: the parameter names
+ * @val: the parameter values
+ *
+ * Sets the value of FW or device parameters. Up to 7 parameters can be
+ * specified at once.
+ */
+int t4_set_params(struct adapter *adap, unsigned int mbox, unsigned int pf,
+ unsigned int vf, unsigned int nparams, const u32 *params,
+ const u32 *val)
+{
+ struct fw_params_cmd c;
+ __be32 *p = &c.param[0].mnem;
+
+ if (nparams > 7)
+ return -EINVAL;
+
+ memset(&c, 0, sizeof(c));
+ c.op_to_vfn = htonl(FW_CMD_OP(FW_PARAMS_CMD) | FW_CMD_REQUEST |
+ FW_CMD_WRITE | FW_PARAMS_CMD_PFN(pf) |
+ FW_PARAMS_CMD_VFN(vf));
+ c.retval_len16 = htonl(FW_LEN16(c));
+ while (nparams--) {
+ *p++ = htonl(*params++);
+ *p++ = htonl(*val++);
+ }
+
+ return t4_wr_mbox(adap, mbox, &c, sizeof(c), NULL);
+}
+
+/**
+ * t4_cfg_pfvf - configure PF/VF resource limits
+ * @adap: the adapter
+ * @mbox: mailbox to use for the FW command
+ * @pf: the PF being configured
+ * @vf: the VF being configured
+ * @txq: the max number of egress queues
+ * @txq_eth_ctrl: the max number of egress Ethernet or control queues
+ * @rxqi: the max number of interrupt-capable ingress queues
+ * @rxq: the max number of interruptless ingress queues
+ * @tc: the PCI traffic class
+ * @vi: the max number of virtual interfaces
+ * @cmask: the channel access rights mask for the PF/VF
+ * @pmask: the port access rights mask for the PF/VF
+ * @nexact: the maximum number of exact MPS filters
+ * @rcaps: read capabilities
+ * @wxcaps: write/execute capabilities
+ *
+ * Configures resource limits and capabilities for a physical or virtual
+ * function.
+ */
+int t4_cfg_pfvf(struct adapter *adap, unsigned int mbox, unsigned int pf,
+ unsigned int vf, unsigned int txq, unsigned int txq_eth_ctrl,
+ unsigned int rxqi, unsigned int rxq, unsigned int tc,
+ unsigned int vi, unsigned int cmask, unsigned int pmask,
+ unsigned int nexact, unsigned int rcaps, unsigned int wxcaps)
+{
+ struct fw_pfvf_cmd c;
+
+ memset(&c, 0, sizeof(c));
+ c.op_to_vfn = htonl(FW_CMD_OP(FW_PFVF_CMD) | FW_CMD_REQUEST |
+ FW_CMD_WRITE | FW_PFVF_CMD_PFN(pf) |
+ FW_PFVF_CMD_VFN(vf));
+ c.retval_len16 = htonl(FW_LEN16(c));
+ c.niqflint_niq = htonl(FW_PFVF_CMD_NIQFLINT(rxqi) |
+ FW_PFVF_CMD_NIQ(rxq));
+ c.cmask_to_neq = htonl(FW_PFVF_CMD_CMASK(cmask) |
+ FW_PFVF_CMD_PMASK(pmask) |
+ FW_PFVF_CMD_NEQ(txq));
+ c.tc_to_nexactf = htonl(FW_PFVF_CMD_TC(tc) | FW_PFVF_CMD_NVI(vi) |
+ FW_PFVF_CMD_NEXACTF(nexact));
+ c.r_caps_to_nethctrl = htonl(FW_PFVF_CMD_R_CAPS(rcaps) |
+ FW_PFVF_CMD_WX_CAPS(wxcaps) |
+ FW_PFVF_CMD_NETHCTRL(txq_eth_ctrl));
+ return t4_wr_mbox(adap, mbox, &c, sizeof(c), NULL);
+}
+
+/**
+ * t4_alloc_vi - allocate a virtual interface
+ * @adap: the adapter
+ * @mbox: mailbox to use for the FW command
+ * @port: physical port associated with the VI
+ * @pf: the PF owning the VI
+ * @vf: the VF owning the VI
+ * @nmac: number of MAC addresses needed (1 to 5)
+ * @mac: the MAC addresses of the VI
+ * @rss_size: size of RSS table slice associated with this VI
+ *
+ * Allocates a virtual interface for the given physical port. If @mac is
+ * not %NULL it contains the MAC addresses of the VI as assigned by FW.
+ * @mac should be large enough to hold @nmac Ethernet addresses, they are
+ * stored consecutively so the space needed is @nmac * 6 bytes.
+ * Returns a negative error number or the non-negative VI id.
+ */
+int t4_alloc_vi(struct adapter *adap, unsigned int mbox, unsigned int port,
+ unsigned int pf, unsigned int vf, unsigned int nmac, u8 *mac,
+ unsigned int *rss_size)
+{
+ int ret;
+ struct fw_vi_cmd c;
+
+ memset(&c, 0, sizeof(c));
+ c.op_to_vfn = htonl(FW_CMD_OP(FW_VI_CMD) | FW_CMD_REQUEST |
+ FW_CMD_WRITE | FW_CMD_EXEC |
+ FW_VI_CMD_PFN(pf) | FW_VI_CMD_VFN(vf));
+ c.alloc_to_len16 = htonl(FW_VI_CMD_ALLOC | FW_LEN16(c));
+ c.portid_pkd = FW_VI_CMD_PORTID(port);
+ c.nmac = nmac - 1;
+
+ ret = t4_wr_mbox(adap, mbox, &c, sizeof(c), &c);
+ if (ret)
+ return ret;
+
+ if (mac) {
+ memcpy(mac, c.mac, sizeof(c.mac));
+ switch (nmac) {
+ case 5:
+ memcpy(mac + 24, c.nmac3, sizeof(c.nmac3));
+ case 4:
+ memcpy(mac + 18, c.nmac2, sizeof(c.nmac2));
+ case 3:
+ memcpy(mac + 12, c.nmac1, sizeof(c.nmac1));
+ case 2:
+ memcpy(mac + 6, c.nmac0, sizeof(c.nmac0));
+ }
+ }
+ if (rss_size)
+ *rss_size = FW_VI_CMD_RSSSIZE_GET(ntohs(c.rsssize_pkd));
+ return ntohs(c.viid_pkd);
+}
+
+/**
+ * t4_free_vi - free a virtual interface
+ * @adap: the adapter
+ * @mbox: mailbox to use for the FW command
+ * @pf: the PF owning the VI
+ * @vf: the VF owning the VI
+ * @viid: virtual interface identifiler
+ *
+ * Free a previously allocated virtual interface.
+ */
+int t4_free_vi(struct adapter *adap, unsigned int mbox, unsigned int pf,
+ unsigned int vf, unsigned int viid)
+{
+ struct fw_vi_cmd c;
+
+ memset(&c, 0, sizeof(c));
+ c.op_to_vfn = htonl(FW_CMD_OP(FW_VI_CMD) | FW_CMD_REQUEST |
+ FW_CMD_EXEC | FW_VI_CMD_PFN(pf) |
+ FW_VI_CMD_VFN(vf));
+ c.alloc_to_len16 = htonl(FW_VI_CMD_FREE | FW_LEN16(c));
+ c.viid_pkd = htons(FW_VI_CMD_VIID(viid));
+ return t4_wr_mbox(adap, mbox, &c, sizeof(c), &c);
+}
+
+/**
+ * t4_set_rxmode - set Rx properties of a virtual interface
+ * @adap: the adapter
+ * @mbox: mailbox to use for the FW command
+ * @viid: the VI id
+ * @mtu: the new MTU or -1
+ * @promisc: 1 to enable promiscuous mode, 0 to disable it, -1 no change
+ * @all_multi: 1 to enable all-multi mode, 0 to disable it, -1 no change
+ * @bcast: 1 to enable broadcast Rx, 0 to disable it, -1 no change
+ * @sleep_ok: if true we may sleep while awaiting command completion
+ *
+ * Sets Rx properties of a virtual interface.
+ */
+int t4_set_rxmode(struct adapter *adap, unsigned int mbox, unsigned int viid,
+ int mtu, int promisc, int all_multi, int bcast, bool sleep_ok)
+{
+ struct fw_vi_rxmode_cmd c;
+
+ /* convert to FW values */
+ if (mtu < 0)
+ mtu = FW_RXMODE_MTU_NO_CHG;
+ if (promisc < 0)
+ promisc = FW_VI_RXMODE_CMD_PROMISCEN_MASK;
+ if (all_multi < 0)
+ all_multi = FW_VI_RXMODE_CMD_ALLMULTIEN_MASK;
+ if (bcast < 0)
+ bcast = FW_VI_RXMODE_CMD_BROADCASTEN_MASK;
+
+ memset(&c, 0, sizeof(c));
+ c.op_to_viid = htonl(FW_CMD_OP(FW_VI_RXMODE_CMD) | FW_CMD_REQUEST |
+ FW_CMD_WRITE | FW_VI_RXMODE_CMD_VIID(viid));
+ c.retval_len16 = htonl(FW_LEN16(c));
+ c.mtu_to_broadcasten = htonl(FW_VI_RXMODE_CMD_MTU(mtu) |
+ FW_VI_RXMODE_CMD_PROMISCEN(promisc) |
+ FW_VI_RXMODE_CMD_ALLMULTIEN(all_multi) |
+ FW_VI_RXMODE_CMD_BROADCASTEN(bcast));
+ return t4_wr_mbox_meat(adap, mbox, &c, sizeof(c), NULL, sleep_ok);
+}
+
+/**
+ * t4_alloc_mac_filt - allocates exact-match filters for MAC addresses
+ * @adap: the adapter
+ * @mbox: mailbox to use for the FW command
+ * @viid: the VI id
+ * @free: if true any existing filters for this VI id are first removed
+ * @naddr: the number of MAC addresses to allocate filters for (up to 7)
+ * @addr: the MAC address(es)
+ * @idx: where to store the index of each allocated filter
+ * @hash: pointer to hash address filter bitmap
+ * @sleep_ok: call is allowed to sleep
+ *
+ * Allocates an exact-match filter for each of the supplied addresses and
+ * sets it to the corresponding address. If @idx is not %NULL it should
+ * have at least @naddr entries, each of which will be set to the index of
+ * the filter allocated for the corresponding MAC address. If a filter
+ * could not be allocated for an address its index is set to 0xffff.
+ * If @hash is not %NULL addresses that fail to allocate an exact filter
+ * are hashed and update the hash filter bitmap pointed at by @hash.
+ *
+ * Returns a negative error number or the number of filters allocated.
+ */
+int t4_alloc_mac_filt(struct adapter *adap, unsigned int mbox,
+ unsigned int viid, bool free, unsigned int naddr,
+ const u8 **addr, u16 *idx, u64 *hash, bool sleep_ok)
+{
+ int i, ret;
+ struct fw_vi_mac_cmd c;
+ struct fw_vi_mac_exact *p;
+
+ if (naddr > 7)
+ return -EINVAL;
+
+ memset(&c, 0, sizeof(c));
+ c.op_to_viid = htonl(FW_CMD_OP(FW_VI_MAC_CMD) | FW_CMD_REQUEST |
+ FW_CMD_WRITE | (free ? FW_CMD_EXEC : 0) |
+ FW_VI_MAC_CMD_VIID(viid));
+ c.freemacs_to_len16 = htonl(FW_VI_MAC_CMD_FREEMACS(free) |
+ FW_CMD_LEN16((naddr + 2) / 2));
+
+ for (i = 0, p = c.u.exact; i < naddr; i++, p++) {
+ p->valid_to_idx = htons(FW_VI_MAC_CMD_VALID |
+ FW_VI_MAC_CMD_IDX(FW_VI_MAC_ADD_MAC));
+ memcpy(p->macaddr, addr[i], sizeof(p->macaddr));
+ }
+
+ ret = t4_wr_mbox_meat(adap, mbox, &c, sizeof(c), &c, sleep_ok);
+ if (ret)
+ return ret;
+
+ for (i = 0, p = c.u.exact; i < naddr; i++, p++) {
+ u16 index = FW_VI_MAC_CMD_IDX_GET(ntohs(p->valid_to_idx));
+
+ if (idx)
+ idx[i] = index >= NEXACT_MAC ? 0xffff : index;
+ if (index < NEXACT_MAC)
+ ret++;
+ else if (hash)
+ *hash |= (1 << hash_mac_addr(addr[i]));
+ }
+ return ret;
+}
+
+/**
+ * t4_change_mac - modifies the exact-match filter for a MAC address
+ * @adap: the adapter
+ * @mbox: mailbox to use for the FW command
+ * @viid: the VI id
+ * @idx: index of existing filter for old value of MAC address, or -1
+ * @addr: the new MAC address value
+ * @persist: whether a new MAC allocation should be persistent
+ * @add_smt: if true also add the address to the HW SMT
+ *
+ * Modifies an exact-match filter and sets it to the new MAC address.
+ * Note that in general it is not possible to modify the value of a given
+ * filter so the generic way to modify an address filter is to free the one
+ * being used by the old address value and allocate a new filter for the
+ * new address value. @idx can be -1 if the address is a new addition.
+ *
+ * Returns a negative error number or the index of the filter with the new
+ * MAC value.
+ */
+int t4_change_mac(struct adapter *adap, unsigned int mbox, unsigned int viid,
+ int idx, const u8 *addr, bool persist, bool add_smt)
+{
+ int ret, mode;
+ struct fw_vi_mac_cmd c;
+ struct fw_vi_mac_exact *p = c.u.exact;
+
+ if (idx < 0) /* new allocation */
+ idx = persist ? FW_VI_MAC_ADD_PERSIST_MAC : FW_VI_MAC_ADD_MAC;
+ mode = add_smt ? FW_VI_MAC_SMT_AND_MPSTCAM : FW_VI_MAC_MPS_TCAM_ENTRY;
+
+ memset(&c, 0, sizeof(c));
+ c.op_to_viid = htonl(FW_CMD_OP(FW_VI_MAC_CMD) | FW_CMD_REQUEST |
+ FW_CMD_WRITE | FW_VI_MAC_CMD_VIID(viid));
+ c.freemacs_to_len16 = htonl(FW_CMD_LEN16(1));
+ p->valid_to_idx = htons(FW_VI_MAC_CMD_VALID |
+ FW_VI_MAC_CMD_SMAC_RESULT(mode) |
+ FW_VI_MAC_CMD_IDX(idx));
+ memcpy(p->macaddr, addr, sizeof(p->macaddr));
+
+ ret = t4_wr_mbox(adap, mbox, &c, sizeof(c), &c);
+ if (ret == 0) {
+ ret = FW_VI_MAC_CMD_IDX_GET(ntohs(p->valid_to_idx));
+ if (ret >= NEXACT_MAC)
+ ret = -ENOMEM;
+ }
+ return ret;
+}
+
+/**
+ * t4_set_addr_hash - program the MAC inexact-match hash filter
+ * @adap: the adapter
+ * @mbox: mailbox to use for the FW command
+ * @viid: the VI id
+ * @ucast: whether the hash filter should also match unicast addresses
+ * @vec: the value to be written to the hash filter
+ * @sleep_ok: call is allowed to sleep
+ *
+ * Sets the 64-bit inexact-match hash filter for a virtual interface.
+ */
+int t4_set_addr_hash(struct adapter *adap, unsigned int mbox, unsigned int viid,
+ bool ucast, u64 vec, bool sleep_ok)
+{
+ struct fw_vi_mac_cmd c;
+
+ memset(&c, 0, sizeof(c));
+ c.op_to_viid = htonl(FW_CMD_OP(FW_VI_MAC_CMD) | FW_CMD_REQUEST |
+ FW_CMD_WRITE | FW_VI_ENABLE_CMD_VIID(viid));
+ c.freemacs_to_len16 = htonl(FW_VI_MAC_CMD_HASHVECEN |
+ FW_VI_MAC_CMD_HASHUNIEN(ucast) |
+ FW_CMD_LEN16(1));
+ c.u.hash.hashvec = cpu_to_be64(vec);
+ return t4_wr_mbox_meat(adap, mbox, &c, sizeof(c), NULL, sleep_ok);
+}
+
+/**
+ * t4_enable_vi - enable/disable a virtual interface
+ * @adap: the adapter
+ * @mbox: mailbox to use for the FW command
+ * @viid: the VI id
+ * @rx_en: 1=enable Rx, 0=disable Rx
+ * @tx_en: 1=enable Tx, 0=disable Tx
+ *
+ * Enables/disables a virtual interface.
+ */
+int t4_enable_vi(struct adapter *adap, unsigned int mbox, unsigned int viid,
+ bool rx_en, bool tx_en)
+{
+ struct fw_vi_enable_cmd c;
+
+ memset(&c, 0, sizeof(c));
+ c.op_to_viid = htonl(FW_CMD_OP(FW_VI_ENABLE_CMD) | FW_CMD_REQUEST |
+ FW_CMD_EXEC | FW_VI_ENABLE_CMD_VIID(viid));
+ c.ien_to_len16 = htonl(FW_VI_ENABLE_CMD_IEN(rx_en) |
+ FW_VI_ENABLE_CMD_EEN(tx_en) | FW_LEN16(c));
+ return t4_wr_mbox(adap, mbox, &c, sizeof(c), NULL);
+}
+
+/**
+ * t4_identify_port - identify a VI's port by blinking its LED
+ * @adap: the adapter
+ * @mbox: mailbox to use for the FW command
+ * @viid: the VI id
+ * @nblinks: how many times to blink LED at 2.5 Hz
+ *
+ * Identifies a VI's port by blinking its LED.
+ */
+int t4_identify_port(struct adapter *adap, unsigned int mbox, unsigned int viid,
+ unsigned int nblinks)
+{
+ struct fw_vi_enable_cmd c;
+
+ c.op_to_viid = htonl(FW_CMD_OP(FW_VI_ENABLE_CMD) | FW_CMD_REQUEST |
+ FW_CMD_EXEC | FW_VI_ENABLE_CMD_VIID(viid));
+ c.ien_to_len16 = htonl(FW_VI_ENABLE_CMD_LED | FW_LEN16(c));
+ c.blinkdur = htons(nblinks);
+ return t4_wr_mbox(adap, mbox, &c, sizeof(c), NULL);
+}
+
+/**
+ * t4_iq_start_stop - enable/disable an ingress queue and its FLs
+ * @adap: the adapter
+ * @mbox: mailbox to use for the FW command
+ * @start: %true to enable the queues, %false to disable them
+ * @pf: the PF owning the queues
+ * @vf: the VF owning the queues
+ * @iqid: ingress queue id
+ * @fl0id: FL0 queue id or 0xffff if no attached FL0
+ * @fl1id: FL1 queue id or 0xffff if no attached FL1
+ *
+ * Starts or stops an ingress queue and its associated FLs, if any.
+ */
+int t4_iq_start_stop(struct adapter *adap, unsigned int mbox, bool start,
+ unsigned int pf, unsigned int vf, unsigned int iqid,
+ unsigned int fl0id, unsigned int fl1id)
+{
+ struct fw_iq_cmd c;
+
+ memset(&c, 0, sizeof(c));
+ c.op_to_vfn = htonl(FW_CMD_OP(FW_IQ_CMD) | FW_CMD_REQUEST |
+ FW_CMD_EXEC | FW_IQ_CMD_PFN(pf) |
+ FW_IQ_CMD_VFN(vf));
+ c.alloc_to_len16 = htonl(FW_IQ_CMD_IQSTART(start) |
+ FW_IQ_CMD_IQSTOP(!start) | FW_LEN16(c));
+ c.iqid = htons(iqid);
+ c.fl0id = htons(fl0id);
+ c.fl1id = htons(fl1id);
+ return t4_wr_mbox(adap, mbox, &c, sizeof(c), NULL);
+}
+
+/**
+ * t4_iq_free - free an ingress queue and its FLs
+ * @adap: the adapter
+ * @mbox: mailbox to use for the FW command
+ * @pf: the PF owning the queues
+ * @vf: the VF owning the queues
+ * @iqtype: the ingress queue type
+ * @iqid: ingress queue id
+ * @fl0id: FL0 queue id or 0xffff if no attached FL0
+ * @fl1id: FL1 queue id or 0xffff if no attached FL1
+ *
+ * Frees an ingress queue and its associated FLs, if any.
+ */
+int t4_iq_free(struct adapter *adap, unsigned int mbox, unsigned int pf,
+ unsigned int vf, unsigned int iqtype, unsigned int iqid,
+ unsigned int fl0id, unsigned int fl1id)
+{
+ struct fw_iq_cmd c;
+
+ memset(&c, 0, sizeof(c));
+ c.op_to_vfn = htonl(FW_CMD_OP(FW_IQ_CMD) | FW_CMD_REQUEST |
+ FW_CMD_EXEC | FW_IQ_CMD_PFN(pf) |
+ FW_IQ_CMD_VFN(vf));
+ c.alloc_to_len16 = htonl(FW_IQ_CMD_FREE | FW_LEN16(c));
+ c.type_to_iqandstindex = htonl(FW_IQ_CMD_TYPE(iqtype));
+ c.iqid = htons(iqid);
+ c.fl0id = htons(fl0id);
+ c.fl1id = htons(fl1id);
+ return t4_wr_mbox(adap, mbox, &c, sizeof(c), NULL);
+}
+
+/**
+ * t4_eth_eq_free - free an Ethernet egress queue
+ * @adap: the adapter
+ * @mbox: mailbox to use for the FW command
+ * @pf: the PF owning the queue
+ * @vf: the VF owning the queue
+ * @eqid: egress queue id
+ *
+ * Frees an Ethernet egress queue.
+ */
+int t4_eth_eq_free(struct adapter *adap, unsigned int mbox, unsigned int pf,
+ unsigned int vf, unsigned int eqid)
+{
+ struct fw_eq_eth_cmd c;
+
+ memset(&c, 0, sizeof(c));
+ c.op_to_vfn = htonl(FW_CMD_OP(FW_EQ_ETH_CMD) | FW_CMD_REQUEST |
+ FW_CMD_EXEC | FW_EQ_ETH_CMD_PFN(pf) |
+ FW_EQ_ETH_CMD_VFN(vf));
+ c.alloc_to_len16 = htonl(FW_EQ_ETH_CMD_FREE | FW_LEN16(c));
+ c.eqid_pkd = htonl(FW_EQ_ETH_CMD_EQID(eqid));
+ return t4_wr_mbox(adap, mbox, &c, sizeof(c), NULL);
+}
+
+/**
+ * t4_ctrl_eq_free - free a control egress queue
+ * @adap: the adapter
+ * @mbox: mailbox to use for the FW command
+ * @pf: the PF owning the queue
+ * @vf: the VF owning the queue
+ * @eqid: egress queue id
+ *
+ * Frees a control egress queue.
+ */
+int t4_ctrl_eq_free(struct adapter *adap, unsigned int mbox, unsigned int pf,
+ unsigned int vf, unsigned int eqid)
+{
+ struct fw_eq_ctrl_cmd c;
+
+ memset(&c, 0, sizeof(c));
+ c.op_to_vfn = htonl(FW_CMD_OP(FW_EQ_CTRL_CMD) | FW_CMD_REQUEST |
+ FW_CMD_EXEC | FW_EQ_CTRL_CMD_PFN(pf) |
+ FW_EQ_CTRL_CMD_VFN(vf));
+ c.alloc_to_len16 = htonl(FW_EQ_CTRL_CMD_FREE | FW_LEN16(c));
+ c.cmpliqid_eqid = htonl(FW_EQ_CTRL_CMD_EQID(eqid));
+ return t4_wr_mbox(adap, mbox, &c, sizeof(c), NULL);
+}
+
+/**
+ * t4_ofld_eq_free - free an offload egress queue
+ * @adap: the adapter
+ * @mbox: mailbox to use for the FW command
+ * @pf: the PF owning the queue
+ * @vf: the VF owning the queue
+ * @eqid: egress queue id
+ *
+ * Frees a control egress queue.
+ */
+int t4_ofld_eq_free(struct adapter *adap, unsigned int mbox, unsigned int pf,
+ unsigned int vf, unsigned int eqid)
+{
+ struct fw_eq_ofld_cmd c;
+
+ memset(&c, 0, sizeof(c));
+ c.op_to_vfn = htonl(FW_CMD_OP(FW_EQ_OFLD_CMD) | FW_CMD_REQUEST |
+ FW_CMD_EXEC | FW_EQ_OFLD_CMD_PFN(pf) |
+ FW_EQ_OFLD_CMD_VFN(vf));
+ c.alloc_to_len16 = htonl(FW_EQ_OFLD_CMD_FREE | FW_LEN16(c));
+ c.eqid_pkd = htonl(FW_EQ_OFLD_CMD_EQID(eqid));
+ return t4_wr_mbox(adap, mbox, &c, sizeof(c), NULL);
+}
+
+/**
+ * t4_handle_fw_rpl - process a FW reply message
+ * @adap: the adapter
+ * @rpl: start of the FW message
+ *
+ * Processes a FW message, such as link state change messages.
+ */
+int t4_handle_fw_rpl(struct adapter *adap, const __be64 *rpl)
+{
+ u8 opcode = *(const u8 *)rpl;
+
+ if (opcode == FW_PORT_CMD) { /* link/module state change message */
+ int speed = 0, fc = 0;
+ const struct fw_port_cmd *p = (void *)rpl;
+ int chan = FW_PORT_CMD_PORTID_GET(ntohl(p->op_to_portid));
+ int port = adap->chan_map[chan];
+ struct port_info *pi = adap2pinfo(adap, port);
+ struct link_config *lc = &pi->link_cfg;
+ u32 stat = ntohl(p->u.info.lstatus_to_modtype);
+ int link_ok = (stat & FW_PORT_CMD_LSTATUS) != 0;
+ u32 mod = FW_PORT_CMD_MODTYPE_GET(stat);
+
+ if (stat & FW_PORT_CMD_RXPAUSE)
+ fc |= PAUSE_RX;
+ if (stat & FW_PORT_CMD_TXPAUSE)
+ fc |= PAUSE_TX;
+ if (stat & FW_PORT_CMD_LSPEED(FW_PORT_CAP_SPEED_100M))
+ speed = SPEED_100;
+ else if (stat & FW_PORT_CMD_LSPEED(FW_PORT_CAP_SPEED_1G))
+ speed = SPEED_1000;
+ else if (stat & FW_PORT_CMD_LSPEED(FW_PORT_CAP_SPEED_10G))
+ speed = SPEED_10000;
+
+ if (link_ok != lc->link_ok || speed != lc->speed ||
+ fc != lc->fc) { /* something changed */
+ lc->link_ok = link_ok;
+ lc->speed = speed;
+ lc->fc = fc;
+ t4_os_link_changed(adap, port, link_ok);
+ }
+ if (mod != pi->mod_type) {
+ pi->mod_type = mod;
+ t4_os_portmod_changed(adap, port);
+ }
+ }
+ return 0;
+}
+
+static void __devinit get_pci_mode(struct adapter *adapter,
+ struct pci_params *p)
+{
+ u16 val;
+ u32 pcie_cap = pci_pcie_cap(adapter->pdev);
+
+ if (pcie_cap) {
+ pci_read_config_word(adapter->pdev, pcie_cap + PCI_EXP_LNKSTA,
+ &val);
+ p->speed = val & PCI_EXP_LNKSTA_CLS;
+ p->width = (val & PCI_EXP_LNKSTA_NLW) >> 4;
+ }
+}
+
+/**
+ * init_link_config - initialize a link's SW state
+ * @lc: structure holding the link state
+ * @caps: link capabilities
+ *
+ * Initializes the SW state maintained for each link, including the link's
+ * capabilities and default speed/flow-control/autonegotiation settings.
+ */
+static void __devinit init_link_config(struct link_config *lc,
+ unsigned int caps)
+{
+ lc->supported = caps;
+ lc->requested_speed = 0;
+ lc->speed = 0;
+ lc->requested_fc = lc->fc = PAUSE_RX | PAUSE_TX;
+ if (lc->supported & FW_PORT_CAP_ANEG) {
+ lc->advertising = lc->supported & ADVERT_MASK;
+ lc->autoneg = AUTONEG_ENABLE;
+ lc->requested_fc |= PAUSE_AUTONEG;
+ } else {
+ lc->advertising = 0;
+ lc->autoneg = AUTONEG_DISABLE;
+ }
+}
+
+static int __devinit wait_dev_ready(struct adapter *adap)
+{
+ if (t4_read_reg(adap, PL_WHOAMI) != 0xffffffff)
+ return 0;
+ msleep(500);
+ return t4_read_reg(adap, PL_WHOAMI) != 0xffffffff ? 0 : -EIO;
+}
+
+/**
+ * t4_prep_adapter - prepare SW and HW for operation
+ * @adapter: the adapter
+ * @reset: if true perform a HW reset
+ *
+ * Initialize adapter SW state for the various HW modules, set initial
+ * values for some adapter tunables, take PHYs out of reset, and
+ * initialize the MDIO interface.
+ */
+int __devinit t4_prep_adapter(struct adapter *adapter)
+{
+ int ret;
+
+ ret = wait_dev_ready(adapter);
+ if (ret < 0)
+ return ret;
+
+ get_pci_mode(adapter, &adapter->params.pci);
+ adapter->params.rev = t4_read_reg(adapter, PL_REV);
+
+ ret = get_vpd_params(adapter, &adapter->params.vpd);
+ if (ret < 0)
+ return ret;
+
+ init_cong_ctrl(adapter->params.a_wnd, adapter->params.b_wnd);
+
+ /*
+ * Default port for debugging in case we can't reach FW.
+ */
+ adapter->params.nports = 1;
+ adapter->params.portvec = 1;
+ return 0;
+}
+
+int __devinit t4_port_init(struct adapter *adap, int mbox, int pf, int vf)
+{
+ u8 addr[6];
+ int ret, i, j = 0;
+ struct fw_port_cmd c;
+
+ memset(&c, 0, sizeof(c));
+
+ for_each_port(adap, i) {
+ unsigned int rss_size;
+ struct port_info *p = adap2pinfo(adap, i);
+
+ while ((adap->params.portvec & (1 << j)) == 0)
+ j++;
+
+ c.op_to_portid = htonl(FW_CMD_OP(FW_PORT_CMD) |
+ FW_CMD_REQUEST | FW_CMD_READ |
+ FW_PORT_CMD_PORTID(j));
+ c.action_to_len16 = htonl(
+ FW_PORT_CMD_ACTION(FW_PORT_ACTION_GET_PORT_INFO) |
+ FW_LEN16(c));
+ ret = t4_wr_mbox(adap, mbox, &c, sizeof(c), &c);
+ if (ret)
+ return ret;
+
+ ret = t4_alloc_vi(adap, mbox, j, pf, vf, 1, addr, &rss_size);
+ if (ret < 0)
+ return ret;
+
+ p->viid = ret;
+ p->tx_chan = j;
+ p->lport = j;
+ p->rss_size = rss_size;
+ memcpy(adap->port[i]->dev_addr, addr, ETH_ALEN);
+ memcpy(adap->port[i]->perm_addr, addr, ETH_ALEN);
+
+ ret = ntohl(c.u.info.lstatus_to_modtype);
+ p->mdio_addr = (ret & FW_PORT_CMD_MDIOCAP) ?
+ FW_PORT_CMD_MDIOADDR_GET(ret) : -1;
+ p->port_type = FW_PORT_CMD_PTYPE_GET(ret);
+ p->mod_type = FW_PORT_CMD_MODTYPE_GET(ret);
+
+ init_link_config(&p->link_cfg, ntohs(c.u.info.pcap));
+ j++;
+ }
+ return 0;
+}
--- /dev/null
+/*
+ * This file is part of the Chelsio T4 Ethernet driver for Linux.
+ *
+ * Copyright (c) 2003-2010 Chelsio Communications, Inc. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef __T4_HW_H
+#define __T4_HW_H
+
+#include <linux/types.h>
+
+enum {
+ NCHAN = 4, /* # of HW channels */
+ MAX_MTU = 9600, /* max MAC MTU, excluding header + FCS */
+ EEPROMSIZE = 17408, /* Serial EEPROM physical size */
+ EEPROMVSIZE = 32768, /* Serial EEPROM virtual address space size */
+ RSS_NENTRIES = 2048, /* # of entries in RSS mapping table */
+ TCB_SIZE = 128, /* TCB size */
+ NMTUS = 16, /* size of MTU table */
+ NCCTRL_WIN = 32, /* # of congestion control windows */
+ NEXACT_MAC = 336, /* # of exact MAC address filters */
+ L2T_SIZE = 4096, /* # of L2T entries */
+ MBOX_LEN = 64, /* mailbox size in bytes */
+ TRACE_LEN = 112, /* length of trace data and mask */
+ FILTER_OPT_LEN = 36, /* filter tuple width for optional components */
+ NWOL_PAT = 8, /* # of WoL patterns */
+ WOL_PAT_LEN = 128, /* length of WoL patterns */
+};
+
+enum {
+ SF_PAGE_SIZE = 256, /* serial flash page size */
+ SF_SEC_SIZE = 64 * 1024, /* serial flash sector size */
+ SF_SIZE = SF_SEC_SIZE * 16, /* serial flash size */
+};
+
+enum { RSP_TYPE_FLBUF, RSP_TYPE_CPL, RSP_TYPE_INTR }; /* response entry types */
+
+enum { MBOX_OWNER_NONE, MBOX_OWNER_FW, MBOX_OWNER_DRV }; /* mailbox owners */
+
+enum {
+ SGE_MAX_WR_LEN = 512, /* max WR size in bytes */
+ SGE_NTIMERS = 6, /* # of interrupt holdoff timer values */
+ SGE_NCOUNTERS = 4, /* # of interrupt packet counter values */
+};
+
+struct sge_qstat { /* data written to SGE queue status entries */
+ __be32 qid;
+ __be16 cidx;
+ __be16 pidx;
+};
+
+/*
+ * Structure for last 128 bits of response descriptors
+ */
+struct rsp_ctrl {
+ __be32 hdrbuflen_pidx;
+ __be32 pldbuflen_qid;
+ union {
+ u8 type_gen;
+ __be64 last_flit;
+ };
+};
+
+#define RSPD_NEWBUF 0x80000000U
+#define RSPD_LEN 0x7fffffffU
+
+#define RSPD_GEN(x) ((x) >> 7)
+#define RSPD_TYPE(x) (((x) >> 4) & 3)
+
+#define QINTR_CNT_EN 0x1
+#define QINTR_TIMER_IDX(x) ((x) << 1)
+#endif /* __T4_HW_H */
--- /dev/null
+/*
+ * This file is part of the Chelsio T4 Ethernet driver for Linux.
+ *
+ * Copyright (c) 2003-2010 Chelsio Communications, Inc. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef __T4_MSG_H
+#define __T4_MSG_H
+
+#include <linux/types.h>
+
+enum {
+ CPL_PASS_OPEN_REQ = 0x1,
+ CPL_PASS_ACCEPT_RPL = 0x2,
+ CPL_ACT_OPEN_REQ = 0x3,
+ CPL_SET_TCB_FIELD = 0x5,
+ CPL_GET_TCB = 0x6,
+ CPL_CLOSE_CON_REQ = 0x8,
+ CPL_CLOSE_LISTSRV_REQ = 0x9,
+ CPL_ABORT_REQ = 0xA,
+ CPL_ABORT_RPL = 0xB,
+ CPL_RX_DATA_ACK = 0xD,
+ CPL_TX_PKT = 0xE,
+ CPL_L2T_WRITE_REQ = 0x12,
+ CPL_TID_RELEASE = 0x1A,
+
+ CPL_CLOSE_LISTSRV_RPL = 0x20,
+ CPL_L2T_WRITE_RPL = 0x23,
+ CPL_PASS_OPEN_RPL = 0x24,
+ CPL_ACT_OPEN_RPL = 0x25,
+ CPL_PEER_CLOSE = 0x26,
+ CPL_ABORT_REQ_RSS = 0x2B,
+ CPL_ABORT_RPL_RSS = 0x2D,
+
+ CPL_CLOSE_CON_RPL = 0x32,
+ CPL_ISCSI_HDR = 0x33,
+ CPL_RDMA_CQE = 0x35,
+ CPL_RDMA_CQE_READ_RSP = 0x36,
+ CPL_RDMA_CQE_ERR = 0x37,
+ CPL_RX_DATA = 0x39,
+ CPL_SET_TCB_RPL = 0x3A,
+ CPL_RX_PKT = 0x3B,
+ CPL_RX_DDP_COMPLETE = 0x3F,
+
+ CPL_ACT_ESTABLISH = 0x40,
+ CPL_PASS_ESTABLISH = 0x41,
+ CPL_RX_DATA_DDP = 0x42,
+ CPL_PASS_ACCEPT_REQ = 0x44,
+
+ CPL_RDMA_READ_REQ = 0x60,
+
+ CPL_PASS_OPEN_REQ6 = 0x81,
+ CPL_ACT_OPEN_REQ6 = 0x83,
+
+ CPL_RDMA_TERMINATE = 0xA2,
+ CPL_RDMA_WRITE = 0xA4,
+ CPL_SGE_EGR_UPDATE = 0xA5,
+
+ CPL_TRACE_PKT = 0xB0,
+
+ CPL_FW4_MSG = 0xC0,
+ CPL_FW4_PLD = 0xC1,
+ CPL_FW4_ACK = 0xC3,
+
+ CPL_FW6_MSG = 0xE0,
+ CPL_FW6_PLD = 0xE1,
+ CPL_TX_PKT_LSO = 0xED,
+ CPL_TX_PKT_XT = 0xEE,
+
+ NUM_CPL_CMDS
+};
+
+enum CPL_error {
+ CPL_ERR_NONE = 0,
+ CPL_ERR_TCAM_FULL = 3,
+ CPL_ERR_BAD_LENGTH = 15,
+ CPL_ERR_BAD_ROUTE = 18,
+ CPL_ERR_CONN_RESET = 20,
+ CPL_ERR_CONN_EXIST_SYNRECV = 21,
+ CPL_ERR_CONN_EXIST = 22,
+ CPL_ERR_ARP_MISS = 23,
+ CPL_ERR_BAD_SYN = 24,
+ CPL_ERR_CONN_TIMEDOUT = 30,
+ CPL_ERR_XMIT_TIMEDOUT = 31,
+ CPL_ERR_PERSIST_TIMEDOUT = 32,
+ CPL_ERR_FINWAIT2_TIMEDOUT = 33,
+ CPL_ERR_KEEPALIVE_TIMEDOUT = 34,
+ CPL_ERR_RTX_NEG_ADVICE = 35,
+ CPL_ERR_PERSIST_NEG_ADVICE = 36,
+ CPL_ERR_ABORT_FAILED = 42,
+ CPL_ERR_IWARP_FLM = 50,
+};
+
+enum {
+ ULP_MODE_NONE = 0,
+ ULP_MODE_ISCSI = 2,
+ ULP_MODE_RDMA = 4,
+ ULP_MODE_FCOE = 6,
+};
+
+enum {
+ ULP_CRC_HEADER = 1 << 0,
+ ULP_CRC_DATA = 1 << 1
+};
+
+enum {
+ CPL_ABORT_SEND_RST = 0,
+ CPL_ABORT_NO_RST,
+};
+
+enum { /* TX_PKT_XT checksum types */
+ TX_CSUM_TCP = 0,
+ TX_CSUM_UDP = 1,
+ TX_CSUM_CRC16 = 4,
+ TX_CSUM_CRC32 = 5,
+ TX_CSUM_CRC32C = 6,
+ TX_CSUM_FCOE = 7,
+ TX_CSUM_TCPIP = 8,
+ TX_CSUM_UDPIP = 9,
+ TX_CSUM_TCPIP6 = 10,
+ TX_CSUM_UDPIP6 = 11,
+ TX_CSUM_IP = 12,
+};
+
+union opcode_tid {
+ __be32 opcode_tid;
+ u8 opcode;
+};
+
+#define CPL_OPCODE(x) ((x) << 24)
+#define MK_OPCODE_TID(opcode, tid) (CPL_OPCODE(opcode) | (tid))
+#define OPCODE_TID(cmd) ((cmd)->ot.opcode_tid)
+#define GET_TID(cmd) (ntohl(OPCODE_TID(cmd)) & 0xFFFFFF)
+
+/* partitioning of TID fields that also carry a queue id */
+#define GET_TID_TID(x) ((x) & 0x3fff)
+#define GET_TID_QID(x) (((x) >> 14) & 0x3ff)
+#define TID_QID(x) ((x) << 14)
+
+struct rss_header {
+ u8 opcode;
+#if defined(__LITTLE_ENDIAN_BITFIELD)
+ u8 channel:2;
+ u8 filter_hit:1;
+ u8 filter_tid:1;
+ u8 hash_type:2;
+ u8 ipv6:1;
+ u8 send2fw:1;
+#else
+ u8 send2fw:1;
+ u8 ipv6:1;
+ u8 hash_type:2;
+ u8 filter_tid:1;
+ u8 filter_hit:1;
+ u8 channel:2;
+#endif
+ __be16 qid;
+ __be32 hash_val;
+};
+
+struct work_request_hdr {
+ __be32 wr_hi;
+ __be32 wr_mid;
+ __be64 wr_lo;
+};
+
+#define WR_HDR struct work_request_hdr wr
+
+struct cpl_pass_open_req {
+ WR_HDR;
+ union opcode_tid ot;
+ __be16 local_port;
+ __be16 peer_port;
+ __be32 local_ip;
+ __be32 peer_ip;
+ __be64 opt0;
+#define TX_CHAN(x) ((x) << 2)
+#define DELACK(x) ((x) << 5)
+#define ULP_MODE(x) ((x) << 8)
+#define RCV_BUFSIZ(x) ((x) << 12)
+#define DSCP(x) ((x) << 22)
+#define SMAC_SEL(x) ((u64)(x) << 28)
+#define L2T_IDX(x) ((u64)(x) << 36)
+#define NAGLE(x) ((u64)(x) << 49)
+#define WND_SCALE(x) ((u64)(x) << 50)
+#define KEEP_ALIVE(x) ((u64)(x) << 54)
+#define MSS_IDX(x) ((u64)(x) << 60)
+ __be64 opt1;
+#define SYN_RSS_ENABLE (1 << 0)
+#define SYN_RSS_QUEUE(x) ((x) << 2)
+#define CONN_POLICY_ASK (1 << 22)
+};
+
+struct cpl_pass_open_req6 {
+ WR_HDR;
+ union opcode_tid ot;
+ __be16 local_port;
+ __be16 peer_port;
+ __be64 local_ip_hi;
+ __be64 local_ip_lo;
+ __be64 peer_ip_hi;
+ __be64 peer_ip_lo;
+ __be64 opt0;
+ __be64 opt1;
+};
+
+struct cpl_pass_open_rpl {
+ union opcode_tid ot;
+ u8 rsvd[3];
+ u8 status;
+};
+
+struct cpl_pass_accept_rpl {
+ WR_HDR;
+ union opcode_tid ot;
+ __be32 opt2;
+#define RSS_QUEUE(x) ((x) << 0)
+#define RSS_QUEUE_VALID (1 << 10)
+#define RX_COALESCE_VALID(x) ((x) << 11)
+#define RX_COALESCE(x) ((x) << 12)
+#define TX_QUEUE(x) ((x) << 23)
+#define RX_CHANNEL(x) ((x) << 26)
+#define WND_SCALE_EN(x) ((x) << 28)
+#define TSTAMPS_EN(x) ((x) << 29)
+#define SACK_EN(x) ((x) << 30)
+ __be64 opt0;
+};
+
+struct cpl_act_open_req {
+ WR_HDR;
+ union opcode_tid ot;
+ __be16 local_port;
+ __be16 peer_port;
+ __be32 local_ip;
+ __be32 peer_ip;
+ __be64 opt0;
+ __be32 params;
+ __be32 opt2;
+};
+
+struct cpl_act_open_req6 {
+ WR_HDR;
+ union opcode_tid ot;
+ __be16 local_port;
+ __be16 peer_port;
+ __be64 local_ip_hi;
+ __be64 local_ip_lo;
+ __be64 peer_ip_hi;
+ __be64 peer_ip_lo;
+ __be64 opt0;
+ __be32 params;
+ __be32 opt2;
+};
+
+struct cpl_act_open_rpl {
+ union opcode_tid ot;
+ __be32 atid_status;
+#define GET_AOPEN_STATUS(x) ((x) & 0xff)
+#define GET_AOPEN_ATID(x) (((x) >> 8) & 0xffffff)
+};
+
+struct cpl_pass_establish {
+ union opcode_tid ot;
+ __be32 rsvd;
+ __be32 tos_stid;
+#define GET_POPEN_TID(x) ((x) & 0xffffff)
+#define GET_POPEN_TOS(x) (((x) >> 24) & 0xff)
+ __be16 mac_idx;
+ __be16 tcp_opt;
+#define GET_TCPOPT_WSCALE_OK(x) (((x) >> 5) & 1)
+#define GET_TCPOPT_SACK(x) (((x) >> 6) & 1)
+#define GET_TCPOPT_TSTAMP(x) (((x) >> 7) & 1)
+#define GET_TCPOPT_SND_WSCALE(x) (((x) >> 8) & 0xf)
+#define GET_TCPOPT_MSS(x) (((x) >> 12) & 0xf)
+ __be32 snd_isn;
+ __be32 rcv_isn;
+};
+
+struct cpl_act_establish {
+ union opcode_tid ot;
+ __be32 rsvd;
+ __be32 tos_atid;
+ __be16 mac_idx;
+ __be16 tcp_opt;
+ __be32 snd_isn;
+ __be32 rcv_isn;
+};
+
+struct cpl_get_tcb {
+ WR_HDR;
+ union opcode_tid ot;
+ __be16 reply_ctrl;
+#define QUEUENO(x) ((x) << 0)
+#define REPLY_CHAN(x) ((x) << 14)
+#define NO_REPLY(x) ((x) << 15)
+ __be16 cookie;
+};
+
+struct cpl_set_tcb_field {
+ WR_HDR;
+ union opcode_tid ot;
+ __be16 reply_ctrl;
+ __be16 word_cookie;
+#define TCB_WORD(x) ((x) << 0)
+#define TCB_COOKIE(x) ((x) << 5)
+ __be64 mask;
+ __be64 val;
+};
+
+struct cpl_set_tcb_rpl {
+ union opcode_tid ot;
+ __be16 rsvd;
+ u8 cookie;
+ u8 status;
+ __be64 oldval;
+};
+
+struct cpl_close_con_req {
+ WR_HDR;
+ union opcode_tid ot;
+ __be32 rsvd;
+};
+
+struct cpl_close_con_rpl {
+ union opcode_tid ot;
+ u8 rsvd[3];
+ u8 status;
+ __be32 snd_nxt;
+ __be32 rcv_nxt;
+};
+
+struct cpl_close_listsvr_req {
+ WR_HDR;
+ union opcode_tid ot;
+ __be16 reply_ctrl;
+#define LISTSVR_IPV6 (1 << 14)
+ __be16 rsvd;
+};
+
+struct cpl_close_listsvr_rpl {
+ union opcode_tid ot;
+ u8 rsvd[3];
+ u8 status;
+};
+
+struct cpl_abort_req_rss {
+ union opcode_tid ot;
+ u8 rsvd[3];
+ u8 status;
+};
+
+struct cpl_abort_req {
+ WR_HDR;
+ union opcode_tid ot;
+ __be32 rsvd0;
+ u8 rsvd1;
+ u8 cmd;
+ u8 rsvd2[6];
+};
+
+struct cpl_abort_rpl_rss {
+ union opcode_tid ot;
+ u8 rsvd[3];
+ u8 status;
+};
+
+struct cpl_abort_rpl {
+ WR_HDR;
+ union opcode_tid ot;
+ __be32 rsvd0;
+ u8 rsvd1;
+ u8 cmd;
+ u8 rsvd2[6];
+};
+
+struct cpl_peer_close {
+ union opcode_tid ot;
+ __be32 rcv_nxt;
+};
+
+struct cpl_tid_release {
+ WR_HDR;
+ union opcode_tid ot;
+ __be32 rsvd;
+};
+
+struct cpl_tx_pkt_core {
+ __be32 ctrl0;
+#define TXPKT_VF(x) ((x) << 0)
+#define TXPKT_PF(x) ((x) << 8)
+#define TXPKT_VF_VLD (1 << 11)
+#define TXPKT_OVLAN_IDX(x) ((x) << 12)
+#define TXPKT_INTF(x) ((x) << 16)
+#define TXPKT_INS_OVLAN (1 << 21)
+#define TXPKT_OPCODE(x) ((x) << 24)
+ __be16 pack;
+ __be16 len;
+ __be64 ctrl1;
+#define TXPKT_CSUM_END(x) ((x) << 12)
+#define TXPKT_CSUM_START(x) ((x) << 20)
+#define TXPKT_IPHDR_LEN(x) ((u64)(x) << 20)
+#define TXPKT_CSUM_LOC(x) ((u64)(x) << 30)
+#define TXPKT_ETHHDR_LEN(x) ((u64)(x) << 34)
+#define TXPKT_CSUM_TYPE(x) ((u64)(x) << 40)
+#define TXPKT_VLAN(x) ((u64)(x) << 44)
+#define TXPKT_VLAN_VLD (1ULL << 60)
+#define TXPKT_IPCSUM_DIS (1ULL << 62)
+#define TXPKT_L4CSUM_DIS (1ULL << 63)
+};
+
+struct cpl_tx_pkt {
+ WR_HDR;
+ struct cpl_tx_pkt_core c;
+};
+
+#define cpl_tx_pkt_xt cpl_tx_pkt
+
+struct cpl_tx_pkt_lso {
+ WR_HDR;
+ __be32 lso_ctrl;
+#define LSO_TCPHDR_LEN(x) ((x) << 0)
+#define LSO_IPHDR_LEN(x) ((x) << 4)
+#define LSO_ETHHDR_LEN(x) ((x) << 16)
+#define LSO_IPV6(x) ((x) << 20)
+#define LSO_LAST_SLICE (1 << 22)
+#define LSO_FIRST_SLICE (1 << 23)
+#define LSO_OPCODE(x) ((x) << 24)
+ __be16 ipid_ofst;
+ __be16 mss;
+ __be32 seqno_offset;
+ __be32 len;
+ /* encapsulated CPL (TX_PKT, TX_PKT_XT or TX_DATA) follows here */
+};
+
+struct cpl_iscsi_hdr {
+ union opcode_tid ot;
+ __be16 pdu_len_ddp;
+#define ISCSI_PDU_LEN(x) ((x) & 0x7FFF)
+#define ISCSI_DDP (1 << 15)
+ __be16 len;
+ __be32 seq;
+ __be16 urg;
+ u8 rsvd;
+ u8 status;
+};
+
+struct cpl_rx_data {
+ union opcode_tid ot;
+ __be16 rsvd;
+ __be16 len;
+ __be32 seq;
+ __be16 urg;
+#if defined(__LITTLE_ENDIAN_BITFIELD)
+ u8 dack_mode:2;
+ u8 psh:1;
+ u8 heartbeat:1;
+ u8 ddp_off:1;
+ u8 :3;
+#else
+ u8 :3;
+ u8 ddp_off:1;
+ u8 heartbeat:1;
+ u8 psh:1;
+ u8 dack_mode:2;
+#endif
+ u8 status;
+};
+
+struct cpl_rx_data_ack {
+ WR_HDR;
+ union opcode_tid ot;
+ __be32 credit_dack;
+#define RX_CREDITS(x) ((x) << 0)
+#define RX_FORCE_ACK(x) ((x) << 28)
+};
+
+struct cpl_rx_pkt {
+ u8 opcode;
+#if defined(__LITTLE_ENDIAN_BITFIELD)
+ u8 iff:4;
+ u8 csum_calc:1;
+ u8 ipmi_pkt:1;
+ u8 vlan_ex:1;
+ u8 ip_frag:1;
+#else
+ u8 ip_frag:1;
+ u8 vlan_ex:1;
+ u8 ipmi_pkt:1;
+ u8 csum_calc:1;
+ u8 iff:4;
+#endif
+ __be16 csum;
+ __be16 vlan;
+ __be16 len;
+ __be32 l2info;
+#define RXF_UDP (1 << 22)
+#define RXF_TCP (1 << 23)
+ __be16 hdr_len;
+ __be16 err_vec;
+};
+
+struct cpl_trace_pkt {
+ u8 opcode;
+ u8 intf;
+#if defined(__LITTLE_ENDIAN_BITFIELD)
+ u8 runt:4;
+ u8 filter_hit:4;
+ u8 :6;
+ u8 err:1;
+ u8 trunc:1;
+#else
+ u8 filter_hit:4;
+ u8 runt:4;
+ u8 trunc:1;
+ u8 err:1;
+ u8 :6;
+#endif
+ __be16 rsvd;
+ __be16 len;
+ __be64 tstamp;
+};
+
+struct cpl_l2t_write_req {
+ WR_HDR;
+ union opcode_tid ot;
+ __be16 params;
+#define L2T_W_INFO(x) ((x) << 2)
+#define L2T_W_PORT(x) ((x) << 8)
+#define L2T_W_NOREPLY(x) ((x) << 15)
+ __be16 l2t_idx;
+ __be16 vlan;
+ u8 dst_mac[6];
+};
+
+struct cpl_l2t_write_rpl {
+ union opcode_tid ot;
+ u8 status;
+ u8 rsvd[3];
+};
+
+struct cpl_rdma_terminate {
+ union opcode_tid ot;
+ __be16 rsvd;
+ __be16 len;
+};
+
+struct cpl_sge_egr_update {
+ __be32 opcode_qid;
+#define EGR_QID(x) ((x) & 0x1FFFF)
+ __be16 cidx;
+ __be16 pidx;
+};
+
+struct cpl_fw4_pld {
+ u8 opcode;
+ u8 rsvd0[3];
+ u8 type;
+ u8 rsvd1;
+ __be16 len;
+ __be64 data;
+ __be64 rsvd2;
+};
+
+struct cpl_fw6_pld {
+ u8 opcode;
+ u8 rsvd[5];
+ __be16 len;
+ __be64 data[4];
+};
+
+struct cpl_fw4_msg {
+ u8 opcode;
+ u8 type;
+ __be16 rsvd0;
+ __be32 rsvd1;
+ __be64 data[2];
+};
+
+struct cpl_fw4_ack {
+ union opcode_tid ot;
+ u8 credits;
+ u8 rsvd0[2];
+ u8 seq_vld;
+ __be32 snd_nxt;
+ __be32 snd_una;
+ __be64 rsvd1;
+};
+
+struct cpl_fw6_msg {
+ u8 opcode;
+ u8 type;
+ __be16 rsvd0;
+ __be32 rsvd1;
+ __be64 data[4];
+};
+
+enum {
+ ULP_TX_MEM_READ = 2,
+ ULP_TX_MEM_WRITE = 3,
+ ULP_TX_PKT = 4
+};
+
+enum {
+ ULP_TX_SC_NOOP = 0x80,
+ ULP_TX_SC_IMM = 0x81,
+ ULP_TX_SC_DSGL = 0x82,
+ ULP_TX_SC_ISGL = 0x83
+};
+
+struct ulptx_sge_pair {
+ __be32 len[2];
+ __be64 addr[2];
+};
+
+struct ulptx_sgl {
+ __be32 cmd_nsge;
+#define ULPTX_CMD(x) ((x) << 24)
+#define ULPTX_NSGE(x) ((x) << 0)
+ __be32 len0;
+ __be64 addr0;
+ struct ulptx_sge_pair sge[0];
+};
+
+struct ulp_mem_io {
+ WR_HDR;
+ __be32 cmd;
+#define ULP_MEMIO_ORDER(x) ((x) << 23)
+ __be32 len16; /* command length */
+ __be32 dlen; /* data length in 32-byte units */
+#define ULP_MEMIO_DATA_LEN(x) ((x) << 0)
+ __be32 lock_addr;
+#define ULP_MEMIO_ADDR(x) ((x) << 0)
+#define ULP_MEMIO_LOCK(x) ((x) << 31)
+};
+
+#endif /* __T4_MSG_H */
--- /dev/null
+/*
+ * This file is part of the Chelsio T4 Ethernet driver for Linux.
+ *
+ * Copyright (c) 2010 Chelsio Communications, Inc. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef __T4_REGS_H
+#define __T4_REGS_H
+
+#define MYPF_BASE 0x1b000
+#define MYPF_REG(reg_addr) (MYPF_BASE + (reg_addr))
+
+#define PF0_BASE 0x1e000
+#define PF0_REG(reg_addr) (PF0_BASE + (reg_addr))
+
+#define PF_STRIDE 0x400
+#define PF_BASE(idx) (PF0_BASE + (idx) * PF_STRIDE)
+#define PF_REG(idx, reg) (PF_BASE(idx) + (reg))
+
+#define MYPORT_BASE 0x1c000
+#define MYPORT_REG(reg_addr) (MYPORT_BASE + (reg_addr))
+
+#define PORT0_BASE 0x20000
+#define PORT0_REG(reg_addr) (PORT0_BASE + (reg_addr))
+
+#define PORT_STRIDE 0x2000
+#define PORT_BASE(idx) (PORT0_BASE + (idx) * PORT_STRIDE)
+#define PORT_REG(idx, reg) (PORT_BASE(idx) + (reg))
+
+#define EDC_STRIDE (EDC_1_BASE_ADDR - EDC_0_BASE_ADDR)
+#define EDC_REG(reg, idx) (reg + EDC_STRIDE * idx)
+
+#define PCIE_MEM_ACCESS_REG(reg_addr, idx) ((reg_addr) + (idx) * 8)
+#define PCIE_MAILBOX_REG(reg_addr, idx) ((reg_addr) + (idx) * 8)
+#define MC_BIST_STATUS_REG(reg_addr, idx) ((reg_addr) + (idx) * 4)
+#define EDC_BIST_STATUS_REG(reg_addr, idx) ((reg_addr) + (idx) * 4)
+
+#define SGE_PF_KDOORBELL 0x0
+#define QID_MASK 0xffff8000U
+#define QID_SHIFT 15
+#define QID(x) ((x) << QID_SHIFT)
+#define DBPRIO 0x00004000U
+#define PIDX_MASK 0x00003fffU
+#define PIDX_SHIFT 0
+#define PIDX(x) ((x) << PIDX_SHIFT)
+
+#define SGE_PF_GTS 0x4
+#define INGRESSQID_MASK 0xffff0000U
+#define INGRESSQID_SHIFT 16
+#define INGRESSQID(x) ((x) << INGRESSQID_SHIFT)
+#define TIMERREG_MASK 0x0000e000U
+#define TIMERREG_SHIFT 13
+#define TIMERREG(x) ((x) << TIMERREG_SHIFT)
+#define SEINTARM_MASK 0x00001000U
+#define SEINTARM_SHIFT 12
+#define SEINTARM(x) ((x) << SEINTARM_SHIFT)
+#define CIDXINC_MASK 0x00000fffU
+#define CIDXINC_SHIFT 0
+#define CIDXINC(x) ((x) << CIDXINC_SHIFT)
+
+#define SGE_CONTROL 0x1008
+#define DCASYSTYPE 0x00080000U
+#define RXPKTCPLMODE 0x00040000U
+#define EGRSTATUSPAGESIZE 0x00020000U
+#define PKTSHIFT_MASK 0x00001c00U
+#define PKTSHIFT_SHIFT 10
+#define PKTSHIFT(x) ((x) << PKTSHIFT_SHIFT)
+#define INGPCIEBOUNDARY_MASK 0x00000380U
+#define INGPCIEBOUNDARY_SHIFT 7
+#define INGPCIEBOUNDARY(x) ((x) << INGPCIEBOUNDARY_SHIFT)
+#define INGPADBOUNDARY_MASK 0x00000070U
+#define INGPADBOUNDARY_SHIFT 4
+#define INGPADBOUNDARY(x) ((x) << INGPADBOUNDARY_SHIFT)
+#define EGRPCIEBOUNDARY_MASK 0x0000000eU
+#define EGRPCIEBOUNDARY_SHIFT 1
+#define EGRPCIEBOUNDARY(x) ((x) << EGRPCIEBOUNDARY_SHIFT)
+#define GLOBALENABLE 0x00000001U
+
+#define SGE_HOST_PAGE_SIZE 0x100c
+#define HOSTPAGESIZEPF0_MASK 0x0000000fU
+#define HOSTPAGESIZEPF0_SHIFT 0
+#define HOSTPAGESIZEPF0(x) ((x) << HOSTPAGESIZEPF0_SHIFT)
+
+#define SGE_EGRESS_QUEUES_PER_PAGE_PF 0x1010
+#define QUEUESPERPAGEPF0_MASK 0x0000000fU
+#define QUEUESPERPAGEPF0_GET(x) ((x) & QUEUESPERPAGEPF0_MASK)
+
+#define SGE_INT_CAUSE1 0x1024
+#define SGE_INT_CAUSE2 0x1030
+#define SGE_INT_CAUSE3 0x103c
+#define ERR_FLM_DBP 0x80000000U
+#define ERR_FLM_IDMA1 0x40000000U
+#define ERR_FLM_IDMA0 0x20000000U
+#define ERR_FLM_HINT 0x10000000U
+#define ERR_PCIE_ERROR3 0x08000000U
+#define ERR_PCIE_ERROR2 0x04000000U
+#define ERR_PCIE_ERROR1 0x02000000U
+#define ERR_PCIE_ERROR0 0x01000000U
+#define ERR_TIMER_ABOVE_MAX_QID 0x00800000U
+#define ERR_CPL_EXCEED_IQE_SIZE 0x00400000U
+#define ERR_INVALID_CIDX_INC 0x00200000U
+#define ERR_ITP_TIME_PAUSED 0x00100000U
+#define ERR_CPL_OPCODE_0 0x00080000U
+#define ERR_DROPPED_DB 0x00040000U
+#define ERR_DATA_CPL_ON_HIGH_QID1 0x00020000U
+#define ERR_DATA_CPL_ON_HIGH_QID0 0x00010000U
+#define ERR_BAD_DB_PIDX3 0x00008000U
+#define ERR_BAD_DB_PIDX2 0x00004000U
+#define ERR_BAD_DB_PIDX1 0x00002000U
+#define ERR_BAD_DB_PIDX0 0x00001000U
+#define ERR_ING_PCIE_CHAN 0x00000800U
+#define ERR_ING_CTXT_PRIO 0x00000400U
+#define ERR_EGR_CTXT_PRIO 0x00000200U
+#define DBFIFO_HP_INT 0x00000100U
+#define DBFIFO_LP_INT 0x00000080U
+#define REG_ADDRESS_ERR 0x00000040U
+#define INGRESS_SIZE_ERR 0x00000020U
+#define EGRESS_SIZE_ERR 0x00000010U
+#define ERR_INV_CTXT3 0x00000008U
+#define ERR_INV_CTXT2 0x00000004U
+#define ERR_INV_CTXT1 0x00000002U
+#define ERR_INV_CTXT0 0x00000001U
+
+#define SGE_INT_ENABLE3 0x1040
+#define SGE_FL_BUFFER_SIZE0 0x1044
+#define SGE_FL_BUFFER_SIZE1 0x1048
+#define SGE_INGRESS_RX_THRESHOLD 0x10a0
+#define THRESHOLD_0_MASK 0x3f000000U
+#define THRESHOLD_0_SHIFT 24
+#define THRESHOLD_0(x) ((x) << THRESHOLD_0_SHIFT)
+#define THRESHOLD_0_GET(x) (((x) & THRESHOLD_0_MASK) >> THRESHOLD_0_SHIFT)
+#define THRESHOLD_1_MASK 0x003f0000U
+#define THRESHOLD_1_SHIFT 16
+#define THRESHOLD_1(x) ((x) << THRESHOLD_1_SHIFT)
+#define THRESHOLD_1_GET(x) (((x) & THRESHOLD_1_MASK) >> THRESHOLD_1_SHIFT)
+#define THRESHOLD_2_MASK 0x00003f00U
+#define THRESHOLD_2_SHIFT 8
+#define THRESHOLD_2(x) ((x) << THRESHOLD_2_SHIFT)
+#define THRESHOLD_2_GET(x) (((x) & THRESHOLD_2_MASK) >> THRESHOLD_2_SHIFT)
+#define THRESHOLD_3_MASK 0x0000003fU
+#define THRESHOLD_3_SHIFT 0
+#define THRESHOLD_3(x) ((x) << THRESHOLD_3_SHIFT)
+#define THRESHOLD_3_GET(x) (((x) & THRESHOLD_3_MASK) >> THRESHOLD_3_SHIFT)
+
+#define SGE_TIMER_VALUE_0_AND_1 0x10b8
+#define TIMERVALUE0_MASK 0xffff0000U
+#define TIMERVALUE0_SHIFT 16
+#define TIMERVALUE0(x) ((x) << TIMERVALUE0_SHIFT)
+#define TIMERVALUE0_GET(x) (((x) & TIMERVALUE0_MASK) >> TIMERVALUE0_SHIFT)
+#define TIMERVALUE1_MASK 0x0000ffffU
+#define TIMERVALUE1_SHIFT 0
+#define TIMERVALUE1(x) ((x) << TIMERVALUE1_SHIFT)
+#define TIMERVALUE1_GET(x) (((x) & TIMERVALUE1_MASK) >> TIMERVALUE1_SHIFT)
+
+#define SGE_TIMER_VALUE_2_AND_3 0x10bc
+#define SGE_TIMER_VALUE_4_AND_5 0x10c0
+#define SGE_DEBUG_INDEX 0x10cc
+#define SGE_DEBUG_DATA_HIGH 0x10d0
+#define SGE_DEBUG_DATA_LOW 0x10d4
+#define SGE_INGRESS_QUEUES_PER_PAGE_PF 0x10f4
+
+#define PCIE_PF_CLI 0x44
+#define PCIE_INT_CAUSE 0x3004
+#define UNXSPLCPLERR 0x20000000U
+#define PCIEPINT 0x10000000U
+#define PCIESINT 0x08000000U
+#define RPLPERR 0x04000000U
+#define RXWRPERR 0x02000000U
+#define RXCPLPERR 0x01000000U
+#define PIOTAGPERR 0x00800000U
+#define MATAGPERR 0x00400000U
+#define INTXCLRPERR 0x00200000U
+#define FIDPERR 0x00100000U
+#define CFGSNPPERR 0x00080000U
+#define HRSPPERR 0x00040000U
+#define HREQPERR 0x00020000U
+#define HCNTPERR 0x00010000U
+#define DRSPPERR 0x00008000U
+#define DREQPERR 0x00004000U
+#define DCNTPERR 0x00002000U
+#define CRSPPERR 0x00001000U
+#define CREQPERR 0x00000800U
+#define CCNTPERR 0x00000400U
+#define TARTAGPERR 0x00000200U
+#define PIOREQPERR 0x00000100U
+#define PIOCPLPERR 0x00000080U
+#define MSIXDIPERR 0x00000040U
+#define MSIXDATAPERR 0x00000020U
+#define MSIXADDRHPERR 0x00000010U
+#define MSIXADDRLPERR 0x00000008U
+#define MSIDATAPERR 0x00000004U
+#define MSIADDRHPERR 0x00000002U
+#define MSIADDRLPERR 0x00000001U
+
+#define PCIE_NONFAT_ERR 0x3010
+#define PCIE_MEM_ACCESS_BASE_WIN 0x3068
+#define PCIEOFST_MASK 0xfffffc00U
+#define BIR_MASK 0x00000300U
+#define BIR_SHIFT 8
+#define BIR(x) ((x) << BIR_SHIFT)
+#define WINDOW_MASK 0x000000ffU
+#define WINDOW_SHIFT 0
+#define WINDOW(x) ((x) << WINDOW_SHIFT)
+
+#define PCIE_CORE_UTL_SYSTEM_BUS_AGENT_STATUS 0x5908
+#define RNPP 0x80000000U
+#define RPCP 0x20000000U
+#define RCIP 0x08000000U
+#define RCCP 0x04000000U
+#define RFTP 0x00800000U
+#define PTRP 0x00100000U
+
+#define PCIE_CORE_UTL_PCI_EXPRESS_PORT_STATUS 0x59a4
+#define TPCP 0x40000000U
+#define TNPP 0x20000000U
+#define TFTP 0x10000000U
+#define TCAP 0x08000000U
+#define TCIP 0x04000000U
+#define RCAP 0x02000000U
+#define PLUP 0x00800000U
+#define PLDN 0x00400000U
+#define OTDD 0x00200000U
+#define GTRP 0x00100000U
+#define RDPE 0x00040000U
+#define TDCE 0x00020000U
+#define TDUE 0x00010000U
+
+#define MC_INT_CAUSE 0x7518
+#define ECC_UE_INT_CAUSE 0x00000004U
+#define ECC_CE_INT_CAUSE 0x00000002U
+#define PERR_INT_CAUSE 0x00000001U
+
+#define MC_ECC_STATUS 0x751c
+#define ECC_CECNT_MASK 0xffff0000U
+#define ECC_CECNT_SHIFT 16
+#define ECC_CECNT(x) ((x) << ECC_CECNT_SHIFT)
+#define ECC_CECNT_GET(x) (((x) & ECC_CECNT_MASK) >> ECC_CECNT_SHIFT)
+#define ECC_UECNT_MASK 0x0000ffffU
+#define ECC_UECNT_SHIFT 0
+#define ECC_UECNT(x) ((x) << ECC_UECNT_SHIFT)
+#define ECC_UECNT_GET(x) (((x) & ECC_UECNT_MASK) >> ECC_UECNT_SHIFT)
+
+#define MC_BIST_CMD 0x7600
+#define START_BIST 0x80000000U
+#define BIST_CMD_GAP_MASK 0x0000ff00U
+#define BIST_CMD_GAP_SHIFT 8
+#define BIST_CMD_GAP(x) ((x) << BIST_CMD_GAP_SHIFT)
+#define BIST_OPCODE_MASK 0x00000003U
+#define BIST_OPCODE_SHIFT 0
+#define BIST_OPCODE(x) ((x) << BIST_OPCODE_SHIFT)
+
+#define MC_BIST_CMD_ADDR 0x7604
+#define MC_BIST_CMD_LEN 0x7608
+#define MC_BIST_DATA_PATTERN 0x760c
+#define BIST_DATA_TYPE_MASK 0x0000000fU
+#define BIST_DATA_TYPE_SHIFT 0
+#define BIST_DATA_TYPE(x) ((x) << BIST_DATA_TYPE_SHIFT)
+
+#define MC_BIST_STATUS_RDATA 0x7688
+
+#define MA_EXT_MEMORY_BAR 0x77c8
+#define EXT_MEM_SIZE_MASK 0x00000fffU
+#define EXT_MEM_SIZE_SHIFT 0
+#define EXT_MEM_SIZE_GET(x) (((x) & EXT_MEM_SIZE_MASK) >> EXT_MEM_SIZE_SHIFT)
+
+#define MA_TARGET_MEM_ENABLE 0x77d8
+#define EXT_MEM_ENABLE 0x00000004U
+#define EDRAM1_ENABLE 0x00000002U
+#define EDRAM0_ENABLE 0x00000001U
+
+#define MA_INT_CAUSE 0x77e0
+#define MEM_PERR_INT_CAUSE 0x00000002U
+#define MEM_WRAP_INT_CAUSE 0x00000001U
+
+#define MA_INT_WRAP_STATUS 0x77e4
+#define MEM_WRAP_ADDRESS_MASK 0xfffffff0U
+#define MEM_WRAP_ADDRESS_SHIFT 4
+#define MEM_WRAP_ADDRESS_GET(x) (((x) & MEM_WRAP_ADDRESS_MASK) >> MEM_WRAP_ADDRESS_SHIFT)
+#define MEM_WRAP_CLIENT_NUM_MASK 0x0000000fU
+#define MEM_WRAP_CLIENT_NUM_SHIFT 0
+#define MEM_WRAP_CLIENT_NUM_GET(x) (((x) & MEM_WRAP_CLIENT_NUM_MASK) >> MEM_WRAP_CLIENT_NUM_SHIFT)
+
+#define MA_PARITY_ERROR_STATUS 0x77f4
+
+#define EDC_0_BASE_ADDR 0x7900
+
+#define EDC_BIST_CMD 0x7904
+#define EDC_BIST_CMD_ADDR 0x7908
+#define EDC_BIST_CMD_LEN 0x790c
+#define EDC_BIST_DATA_PATTERN 0x7910
+#define EDC_BIST_STATUS_RDATA 0x7928
+#define EDC_INT_CAUSE 0x7978
+#define ECC_UE_PAR 0x00000020U
+#define ECC_CE_PAR 0x00000010U
+#define PERR_PAR_CAUSE 0x00000008U
+
+#define EDC_ECC_STATUS 0x797c
+
+#define EDC_1_BASE_ADDR 0x7980
+
+#define CIM_PF_MAILBOX_DATA 0x240
+#define CIM_PF_MAILBOX_CTRL 0x280
+#define MBMSGVALID 0x00000008U
+#define MBINTREQ 0x00000004U
+#define MBOWNER_MASK 0x00000003U
+#define MBOWNER_SHIFT 0
+#define MBOWNER(x) ((x) << MBOWNER_SHIFT)
+#define MBOWNER_GET(x) (((x) & MBOWNER_MASK) >> MBOWNER_SHIFT)
+
+#define CIM_PF_HOST_INT_CAUSE 0x28c
+#define MBMSGRDYINT 0x00080000U
+
+#define CIM_HOST_INT_CAUSE 0x7b2c
+#define TIEQOUTPARERRINT 0x00100000U
+#define TIEQINPARERRINT 0x00080000U
+#define MBHOSTPARERR 0x00040000U
+#define MBUPPARERR 0x00020000U
+#define IBQPARERR 0x0001f800U
+#define IBQTP0PARERR 0x00010000U
+#define IBQTP1PARERR 0x00008000U
+#define IBQULPPARERR 0x00004000U
+#define IBQSGELOPARERR 0x00002000U
+#define IBQSGEHIPARERR 0x00001000U
+#define IBQNCSIPARERR 0x00000800U
+#define OBQPARERR 0x000007e0U
+#define OBQULP0PARERR 0x00000400U
+#define OBQULP1PARERR 0x00000200U
+#define OBQULP2PARERR 0x00000100U
+#define OBQULP3PARERR 0x00000080U
+#define OBQSGEPARERR 0x00000040U
+#define OBQNCSIPARERR 0x00000020U
+#define PREFDROPINT 0x00000002U
+#define UPACCNONZERO 0x00000001U
+
+#define CIM_HOST_UPACC_INT_CAUSE 0x7b34
+#define EEPROMWRINT 0x40000000U
+#define TIMEOUTMAINT 0x20000000U
+#define TIMEOUTINT 0x10000000U
+#define RSPOVRLOOKUPINT 0x08000000U
+#define REQOVRLOOKUPINT 0x04000000U
+#define BLKWRPLINT 0x02000000U
+#define BLKRDPLINT 0x01000000U
+#define SGLWRPLINT 0x00800000U
+#define SGLRDPLINT 0x00400000U
+#define BLKWRCTLINT 0x00200000U
+#define BLKRDCTLINT 0x00100000U
+#define SGLWRCTLINT 0x00080000U
+#define SGLRDCTLINT 0x00040000U
+#define BLKWREEPROMINT 0x00020000U
+#define BLKRDEEPROMINT 0x00010000U
+#define SGLWREEPROMINT 0x00008000U
+#define SGLRDEEPROMINT 0x00004000U
+#define BLKWRFLASHINT 0x00002000U
+#define BLKRDFLASHINT 0x00001000U
+#define SGLWRFLASHINT 0x00000800U
+#define SGLRDFLASHINT 0x00000400U
+#define BLKWRBOOTINT 0x00000200U
+#define BLKRDBOOTINT 0x00000100U
+#define SGLWRBOOTINT 0x00000080U
+#define SGLRDBOOTINT 0x00000040U
+#define ILLWRBEINT 0x00000020U
+#define ILLRDBEINT 0x00000010U
+#define ILLRDINT 0x00000008U
+#define ILLWRINT 0x00000004U
+#define ILLTRANSINT 0x00000002U
+#define RSVDSPACEINT 0x00000001U
+
+#define TP_OUT_CONFIG 0x7d04
+#define VLANEXTENABLE_MASK 0x0000f000U
+#define VLANEXTENABLE_SHIFT 12
+
+#define TP_PARA_REG2 0x7d68
+#define MAXRXDATA_MASK 0xffff0000U
+#define MAXRXDATA_SHIFT 16
+#define MAXRXDATA_GET(x) (((x) & MAXRXDATA_MASK) >> MAXRXDATA_SHIFT)
+
+#define TP_TIMER_RESOLUTION 0x7d90
+#define TIMERRESOLUTION_MASK 0x00ff0000U
+#define TIMERRESOLUTION_SHIFT 16
+#define TIMERRESOLUTION_GET(x) (((x) & TIMERRESOLUTION_MASK) >> TIMERRESOLUTION_SHIFT)
+
+#define TP_SHIFT_CNT 0x7dc0
+
+#define TP_CCTRL_TABLE 0x7ddc
+#define TP_MTU_TABLE 0x7de4
+#define MTUINDEX_MASK 0xff000000U
+#define MTUINDEX_SHIFT 24
+#define MTUINDEX(x) ((x) << MTUINDEX_SHIFT)
+#define MTUWIDTH_MASK 0x000f0000U
+#define MTUWIDTH_SHIFT 16
+#define MTUWIDTH(x) ((x) << MTUWIDTH_SHIFT)
+#define MTUWIDTH_GET(x) (((x) & MTUWIDTH_MASK) >> MTUWIDTH_SHIFT)
+#define MTUVALUE_MASK 0x00003fffU
+#define MTUVALUE_SHIFT 0
+#define MTUVALUE(x) ((x) << MTUVALUE_SHIFT)
+#define MTUVALUE_GET(x) (((x) & MTUVALUE_MASK) >> MTUVALUE_SHIFT)
+
+#define TP_RSS_LKP_TABLE 0x7dec
+#define LKPTBLROWVLD 0x80000000U
+#define LKPTBLQUEUE1_MASK 0x000ffc00U
+#define LKPTBLQUEUE1_SHIFT 10
+#define LKPTBLQUEUE1(x) ((x) << LKPTBLQUEUE1_SHIFT)
+#define LKPTBLQUEUE1_GET(x) (((x) & LKPTBLQUEUE1_MASK) >> LKPTBLQUEUE1_SHIFT)
+#define LKPTBLQUEUE0_MASK 0x000003ffU
+#define LKPTBLQUEUE0_SHIFT 0
+#define LKPTBLQUEUE0(x) ((x) << LKPTBLQUEUE0_SHIFT)
+#define LKPTBLQUEUE0_GET(x) (((x) & LKPTBLQUEUE0_MASK) >> LKPTBLQUEUE0_SHIFT)
+
+#define TP_PIO_ADDR 0x7e40
+#define TP_PIO_DATA 0x7e44
+#define TP_MIB_INDEX 0x7e50
+#define TP_MIB_DATA 0x7e54
+#define TP_INT_CAUSE 0x7e74
+#define FLMTXFLSTEMPTY 0x40000000U
+
+#define TP_INGRESS_CONFIG 0x141
+#define VNIC 0x00000800U
+#define CSUM_HAS_PSEUDO_HDR 0x00000400U
+#define RM_OVLAN 0x00000200U
+#define LOOKUPEVERYPKT 0x00000100U
+
+#define TP_MIB_MAC_IN_ERR_0 0x0
+#define TP_MIB_TCP_OUT_RST 0xc
+#define TP_MIB_TCP_IN_SEG_HI 0x10
+#define TP_MIB_TCP_IN_SEG_LO 0x11
+#define TP_MIB_TCP_OUT_SEG_HI 0x12
+#define TP_MIB_TCP_OUT_SEG_LO 0x13
+#define TP_MIB_TCP_RXT_SEG_HI 0x14
+#define TP_MIB_TCP_RXT_SEG_LO 0x15
+#define TP_MIB_TNL_CNG_DROP_0 0x18
+#define TP_MIB_TCP_V6IN_ERR_0 0x28
+#define TP_MIB_TCP_V6OUT_RST 0x2c
+#define TP_MIB_OFD_ARP_DROP 0x36
+#define TP_MIB_TNL_DROP_0 0x44
+#define TP_MIB_OFD_VLN_DROP_0 0x58
+
+#define ULP_TX_INT_CAUSE 0x8dcc
+#define PBL_BOUND_ERR_CH3 0x80000000U
+#define PBL_BOUND_ERR_CH2 0x40000000U
+#define PBL_BOUND_ERR_CH1 0x20000000U
+#define PBL_BOUND_ERR_CH0 0x10000000U
+
+#define PM_RX_INT_CAUSE 0x8fdc
+#define ZERO_E_CMD_ERROR 0x00400000U
+#define PMRX_FRAMING_ERROR 0x003ffff0U
+#define OCSPI_PAR_ERROR 0x00000008U
+#define DB_OPTIONS_PAR_ERROR 0x00000004U
+#define IESPI_PAR_ERROR 0x00000002U
+#define E_PCMD_PAR_ERROR 0x00000001U
+
+#define PM_TX_INT_CAUSE 0x8ffc
+#define PCMD_LEN_OVFL0 0x80000000U
+#define PCMD_LEN_OVFL1 0x40000000U
+#define PCMD_LEN_OVFL2 0x20000000U
+#define ZERO_C_CMD_ERROR 0x10000000U
+#define PMTX_FRAMING_ERROR 0x0ffffff0U
+#define OESPI_PAR_ERROR 0x00000008U
+#define ICSPI_PAR_ERROR 0x00000002U
+#define C_PCMD_PAR_ERROR 0x00000001U
+
+#define MPS_PORT_STAT_TX_PORT_BYTES_L 0x400
+#define MPS_PORT_STAT_TX_PORT_BYTES_H 0x404
+#define MPS_PORT_STAT_TX_PORT_FRAMES_L 0x408
+#define MPS_PORT_STAT_TX_PORT_FRAMES_H 0x40c
+#define MPS_PORT_STAT_TX_PORT_BCAST_L 0x410
+#define MPS_PORT_STAT_TX_PORT_BCAST_H 0x414
+#define MPS_PORT_STAT_TX_PORT_MCAST_L 0x418
+#define MPS_PORT_STAT_TX_PORT_MCAST_H 0x41c
+#define MPS_PORT_STAT_TX_PORT_UCAST_L 0x420
+#define MPS_PORT_STAT_TX_PORT_UCAST_H 0x424
+#define MPS_PORT_STAT_TX_PORT_ERROR_L 0x428
+#define MPS_PORT_STAT_TX_PORT_ERROR_H 0x42c
+#define MPS_PORT_STAT_TX_PORT_64B_L 0x430
+#define MPS_PORT_STAT_TX_PORT_64B_H 0x434
+#define MPS_PORT_STAT_TX_PORT_65B_127B_L 0x438
+#define MPS_PORT_STAT_TX_PORT_65B_127B_H 0x43c
+#define MPS_PORT_STAT_TX_PORT_128B_255B_L 0x440
+#define MPS_PORT_STAT_TX_PORT_128B_255B_H 0x444
+#define MPS_PORT_STAT_TX_PORT_256B_511B_L 0x448
+#define MPS_PORT_STAT_TX_PORT_256B_511B_H 0x44c
+#define MPS_PORT_STAT_TX_PORT_512B_1023B_L 0x450
+#define MPS_PORT_STAT_TX_PORT_512B_1023B_H 0x454
+#define MPS_PORT_STAT_TX_PORT_1024B_1518B_L 0x458
+#define MPS_PORT_STAT_TX_PORT_1024B_1518B_H 0x45c
+#define MPS_PORT_STAT_TX_PORT_1519B_MAX_L 0x460
+#define MPS_PORT_STAT_TX_PORT_1519B_MAX_H 0x464
+#define MPS_PORT_STAT_TX_PORT_DROP_L 0x468
+#define MPS_PORT_STAT_TX_PORT_DROP_H 0x46c
+#define MPS_PORT_STAT_TX_PORT_PAUSE_L 0x470
+#define MPS_PORT_STAT_TX_PORT_PAUSE_H 0x474
+#define MPS_PORT_STAT_TX_PORT_PPP0_L 0x478
+#define MPS_PORT_STAT_TX_PORT_PPP0_H 0x47c
+#define MPS_PORT_STAT_TX_PORT_PPP1_L 0x480
+#define MPS_PORT_STAT_TX_PORT_PPP1_H 0x484
+#define MPS_PORT_STAT_TX_PORT_PPP2_L 0x488
+#define MPS_PORT_STAT_TX_PORT_PPP2_H 0x48c
+#define MPS_PORT_STAT_TX_PORT_PPP3_L 0x490
+#define MPS_PORT_STAT_TX_PORT_PPP3_H 0x494
+#define MPS_PORT_STAT_TX_PORT_PPP4_L 0x498
+#define MPS_PORT_STAT_TX_PORT_PPP4_H 0x49c
+#define MPS_PORT_STAT_TX_PORT_PPP5_L 0x4a0
+#define MPS_PORT_STAT_TX_PORT_PPP5_H 0x4a4
+#define MPS_PORT_STAT_TX_PORT_PPP6_L 0x4a8
+#define MPS_PORT_STAT_TX_PORT_PPP6_H 0x4ac
+#define MPS_PORT_STAT_TX_PORT_PPP7_L 0x4b0
+#define MPS_PORT_STAT_TX_PORT_PPP7_H 0x4b4
+#define MPS_PORT_STAT_LB_PORT_BYTES_L 0x4c0
+#define MPS_PORT_STAT_LB_PORT_BYTES_H 0x4c4
+#define MPS_PORT_STAT_LB_PORT_FRAMES_L 0x4c8
+#define MPS_PORT_STAT_LB_PORT_FRAMES_H 0x4cc
+#define MPS_PORT_STAT_LB_PORT_BCAST_L 0x4d0
+#define MPS_PORT_STAT_LB_PORT_BCAST_H 0x4d4
+#define MPS_PORT_STAT_LB_PORT_MCAST_L 0x4d8
+#define MPS_PORT_STAT_LB_PORT_MCAST_H 0x4dc
+#define MPS_PORT_STAT_LB_PORT_UCAST_L 0x4e0
+#define MPS_PORT_STAT_LB_PORT_UCAST_H 0x4e4
+#define MPS_PORT_STAT_LB_PORT_ERROR_L 0x4e8
+#define MPS_PORT_STAT_LB_PORT_ERROR_H 0x4ec
+#define MPS_PORT_STAT_LB_PORT_64B_L 0x4f0
+#define MPS_PORT_STAT_LB_PORT_64B_H 0x4f4
+#define MPS_PORT_STAT_LB_PORT_65B_127B_L 0x4f8
+#define MPS_PORT_STAT_LB_PORT_65B_127B_H 0x4fc
+#define MPS_PORT_STAT_LB_PORT_128B_255B_L 0x500
+#define MPS_PORT_STAT_LB_PORT_128B_255B_H 0x504
+#define MPS_PORT_STAT_LB_PORT_256B_511B_L 0x508
+#define MPS_PORT_STAT_LB_PORT_256B_511B_H 0x50c
+#define MPS_PORT_STAT_LB_PORT_512B_1023B_L 0x510
+#define MPS_PORT_STAT_LB_PORT_512B_1023B_H 0x514
+#define MPS_PORT_STAT_LB_PORT_1024B_1518B_L 0x518
+#define MPS_PORT_STAT_LB_PORT_1024B_1518B_H 0x51c
+#define MPS_PORT_STAT_LB_PORT_1519B_MAX_L 0x520
+#define MPS_PORT_STAT_LB_PORT_1519B_MAX_H 0x524
+#define MPS_PORT_STAT_LB_PORT_DROP_FRAMES 0x528
+#define MPS_PORT_STAT_RX_PORT_BYTES_L 0x540
+#define MPS_PORT_STAT_RX_PORT_BYTES_H 0x544
+#define MPS_PORT_STAT_RX_PORT_FRAMES_L 0x548
+#define MPS_PORT_STAT_RX_PORT_FRAMES_H 0x54c
+#define MPS_PORT_STAT_RX_PORT_BCAST_L 0x550
+#define MPS_PORT_STAT_RX_PORT_BCAST_H 0x554
+#define MPS_PORT_STAT_RX_PORT_MCAST_L 0x558
+#define MPS_PORT_STAT_RX_PORT_MCAST_H 0x55c
+#define MPS_PORT_STAT_RX_PORT_UCAST_L 0x560
+#define MPS_PORT_STAT_RX_PORT_UCAST_H 0x564
+#define MPS_PORT_STAT_RX_PORT_MTU_ERROR_L 0x568
+#define MPS_PORT_STAT_RX_PORT_MTU_ERROR_H 0x56c
+#define MPS_PORT_STAT_RX_PORT_MTU_CRC_ERROR_L 0x570
+#define MPS_PORT_STAT_RX_PORT_MTU_CRC_ERROR_H 0x574
+#define MPS_PORT_STAT_RX_PORT_CRC_ERROR_L 0x578
+#define MPS_PORT_STAT_RX_PORT_CRC_ERROR_H 0x57c
+#define MPS_PORT_STAT_RX_PORT_LEN_ERROR_L 0x580
+#define MPS_PORT_STAT_RX_PORT_LEN_ERROR_H 0x584
+#define MPS_PORT_STAT_RX_PORT_SYM_ERROR_L 0x588
+#define MPS_PORT_STAT_RX_PORT_SYM_ERROR_H 0x58c
+#define MPS_PORT_STAT_RX_PORT_64B_L 0x590
+#define MPS_PORT_STAT_RX_PORT_64B_H 0x594
+#define MPS_PORT_STAT_RX_PORT_65B_127B_L 0x598
+#define MPS_PORT_STAT_RX_PORT_65B_127B_H 0x59c
+#define MPS_PORT_STAT_RX_PORT_128B_255B_L 0x5a0
+#define MPS_PORT_STAT_RX_PORT_128B_255B_H 0x5a4
+#define MPS_PORT_STAT_RX_PORT_256B_511B_L 0x5a8
+#define MPS_PORT_STAT_RX_PORT_256B_511B_H 0x5ac
+#define MPS_PORT_STAT_RX_PORT_512B_1023B_L 0x5b0
+#define MPS_PORT_STAT_RX_PORT_512B_1023B_H 0x5b4
+#define MPS_PORT_STAT_RX_PORT_1024B_1518B_L 0x5b8
+#define MPS_PORT_STAT_RX_PORT_1024B_1518B_H 0x5bc
+#define MPS_PORT_STAT_RX_PORT_1519B_MAX_L 0x5c0
+#define MPS_PORT_STAT_RX_PORT_1519B_MAX_H 0x5c4
+#define MPS_PORT_STAT_RX_PORT_PAUSE_L 0x5c8
+#define MPS_PORT_STAT_RX_PORT_PAUSE_H 0x5cc
+#define MPS_PORT_STAT_RX_PORT_PPP0_L 0x5d0
+#define MPS_PORT_STAT_RX_PORT_PPP0_H 0x5d4
+#define MPS_PORT_STAT_RX_PORT_PPP1_L 0x5d8
+#define MPS_PORT_STAT_RX_PORT_PPP1_H 0x5dc
+#define MPS_PORT_STAT_RX_PORT_PPP2_L 0x5e0
+#define MPS_PORT_STAT_RX_PORT_PPP2_H 0x5e4
+#define MPS_PORT_STAT_RX_PORT_PPP3_L 0x5e8
+#define MPS_PORT_STAT_RX_PORT_PPP3_H 0x5ec
+#define MPS_PORT_STAT_RX_PORT_PPP4_L 0x5f0
+#define MPS_PORT_STAT_RX_PORT_PPP4_H 0x5f4
+#define MPS_PORT_STAT_RX_PORT_PPP5_L 0x5f8
+#define MPS_PORT_STAT_RX_PORT_PPP5_H 0x5fc
+#define MPS_PORT_STAT_RX_PORT_PPP6_L 0x600
+#define MPS_PORT_STAT_RX_PORT_PPP6_H 0x604
+#define MPS_PORT_STAT_RX_PORT_PPP7_L 0x608
+#define MPS_PORT_STAT_RX_PORT_PPP7_H 0x60c
+#define MPS_PORT_STAT_RX_PORT_LESS_64B_L 0x610
+#define MPS_PORT_STAT_RX_PORT_LESS_64B_H 0x614
+#define MPS_CMN_CTL 0x9000
+#define NUMPORTS_MASK 0x00000003U
+#define NUMPORTS_SHIFT 0
+#define NUMPORTS_GET(x) (((x) & NUMPORTS_MASK) >> NUMPORTS_SHIFT)
+
+#define MPS_INT_CAUSE 0x9008
+#define STATINT 0x00000020U
+#define TXINT 0x00000010U
+#define RXINT 0x00000008U
+#define TRCINT 0x00000004U
+#define CLSINT 0x00000002U
+#define PLINT 0x00000001U
+
+#define MPS_TX_INT_CAUSE 0x9408
+#define PORTERR 0x00010000U
+#define FRMERR 0x00008000U
+#define SECNTERR 0x00004000U
+#define BUBBLE 0x00002000U
+#define TXDESCFIFO 0x00001e00U
+#define TXDATAFIFO 0x000001e0U
+#define NCSIFIFO 0x00000010U
+#define TPFIFO 0x0000000fU
+
+#define MPS_STAT_PERR_INT_CAUSE_SRAM 0x9614
+#define MPS_STAT_PERR_INT_CAUSE_TX_FIFO 0x9620
+#define MPS_STAT_PERR_INT_CAUSE_RX_FIFO 0x962c
+
+#define MPS_STAT_RX_BG_0_MAC_DROP_FRAME_L 0x9640
+#define MPS_STAT_RX_BG_0_MAC_DROP_FRAME_H 0x9644
+#define MPS_STAT_RX_BG_1_MAC_DROP_FRAME_L 0x9648
+#define MPS_STAT_RX_BG_1_MAC_DROP_FRAME_H 0x964c
+#define MPS_STAT_RX_BG_2_MAC_DROP_FRAME_L 0x9650
+#define MPS_STAT_RX_BG_2_MAC_DROP_FRAME_H 0x9654
+#define MPS_STAT_RX_BG_3_MAC_DROP_FRAME_L 0x9658
+#define MPS_STAT_RX_BG_3_MAC_DROP_FRAME_H 0x965c
+#define MPS_STAT_RX_BG_0_LB_DROP_FRAME_L 0x9660
+#define MPS_STAT_RX_BG_0_LB_DROP_FRAME_H 0x9664
+#define MPS_STAT_RX_BG_1_LB_DROP_FRAME_L 0x9668
+#define MPS_STAT_RX_BG_1_LB_DROP_FRAME_H 0x966c
+#define MPS_STAT_RX_BG_2_LB_DROP_FRAME_L 0x9670
+#define MPS_STAT_RX_BG_2_LB_DROP_FRAME_H 0x9674
+#define MPS_STAT_RX_BG_3_LB_DROP_FRAME_L 0x9678
+#define MPS_STAT_RX_BG_3_LB_DROP_FRAME_H 0x967c
+#define MPS_STAT_RX_BG_0_MAC_TRUNC_FRAME_L 0x9680
+#define MPS_STAT_RX_BG_0_MAC_TRUNC_FRAME_H 0x9684
+#define MPS_STAT_RX_BG_1_MAC_TRUNC_FRAME_L 0x9688
+#define MPS_STAT_RX_BG_1_MAC_TRUNC_FRAME_H 0x968c
+#define MPS_STAT_RX_BG_2_MAC_TRUNC_FRAME_L 0x9690
+#define MPS_STAT_RX_BG_2_MAC_TRUNC_FRAME_H 0x9694
+#define MPS_STAT_RX_BG_3_MAC_TRUNC_FRAME_L 0x9698
+#define MPS_STAT_RX_BG_3_MAC_TRUNC_FRAME_H 0x969c
+#define MPS_STAT_RX_BG_0_LB_TRUNC_FRAME_L 0x96a0
+#define MPS_STAT_RX_BG_0_LB_TRUNC_FRAME_H 0x96a4
+#define MPS_STAT_RX_BG_1_LB_TRUNC_FRAME_L 0x96a8
+#define MPS_STAT_RX_BG_1_LB_TRUNC_FRAME_H 0x96ac
+#define MPS_STAT_RX_BG_2_LB_TRUNC_FRAME_L 0x96b0
+#define MPS_STAT_RX_BG_2_LB_TRUNC_FRAME_H 0x96b4
+#define MPS_STAT_RX_BG_3_LB_TRUNC_FRAME_L 0x96b8
+#define MPS_STAT_RX_BG_3_LB_TRUNC_FRAME_H 0x96bc
+#define MPS_TRC_CFG 0x9800
+#define TRCFIFOEMPTY 0x00000010U
+#define TRCIGNOREDROPINPUT 0x00000008U
+#define TRCKEEPDUPLICATES 0x00000004U
+#define TRCEN 0x00000002U
+#define TRCMULTIFILTER 0x00000001U
+
+#define MPS_TRC_RSS_CONTROL 0x9808
+#define RSSCONTROL_MASK 0x00ff0000U
+#define RSSCONTROL_SHIFT 16
+#define RSSCONTROL(x) ((x) << RSSCONTROL_SHIFT)
+#define QUEUENUMBER_MASK 0x0000ffffU
+#define QUEUENUMBER_SHIFT 0
+#define QUEUENUMBER(x) ((x) << QUEUENUMBER_SHIFT)
+
+#define MPS_TRC_FILTER_MATCH_CTL_A 0x9810
+#define TFINVERTMATCH 0x01000000U
+#define TFPKTTOOLARGE 0x00800000U
+#define TFEN 0x00400000U
+#define TFPORT_MASK 0x003c0000U
+#define TFPORT_SHIFT 18
+#define TFPORT(x) ((x) << TFPORT_SHIFT)
+#define TFPORT_GET(x) (((x) & TFPORT_MASK) >> TFPORT_SHIFT)
+#define TFDROP 0x00020000U
+#define TFSOPEOPERR 0x00010000U
+#define TFLENGTH_MASK 0x00001f00U
+#define TFLENGTH_SHIFT 8
+#define TFLENGTH(x) ((x) << TFLENGTH_SHIFT)
+#define TFLENGTH_GET(x) (((x) & TFLENGTH_MASK) >> TFLENGTH_SHIFT)
+#define TFOFFSET_MASK 0x0000001fU
+#define TFOFFSET_SHIFT 0
+#define TFOFFSET(x) ((x) << TFOFFSET_SHIFT)
+#define TFOFFSET_GET(x) (((x) & TFOFFSET_MASK) >> TFOFFSET_SHIFT)
+
+#define MPS_TRC_FILTER_MATCH_CTL_B 0x9820
+#define TFMINPKTSIZE_MASK 0x01ff0000U
+#define TFMINPKTSIZE_SHIFT 16
+#define TFMINPKTSIZE(x) ((x) << TFMINPKTSIZE_SHIFT)
+#define TFMINPKTSIZE_GET(x) (((x) & TFMINPKTSIZE_MASK) >> TFMINPKTSIZE_SHIFT)
+#define TFCAPTUREMAX_MASK 0x00003fffU
+#define TFCAPTUREMAX_SHIFT 0
+#define TFCAPTUREMAX(x) ((x) << TFCAPTUREMAX_SHIFT)
+#define TFCAPTUREMAX_GET(x) (((x) & TFCAPTUREMAX_MASK) >> TFCAPTUREMAX_SHIFT)
+
+#define MPS_TRC_INT_CAUSE 0x985c
+#define MISCPERR 0x00000100U
+#define PKTFIFO 0x000000f0U
+#define FILTMEM 0x0000000fU
+
+#define MPS_TRC_FILTER0_MATCH 0x9c00
+#define MPS_TRC_FILTER0_DONT_CARE 0x9c80
+#define MPS_TRC_FILTER1_MATCH 0x9d00
+#define MPS_CLS_INT_CAUSE 0xd028
+#define PLERRENB 0x00000008U
+#define HASHSRAM 0x00000004U
+#define MATCHTCAM 0x00000002U
+#define MATCHSRAM 0x00000001U
+
+#define MPS_RX_PERR_INT_CAUSE 0x11074
+
+#define CPL_INTR_CAUSE 0x19054
+#define CIM_OP_MAP_PERR 0x00000020U
+#define CIM_OVFL_ERROR 0x00000010U
+#define TP_FRAMING_ERROR 0x00000008U
+#define SGE_FRAMING_ERROR 0x00000004U
+#define CIM_FRAMING_ERROR 0x00000002U
+#define ZERO_SWITCH_ERROR 0x00000001U
+
+#define SMB_INT_CAUSE 0x19090
+#define MSTTXFIFOPARINT 0x00200000U
+#define MSTRXFIFOPARINT 0x00100000U
+#define SLVFIFOPARINT 0x00080000U
+
+#define ULP_RX_INT_CAUSE 0x19158
+#define ULP_RX_ISCSI_TAGMASK 0x19164
+#define ULP_RX_ISCSI_PSZ 0x19168
+#define HPZ3_MASK 0x0f000000U
+#define HPZ3_SHIFT 24
+#define HPZ3(x) ((x) << HPZ3_SHIFT)
+#define HPZ2_MASK 0x000f0000U
+#define HPZ2_SHIFT 16
+#define HPZ2(x) ((x) << HPZ2_SHIFT)
+#define HPZ1_MASK 0x00000f00U
+#define HPZ1_SHIFT 8
+#define HPZ1(x) ((x) << HPZ1_SHIFT)
+#define HPZ0_MASK 0x0000000fU
+#define HPZ0_SHIFT 0
+#define HPZ0(x) ((x) << HPZ0_SHIFT)
+
+#define ULP_RX_TDDP_PSZ 0x19178
+
+#define SF_DATA 0x193f8
+#define SF_OP 0x193fc
+#define BUSY 0x80000000U
+#define SF_LOCK 0x00000010U
+#define SF_CONT 0x00000008U
+#define BYTECNT_MASK 0x00000006U
+#define BYTECNT_SHIFT 1
+#define BYTECNT(x) ((x) << BYTECNT_SHIFT)
+#define OP_WR 0x00000001U
+
+#define PL_PF_INT_CAUSE 0x3c0
+#define PFSW 0x00000008U
+#define PFSGE 0x00000004U
+#define PFCIM 0x00000002U
+#define PFMPS 0x00000001U
+
+#define PL_PF_INT_ENABLE 0x3c4
+#define PL_PF_CTL 0x3c8
+#define SWINT 0x00000001U
+
+#define PL_WHOAMI 0x19400
+#define SOURCEPF_MASK 0x00000700U
+#define SOURCEPF_SHIFT 8
+#define SOURCEPF(x) ((x) << SOURCEPF_SHIFT)
+#define SOURCEPF_GET(x) (((x) & SOURCEPF_MASK) >> SOURCEPF_SHIFT)
+#define ISVF 0x00000080U
+#define VFID_MASK 0x0000007fU
+#define VFID_SHIFT 0
+#define VFID(x) ((x) << VFID_SHIFT)
+#define VFID_GET(x) (((x) & VFID_MASK) >> VFID_SHIFT)
+
+#define PL_INT_CAUSE 0x1940c
+#define ULP_TX 0x08000000U
+#define SGE 0x04000000U
+#define HMA 0x02000000U
+#define CPL_SWITCH 0x01000000U
+#define ULP_RX 0x00800000U
+#define PM_RX 0x00400000U
+#define PM_TX 0x00200000U
+#define MA 0x00100000U
+#define TP 0x00080000U
+#define LE 0x00040000U
+#define EDC1 0x00020000U
+#define EDC0 0x00010000U
+#define MC 0x00008000U
+#define PCIE 0x00004000U
+#define PMU 0x00002000U
+#define XGMAC_KR1 0x00001000U
+#define XGMAC_KR0 0x00000800U
+#define XGMAC1 0x00000400U
+#define XGMAC0 0x00000200U
+#define SMB 0x00000100U
+#define SF 0x00000080U
+#define PL 0x00000040U
+#define NCSI 0x00000020U
+#define MPS 0x00000010U
+#define MI 0x00000008U
+#define DBG 0x00000004U
+#define I2CM 0x00000002U
+#define CIM 0x00000001U
+
+#define PL_INT_MAP0 0x19414
+#define PL_RST 0x19428
+#define PIORST 0x00000002U
+#define PIORSTMODE 0x00000001U
+
+#define PL_PL_INT_CAUSE 0x19430
+#define FATALPERR 0x00000010U
+#define PERRVFID 0x00000001U
+
+#define PL_REV 0x1943c
+
+#define LE_DB_CONFIG 0x19c04
+#define HASHEN 0x00100000U
+
+#define LE_DB_SERVER_INDEX 0x19c18
+#define LE_DB_ACT_CNT_IPV4 0x19c20
+#define LE_DB_ACT_CNT_IPV6 0x19c24
+
+#define LE_DB_INT_CAUSE 0x19c3c
+#define REQQPARERR 0x00010000U
+#define UNKNOWNCMD 0x00008000U
+#define PARITYERR 0x00000040U
+#define LIPMISS 0x00000020U
+#define LIP0 0x00000010U
+
+#define LE_DB_TID_HASHBASE 0x19df8
+
+#define NCSI_INT_CAUSE 0x1a0d8
+#define CIM_DM_PRTY_ERR 0x00000100U
+#define MPS_DM_PRTY_ERR 0x00000080U
+#define TXFIFO_PRTY_ERR 0x00000002U
+#define RXFIFO_PRTY_ERR 0x00000001U
+
+#define XGMAC_PORT_CFG2 0x1018
+#define PATEN 0x00040000U
+#define MAGICEN 0x00020000U
+
+#define XGMAC_PORT_MAGIC_MACID_LO 0x1024
+#define XGMAC_PORT_MAGIC_MACID_HI 0x1028
+
+#define XGMAC_PORT_EPIO_DATA0 0x10c0
+#define XGMAC_PORT_EPIO_DATA1 0x10c4
+#define XGMAC_PORT_EPIO_DATA2 0x10c8
+#define XGMAC_PORT_EPIO_DATA3 0x10cc
+#define XGMAC_PORT_EPIO_OP 0x10d0
+#define EPIOWR 0x00000100U
+#define ADDRESS_MASK 0x000000ffU
+#define ADDRESS_SHIFT 0
+#define ADDRESS(x) ((x) << ADDRESS_SHIFT)
+
+#define XGMAC_PORT_INT_CAUSE 0x10dc
+#endif /* __T4_REGS_H */
--- /dev/null
+/*
+ * This file is part of the Chelsio T4 Ethernet driver for Linux.
+ *
+ * Copyright (c) 2009-2010 Chelsio Communications, Inc. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef _T4FW_INTERFACE_H_
+#define _T4FW_INTERFACE_H_
+
+#define FW_T4VF_SGE_BASE_ADDR 0x0000
+#define FW_T4VF_MPS_BASE_ADDR 0x0100
+#define FW_T4VF_PL_BASE_ADDR 0x0200
+#define FW_T4VF_MBDATA_BASE_ADDR 0x0240
+#define FW_T4VF_CIM_BASE_ADDR 0x0300
+
+enum fw_wr_opcodes {
+ FW_FILTER_WR = 0x02,
+ FW_ULPTX_WR = 0x04,
+ FW_TP_WR = 0x05,
+ FW_ETH_TX_PKT_WR = 0x08,
+ FW_FLOWC_WR = 0x0a,
+ FW_OFLD_TX_DATA_WR = 0x0b,
+ FW_CMD_WR = 0x10,
+ FW_ETH_TX_PKT_VM_WR = 0x11,
+ FW_RI_RES_WR = 0x0c,
+ FW_RI_INIT_WR = 0x0d,
+ FW_RI_RDMA_WRITE_WR = 0x14,
+ FW_RI_SEND_WR = 0x15,
+ FW_RI_RDMA_READ_WR = 0x16,
+ FW_RI_RECV_WR = 0x17,
+ FW_RI_BIND_MW_WR = 0x18,
+ FW_RI_FR_NSMR_WR = 0x19,
+ FW_RI_INV_LSTAG_WR = 0x1a,
+ FW_LASTC2E_WR = 0x40
+};
+
+struct fw_wr_hdr {
+ __be32 hi;
+ __be32 lo;
+};
+
+#define FW_WR_OP(x) ((x) << 24)
+#define FW_WR_ATOMIC(x) ((x) << 23)
+#define FW_WR_FLUSH(x) ((x) << 22)
+#define FW_WR_COMPL(x) ((x) << 21)
+#define FW_WR_IMMDLEN(x) ((x) << 0)
+
+#define FW_WR_EQUIQ (1U << 31)
+#define FW_WR_EQUEQ (1U << 30)
+#define FW_WR_FLOWID(x) ((x) << 8)
+#define FW_WR_LEN16(x) ((x) << 0)
+
+struct fw_ulptx_wr {
+ __be32 op_to_compl;
+ __be32 flowid_len16;
+ u64 cookie;
+};
+
+struct fw_tp_wr {
+ __be32 op_to_immdlen;
+ __be32 flowid_len16;
+ u64 cookie;
+};
+
+struct fw_eth_tx_pkt_wr {
+ __be32 op_immdlen;
+ __be32 equiq_to_len16;
+ __be64 r3;
+};
+
+enum fw_flowc_mnem {
+ FW_FLOWC_MNEM_PFNVFN, /* PFN [15:8] VFN [7:0] */
+ FW_FLOWC_MNEM_CH,
+ FW_FLOWC_MNEM_PORT,
+ FW_FLOWC_MNEM_IQID,
+ FW_FLOWC_MNEM_SNDNXT,
+ FW_FLOWC_MNEM_RCVNXT,
+ FW_FLOWC_MNEM_SNDBUF,
+ FW_FLOWC_MNEM_MSS,
+};
+
+struct fw_flowc_mnemval {
+ u8 mnemonic;
+ u8 r4[3];
+ __be32 val;
+};
+
+struct fw_flowc_wr {
+ __be32 op_to_nparams;
+#define FW_FLOWC_WR_NPARAMS(x) ((x) << 0)
+ __be32 flowid_len16;
+ struct fw_flowc_mnemval mnemval[0];
+};
+
+struct fw_ofld_tx_data_wr {
+ __be32 op_to_immdlen;
+ __be32 flowid_len16;
+ __be32 plen;
+ __be32 tunnel_to_proxy;
+#define FW_OFLD_TX_DATA_WR_TUNNEL(x) ((x) << 19)
+#define FW_OFLD_TX_DATA_WR_SAVE(x) ((x) << 18)
+#define FW_OFLD_TX_DATA_WR_FLUSH(x) ((x) << 17)
+#define FW_OFLD_TX_DATA_WR_URGENT(x) ((x) << 16)
+#define FW_OFLD_TX_DATA_WR_MORE(x) ((x) << 15)
+#define FW_OFLD_TX_DATA_WR_SHOVE(x) ((x) << 14)
+#define FW_OFLD_TX_DATA_WR_ULPMODE(x) ((x) << 10)
+#define FW_OFLD_TX_DATA_WR_ULPSUBMODE(x) ((x) << 6)
+};
+
+struct fw_cmd_wr {
+ __be32 op_dma;
+#define FW_CMD_WR_DMA (1U << 17)
+ __be32 len16_pkd;
+ __be64 cookie_daddr;
+};
+
+struct fw_eth_tx_pkt_vm_wr {
+ __be32 op_immdlen;
+ __be32 equiq_to_len16;
+ __be32 r3[2];
+ u8 ethmacdst[6];
+ u8 ethmacsrc[6];
+ __be16 ethtype;
+ __be16 vlantci;
+};
+
+#define FW_CMD_MAX_TIMEOUT 3000
+
+enum fw_cmd_opcodes {
+ FW_LDST_CMD = 0x01,
+ FW_RESET_CMD = 0x03,
+ FW_HELLO_CMD = 0x04,
+ FW_BYE_CMD = 0x05,
+ FW_INITIALIZE_CMD = 0x06,
+ FW_CAPS_CONFIG_CMD = 0x07,
+ FW_PARAMS_CMD = 0x08,
+ FW_PFVF_CMD = 0x09,
+ FW_IQ_CMD = 0x10,
+ FW_EQ_MNGT_CMD = 0x11,
+ FW_EQ_ETH_CMD = 0x12,
+ FW_EQ_CTRL_CMD = 0x13,
+ FW_EQ_OFLD_CMD = 0x21,
+ FW_VI_CMD = 0x14,
+ FW_VI_MAC_CMD = 0x15,
+ FW_VI_RXMODE_CMD = 0x16,
+ FW_VI_ENABLE_CMD = 0x17,
+ FW_ACL_MAC_CMD = 0x18,
+ FW_ACL_VLAN_CMD = 0x19,
+ FW_VI_STATS_CMD = 0x1a,
+ FW_PORT_CMD = 0x1b,
+ FW_PORT_STATS_CMD = 0x1c,
+ FW_PORT_LB_STATS_CMD = 0x1d,
+ FW_PORT_TRACE_CMD = 0x1e,
+ FW_PORT_TRACE_MMAP_CMD = 0x1f,
+ FW_RSS_IND_TBL_CMD = 0x20,
+ FW_RSS_GLB_CONFIG_CMD = 0x22,
+ FW_RSS_VI_CONFIG_CMD = 0x23,
+ FW_LASTC2E_CMD = 0x40,
+ FW_ERROR_CMD = 0x80,
+ FW_DEBUG_CMD = 0x81,
+};
+
+enum fw_cmd_cap {
+ FW_CMD_CAP_PF = 0x01,
+ FW_CMD_CAP_DMAQ = 0x02,
+ FW_CMD_CAP_PORT = 0x04,
+ FW_CMD_CAP_PORTPROMISC = 0x08,
+ FW_CMD_CAP_PORTSTATS = 0x10,
+ FW_CMD_CAP_VF = 0x80,
+};
+
+/*
+ * Generic command header flit0
+ */
+struct fw_cmd_hdr {
+ __be32 hi;
+ __be32 lo;
+};
+
+#define FW_CMD_OP(x) ((x) << 24)
+#define FW_CMD_OP_GET(x) (((x) >> 24) & 0xff)
+#define FW_CMD_REQUEST (1U << 23)
+#define FW_CMD_READ (1U << 22)
+#define FW_CMD_WRITE (1U << 21)
+#define FW_CMD_EXEC (1U << 20)
+#define FW_CMD_RAMASK(x) ((x) << 20)
+#define FW_CMD_RETVAL(x) ((x) << 8)
+#define FW_CMD_RETVAL_GET(x) (((x) >> 8) & 0xff)
+#define FW_CMD_LEN16(x) ((x) << 0)
+
+enum fw_ldst_addrspc {
+ FW_LDST_ADDRSPC_FIRMWARE = 0x0001,
+ FW_LDST_ADDRSPC_SGE_EGRC = 0x0008,
+ FW_LDST_ADDRSPC_SGE_INGC = 0x0009,
+ FW_LDST_ADDRSPC_SGE_FLMC = 0x000a,
+ FW_LDST_ADDRSPC_SGE_CONMC = 0x000b,
+ FW_LDST_ADDRSPC_TP_PIO = 0x0010,
+ FW_LDST_ADDRSPC_TP_TM_PIO = 0x0011,
+ FW_LDST_ADDRSPC_TP_MIB = 0x0012,
+ FW_LDST_ADDRSPC_MDIO = 0x0018,
+ FW_LDST_ADDRSPC_MPS = 0x0020,
+ FW_LDST_ADDRSPC_FUNC = 0x0028
+};
+
+enum fw_ldst_mps_fid {
+ FW_LDST_MPS_ATRB,
+ FW_LDST_MPS_RPLC
+};
+
+enum fw_ldst_func_access_ctl {
+ FW_LDST_FUNC_ACC_CTL_VIID,
+ FW_LDST_FUNC_ACC_CTL_FID
+};
+
+enum fw_ldst_func_mod_index {
+ FW_LDST_FUNC_MPS
+};
+
+struct fw_ldst_cmd {
+ __be32 op_to_addrspace;
+#define FW_LDST_CMD_ADDRSPACE(x) ((x) << 0)
+ __be32 cycles_to_len16;
+ union fw_ldst {
+ struct fw_ldst_addrval {
+ __be32 addr;
+ __be32 val;
+ } addrval;
+ struct fw_ldst_idctxt {
+ __be32 physid;
+ __be32 msg_pkd;
+ __be32 ctxt_data7;
+ __be32 ctxt_data6;
+ __be32 ctxt_data5;
+ __be32 ctxt_data4;
+ __be32 ctxt_data3;
+ __be32 ctxt_data2;
+ __be32 ctxt_data1;
+ __be32 ctxt_data0;
+ } idctxt;
+ struct fw_ldst_mdio {
+ __be16 paddr_mmd;
+ __be16 raddr;
+ __be16 vctl;
+ __be16 rval;
+ } mdio;
+ struct fw_ldst_mps {
+ __be16 fid_ctl;
+ __be16 rplcpf_pkd;
+ __be32 rplc127_96;
+ __be32 rplc95_64;
+ __be32 rplc63_32;
+ __be32 rplc31_0;
+ __be32 atrb;
+ __be16 vlan[16];
+ } mps;
+ struct fw_ldst_func {
+ u8 access_ctl;
+ u8 mod_index;
+ __be16 ctl_id;
+ __be32 offset;
+ __be64 data0;
+ __be64 data1;
+ } func;
+ } u;
+};
+
+#define FW_LDST_CMD_MSG(x) ((x) << 31)
+#define FW_LDST_CMD_PADDR(x) ((x) << 8)
+#define FW_LDST_CMD_MMD(x) ((x) << 0)
+#define FW_LDST_CMD_FID(x) ((x) << 15)
+#define FW_LDST_CMD_CTL(x) ((x) << 0)
+#define FW_LDST_CMD_RPLCPF(x) ((x) << 0)
+
+struct fw_reset_cmd {
+ __be32 op_to_write;
+ __be32 retval_len16;
+ __be32 val;
+ __be32 r3;
+};
+
+struct fw_hello_cmd {
+ __be32 op_to_write;
+ __be32 retval_len16;
+ __be32 err_to_mbasyncnot;
+#define FW_HELLO_CMD_ERR (1U << 31)
+#define FW_HELLO_CMD_INIT (1U << 30)
+#define FW_HELLO_CMD_MASTERDIS(x) ((x) << 29)
+#define FW_HELLO_CMD_MASTERFORCE(x) ((x) << 28)
+#define FW_HELLO_CMD_MBMASTER(x) ((x) << 24)
+#define FW_HELLO_CMD_MBASYNCNOT(x) ((x) << 20)
+ __be32 fwrev;
+};
+
+struct fw_bye_cmd {
+ __be32 op_to_write;
+ __be32 retval_len16;
+ __be64 r3;
+};
+
+struct fw_initialize_cmd {
+ __be32 op_to_write;
+ __be32 retval_len16;
+ __be64 r3;
+};
+
+enum fw_caps_config_hm {
+ FW_CAPS_CONFIG_HM_PCIE = 0x00000001,
+ FW_CAPS_CONFIG_HM_PL = 0x00000002,
+ FW_CAPS_CONFIG_HM_SGE = 0x00000004,
+ FW_CAPS_CONFIG_HM_CIM = 0x00000008,
+ FW_CAPS_CONFIG_HM_ULPTX = 0x00000010,
+ FW_CAPS_CONFIG_HM_TP = 0x00000020,
+ FW_CAPS_CONFIG_HM_ULPRX = 0x00000040,
+ FW_CAPS_CONFIG_HM_PMRX = 0x00000080,
+ FW_CAPS_CONFIG_HM_PMTX = 0x00000100,
+ FW_CAPS_CONFIG_HM_MC = 0x00000200,
+ FW_CAPS_CONFIG_HM_LE = 0x00000400,
+ FW_CAPS_CONFIG_HM_MPS = 0x00000800,
+ FW_CAPS_CONFIG_HM_XGMAC = 0x00001000,
+ FW_CAPS_CONFIG_HM_CPLSWITCH = 0x00002000,
+ FW_CAPS_CONFIG_HM_T4DBG = 0x00004000,
+ FW_CAPS_CONFIG_HM_MI = 0x00008000,
+ FW_CAPS_CONFIG_HM_I2CM = 0x00010000,
+ FW_CAPS_CONFIG_HM_NCSI = 0x00020000,
+ FW_CAPS_CONFIG_HM_SMB = 0x00040000,
+ FW_CAPS_CONFIG_HM_MA = 0x00080000,
+ FW_CAPS_CONFIG_HM_EDRAM = 0x00100000,
+ FW_CAPS_CONFIG_HM_PMU = 0x00200000,
+ FW_CAPS_CONFIG_HM_UART = 0x00400000,
+ FW_CAPS_CONFIG_HM_SF = 0x00800000,
+};
+
+enum fw_caps_config_nbm {
+ FW_CAPS_CONFIG_NBM_IPMI = 0x00000001,
+ FW_CAPS_CONFIG_NBM_NCSI = 0x00000002,
+};
+
+enum fw_caps_config_link {
+ FW_CAPS_CONFIG_LINK_PPP = 0x00000001,
+ FW_CAPS_CONFIG_LINK_QFC = 0x00000002,
+ FW_CAPS_CONFIG_LINK_DCBX = 0x00000004,
+};
+
+enum fw_caps_config_switch {
+ FW_CAPS_CONFIG_SWITCH_INGRESS = 0x00000001,
+ FW_CAPS_CONFIG_SWITCH_EGRESS = 0x00000002,
+};
+
+enum fw_caps_config_nic {
+ FW_CAPS_CONFIG_NIC = 0x00000001,
+ FW_CAPS_CONFIG_NIC_VM = 0x00000002,
+};
+
+enum fw_caps_config_ofld {
+ FW_CAPS_CONFIG_OFLD = 0x00000001,
+};
+
+enum fw_caps_config_rdma {
+ FW_CAPS_CONFIG_RDMA_RDDP = 0x00000001,
+ FW_CAPS_CONFIG_RDMA_RDMAC = 0x00000002,
+};
+
+enum fw_caps_config_iscsi {
+ FW_CAPS_CONFIG_ISCSI_INITIATOR_PDU = 0x00000001,
+ FW_CAPS_CONFIG_ISCSI_TARGET_PDU = 0x00000002,
+ FW_CAPS_CONFIG_ISCSI_INITIATOR_CNXOFLD = 0x00000004,
+ FW_CAPS_CONFIG_ISCSI_TARGET_CNXOFLD = 0x00000008,
+};
+
+enum fw_caps_config_fcoe {
+ FW_CAPS_CONFIG_FCOE_INITIATOR = 0x00000001,
+ FW_CAPS_CONFIG_FCOE_TARGET = 0x00000002,
+};
+
+struct fw_caps_config_cmd {
+ __be32 op_to_write;
+ __be32 retval_len16;
+ __be32 r2;
+ __be32 hwmbitmap;
+ __be16 nbmcaps;
+ __be16 linkcaps;
+ __be16 switchcaps;
+ __be16 r3;
+ __be16 niccaps;
+ __be16 ofldcaps;
+ __be16 rdmacaps;
+ __be16 r4;
+ __be16 iscsicaps;
+ __be16 fcoecaps;
+ __be32 r5;
+ __be64 r6;
+};
+
+/*
+ * params command mnemonics
+ */
+enum fw_params_mnem {
+ FW_PARAMS_MNEM_DEV = 1, /* device params */
+ FW_PARAMS_MNEM_PFVF = 2, /* function params */
+ FW_PARAMS_MNEM_REG = 3, /* limited register access */
+ FW_PARAMS_MNEM_DMAQ = 4, /* dma queue params */
+ FW_PARAMS_MNEM_LAST
+};
+
+/*
+ * device parameters
+ */
+enum fw_params_param_dev {
+ FW_PARAMS_PARAM_DEV_CCLK = 0x00, /* chip core clock in khz */
+ FW_PARAMS_PARAM_DEV_PORTVEC = 0x01, /* the port vector */
+ FW_PARAMS_PARAM_DEV_NTID = 0x02, /* reads the number of TIDs
+ * allocated by the device's
+ * Lookup Engine
+ */
+ FW_PARAMS_PARAM_DEV_FLOWC_BUFFIFO_SZ = 0x03,
+ FW_PARAMS_PARAM_DEV_INTVER_NIC = 0x04,
+ FW_PARAMS_PARAM_DEV_INTVER_VNIC = 0x05,
+ FW_PARAMS_PARAM_DEV_INTVER_OFLD = 0x06,
+ FW_PARAMS_PARAM_DEV_INTVER_RI = 0x07,
+ FW_PARAMS_PARAM_DEV_INTVER_ISCSIPDU = 0x08,
+ FW_PARAMS_PARAM_DEV_INTVER_ISCSI = 0x09,
+ FW_PARAMS_PARAM_DEV_INTVER_FCOE = 0x0A
+};
+
+/*
+ * physical and virtual function parameters
+ */
+enum fw_params_param_pfvf {
+ FW_PARAMS_PARAM_PFVF_RWXCAPS = 0x00,
+ FW_PARAMS_PARAM_PFVF_ROUTE_START = 0x01,
+ FW_PARAMS_PARAM_PFVF_ROUTE_END = 0x02,
+ FW_PARAMS_PARAM_PFVF_CLIP_START = 0x03,
+ FW_PARAMS_PARAM_PFVF_CLIP_END = 0x04,
+ FW_PARAMS_PARAM_PFVF_FILTER_START = 0x05,
+ FW_PARAMS_PARAM_PFVF_FILTER_END = 0x06,
+ FW_PARAMS_PARAM_PFVF_SERVER_START = 0x07,
+ FW_PARAMS_PARAM_PFVF_SERVER_END = 0x08,
+ FW_PARAMS_PARAM_PFVF_TDDP_START = 0x09,
+ FW_PARAMS_PARAM_PFVF_TDDP_END = 0x0A,
+ FW_PARAMS_PARAM_PFVF_ISCSI_START = 0x0B,
+ FW_PARAMS_PARAM_PFVF_ISCSI_END = 0x0C,
+ FW_PARAMS_PARAM_PFVF_STAG_START = 0x0D,
+ FW_PARAMS_PARAM_PFVF_STAG_END = 0x0E,
+ FW_PARAMS_PARAM_PFVF_RQ_START = 0x1F,
+ FW_PARAMS_PARAM_PFVF_RQ_END = 0x10,
+ FW_PARAMS_PARAM_PFVF_PBL_START = 0x11,
+ FW_PARAMS_PARAM_PFVF_PBL_END = 0x12,
+ FW_PARAMS_PARAM_PFVF_L2T_START = 0x13,
+ FW_PARAMS_PARAM_PFVF_L2T_END = 0x14,
+ FW_PARAMS_PARAM_PFVF_SCHEDCLASS_ETH = 0x20,
+};
+
+/*
+ * dma queue parameters
+ */
+enum fw_params_param_dmaq {
+ FW_PARAMS_PARAM_DMAQ_IQ_DCAEN_DCACPU = 0x00,
+ FW_PARAMS_PARAM_DMAQ_IQ_INTCNTTHRESH = 0x01,
+ FW_PARAMS_PARAM_DMAQ_EQ_CMPLIQID_MNGT = 0x10,
+ FW_PARAMS_PARAM_DMAQ_EQ_CMPLIQID_CTRL = 0x11,
+ FW_PARAMS_PARAM_DMAQ_EQ_SCHEDCLASS_ETH = 0x12,
+};
+
+#define FW_PARAMS_MNEM(x) ((x) << 24)
+#define FW_PARAMS_PARAM_X(x) ((x) << 16)
+#define FW_PARAMS_PARAM_Y(x) ((x) << 8)
+#define FW_PARAMS_PARAM_Z(x) ((x) << 0)
+#define FW_PARAMS_PARAM_XYZ(x) ((x) << 0)
+#define FW_PARAMS_PARAM_YZ(x) ((x) << 0)
+
+struct fw_params_cmd {
+ __be32 op_to_vfn;
+ __be32 retval_len16;
+ struct fw_params_param {
+ __be32 mnem;
+ __be32 val;
+ } param[7];
+};
+
+#define FW_PARAMS_CMD_PFN(x) ((x) << 8)
+#define FW_PARAMS_CMD_VFN(x) ((x) << 0)
+
+struct fw_pfvf_cmd {
+ __be32 op_to_vfn;
+ __be32 retval_len16;
+ __be32 niqflint_niq;
+ __be32 cmask_to_neq;
+ __be32 tc_to_nexactf;
+ __be32 r_caps_to_nethctrl;
+ __be16 nricq;
+ __be16 nriqp;
+ __be32 r4;
+};
+
+#define FW_PFVF_CMD_PFN(x) ((x) << 8)
+#define FW_PFVF_CMD_VFN(x) ((x) << 0)
+
+#define FW_PFVF_CMD_NIQFLINT(x) ((x) << 20)
+#define FW_PFVF_CMD_NIQFLINT_GET(x) (((x) >> 20) & 0xfff)
+
+#define FW_PFVF_CMD_NIQ(x) ((x) << 0)
+#define FW_PFVF_CMD_NIQ_GET(x) (((x) >> 0) & 0xfffff)
+
+#define FW_PFVF_CMD_CMASK(x) ((x) << 24)
+#define FW_PFVF_CMD_CMASK_GET(x) (((x) >> 24) & 0xf)
+
+#define FW_PFVF_CMD_PMASK(x) ((x) << 20)
+#define FW_PFVF_CMD_PMASK_GET(x) (((x) >> 20) & 0xf)
+
+#define FW_PFVF_CMD_NEQ(x) ((x) << 0)
+#define FW_PFVF_CMD_NEQ_GET(x) (((x) >> 0) & 0xfffff)
+
+#define FW_PFVF_CMD_TC(x) ((x) << 24)
+#define FW_PFVF_CMD_TC_GET(x) (((x) >> 24) & 0xff)
+
+#define FW_PFVF_CMD_NVI(x) ((x) << 16)
+#define FW_PFVF_CMD_NVI_GET(x) (((x) >> 16) & 0xff)
+
+#define FW_PFVF_CMD_NEXACTF(x) ((x) << 0)
+#define FW_PFVF_CMD_NEXACTF_GET(x) (((x) >> 0) & 0xffff)
+
+#define FW_PFVF_CMD_R_CAPS(x) ((x) << 24)
+#define FW_PFVF_CMD_R_CAPS_GET(x) (((x) >> 24) & 0xff)
+
+#define FW_PFVF_CMD_WX_CAPS(x) ((x) << 16)
+#define FW_PFVF_CMD_WX_CAPS_GET(x) (((x) >> 16) & 0xff)
+
+#define FW_PFVF_CMD_NETHCTRL(x) ((x) << 0)
+#define FW_PFVF_CMD_NETHCTRL_GET(x) (((x) >> 0) & 0xffff)
+
+enum fw_iq_type {
+ FW_IQ_TYPE_FL_INT_CAP,
+ FW_IQ_TYPE_NO_FL_INT_CAP
+};
+
+struct fw_iq_cmd {
+ __be32 op_to_vfn;
+ __be32 alloc_to_len16;
+ __be16 physiqid;
+ __be16 iqid;
+ __be16 fl0id;
+ __be16 fl1id;
+ __be32 type_to_iqandstindex;
+ __be16 iqdroprss_to_iqesize;
+ __be16 iqsize;
+ __be64 iqaddr;
+ __be32 iqns_to_fl0congen;
+ __be16 fl0dcaen_to_fl0cidxfthresh;
+ __be16 fl0size;
+ __be64 fl0addr;
+ __be32 fl1cngchmap_to_fl1congen;
+ __be16 fl1dcaen_to_fl1cidxfthresh;
+ __be16 fl1size;
+ __be64 fl1addr;
+};
+
+#define FW_IQ_CMD_PFN(x) ((x) << 8)
+#define FW_IQ_CMD_VFN(x) ((x) << 0)
+
+#define FW_IQ_CMD_ALLOC (1U << 31)
+#define FW_IQ_CMD_FREE (1U << 30)
+#define FW_IQ_CMD_MODIFY (1U << 29)
+#define FW_IQ_CMD_IQSTART(x) ((x) << 28)
+#define FW_IQ_CMD_IQSTOP(x) ((x) << 27)
+
+#define FW_IQ_CMD_TYPE(x) ((x) << 29)
+#define FW_IQ_CMD_IQASYNCH(x) ((x) << 28)
+#define FW_IQ_CMD_VIID(x) ((x) << 16)
+#define FW_IQ_CMD_IQANDST(x) ((x) << 15)
+#define FW_IQ_CMD_IQANUS(x) ((x) << 14)
+#define FW_IQ_CMD_IQANUD(x) ((x) << 12)
+#define FW_IQ_CMD_IQANDSTINDEX(x) ((x) << 0)
+
+#define FW_IQ_CMD_IQDROPRSS (1U << 15)
+#define FW_IQ_CMD_IQGTSMODE (1U << 14)
+#define FW_IQ_CMD_IQPCIECH(x) ((x) << 12)
+#define FW_IQ_CMD_IQDCAEN(x) ((x) << 11)
+#define FW_IQ_CMD_IQDCACPU(x) ((x) << 6)
+#define FW_IQ_CMD_IQINTCNTTHRESH(x) ((x) << 4)
+#define FW_IQ_CMD_IQO (1U << 3)
+#define FW_IQ_CMD_IQCPRIO(x) ((x) << 2)
+#define FW_IQ_CMD_IQESIZE(x) ((x) << 0)
+
+#define FW_IQ_CMD_IQNS(x) ((x) << 31)
+#define FW_IQ_CMD_IQRO(x) ((x) << 30)
+#define FW_IQ_CMD_IQFLINTIQHSEN(x) ((x) << 28)
+#define FW_IQ_CMD_IQFLINTCONGEN(x) ((x) << 27)
+#define FW_IQ_CMD_IQFLINTISCSIC(x) ((x) << 26)
+#define FW_IQ_CMD_FL0CNGCHMAP(x) ((x) << 20)
+#define FW_IQ_CMD_FL0CACHELOCK(x) ((x) << 15)
+#define FW_IQ_CMD_FL0DBP(x) ((x) << 14)
+#define FW_IQ_CMD_FL0DATANS(x) ((x) << 13)
+#define FW_IQ_CMD_FL0DATARO(x) ((x) << 12)
+#define FW_IQ_CMD_FL0CONGCIF(x) ((x) << 11)
+#define FW_IQ_CMD_FL0ONCHIP(x) ((x) << 10)
+#define FW_IQ_CMD_FL0STATUSPGNS(x) ((x) << 9)
+#define FW_IQ_CMD_FL0STATUSPGRO(x) ((x) << 8)
+#define FW_IQ_CMD_FL0FETCHNS(x) ((x) << 7)
+#define FW_IQ_CMD_FL0FETCHRO(x) ((x) << 6)
+#define FW_IQ_CMD_FL0HOSTFCMODE(x) ((x) << 4)
+#define FW_IQ_CMD_FL0CPRIO(x) ((x) << 3)
+#define FW_IQ_CMD_FL0PADEN (1U << 2)
+#define FW_IQ_CMD_FL0PACKEN (1U << 1)
+#define FW_IQ_CMD_FL0CONGEN (1U << 0)
+
+#define FW_IQ_CMD_FL0DCAEN(x) ((x) << 15)
+#define FW_IQ_CMD_FL0DCACPU(x) ((x) << 10)
+#define FW_IQ_CMD_FL0FBMIN(x) ((x) << 7)
+#define FW_IQ_CMD_FL0FBMAX(x) ((x) << 4)
+#define FW_IQ_CMD_FL0CIDXFTHRESHO (1U << 3)
+#define FW_IQ_CMD_FL0CIDXFTHRESH(x) ((x) << 0)
+
+#define FW_IQ_CMD_FL1CNGCHMAP(x) ((x) << 20)
+#define FW_IQ_CMD_FL1CACHELOCK(x) ((x) << 15)
+#define FW_IQ_CMD_FL1DBP(x) ((x) << 14)
+#define FW_IQ_CMD_FL1DATANS(x) ((x) << 13)
+#define FW_IQ_CMD_FL1DATARO(x) ((x) << 12)
+#define FW_IQ_CMD_FL1CONGCIF(x) ((x) << 11)
+#define FW_IQ_CMD_FL1ONCHIP(x) ((x) << 10)
+#define FW_IQ_CMD_FL1STATUSPGNS(x) ((x) << 9)
+#define FW_IQ_CMD_FL1STATUSPGRO(x) ((x) << 8)
+#define FW_IQ_CMD_FL1FETCHNS(x) ((x) << 7)
+#define FW_IQ_CMD_FL1FETCHRO(x) ((x) << 6)
+#define FW_IQ_CMD_FL1HOSTFCMODE(x) ((x) << 4)
+#define FW_IQ_CMD_FL1CPRIO(x) ((x) << 3)
+#define FW_IQ_CMD_FL1PADEN (1U << 2)
+#define FW_IQ_CMD_FL1PACKEN (1U << 1)
+#define FW_IQ_CMD_FL1CONGEN (1U << 0)
+
+#define FW_IQ_CMD_FL1DCAEN(x) ((x) << 15)
+#define FW_IQ_CMD_FL1DCACPU(x) ((x) << 10)
+#define FW_IQ_CMD_FL1FBMIN(x) ((x) << 7)
+#define FW_IQ_CMD_FL1FBMAX(x) ((x) << 4)
+#define FW_IQ_CMD_FL1CIDXFTHRESHO (1U << 3)
+#define FW_IQ_CMD_FL1CIDXFTHRESH(x) ((x) << 0)
+
+struct fw_eq_eth_cmd {
+ __be32 op_to_vfn;
+ __be32 alloc_to_len16;
+ __be32 eqid_pkd;
+ __be32 physeqid_pkd;
+ __be32 fetchszm_to_iqid;
+ __be32 dcaen_to_eqsize;
+ __be64 eqaddr;
+ __be32 viid_pkd;
+ __be32 r8_lo;
+ __be64 r9;
+};
+
+#define FW_EQ_ETH_CMD_PFN(x) ((x) << 8)
+#define FW_EQ_ETH_CMD_VFN(x) ((x) << 0)
+#define FW_EQ_ETH_CMD_ALLOC (1U << 31)
+#define FW_EQ_ETH_CMD_FREE (1U << 30)
+#define FW_EQ_ETH_CMD_MODIFY (1U << 29)
+#define FW_EQ_ETH_CMD_EQSTART (1U << 28)
+#define FW_EQ_ETH_CMD_EQSTOP (1U << 27)
+
+#define FW_EQ_ETH_CMD_EQID(x) ((x) << 0)
+#define FW_EQ_ETH_CMD_EQID_GET(x) (((x) >> 0) & 0xfffff)
+#define FW_EQ_ETH_CMD_PHYSEQID(x) ((x) << 0)
+
+#define FW_EQ_ETH_CMD_FETCHSZM(x) ((x) << 26)
+#define FW_EQ_ETH_CMD_STATUSPGNS(x) ((x) << 25)
+#define FW_EQ_ETH_CMD_STATUSPGRO(x) ((x) << 24)
+#define FW_EQ_ETH_CMD_FETCHNS(x) ((x) << 23)
+#define FW_EQ_ETH_CMD_FETCHRO(x) ((x) << 22)
+#define FW_EQ_ETH_CMD_HOSTFCMODE(x) ((x) << 20)
+#define FW_EQ_ETH_CMD_CPRIO(x) ((x) << 19)
+#define FW_EQ_ETH_CMD_ONCHIP(x) ((x) << 18)
+#define FW_EQ_ETH_CMD_PCIECHN(x) ((x) << 16)
+#define FW_EQ_ETH_CMD_IQID(x) ((x) << 0)
+
+#define FW_EQ_ETH_CMD_DCAEN(x) ((x) << 31)
+#define FW_EQ_ETH_CMD_DCACPU(x) ((x) << 26)
+#define FW_EQ_ETH_CMD_FBMIN(x) ((x) << 23)
+#define FW_EQ_ETH_CMD_FBMAX(x) ((x) << 20)
+#define FW_EQ_ETH_CMD_CIDXFTHRESHO(x) ((x) << 19)
+#define FW_EQ_ETH_CMD_CIDXFTHRESH(x) ((x) << 16)
+#define FW_EQ_ETH_CMD_EQSIZE(x) ((x) << 0)
+
+#define FW_EQ_ETH_CMD_VIID(x) ((x) << 16)
+
+struct fw_eq_ctrl_cmd {
+ __be32 op_to_vfn;
+ __be32 alloc_to_len16;
+ __be32 cmpliqid_eqid;
+ __be32 physeqid_pkd;
+ __be32 fetchszm_to_iqid;
+ __be32 dcaen_to_eqsize;
+ __be64 eqaddr;
+};
+
+#define FW_EQ_CTRL_CMD_PFN(x) ((x) << 8)
+#define FW_EQ_CTRL_CMD_VFN(x) ((x) << 0)
+
+#define FW_EQ_CTRL_CMD_ALLOC (1U << 31)
+#define FW_EQ_CTRL_CMD_FREE (1U << 30)
+#define FW_EQ_CTRL_CMD_MODIFY (1U << 29)
+#define FW_EQ_CTRL_CMD_EQSTART (1U << 28)
+#define FW_EQ_CTRL_CMD_EQSTOP (1U << 27)
+
+#define FW_EQ_CTRL_CMD_CMPLIQID(x) ((x) << 20)
+#define FW_EQ_CTRL_CMD_EQID(x) ((x) << 0)
+#define FW_EQ_CTRL_CMD_EQID_GET(x) (((x) >> 0) & 0xfffff)
+#define FW_EQ_CTRL_CMD_PHYSEQID_GET(x) (((x) >> 0) & 0xfffff)
+
+#define FW_EQ_CTRL_CMD_FETCHSZM (1U << 26)
+#define FW_EQ_CTRL_CMD_STATUSPGNS (1U << 25)
+#define FW_EQ_CTRL_CMD_STATUSPGRO (1U << 24)
+#define FW_EQ_CTRL_CMD_FETCHNS (1U << 23)
+#define FW_EQ_CTRL_CMD_FETCHRO (1U << 22)
+#define FW_EQ_CTRL_CMD_HOSTFCMODE(x) ((x) << 20)
+#define FW_EQ_CTRL_CMD_CPRIO(x) ((x) << 19)
+#define FW_EQ_CTRL_CMD_ONCHIP(x) ((x) << 18)
+#define FW_EQ_CTRL_CMD_PCIECHN(x) ((x) << 16)
+#define FW_EQ_CTRL_CMD_IQID(x) ((x) << 0)
+
+#define FW_EQ_CTRL_CMD_DCAEN(x) ((x) << 31)
+#define FW_EQ_CTRL_CMD_DCACPU(x) ((x) << 26)
+#define FW_EQ_CTRL_CMD_FBMIN(x) ((x) << 23)
+#define FW_EQ_CTRL_CMD_FBMAX(x) ((x) << 20)
+#define FW_EQ_CTRL_CMD_CIDXFTHRESHO(x) ((x) << 19)
+#define FW_EQ_CTRL_CMD_CIDXFTHRESH(x) ((x) << 16)
+#define FW_EQ_CTRL_CMD_EQSIZE(x) ((x) << 0)
+
+struct fw_eq_ofld_cmd {
+ __be32 op_to_vfn;
+ __be32 alloc_to_len16;
+ __be32 eqid_pkd;
+ __be32 physeqid_pkd;
+ __be32 fetchszm_to_iqid;
+ __be32 dcaen_to_eqsize;
+ __be64 eqaddr;
+};
+
+#define FW_EQ_OFLD_CMD_PFN(x) ((x) << 8)
+#define FW_EQ_OFLD_CMD_VFN(x) ((x) << 0)
+
+#define FW_EQ_OFLD_CMD_ALLOC (1U << 31)
+#define FW_EQ_OFLD_CMD_FREE (1U << 30)
+#define FW_EQ_OFLD_CMD_MODIFY (1U << 29)
+#define FW_EQ_OFLD_CMD_EQSTART (1U << 28)
+#define FW_EQ_OFLD_CMD_EQSTOP (1U << 27)
+
+#define FW_EQ_OFLD_CMD_EQID(x) ((x) << 0)
+#define FW_EQ_OFLD_CMD_EQID_GET(x) (((x) >> 0) & 0xfffff)
+#define FW_EQ_OFLD_CMD_PHYSEQID_GET(x) (((x) >> 0) & 0xfffff)
+
+#define FW_EQ_OFLD_CMD_FETCHSZM(x) ((x) << 26)
+#define FW_EQ_OFLD_CMD_STATUSPGNS(x) ((x) << 25)
+#define FW_EQ_OFLD_CMD_STATUSPGRO(x) ((x) << 24)
+#define FW_EQ_OFLD_CMD_FETCHNS(x) ((x) << 23)
+#define FW_EQ_OFLD_CMD_FETCHRO(x) ((x) << 22)
+#define FW_EQ_OFLD_CMD_HOSTFCMODE(x) ((x) << 20)
+#define FW_EQ_OFLD_CMD_CPRIO(x) ((x) << 19)
+#define FW_EQ_OFLD_CMD_ONCHIP(x) ((x) << 18)
+#define FW_EQ_OFLD_CMD_PCIECHN(x) ((x) << 16)
+#define FW_EQ_OFLD_CMD_IQID(x) ((x) << 0)
+
+#define FW_EQ_OFLD_CMD_DCAEN(x) ((x) << 31)
+#define FW_EQ_OFLD_CMD_DCACPU(x) ((x) << 26)
+#define FW_EQ_OFLD_CMD_FBMIN(x) ((x) << 23)
+#define FW_EQ_OFLD_CMD_FBMAX(x) ((x) << 20)
+#define FW_EQ_OFLD_CMD_CIDXFTHRESHO(x) ((x) << 19)
+#define FW_EQ_OFLD_CMD_CIDXFTHRESH(x) ((x) << 16)
+#define FW_EQ_OFLD_CMD_EQSIZE(x) ((x) << 0)
+
+/*
+ * Macros for VIID parsing:
+ * VIID - [10:8] PFN, [7] VI Valid, [6:0] VI number
+ */
+#define FW_VIID_PFN_GET(x) (((x) >> 8) & 0x7)
+#define FW_VIID_VIVLD_GET(x) (((x) >> 7) & 0x1)
+#define FW_VIID_VIN_GET(x) (((x) >> 0) & 0x7F)
+
+struct fw_vi_cmd {
+ __be32 op_to_vfn;
+ __be32 alloc_to_len16;
+ __be16 viid_pkd;
+ u8 mac[6];
+ u8 portid_pkd;
+ u8 nmac;
+ u8 nmac0[6];
+ __be16 rsssize_pkd;
+ u8 nmac1[6];
+ __be16 r7;
+ u8 nmac2[6];
+ __be16 r8;
+ u8 nmac3[6];
+ __be64 r9;
+ __be64 r10;
+};
+
+#define FW_VI_CMD_PFN(x) ((x) << 8)
+#define FW_VI_CMD_VFN(x) ((x) << 0)
+#define FW_VI_CMD_ALLOC (1U << 31)
+#define FW_VI_CMD_FREE (1U << 30)
+#define FW_VI_CMD_VIID(x) ((x) << 0)
+#define FW_VI_CMD_PORTID(x) ((x) << 4)
+#define FW_VI_CMD_RSSSIZE_GET(x) (((x) >> 0) & 0x7ff)
+
+/* Special VI_MAC command index ids */
+#define FW_VI_MAC_ADD_MAC 0x3FF
+#define FW_VI_MAC_ADD_PERSIST_MAC 0x3FE
+#define FW_VI_MAC_MAC_BASED_FREE 0x3FD
+
+enum fw_vi_mac_smac {
+ FW_VI_MAC_MPS_TCAM_ENTRY,
+ FW_VI_MAC_MPS_TCAM_ONLY,
+ FW_VI_MAC_SMT_ONLY,
+ FW_VI_MAC_SMT_AND_MPSTCAM
+};
+
+enum fw_vi_mac_result {
+ FW_VI_MAC_R_SUCCESS,
+ FW_VI_MAC_R_F_NONEXISTENT_NOMEM,
+ FW_VI_MAC_R_SMAC_FAIL,
+ FW_VI_MAC_R_F_ACL_CHECK
+};
+
+struct fw_vi_mac_cmd {
+ __be32 op_to_viid;
+ __be32 freemacs_to_len16;
+ union fw_vi_mac {
+ struct fw_vi_mac_exact {
+ __be16 valid_to_idx;
+ u8 macaddr[6];
+ } exact[7];
+ struct fw_vi_mac_hash {
+ __be64 hashvec;
+ } hash;
+ } u;
+};
+
+#define FW_VI_MAC_CMD_VIID(x) ((x) << 0)
+#define FW_VI_MAC_CMD_FREEMACS(x) ((x) << 31)
+#define FW_VI_MAC_CMD_HASHVECEN (1U << 23)
+#define FW_VI_MAC_CMD_HASHUNIEN(x) ((x) << 22)
+#define FW_VI_MAC_CMD_VALID (1U << 15)
+#define FW_VI_MAC_CMD_PRIO(x) ((x) << 12)
+#define FW_VI_MAC_CMD_SMAC_RESULT(x) ((x) << 10)
+#define FW_VI_MAC_CMD_SMAC_RESULT_GET(x) (((x) >> 10) & 0x3)
+#define FW_VI_MAC_CMD_IDX(x) ((x) << 0)
+#define FW_VI_MAC_CMD_IDX_GET(x) (((x) >> 0) & 0x3ff)
+
+#define FW_RXMODE_MTU_NO_CHG 65535
+
+struct fw_vi_rxmode_cmd {
+ __be32 op_to_viid;
+ __be32 retval_len16;
+ __be32 mtu_to_broadcasten;
+ __be32 r4_lo;
+};
+
+#define FW_VI_RXMODE_CMD_VIID(x) ((x) << 0)
+#define FW_VI_RXMODE_CMD_MTU(x) ((x) << 16)
+#define FW_VI_RXMODE_CMD_PROMISCEN_MASK 0x3
+#define FW_VI_RXMODE_CMD_PROMISCEN(x) ((x) << 14)
+#define FW_VI_RXMODE_CMD_ALLMULTIEN_MASK 0x3
+#define FW_VI_RXMODE_CMD_ALLMULTIEN(x) ((x) << 12)
+#define FW_VI_RXMODE_CMD_BROADCASTEN_MASK 0x3
+#define FW_VI_RXMODE_CMD_BROADCASTEN(x) ((x) << 10)
+
+struct fw_vi_enable_cmd {
+ __be32 op_to_viid;
+ __be32 ien_to_len16;
+ __be16 blinkdur;
+ __be16 r3;
+ __be32 r4;
+};
+
+#define FW_VI_ENABLE_CMD_VIID(x) ((x) << 0)
+#define FW_VI_ENABLE_CMD_IEN(x) ((x) << 31)
+#define FW_VI_ENABLE_CMD_EEN(x) ((x) << 30)
+#define FW_VI_ENABLE_CMD_LED (1U << 29)
+
+/* VI VF stats offset definitions */
+#define VI_VF_NUM_STATS 16
+enum fw_vi_stats_vf_index {
+ FW_VI_VF_STAT_TX_BCAST_BYTES_IX,
+ FW_VI_VF_STAT_TX_BCAST_FRAMES_IX,
+ FW_VI_VF_STAT_TX_MCAST_BYTES_IX,
+ FW_VI_VF_STAT_TX_MCAST_FRAMES_IX,
+ FW_VI_VF_STAT_TX_UCAST_BYTES_IX,
+ FW_VI_VF_STAT_TX_UCAST_FRAMES_IX,
+ FW_VI_VF_STAT_TX_DROP_FRAMES_IX,
+ FW_VI_VF_STAT_TX_OFLD_BYTES_IX,
+ FW_VI_VF_STAT_TX_OFLD_FRAMES_IX,
+ FW_VI_VF_STAT_RX_BCAST_BYTES_IX,
+ FW_VI_VF_STAT_RX_BCAST_FRAMES_IX,
+ FW_VI_VF_STAT_RX_MCAST_BYTES_IX,
+ FW_VI_VF_STAT_RX_MCAST_FRAMES_IX,
+ FW_VI_VF_STAT_RX_UCAST_BYTES_IX,
+ FW_VI_VF_STAT_RX_UCAST_FRAMES_IX,
+ FW_VI_VF_STAT_RX_ERR_FRAMES_IX
+};
+
+/* VI PF stats offset definitions */
+#define VI_PF_NUM_STATS 17
+enum fw_vi_stats_pf_index {
+ FW_VI_PF_STAT_TX_BCAST_BYTES_IX,
+ FW_VI_PF_STAT_TX_BCAST_FRAMES_IX,
+ FW_VI_PF_STAT_TX_MCAST_BYTES_IX,
+ FW_VI_PF_STAT_TX_MCAST_FRAMES_IX,
+ FW_VI_PF_STAT_TX_UCAST_BYTES_IX,
+ FW_VI_PF_STAT_TX_UCAST_FRAMES_IX,
+ FW_VI_PF_STAT_TX_OFLD_BYTES_IX,
+ FW_VI_PF_STAT_TX_OFLD_FRAMES_IX,
+ FW_VI_PF_STAT_RX_BYTES_IX,
+ FW_VI_PF_STAT_RX_FRAMES_IX,
+ FW_VI_PF_STAT_RX_BCAST_BYTES_IX,
+ FW_VI_PF_STAT_RX_BCAST_FRAMES_IX,
+ FW_VI_PF_STAT_RX_MCAST_BYTES_IX,
+ FW_VI_PF_STAT_RX_MCAST_FRAMES_IX,
+ FW_VI_PF_STAT_RX_UCAST_BYTES_IX,
+ FW_VI_PF_STAT_RX_UCAST_FRAMES_IX,
+ FW_VI_PF_STAT_RX_ERR_FRAMES_IX
+};
+
+struct fw_vi_stats_cmd {
+ __be32 op_to_viid;
+ __be32 retval_len16;
+ union fw_vi_stats {
+ struct fw_vi_stats_ctl {
+ __be16 nstats_ix;
+ __be16 r6;
+ __be32 r7;
+ __be64 stat0;
+ __be64 stat1;
+ __be64 stat2;
+ __be64 stat3;
+ __be64 stat4;
+ __be64 stat5;
+ } ctl;
+ struct fw_vi_stats_pf {
+ __be64 tx_bcast_bytes;
+ __be64 tx_bcast_frames;
+ __be64 tx_mcast_bytes;
+ __be64 tx_mcast_frames;
+ __be64 tx_ucast_bytes;
+ __be64 tx_ucast_frames;
+ __be64 tx_offload_bytes;
+ __be64 tx_offload_frames;
+ __be64 rx_pf_bytes;
+ __be64 rx_pf_frames;
+ __be64 rx_bcast_bytes;
+ __be64 rx_bcast_frames;
+ __be64 rx_mcast_bytes;
+ __be64 rx_mcast_frames;
+ __be64 rx_ucast_bytes;
+ __be64 rx_ucast_frames;
+ __be64 rx_err_frames;
+ } pf;
+ struct fw_vi_stats_vf {
+ __be64 tx_bcast_bytes;
+ __be64 tx_bcast_frames;
+ __be64 tx_mcast_bytes;
+ __be64 tx_mcast_frames;
+ __be64 tx_ucast_bytes;
+ __be64 tx_ucast_frames;
+ __be64 tx_drop_frames;
+ __be64 tx_offload_bytes;
+ __be64 tx_offload_frames;
+ __be64 rx_bcast_bytes;
+ __be64 rx_bcast_frames;
+ __be64 rx_mcast_bytes;
+ __be64 rx_mcast_frames;
+ __be64 rx_ucast_bytes;
+ __be64 rx_ucast_frames;
+ __be64 rx_err_frames;
+ } vf;
+ } u;
+};
+
+#define FW_VI_STATS_CMD_VIID(x) ((x) << 0)
+#define FW_VI_STATS_CMD_NSTATS(x) ((x) << 12)
+#define FW_VI_STATS_CMD_IX(x) ((x) << 0)
+
+struct fw_acl_mac_cmd {
+ __be32 op_to_vfn;
+ __be32 en_to_len16;
+ u8 nmac;
+ u8 r3[7];
+ __be16 r4;
+ u8 macaddr0[6];
+ __be16 r5;
+ u8 macaddr1[6];
+ __be16 r6;
+ u8 macaddr2[6];
+ __be16 r7;
+ u8 macaddr3[6];
+};
+
+#define FW_ACL_MAC_CMD_PFN(x) ((x) << 8)
+#define FW_ACL_MAC_CMD_VFN(x) ((x) << 0)
+#define FW_ACL_MAC_CMD_EN(x) ((x) << 31)
+
+struct fw_acl_vlan_cmd {
+ __be32 op_to_vfn;
+ __be32 en_to_len16;
+ u8 nvlan;
+ u8 dropnovlan_fm;
+ u8 r3_lo[6];
+ __be16 vlanid[16];
+};
+
+#define FW_ACL_VLAN_CMD_PFN(x) ((x) << 8)
+#define FW_ACL_VLAN_CMD_VFN(x) ((x) << 0)
+#define FW_ACL_VLAN_CMD_EN(x) ((x) << 31)
+#define FW_ACL_VLAN_CMD_DROPNOVLAN(x) ((x) << 7)
+#define FW_ACL_VLAN_CMD_FM(x) ((x) << 6)
+
+enum fw_port_cap {
+ FW_PORT_CAP_SPEED_100M = 0x0001,
+ FW_PORT_CAP_SPEED_1G = 0x0002,
+ FW_PORT_CAP_SPEED_2_5G = 0x0004,
+ FW_PORT_CAP_SPEED_10G = 0x0008,
+ FW_PORT_CAP_SPEED_40G = 0x0010,
+ FW_PORT_CAP_SPEED_100G = 0x0020,
+ FW_PORT_CAP_FC_RX = 0x0040,
+ FW_PORT_CAP_FC_TX = 0x0080,
+ FW_PORT_CAP_ANEG = 0x0100,
+ FW_PORT_CAP_MDI_0 = 0x0200,
+ FW_PORT_CAP_MDI_1 = 0x0400,
+ FW_PORT_CAP_BEAN = 0x0800,
+ FW_PORT_CAP_PMA_LPBK = 0x1000,
+ FW_PORT_CAP_PCS_LPBK = 0x2000,
+ FW_PORT_CAP_PHYXS_LPBK = 0x4000,
+ FW_PORT_CAP_FAR_END_LPBK = 0x8000,
+};
+
+enum fw_port_mdi {
+ FW_PORT_MDI_UNCHANGED,
+ FW_PORT_MDI_AUTO,
+ FW_PORT_MDI_F_STRAIGHT,
+ FW_PORT_MDI_F_CROSSOVER
+};
+
+#define FW_PORT_MDI(x) ((x) << 9)
+
+enum fw_port_action {
+ FW_PORT_ACTION_L1_CFG = 0x0001,
+ FW_PORT_ACTION_L2_CFG = 0x0002,
+ FW_PORT_ACTION_GET_PORT_INFO = 0x0003,
+ FW_PORT_ACTION_L2_PPP_CFG = 0x0004,
+ FW_PORT_ACTION_L2_DCB_CFG = 0x0005,
+ FW_PORT_ACTION_LOW_PWR_TO_NORMAL = 0x0010,
+ FW_PORT_ACTION_L1_LOW_PWR_EN = 0x0011,
+ FW_PORT_ACTION_L2_WOL_MODE_EN = 0x0012,
+ FW_PORT_ACTION_LPBK_TO_NORMAL = 0x0020,
+ FW_PORT_ACTION_L1_LPBK = 0x0021,
+ FW_PORT_ACTION_L1_PMA_LPBK = 0x0022,
+ FW_PORT_ACTION_L1_PCS_LPBK = 0x0023,
+ FW_PORT_ACTION_L1_PHYXS_CSIDE_LPBK = 0x0024,
+ FW_PORT_ACTION_L1_PHYXS_ESIDE_LPBK = 0x0025,
+ FW_PORT_ACTION_PHY_RESET = 0x0040,
+ FW_PORT_ACTION_PMA_RESET = 0x0041,
+ FW_PORT_ACTION_PCS_RESET = 0x0042,
+ FW_PORT_ACTION_PHYXS_RESET = 0x0043,
+ FW_PORT_ACTION_DTEXS_REEST = 0x0044,
+ FW_PORT_ACTION_AN_RESET = 0x0045
+};
+
+enum fw_port_l2cfg_ctlbf {
+ FW_PORT_L2_CTLBF_OVLAN0 = 0x01,
+ FW_PORT_L2_CTLBF_OVLAN1 = 0x02,
+ FW_PORT_L2_CTLBF_OVLAN2 = 0x04,
+ FW_PORT_L2_CTLBF_OVLAN3 = 0x08,
+ FW_PORT_L2_CTLBF_IVLAN = 0x10,
+ FW_PORT_L2_CTLBF_TXIPG = 0x20
+};
+
+enum fw_port_dcb_cfg {
+ FW_PORT_DCB_CFG_PG = 0x01,
+ FW_PORT_DCB_CFG_PFC = 0x02,
+ FW_PORT_DCB_CFG_APPL = 0x04
+};
+
+enum fw_port_dcb_cfg_rc {
+ FW_PORT_DCB_CFG_SUCCESS = 0x0,
+ FW_PORT_DCB_CFG_ERROR = 0x1
+};
+
+struct fw_port_cmd {
+ __be32 op_to_portid;
+ __be32 action_to_len16;
+ union fw_port {
+ struct fw_port_l1cfg {
+ __be32 rcap;
+ __be32 r;
+ } l1cfg;
+ struct fw_port_l2cfg {
+ __be16 ctlbf_to_ivlan0;
+ __be16 ivlantype;
+ __be32 txipg_pkd;
+ __be16 ovlan0mask;
+ __be16 ovlan0type;
+ __be16 ovlan1mask;
+ __be16 ovlan1type;
+ __be16 ovlan2mask;
+ __be16 ovlan2type;
+ __be16 ovlan3mask;
+ __be16 ovlan3type;
+ } l2cfg;
+ struct fw_port_info {
+ __be32 lstatus_to_modtype;
+ __be16 pcap;
+ __be16 acap;
+ } info;
+ struct fw_port_ppp {
+ __be32 pppen_to_ncsich;
+ __be32 r11;
+ } ppp;
+ struct fw_port_dcb {
+ __be16 cfg;
+ u8 up_map;
+ u8 sf_cfgrc;
+ __be16 prot_ix;
+ u8 pe7_to_pe0;
+ u8 numTCPFCs;
+ __be32 pgid0_to_pgid7;
+ __be32 numTCs_oui;
+ u8 pgpc[8];
+ } dcb;
+ } u;
+};
+
+#define FW_PORT_CMD_READ (1U << 22)
+
+#define FW_PORT_CMD_PORTID(x) ((x) << 0)
+#define FW_PORT_CMD_PORTID_GET(x) (((x) >> 0) & 0xf)
+
+#define FW_PORT_CMD_ACTION(x) ((x) << 16)
+
+#define FW_PORT_CMD_CTLBF(x) ((x) << 10)
+#define FW_PORT_CMD_OVLAN3(x) ((x) << 7)
+#define FW_PORT_CMD_OVLAN2(x) ((x) << 6)
+#define FW_PORT_CMD_OVLAN1(x) ((x) << 5)
+#define FW_PORT_CMD_OVLAN0(x) ((x) << 4)
+#define FW_PORT_CMD_IVLAN0(x) ((x) << 3)
+
+#define FW_PORT_CMD_TXIPG(x) ((x) << 19)
+
+#define FW_PORT_CMD_LSTATUS (1U << 31)
+#define FW_PORT_CMD_LSPEED(x) ((x) << 24)
+#define FW_PORT_CMD_LSPEED_GET(x) (((x) >> 24) & 0x3f)
+#define FW_PORT_CMD_TXPAUSE (1U << 23)
+#define FW_PORT_CMD_RXPAUSE (1U << 22)
+#define FW_PORT_CMD_MDIOCAP (1U << 21)
+#define FW_PORT_CMD_MDIOADDR_GET(x) (((x) >> 16) & 0x1f)
+#define FW_PORT_CMD_LPTXPAUSE (1U << 15)
+#define FW_PORT_CMD_LPRXPAUSE (1U << 14)
+#define FW_PORT_CMD_PTYPE_MASK 0x1f
+#define FW_PORT_CMD_PTYPE_GET(x) (((x) >> 8) & FW_PORT_CMD_PTYPE_MASK)
+#define FW_PORT_CMD_MODTYPE_MASK 0x1f
+#define FW_PORT_CMD_MODTYPE_GET(x) (((x) >> 0) & FW_PORT_CMD_MODTYPE_MASK)
+
+#define FW_PORT_CMD_PPPEN(x) ((x) << 31)
+#define FW_PORT_CMD_TPSRC(x) ((x) << 28)
+#define FW_PORT_CMD_NCSISRC(x) ((x) << 24)
+
+#define FW_PORT_CMD_CH0(x) ((x) << 20)
+#define FW_PORT_CMD_CH1(x) ((x) << 16)
+#define FW_PORT_CMD_CH2(x) ((x) << 12)
+#define FW_PORT_CMD_CH3(x) ((x) << 8)
+#define FW_PORT_CMD_NCSICH(x) ((x) << 4)
+
+enum fw_port_type {
+ FW_PORT_TYPE_FIBER,
+ FW_PORT_TYPE_KX4,
+ FW_PORT_TYPE_BT_SGMII,
+ FW_PORT_TYPE_KX,
+ FW_PORT_TYPE_BT_XAUI,
+ FW_PORT_TYPE_KR,
+ FW_PORT_TYPE_CX4,
+ FW_PORT_TYPE_TWINAX,
+
+ FW_PORT_TYPE_NONE = FW_PORT_CMD_PTYPE_MASK
+};
+
+enum fw_port_module_type {
+ FW_PORT_MOD_TYPE_NA,
+ FW_PORT_MOD_TYPE_LR,
+ FW_PORT_MOD_TYPE_SR,
+ FW_PORT_MOD_TYPE_ER,
+
+ FW_PORT_MOD_TYPE_NONE = FW_PORT_CMD_MODTYPE_MASK
+};
+
+/* port stats */
+#define FW_NUM_PORT_STATS 50
+#define FW_NUM_PORT_TX_STATS 23
+#define FW_NUM_PORT_RX_STATS 27
+
+enum fw_port_stats_tx_index {
+ FW_STAT_TX_PORT_BYTES_IX,
+ FW_STAT_TX_PORT_FRAMES_IX,
+ FW_STAT_TX_PORT_BCAST_IX,
+ FW_STAT_TX_PORT_MCAST_IX,
+ FW_STAT_TX_PORT_UCAST_IX,
+ FW_STAT_TX_PORT_ERROR_IX,
+ FW_STAT_TX_PORT_64B_IX,
+ FW_STAT_TX_PORT_65B_127B_IX,
+ FW_STAT_TX_PORT_128B_255B_IX,
+ FW_STAT_TX_PORT_256B_511B_IX,
+ FW_STAT_TX_PORT_512B_1023B_IX,
+ FW_STAT_TX_PORT_1024B_1518B_IX,
+ FW_STAT_TX_PORT_1519B_MAX_IX,
+ FW_STAT_TX_PORT_DROP_IX,
+ FW_STAT_TX_PORT_PAUSE_IX,
+ FW_STAT_TX_PORT_PPP0_IX,
+ FW_STAT_TX_PORT_PPP1_IX,
+ FW_STAT_TX_PORT_PPP2_IX,
+ FW_STAT_TX_PORT_PPP3_IX,
+ FW_STAT_TX_PORT_PPP4_IX,
+ FW_STAT_TX_PORT_PPP5_IX,
+ FW_STAT_TX_PORT_PPP6_IX,
+ FW_STAT_TX_PORT_PPP7_IX
+};
+
+enum fw_port_stat_rx_index {
+ FW_STAT_RX_PORT_BYTES_IX,
+ FW_STAT_RX_PORT_FRAMES_IX,
+ FW_STAT_RX_PORT_BCAST_IX,
+ FW_STAT_RX_PORT_MCAST_IX,
+ FW_STAT_RX_PORT_UCAST_IX,
+ FW_STAT_RX_PORT_MTU_ERROR_IX,
+ FW_STAT_RX_PORT_MTU_CRC_ERROR_IX,
+ FW_STAT_RX_PORT_CRC_ERROR_IX,
+ FW_STAT_RX_PORT_LEN_ERROR_IX,
+ FW_STAT_RX_PORT_SYM_ERROR_IX,
+ FW_STAT_RX_PORT_64B_IX,
+ FW_STAT_RX_PORT_65B_127B_IX,
+ FW_STAT_RX_PORT_128B_255B_IX,
+ FW_STAT_RX_PORT_256B_511B_IX,
+ FW_STAT_RX_PORT_512B_1023B_IX,
+ FW_STAT_RX_PORT_1024B_1518B_IX,
+ FW_STAT_RX_PORT_1519B_MAX_IX,
+ FW_STAT_RX_PORT_PAUSE_IX,
+ FW_STAT_RX_PORT_PPP0_IX,
+ FW_STAT_RX_PORT_PPP1_IX,
+ FW_STAT_RX_PORT_PPP2_IX,
+ FW_STAT_RX_PORT_PPP3_IX,
+ FW_STAT_RX_PORT_PPP4_IX,
+ FW_STAT_RX_PORT_PPP5_IX,
+ FW_STAT_RX_PORT_PPP6_IX,
+ FW_STAT_RX_PORT_PPP7_IX,
+ FW_STAT_RX_PORT_LESS_64B_IX
+};
+
+struct fw_port_stats_cmd {
+ __be32 op_to_portid;
+ __be32 retval_len16;
+ union fw_port_stats {
+ struct fw_port_stats_ctl {
+ u8 nstats_bg_bm;
+ u8 tx_ix;
+ __be16 r6;
+ __be32 r7;
+ __be64 stat0;
+ __be64 stat1;
+ __be64 stat2;
+ __be64 stat3;
+ __be64 stat4;
+ __be64 stat5;
+ } ctl;
+ struct fw_port_stats_all {
+ __be64 tx_bytes;
+ __be64 tx_frames;
+ __be64 tx_bcast;
+ __be64 tx_mcast;
+ __be64 tx_ucast;
+ __be64 tx_error;
+ __be64 tx_64b;
+ __be64 tx_65b_127b;
+ __be64 tx_128b_255b;
+ __be64 tx_256b_511b;
+ __be64 tx_512b_1023b;
+ __be64 tx_1024b_1518b;
+ __be64 tx_1519b_max;
+ __be64 tx_drop;
+ __be64 tx_pause;
+ __be64 tx_ppp0;
+ __be64 tx_ppp1;
+ __be64 tx_ppp2;
+ __be64 tx_ppp3;
+ __be64 tx_ppp4;
+ __be64 tx_ppp5;
+ __be64 tx_ppp6;
+ __be64 tx_ppp7;
+ __be64 rx_bytes;
+ __be64 rx_frames;
+ __be64 rx_bcast;
+ __be64 rx_mcast;
+ __be64 rx_ucast;
+ __be64 rx_mtu_error;
+ __be64 rx_mtu_crc_error;
+ __be64 rx_crc_error;
+ __be64 rx_len_error;
+ __be64 rx_sym_error;
+ __be64 rx_64b;
+ __be64 rx_65b_127b;
+ __be64 rx_128b_255b;
+ __be64 rx_256b_511b;
+ __be64 rx_512b_1023b;
+ __be64 rx_1024b_1518b;
+ __be64 rx_1519b_max;
+ __be64 rx_pause;
+ __be64 rx_ppp0;
+ __be64 rx_ppp1;
+ __be64 rx_ppp2;
+ __be64 rx_ppp3;
+ __be64 rx_ppp4;
+ __be64 rx_ppp5;
+ __be64 rx_ppp6;
+ __be64 rx_ppp7;
+ __be64 rx_less_64b;
+ __be64 rx_bg_drop;
+ __be64 rx_bg_trunc;
+ } all;
+ } u;
+};
+
+#define FW_PORT_STATS_CMD_NSTATS(x) ((x) << 4)
+#define FW_PORT_STATS_CMD_BG_BM(x) ((x) << 0)
+#define FW_PORT_STATS_CMD_TX(x) ((x) << 7)
+#define FW_PORT_STATS_CMD_IX(x) ((x) << 0)
+
+/* port loopback stats */
+#define FW_NUM_LB_STATS 16
+enum fw_port_lb_stats_index {
+ FW_STAT_LB_PORT_BYTES_IX,
+ FW_STAT_LB_PORT_FRAMES_IX,
+ FW_STAT_LB_PORT_BCAST_IX,
+ FW_STAT_LB_PORT_MCAST_IX,
+ FW_STAT_LB_PORT_UCAST_IX,
+ FW_STAT_LB_PORT_ERROR_IX,
+ FW_STAT_LB_PORT_64B_IX,
+ FW_STAT_LB_PORT_65B_127B_IX,
+ FW_STAT_LB_PORT_128B_255B_IX,
+ FW_STAT_LB_PORT_256B_511B_IX,
+ FW_STAT_LB_PORT_512B_1023B_IX,
+ FW_STAT_LB_PORT_1024B_1518B_IX,
+ FW_STAT_LB_PORT_1519B_MAX_IX,
+ FW_STAT_LB_PORT_DROP_FRAMES_IX
+};
+
+struct fw_port_lb_stats_cmd {
+ __be32 op_to_lbport;
+ __be32 retval_len16;
+ union fw_port_lb_stats {
+ struct fw_port_lb_stats_ctl {
+ u8 nstats_bg_bm;
+ u8 ix_pkd;
+ __be16 r6;
+ __be32 r7;
+ __be64 stat0;
+ __be64 stat1;
+ __be64 stat2;
+ __be64 stat3;
+ __be64 stat4;
+ __be64 stat5;
+ } ctl;
+ struct fw_port_lb_stats_all {
+ __be64 tx_bytes;
+ __be64 tx_frames;
+ __be64 tx_bcast;
+ __be64 tx_mcast;
+ __be64 tx_ucast;
+ __be64 tx_error;
+ __be64 tx_64b;
+ __be64 tx_65b_127b;
+ __be64 tx_128b_255b;
+ __be64 tx_256b_511b;
+ __be64 tx_512b_1023b;
+ __be64 tx_1024b_1518b;
+ __be64 tx_1519b_max;
+ __be64 rx_lb_drop;
+ __be64 rx_lb_trunc;
+ } all;
+ } u;
+};
+
+#define FW_PORT_LB_STATS_CMD_LBPORT(x) ((x) << 0)
+#define FW_PORT_LB_STATS_CMD_NSTATS(x) ((x) << 4)
+#define FW_PORT_LB_STATS_CMD_BG_BM(x) ((x) << 0)
+#define FW_PORT_LB_STATS_CMD_IX(x) ((x) << 0)
+
+struct fw_rss_ind_tbl_cmd {
+ __be32 op_to_viid;
+#define FW_RSS_IND_TBL_CMD_VIID(x) ((x) << 0)
+ __be32 retval_len16;
+ __be16 niqid;
+ __be16 startidx;
+ __be32 r3;
+ __be32 iq0_to_iq2;
+#define FW_RSS_IND_TBL_CMD_IQ0(x) ((x) << 20)
+#define FW_RSS_IND_TBL_CMD_IQ1(x) ((x) << 10)
+#define FW_RSS_IND_TBL_CMD_IQ2(x) ((x) << 0)
+ __be32 iq3_to_iq5;
+ __be32 iq6_to_iq8;
+ __be32 iq9_to_iq11;
+ __be32 iq12_to_iq14;
+ __be32 iq15_to_iq17;
+ __be32 iq18_to_iq20;
+ __be32 iq21_to_iq23;
+ __be32 iq24_to_iq26;
+ __be32 iq27_to_iq29;
+ __be32 iq30_iq31;
+ __be32 r15_lo;
+};
+
+struct fw_rss_glb_config_cmd {
+ __be32 op_to_write;
+ __be32 retval_len16;
+ union fw_rss_glb_config {
+ struct fw_rss_glb_config_manual {
+ __be32 mode_pkd;
+ __be32 r3;
+ __be64 r4;
+ __be64 r5;
+ } manual;
+ struct fw_rss_glb_config_basicvirtual {
+ __be32 mode_pkd;
+ __be32 synmapen_to_hashtoeplitz;
+#define FW_RSS_GLB_CONFIG_CMD_SYNMAPEN (1U << 8)
+#define FW_RSS_GLB_CONFIG_CMD_SYN4TUPENIPV6 (1U << 7)
+#define FW_RSS_GLB_CONFIG_CMD_SYN2TUPENIPV6 (1U << 6)
+#define FW_RSS_GLB_CONFIG_CMD_SYN4TUPENIPV4 (1U << 5)
+#define FW_RSS_GLB_CONFIG_CMD_SYN2TUPENIPV4 (1U << 4)
+#define FW_RSS_GLB_CONFIG_CMD_OFDMAPEN (1U << 3)
+#define FW_RSS_GLB_CONFIG_CMD_TNLMAPEN (1U << 2)
+#define FW_RSS_GLB_CONFIG_CMD_TNLALLLKP (1U << 1)
+#define FW_RSS_GLB_CONFIG_CMD_HASHTOEPLITZ (1U << 0)
+ __be64 r8;
+ __be64 r9;
+ } basicvirtual;
+ } u;
+};
+
+#define FW_RSS_GLB_CONFIG_CMD_MODE(x) ((x) << 28)
+
+#define FW_RSS_GLB_CONFIG_CMD_MODE_MANUAL 0
+#define FW_RSS_GLB_CONFIG_CMD_MODE_BASICVIRTUAL 1
+
+struct fw_rss_vi_config_cmd {
+ __be32 op_to_viid;
+#define FW_RSS_VI_CONFIG_CMD_VIID(x) ((x) << 0)
+ __be32 retval_len16;
+ union fw_rss_vi_config {
+ struct fw_rss_vi_config_manual {
+ __be64 r3;
+ __be64 r4;
+ __be64 r5;
+ } manual;
+ struct fw_rss_vi_config_basicvirtual {
+ __be32 r6;
+ __be32 defaultq_to_ip4udpen;
+#define FW_RSS_VI_CONFIG_CMD_DEFAULTQ(x) ((x) << 16)
+#define FW_RSS_VI_CONFIG_CMD_IP6FOURTUPEN (1U << 4)
+#define FW_RSS_VI_CONFIG_CMD_IP6TWOTUPEN (1U << 3)
+#define FW_RSS_VI_CONFIG_CMD_IP4FOURTUPEN (1U << 2)
+#define FW_RSS_VI_CONFIG_CMD_IP4TWOTUPEN (1U << 1)
+#define FW_RSS_VI_CONFIG_CMD_IP4UDPEN (1U << 0)
+ __be64 r9;
+ __be64 r10;
+ } basicvirtual;
+ } u;
+};
+
+enum fw_error_type {
+ FW_ERROR_TYPE_EXCEPTION = 0x0,
+ FW_ERROR_TYPE_HWMODULE = 0x1,
+ FW_ERROR_TYPE_WR = 0x2,
+ FW_ERROR_TYPE_ACL = 0x3,
+};
+
+struct fw_error_cmd {
+ __be32 op_to_type;
+ __be32 len16_pkd;
+ union fw_error {
+ struct fw_error_exception {
+ __be32 info[6];
+ } exception;
+ struct fw_error_hwmodule {
+ __be32 regaddr;
+ __be32 regval;
+ } hwmodule;
+ struct fw_error_wr {
+ __be16 cidx;
+ __be16 pfn_vfn;
+ __be32 eqid;
+ u8 wrhdr[16];
+ } wr;
+ struct fw_error_acl {
+ __be16 cidx;
+ __be16 pfn_vfn;
+ __be32 eqid;
+ __be16 mv_pkd;
+ u8 val[6];
+ __be64 r4;
+ } acl;
+ } u;
+};
+
+struct fw_debug_cmd {
+ __be32 op_type;
+#define FW_DEBUG_CMD_TYPE_GET(x) ((x) & 0xff)
+ __be32 len16_pkd;
+ union fw_debug {
+ struct fw_debug_assert {
+ __be32 fcid;
+ __be32 line;
+ __be32 x;
+ __be32 y;
+ u8 filename_0_7[8];
+ u8 filename_8_15[8];
+ __be64 r3;
+ } assert;
+ struct fw_debug_prt {
+ __be16 dprtstridx;
+ __be16 r3[3];
+ __be32 dprtstrparam0;
+ __be32 dprtstrparam1;
+ __be32 dprtstrparam2;
+ __be32 dprtstrparam3;
+ } prt;
+ } u;
+};
+
+struct fw_hdr {
+ u8 ver;
+ u8 reserved1;
+ __be16 len512; /* bin length in units of 512-bytes */
+ __be32 fw_ver; /* firmware version */
+ __be32 tp_microcode_ver;
+ u8 intfver_nic;
+ u8 intfver_vnic;
+ u8 intfver_ofld;
+ u8 intfver_ri;
+ u8 intfver_iscsipdu;
+ u8 intfver_iscsi;
+ u8 intfver_fcoe;
+ u8 reserved2;
+ __be32 reserved3[27];
+};
+
+#define FW_HDR_FW_VER_MAJOR_GET(x) (((x) >> 24) & 0xff)
+#define FW_HDR_FW_VER_MINOR_GET(x) (((x) >> 16) & 0xff)
+#define FW_HDR_FW_VER_MICRO_GET(x) (((x) >> 8) & 0xff)
+#define FW_HDR_FW_VER_BUILD_GET(x) (((x) >> 0) & 0xff)
+#endif /* _T4FW_INTERFACE_H_ */
#include <linux/delay.h>
#include <linux/platform_device.h>
#include <linux/irq.h>
+#include <linux/slab.h>
#include <asm/delay.h>
#include <asm/irq.h>
/* TX */
struct e1000_tx_ring *tx_ring; /* One per active queue */
unsigned int restart_queue;
- unsigned long tx_queue_len;
u32 txd_cmd;
u32 tx_int_delay;
u32 tx_abs_int_delay;
adapter->alloc_rx_buf(adapter, ring,
E1000_DESC_UNUSED(ring));
}
-
- adapter->tx_queue_len = netdev->tx_queue_len;
}
int e1000_up(struct e1000_adapter *adapter)
del_timer_sync(&adapter->watchdog_timer);
del_timer_sync(&adapter->phy_info_timer);
- netdev->tx_queue_len = adapter->tx_queue_len;
adapter->link_speed = 0;
adapter->link_duplex = 0;
netif_carrier_off(netdev);
E1000_CTRL_RFCE) ? "RX" : ((ctrl &
E1000_CTRL_TFCE) ? "TX" : "None" )));
- /* tweak tx_queue_len according to speed/duplex
- * and adjust the timeout factor */
- netdev->tx_queue_len = adapter->tx_queue_len;
+ /* adjust timeout factor according to speed/duplex */
adapter->tx_timeout_factor = 1;
switch (adapter->link_speed) {
case SPEED_10:
txb2b = false;
- netdev->tx_queue_len = 10;
adapter->tx_timeout_factor = 16;
break;
case SPEED_100:
txb2b = false;
- netdev->tx_queue_len = 100;
/* maybe add some timeout factor ? */
break;
}
struct napi_struct napi;
- unsigned long tx_queue_len;
unsigned int restart_queue;
u32 txd_cmd;
#include <linux/netdevice.h>
#include <linux/ethtool.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include "e1000.h"
#include <linux/netdevice.h>
#include <linux/tcp.h>
#include <linux/ipv6.h>
+#include <linux/slab.h>
#include <net/checksum.h>
#include <net/ip6_checksum.h>
#include <linux/mii.h>
ew32(TCTL, tctl);
e1000e_config_collision_dist(hw);
-
- adapter->tx_queue_len = adapter->netdev->tx_queue_len;
}
/**
del_timer_sync(&adapter->watchdog_timer);
del_timer_sync(&adapter->phy_info_timer);
- netdev->tx_queue_len = adapter->tx_queue_len;
netif_carrier_off(netdev);
adapter->link_speed = 0;
adapter->link_duplex = 0;
"link gets many collisions.\n");
}
- /*
- * tweak tx_queue_len according to speed/duplex
- * and adjust the timeout factor
- */
- netdev->tx_queue_len = adapter->tx_queue_len;
+ /* adjust timeout factor according to speed/duplex */
adapter->tx_timeout_factor = 1;
switch (adapter->link_speed) {
case SPEED_10:
txb2b = 0;
- netdev->tx_queue_len = 10;
adapter->tx_timeout_factor = 16;
break;
case SPEED_100:
txb2b = 0;
- netdev->tx_queue_len = 100;
adapter->tx_timeout_factor = 10;
break;
}
#include <linux/interrupt.h>
#include <linux/ioport.h>
#include <linux/in.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/netdevice.h>
#include <linux/netdevice.h>
#include <linux/etherdevice.h>
#include <linux/skbuff.h>
-#include <linux/slab.h>
#include <linux/mca-legacy.h>
#include <linux/spinlock.h>
#include <linux/bitops.h>
#include <linux/udp.h>
#include <linux/if.h>
#include <linux/list.h>
+#include <linux/slab.h>
#include <linux/if_ether.h>
#include <linux/notifier.h>
#include <linux/reboot.h>
*/
#include <linux/mm.h>
+#include <linux/slab.h>
#include "ehea.h"
#include "ehea_phyp.h"
#include "ehea_qmr.h"
#include <linux/types.h>
#include <linux/fcntl.h>
#include <linux/interrupt.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/init.h>
#include <linux/pci.h>
#include <linux/delay.h>
#include <linux/if_ether.h>
+#include <linux/slab.h>
#include "vnic_resource.h"
#include "vnic_devcmd.h"
#include <linux/types.h>
#include <linux/pci.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include "vnic_dev.h"
#include "vnic_rq.h"
#include <linux/types.h>
#include <linux/pci.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include "vnic_dev.h"
#include "vnic_wq.h"
#include <linux/timer.h>
#include <linux/errno.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
#include <linux/delay.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/timer.h>
#include <linux/netdevice.h>
#include <net/net_namespace.h>
#include <linux/interrupt.h>
#include <linux/ioport.h>
#include <linux/in.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/init.h>
#include <linux/phy.h>
#include <linux/platform_device.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <net/ethoc.h>
static int buffer_size = 0x8000; /* 32 KBytes */
#include <linux/timer.h>
#include <linux/errno.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
#include <linux/netdevice.h>
#include <linux/kernel.h>
#include <linux/types.h>
#include <linux/spinlock.h>
+#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/init.h>
#include <linux/crc32.h>
#include <linux/netdevice.h>
#include <linux/phy.h>
#include <linux/of_platform.h>
+#include <linux/slab.h>
#include <linux/of_mdio.h>
#include <asm/io.h>
#include <asm/mpc52xx.h>
#include <linux/init.h>
#include <linux/if_vlan.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <asm/irq.h>
#include <asm/io.h>
#include <linux/ptrace.h>
#include <linux/errno.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/init.h>
#include <linux/delay.h>
#include <linux/platform_device.h>
#include <linux/phy.h>
#include <linux/of_device.h>
+#include <linux/gfp.h>
#include <asm/immap_cpm2.h>
#include <asm/mpc8260.h>
#include <linux/ptrace.h>
#include <linux/errno.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/init.h>
#include <linux/delay.h>
#include <linux/fs.h>
#include <linux/platform_device.h>
#include <linux/of_device.h>
+#include <linux/gfp.h>
#include <asm/irq.h>
#include <asm/uaccess.h>
#include <linux/ptrace.h>
#include <linux/errno.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/init.h>
#include <linux/delay.h>
priv->rx_queue[i] = NULL;
for (i = 0; i < priv->num_tx_queues; i++) {
- priv->tx_queue[i] = (struct gfar_priv_tx_q *)kmalloc(
+ priv->tx_queue[i] = (struct gfar_priv_tx_q *)kzalloc(
sizeof (struct gfar_priv_tx_q), GFP_KERNEL);
if (!priv->tx_queue[i]) {
err = -ENOMEM;
}
for (i = 0; i < priv->num_rx_queues; i++) {
- priv->rx_queue[i] = (struct gfar_priv_rx_q *)kmalloc(
+ priv->rx_queue[i] = (struct gfar_priv_rx_q *)kzalloc(
sizeof (struct gfar_priv_rx_q), GFP_KERNEL);
if (!priv->rx_queue[i]) {
err = -ENOMEM;
/* provided which set of benchmarks. */
printk(KERN_INFO "%s: Running with NAPI enabled\n", dev->name);
for (i = 0; i < priv->num_rx_queues; i++)
- printk(KERN_INFO "%s: :RX BD ring size for Q[%d]: %d\n",
+ printk(KERN_INFO "%s: RX BD ring size for Q[%d]: %d\n",
dev->name, i, priv->rx_queue[i]->rx_ring_size);
for(i = 0; i < priv->num_tx_queues; i++)
- printk(KERN_INFO "%s:TX BD ring size for Q[%d]: %d\n",
+ printk(KERN_INFO "%s: TX BD ring size for Q[%d]: %d\n",
dev->name, i, priv->tx_queue[i]->tx_ring_size);
return 0;
/* Go through all the buffer descriptors and free their data buffers */
for (i = 0; i < priv->num_tx_queues; i++) {
tx_queue = priv->tx_queue[i];
- if(!tx_queue->tx_skbuff)
+ if(tx_queue->tx_skbuff)
free_skb_tx_queue(tx_queue);
}
for (i = 0; i < priv->num_rx_queues; i++) {
rx_queue = priv->rx_queue[i];
- if(!rx_queue->rx_skbuff)
+ if(rx_queue->rx_skbuff)
free_skb_rx_queue(rx_queue);
}
* as many bytes as needed to align the data properly
*/
skb_reserve(skb, alignamount);
+ GFAR_CB(skb)->alignamount = alignamount;
return skb;
}
newskb = skb;
else if (skb) {
/*
- * We need to reset ->data to what it
+ * We need to un-reserve() the skb to what it
* was before gfar_new_skb() re-aligned
* it to an RXBUF_ALIGNMENT boundary
* before we put the skb back on the
* recycle list.
*/
- skb->data = skb->head + NET_SKB_PAD;
+ skb_reserve(skb, -GFAR_CB(skb)->alignamount);
__skb_queue_head(&priv->rx_recycle, skb);
}
} else {
u16 vlctl; /* VLAN control word */
};
+struct gianfar_skb_cb {
+ int alignamount;
+};
+
+#define GFAR_CB(skb) ((struct gianfar_skb_cb *)((skb)->cb))
+
struct rmon_mib
{
u32 tr64; /* 0x.680 - Transmit and Receive 64-byte Frame Counter */
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/errno.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/init.h>
#include <linux/delay.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/unistd.h>
-#include <linux/slab.h>
#include <linux/init.h>
#include <linux/delay.h>
#include <linux/etherdevice.h>
#include <linux/mii.h>
#include <linux/of_device.h>
#include <linux/of_platform.h>
+#include <linux/slab.h>
#include <asm/cacheflush.h>
#include <asm/byteorder.h>
#include <linux/time.h>
#include <linux/errno.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
#include <linux/init.h>
#include <linux/errno.h>
#include <linux/netdevice.h>
#include <linux/timer.h>
+#include <linux/slab.h>
#include <net/ax25.h>
#include <linux/etherdevice.h>
#include <linux/skbuff.h>
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/net.h>
+#include <linux/slab.h>
#include <net/ax25.h>
#include <linux/inet.h>
#include <linux/netdevice.h>
#include <linux/kernel.h>
#include <linux/mm.h>
#include <linux/netdevice.h>
+#include <linux/slab.h>
#include <linux/rtnetlink.h>
#include <linux/sockios.h>
#include <linux/workqueue.h>
#include <linux/net.h>
#include <linux/in.h>
#include <linux/if.h>
-#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/init.h>
#include <linux/bitops.h>
#include <linux/interrupt.h>
#include <linux/in.h>
#include <linux/inet.h>
+#include <linux/slab.h>
#include <linux/tty.h>
#include <linux/errno.h>
#include <linux/netdevice.h>
#include <linux/in.h>
#include <linux/fcntl.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/skbuff.h>
#include <linux/netdevice.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/eisa.h>
#include <linux/pci.h>
#include <linux/types.h>
#include <linux/interrupt.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/netdevice.h>
#include <linux/etherdevice.h>
#include <linux/bitops.h>
#include <linux/workqueue.h>
#include <linux/of.h>
+#include <linux/slab.h>
#include <asm/processor.h>
#include <asm/io.h>
#include <linux/dma-mapping.h>
#include <linux/spinlock.h>
#include <linux/of_platform.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/dcr.h>
*/
#include <linux/delay.h>
+#include <linux/slab.h>
#include "core.h"
#include <asm/dcr-regs.h>
* option) any later version.
*
*/
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/ethtool.h>
#include <asm/io.h>
* option) any later version.
*
*/
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/ethtool.h>
#include <asm/io.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/delay.h>
#include <linux/time.h>
#include <linux/proc_fs.h>
#include <linux/in.h>
#include <linux/ip.h>
+#include <linux/slab.h>
#include <net/net_namespace.h>
#include <asm/hvcall.h>
#include <asm/atomic.h>
*/
#include <linux/types.h>
-#include <linux/slab.h>
#include <linux/if_ether.h>
#include "e1000_mac.h"
case E1000_DEV_ID_82576_FIBER:
case E1000_DEV_ID_82576_SERDES:
case E1000_DEV_ID_82576_QUAD_COPPER:
+ case E1000_DEV_ID_82576_QUAD_COPPER_ET2:
case E1000_DEV_ID_82576_SERDES_QUAD:
mac->type = e1000_82576;
break;
#define E1000_DEV_ID_82576_FIBER 0x10E6
#define E1000_DEV_ID_82576_SERDES 0x10E7
#define E1000_DEV_ID_82576_QUAD_COPPER 0x10E8
+#define E1000_DEV_ID_82576_QUAD_COPPER_ET2 0x1526
#define E1000_DEV_ID_82576_NS 0x150A
#define E1000_DEV_ID_82576_NS_SERDES 0x1518
#define E1000_DEV_ID_82576_SERDES_QUAD 0x150D
* igb_enable_mng_pass_thru - Enable processing of ARP's
* @hw: pointer to the HW structure
*
- * Verifies the hardware needs to allow ARPs to be processed by the host.
+ * Verifies the hardware needs to leave interface enabled so that frames can
+ * be directed to and from the management interface.
**/
bool igb_enable_mng_pass_thru(struct e1000_hw *hw)
{
manc = rd32(E1000_MANC);
- if (!(manc & E1000_MANC_RCV_TCO_EN) ||
- !(manc & E1000_MANC_EN_MAC_ADDR_FILTER))
+ if (!(manc & E1000_MANC_RCV_TCO_EN))
goto out;
if (hw->mac.arc_subsystem_valid) {
/* TX */
struct igb_ring *tx_ring[16];
- unsigned long tx_queue_len;
u32 tx_timeout_count;
/* RX */
#include <linux/if_ether.h>
#include <linux/ethtool.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include "igb.h"
#include <linux/pagemap.h>
#include <linux/netdevice.h>
#include <linux/ipv6.h>
+#include <linux/slab.h>
#include <net/checksum.h>
#include <net/ip6_checksum.h>
#include <linux/net_tstamp.h>
{ PCI_VDEVICE(INTEL, E1000_DEV_ID_82576_FIBER), board_82575 },
{ PCI_VDEVICE(INTEL, E1000_DEV_ID_82576_SERDES), board_82575 },
{ PCI_VDEVICE(INTEL, E1000_DEV_ID_82576_SERDES_QUAD), board_82575 },
+ { PCI_VDEVICE(INTEL, E1000_DEV_ID_82576_QUAD_COPPER_ET2), board_82575 },
{ PCI_VDEVICE(INTEL, E1000_DEV_ID_82576_QUAD_COPPER), board_82575 },
{ PCI_VDEVICE(INTEL, E1000_DEV_ID_82575EB_COPPER), board_82575 },
{ PCI_VDEVICE(INTEL, E1000_DEV_ID_82575EB_FIBER_SERDES), board_82575 },
struct igb_ring *ring = adapter->rx_ring[i];
igb_alloc_rx_buffers_adv(ring, igb_desc_unused(ring));
}
-
-
- adapter->tx_queue_len = netdev->tx_queue_len;
}
/**
del_timer_sync(&adapter->watchdog_timer);
del_timer_sync(&adapter->phy_info_timer);
- netdev->tx_queue_len = adapter->tx_queue_len;
netif_carrier_off(netdev);
/* record the stats before reset*/
((ctrl & E1000_CTRL_RFCE) ? "RX" :
((ctrl & E1000_CTRL_TFCE) ? "TX" : "None")));
- /* tweak tx_queue_len according to speed/duplex and
- * adjust the timeout factor */
- netdev->tx_queue_len = adapter->tx_queue_len;
+ /* adjust timeout factor according to speed/duplex */
adapter->tx_timeout_factor = 1;
switch (adapter->link_speed) {
case SPEED_10:
- netdev->tx_queue_len = 10;
adapter->tx_timeout_factor = 14;
break;
case SPEED_100:
- netdev->tx_queue_len = 100;
/* maybe add some timeout factor ? */
break;
}
struct net_device_stats *net_stats = igb_get_stats(adapter->netdev);
struct e1000_hw *hw = &adapter->hw;
struct pci_dev *pdev = adapter->pdev;
- u32 rnbc, reg;
+ u32 reg, mpc;
u16 phy_tmp;
int i;
u64 bytes, packets;
adapter->stats.symerrs += rd32(E1000_SYMERRS);
adapter->stats.sec += rd32(E1000_SEC);
- adapter->stats.mpc += rd32(E1000_MPC);
+ mpc = rd32(E1000_MPC);
+ adapter->stats.mpc += mpc;
+ net_stats->rx_fifo_errors += mpc;
adapter->stats.scc += rd32(E1000_SCC);
adapter->stats.ecol += rd32(E1000_ECOL);
adapter->stats.mcc += rd32(E1000_MCC);
adapter->stats.gptc += rd32(E1000_GPTC);
adapter->stats.gotc += rd32(E1000_GOTCL);
rd32(E1000_GOTCH); /* clear GOTCL */
- rnbc = rd32(E1000_RNBC);
- adapter->stats.rnbc += rnbc;
- net_stats->rx_fifo_errors += rnbc;
+ adapter->stats.rnbc += rd32(E1000_RNBC);
adapter->stats.ruc += rd32(E1000_RUC);
adapter->stats.rfc += rd32(E1000_RFC);
adapter->stats.rjc += rd32(E1000_RJC);
{
struct igb_adapter *adapter = q_vector->adapter;
- if (vlan_tag)
+ if (vlan_tag && adapter->vlgrp)
vlan_gro_receive(&q_vector->napi, adapter->vlgrp,
vlan_tag, skb);
else
struct igbvf_ring *tx_ring /* One per active queue */
____cacheline_aligned_in_smp;
- unsigned long tx_queue_len;
unsigned int restart_queue;
u32 txd_cmd;
#include <linux/netdevice.h>
#include <linux/tcp.h>
#include <linux/ipv6.h>
+#include <linux/slab.h>
#include <net/checksum.h>
#include <net/ip6_checksum.h>
#include <linux/mii.h>
/* enable Report Status bit */
adapter->txd_cmd |= E1000_ADVTXD_DCMD_RS;
-
- adapter->tx_queue_len = adapter->netdev->tx_queue_len;
}
/**
del_timer_sync(&adapter->watchdog_timer);
- netdev->tx_queue_len = adapter->tx_queue_len;
netif_carrier_off(netdev);
/* record the stats before reset*/
&adapter->link_duplex);
igbvf_print_link_info(adapter);
- /*
- * tweak tx_queue_len according to speed/duplex
- * and adjust the timeout factor
- */
- netdev->tx_queue_len = adapter->tx_queue_len;
+ /* adjust timeout factor according to speed/duplex */
adapter->tx_timeout_factor = 1;
switch (adapter->link_speed) {
case SPEED_10:
txb2b = 0;
- netdev->tx_queue_len = 10;
adapter->tx_timeout_factor = 16;
break;
case SPEED_100:
txb2b = 0;
- netdev->tx_queue_len = 100;
/* maybe add some timeout factor ? */
break;
}
#include <linux/tcp.h>
#include <linux/udp.h>
#include <linux/dma-mapping.h>
+#include <linux/gfp.h>
#ifdef CONFIG_SERIAL_8250
#include <linux/serial_core.h>
*/
#include <linux/crc32.h>
#include <linux/ethtool.h>
+#include <linux/gfp.h>
#include <linux/mii.h>
#include <linux/mutex.h>
********************************************************************/
#include <linux/module.h>
+#include <linux/gfp.h>
#include <linux/kernel.h>
#include <linux/types.h>
#include <linux/netdevice.h>
#include <linux/ioport.h>
#include <linux/delay.h>
-#include <linux/slab.h>
#include <linux/init.h>
#include <linux/rtnetlink.h>
#include <linux/serial_reg.h>
#include <linux/delay.h>
#include <linux/platform_device.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <net/irda/irda.h>
#include <net/irda/wrapper.h>
#include <linux/module.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/tty.h>
#include <linux/init.h>
#include <asm/uaccess.h>
********************************************************************/
#include <linux/module.h>
+#include <linux/gfp.h>
#include <linux/kernel.h>
#include <linux/types.h>
#include <linux/netdevice.h>
#include <linux/ioport.h>
#include <linux/delay.h>
-#include <linux/slab.h>
#include <linux/init.h>
#include <linux/rtnetlink.h>
#include <linux/dma-mapping.h>
#include <linux/platform_device.h>
#include <linux/clk.h>
#include <linux/gpio.h>
+#include <linux/slab.h>
#include <net/irda/irda.h>
#include <net/irda/irmod.h>
#include <linux/module.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <net/irda/wrapper.h>
#include <net/irda/irda_device.h>
#include <asm/clock.h>
#include <linux/module.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/delay.h>
#include <linux/netdevice.h>
#include <linux/ioport.h>
#include <linux/delay.h>
-#include <linux/slab.h>
#include <linux/init.h>
#include <linux/rtnetlink.h>
#include <linux/serial_reg.h>
#include <linux/dma-mapping.h>
#include <linux/pnp.h>
#include <linux/platform_device.h>
+#include <linux/gfp.h>
#include <asm/io.h>
#include <asm/dma.h>
#include <linux/netdevice.h>
#include <linux/ioport.h>
#include <linux/delay.h>
-#include <linux/slab.h>
#include <linux/init.h>
#include <linux/rtnetlink.h>
#include <linux/pci.h>
#include <linux/dma-mapping.h>
+#include <linux/gfp.h>
#include <asm/io.h>
#include <asm/dma.h>
#include <linux/netdevice.h>
#include <linux/ioport.h>
#include <linux/delay.h>
-#include <linux/slab.h>
#include <linux/init.h>
#include <linux/rtnetlink.h>
#include <linux/dma-mapping.h>
+#include <linux/gfp.h>
#include <asm/io.h>
#include <asm/dma.h>
#include <linux/mm.h>
#include <linux/ethtool.h>
#include <linux/if_ether.h>
+#include <linux/slab.h>
#include <asm/abs_addr.h>
#include <asm/iseries/mf.h>
#define IXGBE_MAX_FDIR_INDICES 64
#ifdef IXGBE_FCOE
#define IXGBE_MAX_FCOE_INDICES 8
+#define MAX_RX_QUEUES (IXGBE_MAX_FDIR_INDICES + IXGBE_MAX_FCOE_INDICES)
+#define MAX_TX_QUEUES (IXGBE_MAX_FDIR_INDICES + IXGBE_MAX_FCOE_INDICES)
+#else
+#define MAX_RX_QUEUES IXGBE_MAX_FDIR_INDICES
+#define MAX_TX_QUEUES IXGBE_MAX_FDIR_INDICES
#endif /* IXGBE_FCOE */
struct ixgbe_ring_feature {
int indices;
int mask;
} ____cacheline_internodealigned_in_smp;
-#define MAX_RX_QUEUES 128
-#define MAX_TX_QUEUES 128
#define MAX_RX_PACKET_BUFFERS ((adapter->flags & IXGBE_FLAG_DCB_ENABLED) \
? 8 : 1)
#define IXGBE_82599_MC_TBL_SIZE 128
#define IXGBE_82599_VFT_TBL_SIZE 128
+void ixgbe_flap_tx_laser_multispeed_fiber(struct ixgbe_hw *hw);
s32 ixgbe_setup_mac_link_multispeed_fiber(struct ixgbe_hw *hw,
ixgbe_link_speed speed,
bool autoneg,
if (hw->phy.multispeed_fiber) {
/* Set up dual speed SFP+ support */
mac->ops.setup_link = &ixgbe_setup_mac_link_multispeed_fiber;
+ mac->ops.flap_tx_laser = &ixgbe_flap_tx_laser_multispeed_fiber;
} else {
+ mac->ops.flap_tx_laser = NULL;
if ((mac->ops.get_media_type(hw) ==
ixgbe_media_type_backplane) &&
(hw->phy.smart_speed == ixgbe_smart_speed_auto ||
return status;
}
+/**
+ * ixgbe_flap_tx_laser_multispeed_fiber - Flap Tx laser
+ * @hw: pointer to hardware structure
+ *
+ * When the driver changes the link speeds that it can support,
+ * it sets autotry_restart to true to indicate that we need to
+ * initiate a new autotry session with the link partner. To do
+ * so, we set the speed then disable and re-enable the tx laser, to
+ * alert the link partner that it also needs to restart autotry on its
+ * end. This is consistent with true clause 37 autoneg, which also
+ * involves a loss of signal.
+ **/
+void ixgbe_flap_tx_laser_multispeed_fiber(struct ixgbe_hw *hw)
+{
+ u32 esdp_reg = IXGBE_READ_REG(hw, IXGBE_ESDP);
+
+ hw_dbg(hw, "ixgbe_flap_tx_laser_multispeed_fiber\n");
+
+ if (hw->mac.autotry_restart) {
+ /* Disable tx laser; allow 100us to go dark per spec */
+ esdp_reg |= IXGBE_ESDP_SDP3;
+ IXGBE_WRITE_REG(hw, IXGBE_ESDP, esdp_reg);
+ IXGBE_WRITE_FLUSH(hw);
+ udelay(100);
+
+ /* Enable tx laser; allow 100ms to light up */
+ esdp_reg &= ~IXGBE_ESDP_SDP3;
+ IXGBE_WRITE_REG(hw, IXGBE_ESDP, esdp_reg);
+ IXGBE_WRITE_FLUSH(hw);
+ msleep(100);
+
+ hw->mac.autotry_restart = false;
+ }
+}
+
/**
* ixgbe_setup_mac_link_multispeed_fiber - Set MAC link speed
* @hw: pointer to hardware structure
hw->mac.ops.get_link_capabilities(hw, &phy_link_speed, &negotiation);
speed &= phy_link_speed;
- /*
- * When the driver changes the link speeds that it can support,
- * it sets autotry_restart to true to indicate that we need to
- * initiate a new autotry session with the link partner. To do
- * so, we set the speed then disable and re-enable the tx laser, to
- * alert the link partner that it also needs to restart autotry on its
- * end. This is consistent with true clause 37 autoneg, which also
- * involves a loss of signal.
- */
-
/*
* Try each speed one by one, highest priority first. We do this in
* software because 10gb fiber doesn't support speed autonegotiation.
/* Set the module link speed */
esdp_reg |= (IXGBE_ESDP_SDP5_DIR | IXGBE_ESDP_SDP5);
IXGBE_WRITE_REG(hw, IXGBE_ESDP, esdp_reg);
+ IXGBE_WRITE_FLUSH(hw);
/* Allow module to change analog characteristics (1G->10G) */
msleep(40);
return status;
/* Flap the tx laser if it has not already been done */
- if (hw->mac.autotry_restart) {
- /* Disable tx laser; allow 100us to go dark per spec */
- esdp_reg |= IXGBE_ESDP_SDP3;
- IXGBE_WRITE_REG(hw, IXGBE_ESDP, esdp_reg);
- udelay(100);
-
- /* Enable tx laser; allow 2ms to light up per spec */
- esdp_reg &= ~IXGBE_ESDP_SDP3;
- IXGBE_WRITE_REG(hw, IXGBE_ESDP, esdp_reg);
- msleep(2);
-
- hw->mac.autotry_restart = false;
- }
+ hw->mac.ops.flap_tx_laser(hw);
/*
* Wait for the controller to acquire link. Per IEEE 802.3ap,
esdp_reg &= ~IXGBE_ESDP_SDP5;
esdp_reg |= IXGBE_ESDP_SDP5_DIR;
IXGBE_WRITE_REG(hw, IXGBE_ESDP, esdp_reg);
+ IXGBE_WRITE_FLUSH(hw);
/* Allow module to change analog characteristics (10G->1G) */
msleep(40);
return status;
/* Flap the tx laser if it has not already been done */
- if (hw->mac.autotry_restart) {
- /* Disable tx laser; allow 100us to go dark per spec */
- esdp_reg |= IXGBE_ESDP_SDP3;
- IXGBE_WRITE_REG(hw, IXGBE_ESDP, esdp_reg);
- udelay(100);
-
- /* Enable tx laser; allow 2ms to light up per spec */
- esdp_reg &= ~IXGBE_ESDP_SDP3;
- IXGBE_WRITE_REG(hw, IXGBE_ESDP, esdp_reg);
- msleep(2);
-
- hw->mac.autotry_restart = false;
- }
+ hw->mac.ops.flap_tx_laser(hw);
/* Wait for the link partner to also set speed */
msleep(100);
#include <linux/types.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/netdevice.h>
#include <linux/ethtool.h>
if (ixgbe_link_test(adapter, &data[4]))
eth_test->flags |= ETH_TEST_FL_FAILED;
+ if (adapter->flags & IXGBE_FLAG_SRIOV_ENABLED) {
+ int i;
+ for (i = 0; i < adapter->num_vfs; i++) {
+ if (adapter->vfinfo[i].clear_to_send) {
+ netdev_warn(netdev, "%s",
+ "offline diagnostic is not "
+ "supported when VFs are "
+ "present\n");
+ data[0] = 1;
+ data[1] = 1;
+ data[2] = 1;
+ data[3] = 1;
+ eth_test->flags |= ETH_TEST_FL_FAILED;
+ clear_bit(__IXGBE_TESTING,
+ &adapter->state);
+ goto skip_ol_tests;
+ }
+ }
+ }
+
if (if_running)
/* indicate we're in test mode */
dev_close(netdev);
clear_bit(__IXGBE_TESTING, &adapter->state);
}
+skip_ol_tests:
msleep_interruptible(4 * 1000);
}
#include "ixgbe_dcb_82599.h"
#endif /* CONFIG_IXGBE_DCB */
#include <linux/if_ether.h>
+#include <linux/gfp.h>
#include <scsi/scsi_cmnd.h>
#include <scsi/scsi_device.h>
#include <scsi/fc/fc_fs.h>
addr = sg_dma_address(sg);
len = sg_dma_len(sg);
while (len) {
+ /* max number of buffers allowed in one DDP context */
+ if (j >= IXGBE_BUFFCNT_MAX) {
+ netif_err(adapter, drv, adapter->netdev,
+ "xid=%x:%d,%d,%d:addr=%llx "
+ "not enough descriptors\n",
+ xid, i, j, dmacount, (u64)addr);
+ goto out_noddp_free;
+ }
+
/* get the offset of length of current buffer */
thisoff = addr & ((dma_addr_t)bufflen - 1);
thislen = min((bufflen - thisoff), len);
len -= thislen;
addr += thislen;
j++;
- /* max number of buffers allowed in one DDP context */
- if (j > IXGBE_BUFFCNT_MAX) {
- DPRINTK(DRV, ERR, "xid=%x:%d,%d,%d:addr=%llx "
- "not enough descriptors\n",
- xid, i, j, dmacount, (u64)addr);
- goto out_noddp_free;
- }
}
}
/* only the last buffer may have non-full bufflen */
lastsize = thisoff + thislen;
fcbuff = (IXGBE_FCBUFF_4KB << IXGBE_FCBUFF_BUFFSIZE_SHIFT);
- fcbuff |= (j << IXGBE_FCBUFF_BUFFCNT_SHIFT);
+ fcbuff |= ((j & 0xff) << IXGBE_FCBUFF_BUFFCNT_SHIFT);
fcbuff |= (firstoff << IXGBE_FCBUFF_OFFSET_SHIFT);
fcbuff |= (IXGBE_FCBUFF_VALID);
/* Enable L2 eth type filter for FCoE */
IXGBE_WRITE_REG(hw, IXGBE_ETQF(IXGBE_ETQF_FILTER_FCOE),
(ETH_P_FCOE | IXGBE_ETQF_FCOE | IXGBE_ETQF_FILTER_EN));
+ /* Enable L2 eth type filter for FIP */
+ IXGBE_WRITE_REG(hw, IXGBE_ETQF(IXGBE_ETQF_FILTER_FIP),
+ (ETH_P_FIP | IXGBE_ETQF_FILTER_EN));
if (adapter->ring_feature[RING_F_FCOE].indices) {
/* Use multiple rx queues for FCoE by redirection table */
for (i = 0; i < IXGBE_FCRETA_SIZE; i++) {
}
IXGBE_WRITE_REG(hw, IXGBE_FCRECTL, IXGBE_FCRECTL_ENA);
IXGBE_WRITE_REG(hw, IXGBE_ETQS(IXGBE_ETQF_FILTER_FCOE), 0);
+ fcoe_i = f->mask;
+ fcoe_i &= IXGBE_FCRETA_ENTRY_MASK;
+ fcoe_q = adapter->rx_ring[fcoe_i]->reg_idx;
+ IXGBE_WRITE_REG(hw, IXGBE_ETQS(IXGBE_ETQF_FILTER_FIP),
+ IXGBE_ETQS_QUEUE_EN |
+ (fcoe_q << IXGBE_ETQS_RX_QUEUE_SHIFT));
} else {
/* Use single rx queue for FCoE */
fcoe_i = f->mask;
IXGBE_ETQS_QUEUE_EN |
(fcoe_q << IXGBE_ETQS_RX_QUEUE_SHIFT));
}
+ /* send FIP frames to the first FCoE queue */
+ fcoe_i = f->mask;
+ fcoe_q = adapter->rx_ring[fcoe_i]->reg_idx;
+ IXGBE_WRITE_REG(hw, IXGBE_ETQS(IXGBE_ETQF_FILTER_FIP),
+ IXGBE_ETQS_QUEUE_EN |
+ (fcoe_q << IXGBE_ETQS_RX_QUEUE_SHIFT));
IXGBE_WRITE_REG(hw, IXGBE_FCRXCTRL,
IXGBE_FCRXCTRL_FCOELLI |
netdev->vlan_features |= NETIF_F_FSO;
netdev->vlan_features |= NETIF_F_FCOE_MTU;
netdev->fcoe_ddp_xid = IXGBE_FCOE_DDP_MAX - 1;
- netdev_features_change(netdev);
ixgbe_init_interrupt_scheme(adapter);
+ netdev_features_change(netdev);
if (netif_running(netdev))
netdev->netdev_ops->ndo_open(netdev);
netdev->vlan_features &= ~NETIF_F_FSO;
netdev->vlan_features &= ~NETIF_F_FCOE_MTU;
netdev->fcoe_ddp_xid = 0;
- netdev_features_change(netdev);
ixgbe_cleanup_fcoe(adapter);
-
ixgbe_init_interrupt_scheme(adapter);
+ netdev_features_change(netdev);
+
if (netif_running(netdev))
netdev->netdev_ops->ndo_open(netdev);
rc = 0;
#include <linux/tcp.h>
#include <linux/pkt_sched.h>
#include <linux/ipv6.h>
+#include <linux/slab.h>
#include <net/checksum.h>
#include <net/ip6_checksum.h>
#include <linux/ethtool.h>
if (skb->prev)
skb = ixgbe_transform_rsc_queue(skb, &(rx_ring->rsc_count));
if (adapter->flags2 & IXGBE_FLAG2_RSC_ENABLED) {
- if (IXGBE_RSC_CB(skb)->dma)
+ if (IXGBE_RSC_CB(skb)->dma) {
pci_unmap_single(pdev, IXGBE_RSC_CB(skb)->dma,
rx_ring->rx_buf_len,
PCI_DMA_FROMDEVICE);
+ IXGBE_RSC_CB(skb)->dma = 0;
+ }
if (rx_ring->flags & IXGBE_RING_RX_PS_ENABLED)
rx_ring->rsc_count += skb_shinfo(skb)->nr_frags;
else
while (test_and_set_bit(__IXGBE_RESETTING, &adapter->state))
msleep(1);
ixgbe_down(adapter);
+ /*
+ * If SR-IOV enabled then wait a bit before bringing the adapter
+ * back up to give the VFs time to respond to the reset. The
+ * two second wait is based upon the watchdog timer cycle in
+ * the VF driver.
+ */
+ if (adapter->flags & IXGBE_FLAG_SRIOV_ENABLED)
+ msleep(2000);
ixgbe_up(adapter);
clear_bit(__IXGBE_RESETTING, &adapter->state);
}
rx_buffer_info->skb = NULL;
do {
struct sk_buff *this = skb;
- if (IXGBE_RSC_CB(this)->dma)
+ if (IXGBE_RSC_CB(this)->dma) {
pci_unmap_single(pdev, IXGBE_RSC_CB(this)->dma,
rx_ring->rx_buf_len,
PCI_DMA_FROMDEVICE);
+ IXGBE_RSC_CB(this)->dma = 0;
+ }
skb = skb->prev;
dev_kfree_skb(this);
} while (skb);
/* disable receive for all VFs and wait one second */
if (adapter->num_vfs) {
- for (i = 0 ; i < adapter->num_vfs; i++)
- adapter->vfinfo[i].clear_to_send = 0;
-
/* ping all the active vfs to let them know we are going down */
ixgbe_ping_all_vfs(adapter);
+
/* Disable all VFTE/VFRE TX/RX */
ixgbe_disable_tx_rx(adapter);
+
+ /* Mark all the VFs as inactive */
+ for (i = 0 ; i < adapter->num_vfs; i++)
+ adapter->vfinfo[i].clear_to_send = 0;
}
/* disable receives */
autoneg = hw->phy.autoneg_advertised;
if ((!autoneg) && (hw->mac.ops.get_link_capabilities))
hw->mac.ops.get_link_capabilities(hw, &autoneg, &negotiation);
+ hw->mac.autotry_restart = false;
if (hw->mac.ops.setup_link)
hw->mac.ops.setup_link(hw, autoneg, negotiation, true);
adapter->flags |= IXGBE_FLAG_NEED_LINK_UPDATE;
#ifdef IXGBE_FCOE
if ((adapter->flags & IXGBE_FLAG_FCOE_ENABLED) &&
- (skb->protocol == htons(ETH_P_FCOE))) {
+ ((skb->protocol == htons(ETH_P_FCOE)) ||
+ (skb->protocol == htons(ETH_P_FIP)))) {
txq &= (adapter->ring_feature[RING_F_FCOE].indices - 1);
txq += adapter->ring_feature[RING_F_FCOE].mask;
return txq;
tx_ring = adapter->tx_ring[skb->queue_mapping];
- if ((adapter->flags & IXGBE_FLAG_FCOE_ENABLED) &&
- (skb->protocol == htons(ETH_P_FCOE))) {
- tx_flags |= IXGBE_TX_FLAGS_FCOE;
#ifdef IXGBE_FCOE
+ if (adapter->flags & IXGBE_FLAG_FCOE_ENABLED) {
#ifdef CONFIG_IXGBE_DCB
- tx_flags &= ~(IXGBE_TX_FLAGS_VLAN_PRIO_MASK
- << IXGBE_TX_FLAGS_VLAN_SHIFT);
- tx_flags |= ((adapter->fcoe.up << 13)
- << IXGBE_TX_FLAGS_VLAN_SHIFT);
-#endif
+ /* for FCoE with DCB, we force the priority to what
+ * was specified by the switch */
+ if ((skb->protocol == htons(ETH_P_FCOE)) ||
+ (skb->protocol == htons(ETH_P_FIP))) {
+ tx_flags &= ~(IXGBE_TX_FLAGS_VLAN_PRIO_MASK
+ << IXGBE_TX_FLAGS_VLAN_SHIFT);
+ tx_flags |= ((adapter->fcoe.up << 13)
+ << IXGBE_TX_FLAGS_VLAN_SHIFT);
+ }
#endif
+ /* flag for FCoE offloads */
+ if (skb->protocol == htons(ETH_P_FCOE))
+ tx_flags |= IXGBE_TX_FLAGS_FCOE;
}
+#endif
+
/* four things can cause us to need a context descriptor */
if (skb_is_gso(skb) ||
(skb->ip_summed == CHECKSUM_PARTIAL) ||
indices += min_t(unsigned int, num_possible_cpus(),
IXGBE_MAX_FCOE_INDICES);
#endif
- indices = min_t(unsigned int, indices, MAX_TX_QUEUES);
netdev = alloc_etherdev_mq(sizeof(struct ixgbe_adapter), indices);
if (!netdev) {
err = -ENOMEM;
case IXGBE_DEV_ID_82599_KX4:
adapter->wol = (IXGBE_WUFC_MAG | IXGBE_WUFC_EX |
IXGBE_WUFC_MC | IXGBE_WUFC_BC);
- /* Enable ACPI wakeup in GRC */
- IXGBE_WRITE_REG(hw, IXGBE_GRC,
- (IXGBE_READ_REG(hw, IXGBE_GRC) & ~IXGBE_GRC_APME));
break;
default:
adapter->wol = 0;
del_timer_sync(&adapter->sfp_timer);
cancel_work_sync(&adapter->watchdog_task);
cancel_work_sync(&adapter->sfp_task);
+ if (adapter->hw.phy.multispeed_fiber) {
+ struct ixgbe_hw *hw = &adapter->hw;
+ /*
+ * Restart clause 37 autoneg, disable and re-enable
+ * the tx laser, to clear & alert the link partner
+ * that it needs to restart autotry
+ */
+ hw->mac.autotry_restart = true;
+ hw->mac.ops.flap_tx_laser(hw);
+ }
cancel_work_sync(&adapter->multispeed_fiber_task);
cancel_work_sync(&adapter->sfp_config_module_task);
if (adapter->flags & IXGBE_FLAG_FDIR_HASH_CAPABLE ||
#define IXGBE_ETQF_FILTER_BCN 1
#define IXGBE_ETQF_FILTER_FCOE 2
#define IXGBE_ETQF_FILTER_1588 3
+#define IXGBE_ETQF_FILTER_FIP 4
/* VLAN Control Bit Masks */
#define IXGBE_VLNCTRL_VET 0x0000FFFF /* bits 0-15 */
#define IXGBE_VLNCTRL_CFI 0x10000000 /* bit 28 */
s32 (*enable_rx_dma)(struct ixgbe_hw *, u32);
/* Link */
+ void (*flap_tx_laser)(struct ixgbe_hw *);
s32 (*setup_link)(struct ixgbe_hw *, ixgbe_link_speed, bool, bool);
s32 (*check_link)(struct ixgbe_hw *, ixgbe_link_speed *, bool *, bool);
s32 (*get_link_capabilities)(struct ixgbe_hw *, ixgbe_link_speed *,
#include <linux/types.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/netdevice.h>
#include <linux/ethtool.h>
int sizeof_stat;
int stat_offset;
int base_stat_offset;
+ int saved_reset_offset;
};
-#define IXGBEVF_STAT(m, b) sizeof(((struct ixgbevf_adapter *)0)->m), \
- offsetof(struct ixgbevf_adapter, m), \
- offsetof(struct ixgbevf_adapter, b)
+#define IXGBEVF_STAT(m, b, r) sizeof(((struct ixgbevf_adapter *)0)->m), \
+ offsetof(struct ixgbevf_adapter, m), \
+ offsetof(struct ixgbevf_adapter, b), \
+ offsetof(struct ixgbevf_adapter, r)
static struct ixgbe_stats ixgbe_gstrings_stats[] = {
- {"rx_packets", IXGBEVF_STAT(stats.vfgprc, stats.base_vfgprc)},
- {"tx_packets", IXGBEVF_STAT(stats.vfgptc, stats.base_vfgptc)},
- {"rx_bytes", IXGBEVF_STAT(stats.vfgorc, stats.base_vfgorc)},
- {"tx_bytes", IXGBEVF_STAT(stats.vfgotc, stats.base_vfgotc)},
- {"tx_busy", IXGBEVF_STAT(tx_busy, zero_base)},
- {"multicast", IXGBEVF_STAT(stats.vfmprc, stats.base_vfmprc)},
- {"rx_csum_offload_good", IXGBEVF_STAT(hw_csum_rx_good, zero_base)},
- {"rx_csum_offload_errors", IXGBEVF_STAT(hw_csum_rx_error, zero_base)},
- {"tx_csum_offload_ctxt", IXGBEVF_STAT(hw_csum_tx_good, zero_base)},
- {"rx_header_split", IXGBEVF_STAT(rx_hdr_split, zero_base)},
+ {"rx_packets", IXGBEVF_STAT(stats.vfgprc, stats.base_vfgprc,
+ stats.saved_reset_vfgprc)},
+ {"tx_packets", IXGBEVF_STAT(stats.vfgptc, stats.base_vfgptc,
+ stats.saved_reset_vfgptc)},
+ {"rx_bytes", IXGBEVF_STAT(stats.vfgorc, stats.base_vfgorc,
+ stats.saved_reset_vfgorc)},
+ {"tx_bytes", IXGBEVF_STAT(stats.vfgotc, stats.base_vfgotc,
+ stats.saved_reset_vfgotc)},
+ {"tx_busy", IXGBEVF_STAT(tx_busy, zero_base, zero_base)},
+ {"multicast", IXGBEVF_STAT(stats.vfmprc, stats.base_vfmprc,
+ stats.saved_reset_vfmprc)},
+ {"rx_csum_offload_good", IXGBEVF_STAT(hw_csum_rx_good, zero_base,
+ zero_base)},
+ {"rx_csum_offload_errors", IXGBEVF_STAT(hw_csum_rx_error, zero_base,
+ zero_base)},
+ {"tx_csum_offload_ctxt", IXGBEVF_STAT(hw_csum_tx_good, zero_base,
+ zero_base)},
+ {"rx_header_split", IXGBEVF_STAT(rx_hdr_split, zero_base, zero_base)},
};
#define IXGBE_QUEUE_STATS_LEN 0
ixgbe_gstrings_stats[i].stat_offset;
char *b = (char *)adapter +
ixgbe_gstrings_stats[i].base_stat_offset;
+ char *r = (char *)adapter +
+ ixgbe_gstrings_stats[i].saved_reset_offset;
data[i] = ((ixgbe_gstrings_stats[i].sizeof_stat ==
sizeof(u64)) ? *(u64 *)p : *(u32 *)p) -
((ixgbe_gstrings_stats[i].sizeof_stat ==
- sizeof(u64)) ? *(u64 *)b : *(u32 *)b);
+ sizeof(u64)) ? *(u64 *)b : *(u32 *)b) +
+ ((ixgbe_gstrings_stats[i].sizeof_stat ==
+ sizeof(u64)) ? *(u64 *)r : *(u32 *)r);
}
}
#include <linux/ip.h>
#include <linux/tcp.h>
#include <linux/ipv6.h>
+#include <linux/slab.h>
#include <net/checksum.h>
#include <net/ip6_checksum.h>
#include <linux/ethtool.h>
if ((msg & IXGBE_MBVFICR_VFREQ_MASK) == IXGBE_PF_CONTROL_MSG)
mod_timer(&adapter->watchdog_timer,
- round_jiffies(jiffies + 10));
+ round_jiffies(jiffies + 1));
return IRQ_HANDLED;
}
(adapter->rx_ring[rxr].count - 1));
}
+static void ixgbevf_save_reset_stats(struct ixgbevf_adapter *adapter)
+{
+ /* Only save pre-reset stats if there are some */
+ if (adapter->stats.vfgprc || adapter->stats.vfgptc) {
+ adapter->stats.saved_reset_vfgprc += adapter->stats.vfgprc -
+ adapter->stats.base_vfgprc;
+ adapter->stats.saved_reset_vfgptc += adapter->stats.vfgptc -
+ adapter->stats.base_vfgptc;
+ adapter->stats.saved_reset_vfgorc += adapter->stats.vfgorc -
+ adapter->stats.base_vfgorc;
+ adapter->stats.saved_reset_vfgotc += adapter->stats.vfgotc -
+ adapter->stats.base_vfgotc;
+ adapter->stats.saved_reset_vfmprc += adapter->stats.vfmprc -
+ adapter->stats.base_vfmprc;
+ }
+}
+
+static void ixgbevf_init_last_counter_stats(struct ixgbevf_adapter *adapter)
+{
+ struct ixgbe_hw *hw = &adapter->hw;
+
+ adapter->stats.last_vfgprc = IXGBE_READ_REG(hw, IXGBE_VFGPRC);
+ adapter->stats.last_vfgorc = IXGBE_READ_REG(hw, IXGBE_VFGORC_LSB);
+ adapter->stats.last_vfgorc |=
+ (((u64)(IXGBE_READ_REG(hw, IXGBE_VFGORC_MSB))) << 32);
+ adapter->stats.last_vfgptc = IXGBE_READ_REG(hw, IXGBE_VFGPTC);
+ adapter->stats.last_vfgotc = IXGBE_READ_REG(hw, IXGBE_VFGOTC_LSB);
+ adapter->stats.last_vfgotc |=
+ (((u64)(IXGBE_READ_REG(hw, IXGBE_VFGOTC_MSB))) << 32);
+ adapter->stats.last_vfmprc = IXGBE_READ_REG(hw, IXGBE_VFMPRC);
+
+ adapter->stats.base_vfgprc = adapter->stats.last_vfgprc;
+ adapter->stats.base_vfgorc = adapter->stats.last_vfgorc;
+ adapter->stats.base_vfgptc = adapter->stats.last_vfgptc;
+ adapter->stats.base_vfgotc = adapter->stats.last_vfgotc;
+ adapter->stats.base_vfmprc = adapter->stats.last_vfmprc;
+}
+
static int ixgbevf_up_complete(struct ixgbevf_adapter *adapter)
{
struct net_device *netdev = adapter->netdev;
/* enable transmits */
netif_tx_start_all_queues(netdev);
+ ixgbevf_save_reset_stats(adapter);
+ ixgbevf_init_last_counter_stats(adapter);
+
/* bring the link up in the watchdog, this could race with our first
* link up interrupt but shouldn't be a problem */
adapter->flags |= IXGBE_FLAG_NEED_LINK_UPDATE;
return err;
}
-static void ixgbevf_init_last_counter_stats(struct ixgbevf_adapter *adapter)
-{
- struct ixgbe_hw *hw = &adapter->hw;
-
- adapter->stats.last_vfgprc = IXGBE_READ_REG(hw, IXGBE_VFGPRC);
- adapter->stats.last_vfgorc = IXGBE_READ_REG(hw, IXGBE_VFGORC_LSB);
- adapter->stats.last_vfgorc |=
- (((u64)(IXGBE_READ_REG(hw, IXGBE_VFGORC_MSB))) << 32);
- adapter->stats.last_vfgptc = IXGBE_READ_REG(hw, IXGBE_VFGPTC);
- adapter->stats.last_vfgotc = IXGBE_READ_REG(hw, IXGBE_VFGOTC_LSB);
- adapter->stats.last_vfgotc |=
- (((u64)(IXGBE_READ_REG(hw, IXGBE_VFGOTC_MSB))) << 32);
- adapter->stats.last_vfmprc = IXGBE_READ_REG(hw, IXGBE_VFMPRC);
-
- adapter->stats.base_vfgprc = adapter->stats.last_vfgprc;
- adapter->stats.base_vfgorc = adapter->stats.last_vfgorc;
- adapter->stats.base_vfgptc = adapter->stats.last_vfgptc;
- adapter->stats.base_vfgotc = adapter->stats.last_vfgotc;
- adapter->stats.base_vfmprc = adapter->stats.last_vfmprc;
-}
-
#define UPDATE_VF_COUNTER_32bit(reg, last_counter, counter) \
{ \
u32 current_counter = IXGBE_READ_REG(hw, reg); \
if (!netif_carrier_ok(netdev)) {
hw_dbg(&adapter->hw, "NIC Link is Up %s, ",
((link_speed == IXGBE_LINK_SPEED_10GB_FULL) ?
- "10 Gbps" : "1 Gbps"));
+ "10 Gbps\n" : "1 Gbps\n"));
netif_carrier_on(netdev);
netif_tx_wake_all_queues(netdev);
} else {
}
}
-pf_has_reset:
ixgbevf_update_stats(adapter);
+pf_has_reset:
/* Force detection of hung controller every watchdog period */
adapter->detect_tx_hung = true;
if (hw->adapter_stopped) {
err = IXGBE_ERR_MBX;
printk(KERN_ERR "Unable to start - perhaps the PF"
- "Driver isn't up yet\n");
+ " Driver isn't up yet\n");
goto err_setup_reset;
}
}
struct ixgbevf_tx_buffer *tx_buffer_info;
unsigned int len;
unsigned int total = skb->len;
- unsigned int offset = 0, size, count = 0, i;
+ unsigned int offset = 0, size, count = 0;
unsigned int nr_frags = skb_shinfo(skb)->nr_frags;
unsigned int f;
+ int i;
i = tx_ring->next_to_use;
/* setup the private structure */
err = ixgbevf_sw_init(adapter);
- ixgbevf_init_last_counter_stats(adapter);
-
#ifdef MAX_SKB_FRAGS
netdev->features = NETIF_F_SG |
NETIF_F_IP_CSUM |
adapter->netdev_registered = true;
+ ixgbevf_init_last_counter_stats(adapter);
+
/* print the MAC address */
hw_dbg(hw, "%2.2x:%2.2x:%2.2x:%2.2x:%2.2x:%2.2x\n",
netdev->dev_addr[0],
u64 vfgorc;
u64 vfgotc;
u64 vfmprc;
+
+ u64 saved_reset_vfgprc;
+ u64 saved_reset_vfgptc;
+ u64 saved_reset_vfgorc;
+ u64 saved_reset_vfgotc;
+ u64 saved_reset_vfmprc;
};
struct ixgbevf_info {
#include <linux/etherdevice.h>
#include <linux/init.h>
#include <linux/moduleparam.h>
+#include <linux/gfp.h>
#include <asm/hardware/uengine.h>
#include <asm/io.h>
#include "ixp2400_rx.ucode"
#include <linux/module.h>
#include <linux/types.h>
#include <linux/fcntl.h>
+#include <linux/gfp.h>
#include <linux/interrupt.h>
#include <linux/init.h>
#include <linux/ioport.h>
#include <linux/in.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/delay.h>
#include <linux/errno.h>
#include <linux/skbuff.h>
#include <linux/platform_device.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <asm/bootinfo.h>
#include <asm/system.h>
#include <linux/tcp.h>
#include <linux/udp.h>
#include <linux/if_vlan.h>
+#include <linux/slab.h>
#include <net/ip6_checksum.h>
#include "jme.h"
jme->jme_vlan_rx(skb, jme->vlgrp,
le16_to_cpu(rxdesc->descwb.vlan));
NET_STAT(jme).rx_bytes += 4;
+ } else {
+ dev_kfree_skb(skb);
}
} else {
jme->jme_rx(skb);
jme_reset_link(jme);
}
+static inline void jme_pause_rx(struct jme_adapter *jme)
+{
+ atomic_dec(&jme->link_changing);
+
+ jme_set_rx_pcc(jme, PCC_OFF);
+ if (test_bit(JME_FLAG_POLL, &jme->flags)) {
+ JME_NAPI_DISABLE(jme);
+ } else {
+ tasklet_disable(&jme->rxclean_task);
+ tasklet_disable(&jme->rxempty_task);
+ }
+}
+
+static inline void jme_resume_rx(struct jme_adapter *jme)
+{
+ struct dynpcc_info *dpi = &(jme->dpi);
+
+ if (test_bit(JME_FLAG_POLL, &jme->flags)) {
+ JME_NAPI_ENABLE(jme);
+ } else {
+ tasklet_hi_enable(&jme->rxclean_task);
+ tasklet_hi_enable(&jme->rxempty_task);
+ }
+ dpi->cur = PCC_P1;
+ dpi->attempt = PCC_P1;
+ dpi->cnt = 0;
+ jme_set_rx_pcc(jme, PCC_P1);
+
+ atomic_inc(&jme->link_changing);
+}
+
static void
jme_vlan_rx_register(struct net_device *netdev, struct vlan_group *grp)
{
struct jme_adapter *jme = netdev_priv(netdev);
+ jme_pause_rx(jme);
jme->vlgrp = grp;
+ jme_resume_rx(jme);
}
static void
#define __JME_H_INCLUDED__
#define DRV_NAME "jme"
-#define DRV_VERSION "1.0.5"
+#define DRV_VERSION "1.0.6"
#define PFX DRV_NAME ": "
#define PCI_DEVICE_ID_JMICRON_JMC250 0x0250
crc >>= (32 - 6); /* get top six bits */
rxctrl.mchash[crc >> 4] |= (1 << (crc & 0xf));
- mcptr = mcptr->next;
}
rxctrl.rxcr1 = RXCR1_RXME | RXCR1_RXPAFMA;
#include <linux/mii.h>
#include <linux/platform_device.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#define DRV_NAME "ks8851_mll"
#include <linux/if_vlan.h>
#include <linux/crc32.h>
#include <linux/sched.h>
+#include <linux/slab.h>
/* DMA Registers */
int len;
if (eeprom->magic != EEPROM_MAGIC)
- return 1;
+ return -EINVAL;
len = (eeprom->offset + eeprom->len + 1) / 2;
for (i = eeprom->offset / 2; i < len; i++)
#include <linux/ptrace.h>
#include <linux/errno.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/delay.h>
#include <linux/netdevice.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/delay.h>
#include <linux/netdevice.h>
#include <linux/dma-mapping.h>
#include <linux/io.h>
#include <linux/irq.h>
+#include <linux/gfp.h>
/* DEBUG flags
*/
#include <linux/in.h>
#include <linux/io.h>
#include <linux/ip.h>
+#include <linux/slab.h>
#include "ll_temac.h"
#include <linux/phy.h>
#include <linux/of.h>
#include <linux/of_device.h>
+#include <linux/slab.h>
#include <linux/of_mdio.h>
#include "ll_temac.h"
#include <linux/ioport.h>
#include <linux/nubus.h>
#include <linux/in.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/ioport.h>
#include <linux/in.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/nubus.h>
#include <linux/errno.h>
#include <linux/skbuff.h>
#include <linux/delay.h>
#include <linux/bitops.h>
+#include <linux/gfp.h>
#include <asm/system.h>
#include <asm/io.h>
#include <linux/crc32.h>
#include <linux/spinlock.h>
#include <linux/bitrev.h>
+#include <linux/slab.h>
#include <asm/prom.h>
#include <asm/dbdma.h>
#include <asm/io.h>
#include <linux/bitrev.h>
#include <linux/dma-mapping.h>
#include <linux/platform_device.h>
+#include <linux/gfp.h>
#include <asm/io.h>
#include <asm/irq.h>
#include <asm/macintosh.h>
#include <linux/module.h>
#include <linux/types.h>
#include <linux/fcntl.h>
+#include <linux/gfp.h>
#include <linux/interrupt.h>
#include <linux/init.h>
#include <linux/ioport.h>
#include <linux/in.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/delay.h>
#include <linux/nubus.h>
#include <linux/platform_device.h>
#include <linux/dma-mapping.h>
#include <linux/bitrev.h>
+#include <linux/slab.h>
#include <asm/bootinfo.h>
#include <asm/system.h>
#include <linux/cache.h>
#include <linux/sched.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/wait.h>
#include <linux/cdev.h>
*/
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/errno.h>
*/
#include <linux/hardirq.h>
+#include <linux/gfp.h>
#include <linux/mlx4/cmd.h>
#include <linux/mlx4/cq.h>
#include <linux/module.h>
#include <linux/delay.h>
#include <linux/netdevice.h>
+#include <linux/slab.h>
#include <linux/mlx4/driver.h>
#include <linux/mlx4/device.h>
#include <linux/tcp.h>
#include <linux/if_vlan.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/mlx4/driver.h>
#include <linux/mlx4/device.h>
*
*/
+#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/mlx4/qp.h>
*/
#include <linux/mlx4/cq.h>
+#include <linux/slab.h>
#include <linux/mlx4/qp.h>
#include <linux/skbuff.h>
#include <linux/if_ether.h>
#include <asm/page.h>
#include <linux/mlx4/cq.h>
+#include <linux/slab.h>
#include <linux/mlx4/qp.h>
#include <linux/skbuff.h>
#include <linux/if_vlan.h>
*/
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/dma-mapping.h>
#include <linux/errno.h>
#include <linux/mm.h>
#include <linux/scatterlist.h>
+#include <linux/slab.h>
#include <linux/mlx4/cmd.h>
* SOFTWARE.
*/
+#include <linux/slab.h>
+
#include "mlx4.h"
struct mlx4_device_context {
#include <linux/errno.h>
#include <linux/pci.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <linux/mlx4/device.h>
#include <linux/mlx4/doorbell.h>
info->port_attr.attr.mode = S_IRUGO | S_IWUSR;
info->port_attr.show = show_port_type;
info->port_attr.store = set_port_type;
+ sysfs_attr_init(&info->port_attr.attr);
err = device_create_file(&dev->pdev->dev, &info->port_attr);
if (err) {
*/
#include <linux/string.h>
-#include <linux/slab.h>
#include <linux/mlx4/cmd.h>
*/
#include <linux/errno.h>
+#include <linux/slab.h>
#include <linux/mlx4/cmd.h>
* SOFTWARE.
*/
+#include <linux/slab.h>
+
#include "mlx4.h"
#include "fw.h"
* SOFTWARE.
*/
+#include <linux/gfp.h>
#include <linux/mlx4/cmd.h>
#include <linux/mlx4/qp.h>
*/
#include <linux/mlx4/cmd.h>
+#include <linux/gfp.h>
#include "mlx4.h"
#include "icm.h"
#include <linux/io.h>
#include <linux/types.h>
#include <linux/inet_lro.h>
+#include <linux/slab.h>
#include <asm/system.h>
static char mv643xx_eth_driver_name[] = "mv643xx_eth";
#include <linux/types.h>
#include <linux/interrupt.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/errno.h>
+#include <linux/gfp.h>
/* Used for the temporal inet entries and routing */
#include <linux/socket.h>
#include <linux/route.h>
#include <linux/moduleparam.h>
#include <linux/io.h>
#include <linux/log2.h>
+#include <linux/slab.h>
#include <net/checksum.h>
#include <net/ip.h>
#include <net/tcp.h>
#include <linux/interrupt.h>
#include <linux/ioport.h>
#include <linux/in.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/of.h>
#include <linux/of_device.h>
#include <linux/firmware.h>
+#include <linux/gfp.h>
#include <net/dst.h>
#include <net/arp.h>
#include <linux/interrupt.h>
#include <linux/ioport.h>
#include <linux/in.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/init.h>
#include <linux/mm.h>
#include <linux/init.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/console.h>
#include <linux/moduleparam.h>
#include <linux/string.h>
#define _NETXEN_NIC_LINUX_MAJOR 4
#define _NETXEN_NIC_LINUX_MINOR 0
-#define _NETXEN_NIC_LINUX_SUBVERSION 72
-#define NETXEN_NIC_LINUX_VERSIONID "4.0.72"
+#define _NETXEN_NIC_LINUX_SUBVERSION 73
+#define NETXEN_NIC_LINUX_VERSIONID "4.0.73"
#define NETXEN_VERSION_CODE(a, b, c) (((a) << 24) + ((b) << 16) + (c))
#define _major(v) (((v) >> 24) & 0xff)
}
sds_ring->desc_head = (struct status_desc *)addr;
- sds_ring->crb_sts_consumer =
- netxen_get_ioaddr(adapter,
- recv_crb_registers[port].crb_sts_consumer[ring]);
+ if (NX_IS_REVISION_P2(adapter->ahw.revision_id)) {
+ sds_ring->crb_sts_consumer =
+ netxen_get_ioaddr(adapter,
+ recv_crb_registers[port].crb_sts_consumer[ring]);
- sds_ring->crb_intr_mask =
- netxen_get_ioaddr(adapter,
- recv_crb_registers[port].sw_int_mask[ring]);
+ sds_ring->crb_intr_mask =
+ netxen_get_ioaddr(adapter,
+ recv_crb_registers[port].sw_int_mask[ring]);
+ }
}
*
*/
+#include <linux/slab.h>
#include "netxen_nic.h"
#include "netxen_nic_hw.h"
#include <linux/netdevice.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include "netxen_nic.h"
#include "netxen_nic_hw.h"
if (adapter->fw_type == NX_UNIFIED_ROMIMAGE) {
bios_ver = cpu_to_le32(*((u32 *) (&fw->data[prd_off])
+ NX_UNI_BIOS_VERSION_OFF));
- return (bios_ver << 24) + ((bios_ver >> 8) & 0xff00) +
+ return (bios_ver << 16) + ((bios_ver >> 8) & 0xff00) +
(bios_ver >> 24);
} else
return cpu_to_le32(*(u32 *)&fw->data[NX_BIOS_VERSION_OFFSET]);
*
*/
+#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/interrupt.h>
#include "netxen_nic_hw.h"
static int
netxen_setup_pci_map(struct netxen_adapter *adapter)
{
- void __iomem *mem_ptr0 = NULL;
- void __iomem *mem_ptr1 = NULL;
- void __iomem *mem_ptr2 = NULL;
void __iomem *db_ptr = NULL;
resource_size_t mem_base, db_base;
- unsigned long mem_len, db_len = 0, pci_len0 = 0;
+ unsigned long mem_len, db_len = 0;
struct pci_dev *pdev = adapter->pdev;
int pci_func = adapter->ahw.pci_func;
+ struct netxen_hardware_context *ahw = &adapter->ahw;
int err = 0;
/* 128 Meg of memory */
if (mem_len == NETXEN_PCI_128MB_SIZE) {
- mem_ptr0 = ioremap(mem_base, FIRST_PAGE_GROUP_SIZE);
- mem_ptr1 = ioremap(mem_base + SECOND_PAGE_GROUP_START,
+
+ ahw->pci_base0 = ioremap(mem_base, FIRST_PAGE_GROUP_SIZE);
+ ahw->pci_base1 = ioremap(mem_base + SECOND_PAGE_GROUP_START,
SECOND_PAGE_GROUP_SIZE);
- mem_ptr2 = ioremap(mem_base + THIRD_PAGE_GROUP_START,
+ ahw->pci_base2 = ioremap(mem_base + THIRD_PAGE_GROUP_START,
THIRD_PAGE_GROUP_SIZE);
- pci_len0 = FIRST_PAGE_GROUP_SIZE;
+ if (ahw->pci_base0 == NULL || ahw->pci_base1 == NULL ||
+ ahw->pci_base2 == NULL) {
+ dev_err(&pdev->dev, "failed to map PCI bar 0\n");
+ err = -EIO;
+ goto err_out;
+ }
+
+ ahw->pci_len0 = FIRST_PAGE_GROUP_SIZE;
+
} else if (mem_len == NETXEN_PCI_32MB_SIZE) {
- mem_ptr1 = ioremap(mem_base, SECOND_PAGE_GROUP_SIZE);
- mem_ptr2 = ioremap(mem_base + THIRD_PAGE_GROUP_START -
+
+ ahw->pci_base1 = ioremap(mem_base, SECOND_PAGE_GROUP_SIZE);
+ ahw->pci_base2 = ioremap(mem_base + THIRD_PAGE_GROUP_START -
SECOND_PAGE_GROUP_START, THIRD_PAGE_GROUP_SIZE);
+ if (ahw->pci_base1 == NULL || ahw->pci_base2 == NULL) {
+ dev_err(&pdev->dev, "failed to map PCI bar 0\n");
+ err = -EIO;
+ goto err_out;
+ }
+
} else if (mem_len == NETXEN_PCI_2MB_SIZE) {
- mem_ptr0 = pci_ioremap_bar(pdev, 0);
- if (mem_ptr0 == NULL) {
+ ahw->pci_base0 = pci_ioremap_bar(pdev, 0);
+ if (ahw->pci_base0 == NULL) {
dev_err(&pdev->dev, "failed to map PCI bar 0\n");
return -EIO;
}
- pci_len0 = mem_len;
+ ahw->pci_len0 = mem_len;
} else {
return -EIO;
}
dev_info(&pdev->dev, "%dMB memory map\n", (int)(mem_len>>20));
- adapter->ahw.pci_base0 = mem_ptr0;
- adapter->ahw.pci_len0 = pci_len0;
- adapter->ahw.pci_base1 = mem_ptr1;
- adapter->ahw.pci_base2 = mem_ptr2;
-
if (NX_IS_REVISION_P3P(adapter->ahw.revision_id)) {
adapter->ahw.ocm_win_crb = netxen_get_ioaddr(adapter,
NETXEN_PCIX_PS_REG(PCIX_OCM_WINDOW_REG(pci_func)));
int pci_func_id = PCI_FUNC(pdev->devfn);
uint8_t revision_id;
- if (pdev->revision >= NX_P3_A0 && pdev->revision < NX_P3_B1) {
- pr_warning("%s: chip revisions between 0x%x-0x%x"
+ if (pdev->revision >= NX_P3_A0 && pdev->revision <= NX_P3_B1) {
+ pr_warning("%s: chip revisions between 0x%x-0x%x "
"will not be enabled.\n",
module_name(THIS_MODULE), NX_P3_A0, NX_P3_B1);
return -ENODEV;
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/jiffies.h>
#include <linux/crc32.h>
#include <linux/list.h>
+#include <linux/slab.h>
#include <linux/io.h>
#include <linux/if_vlan.h>
#include <linux/rtnetlink.h>
#include <linux/jiffies.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/uaccess.h>
#include <linux/netdevice.h>
#include <linux/etherdevice.h>
#include <linux/if_vlan.h>
+#include <linux/slab.h>
#include <linux/phy.h>
#include <linux/spinlock.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/dmaengine.h>
#include <linux/delay.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/timer.h>
#include <linux/delay.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/timer.h>
#include <linux/delay.h>
PCMCIA_PFC_DEVICE_MANF_CARD(0, 0x021b, 0x0101),
PCMCIA_PFC_DEVICE_MANF_CARD(0, 0x08a1, 0xc0ab),
PCMCIA_PFC_DEVICE_PROD_ID12(0, "AnyCom", "Fast Ethernet + 56K COMBO", 0x578ba6e7, 0xb0ac62c4),
+ PCMCIA_PFC_DEVICE_PROD_ID12(0, "ATKK", "LM33-PCM-T", 0xba9eb7e2, 0x077c174e),
PCMCIA_PFC_DEVICE_PROD_ID12(0, "D-Link", "DME336T", 0x1a424a1c, 0xb23897ff),
PCMCIA_PFC_DEVICE_PROD_ID12(0, "Grey Cell", "GCS3000", 0x2a151fac, 0x48b932ae),
PCMCIA_PFC_DEVICE_PROD_ID12(0, "Linksys", "EtherFast 10&100 + 56K PC Card (PCMLM56)", 0x0733cc81, 0xb3765033),
PCMCIA_MFC_DEVICE_CIS_PROD_ID12(0, "DAYNA COMMUNICATIONS", "LAN AND MODEM MULTIFUNCTION", 0x8fdf8f89, 0xdd5ed9e8, "cis/DP83903.cis"),
PCMCIA_MFC_DEVICE_CIS_PROD_ID4(0, "NSC MF LAN/Modem", 0x58fc6056, "cis/DP83903.cis"),
PCMCIA_MFC_DEVICE_CIS_MANF_CARD(0, 0x0175, 0x0000, "cis/DP83903.cis"),
- PCMCIA_DEVICE_CIS_MANF_CARD(0xc00f, 0x0002, "cis/LA-PCM.cis"),
+ PCMCIA_DEVICE_CIS_PROD_ID12("Allied Telesis,K.K", "Ethernet LAN Card", 0x2ad62f3c, 0x9fd2f0a2, "cis/LA-PCM.cis"),
PCMCIA_DEVICE_CIS_PROD_ID12("KTI", "PE520 PLUS", 0xad180345, 0x9d58d392, "cis/PE520.cis"),
PCMCIA_DEVICE_CIS_PROD_ID12("NDC", "Ethernet", 0x01c43ae1, 0x00b2e941, "cis/NE2K.cis"),
PCMCIA_DEVICE_CIS_PROD_ID12("PMX ", "PE-200", 0x34f3f1c8, 0x10b59f8c, "cis/PE-200.cis"),
{
struct net_device *dev = priv;
cisparse_t parse;
+ u8 *buf;
if (pcmcia_parse_tuple(tuple, &parse))
return -EINVAL;
- if ((parse.version_1.ns > 3) &&
- (cvt_ascii_address(dev,
- (parse.version_1.str + parse.version_1.ofs[3]))))
+ buf = parse.version_1.str + parse.version_1.ofs[3];
+
+ if ((parse.version_1.ns > 3) && (cvt_ascii_address(dev, buf) == 0))
return 0;
return -EINVAL;
len = pcmcia_get_tuple(link, 0x81, &buf);
if (buf && len >= 13) {
buf[12] = '\0';
- if (cvt_ascii_address(dev, buf))
+ if (cvt_ascii_address(dev, buf) == 0)
rc = 0;
}
kfree(buf);
if (i != 0) {
printk(KERN_NOTICE "smc91c92_cs: Unable to find hardware address.\n");
- goto config_undo;
+ goto config_failed;
}
smc->duplex = 0;
unregister_netdev(dev);
config_failed:
smc91c92_release(link);
+ free_netdev(dev);
return -ENODEV;
} /* smc91c92_config */
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/unistd.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/init.h>
#include <linux/delay.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/unistd.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/init.h>
#include <linux/delay.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/unistd.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/init.h>
#include <linux/delay.h>
#include <linux/phy.h>
#include <linux/phy_fixed.h>
#include <linux/err.h>
+#include <linux/slab.h>
#define MII_REGS_NUM 29
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/unistd.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/init.h>
#include <linux/delay.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/unistd.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/init.h>
#include <linux/delay.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/unistd.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/init.h>
#include <linux/delay.h>
#include <linux/module.h>
#include <linux/mdio-bitbang.h>
-#include <linux/slab.h>
#include <linux/types.h>
#include <linux/delay.h>
* Copyright (C) 2009 Cavium Networks
*/
+#include <linux/gfp.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/platform_device.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/unistd.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/init.h>
#include <linux/delay.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/unistd.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/init.h>
#include <linux/delay.h>
#include <linux/fcntl.h>
#include <linux/interrupt.h>
#include <linux/string.h>
+#include <linux/slab.h>
#include <linux/if_ether.h>
#include <linux/in.h>
#include <linux/errno.h>
#include <linux/spinlock.h>
#include <linux/init.h>
#include <linux/jiffies.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/string.h>
#include <linux/stddef.h>
#include <linux/device.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <net/slhc_vj.h>
#include <asm/atomic.h>
#include <linux/spinlock.h>
#include <linux/completion.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#define PPP_VERSION "2.4.2"
#include <linux/string.h>
#include <linux/module.h>
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/netdevice.h>
#include <linux/net.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/etherdevice.h>
#include <linux/ethtool.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/etherdevice.h>
#include <linux/ethtool.h>
#include "qlcnic.h"
+#include <linux/slab.h>
#include <net/ip.h>
#define MASK(n) ((1ULL<<(n))-1)
#include <linux/netdevice.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include "qlcnic.h"
struct crb_addr_pair {
*
*/
+#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
+
#include "qlge.h"
/* Read a NIC register from the alternate function. */
#include <linux/dma-mapping.h>
#include <linux/pagemap.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/dmapool.h>
#include <linux/mempool.h>
#include <linux/spinlock.h>
#include <linux/timer.h>
#include <linux/errno.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
#include <linux/netdevice.h>
MODULE_DEVICE_TABLE(pci, rtl8169_pci_tbl);
-static int rx_copybreak = 200;
-static int use_dac = -1;
+/*
+ * we set our copybreak very high so that we don't have
+ * to allocate 16k frames all the time (see note in
+ * rtl8169_open()
+ */
+static int rx_copybreak = 16383;
+static int use_dac;
static struct {
u32 msg_enable;
} debug = { -1 };
module_param(rx_copybreak, int, 0);
MODULE_PARM_DESC(rx_copybreak, "Copy breakpoint for copy-only-tiny-frames");
module_param(use_dac, int, 0);
-MODULE_PARM_DESC(use_dac, "Enable PCI DAC. -1 defaults on for PCI Express only."
-" Unsafe on 32 bit PCI slot.");
+MODULE_PARM_DESC(use_dac, "Enable PCI DAC. Unsafe on 32 bit PCI slot.");
module_param_named(debug, debug.msg_enable, int, 0);
MODULE_PARM_DESC(debug, "Debug verbosity level (0=none, ..., 16=all)");
MODULE_LICENSE("GPL");
spin_lock_irq(&tp->lock);
RTL_W8(Cfg9346, Cfg9346_Unlock);
- RTL_W32(MAC0, low);
RTL_W32(MAC4, high);
+ RTL_W32(MAC0, low);
RTL_W8(Cfg9346, Cfg9346_Lock);
spin_unlock_irq(&tp->lock);
void __iomem *ioaddr;
unsigned int i;
int rc;
- int this_use_dac = use_dac;
if (netif_msg_drv(&debug)) {
printk(KERN_INFO "%s Gigabit Ethernet driver %s loaded\n",
tp->cp_cmd = PCIMulRW | RxChkSum;
- tp->pcie_cap = pci_find_capability(pdev, PCI_CAP_ID_EXP);
- if (!tp->pcie_cap)
- netif_info(tp, probe, dev, "no PCI Express capability\n");
-
- if (this_use_dac < 0)
- this_use_dac = tp->pcie_cap != 0;
-
if ((sizeof(dma_addr_t) > 4) &&
- this_use_dac &&
- !pci_set_dma_mask(pdev, DMA_BIT_MASK(64))) {
- netif_info(tp, probe, dev, "using 64-bit DMA\n");
+ !pci_set_dma_mask(pdev, DMA_BIT_MASK(64)) && use_dac) {
tp->cp_cmd |= PCIDAC;
dev->features |= NETIF_F_HIGHDMA;
} else {
goto err_out_free_res_4;
}
+ tp->pcie_cap = pci_find_capability(pdev, PCI_CAP_ID_EXP);
+ if (!tp->pcie_cap)
+ netif_info(tp, probe, dev, "no PCI Express capability\n");
+
RTL_W16(IntrMask, 0x0000);
/* Soft reset the chip. */
}
static void rtl8169_set_rxbufsize(struct rtl8169_private *tp,
- struct net_device *dev)
+ unsigned int mtu)
{
- unsigned int max_frame = dev->mtu + VLAN_ETH_HLEN + ETH_FCS_LEN;
+ unsigned int max_frame = mtu + VLAN_ETH_HLEN + ETH_FCS_LEN;
+
+ if (max_frame != 16383)
+ printk(KERN_WARNING PFX "WARNING! Changing of MTU on this "
+ "NIC may lead to frame reception errors!\n");
tp->rx_buf_sz = (max_frame > RX_BUF_SIZE) ? max_frame : RX_BUF_SIZE;
}
int retval = -ENOMEM;
- rtl8169_set_rxbufsize(tp, dev);
+ /*
+ * Note that we use a magic value here, its wierd I know
+ * its done because, some subset of rtl8169 hardware suffers from
+ * a problem in which frames received that are longer than
+ * the size set in RxMaxSize register return garbage sizes
+ * when received. To avoid this we need to turn off filtering,
+ * which is done by setting a value of 16383 in the RxMaxSize register
+ * and allocating 16k frames to handle the largest possible rx value
+ * thats what the magic math below does.
+ */
+ rtl8169_set_rxbufsize(tp, 16383 - VLAN_ETH_HLEN - ETH_FCS_LEN);
/*
* Rx and Tx desscriptors needs 256 bytes alignment.
rtl8169_down(dev);
- rtl8169_set_rxbufsize(tp, dev);
+ rtl8169_set_rxbufsize(tp, dev->mtu);
ret = rtl8169_init_ring(dev);
if (ret < 0)
mc_filter[1] = swab32(data);
}
- RTL_W32(MAR0 + 0, mc_filter[0]);
RTL_W32(MAR0 + 4, mc_filter[1]);
+ RTL_W32(MAR0 + 0, mc_filter[0]);
RTL_W32(RxConfig, tmp);
#include <linux/delay.h>
#include <linux/rio.h>
#include <linux/rio_drv.h>
+#include <linux/slab.h>
#include <linux/rio_ids.h>
#include <linux/netdevice.h>
#include <linux/init.h>
#include <linux/delay.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <asm/system.h>
#include <linux/tcp.h>
#include <linux/uaccess.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <net/tcp.h>
#include <asm/system.h>
#include <linux/errno.h>
#include <linux/if_cablemodem.h> /* for SIOGCM/SIOSCM stuff */
#include <linux/in.h>
-#include <linux/slab.h>
#include <linux/ioport.h>
#include <linux/netdevice.h>
#include <linux/if_arp.h>
#include <linux/pnp.h>
#include <linux/init.h>
#include <linux/bitops.h>
+#include <linux/gfp.h>
#include <asm/io.h>
#include <asm/processor.h>
#include <linux/interrupt.h>
#include <linux/ioport.h>
#include <linux/in.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/init.h>
#include <linux/delay.h>
#include <linux/crc32.h>
#include <linux/ethtool.h>
#include <linux/topology.h>
+#include <linux/gfp.h>
#include "net_driver.h"
#include "efx.h"
#include "mdio_10g.h"
#include <linux/seq_file.h>
#include <linux/i2c.h>
#include <linux/mii.h>
+#include <linux/slab.h>
#include "net_driver.h"
#include "bitfield.h"
#include "efx.h"
* Driver for PHY related operations via MCDI.
*/
+#include <linux/slab.h>
#include "efx.h"
#include "phy.h"
#include "mcdi.h"
#include <linux/module.h>
#include <linux/mtd/mtd.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/rtnetlink.h>
#define EFX_DRIVER_NAME "sfc_mtd"
* Driver for AMCC QT202x SFP+ and XFP adapters; see www.amcc.com for details
*/
+#include <linux/slab.h>
#include <linux/timer.h>
#include <linux/delay.h>
#include "efx.h"
#include <linux/socket.h>
#include <linux/in.h>
+#include <linux/slab.h>
#include <linux/ip.h>
#include <linux/tcp.h>
#include <linux/udp.h>
#include <linux/in.h>
#include <linux/udp.h>
#include <linux/rtnetlink.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include "net_driver.h"
#include "efx.h"
#include <linux/delay.h>
#include <linux/pci.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include "net_driver.h"
#include "bitfield.h"
#include "efx.h"
#include <linux/delay.h>
#include <linux/rtnetlink.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include "efx.h"
#include "mdio_10g.h"
#include "nic.h"
#include <linux/ip.h>
#include <linux/in.h>
#include <linux/ipv6.h>
+#include <linux/slab.h>
#include <net/ipv6.h>
#include <linux/if_ether.h>
#include <linux/highmem.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/init.h>
#include <linux/types.h>
/* Setup... */
len = skb->len;
if (len < ETH_ZLEN) {
- if (skb_padto(skb, ETH_ZLEN))
+ if (skb_padto(skb, ETH_ZLEN)) {
+ spin_unlock_irqrestore(&sp->tx_lock, flags);
return NETDEV_TX_OK;
+ }
len = ETH_ZLEN;
}
#include <linux/cache.h>
#include <linux/io.h>
#include <linux/pm_runtime.h>
+#include <linux/slab.h>
#include <asm/cacheflush.h>
#include "sh_eth.h"
#include <linux/delay.h>
#include <linux/crc32.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <asm/irq.h>
#define PHY_MAX_ADDR 32
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
#include <linux/netdevice.h>
#include <linux/fddidevice.h>
#include <linux/skbuff.h>
#include <linux/bitops.h>
+#include <linux/gfp.h>
#include <asm/byteorder.h>
#include <asm/io.h>
#include <linux/sched.h>
#include <linux/seq_file.h>
#include <linux/mii.h>
+#include <linux/slab.h>
#include <asm/irq.h>
#include "skge.h"
#include <linux/ethtool.h>
#include <linux/pci.h>
#include <linux/ip.h>
+#include <linux/slab.h>
#include <net/ip.h>
#include <linux/tcp.h>
#include <linux/in.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/compat.h>
#include <linux/delay.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include "slip.h"
#ifdef CONFIG_INET
#include <linux/ip.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/errno.h>
#include <linux/interrupt.h>
#include <linux/ioport.h>
#include <linux/in.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/init.h>
#include <linux/crc32.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/errno.h>
#include <linux/netdevice.h>
#include <linux/platform_device.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/timer.h>
#include <linux/bug.h>
#include <linux/bitops.h>
#include <linux/if_vlan.h>
#include <linux/dma-mapping.h>
#include <linux/crc32.h>
+#include <linux/slab.h>
#include <asm/unaligned.h>
#include "smsc9420.h"
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/delay.h>
#include <linux/netdevice.h>
#include <linux/if_vlan.h>
#include <linux/in.h>
#include <linux/init.h>
+#include <linux/gfp.h>
#include <linux/ioport.h>
#include <linux/ip.h>
#include <linux/kernel.h>
#include <linux/device.h>
#include <linux/pci.h>
#include <linux/skbuff.h>
-#include <linux/slab.h>
#include <linux/tcp.h>
#include <linux/types.h>
#include <linux/vmalloc.h>
tristate "STMicroelectronics 10/100/1000 Ethernet driver"
select MII
select PHYLIB
+ select CRC32
depends on NETDEVICES && CPU_SUBTYPE_ST40
help
This is the driver for the Ethernet IPs are built around a
#include <linux/crc32.h>
#include <linux/mii.h>
#include <linux/phy.h>
+#include <linux/slab.h>
#include "common.h"
#include "dwmac100.h"
*******************************************************************************/
#include <linux/crc32.h>
+#include <linux/slab.h>
#include "dwmac1000.h"
static void dwmac1000_core_init(unsigned long ioaddr)
#include <linux/phy.h>
#include <linux/if_vlan.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include "stmmac.h"
#define STMMAC_RESOURCE_NAME "stmmaceth"
#include <linux/mii.h>
#include <linux/phy.h>
+#include <linux/slab.h>
#include "stmmac.h"
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/errno.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/init.h>
#include <linux/ioport.h>
#include <linux/interrupt.h>
#include <linux/ioport.h>
#include <linux/in.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/dma-mapping.h>
#include <linux/of.h>
#include <linux/of_device.h>
+#include <linux/gfp.h>
#include <asm/auxio.h>
#include <asm/byteorder.h>
#include <linux/timer.h>
#include <linux/errno.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
#include <linux/netdevice.h>
#include <linux/ioport.h>
#include <linux/in.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/bitops.h>
#include <linux/mutex.h>
#include <linux/mm.h>
+#include <linux/gfp.h>
#include <asm/system.h>
#include <asm/io.h>
#include <linux/interrupt.h>
#include <linux/ioport.h>
#include <linux/in.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/dma-mapping.h>
#include <linux/of.h>
#include <linux/of_device.h>
+#include <linux/gfp.h>
#include <asm/system.h>
#include <asm/io.h>
#include <linux/firmware.h>
#include <asm/byteorder.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
/* Compile Time Switches */
/* start */
#include <linux/spinlock.h>
#include <linux/bitops.h>
#include <linux/firmware.h>
+#include <linux/slab.h>
#include <net/checksum.h>
#include <linux/spinlock.h>
#include <linux/bitops.h>
#include <linux/jiffies.h>
+#include <linux/slab.h>
#include <net/net_namespace.h>
#include <net/checksum.h>
#include <linux/module.h>
#include <linux/mca.h>
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/init.h>
#include <linux/ptrace.h>
#include <linux/ioport.h>
#include <linux/in.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/time.h>
#include <linux/errno.h>
#include <linux/ptrace.h>
#include <linux/ioport.h>
#include <linux/in.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/time.h>
#include <linux/errno.h>
#include <linux/etherdevice.h>
#include <linux/ethtool.h>
#include <linux/skbuff.h>
-#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/delay.h>
#include <linux/crc32.h>
#include <linux/rtnetlink.h>
#include <linux/timer.h>
#include <linux/platform_device.h>
+#include <linux/gfp.h>
#include <asm/system.h>
#include <asm/io.h>
#include <linux/compiler.h>
#include <linux/rtnetlink.h>
#include <linux/crc32.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/irq.h>
#include <linux/ptrace.h>
#include <linux/errno.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/eisa.h>
#include <linux/delay.h>
#include <linux/dma-mapping.h>
#include <linux/moduleparam.h>
#include <linux/bitops.h>
+#include <linux/gfp.h>
#include <asm/io.h>
#include <asm/dma.h>
#include <linux/ptrace.h>
#include <linux/errno.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
#include <linux/dma-mapping.h>
*/
#include <linux/pci.h>
+#include <linux/slab.h>
#include "tulip.h"
#include <linux/init.h>
#include <asm/unaligned.h>
#include <linux/module.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include "tulip.h"
#include <linux/init.h>
#include <linux/etherdevice.h>
#include <linux/timer.h>
#include <linux/errno.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
#include <linux/init.h>
if ( !(rdes0 & 0x8000) ||
((db->cr6_data & CR6_PM) && (rxlen>6)) ) {
+ struct sk_buff *new_skb = NULL;
+
skb = rxptr->rx_skb_ptr;
/* Good packet, send to upper layer */
/* Shorst packet used new SKB */
- if ( (rxlen < RX_COPY_SIZE) &&
- ( (skb = dev_alloc_skb(rxlen + 2) )
- != NULL) ) {
+ if ((rxlen < RX_COPY_SIZE) &&
+ (((new_skb = dev_alloc_skb(rxlen + 2)) != NULL))) {
+ skb = new_skb;
/* size less than COPY_SIZE, allocate a rxlen SKB */
skb_reserve(skb, 2); /* 16byte align */
memcpy(skb_put(skb, rxlen),
#include <linux/timer.h>
#include <linux/errno.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
#include <linux/dma-mapping.h>
#include <linux/timer.h>
#include <linux/errno.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
#include <linux/netdevice.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/errno.h>
-#include <linux/slab.h>
#include <linux/stddef.h>
#include <linux/interrupt.h>
#include <linux/netdevice.h>
#include <linux/usb.h>
#include <linux/crc32.h>
#include <linux/usb/usbnet.h>
+#include <linux/slab.h>
#define DRIVER_VERSION "14-Jun-2006"
static const char driver_name [] = "asix";
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/string.h>
-#include <linux/slab.h>
#include <linux/netdevice.h>
#include <linux/etherdevice.h>
#include <linux/skbuff.h>
#include <linux/ethtool.h>
#include <linux/crc32.h>
#include <linux/bitops.h>
+#include <linux/gfp.h>
#include <asm/uaccess.h>
#undef DEBUG
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/gfp.h>
#include <linux/usb.h>
#include <linux/usb/cdc.h>
#include <linux/netdevice.h>
#include <linux/crc32.h>
#include <linux/usb/cdc.h>
#include <linux/usb/usbnet.h>
+#include <linux/gfp.h>
/*
#include <linux/usb.h>
#include <linux/crc32.h>
#include <linux/usb/usbnet.h>
+#include <linux/slab.h>
/* datasheet:
http://ptm2.cc.utu.fi/ftp/network/cards/DM9601/From_NET/DM9601-DS-P01-930914.pdf
#include <linux/mii.h>
#include <linux/usb.h>
#include <linux/usb/usbnet.h>
+#include <linux/gfp.h>
/*
#include <linux/netdevice.h>
#include <linux/etherdevice.h>
#include <linux/ethtool.h>
+#include <linux/slab.h>
#include <linux/mii.h>
#include <linux/usb.h>
#include <linux/usb/usbnet.h>
#include <linux/mii.h>
#include <linux/module.h>
#include <linux/netdevice.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include <linux/usb/usbnet.h>
#include <linux/mii.h>
#include <linux/usb.h>
#include <linux/usb/usbnet.h>
+#include <linux/slab.h>
#include <asm/unaligned.h>
#include <linux/etherdevice.h>
#include <linux/ethtool.h>
#include <linux/workqueue.h>
+#include <linux/slab.h>
#include <linux/mii.h>
#include <linux/usb.h>
#include <linux/usb/cdc.h>
#include <linux/usb.h>
#include <linux/crc32.h>
#include <linux/usb/usbnet.h>
+#include <linux/slab.h>
#include "smsc75xx.h"
#define SMSC_CHIPNAME "smsc75xx"
#include <linux/usb.h>
#include <linux/crc32.h>
#include <linux/usb/usbnet.h>
+#include <linux/slab.h>
#include "smsc95xx.h"
#define SMSC_CHIPNAME "smsc95xx"
}
if (csum) {
- u32 csum_preamble = smsc95xx_calc_csum_preamble(skb);
- skb_push(skb, 4);
- memcpy(skb->data, &csum_preamble, 4);
+ if (skb->len <= 45) {
+ /* workaround - hardware tx checksum does not work
+ * properly with extremely small packets */
+ long csstart = skb->csum_start - skb_headroom(skb);
+ __wsum calc = csum_partial(skb->data + csstart,
+ skb->len - csstart, 0);
+ *((__sum16 *)(skb->data + csstart
+ + skb->csum_offset)) = csum_fold(calc);
+
+ csum = false;
+ } else {
+ u32 csum_preamble = smsc95xx_calc_csum_preamble(skb);
+ skb_push(skb, 4);
+ memcpy(skb->data, &csum_preamble, 4);
+ }
}
skb_push(skb, 4);
#include <linux/mii.h>
#include <linux/usb.h>
#include <linux/usb/usbnet.h>
+#include <linux/slab.h>
#define DRIVER_VERSION "22-Aug-2005"
*/
#include <linux/netdevice.h>
+#include <linux/slab.h>
#include <linux/ethtool.h>
#include <linux/etherdevice.h>
#include <linux/timer.h>
#include <linux/errno.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
#include <linux/dma-mapping.h>
case FLOW_CNTL_TX_RX:
MII_REG_BITS_ON(ANAR_PAUSE, MII_REG_ANAR, vptr->mac_regs);
- MII_REG_BITS_ON(ANAR_ASMDIR, MII_REG_ANAR, vptr->mac_regs);
+ MII_REG_BITS_OFF(ANAR_ASMDIR, MII_REG_ANAR, vptr->mac_regs);
break;
case FLOW_CNTL_DISABLE:
#include <linux/virtio_net.h>
#include <linux/scatterlist.h>
#include <linux/if_vlan.h>
+#include <linux/slab.h>
static int napi_weight = 128;
module_param(napi_weight, int, 0444);
#include <linux/etherdevice.h>
#include <linux/pci.h>
#include <linux/pci_hotplug.h>
+#include <linux/slab.h>
#include "vxge-traffic.h"
#include "vxge-config.h"
#ifndef VXGE_CONFIG_H
#define VXGE_CONFIG_H
#include <linux/list.h>
+#include <linux/slab.h>
#ifndef VXGE_CACHE_LINE_SIZE
#define VXGE_CACHE_LINE_SIZE 128
* Copyright(c) 2002-2009 Neterion Inc.
******************************************************************************/
#include<linux/ethtool.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/etherdevice.h>
#include <linux/if_vlan.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/tcp.h>
#include <net/ip.h>
#include <linux/netdevice.h>
#include <linux/pci.h>
#include <linux/kernel.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include <asm/system.h>
#include <asm/cache.h>
#include <linux/version.h>
#include <linux/pci.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/ioport.h>
#include <linux/init.h>
#include <linux/if.h>
#include <linux/module.h>
#include <linux/netdevice.h>
#include <linux/skbuff.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/types.h>
#include <asm/io.h>
#include <linux/module.h>
#include <linux/netdevice.h>
#include <linux/skbuff.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/types.h>
#include <asm/io.h>
#include <linux/poll.h>
#include <linux/rtnetlink.h>
#include <linux/skbuff.h>
-#include <linux/slab.h>
#undef DEBUG_HARD_HEADER
#include <linux/poll.h>
#include <linux/rtnetlink.h>
#include <linux/skbuff.h>
-#include <linux/slab.h>
static int raw_ioctl(struct net_device *dev, struct ifreq *ifr);
#include <linux/errno.h>
#include <linux/etherdevice.h>
+#include <linux/gfp.h>
#include <linux/hdlc.h>
#include <linux/if_arp.h>
#include <linux/inetdevice.h>
#include <linux/poll.h>
#include <linux/rtnetlink.h>
#include <linux/skbuff.h>
-#include <linux/slab.h>
static int raw_eth_ioctl(struct net_device *dev, struct ifreq *ifr);
*/
#include <linux/errno.h>
+#include <linux/gfp.h>
#include <linux/hdlc.h>
#include <linux/if_arp.h>
#include <linux/inetdevice.h>
#include <linux/poll.h>
#include <linux/rtnetlink.h>
#include <linux/skbuff.h>
-#include <linux/slab.h>
#include <net/x25device.h>
static int x25_ioctl(struct net_device *dev, struct ifreq *ifr);
#include <linux/delay.h>
#include <linux/hdlc.h>
#include <linux/ioport.h>
+#include <linux/slab.h>
#include <net/arp.h>
#include <asm/irq.h>
#include <linux/kernel.h>
#include <linux/platform_device.h>
#include <linux/poll.h>
+#include <linux/slab.h>
#include <mach/npe.h>
#include <mach/qmgr.h>
#include <linux/types.h>
#include <linux/socket.h>
#include <linux/in.h>
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/net.h>
#include <linux/ptrace.h>
#include <linux/errno.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/in.h>
#include <linux/if_arp.h>
#include <linux/ptrace.h>
#include <linux/errno.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/in.h>
#include <linux/if_arp.h>
#include <linux/etherdevice.h>
#include <linux/spinlock.h>
#include <linux/if.h>
+#include <linux/slab.h>
#include <net/arp.h>
#include <asm/io.h>
#include <linux/fcntl.h>
#include <linux/ioport.h>
#include <linux/interrupt.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/netdevice.h>
#include <linux/hdlc.h>
#include <linux/ioport.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <net/arp.h>
#include <asm/irq.h>
#include <linux/init.h>
#include <linux/rtnetlink.h>
#include <linux/compat.h>
+#include <linux/slab.h>
#include "x25_asy.h"
#include <net/x25device.h>
#include <linux/hdlc.h>
#include <linux/ioport.h>
#include <linux/init.h>
+#include <linux/gfp.h>
#include <asm/dma.h>
#include <asm/io.h>
#define RT_LOCK
#include <stdarg.h>
#include "i2400m.h"
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/wimax/i2400m.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/suspend.h>
+#include <linux/slab.h>
#define D_SUBMODULE driver
#include "debug-levels.h"
*/
#include <linux/firmware.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include "i2400m.h"
* alloc_netdev.
*/
#include <linux/if_arp.h>
+#include <linux/slab.h>
#include <linux/netdevice.h>
#include <linux/ethtool.h>
#include "i2400m.h"
*/
#include "i2400m.h"
#include <linux/wimax/i2400m.h>
+#include <linux/slab.h>
* i2400m_msg_size_check
* wimax_msg
*/
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/if_arp.h>
#include <linux/netdevice.h>
#include <linux/skbuff.h>
#include <linux/mmc/sdio.h>
#include <linux/mmc/sdio_func.h>
+#include <linux/slab.h>
#include "i2400m-sdio.h"
#define D_SUBMODULE rx
* __i2400ms_send_barker()
*/
+#include <linux/slab.h>
#include <linux/debugfs.h>
#include <linux/mmc/sdio_ids.h>
#include <linux/mmc/sdio.h>
* (FIFO empty).
*/
#include <linux/netdevice.h>
+#include <linux/slab.h>
#include "i2400m.h"
* i2400m_notif_submit
*/
#include <linux/usb.h>
+#include <linux/gfp.h>
#include "i2400m-usb.h"
* i2400mu_rx_kick()
*/
#include <linux/usb.h>
+#include <linux/slab.h>
#include "i2400m-usb.h"
* i2400mu_rx_release() called from i2400mu_bus_dev_stop()
*/
#include <linux/workqueue.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include "i2400m-usb.h"
#include "i2400m-usb.h"
#include <linux/wimax/i2400m.h>
#include <linux/debugfs.h>
+#include <linux/slab.h>
#define D_SUBMODULE usb
#include <linux/init.h>
#include <linux/if.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <linux/etherdevice.h>
#include <linux/pci.h>
#include <linux/delay.h>
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/etherdevice.h>
#include <net/mac80211.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include <linux/firmware.h>
#include <linux/etherdevice.h>
{ USB_DEVICE(0x04bb, 0x093f) },
/* AVM FRITZ!WLAN USB Stick N */
{ USB_DEVICE(0x057C, 0x8401) },
+ /* NEC WL300NU-G */
+ { USB_DEVICE(0x0409, 0x0249) },
/* AVM FRITZ!WLAN USB Stick N 2.4 */
{ USB_DEVICE(0x057C, 0x8402), .driver_info = AR9170_REQ_FW1_ONLY },
spin_unlock_irqrestore(&aru->common.cmdlock, flags);
usb_fill_int_urb(urb, aru->udev,
- usb_sndbulkpipe(aru->udev, AR9170_EP_CMD),
+ usb_sndintpipe(aru->udev, AR9170_EP_CMD),
aru->common.cmdbuf, plen + 4,
ar9170_usb_tx_urb_complete, NULL, 1);
\*************************************/
#include <linux/pci.h>
+#include <linux/slab.h>
#include "ath5k.h"
#include "reg.h"
#include "debug.h"
#include <linux/pci.h>
#include <linux/ethtool.h>
#include <linux/uaccess.h>
+#include <linux/slab.h>
#include <net/ieee80211_radiotap.h>
* EEPROM access functions and helpers *
\*************************************/
+#include <linux/slab.h>
+
#include "ath5k.h"
#include "reg.h"
#include "debug.h"
#define _ATH5K_PHY
#include <linux/delay.h>
+#include <linux/slab.h>
#include "ath5k.h"
#include "reg.h"
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
+#include <linux/slab.h>
#include <asm/unaligned.h>
#include "ath9k.h"
*/
#include <linux/io.h>
+#include <linux/slab.h>
#include <asm/unaligned.h>
#include "hw.h"
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
+#include <linux/slab.h>
+
#include "ath9k.h"
static char *dev_info = "ath9k";
* AR9287 - 11n single-band 1x1 MIMO for USB
*/
+#include <linux/slab.h>
+
#include "hw.h"
/**
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
+#include <linux/slab.h>
+
#include "ath9k.h"
static const struct ath_rate_table ar5416_11na_ratetable = {
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
+#include <linux/slab.h>
+
#include "ath9k.h"
struct ath9k_vif_iter_data {
return htype;
}
-static bool is_pae(struct sk_buff *skb)
-{
- struct ieee80211_hdr *hdr;
- __le16 fc;
-
- hdr = (struct ieee80211_hdr *)skb->data;
- fc = hdr->frame_control;
-
- if (ieee80211_is_data(fc)) {
- if (ieee80211_is_nullfunc(fc) ||
- /* Port Access Entity (IEEE 802.1X) */
- (skb->protocol == cpu_to_be16(ETH_P_PAE))) {
- return true;
- }
- }
-
- return false;
-}
-
static int get_hw_crypto_keytype(struct sk_buff *skb)
{
struct ieee80211_tx_info *tx_info = IEEE80211_SKB_CB(skb);
goto tx_done;
}
- if ((tx_info->flags & IEEE80211_TX_CTL_AMPDU) && !is_pae(skb)) {
+ if (tx_info->flags & IEEE80211_TX_CTL_AMPDU) {
/*
* Try aggregation if it's a unicast data frame
* and the destination is HT capable.
*/
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <net/cfg80211.h>
#include <net/mac80211.h>
#include "regd.h"
#include <linux/delay.h>
#include <linux/skbuff.h>
#include <linux/etherdevice.h>
+#include <linux/slab.h>
#include <asm/div64.h>
#include <linux/delay.h>
#include <linux/sched.h>
+#include <linux/slab.h>
static struct b43_lo_calib *b43_find_lo_calib(struct b43_txpower_lo_control *lo,
#include <linux/skbuff.h>
#include <linux/io.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <asm/unaligned.h>
#include "b43.h"
#include "pcmcia.h"
#include <linux/ssb/ssb.h>
+#include <linux/slab.h>
#include <pcmcia/cs_types.h>
#include <pcmcia/cs.h>
*/
+#include <linux/slab.h>
+
#include "b43.h"
#include "phy_a.h"
#include "phy_common.h"
#include "main.h"
#include <linux/bitrev.h>
+#include <linux/slab.h>
static const s8 b43_tssi2dbm_g_table[] = {
*/
+#include <linux/slab.h>
+
#include "b43.h"
#include "main.h"
#include "phy_lp.h"
*/
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include "b43.h"
#include <linux/delay.h>
#include <linux/sched.h>
+#include <linux/slab.h>
static u16 generate_cookie(struct b43_pio_txqueue *q,
#include <linux/mmc/card.h>
#include <linux/mmc/sdio_func.h>
#include <linux/mmc/sdio_ids.h>
+#include <linux/slab.h>
#include <linux/ssb/ssb.h>
#include "sdio.h"
#include <linux/pci.h>
#include <linux/delay.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <net/dst.h>
/* 32bit DMA ops. */
#include <linux/sched.h>
#include <linux/skbuff.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <net/dst.h>
#include <asm/unaligned.h>
#include <linux/delay.h>
#include <linux/pci.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include "b43legacy.h"
#include "xmit.h"
#include <linux/delay.h>
+#include <linux/slab.h>
static void tx_start(struct b43legacy_pioqueue *queue)
#include <linux/etherdevice.h>
+#include <linux/slab.h>
#include <net/lib80211.h>
#include <linux/if_arp.h>
+#include <linux/slab.h>
+
#include "hostap_80211.h"
#include "hostap_common.h"
#include "hostap_wlan.h"
#include <linux/delay.h>
#include <linux/random.h>
#include <linux/if_arp.h>
+#include <linux/slab.h>
#include "hostap_wlan.h"
#include "hostap.h"
#include <linux/module.h>
#include <linux/init.h>
#include <linux/if.h>
+#include <linux/slab.h>
#include <linux/wait.h>
#include <linux/timer.h>
#include <linux/skbuff.h>
#include <linux/if_arp.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include "hostap_wlan.h"
#include "hostap.h"
#include "hostap_ap.h"
/* ioctl() (mostly Linux Wireless Extensions) routines for Host AP driver */
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/sched.h>
#include <linux/ethtool.h>
#include <linux/if.h>
#include <linux/skbuff.h>
#include <linux/netdevice.h>
+#include <linux/slab.h>
#include <linux/workqueue.h>
#include <linux/wireless.h>
#include <net/iw_handler.h>
#include <linux/if.h>
#include <linux/skbuff.h>
#include <linux/netdevice.h>
+#include <linux/slab.h>
#include <linux/workqueue.h>
#include <linux/wireless.h>
#include <net/iw_handler.h>
******************************************************************************/
#include <linux/sched.h>
+#include <linux/slab.h>
#include "ipw2200.h"
#include <linux/netdevice.h>
#include <linux/proc_fs.h>
#include <linux/skbuff.h>
-#include <linux/slab.h>
#include <linux/tcp.h>
#include <linux/types.h>
#include <linux/wireless.h>
#include <linux/errno.h>
#include <linux/if_arp.h>
#include <linux/in6.h>
+#include <linux/gfp.h>
#include <linux/in.h>
#include <linux/ip.h>
#include <linux/kernel.h>
#include <linux/netdevice.h>
#include <linux/proc_fs.h>
#include <linux/skbuff.h>
-#include <linux/slab.h>
#include <linux/tcp.h>
#include <linux/types.h>
#include <linux/wireless.h>
******************************************************************************/
#include <linux/kmod.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/jiffies.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <linux/wireless.h>
#include <net/mac80211.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/dma-mapping.h>
#include <linux/delay.h>
tx_resp->failure_frame);
freed = iwl_tx_queue_reclaim(priv, txq_id, index);
- if (qc && likely(sta_id != IWL_INVALID_STATION))
- priv->stations[sta_id].tid[tid].tfds_in_queue -= freed;
+ iwl_free_tfds_in_queue(priv, sta_id, tid, freed);
if (priv->mac80211_registered &&
(iwl_queue_space(&txq->q) > txq->q.low_mark))
iwl_wake_queue(priv, txq_id);
}
- if (qc && likely(sta_id != IWL_INVALID_STATION))
- iwl_txq_check_empty(priv, sta_id, tid, txq_id);
+ iwl_txq_check_empty(priv, sta_id, tid, txq_id);
if (iwl_check_bits(status, TX_ABORT_REQUIRED_MSK))
IWL_ERR(priv, "TODO: Implement Tx ABORT REQUIRED!!!\n");
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <linux/wireless.h>
#include <net/mac80211.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/dma-mapping.h>
#include <linux/delay.h>
#include <linux/sched.h>
/* Ack/clear/reset pending uCode interrupts.
* Note: Some bits in CSR_INT are "OR" of bits in CSR_FH_INT_STATUS,
*/
- iwl_write32(priv, CSR_INT, priv->inta);
+ /* There is a hardware bug in the interrupt mask function that some
+ * interrupts (i.e. CSR_INT_BIT_SCD) can still be generated even if
+ * they are disabled in the CSR_INT_MASK register. Furthermore the
+ * ICT interrupt handling mechanism has another bug that might cause
+ * these unmasked interrupts fail to be detected. We workaround the
+ * hardware bugs here by ACKing all the possible interrupts so that
+ * interrupt coalescing can still be achieved.
+ */
+ iwl_write32(priv, CSR_INT, priv->inta | ~priv->inta_mask);
inta = priv->inta;
BIT(NL80211_IFTYPE_STATION) |
BIT(NL80211_IFTYPE_ADHOC);
- hw->wiphy->flags |= WIPHY_FLAG_STRICT_REGULATORY |
+ hw->wiphy->flags |= WIPHY_FLAG_CUSTOM_REGULATORY |
WIPHY_FLAG_DISABLE_BEACON_HINTS;
/*
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*****************************************************************************/
+#include <linux/slab.h>
#include <net/mac80211.h>
#include "iwl-dev.h"
#include <linux/module.h>
#include <linux/etherdevice.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <net/mac80211.h>
#include "iwl-eeprom.h"
* Intel Corporation, 5200 N.E. Elam Young Parkway, Hillsboro, OR 97124-6497
*****************************************************************************/
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/debugfs.h>
/* sparse doesn't like tracepoint macros */
#ifndef __CHECKER__
+#include "iwl-dev.h"
+
#define CREATE_TRACE_POINTS
#include "iwl-devtrace.h"
#define __IWLWIFI_DEVICE_TRACE
#include <linux/tracepoint.h>
-#include "iwl-dev.h"
#if !defined(CONFIG_IWLWIFI_DEVICE_TRACING) || defined(__CHECKER__)
#undef TRACE_EVENT
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <net/mac80211.h>
#include <linux/io.h>
+#include "iwl-dev.h"
#include "iwl-debug.h"
#include "iwl-devtrace.h"
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <net/mac80211.h>
*****************************************************************************/
#include <linux/etherdevice.h>
+#include <linux/slab.h>
#include <net/mac80211.h>
#include <asm/unaligned.h>
#include "iwl-eeprom.h"
* Intel Linux Wireless <ilw@linux.intel.com>
* Intel Corporation, 5200 N.E. Elam Young Parkway, Hillsboro, OR 97124-6497
*****************************************************************************/
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/etherdevice.h>
#include <net/mac80211.h>
#include <linux/etherdevice.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <net/mac80211.h>
#include "iwl-eeprom.h"
#include "iwl-dev.h"
if (priv->stations[sta_id].tid[tid].tfds_in_queue >= freed)
priv->stations[sta_id].tid[tid].tfds_in_queue -= freed;
else {
- IWL_ERR(priv, "free more than tfds_in_queue (%u:%d)\n",
+ IWL_DEBUG_TX(priv, "free more than tfds_in_queue (%u:%d)\n",
priv->stations[sta_id].tid[tid].tfds_in_queue,
freed);
priv->stations[sta_id].tid[tid].tfds_in_queue = 0;
#include <linux/module.h>
#include <linux/init.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/dma-mapping.h>
#include <linux/delay.h>
#include <linux/sched.h>
{
int i;
- for (i = 0; i < IWL_RATE_COUNT; i++) {
+ for (i = 0; i < IWL_RATE_COUNT_LEGACY; i++) {
rates[i].bitrate = iwl3945_rates[i].ieee * 5;
rates[i].hw_value = i; /* Rate scaling will work on indexes */
rates[i].hw_value_short = i;
BIT(NL80211_IFTYPE_STATION) |
BIT(NL80211_IFTYPE_ADHOC);
- hw->wiphy->flags |= WIPHY_FLAG_STRICT_REGULATORY |
+ hw->wiphy->flags |= WIPHY_FLAG_CUSTOM_REGULATORY |
WIPHY_FLAG_DISABLE_BEACON_HINTS;
hw->wiphy->max_scan_ssids = PROBE_OPTION_MAX_3945;
#include <linux/etherdevice.h>
#include <linux/wireless.h>
#include <linux/ieee80211.h>
+#include <linux/slab.h>
#include <net/cfg80211.h>
#include "iwm.h"
#include <linux/etherdevice.h>
#include <linux/ieee80211.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include "iwm.h"
#include "bus.h"
*
*/
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/bitops.h>
#include <linux/debugfs.h>
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include "iwm.h"
#include "umac.h"
*/
#include <linux/kernel.h>
#include <linux/netdevice.h>
+#include <linux/slab.h>
#include "iwm.h"
#include "bus.h"
#include <linux/sched.h>
#include <linux/ieee80211.h>
#include <linux/wireless.h>
+#include <linux/slab.h>
#include "iwm.h"
#include "debug.h"
* -> sdio_disable_func()
*/
#include <linux/netdevice.h>
+#include <linux/slab.h>
#include "iwm.h"
#include "commands.h"
#include <linux/ieee80211.h>
#include <linux/if_arp.h>
#include <linux/list.h>
+#include <linux/slab.h>
#include <net/iw_handler.h>
#include "iwm.h"
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/netdevice.h>
#include <linux/debugfs.h>
#include <linux/mmc/sdio_ids.h>
* (i.e. half of the max size). [iwm_tx_worker]
*/
+#include <linux/slab.h>
#include <linux/skbuff.h>
#include <linux/netdevice.h>
#include <linux/ieee80211.h>
#include <linux/etherdevice.h>
#include <linux/ieee80211.h>
#include <linux/if_arp.h>
+#include <linux/slab.h>
#include <net/lib80211.h>
#include "assoc.h"
*
*/
+#include <linux/slab.h>
#include <net/cfg80211.h>
#include "cfg.h"
if (ret < 0)
lbs_pr_err("cannot register wiphy device\n");
+ priv->wiphy_registered = true;
+
ret = register_netdev(priv->dev);
if (ret)
lbs_pr_err("cannot register network device\n");
if (!wdev)
return;
- if (wdev->wiphy) {
+ if (priv->wiphy_registered)
wiphy_unregister(wdev->wiphy);
+
+ if (wdev->wiphy)
wiphy_free(wdev->wiphy);
- }
+
kfree(wdev);
}
#include <linux/kfifo.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include "host.h"
#include "decl.h"
* This file contains the handling of command
* responses as well as events generated by firmware.
*/
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/sched.h>
#include <linux/if_arp.h>
#include <linux/delay.h>
#include <linux/mm.h>
#include <linux/string.h>
+#include <linux/slab.h>
#include <net/iw_handler.h>
#include <net/lib80211.h>
/* CFG80211 */
struct wireless_dev *wdev;
+ bool wiphy_registered;
/* Mesh */
struct net_device *mesh_dev; /* Virtual device */
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/moduleparam.h>
#include <linux/firmware.h>
#include <linux/kernel.h>
#include <linux/moduleparam.h>
+#include <linux/slab.h>
#include <linux/firmware.h>
#include <linux/netdevice.h>
#include <linux/delay.h>
#include <linux/list.h>
#include <linux/netdevice.h>
#include <linux/semaphore.h>
+#include <linux/slab.h>
#include <linux/spi/libertas_spi.h>
#include <linux/spi/spi.h>
#include <linux/moduleparam.h>
#include <linux/firmware.h>
#include <linux/netdevice.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#ifdef CONFIG_OLPC
#include <linux/kfifo.h>
#include <linux/stddef.h>
#include <linux/ieee80211.h>
+#include <linux/slab.h>
#include <net/iw_handler.h>
#include <net/cfg80211.h>
* This file contains the handling of RX in wlan driver.
*/
#include <linux/etherdevice.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include "host.h"
* IOCTL handlers as well as command preperation and response routines
* for sending scan commands to the firmware.
*/
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/etherdevice.h>
* This file contains ioctl functions
*/
#include <linux/ctype.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/if.h>
#include <linux/if_arp.h>
* the Free Software Foundation; either version 2 of the License, or (at
* your option) any later version.
*/
+#include <linux/slab.h>
+
#include "libertas_tf.h"
static const struct channel_range channel_ranges[] = {
#include <linux/moduleparam.h>
#include <linux/firmware.h>
#include <linux/netdevice.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#define DRV_NAME "lbtf_usb"
* the Free Software Foundation; either version 2 of the License, or (at
* your option) any later version.
*/
+#include <linux/slab.h>
+
#include "libertas_tf.h"
#include "linux/etherdevice.h"
*/
#include <linux/list.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <net/dst.h>
#include <net/xfrm.h>
#include <linux/delay.h>
#include <linux/completion.h>
#include <linux/etherdevice.h>
+#include <linux/slab.h>
#include <net/mac80211.h>
#include <linux/moduleparam.h>
#include <linux/firmware.h>
MODULE_FIRMWARE("mwl8k/fmimage_8366.fw");
static DEFINE_PCI_DEVICE_TABLE(mwl8k_pci_id_table) = {
+ { PCI_VDEVICE(MARVELL, 0x2a0a), .driver_data = MWL8363, },
{ PCI_VDEVICE(MARVELL, 0x2a0c), .driver_data = MWL8363, },
{ PCI_VDEVICE(MARVELL, 0x2a24), .driver_data = MWL8363, },
{ PCI_VDEVICE(MARVELL, 0x2a2b), .driver_data = MWL8687, },
* See copyright notice in main.c
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/firmware.h>
#include <linux/device.h>
#include <linux/module.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/delay.h>
#include <linux/device.h>
* See copyright notice in main.c
*/
+#include <linux/gfp.h>
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/ieee80211.h>
*
* See copyright notice in main.c
*/
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/if_arp.h>
#include <linux/wireless.h>
#include <linux/firmware.h>
#include <linux/etherdevice.h>
#include <linux/sort.h>
+#include <linux/slab.h>
#include <net/mac80211.h>
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/firmware.h>
#include <linux/etherdevice.h>
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/firmware.h>
#include <linux/etherdevice.h>
#include <linux/init.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/firmware.h>
#include <linux/etherdevice.h>
#include <linux/delay.h>
#include <linux/spi/spi.h>
#include <linux/etherdevice.h>
#include <linux/gpio.h>
+#include <linux/slab.h>
#include "p54spi.h"
#include "p54spi_eeprom.h"
#include <linux/init.h>
#include <linux/usb.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/firmware.h>
#include <linux/etherdevice.h>
#include <linux/delay.h>
static struct usb_device_id p54u_table[] __devinitdata = {
/* Version 1 devices (pci chip + net2280) */
{USB_DEVICE(0x0506, 0x0a11)}, /* 3COM 3CRWE254G72 */
+ {USB_DEVICE(0x06b9, 0x0120)}, /* Thomson SpeedTouch 120g */
{USB_DEVICE(0x0707, 0xee06)}, /* SMC 2862W-G */
{USB_DEVICE(0x07aa, 0x001c)}, /* Corega CG-WLUSB2GT */
{USB_DEVICE(0x083a, 0x4501)}, /* Accton 802.11g WN4501 USB */
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/if_arp.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <asm/uaccess.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/netdevice.h>
#include <linux/ethtool.h>
*/
#include <linux/module.h>
+#include <linux/gfp.h>
#include <linux/pci.h>
#include <linux/delay.h>
#include <linux/module.h>
#include <linux/pci.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/system.h>
#include <linux/wireless.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
/*
* Function definitions
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include "prismcompat.h"
#include "islpci_dev.h"
#include <linux/proc_fs.h>
#include <linux/ptrace.h>
#include <linux/seq_file.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/timer.h>
#include <linux/init.h>
#include <linux/if_arp.h>
#include <linux/ctype.h>
#include <linux/spinlock.h>
+#include <linux/slab.h>
#include <net/iw_handler.h>
#include <net/cfg80211.h>
#include <linux/usb/usbnet.h>
#include <linux/module.h>
#include <linux/pci.h>
#include <linux/eeprom_93cx6.h>
+#include <linux/slab.h>
#include "rt2x00.h"
#include "rt2x00pci.h"
#include <linux/module.h>
#include <linux/pci.h>
#include <linux/eeprom_93cx6.h>
+#include <linux/slab.h>
#include "rt2x00.h"
#include "rt2x00pci.h"
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include "rt2x00.h"
char *tx_power;
unsigned int i;
+ /*
+ * Disable powersaving as default.
+ */
+ rt2x00dev->hw->wiphy->flags &= ~WIPHY_FLAG_PS_ON_BY_DEFAULT;
+
/*
* Initialize all hw fields.
*/
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include "rt2x00.h"
#if defined(CONFIG_RT2X00_LIB_USB) || defined(CONFIG_RT2X00_LIB_USB_MODULE)
rt2800_rfcsr_write(rt2x00dev, 24,
rt2x00dev->calibration[conf_is_ht40(conf)]);
- rt2800_rfcsr_read(rt2x00dev, 23, &rfcsr);
+ rt2800_rfcsr_read(rt2x00dev, 7, &rfcsr);
rt2x00_set_field8(&rfcsr, RFCSR7_RF_TUNING, 1);
- rt2800_rfcsr_write(rt2x00dev, 23, rfcsr);
+ rt2800_rfcsr_write(rt2x00dev, 7, rfcsr);
}
static void rt2800_config_channel(struct rt2x00_dev *rt2x00dev,
#include <linux/module.h>
#include <linux/poll.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/uaccess.h>
#include "rt2x00.h"
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include "rt2x00.h"
#include "rt2x00lib.h"
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include "rt2x00.h"
#include "rt2x00pci.h"
Abstract: rt2x00 queue specific routines.
*/
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/dma-mapping.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include "rt2x00.h"
#include "rt2x00soc.h"
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include <linux/bug.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/eeprom_93cx6.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include "rt2x00.h"
#include <linux/init.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/etherdevice.h>
#include <linux/eeprom_93cx6.h>
#include <linux/init.h>
#include <linux/usb.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/etherdevice.h>
#include <linux/eeprom_93cx6.h>
#include "wl1251_acx.h"
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/crc7.h>
#include "wl1251.h"
*/
#include <linux/gpio.h>
+#include <linux/slab.h>
#include "wl1251_reg.h"
#include "wl1251_boot.h"
#include "wl1251_cmd.h"
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/crc7.h>
#include "wl1251.h"
#include "wl1251_debugfs.h"
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include "wl1251.h"
#include "wl1251_acx.h"
void wl1251_debugfs_reset(struct wl1251 *wl)
{
- memset(wl->stats.fw_stats, 0, sizeof(*wl->stats.fw_stats));
+ if (wl->stats.fw_stats != NULL)
+ memset(wl->stats.fw_stats, 0, sizeof(*wl->stats.fw_stats));
wl->stats.retry_count = 0;
wl->stats.excessive_retries = 0;
}
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include "wl1251_init.h"
#include "wl12xx_80211.h"
#include <linux/crc32.h>
#include <linux/etherdevice.h>
#include <linux/vmalloc.h>
+#include <linux/slab.h>
#include "wl1251.h"
#include "wl12xx_80211.h"
*/
#include <linux/skbuff.h>
+#include <linux/gfp.h>
#include <net/mac80211.h>
#include "wl1251.h"
#include <linux/irq.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/crc7.h>
#include <linux/spi/spi.h>
#include <linux/spi/wl12xx.h>
#include <linux/platform_device.h>
#include <linux/crc7.h>
#include <linux/spi/spi.h>
+#include <linux/slab.h>
#include "wl1271.h"
#include "wl12xx_80211.h"
*/
#include <linux/gpio.h>
+#include <linux/slab.h>
#include "wl1271_acx.h"
#include "wl1271_reg.h"
#include <linux/crc7.h>
#include <linux/spi/spi.h>
#include <linux/etherdevice.h>
+#include <linux/slab.h>
#include "wl1271.h"
#include "wl1271_reg.h"
#include "wl1271_debugfs.h"
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include "wl1271.h"
#include "wl1271_acx.h"
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include "wl1271_init.h"
#include "wl12xx_80211.h"
#include <linux/vmalloc.h>
#include <linux/spi/wl12xx.h>
#include <linux/inetdevice.h>
+#include <linux/slab.h>
#include "wl1271.h"
#include "wl12xx_80211.h"
*
*/
+#include <linux/gfp.h>
+
#include "wl1271.h"
#include "wl1271_acx.h"
#include "wl1271_reg.h"
#include <linux/platform_device.h>
#include <linux/crc7.h>
#include <linux/spi/spi.h>
+#include <linux/slab.h>
#include "wl1271.h"
#include "wl12xx_80211.h"
*/
#include "wl1271_testmode.h"
+#include <linux/slab.h>
#include <net/genetlink.h>
#include "wl1271.h"
#include <linux/module.h>
#include <linux/usb.h>
+#include <linux/slab.h>
#include <linux/netdevice.h>
#include <linux/etherdevice.h>
#include <linux/wireless.h>
#include <linux/kernel.h>
#include <linux/errno.h>
+#include <linux/slab.h>
#include "zd_def.h"
#include "zd_chip.h"
#include <linux/netdevice.h>
#include <linux/etherdevice.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include <linux/jiffies.h>
#include <net/ieee80211_radiotap.h>
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include "zd_rf.h"
#include "zd_usb.h"
#include <linux/firmware.h>
#include <linux/device.h>
#include <linux/errno.h>
+#include <linux/slab.h>
#include <linux/skbuff.h>
#include <linux/usb.h>
#include <linux/workqueue.h>
#include <linux/udp.h>
#include <linux/moduleparam.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include <net/ip.h>
#include <xen/xen.h>
#include <linux/etherdevice.h>
#include <linux/skbuff.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <linux/of_device.h>
#include <linux/of_platform.h>
#include <linux/module.h>
#include <linux/types.h>
#include <linux/fcntl.h>
+#include <linux/gfp.h>
#include <linux/interrupt.h>
#include <linux/init.h>
#include <linux/ioport.h>
#include <linux/in.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/delay.h>
#include <linux/errno.h>
#include <linux/skbuff.h>
#include <linux/platform_device.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/pgtable.h>
#include <linux/timer.h>
#include <linux/errno.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/string.h>
+#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/interrupt.h>
#include <linux/ioport.h>
#include <linux/init.h>
#include <linux/delay.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <asm/setup.h>
#include <asm/system.h>
#include <asm/page.h>
#include <linux/module.h>
#include <linux/of.h>
#include <linux/spinlock.h>
+#include <linux/slab.h>
#include <linux/proc_fs.h>
struct device_node *allnodes;
if (!np->type)
np->type = "<NULL>";
}
- while (tag == OF_DT_BEGIN_NODE) {
- mem = unflatten_dt_node(mem, p, np, allnextpp, fpsize);
+ while (tag == OF_DT_BEGIN_NODE || tag == OF_DT_NOP) {
+ if (tag == OF_DT_NOP)
+ *p += 4;
+ else
+ mem = unflatten_dt_node(mem, p, np, allnextpp, fpsize);
tag = be32_to_cpup((__be32 *)(*p));
}
if (tag != OF_DT_END_NODE) {
#include <linux/errno.h>
#include <linux/io.h>
#include <linux/of.h>
+#include <linux/slab.h>
#include <linux/of_gpio.h>
#include <asm/prom.h>
#include <linux/fs.h>
#include <linux/oprofile.h>
#include <linux/sched.h>
+#include <linux/gfp.h>
#include "oprofile_stats.h"
#include "event_buffer.h"
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/module.h>
-#include <linux/slab.h>
#include <linux/types.h>
#include <asm/io.h>
#include <asm/led.h>
#include <linux/mm.h>
#include <linux/string.h>
#include <linux/pci.h>
+#include <linux/gfp.h>
#include <asm/uaccess.h>
#include <linux/interrupt.h>
#include <linux/ioport.h>
#include <linux/module.h>
-#include <linux/slab.h>
#include <linux/types.h>
#include <asm/hardware.h>
#include <linux/module.h>
#include <linux/parport.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#include <asm/current.h>
#include <linux/interrupt.h>
#include <linux/errno.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/irq.h>
#include <linux/module.h>
#include <linux/parport.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/stddef.h>
#include <linux/types.h>
#include <linux/types.h>
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/interrupt.h>
#include <linux/parport.h>
#include <linux/parport.h>
#include <linux/ctype.h>
#include <linux/string.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
static const struct {
#include <linux/pci.h>
#include <linux/module.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/ioport.h>
#include <linux/wait.h>
#include <linux/ioport.h>
#include <linux/proc_fs.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include "pci.h"
#include <linux/interrupt.h>
#include <linux/tboot.h>
#include <linux/dmi.h>
+#include <linux/slab.h>
#define PREFIX "DMAR: "
#include <linux/pci_hotplug.h>
#include <linux/acpi.h>
#include <linux/pci-acpi.h>
+#include <linux/slab.h>
#define MY_NAME "acpi_pcihp"
#include <linux/pci_hotplug.h>
#include <linux/pci-acpi.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include "../pci.h"
#include "acpiphp.h"
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <acpi/acpi_bus.h>
#include <linux/module.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/proc_fs.h>
#include <linux/workqueue.h>
#include <linux/init.h>
#include <linux/pci.h>
#include <linux/device.h>
+#include <linux/slab.h>
#include "../pci.h"
struct legacy_slot {
#include <linux/kobject.h>
#include <linux/sysfs.h>
#include <linux/pagemap.h>
-#include <linux/slab.h>
#include <linux/init.h>
#include <linux/mount.h>
#include <linux/namei.h>
#include <linux/acpi.h>
#include <linux/pci.h>
#include <linux/pci_hotplug.h>
+#include <linux/slab.h>
#include "pciehp.h"
#define PCIEHP_DETECT_PCIE (0)
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/pci.h>
#include "pciehp.h"
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/workqueue.h>
#include "../pci.h"
#include <linux/pci.h>
#include <linux/interrupt.h>
#include <linux/time.h>
+#include <linux/slab.h>
#include "../pci.h"
#include "pciehp.h"
for (i = 0; i < DEVICE_COUNT_RESOURCE; i++) {
if (!pci_resource_len(pdev, i))
continue;
- ctrl_info(ctrl, " PCI resource [%d] : 0x%llx@0x%llx\n",
- i, (unsigned long long)pci_resource_len(pdev, i),
- (unsigned long long)pci_resource_start(pdev, i));
+ ctrl_info(ctrl, " PCI resource [%d] : %pR\n",
+ i, &pdev->resource[i]);
}
ctrl_info(ctrl, "Slot Capabilities : 0x%08x\n", ctrl->slot_cap);
ctrl_info(ctrl, " Physical Slot Number : %d\n", PSN(ctrl));
#include <linux/moduleparam.h>
#include <linux/pci.h>
#include <linux/pci_hotplug.h>
-#include <linux/slab.h>
#include <linux/smp.h>
#include <linux/init.h>
#include <asm/eeh.h> /* for eeh_add_device() */
#include <linux/pci.h>
#include <linux/pci_hotplug.h>
#include <linux/proc_fs.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/mutex.h>
#include <linux/moduleparam.h>
#include <linux/kernel.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/workqueue.h>
#include "shpchp.h"
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/workqueue.h>
#include "../pci.h"
#include <linux/pci.h>
#include <linux/spinlock.h>
#include <linux/slab.h>
-#include <linux/gfp.h>
#include <linux/htirq.h>
/* Global ht irq lock.
#include <linux/interrupt.h>
#include <linux/dmar.h>
#include <linux/spinlock.h>
+#include <linux/slab.h>
#include <linux/jiffies.h>
#include <linux/hpet.h>
#include <linux/pci.h>
#include <linux/pci.h>
#include <linux/acpi.h>
+#include <linux/slab.h>
#include <acpi/acpi_bus.h>
struct ioapic {
acpi_status status;
unsigned long long gsb;
struct ioapic *ioapic;
- u64 addr;
int ret;
char *type;
+ struct resource *res;
handle = DEVICE_ACPI_HANDLE(&dev->dev);
if (!handle)
if (pci_request_region(dev, 0, type))
goto exit_disable;
- addr = pci_resource_start(dev, 0);
- if (acpi_register_ioapic(ioapic->handle, addr, ioapic->gsi_base))
+ res = &dev->resource[0];
+ if (acpi_register_ioapic(ioapic->handle, res->start, ioapic->gsi_base))
goto exit_release;
pci_set_drvdata(dev, ioapic);
- dev_info(&dev->dev, "%s at %#llx, GSI %u\n", type, addr,
- ioapic->gsi_base);
+ dev_info(&dev->dev, "%s at %pR, GSI %u\n", type, res, ioapic->gsi_base);
return 0;
exit_release:
*/
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/mutex.h>
#include <linux/string.h>
#include <linux/delay.h>
#include <linux/smp.h>
#include <linux/errno.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include "pci.h"
#include "msi.h"
#include <linux/mm.h>
#include <linux/capability.h>
#include <linux/pci-aspm.h>
+#include <linux/slab.h>
#include "pci.h"
static int sysfs_initialized; /* = 0 */
goto legacy_io_err;
/* Allocated above after the legacy_io struct */
- sysfs_bin_attr_init(b->legacy_mem);
b->legacy_mem = b->legacy_io + 1;
+ sysfs_bin_attr_init(b->legacy_mem);
b->legacy_mem->attr.name = "legacy_mem";
b->legacy_mem->size = 1024*1024;
b->legacy_mem->attr.mode = S_IRUSR | S_IWUSR;
#include <linux/init.h>
#include <linux/pci.h>
#include <linux/pm.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/spinlock.h>
#include <linux/string.h>
*/
int pcix_get_max_mmrbc(struct pci_dev *dev)
{
- int err, cap;
+ int cap;
u32 stat;
cap = pci_find_capability(dev, PCI_CAP_ID_PCIX);
if (!cap)
return -EINVAL;
- err = pci_read_config_dword(dev, cap + PCI_X_STATUS, &stat);
- if (err)
+ if (pci_read_config_dword(dev, cap + PCI_X_STATUS, &stat))
return -EINVAL;
- return (stat & PCI_X_STATUS_MAX_READ) >> 12;
+ return 512 << ((stat & PCI_X_STATUS_MAX_READ) >> 21);
}
EXPORT_SYMBOL(pcix_get_max_mmrbc);
*/
int pcix_get_mmrbc(struct pci_dev *dev)
{
- int ret, cap;
- u32 cmd;
+ int cap;
+ u16 cmd;
cap = pci_find_capability(dev, PCI_CAP_ID_PCIX);
if (!cap)
return -EINVAL;
- ret = pci_read_config_dword(dev, cap + PCI_X_CMD, &cmd);
- if (!ret)
- ret = 512 << ((cmd & PCI_X_CMD_MAX_READ) >> 2);
+ if (pci_read_config_word(dev, cap + PCI_X_CMD, &cmd))
+ return -EINVAL;
- return ret;
+ return 512 << ((cmd & PCI_X_CMD_MAX_READ) >> 2);
}
EXPORT_SYMBOL(pcix_get_mmrbc);
*/
int pcix_set_mmrbc(struct pci_dev *dev, int mmrbc)
{
- int cap, err = -EINVAL;
- u32 stat, cmd, v, o;
+ int cap;
+ u32 stat, v, o;
+ u16 cmd;
if (mmrbc < 512 || mmrbc > 4096 || !is_power_of_2(mmrbc))
- goto out;
+ return -EINVAL;
v = ffs(mmrbc) - 10;
cap = pci_find_capability(dev, PCI_CAP_ID_PCIX);
if (!cap)
- goto out;
+ return -EINVAL;
- err = pci_read_config_dword(dev, cap + PCI_X_STATUS, &stat);
- if (err)
- goto out;
+ if (pci_read_config_dword(dev, cap + PCI_X_STATUS, &stat))
+ return -EINVAL;
if (v > (stat & PCI_X_STATUS_MAX_READ) >> 21)
return -E2BIG;
- err = pci_read_config_dword(dev, cap + PCI_X_CMD, &cmd);
- if (err)
- goto out;
+ if (pci_read_config_word(dev, cap + PCI_X_CMD, &cmd))
+ return -EINVAL;
o = (cmd & PCI_X_CMD_MAX_READ) >> 2;
if (o != v) {
cmd &= ~PCI_X_CMD_MAX_READ;
cmd |= v << 2;
- err = pci_write_config_dword(dev, cap + PCI_X_CMD, cmd);
+ if (pci_write_config_word(dev, cap + PCI_X_CMD, cmd))
+ return -EIO;
}
-out:
- return err;
+ return 0;
}
EXPORT_SYMBOL(pcix_set_mmrbc);
EXPORT_SYMBOL(pci_disable_device);
EXPORT_SYMBOL(pci_find_capability);
EXPORT_SYMBOL(pci_bus_find_capability);
-EXPORT_SYMBOL(pci_register_set_vga_state);
EXPORT_SYMBOL(pci_release_regions);
EXPORT_SYMBOL(pci_request_regions);
EXPORT_SYMBOL(pci_request_regions_exclusive);
#include <linux/init.h>
#include <linux/miscdevice.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/fs.h>
#include <linux/uaccess.h>
#include <linux/stddef.h>
#include <linux/interrupt.h>
#include <linux/delay.h>
#include <linux/pcieport_if.h>
+#include <linux/slab.h>
#include "aerdrv.h"
#include "../../pci.h"
#include <linux/pm.h>
#include <linux/suspend.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include "aerdrv.h"
static int forceload;
#include <linux/pci.h>
#include <linux/kernel.h>
#include <linux/errno.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/device.h>
#include <linux/errno.h>
#include <linux/pm.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/pcieport_if.h>
#include <linux/aer.h>
#include <linux/dmi.h>
pci_read_config_dword(dev, pos, &sz);
pci_write_config_dword(dev, pos, l);
+ if (!sz)
+ goto fail; /* BAR not implemented */
+
/*
* All bits set in sz means the device isn't working properly.
- * If the BAR isn't implemented, all bits must be 0. If it's a
- * memory BAR or a ROM, bit 0 must be clear; if it's an io BAR, bit
- * 1 must be clear.
+ * If it's a memory BAR or a ROM, bit 0 must be clear; if it's
+ * an io BAR, bit 1 must be clear.
*/
- if (!sz || sz == 0xffffffff)
+ if (sz == 0xffffffff) {
+ dev_err(&dev->dev, "reg %x: invalid size %#x; broken device?\n",
+ pos, sz);
goto fail;
+ }
/*
* I don't know how l can have all bits set. Copied from old code.
pos, res);
}
} else {
- sz = pci_size(l, sz, mask);
+ u32 size = pci_size(l, sz, mask);
- if (!sz)
+ if (!size) {
+ dev_err(&dev->dev, "reg %x: invalid size "
+ "(l %#x sz %#x mask %#x); broken device?",
+ pos, l, sz, mask);
goto fail;
+ }
res->start = l;
- res->end = l + sz;
+ res->end = l + size;
dev_printk(KERN_DEBUG, &dev->dev, "reg %x: %pR\n", pos, res);
}
dev_printk(KERN_DEBUG, &dev->dev, " bridge window %pR\n", res);
} else {
dev_printk(KERN_DEBUG, &dev->dev,
- " bridge window [io %04lx - %04lx] reg reading\n",
+ " bridge window [io %#06lx-%#06lx] (disabled)\n",
base, limit);
}
}
dev_printk(KERN_DEBUG, &dev->dev, " bridge window %pR\n", res);
} else {
dev_printk(KERN_DEBUG, &dev->dev,
- " bridge window [mem 0x%08lx - 0x%08lx] reg reading\n",
+ " bridge window [mem %#010lx-%#010lx] (disabled)\n",
base, limit + 0xfffff);
}
}
dev_printk(KERN_DEBUG, &dev->dev, " bridge window %pR\n", res);
} else {
dev_printk(KERN_DEBUG, &dev->dev,
- " bridge window [mem 0x%08lx - %08lx pref] reg reading\n",
+ " bridge window [mem %#010lx-%#010lx pref] (disabled)\n",
base, limit + 0xfffff);
}
}
int is_cardbus = (dev->hdr_type == PCI_HEADER_TYPE_CARDBUS);
u32 buses, i, j = 0;
u16 bctl;
+ u8 primary, secondary, subordinate;
int broken = 0;
pci_read_config_dword(dev, PCI_PRIMARY_BUS, &buses);
+ primary = buses & 0xFF;
+ secondary = (buses >> 8) & 0xFF;
+ subordinate = (buses >> 16) & 0xFF;
- dev_dbg(&dev->dev, "scanning behind bridge, config %06x, pass %d\n",
- buses & 0xffffff, pass);
+ dev_dbg(&dev->dev, "scanning [bus %02x-%02x] behind bridge, pass %d\n",
+ secondary, subordinate, pass);
/* Check if setup is sensible at all */
if (!pass &&
- ((buses & 0xff) != bus->number || ((buses >> 8) & 0xff) <= bus->number)) {
+ (primary != bus->number || secondary <= bus->number)) {
dev_dbg(&dev->dev, "bus configuration invalid, reconfiguring\n");
broken = 1;
}
pci_write_config_word(dev, PCI_BRIDGE_CONTROL,
bctl & ~PCI_BRIDGE_CTL_MASTER_ABORT);
- if ((buses & 0xffff00) && !pcibios_assign_all_busses() && !is_cardbus && !broken) {
- unsigned int cmax, busnr;
+ if ((secondary || subordinate) && !pcibios_assign_all_busses() &&
+ !is_cardbus && !broken) {
+ unsigned int cmax;
/*
* Bus already configured by firmware, process it in the first
* pass and just note the configuration.
*/
if (pass)
goto out;
- busnr = (buses >> 8) & 0xFF;
/*
* If we already got to this bus through a different bridge,
* However, we continue to descend down the hierarchy and
* scan remaining child buses.
*/
- child = pci_find_bus(pci_domain_nr(bus), busnr);
+ child = pci_find_bus(pci_domain_nr(bus), secondary);
if (!child) {
- child = pci_add_new_bus(bus, dev, busnr);
+ child = pci_add_new_bus(bus, dev, secondary);
if (!child)
goto out;
- child->primary = buses & 0xFF;
- child->subordinate = (buses >> 16) & 0xFF;
+ child->primary = primary;
+ child->subordinate = subordinate;
child->bridge_ctl = bctl;
}
#include <linux/init.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
bus_region.end = res->end;
pcibios_bus_to_resource(dev, res, &bus_region);
- pci_claim_resource(dev, nr);
- dev_info(&dev->dev, "quirk: %pR claimed by %s\n", res, name);
+ if (pci_claim_resource(dev, nr) == 0)
+ dev_info(&dev->dev, "quirk: %pR claimed by %s\n",
+ res, name);
}
}
/*
* Disable PCI Bus Parking and PCI Master read caching on CX700
* which causes unspecified timing errors with a VT6212L on the PCI
- * bus leading to USB2.0 packet loss. The defaults are that these
- * features are turned off but some BIOSes turn them on.
+ * bus leading to USB2.0 packet loss.
+ *
+ * This quirk is only enabled if a second (on the external PCI bus)
+ * VT6212L is found -- the CX700 core itself also contains a USB
+ * host controller with the same PCI ID as the VT6212L.
*/
+ /* Count VT6212L instances */
+ struct pci_dev *p = pci_get_device(PCI_VENDOR_ID_VIA,
+ PCI_DEVICE_ID_VIA_8235_USB_2, NULL);
uint8_t b;
+
+ /* p should contain the first (internal) VT6212L -- see if we have
+ an external one by searching again */
+ p = pci_get_device(PCI_VENDOR_ID_VIA, PCI_DEVICE_ID_VIA_8235_USB_2, p);
+ if (!p)
+ return;
+ pci_dev_put(p);
+
if (pci_read_config_byte(dev, 0x76, &b) == 0) {
if (b & 0x40) {
/* Turn off PCI Bus Parking */
}
}
}
-DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_VIA, 0x324e, quirk_via_cx700_pci_parking_caching);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_VIA, 0x324e, quirk_via_cx700_pci_parking_caching);
/*
* For Broadcom 5706, 5708, 5709 rev. A nics, any read beyond the
}
}
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_AMD, PCI_DEVICE_ID_AMD_8131_BRIDGE, quirk_disable_msi);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_AMD, 0x9602, quirk_disable_msi);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_ASUSTEK, 0x9602, quirk_disable_msi);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_AI, 0x9602, quirk_disable_msi);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_VIA, 0xa238, quirk_disable_msi);
/* Go through the list of Hypertransport capabilities and
* return 1 if a HT MSI capability is found and enabled */
#include <linux/init.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/interrupt.h>
#include "pci.h"
int pci_claim_resource(struct pci_dev *dev, int resource)
{
struct resource *res = &dev->resource[resource];
- struct resource *root;
- int err;
+ struct resource *root, *conflict;
root = pci_find_parent_resource(dev, res);
if (!root) {
return -EINVAL;
}
- err = request_resource(root, res);
- if (err)
+ conflict = request_resource_conflict(root, res);
+ if (conflict) {
dev_err(&dev->dev,
- "address space collision: %pR already in use\n", res);
+ "address space collision: %pR conflicts with %s %pR\n",
+ res, conflict->name, conflict);
+ return -EBUSY;
+ }
- return err;
+ return 0;
}
EXPORT_SYMBOL(pci_claim_resource);
*/
#include <linux/kobject.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/err.h>
#include "pci.h"
#include <linux/errno.h>
#include <linux/init.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <pcmcia/ss.h>
struct at91_cf_socket *cf = platform_get_drvdata(pdev);
struct at91_cf_data *board = cf->board;
- pcmcia_socket_dev_suspend(&pdev->dev);
if (device_may_wakeup(&pdev->dev)) {
enable_irq_wake(board->det_pin);
if (board->irq_pin)
disable_irq_wake(board->irq_pin);
}
- pcmcia_socket_dev_resume(&pdev->dev);
return 0;
}
#include <linux/spinlock.h>
#include <linux/mutex.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/irq.h>
return ret;
}
-static int au1x00_drv_pcmcia_suspend(struct platform_device *dev,
- pm_message_t state)
-{
- return pcmcia_socket_dev_suspend(&dev->dev);
-}
-
-static int au1x00_drv_pcmcia_resume(struct platform_device *dev)
-{
- return pcmcia_socket_dev_resume(&dev->dev);
-}
-
static struct platform_driver au1x00_pcmcia_driver = {
.driver = {
.name = "au1x00-pcmcia",
},
.probe = au1x00_drv_pcmcia_probe,
.remove = au1x00_drv_pcmcia_remove,
- .suspend = au1x00_drv_pcmcia_suspend,
- .resume = au1x00_drv_pcmcia_resume,
};
#include <linux/ioport.h>
#include <linux/timer.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/pci.h>
#include <linux/gpio.h>
#include <linux/platform_device.h>
#include <linux/errno.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/irq.h>
return 0;
}
-static int bfin_cf_suspend(struct platform_device *pdev, pm_message_t mesg)
-{
- return pcmcia_socket_dev_suspend(&pdev->dev);
-}
-
-static int bfin_cf_resume(struct platform_device *pdev)
-{
- return pcmcia_socket_dev_resume(&pdev->dev);
-}
-
static struct platform_driver bfin_cf_driver = {
.driver = {
.name = (char *)driver_name,
},
.probe = bfin_cf_probe,
.remove = __devexit_p(bfin_cf_remove),
- .suspend = bfin_cf_suspend,
- .resume = bfin_cf_resume,
};
static int __init bfin_cf_init(void)
EXPORT_SYMBOL(pcmcia_socket_list_rwsem);
-/*
- * Low-level PCMCIA socket drivers need to register with the PCCard
- * core using pcmcia_register_socket.
- *
- * socket drivers are expected to use the following callbacks in their
- * .drv struct:
- * - pcmcia_socket_dev_suspend
- * - pcmcia_socket_dev_resume
- * These functions check for the appropriate struct pcmcia_soket arrays,
- * and pass them to the low-level functions pcmcia_{suspend,resume}_socket
- */
-static int socket_early_resume(struct pcmcia_socket *skt);
-static int socket_late_resume(struct pcmcia_socket *skt);
-static int socket_resume(struct pcmcia_socket *skt);
-static int socket_suspend(struct pcmcia_socket *skt);
-
-static void pcmcia_socket_dev_run(struct device *dev,
- int (*cb)(struct pcmcia_socket *))
-{
- struct pcmcia_socket *socket;
-
- down_read(&pcmcia_socket_list_rwsem);
- list_for_each_entry(socket, &pcmcia_socket_list, socket_list) {
- if (socket->dev.parent != dev)
- continue;
- mutex_lock(&socket->skt_mutex);
- cb(socket);
- mutex_unlock(&socket->skt_mutex);
- }
- up_read(&pcmcia_socket_list_rwsem);
-}
-
-int pcmcia_socket_dev_suspend(struct device *dev)
-{
- pcmcia_socket_dev_run(dev, socket_suspend);
- return 0;
-}
-EXPORT_SYMBOL(pcmcia_socket_dev_suspend);
-
-void pcmcia_socket_dev_early_resume(struct device *dev)
-{
- pcmcia_socket_dev_run(dev, socket_early_resume);
-}
-EXPORT_SYMBOL(pcmcia_socket_dev_early_resume);
-
-void pcmcia_socket_dev_late_resume(struct device *dev)
-{
- pcmcia_socket_dev_run(dev, socket_late_resume);
-}
-EXPORT_SYMBOL(pcmcia_socket_dev_late_resume);
-
-int pcmcia_socket_dev_resume(struct device *dev)
-{
- pcmcia_socket_dev_run(dev, socket_resume);
- return 0;
-}
-EXPORT_SYMBOL(pcmcia_socket_dev_resume);
-
-
struct pcmcia_socket *pcmcia_get_socket(struct pcmcia_socket *skt)
{
struct device *dev = get_device(&skt->dev);
static int socket_late_resume(struct pcmcia_socket *skt)
{
+ int ret;
+
mutex_lock(&skt->ops_mutex);
skt->state &= ~SOCKET_SUSPEND;
mutex_unlock(&skt->ops_mutex);
- if (!(skt->state & SOCKET_PRESENT))
- return socket_insert(skt);
+ if (!(skt->state & SOCKET_PRESENT)) {
+ ret = socket_insert(skt);
+ if (ret == -ENODEV)
+ ret = 0;
+ return ret;
+ }
if (skt->resume_status) {
socket_shutdown(skt);
}
+#ifdef CONFIG_PM
+
+static int __pcmcia_pm_op(struct device *dev,
+ int (*callback) (struct pcmcia_socket *skt))
+{
+ struct pcmcia_socket *s = container_of(dev, struct pcmcia_socket, dev);
+ int ret;
+
+ mutex_lock(&s->skt_mutex);
+ ret = callback(s);
+ mutex_unlock(&s->skt_mutex);
+
+ return ret;
+}
+
+static int pcmcia_socket_dev_suspend_noirq(struct device *dev)
+{
+ return __pcmcia_pm_op(dev, socket_suspend);
+}
+
+static int pcmcia_socket_dev_resume_noirq(struct device *dev)
+{
+ return __pcmcia_pm_op(dev, socket_early_resume);
+}
+
+static int pcmcia_socket_dev_resume(struct device *dev)
+{
+ return __pcmcia_pm_op(dev, socket_late_resume);
+}
+
+static const struct dev_pm_ops pcmcia_socket_pm_ops = {
+ /* dev_resume may be called with IRQs enabled */
+ SET_SYSTEM_SLEEP_PM_OPS(NULL,
+ pcmcia_socket_dev_resume)
+
+ /* late suspend must be called with IRQs disabled */
+ .suspend_noirq = pcmcia_socket_dev_suspend_noirq,
+ .freeze_noirq = pcmcia_socket_dev_suspend_noirq,
+ .poweroff_noirq = pcmcia_socket_dev_suspend_noirq,
+
+ /* early resume must be called with IRQs disabled */
+ .resume_noirq = pcmcia_socket_dev_resume_noirq,
+ .thaw_noirq = pcmcia_socket_dev_resume_noirq,
+ .restore_noirq = pcmcia_socket_dev_resume_noirq,
+};
+
+#define PCMCIA_SOCKET_CLASS_PM_OPS (&pcmcia_socket_pm_ops)
+
+#else /* CONFIG_PM */
+
+#define PCMCIA_SOCKET_CLASS_PM_OPS NULL
+
+#endif /* CONFIG_PM */
+
struct class pcmcia_socket_class = {
.name = "pcmcia_socket",
.dev_uevent = pcmcia_socket_uevent,
.dev_release = pcmcia_release_socket,
.class_release = pcmcia_release_socket_class,
+ .pm = PCMCIA_SOCKET_CLASS_PM_OPS,
};
EXPORT_SYMBOL(pcmcia_socket_class);
#include <linux/pm.h>
#include <linux/platform_device.h>
#include <linux/resource.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <pcmcia/cs_types.h>
return 0;
}
-#ifdef CONFIG_PM
-static int db1x_pcmcia_suspend(struct device *dev)
-{
- return pcmcia_socket_dev_suspend(dev);
-}
-
-static int db1x_pcmcia_resume(struct device *dev)
-{
- return pcmcia_socket_dev_resume(dev);
-}
-
-static struct dev_pm_ops db1x_pcmcia_pmops = {
- .resume = db1x_pcmcia_resume,
- .suspend = db1x_pcmcia_suspend,
- .thaw = db1x_pcmcia_resume,
- .freeze = db1x_pcmcia_suspend,
-};
-
-#define DB1XXX_SS_PMOPS &db1x_pcmcia_pmops
-
-#else
-
-#define DB1XXX_SS_PMOPS NULL
-
-#endif
-
static struct platform_driver db1x_pcmcia_socket_driver = {
.driver = {
.name = "db1xxx_pcmcia",
.owner = THIS_MODULE,
- .pm = DB1XXX_SS_PMOPS
},
.probe = db1x_pcmcia_socket_probe,
.remove = __devexit_p(db1x_pcmcia_socket_remove),
#include <linux/firmware.h>
#include <linux/kref.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <pcmcia/cs_types.h>
#include <pcmcia/cs.h>
p_dev->device_no = (s->device_count++);
mutex_unlock(&s->ops_mutex);
- /* max of 2 devices per card */
- if (p_dev->device_no >= 2)
+ /* max of 2 PFC devices */
+ if ((p_dev->device_no >= 2) && (function == 0))
+ goto err_free;
+
+ /* max of 4 devices overall */
+ if (p_dev->device_no >= 4)
goto err_free;
p_dev->socket = s;
#include <linux/mm.h>
#include <linux/vmalloc.h>
#include <linux/of_platform.h>
+#include <linux/slab.h>
#include <pcmcia/ss.h>
};
MODULE_DEVICE_TABLE(pci, i82092aa_pci_ids);
-#ifdef CONFIG_PM
-static int i82092aa_socket_suspend (struct pci_dev *dev, pm_message_t state)
-{
- return pcmcia_socket_dev_suspend(&dev->dev);
-}
-
-static int i82092aa_socket_resume (struct pci_dev *dev)
-{
- return pcmcia_socket_dev_resume(&dev->dev);
-}
-#endif
-
static struct pci_driver i82092aa_pci_driver = {
.name = "i82092aa",
.id_table = i82092aa_pci_ids,
.probe = i82092aa_pci_probe,
.remove = __devexit_p(i82092aa_pci_remove),
-#ifdef CONFIG_PM
- .suspend = i82092aa_socket_suspend,
- .resume = i82092aa_socket_resume,
-#endif
};
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/timer.h>
-#include <linux/slab.h>
#include <linux/ioport.h>
#include <linux/delay.h>
#include <linux/workqueue.h>
return 0;
}
-static int i82365_drv_pcmcia_suspend(struct platform_device *dev,
- pm_message_t state)
-{
- return pcmcia_socket_dev_suspend(&dev->dev);
-}
-static int i82365_drv_pcmcia_resume(struct platform_device *dev)
-{
- return pcmcia_socket_dev_resume(&dev->dev);
-}
static struct pccard_operations pcic_operations = {
.init = pcic_init,
.get_status = pcic_get_status,
.name = "i82365",
.owner = THIS_MODULE,
},
- .suspend = i82365_drv_pcmcia_suspend,
- .resume = i82365_drv_pcmcia_resume,
};
static struct platform_device *i82365_device;
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/timer.h>
-#include <linux/slab.h>
#include <linux/ioport.h>
#include <linux/delay.h>
#include <linux/workqueue.h>
.set_mem_map = pcc_set_mem_map,
};
-static int cfc_drv_pcmcia_suspend(struct platform_device *dev,
- pm_message_t state)
-{
- return pcmcia_socket_dev_suspend(&dev->dev);
-}
-static int cfc_drv_pcmcia_resume(struct platform_device *dev)
-{
- return pcmcia_socket_dev_resume(&dev->dev);
-}
/*====================================================================*/
static struct platform_driver pcc_driver = {
.name = "cfc",
.owner = THIS_MODULE,
},
- .suspend = cfc_drv_pcmcia_suspend,
- .resume = cfc_drv_pcmcia_resume,
};
static struct platform_device pcc_device = {
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/timer.h>
-#include <linux/slab.h>
#include <linux/ioport.h>
#include <linux/delay.h>
#include <linux/workqueue.h>
.set_mem_map = pcc_set_mem_map,
};
-static int pcc_drv_pcmcia_suspend(struct platform_device *dev,
- pm_message_t state)
-{
- return pcmcia_socket_dev_suspend(&dev->dev);
-}
-
-static int pcc_drv_pcmcia_resume(struct platform_device *dev)
-{
- return pcmcia_socket_dev_resume(&dev->dev);
-}
/*====================================================================*/
static struct platform_driver pcc_driver = {
.name = "pcc",
.owner = THIS_MODULE,
},
- .suspend = pcc_drv_pcmcia_suspend,
- .resume = pcc_drv_pcmcia_resume,
};
static struct platform_device pcc_device = {
#include <linux/kernel.h>
#include <linux/errno.h>
-#include <linux/slab.h>
#include <linux/timer.h>
#include <linux/ioport.h>
#include <linux/delay.h>
return 0;
}
-#ifdef CONFIG_PM
-static int m8xx_suspend(struct platform_device *pdev, pm_message_t state)
-{
- return pcmcia_socket_dev_suspend(&pdev->dev);
-}
-
-static int m8xx_resume(struct platform_device *pdev)
-{
- return pcmcia_socket_dev_resume(&pdev->dev);
-}
-#else
-#define m8xx_suspend NULL
-#define m8xx_resume NULL
-#endif
-
static const struct of_device_id m8xx_pcmcia_match[] = {
{
.type = "pcmcia",
.match_table = m8xx_pcmcia_match,
.probe = m8xx_probe,
.remove = m8xx_remove,
- .suspend = m8xx_suspend,
- .resume = m8xx_resume,
};
static int __init m8xx_init(void)
#include <linux/init.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <pcmcia/ss.h>
return 0;
}
-static int omap_cf_suspend(struct platform_device *pdev, pm_message_t mesg)
-{
- return pcmcia_socket_dev_suspend(&pdev->dev);
-}
-
-static int omap_cf_resume(struct platform_device *pdev)
-{
- return pcmcia_socket_dev_resume(&pdev->dev);
-}
-
static struct platform_driver omap_cf_driver = {
.driver = {
.name = (char *) driver_name,
.owner = THIS_MODULE,
},
.remove = __exit_p(omap_cf_remove),
- .suspend = omap_cf_suspend,
- .resume = omap_cf_resume,
};
static int __init omap_cf_init(void)
#include <linux/proc_fs.h>
#include <linux/poll.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/seq_file.h>
#include <linux/smp_lock.h>
#include <linux/workqueue.h>
#include <linux/pci.h>
#include <linux/device.h>
#include <linux/netdevice.h>
+#include <linux/slab.h>
#include <pcmcia/cs_types.h>
#include <pcmcia/ss.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/init.h>
#include <linux/workqueue.h>
#include <linux/interrupt.h>
#include <linux/device.h>
+#include <linux/io.h>
#include <pcmcia/cs_types.h>
#include <pcmcia/ss.h>
#include <pcmcia/cs.h>
#include <asm/system.h>
-#include <asm/io.h>
#include "pd6729.h"
#include "i82365.h"
? SS_READY : 0;
}
- if (events) {
+ if (events)
pcmcia_parse_events(&socket[i].socket, events);
- }
+
active |= events;
}
status = indirect_read(socket, I365_STATUS);
*value = 0;
- if ((status & I365_CS_DETECT) == I365_CS_DETECT) {
+ if ((status & I365_CS_DETECT) == I365_CS_DETECT)
*value |= SS_DETECT;
- }
/*
* IO cards have a different meaning of bits 0,1
socket->card_irq = state->io_irq;
reg = 0;
- /* The reset bit has "inverse" logic */
+ /* The reset bit has "inverse" logic */
if (!(state->flags & SS_RESET))
reg |= I365_PC_RESET;
if (state->flags & SS_IOCARD)
indirect_write(socket, I365_POWER, reg);
if (irq_mode == 1) {
- /* all interrupts are to be done as PCI interrupts */
+ /* all interrupts are to be done as PCI interrupts */
data = PD67_EC1_INV_MGMT_IRQ | PD67_EC1_INV_CARD_IRQ;
} else
data = 0;
/* Enable specific interrupt events */
reg = 0x00;
- if (state->csc_mask & SS_DETECT) {
+ if (state->csc_mask & SS_DETECT)
reg |= I365_CSC_DETECT;
- }
+
if (state->flags & SS_IOCARD) {
if (state->csc_mask & SS_STSCHG)
reg |= I365_CSC_STSCHG;
ioctl = indirect_read(socket, I365_IOCTL) & ~I365_IOCTL_MASK(map);
- if (io->flags & MAP_0WS) ioctl |= I365_IOCTL_0WS(map);
- if (io->flags & MAP_16BIT) ioctl |= I365_IOCTL_16BIT(map);
- if (io->flags & MAP_AUTOSZ) ioctl |= I365_IOCTL_IOCS16(map);
+ if (io->flags & MAP_0WS)
+ ioctl |= I365_IOCTL_0WS(map);
+ if (io->flags & MAP_16BIT)
+ ioctl |= I365_IOCTL_16BIT(map);
+ if (io->flags & MAP_AUTOSZ)
+ ioctl |= I365_IOCTL_IOCS16(map);
indirect_write(socket, I365_IOCTL, ioctl);
/* write the stop address */
- i= (mem->res->end >> 12) & 0x0fff;
+ i = (mem->res->end >> 12) & 0x0fff;
switch (to_cycles(mem->speed)) {
case 0:
break;
/* the pccard structure and its functions */
static struct pccard_operations pd6729_operations = {
- .init = pd6729_init,
+ .init = pd6729_init,
.get_status = pd6729_get_status,
.set_socket = pd6729_set_socket,
.set_io_map = pd6729_set_io_map,
static int pd6729_check_irq(int irq)
{
- if (request_irq(irq, pd6729_test, IRQF_PROBE_SHARED, "x", pd6729_test)
- != 0) return -1;
+ int ret;
+
+ ret = request_irq(irq, pd6729_test, IRQF_PROBE_SHARED, "x",
+ pd6729_test);
+ if (ret)
+ return -1;
+
free_irq(irq, pd6729_test);
return 0;
}
if (irq_mode == 1) {
printk(KERN_INFO "pd6729: PCI card interrupts, "
- "PCI status changes\n");
+ "PCI status changes\n");
return 0;
}
if (mask & (1<<i))
printk("%s%d", ((mask & ((1<<i)-1)) ? "," : ""), i);
- if (mask == 0) printk("none!");
-
- printk(" polling status changes.\n");
+ if (mask == 0)
+ printk("none!");
+ else
+ printk(" polling status changes.\n");
return mask;
}
socket = kzalloc(sizeof(struct pd6729_socket) * MAX_SOCKETS,
GFP_KERNEL);
- if (!socket)
+ if (!socket) {
+ dev_warn(&dev->dev, "failed to kzalloc socket.\n");
return -ENOMEM;
+ }
- if ((ret = pci_enable_device(dev)))
+ ret = pci_enable_device(dev);
+ if (ret) {
+ dev_warn(&dev->dev, "failed to enable pci_device.\n");
goto err_out_free_mem;
+ }
if (!pci_resource_start(dev, 0)) {
dev_warn(&dev->dev, "refusing to load the driver as the "
dev_info(&dev->dev, "Cirrus PD6729 PCI to PCMCIA Bridge at 0x%llx "
"on irq %d\n",
(unsigned long long)pci_resource_start(dev, 0), dev->irq);
- /*
+ /*
* Since we have no memory BARs some firmware may not
* have had PCI_COMMAND_MEMORY enabled, yet the device needs it.
*/
pci_set_drvdata(dev, socket);
if (irq_mode == 1) {
/* Register the interrupt handler */
- if ((ret = request_irq(dev->irq, pd6729_interrupt, IRQF_SHARED,
- "pd6729", socket))) {
+ ret = request_irq(dev->irq, pd6729_interrupt, IRQF_SHARED,
+ "pd6729", socket);
+ if (ret) {
dev_err(&dev->dev, "Failed to register irq %d\n",
dev->irq);
goto err_out_free_res;
kfree(socket);
}
-#ifdef CONFIG_PM
-static int pd6729_socket_suspend(struct pci_dev *dev, pm_message_t state)
-{
- return pcmcia_socket_dev_suspend(&dev->dev);
-}
-
-static int pd6729_socket_resume(struct pci_dev *dev)
-{
- return pcmcia_socket_dev_resume(&dev->dev);
-}
-#endif
-
static struct pci_device_id pd6729_pci_ids[] = {
{
.vendor = PCI_VENDOR_ID_CIRRUS,
.id_table = pd6729_pci_ids,
.probe = pd6729_pci_probe,
.remove = __devexit_p(pd6729_pci_remove),
-#ifdef CONFIG_PM
- .suspend = pd6729_socket_suspend,
- .resume = pd6729_socket_resume,
-#endif
};
static int pd6729_module_init(void)
======================================================================*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/cpufreq.h>
#include <linux/ioport.h>
return 0;
}
-static int pxa2xx_drv_pcmcia_suspend(struct device *dev)
-{
- return pcmcia_socket_dev_suspend(dev);
-}
-
static int pxa2xx_drv_pcmcia_resume(struct device *dev)
{
pxa2xx_configure_sockets(dev);
- return pcmcia_socket_dev_resume(dev);
+ return 0;
}
static const struct dev_pm_ops pxa2xx_drv_pcmcia_pm_ops = {
- .suspend = pxa2xx_drv_pcmcia_suspend,
.resume = pxa2xx_drv_pcmcia_resume,
};
* (C) 1999 David A. Hinds
*/
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/kernel.h>
unsigned long size = end - start + 1;
int ret = 0;
+#if defined(CONFIG_X86)
+ /* on x86, avoid anything < 0x100 for it is often used for
+ * legacy platform devices */
+ if (start < 0x100)
+ start = 0x100;
+#endif
+
if (end < start)
return -EINVAL;
if (res == &ioport_resource)
continue;
dev_printk(KERN_INFO, &s->cb_dev->dev,
- "pcmcia: parent PCI bridge I/O "
- "window: 0x%llx - 0x%llx\n",
- (unsigned long long)res->start,
- (unsigned long long)res->end);
+ "pcmcia: parent PCI bridge window: %pR\n",
+ res);
if (!adjust_io(s, ADD_MANAGED_RESOURCE, res->start, res->end))
done |= IORESOURCE_IO;
if (res == &iomem_resource)
continue;
dev_printk(KERN_INFO, &s->cb_dev->dev,
- "pcmcia: parent PCI bridge Memory "
- "window: 0x%llx - 0x%llx\n",
- (unsigned long long)res->start,
- (unsigned long long)res->end);
+ "pcmcia: parent PCI bridge window: %pR\n",
+ res);
if (!adjust_memory(s, ADD_MANAGED_RESOURCE, res->start, res->end))
done |= IORESOURCE_MEM;
}
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/platform_device.h>
#include <pcmcia/cs_types.h>
return 0;
}
-static int sa11x0_drv_pcmcia_suspend(struct platform_device *dev,
- pm_message_t state)
-{
- return pcmcia_socket_dev_suspend(&dev->dev);
-}
-
-static int sa11x0_drv_pcmcia_resume(struct platform_device *dev)
-{
- return pcmcia_socket_dev_resume(&dev->dev);
-}
-
static struct platform_driver sa11x0_pcmcia_driver = {
.driver = {
.name = "sa11x0-pcmcia",
},
.probe = sa11x0_drv_pcmcia_probe,
.remove = sa11x0_drv_pcmcia_remove,
- .suspend = sa11x0_drv_pcmcia_suspend,
- .resume = sa11x0_drv_pcmcia_resume,
};
/* sa11x0_pcmcia_init()
#include <linux/interrupt.h>
#include <linux/init.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <pcmcia/ss.h>
return 0;
}
-static int pcmcia_suspend(struct sa1111_dev *dev, pm_message_t state)
-{
- return pcmcia_socket_dev_suspend(&dev->dev);
-}
-
-static int pcmcia_resume(struct sa1111_dev *dev)
-{
- return pcmcia_socket_dev_resume(&dev->dev);
-}
-
static struct sa1111_driver pcmcia_driver = {
.drv = {
.name = "sa1111-pcmcia",
.devid = SA1111_DEVID_PCMCIA,
.probe = pcmcia_probe,
.remove = __devexit_p(pcmcia_remove),
- .suspend = pcmcia_suspend,
- .resume = pcmcia_resume,
};
static int __init sa1111_drv_pcmcia_init(void)
#include <linux/kernel.h>
#include <linux/spinlock.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <mach/hardware.h>
#include <asm/irq.h>
#include <linux/string.h>
#include <linux/major.h>
#include <linux/errno.h>
-#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/interrupt.h>
#include <linux/timer.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/interrupt.h>
-#include <linux/slab.h>
#include <linux/timer.h>
#include <linux/ioport.h>
#include <linux/delay.h>
return id;
}
-static int tcic_drv_pcmcia_suspend(struct platform_device *dev,
- pm_message_t state)
-{
- return pcmcia_socket_dev_suspend(&dev->dev);
-}
-
-static int tcic_drv_pcmcia_resume(struct platform_device *dev)
-{
- return pcmcia_socket_dev_resume(&dev->dev);
-}
/*====================================================================*/
static struct platform_driver tcic_driver = {
.name = "tcic-pcmcia",
.owner = THIS_MODULE,
},
- .suspend = tcic_drv_pcmcia_suspend,
- .resume = tcic_drv_pcmcia_resume,
};
static struct platform_device tcic_device = {
__setup("vrc4171_card=", vrc4171_card_setup);
-static int vrc4171_card_suspend(struct platform_device *dev,
- pm_message_t state)
-{
- return pcmcia_socket_dev_suspend(&dev->dev);
-}
-
-static int vrc4171_card_resume(struct platform_device *dev)
-{
- return pcmcia_socket_dev_resume(&dev->dev);
-}
-
static struct platform_driver vrc4171_card_driver = {
.driver = {
.name = vrc4171_card_name,
.owner = THIS_MODULE,
},
- .suspend = vrc4171_card_suspend,
- .resume = vrc4171_card_resume,
};
static int __devinit vrc4171_card_init(void)
#include <linux/platform_device.h>
#include <linux/pm.h>
#include <linux/resource.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <pcmcia/cs_types.h>
#include <linux/delay.h>
#include <linux/module.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <pcmcia/cs_types.h>
#include <pcmcia/ss.h>
{
struct pci_dev *pdev = to_pci_dev(dev);
struct yenta_socket *socket = pci_get_drvdata(pdev);
- int ret;
-
- ret = pcmcia_socket_dev_suspend(dev);
if (!socket)
- return ret;
+ return 0;
if (socket->type && socket->type->save_state)
socket->type->save_state(socket);
*/
/* pci_set_power_state(dev, 3); */
- return ret;
+ return 0;
}
static int yenta_dev_resume_noirq(struct device *dev)
if (socket->type && socket->type->restore_state)
socket->type->restore_state(socket);
- pcmcia_socket_dev_early_resume(dev);
- return 0;
-}
-
-static int yenta_dev_resume(struct device *dev)
-{
- pcmcia_socket_dev_late_resume(dev);
return 0;
}
static const struct dev_pm_ops yenta_pm_ops = {
.suspend_noirq = yenta_dev_suspend_noirq,
.resume_noirq = yenta_dev_resume_noirq,
- .resume = yenta_dev_resume,
.freeze_noirq = yenta_dev_suspend_noirq,
.thaw_noirq = yenta_dev_resume_noirq,
- .thaw = yenta_dev_resume,
.poweroff_noirq = yenta_dev_suspend_noirq,
.restore_noirq = yenta_dev_resume_noirq,
- .restore = yenta_dev_resume,
};
#define YENTA_PM_OPS (¥ta_pm_ops)
If you have an Eee PC laptop, say Y or M here.
+config EEEPC_WMI
+ tristate "Eee PC WMI Hotkey Driver (EXPERIMENTAL)"
+ depends on ACPI_WMI
+ depends on INPUT
+ depends on EXPERIMENTAL
+ ---help---
+ Say Y here if you want to support WMI-based hotkeys on Eee PC laptops.
+
+ To compile this driver as a module, choose M here: the module will
+ be called eeepc-wmi.
config ACPI_WMI
tristate "WMI"
#
obj-$(CONFIG_ASUS_LAPTOP) += asus-laptop.o
obj-$(CONFIG_EEEPC_LAPTOP) += eeepc-laptop.o
+obj-$(CONFIG_EEEPC_WMI) += eeepc-wmi.o
obj-$(CONFIG_MSI_LAPTOP) += msi-laptop.o
obj-$(CONFIG_ACPI_CMPC) += classmate-laptop.o
obj-$(CONFIG_COMPAL_LAPTOP) += compal-laptop.o
#include <linux/rfkill.h>
#include <linux/workqueue.h>
#include <linux/debugfs.h>
+#include <linux/slab.h>
#include <acpi/acpi_drivers.h>
#include <linux/input.h>
#include <linux/input/sparse-keymap.h>
#include <linux/rfkill.h>
+#include <linux/slab.h>
#include <acpi/acpi_drivers.h>
#include <acpi/acpi_bus.h>
/* Backlight */
static acpi_handle lcd_switch_handle;
-static const char *lcd_switch_paths[] = {
+static char *lcd_switch_paths[] = {
"\\_SB.PCI0.SBRG.EC0._Q10", /* All new models */
"\\_SB.PCI0.ISA.EC0._Q10", /* A1x */
"\\_SB.PCI0.PX40.ECD0._Q10", /* L3C */
#define METHOD_SWITCH_DISPLAY "SDSP"
static acpi_handle display_get_handle;
-static const char *display_get_paths[] = {
+static char *display_get_paths[] = {
/* A6B, A6K A6R A7D F3JM L4R M6R A3G M6A M6V VX-1 V6J V6V W3Z */
"\\_SB.PCI0.P0P1.VGA.GETD",
/* A3E A4K, A4D A4L A6J A7J A8J Z71V M9V S5A M5A z33A W1Jc W2V G1 */
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/types.h>
#include <linux/proc_fs.h>
#include <linux/init.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/workqueue.h>
#include <acpi/acpi_drivers.h>
#include <linux/backlight.h>
#include <linux/acpi.h>
#include <linux/mm.h>
#include <linux/i8042.h>
+#include <linux/slab.h>
#include "../../firmware/dcdbas.h"
#define BRIGHTNESS_TOKEN 0x7d
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/input.h>
#include <acpi/acpi_drivers.h>
#include <linux/fb.h>
#include <linux/hwmon.h>
#include <linux/hwmon-sysfs.h>
+#include <linux/slab.h>
#include <acpi/acpi_drivers.h>
#include <acpi/acpi_bus.h>
#include <linux/uaccess.h>
--- /dev/null
+/*
+ * Eee PC WMI hotkey driver
+ *
+ * Copyright(C) 2010 Intel Corporation.
+ *
+ * Portions based on wistron_btns.c:
+ * Copyright (C) 2005 Miloslav Trmac <mitr@volny.cz>
+ * Copyright (C) 2005 Bernhard Rosenkraenzer <bero@arklinux.org>
+ * Copyright (C) 2005 Dmitry Torokhov <dtor@mail.ru>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/types.h>
+#include <linux/slab.h>
+#include <linux/input.h>
+#include <linux/input/sparse-keymap.h>
+#include <acpi/acpi_bus.h>
+#include <acpi/acpi_drivers.h>
+
+MODULE_AUTHOR("Yong Wang <yong.y.wang@intel.com>");
+MODULE_DESCRIPTION("Eee PC WMI Hotkey Driver");
+MODULE_LICENSE("GPL");
+
+#define EEEPC_WMI_EVENT_GUID "ABBC0F72-8EA1-11D1-00A0-C90629100000"
+
+MODULE_ALIAS("wmi:"EEEPC_WMI_EVENT_GUID);
+
+#define NOTIFY_BRNUP_MIN 0x11
+#define NOTIFY_BRNUP_MAX 0x1f
+#define NOTIFY_BRNDOWN_MIN 0x20
+#define NOTIFY_BRNDOWN_MAX 0x2e
+
+static const struct key_entry eeepc_wmi_keymap[] = {
+ /* Sleep already handled via generic ACPI code */
+ { KE_KEY, 0x5d, { KEY_WLAN } },
+ { KE_KEY, 0x32, { KEY_MUTE } },
+ { KE_KEY, 0x31, { KEY_VOLUMEDOWN } },
+ { KE_KEY, 0x30, { KEY_VOLUMEUP } },
+ { KE_IGNORE, NOTIFY_BRNDOWN_MIN, { KEY_BRIGHTNESSDOWN } },
+ { KE_IGNORE, NOTIFY_BRNUP_MIN, { KEY_BRIGHTNESSUP } },
+ { KE_KEY, 0xcc, { KEY_SWITCHVIDEOMODE } },
+ { KE_END, 0},
+};
+
+static struct input_dev *eeepc_wmi_input_dev;
+
+static void eeepc_wmi_notify(u32 value, void *context)
+{
+ struct acpi_buffer response = { ACPI_ALLOCATE_BUFFER, NULL };
+ union acpi_object *obj;
+ acpi_status status;
+ int code;
+
+ status = wmi_get_event_data(value, &response);
+ if (status != AE_OK) {
+ pr_err("EEEPC WMI: bad event status 0x%x\n", status);
+ return;
+ }
+
+ obj = (union acpi_object *)response.pointer;
+
+ if (obj && obj->type == ACPI_TYPE_INTEGER) {
+ code = obj->integer.value;
+
+ if (code >= NOTIFY_BRNUP_MIN && code <= NOTIFY_BRNUP_MAX)
+ code = NOTIFY_BRNUP_MIN;
+ else if (code >= NOTIFY_BRNDOWN_MIN && code <= NOTIFY_BRNDOWN_MAX)
+ code = NOTIFY_BRNDOWN_MIN;
+
+ if (!sparse_keymap_report_event(eeepc_wmi_input_dev,
+ code, 1, true))
+ pr_info("EEEPC WMI: Unknown key %x pressed\n", code);
+ }
+
+ kfree(obj);
+}
+
+static int eeepc_wmi_input_setup(void)
+{
+ int err;
+
+ eeepc_wmi_input_dev = input_allocate_device();
+ if (!eeepc_wmi_input_dev)
+ return -ENOMEM;
+
+ eeepc_wmi_input_dev->name = "Eee PC WMI hotkeys";
+ eeepc_wmi_input_dev->phys = "wmi/input0";
+ eeepc_wmi_input_dev->id.bustype = BUS_HOST;
+
+ err = sparse_keymap_setup(eeepc_wmi_input_dev, eeepc_wmi_keymap, NULL);
+ if (err)
+ goto err_free_dev;
+
+ err = input_register_device(eeepc_wmi_input_dev);
+ if (err)
+ goto err_free_keymap;
+
+ return 0;
+
+err_free_keymap:
+ sparse_keymap_free(eeepc_wmi_input_dev);
+err_free_dev:
+ input_free_device(eeepc_wmi_input_dev);
+ return err;
+}
+
+static int __init eeepc_wmi_init(void)
+{
+ int err;
+ acpi_status status;
+
+ if (!wmi_has_guid(EEEPC_WMI_EVENT_GUID)) {
+ pr_warning("EEEPC WMI: No known WMI GUID found\n");
+ return -ENODEV;
+ }
+
+ err = eeepc_wmi_input_setup();
+ if (err)
+ return err;
+
+ status = wmi_install_notify_handler(EEEPC_WMI_EVENT_GUID,
+ eeepc_wmi_notify, NULL);
+ if (ACPI_FAILURE(status)) {
+ sparse_keymap_free(eeepc_wmi_input_dev);
+ input_unregister_device(eeepc_wmi_input_dev);
+ pr_err("EEEPC WMI: Unable to register notify handler - %d\n",
+ status);
+ return -ENODEV;
+ }
+
+ return 0;
+}
+
+static void __exit eeepc_wmi_exit(void)
+{
+ wmi_remove_notify_handler(EEEPC_WMI_EVENT_GUID);
+ sparse_keymap_free(eeepc_wmi_input_dev);
+ input_unregister_device(eeepc_wmi_input_dev);
+}
+
+module_init(eeepc_wmi_init);
+module_exit(eeepc_wmi_exit);
#include <linux/kfifo.h>
#include <linux/video_output.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#if defined(CONFIG_LEDS_CLASS) || defined(CONFIG_LEDS_CLASS_MODULE)
#include <linux/leds.h>
#endif
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/input.h>
#include <acpi/acpi_drivers.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/pci.h>
#include <linux/pm.h>
#include <linux/input/sparse-keymap.h>
#include <linux/acpi.h>
#include <linux/backlight.h>
+#include <linux/slab.h>
MODULE_AUTHOR("Thomas Renninger <trenn@suse.de>");
MODULE_DESCRIPTION("MSI laptop WMI hotkeys driver");
#include <linux/ctype.h>
#include <linux/seq_file.h>
#include <linux/uaccess.h>
+#include <linux/slab.h>
#include <acpi/acpi_bus.h>
#include <acpi/acpi_drivers.h>
#include <linux/input.h>
#include <linux/kfifo.h>
#include <linux/workqueue.h>
#include <linux/acpi.h>
+#include <linux/slab.h>
#include <acpi/acpi_drivers.h>
#include <acpi/acpi_bus.h>
#include <asm/uaccess.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/types.h>
#include <acpi/acpi.h>
#include <linux/kthread.h>
#include <linux/freezer.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/nvram.h>
#include <linux/proc_fs.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/acpi.h>
#include <linux/input.h>
#include <linux/platform_device.h>
#include <linux/rfkill.h>
#include <linux/input.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <linux/device.h>
#include <linux/list.h>
#include <linux/acpi.h>
+#include <linux/slab.h>
#include <acpi/acpi_bus.h>
#include <acpi/acpi_drivers.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/errno.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/isapnp.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/pnp.h>
-#include <linux/slab.h>
#include <linux/bitmap.h>
#include <linux/mutex.h>
#include "base.h"
#include <linux/acpi.h>
#include <linux/pnp.h>
+#include <linux/slab.h>
#include <linux/mod_devicetable.h>
#include <acpi/acpi_bus.h>
#include <linux/acpi.h>
#include <linux/pci.h>
#include <linux/pnp.h>
+#include <linux/slab.h>
#include "../base.h"
#include "pnpacpi.h"
#include <linux/pnp.h>
#include <linux/mm.h>
#include <linux/smp.h>
-#include <linux/slab.h>
#include <linux/kmod.h>
#include <linux/completion.h>
#include <linux/spinlock.h>
#include <linux/ctype.h>
#include <linux/pnp.h>
#include <linux/string.h>
-#include <linux/slab.h>
#ifdef CONFIG_PCI
#include <linux/pci.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/interrupt.h>
#include <linux/kernel.h>
#include <linux/power_supply.h>
#include <linux/idr.h>
#include <linux/i2c.h>
+#include <linux/slab.h>
#include <asm/unaligned.h>
#define DRIVER_VERSION "1.1.0"
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/types.h>
#include <linux/device.h>
#include <linux/jiffies.h>
#include <linux/workqueue.h>
#include <linux/pm.h>
+#include <linux/slab.h>
#include <linux/platform_device.h>
#include <linux/power_supply.h>
#include <linux/i2c.h>
#include <linux/idr.h>
#include <linux/power_supply.h>
+#include <linux/slab.h>
#define DS2782_REG_RARC 0x06 /* Remaining active relative capacity */
#include <linux/delay.h>
#include <linux/power_supply.h>
#include <linux/max17040_battery.h>
+#include <linux/slab.h>
#define MAX17040_VCELL_MSB 0x02
#define MAX17040_VCELL_LSB 0x03
#include <linux/module.h>
#include <linux/err.h>
+#include <linux/slab.h>
#include <linux/i2c.h>
#include <linux/interrupt.h>
#include <linux/platform_device.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/types.h>
#include <linux/device.h>
#include <linux/power_supply.h>
#include <linux/adb.h>
#include <linux/pmu.h>
+#include <linux/slab.h>
static struct pmu_battery_dev {
struct power_supply bat;
#include <linux/kernel.h>
#include <linux/power_supply.h>
+#include <linux/slab.h>
#include "power_supply.h"
#include <linux/ctype.h>
#include <linux/power_supply.h>
+#include <linux/slab.h>
#include "power_supply.h"
#include <linux/err.h>
#include <linux/platform_device.h>
#include <linux/power_supply.h>
+#include <linux/slab.h>
#include <linux/mfd/wm831x/core.h>
#include <linux/mfd/wm831x/auxadc.h>
#include <linux/err.h>
#include <linux/platform_device.h>
#include <linux/power_supply.h>
+#include <linux/slab.h>
#include <linux/mfd/wm831x/core.h>
#include <linux/mfd/wm831x/auxadc.h>
#include <linux/interrupt.h>
#include <linux/gpio.h>
#include <linux/irq.h>
+#include <linux/slab.h>
static DEFINE_MUTEX(bat_lock);
static struct work_struct bat_work;
#include <linux/idr.h>
#include <linux/fs.h>
#include <linux/pps_kernel.h>
+#include <linux/slab.h>
/*
* Global variables
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/interrupt.h>
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/interrupt.h>
#include <linux/workqueue.h>
#include <linux/notifier.h>
#include <linux/ioctl.h>
#include <linux/fb.h>
+#include <linux/slab.h>
#include <asm/firmware.h>
#include <asm/ps3av.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/device.h>
+#include <linux/slab.h>
#include <linux/err.h>
#include <linux/mutex.h>
#include <linux/suspend.h>
goto overflow_err;
regulator->dev = dev;
+ sysfs_attr_init(®ulator->dev_attr.attr);
regulator->dev_attr.attr.name = kstrdup(buf, GFP_KERNEL);
if (regulator->dev_attr.attr.name == NULL)
goto attr_name_err;
#include <linux/regulator/fixed.h>
#include <linux/gpio.h>
#include <linux/delay.h>
+#include <linux/slab.h>
struct fixed_voltage_data {
struct regulator_desc desc;
#include <linux/kernel.h>
#include <linux/regulator/driver.h>
#include <linux/regulator/lp3971.h>
+#include <linux/slab.h>
struct lp3971 {
struct device *dev;
LP3971_BUCK2 -> 4
LP3971_BUCK3 -> 6
*/
-#define BUCK_VOL_CHANGE_SHIFT(x) (((1 << x) & ~0x01) << 1)
+#define BUCK_VOL_CHANGE_SHIFT(x) (((!!x) << 2) | (x & ~0x01))
#define BUCK_VOL_CHANGE_FLAG_GO 0x01
#define BUCK_VOL_CHANGE_FLAG_TARGET 0x02
#define BUCK_VOL_CHANGE_FLAG_MASK 0x03
return -EINVAL;
return lp3971_set_bits(lp3971, LP3971_LDO_VOL_CONTR_REG(ldo),
- LDO_VOL_CONTR_MASK << LDO_VOL_CONTR_SHIFT(ldo), val);
+ LDO_VOL_CONTR_MASK << LDO_VOL_CONTR_SHIFT(ldo),
+ val << LDO_VOL_CONTR_SHIFT(ldo));
}
static struct regulator_ops lp3971_ldo_ops = {
lp3971->num_regulators = pdata->num_regulators;
lp3971->rdev = kcalloc(pdata->num_regulators,
sizeof(struct regulator_dev *), GFP_KERNEL);
+ if (!lp3971->rdev) {
+ err = -ENOMEM;
+ goto err_nomem;
+ }
/* Instantiate the regulators */
for (i = 0; i < pdata->num_regulators; i++) {
regulator_unregister(lp3971->rdev[i]);
kfree(lp3971->rdev);
lp3971->rdev = NULL;
+err_nomem:
return err;
}
#include <linux/i2c.h>
#include <linux/platform_device.h>
#include <linux/regulator/driver.h>
+#include <linux/slab.h>
#include <linux/regulator/max1586.h>
#define MAX1586_V3_MAX_VSEL 31
for (i = 0; i <= MAX1586_V6; i++)
if (rdev[i])
regulator_unregister(rdev[i]);
- kfree(rdev);
i2c_set_clientdata(client, NULL);
+ kfree(rdev);
return 0;
}
#include <linux/i2c.h>
#include <linux/platform_device.h>
#include <linux/regulator/driver.h>
+#include <linux/slab.h>
#include <linux/regulator/max8649.h>
#define MAX8649_DCDC_VMIN 750000 /* uV */
dev_info(info->dev, "Max8649 regulator device is detected.\n");
return 0;
out:
+ i2c_set_clientdata(client, NULL);
kfree(info);
return ret;
}
if (info) {
if (info->regulator)
regulator_unregister(info->regulator);
+ i2c_set_clientdata(client, NULL);
kfree(info);
}
- i2c_set_clientdata(client, NULL);
return 0;
}
#include <linux/i2c.h>
#include <linux/platform_device.h>
#include <linux/regulator/driver.h>
+#include <linux/slab.h>
#include <linux/regulator/max8660.h>
#define MAX8660_DCDC_MIN_UV 725000
for (i = 0; i < MAX8660_V_END; i++)
if (rdev[i])
regulator_unregister(rdev[i]);
- kfree(rdev);
i2c_set_clientdata(client, NULL);
+ kfree(rdev);
return 0;
}
MAX8925_LDO(20, 750, 3900, 50),
};
-static inline struct max8925_regulator_info *find_regulator_info(int id)
+static struct max8925_regulator_info * __devinit find_regulator_info(int id)
{
struct max8925_regulator_info *ri;
int i;
{
struct max8925_chip *chip = dev_get_drvdata(pdev->dev.parent);
struct max8925_platform_data *pdata = chip->dev->platform_data;
- struct max8925_regulator_info *ri = NULL;
+ struct max8925_regulator_info *ri;
struct regulator_dev *rdev;
ri = find_regulator_info(pdev->id);
{
struct regulator_dev *rdev = platform_get_drvdata(pdev);
+ platform_set_drvdata(pdev, NULL);
regulator_unregister(rdev);
+
return 0;
}
#include <linux/regulator/driver.h>
#include <linux/platform_device.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/err.h>
#include <linux/regulator/machine.h>
#include <linux/i2c.h>
#include <linux/delay.h>
+#include <linux/slab.h>
/* Register definitions */
#define TPS65023_REG_VERSION 0
#include <linux/regulator/machine.h>
#include <linux/i2c.h>
#include <linux/delay.h>
+#include <linux/slab.h>
/* Register definitions */
#define TPS6507X_REG_PPATH1 0X01
#include <linux/platform_device.h>
#include <linux/regulator/consumer.h>
#include <linux/regulator/userspace-consumer.h>
+#include <linux/slab.h>
struct userspace_consumer_data {
const char *name;
#include <linux/mutex.h>
#include <linux/platform_device.h>
#include <linux/regulator/consumer.h>
+#include <linux/slab.h>
struct virtual_consumer_data {
struct mutex lock;
#include <linux/regulator/driver.h>
#include <linux/regulator/machine.h>
#include <linux/gpio.h>
+#include <linux/slab.h>
#include <linux/mfd/wm831x/core.h>
#include <linux/mfd/wm831x/regulator.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
#include <linux/regulator/driver.h>
+#include <linux/slab.h>
#include <linux/mfd/wm831x/core.h>
#include <linux/mfd/wm831x/regulator.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
#include <linux/regulator/driver.h>
+#include <linux/slab.h>
#include <linux/mfd/wm831x/core.h>
#include <linux/mfd/wm831x/regulator.h>
#include <linux/platform_device.h>
#include <linux/regulator/driver.h>
#include <linux/gpio.h>
+#include <linux/slab.h>
#include <linux/mfd/wm8994/core.h>
#include <linux/mfd/wm8994/registers.h>
#include <linux/rtc.h>
#include <linux/kdev_t.h>
#include <linux/idr.h>
+#include <linux/slab.h>
#include "rtc-core.h"
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/rtc.h>
#include <linux/io.h>
#include <linux/rtc.h>
#include <linux/interrupt.h>
#include <linux/ioctl.h>
+#include <linux/slab.h>
#include <mach/board.h>
#include <mach/at91_rtt.h>
#include <linux/platform_device.h>
#include <linux/rtc.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include <asm/blackfin.h>
#include <linux/platform_device.h>
#include <linux/rtc.h>
#include <linux/bcd.h>
+#include <linux/slab.h>
MODULE_AUTHOR("David S. Miller <davem@davemloft.net>");
MODULE_DESCRIPTION("TI BQ4802 RTC driver");
#include <linux/pm.h>
#include <linux/platform_device.h>
#include <linux/io.h>
+#include <linux/slab.h>
/*
* Registers in the COH 901 331
#include <linux/rtc.h>
#include <linux/platform_device.h>
#include <linux/bcd.h>
+#include <linux/slab.h>
#define DRV_VERSION "0.2"
#include <linux/bcd.h>
#include <linux/ds1286.h>
#include <linux/io.h>
+#include <linux/slab.h>
#define DRV_VERSION "1.0"
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/bcd.h>
+#include <linux/slab.h>
#include <linux/rtc.h>
#include <linux/workqueue.h>
#include <linux/rtc.h>
#include <linux/bcd.h>
#include <linux/workqueue.h>
+#include <linux/slab.h>
#define DS1374_REG_TOD0 0x00 /* Time of Day */
#define DS1374_REG_TOD1 0x01
#include <linux/rtc.h>
#include <linux/spi/spi.h>
#include <linux/bcd.h>
+#include <linux/slab.h>
#define DS1390_REG_100THS 0x00
#define DS1390_REG_SECONDS 0x01
#include <linux/bcd.h>
#include <linux/init.h>
#include <linux/kernel.h>
+#include <linux/gfp.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/rtc.h>
#include <linux/bcd.h>
#include <linux/init.h>
#include <linux/kernel.h>
+#include <linux/gfp.h>
#include <linux/delay.h>
#include <linux/jiffies.h>
#include <linux/interrupt.h>
#include <linux/bcd.h>
#include <linux/init.h>
#include <linux/kernel.h>
+#include <linux/gfp.h>
#include <linux/delay.h>
#include <linux/jiffies.h>
#include <linux/rtc.h>
#include <linux/rtc.h>
#include <linux/platform_device.h>
#include <linux/io.h>
+#include <linux/gfp.h>
#define EP93XX_RTC_DATA 0x000
#define EP93XX_RTC_MATCH 0x004
#include <linux/i2c.h>
#include <linux/rtc.h>
#include <linux/bcd.h>
+#include <linux/slab.h>
#define FM3130_RTC_CONTROL (0x0)
#define FM3130_CAL_CONTROL (0x1)
#include <linux/module.h>
#include <linux/rtc.h>
+#include <linux/slab.h>
#include <linux/platform_device.h>
#include <linux/bcd.h>
#include <linux/io.h>
#include <linux/rtc.h>
#include <linux/rtc/m48t59.h>
#include <linux/bcd.h>
+#include <linux/slab.h>
#ifndef NO_IRQ
#define NO_IRQ (-1)
#include <linux/module.h>
#include <linux/i2c.h>
+#include <linux/slab.h>
#include <linux/rtc.h>
#include <linux/platform_device.h>
#include <linux/mfd/max8925.h>
#include <linux/platform_device.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/rtc.h>
#define DRIVER_NAME "mc13783-rtc"
{
int ret;
struct mc13783_rtc *priv;
+ struct mc13783 *mc13783;
int rtcrst_pending;
priv = kzalloc(sizeof(*priv), GFP_KERNEL);
if (!priv)
return -ENOMEM;
- priv->mc13783 = dev_get_drvdata(pdev->dev.parent);
+ mc13783 = dev_get_drvdata(pdev->dev.parent);
+ priv->mc13783 = mc13783;
+
platform_set_drvdata(pdev, priv);
- mc13783_lock(priv->mc13783);
+ mc13783_lock(mc13783);
- ret = mc13783_irq_request(priv->mc13783, MC13783_IRQ_RTCRST,
+ ret = mc13783_irq_request(mc13783, MC13783_IRQ_RTCRST,
mc13783_rtc_reset_handler, DRIVER_NAME, priv);
if (ret)
goto err_reset_irq_request;
- ret = mc13783_irq_status(priv->mc13783, MC13783_IRQ_RTCRST,
+ ret = mc13783_irq_status(mc13783, MC13783_IRQ_RTCRST,
NULL, &rtcrst_pending);
if (ret)
goto err_reset_irq_status;
priv->valid = !rtcrst_pending;
- ret = mc13783_irq_request_nounmask(priv->mc13783, MC13783_IRQ_1HZ,
+ ret = mc13783_irq_request_nounmask(mc13783, MC13783_IRQ_1HZ,
mc13783_rtc_update_handler, DRIVER_NAME, priv);
if (ret)
goto err_update_irq_request;
- ret = mc13783_irq_request_nounmask(priv->mc13783, MC13783_IRQ_TODA,
+ ret = mc13783_irq_request_nounmask(mc13783, MC13783_IRQ_TODA,
mc13783_rtc_alarm_handler, DRIVER_NAME, priv);
if (ret)
goto err_alarm_irq_request;
if (IS_ERR(priv->rtc)) {
ret = PTR_ERR(priv->rtc);
- mc13783_irq_free(priv->mc13783, MC13783_IRQ_TODA, priv);
+ mc13783_irq_free(mc13783, MC13783_IRQ_TODA, priv);
err_alarm_irq_request:
- mc13783_irq_free(priv->mc13783, MC13783_IRQ_1HZ, priv);
+ mc13783_irq_free(mc13783, MC13783_IRQ_1HZ, priv);
err_update_irq_request:
err_reset_irq_status:
- mc13783_irq_free(priv->mc13783, MC13783_IRQ_RTCRST, priv);
+ mc13783_irq_free(mc13783, MC13783_IRQ_RTCRST, priv);
err_reset_irq_request:
platform_set_drvdata(pdev, NULL);
kfree(priv);
}
- mc13783_unlock(priv->mc13783);
+ mc13783_unlock(mc13783);
return ret;
}
#include <linux/of_device.h>
#include <linux/of_platform.h>
#include <linux/io.h>
+#include <linux/slab.h>
struct mpc5121_rtc_regs {
u8 set_time; /* RTC + 0x00 */
#include <linux/module.h>
#include <linux/platform_device.h>
#include <linux/rtc.h>
+#include <linux/slab.h>
enum {
#include <linux/io.h>
#include <linux/platform_device.h>
#include <linux/delay.h>
+#include <linux/gfp.h>
#define RTC_TIME_REG_OFFS 0
#include <linux/io.h>
#include <linux/rtc.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/platform_device.h>
#include <linux/clk.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/rtc.h>
#include <linux/delay.h>
#include <linux/io.h>
#include <linux/init.h>
#include <linux/mfd/ezx-pcap.h>
#include <linux/rtc.h>
+#include <linux/slab.h>
#include <linux/platform_device.h>
struct pcap_rtc {
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/string.h>
+#include <linux/slab.h>
#include <linux/rtc.h>
#include <linux/spi/spi.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/device.h>
+#include <linux/slab.h>
#include <linux/platform_device.h>
#include <linux/rtc.h>
#include <linux/bcd.h>
#include <linux/i2c.h>
#include <linux/bcd.h>
#include <linux/rtc.h>
+#include <linux/slab.h>
#define DRV_VERSION "0.4.3"
#include <linux/interrupt.h>
#include <linux/amba/bus.h>
#include <linux/io.h>
+#include <linux/slab.h>
#define RTC_DR (0)
#define RTC_MR (4)
#include <linux/bcd.h>
#include <linux/delay.h>
#include <linux/version.h>
+#include <linux/slab.h>
/*
* Register definitions
#include <linux/seq_file.h>
#include <linux/interrupt.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <mach/hardware.h>
#include <linux/module.h>
#include <linux/platform_device.h>
#include <linux/rtc.h>
+#include <linux/slab.h>
enum {
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/string.h>
+#include <linux/slab.h>
#include <linux/rtc.h>
#include <linux/workqueue.h>
#include <linux/spi/spi.h>
#include <linux/i2c.h>
#include <linux/rtc.h>
#include <linux/bcd.h>
+#include <linux/slab.h>
#define DRV_VERSION "0.6"
*/
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/bcd.h>
#include <linux/i2c.h>
#include <linux/bcd.h>
#include <linux/clk.h>
#include <linux/log2.h>
+#include <linux/slab.h>
#include <mach/hardware.h>
#include <asm/uaccess.h>
#include <linux/io.h>
#include <linux/log2.h>
#include <linux/clk.h>
+#include <linux/slab.h>
#include <asm/rtc.h>
#define DRV_NAME "sh-rtc"
#include <linux/bcd.h>
#include <linux/init.h>
#include <linux/kernel.h>
+#include <linux/gfp.h>
#include <linux/delay.h>
#include <linux/jiffies.h>
#include <linux/interrupt.h>
#include <linux/platform_device.h>
#include <linux/interrupt.h>
#include <linux/rtc.h>
+#include <linux/slab.h>
#include <mach/platform.h>
#include <mach/stmp3xxx.h>
#include <linux/platform_device.h>
#include <linux/interrupt.h>
#include <linux/io.h>
+#include <linux/gfp.h>
#include <asm/txx9/tx4939.h>
struct tx4939rtc_plat_data {
#include <linux/rtc-v3020.h>
#include <linux/delay.h>
#include <linux/gpio.h>
+#include <linux/slab.h>
#include <linux/io.h>
#include <linux/kernel.h>
#include <linux/time.h>
#include <linux/rtc.h>
+#include <linux/slab.h>
#include <linux/bcd.h>
#include <linux/interrupt.h>
#include <linux/ioctl.h>
#define KMSG_COMPONENT "dasd-eckd"
#include <linux/timer.h>
-#include <linux/slab.h>
#include <asm/idals.h>
#define PRINTK_HEADER "dasd_erp(3990): "
if (cqr->cpmode == 1) {
cplength = 0;
- datasize = sizeof(struct tcw) + sizeof(struct tsb);
+ /* TCW needs to be 64 byte aligned, so leave enough room */
+ datasize = 64 + sizeof(struct tcw) + sizeof(struct tsb);
} else {
cplength = 2;
datasize = 0;
if (cqr->cpmode == 1) {
/* make a shallow copy of the original tcw but set new tsb */
erp->cpmode = 1;
- erp->cpaddr = erp->data;
- tcw = erp->data;
+ erp->cpaddr = PTR_ALIGN(erp->data, 64);
+ tcw = erp->cpaddr;
tsb = (struct tsb *) &tcw[1];
*tcw = *((struct tcw *)cqr->cpaddr);
tcw->tsb = (long)tsb;
#define KMSG_COMPONENT "dasd-eckd"
#include <linux/list.h>
+#include <linux/slab.h>
#include <asm/ebcdic.h>
#include "dasd_int.h"
#include "dasd_eckd.h"
#include <linux/ctype.h>
#include <linux/init.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <asm/debug.h>
#include <asm/uaccess.h>
tsb = NULL;
sense = NULL;
- if (irb->scsw.tm.tcw)
+ if (irb->scsw.tm.tcw && (irb->scsw.tm.fcxs == 0x01))
tsb = tcw_get_tsb(
(struct tcw *)(unsigned long)irb->scsw.tm.tcw);
- if (tsb && (irb->scsw.tm.fcxs == 0x01)) {
+ if (tsb) {
len += sprintf(page + len, KERN_ERR PRINTK_HEADER
" tsb->length %d\n", tsb->length);
len += sprintf(page + len, KERN_ERR PRINTK_HEADER
#include <linux/mutex.h>
#include <linux/smp_lock.h>
#include <linux/err.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/atomic.h>
#include <linux/fs.h>
#include <linux/blkpg.h>
#include <linux/smp_lock.h>
+#include <linux/slab.h>
#include <asm/compat.h>
#include <asm/ccwdev.h>
#include <asm/cmb.h>
#define KMSG_COMPONENT "dasd"
#include <linux/ctype.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include <linux/seq_file.h>
#include <linux/vmalloc.h>
#include <linux/ctype.h> /* isdigit, isxdigit */
#include <linux/errno.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/blkdev.h>
#include <linux/blkpg.h>
#include <linux/hdreg.h> /* HDIO_GETGEO */
#include <linux/bio.h>
#include <linux/suspend.h>
#include <linux/platform_device.h>
+#include <linux/gfp.h>
#include <asm/uaccess.h>
#define XPRAM_NAME "xpram"
#include <linux/interrupt.h>
#include <linux/list.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/err.h>
#include <linux/reboot.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/list.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/smp_lock.h>
#include <linux/module.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/sysrq.h>
#include <linux/consolemap.h>
#include <linux/interrupt.h>
#include <linux/poll.h>
#include <linux/device.h>
+#include <linux/slab.h>
#include <net/iucv/iucv.h>
#include <asm/uaccess.h>
#include <asm/ebcdic.h>
#include <linux/poll.h>
#include <linux/mutex.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/ebcdic.h>
#include <asm/io.h>
#include <linux/device.h>
#include <linux/stat.h>
#include <linux/string.h>
+#include <linux/slab.h>
#include <linux/ctype.h>
#include <linux/kmod.h>
#include <linux/err.h>
rc = copy_from_user(buf, buffer, sizeof(buf));
if (rc != 0)
return -EFAULT;
+ buf[len - 1] = '\0';
if (strict_strtoul(buf, 0, &val) != 0)
return -EINVAL;
if (val != 0 && val != 1)
u16 rn;
} __packed;
+int arch_get_memory_phys_device(unsigned long start_pfn)
+{
+ if (!rzm)
+ return 0;
+ return PFN_PHYS(start_pfn) >> ilog2(rzm);
+}
+
static unsigned long long rn2addr(u16 rn)
{
return (unsigned long long) (rn - 1) * rzm;
return do_chp_configure(SCLP_CMDW_DECONFIGURE_CHPATH | chpid.id << 8);
}
-int arch_get_memory_phys_device(unsigned long start_pfn)
-{
- if (!rzm)
- return 0;
- return PFN_PHYS(start_pfn) / rzm;
-}
-
struct chp_info_sccb {
struct sccb_header header;
u8 recognized[SCLP_CHP_INFO_MASK_SIZE];
#include <linux/termios.h>
#include <linux/err.h>
#include <linux/reboot.h>
+#include <linux/gfp.h>
#include "sclp.h"
#include "sclp_rw.h"
#include <linux/tty.h>
#include <linux/tty_driver.h>
#include <linux/tty_flip.h>
-#include <linux/slab.h>
#include <linux/err.h>
#include <linux/init.h>
#include <linux/interrupt.h>
+#include <linux/gfp.h>
#include <asm/uaccess.h>
#include "ctrlchar.h"
#include <linux/interrupt.h>
#include <linux/init.h>
#include <linux/reboot.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include "sclp.h"
#include <linux/init.h>
#include <linux/bio.h>
#include <linux/workqueue.h>
+#include <linux/slab.h>
#define TAPE_DBF_AREA tape_34xx_dbf
#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/bio.h>
#include <asm/ebcdic.h>
#define KMSG_COMPONENT "tape"
#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
+#include <linux/slab.h>
+
#include "tape_class.h"
MODULE_AUTHOR("Stefan Bader <shbader@de.ibm.com>");
#include <linux/spinlock.h> // for locks
#include <linux/vmalloc.h>
#include <linux/list.h>
+#include <linux/slab.h>
#include <asm/types.h> // for variable types
#include <linux/kernel.h>
#include <linux/miscdevice.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <asm/compat.h>
#include <asm/cpcmd.h>
#include <asm/debug.h>
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/types.h>
#include <linux/interrupt.h>
#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
#include <linux/cdev.h>
+#include <linux/slab.h>
#include <linux/smp_lock.h>
#include <asm/uaccess.h>
#include <linux/miscdevice.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
+#include <linux/slab.h>
#include <linux/suspend.h>
#include <linux/watchdog.h>
#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/miscdevice.h>
#include <linux/debugfs.h>
#include <asm/asm-offsets.h>
return memcpy_hsa(dest, src, count, TO_KERNEL);
}
-static int memcpy_real(void *dest, unsigned long src, size_t count)
-{
- unsigned long flags;
- int rc = -EFAULT;
- register unsigned long _dest asm("2") = (unsigned long) dest;
- register unsigned long _len1 asm("3") = (unsigned long) count;
- register unsigned long _src asm("4") = src;
- register unsigned long _len2 asm("5") = (unsigned long) count;
-
- if (count == 0)
- return 0;
- flags = __raw_local_irq_stnsm(0xf8UL); /* switch to real mode */
- asm volatile (
- "0: mvcle %1,%2,0x0\n"
- "1: jo 0b\n"
- " lhi %0,0x0\n"
- "2:\n"
- EX_TABLE(1b,2b)
- : "+d" (rc), "+d" (_dest), "+d" (_src), "+d" (_len1),
- "+d" (_len2), "=m" (*((long*)dest))
- : "m" (*((long*)src))
- : "cc", "memory");
- __raw_local_irq_ssm(flags);
-
- return rc;
-}
-
static int memcpy_real_user(void __user *dest, unsigned long src, size_t count)
{
static char buf[4096];
while (offs < count) {
size = min(sizeof(buf), count - offs);
- if (memcpy_real(buf, src + offs, size))
+ if (memcpy_real(buf, (void *) src + offs, size))
return -EFAULT;
if (copy_to_user(dest + offs, buf, size))
return -EFAULT;
if (ipib_info.ipib < ZFCPDUMP_HSA_SIZE)
rc = memcpy_hsa_kernel(ipl_block, ipib_info.ipib, PAGE_SIZE);
else
- rc = memcpy_real(ipl_block, ipib_info.ipib, PAGE_SIZE);
+ rc = memcpy_real(ipl_block, (void *) ipib_info.ipib, PAGE_SIZE);
if (rc) {
free_page((unsigned long) ipl_block);
return rc;
#include <linux/init.h>
#include <linux/vmalloc.h>
-#include <linux/slab.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
#include <linux/ctype.h>
#include <linux/wait.h>
#include <linux/mutex.h>
#include <linux/errno.h>
+#include <linux/slab.h>
#include <asm/chpid.h>
#include <asm/sclp.h>
#include <asm/crw.h>
*
*/
+#include <linux/slab.h>
#include <linux/device.h>
#include <linux/module.h>
#include <linux/uaccess.h>
#include <linux/kernel.h>
#include <linux/timer.h>
#include <linux/delay.h>
+#include <linux/gfp.h>
#include <asm/atomic.h>
#include <asm/debug.h>
#include <asm/qdio.h>
* Jan Glauber <jang@linux.vnet.ibm.com>
*/
#include <linux/io.h>
+#include <linux/slab.h>
#include <asm/atomic.h>
#include <asm/debug.h>
#include <asm/qdio.h>
#include <linux/err.h>
#include <linux/interrupt.h>
#include <linux/workqueue.h>
+#include <linux/slab.h>
#include <linux/notifier.h>
#include <linux/kthread.h>
#include <linux/mutex.h>
#include <linux/seq_file.h>
#include <linux/compat.h>
#include <linux/smp_lock.h>
+#include <linux/slab.h>
#include <asm/atomic.h>
#include <asm/uaccess.h>
#include <linux/hw_random.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/err.h>
#include <asm/atomic.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/err.h>
#include <asm/atomic.h>
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/gfp.h>
#include <linux/err.h>
#include <asm/atomic.h>
#include <asm/uaccess.h>
#include <linux/init.h>
#include <linux/err.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <asm/atomic.h>
#include <asm/uaccess.h>
#include <linux/err.h>
#include <linux/virtio.h>
#include <linux/virtio_config.h>
+#include <linux/slab.h>
#include <linux/virtio_console.h>
#include <linux/interrupt.h>
#include <linux/virtio_ring.h>
#include <linux/string.h>
#include <linux/kernel.h>
#include <linux/errno.h>
-#include <linux/slab.h>
#include <linux/ctype.h>
#include <linux/sysctl.h>
#include <linux/module.h>
#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
#include <linux/sysfs.h>
+#include <linux/slab.h>
#include "ctcm_main.h"
/*
#include "fsm.h"
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/timer.h>
MODULE_AUTHOR("(C) 2000 IBM Corp. by Fritz Elfert (felfert@millenux.com)");
#include <linux/igmp.h>
#include <linux/delay.h>
#include <linux/kthread.h>
+#include <linux/slab.h>
#include <net/arp.h>
#include <net/ip.h>
#include <linux/tcp.h>
#include <linux/mii.h>
#include <linux/kthread.h>
+#include <linux/slab.h>
#include <asm/ebcdic.h>
#include <asm/io.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/etherdevice.h>
#include <linux/mii.h>
#include <linux/ip.h>
#include <linux/ipv6.h>
#include <linux/inetdevice.h>
#include <linux/igmp.h>
+#include <linux/slab.h>
#include <net/ip.h>
#include <net/arp.h>
* Frank Blaschka <frank.blaschka@de.ibm.com>
*/
+#include <linux/slab.h>
+
#include "qeth_l3.h"
#define QETH_DEVICE_ATTR(_id, _name, _mode, _show, _store) \
#include <linux/init.h>
#include <linux/errno.h>
#include <linux/device.h>
+#include <linux/slab.h>
#include <net/iucv/iucv.h>
#include <asm/cpcmd.h>
#include <asm/ebcdic.h>
#include <linux/list.h>
#include <linux/kobject.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/workqueue.h>
#include <net/iucv/iucv.h>
#include <linux/miscdevice.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include "zfcp_ext.h"
#include "zfcp_fc.h"
#include "zfcp_reqlist.h"
#define KMSG_COMPONENT "zfcp"
#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/miscdevice.h>
#include <asm/compat.h>
#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
#include <linux/ctype.h>
+#include <linux/slab.h>
#include <asm/debug.h>
#include "zfcp_dbf.h"
#include "zfcp_ext.h"
#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
#include <linux/types.h>
+#include <linux/slab.h>
#include <scsi/fc/fc_els.h>
#include <scsi/libfc.h>
#include "zfcp_ext.h"
#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
#include <linux/blktrace_api.h>
+#include <linux/slab.h>
#include <scsi/fc/fc_els.h>
#include "zfcp_ext.h"
#include "zfcp_fc.h"
#define KMSG_COMPONENT "zfcp"
#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
+#include <linux/slab.h>
#include "zfcp_ext.h"
#include "zfcp_qdio.h"
#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
#include <linux/types.h>
+#include <linux/slab.h>
#include <scsi/fc/fc_fcp.h>
#include <asm/atomic.h>
#include "zfcp_ext.h"
#define KMSG_COMPONENT "zfcp"
#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
+#include <linux/slab.h>
#include "zfcp_ext.h"
#define ZFCP_DEV_ATTR(_feat, _name, _mode, _show, _store) \
#include <linux/kmod.h>
#include <linux/reboot.h>
#include <linux/of.h>
+#include <linux/slab.h>
#include <linux/of_device.h>
#include <asm/oplib.h>
#include <linux/init.h>
#include <linux/miscdevice.h>
#include <linux/ioport.h> /* request_region */
+#include <linux/slab.h>
#include <linux/smp_lock.h>
#include <linux/of.h>
#include <linux/of_device.h>
#include <linux/miscdevice.h>
#include <linux/kmod.h>
#include <linux/reboot.h>
+#include <linux/slab.h>
#include <linux/smp_lock.h>
#include <linux/of.h>
#include <linux/of_device.h>
#include <linux/types.h>
#include <linux/errno.h>
#include <linux/miscdevice.h>
-#include <linux/slab.h>
#include <linux/fcntl.h>
#include <linux/poll.h>
#include <linux/init.h>
#include <linux/types.h>
#include <linux/errno.h>
#include <linux/miscdevice.h>
-#include <linux/slab.h>
#include <linux/fcntl.h>
#include <linux/poll.h>
#include <linux/init.h>
#include <linux/time.h>
#include <linux/mutex.h>
#include <linux/smp_lock.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/irq.h>
#include <asm/uaccess.h>
#include <linux/time.h>
#include <linux/mutex.h>
#include <linux/smp_lock.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/irq.h>
#include <asm/uaccess.h>
#include <linux/errno.h>
#include <linux/types.h>
#include <linux/delay.h>
+#include <linux/gfp.h>
#include <linux/pci.h>
#include <linux/time.h>
#include <linux/mutex.h>
#include <linux/kernel.h>
#include <linux/types.h>
#include <linux/string.h>
+#include <linux/slab.h>
#include <linux/ioport.h>
#include <linux/delay.h>
#include <linux/spinlock.h>
#include <linux/spinlock.h>
#include <linux/jiffies.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <scsi/scsicam.h>
#include <asm/dma.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/mca.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <scsi/scsi_host.h>
#include <scsi/scsi_device.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/mca.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/init.h>
#include <linux/delay.h>
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/dma-mapping.h>
#include <asm/io.h>
#include <linux/types.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include <linux/blkdev.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/types.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include <linux/blkdev.h>
#include <linux/ioport.h>
#include <linux/init.h>
#include <linux/platform_device.h>
#include <linux/init.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <asm/amigahw.h>
#include <asm/amigaints.h>
#include <scsi/scsi_host.h>
#include <linux/types.h>
#include <linux/pci.h>
#include <linux/spinlock.h>
-#include <linux/slab.h>
#include <linux/blkdev.h>
#include <linux/delay.h>
#include <linux/completion.h>
#include <linux/types.h>
#include <linux/pci.h>
#include <linux/spinlock.h>
-#include <linux/slab.h>
#include <linux/blkdev.h>
#include <linux/delay.h>
#include <linux/completion.h>
if (err) {
printk(KERN_ERR "Failed to load image \"%s\" err %d\n",
fwname, err);
+ asc_dvc->err_code |= ASC_IERR_MCODE_CHKSUM;
return err;
}
if (fw->size < 4) {
printk(KERN_ERR "Bogus length %zu in image \"%s\"\n",
fw->size, fwname);
release_firmware(fw);
+ asc_dvc->err_code |= ASC_IERR_MCODE_CHKSUM;
return -EINVAL;
}
chksum = (fw->data[3] << 24) | (fw->data[2] << 16) |
if (err) {
printk(KERN_ERR "Failed to load image \"%s\" err %d\n",
fwname, err);
+ asc_dvc->err_code = ASC_IERR_MCODE_CHKSUM;
return err;
}
if (fw->size < 4) {
printk(KERN_ERR "Bogus length %zu in image \"%s\"\n",
fw->size, fwname);
release_firmware(fw);
+ asc_dvc->err_code = ASC_IERR_MCODE_CHKSUM;
return -EINVAL;
}
chksum = (fw->data[3] << 24) | (fw->data[2] << 16) |
if (err) {
printk(KERN_ERR "Failed to load image \"%s\" err %d\n",
fwname, err);
+ asc_dvc->err_code = ASC_IERR_MCODE_CHKSUM;
return err;
}
if (fw->size < 4) {
printk(KERN_ERR "Bogus length %zu in image \"%s\"\n",
fw->size, fwname);
release_firmware(fw);
+ asc_dvc->err_code = ASC_IERR_MCODE_CHKSUM;
return -EINVAL;
}
chksum = (fw->data[3] << 24) | (fw->data[2] << 16) |
if (err) {
printk(KERN_ERR "Failed to load image \"%s\" err %d\n",
fwname, err);
+ asc_dvc->err_code = ASC_IERR_MCODE_CHKSUM;
return err;
}
if (fw->size < 4) {
printk(KERN_ERR "Bogus length %zu in image \"%s\"\n",
fw->size, fwname);
release_firmware(fw);
+ asc_dvc->err_code = ASC_IERR_MCODE_CHKSUM;
return -EINVAL;
}
chksum = (fw->data[3] << 24) | (fw->data[2] << 16) |
#include <linux/spinlock.h>
#include <linux/workqueue.h>
#include <linux/list.h>
+#include <linux/slab.h>
#include <scsi/scsicam.h>
#include "scsi.h"
#include <linux/blkdev.h>
#include <linux/mca.h>
#include <linux/mca-legacy.h>
+#include <linux/slab.h>
#include <asm/dma.h>
#include <asm/system.h>
#include <linux/device.h>
#include <linux/eisa.h>
#include <linux/dma-mapping.h>
+#include <linux/gfp.h>
#include <asm/dma.h>
#include <asm/system.h>
#include <linux/blkdev.h> /* For block_size() */
#include <linux/delay.h> /* For ssleep/msleep */
#include <linux/device.h>
+#include <linux/slab.h>
/*
* Bucket size for counting good commands in between bad ones.
#include <linux/mm.h> /* For fetching system memory size */
#include <linux/blkdev.h> /* For block_size() */
#include <linux/delay.h> /* For ssleep/msleep */
+#include <linux/slab.h>
/*
*/
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/module.h>
#include <linux/firmware.h>
#include <linux/pci.h>
#include <linux/delay.h>
#include <linux/firmware.h>
+#include <linux/slab.h>
#include <scsi/scsi_host.h>
*
*/
+#include <linux/gfp.h>
#include <scsi/scsi_host.h>
#include "aic94xx.h"
*/
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include "aic94xx.h"
*/
#include <linux/delay.h>
+#include <linux/gfp.h>
#include <linux/pci.h>
#include <linux/module.h>
#include <linux/firmware.h>
*/
#include <linux/spinlock.h>
+#include <linux/gfp.h>
#include "aic94xx.h"
#include "aic94xx_sas.h"
#include "aic94xx_hwi.h"
#include <linux/timer.h>
#include <linux/pci.h>
#include <linux/aer.h>
+#include <linux/slab.h>
#include <asm/dma.h>
#include <asm/io.h>
#include <asm/system.h>
* interrupt or bottom half.
*/
+#include <linux/gfp.h>
#include <linux/workqueue.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
#include <linux/blkdev.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <asm/system.h>
#include <asm/io.h>
*/
#include <linux/reboot.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/blkdev.h>
#include <linux/pci.h>
* bfad.c Linux driver PCI interface module.
*/
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/kthread.h>
#include "bfad_drv.h"
* bfa_attr.c Linux driver configuration interface module.
*/
+#include <linux/slab.h>
#include "bfad_drv.h"
#include "bfad_im.h"
#include "bfad_trcmod.h"
* bfad_im.c Linux driver IM module.
*/
+#include <linux/slab.h>
#include "bfad_drv.h"
#include "bfad_im.h"
#include "bfad_trcmod.h"
* rport.c Remote port implementation.
*/
+#include <linux/slab.h>
#include <bfa.h>
#include <bfa_svc.h>
#include "fcbuild.h"
* Written by: Anil Veerabhadrappa (anilgv@broadcom.com)
*/
+#include <linux/gfp.h>
#include <scsi/scsi_tcq.h>
#include <scsi/libiscsi.h>
#include "bnx2i.h"
* Written by: Anil Veerabhadrappa (anilgv@broadcom.com)
*/
+#include <linux/slab.h>
#include <scsi/scsi_tcq.h>
#include <scsi/libiscsi.h>
#include "bnx2i.h"
#include <linux/platform_device.h>
#include <linux/init.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <asm/bvme6000hw.h>
#include <scsi/scsi_host.h>
#include <scsi/scsi_device.h>
#include <linux/mutex.h>
#include <linux/idr.h>
#include <linux/smp_lock.h>
+#include <linux/slab.h>
#include <scsi/scsi.h>
#include <scsi/scsi_cmnd.h>
* Written by: Karen Xie (kxie@chelsio.com)
*/
+#include <linux/slab.h>
#include <linux/skbuff.h>
#include <linux/scatterlist.h>
#ifndef __CXGB3I_ULP2_DDP_H__
#define __CXGB3I_ULP2_DDP_H__
+#include <linux/slab.h>
#include <linux/vmalloc.h>
/**
*/
#include <linux/inet.h>
+#include <linux/slab.h>
#include <linux/crypto.h>
#include <linux/if_vlan.h>
#include <net/dst.h>
*/
#include <linux/if_vlan.h>
+#include <linux/slab.h>
#include <linux/version.h>
#include "cxgb3_defs.h"
* Written by: Karen Xie (kxie@chelsio.com)
*/
+#include <linux/slab.h>
#include <linux/skbuff.h>
#include <linux/crypto.h>
#include <scsi/scsi_cmnd.h>
#include <linux/pci.h>
#include <linux/list.h>
#include <linux/vmalloc.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <scsi/scsi.h>
* Mike Anderson <andmike@linux.vnet.ibm.com>
*/
+#include <linux/slab.h>
#include <scsi/scsi_dh.h>
#include "../scsi_priv.h"
* Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
*
*/
+#include <linux/slab.h>
#include <scsi/scsi.h>
#include <scsi/scsi_eh.h>
#include <scsi/scsi_dh.h>
* along with this program; see the file COPYING. If not, write to
* the Free Software Foundation, 675 Mass Ave, Cambridge, MA 02139, USA.
*/
+#include <linux/slab.h>
#include <scsi/scsi.h>
#include <scsi/scsi_eh.h>
#include <scsi/scsi_dh.h>
* the Free Software Foundation, 675 Mass Ave, Cambridge, MA 02139, USA.
*/
+#include <linux/slab.h>
#include <scsi/scsi.h>
#include <scsi/scsi_dbg.h>
#include <scsi/scsi_eh.h>
#include <scsi/scsi_eh.h>
#include <scsi/scsi_dh.h>
#include <linux/workqueue.h>
+#include <linux/slab.h>
#define RDAC_NAME "rdac"
#define RDAC_RETRY_COUNT 5
#include <linux/ctype.h>
#include <linux/spinlock.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <asm/byteorder.h>
#include <asm/dma.h>
#include <asm/io.h>
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/ioport.h>
-#include <linux/slab.h>
#include <linux/in.h>
#include <linux/pci.h>
#include <linux/proc_fs.h>
#include <linux/if_ether.h>
#include <linux/if_vlan.h>
#include <linux/crc32.h>
+#include <linux/slab.h>
#include <linux/cpu.h>
#include <linux/fs.h>
#include <linux/sysfs.h>
#include <linux/if_vlan.h>
#include <linux/errno.h>
#include <linux/bitops.h>
+#include <linux/slab.h>
#include <net/rtnetlink.h>
#include <scsi/fc/fc_els.h>
#include <linux/delay.h>
#include <linux/mca.h>
#include <linux/spinlock.h>
+#include <linux/slab.h>
#include <scsi/scsicam.h>
#include <linux/mca-legacy.h>
#include <linux/stat.h>
#include <linux/delay.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <scsi/scsicam.h>
#include <asm/system.h>
*/
#include <linux/errno.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/skbuff.h>
#include <linux/interrupt.h>
#include <linux/spinlock.h>
#include <linux/module.h>
#include <linux/mempool.h>
#include <linux/string.h>
+#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/init.h>
#include <linux/pci.h>
#include <linux/if_ether.h>
#include <linux/if_vlan.h>
#include <linux/delay.h>
+#include <linux/gfp.h>
#include <scsi/scsi.h>
#include <scsi/scsi_host.h>
#include <scsi/scsi_device.h>
#include <linux/pci.h>
#include <linux/delay.h>
#include <linux/if_ether.h>
+#include <linux/slab.h>
#include "vnic_resource.h"
#include "vnic_devcmd.h"
#include "vnic_dev.h"
#include <linux/types.h>
#include <linux/pci.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include "vnic_dev.h"
#include "vnic_rq.h"
#include <linux/types.h>
#include <linux/pci.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include "vnic_dev.h"
#include "vnic_wq.h"
#include <linux/dma-mapping.h>
#include <linux/list.h>
#include <linux/smp_lock.h>
+#include <linux/slab.h>
#ifdef GDTH_RTC
#include <linux/mc146818rtc.h>
*/
#include <linux/completion.h>
+#include <linux/slab.h>
int gdth_proc_info(struct Scsi_Host *host, char *buffer,char **start,off_t offset,int length,
int inout)
#include <linux/types.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include <linux/blkdev.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/module.h>
#include <linux/blkdev.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/kthread.h>
#include <linux/string.h>
#include <linux/mm.h>
#include <linux/delay.h>
#include <linux/timer.h>
#include <linux/spinlock.h>
+#include <linux/gfp.h>
#include <asm/uaccess.h>
#include <asm/io.h>
#include <asm/div64.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/kthread.h>
+#include <linux/slab.h>
#include <linux/of.h>
#include <linux/pm.h>
#include <linux/stringify.h>
#include <linux/moduleparam.h>
#include <linux/dma-mapping.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/of.h>
#include <linux/pm.h>
#include <asm/firmware.h>
*/
#include <linux/interrupt.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <scsi/scsi.h>
#include <scsi/scsi_host.h>
#include <scsi/scsi_transport_srp.h>
#include <asm/iommu.h>
#include <asm/hvcall.h>
#include <linux/dma-mapping.h>
+#include <linux/gfp.h>
#include <linux/interrupt.h>
#include "ibmvscsi.h"
#include <linux/parport.h>
#include <linux/workqueue.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <scsi/scsi.h>
#include <linux/types.h>
#include <linux/errno.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/ioport.h>
#include <linux/delay.h>
#include <linux/pci.h>
#include <linux/types.h>
#include <linux/inet.h>
+#include <linux/slab.h>
#include <linux/file.h>
#include <linux/blkdev.h>
#include <linux/crypto.h>
*/
#include <linux/kernel.h>
+#include <linux/gfp.h>
#include <linux/types.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/blkdev.h>
#include <linux/ioport.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <asm/page.h>
#include <asm/pgtable.h>
*/
#include <linux/timer.h>
+#include <linux/slab.h>
#include <linux/err.h>
#include <asm/unaligned.h>
*/
#include <linux/timer.h>
-#include <linux/gfp.h>
+#include <linux/slab.h>
#include <linux/err.h>
#include <scsi/fc/fc_fc2.h>
#include <linux/scatterlist.h>
#include <linux/err.h>
#include <linux/crc32.h>
+#include <linux/slab.h>
#include <scsi/scsi_tcq.h>
#include <scsi/scsi.h>
#include <linux/kernel.h>
#include <linux/skbuff.h>
#include <linux/crc32.h>
+#include <linux/gfp.h>
#include <scsi/fc_frame.h>
*/
#include <linux/timer.h>
+#include <linux/slab.h>
#include <asm/unaligned.h>
#include <scsi/fc/fc_gs.h>
#include <linux/kernel.h>
#include <linux/spinlock.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <linux/rcupdate.h>
#include <linux/timer.h>
#include <linux/workqueue.h>
#include <linux/kfifo.h>
#include <linux/delay.h>
#include <linux/log2.h>
+#include <linux/slab.h>
#include <asm/unaligned.h>
#include <net/tcp.h>
#include <scsi/scsi_cmnd.h>
session->state = ISCSI_STATE_TERMINATE;
else if (conn->stop_stage != STOP_CONN_RECOVER)
session->state = ISCSI_STATE_IN_RECOVERY;
+
+ old_stop_stage = conn->stop_stage;
+ conn->stop_stage = flag;
spin_unlock_bh(&session->lock);
del_timer_sync(&conn->transport_timer);
iscsi_suspend_tx(conn);
spin_lock_bh(&session->lock);
- old_stop_stage = conn->stop_stage;
- conn->stop_stage = flag;
conn->c_stage = ISCSI_CONN_STOPPED;
spin_unlock_bh(&session->lock);
#include <linux/types.h>
#include <linux/list.h>
#include <linux/inet.h>
+#include <linux/slab.h>
#include <linux/file.h>
#include <linux/blkdev.h>
#include <linux/crypto.h>
*/
#include <linux/scatterlist.h>
+#include <linux/slab.h>
#include <scsi/sas_ata.h>
#include "sas_internal.h"
*/
#include <linux/scatterlist.h>
+#include <linux/slab.h>
#include <scsi/scsi_host.h>
#include <scsi/scsi_eh.h>
#include "sas_internal.h"
#include <linux/scatterlist.h>
#include <linux/blkdev.h>
+#include <linux/slab.h>
#include "sas_internal.h"
*/
#include <linux/scatterlist.h>
#include <linux/blkdev.h>
+#include <linux/slab.h>
#include "sas_internal.h"
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/device.h>
#include <linux/spinlock.h>
#include <linux/err.h>
#include <linux/blkdev.h>
#include <linux/freezer.h>
+#include <linux/gfp.h>
#include <linux/scatterlist.h>
#include <linux/libata.h>
* 02110-1301 USA
*/
#include <linux/err.h>
+#include <linux/slab.h>
#include <linux/kfifo.h>
#include <linux/scatterlist.h>
#include <linux/dma-mapping.h>
#include <linux/pci.h>
#include <linux/interrupt.h>
#include <linux/aer.h>
+#include <linux/gfp.h>
#include <scsi/scsi.h>
#include <scsi/scsi_device.h>
#include <linux/interrupt.h>
#include <linux/mempool.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <scsi/scsi.h>
#include <linux/blkdev.h>
#include <linux/pci.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <linux/utsname.h>
#include <scsi/scsi.h>
#include <linux/idr.h>
#include <linux/interrupt.h>
#include <linux/kthread.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/spinlock.h>
#include <linux/ctype.h>
/* See Fibre Channel protocol T11 FC-LS for details */
#include <linux/blkdev.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/interrupt.h>
#include <scsi/scsi.h>
*******************************************************************/
#include <linux/blkdev.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/kthread.h>
#include <linux/interrupt.h>
#include <linux/spinlock.h>
#include <linux/ctype.h>
#include <linux/aer.h>
+#include <linux/slab.h>
#include <scsi/scsi.h>
#include <scsi/scsi_device.h>
#include <linux/blkdev.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/interrupt.h>
#include <scsi/scsi_device.h>
*******************************************************************/
#include <linux/mempool.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/interrupt.h>
#include <linux/blkdev.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/interrupt.h>
#include <scsi/scsi.h>
* included with this package. *
*******************************************************************/
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/delay.h>
#include <asm/unaligned.h>
#include <linux/pci.h>
#include <linux/interrupt.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <scsi/scsi.h>
#include <scsi/scsi_cmnd.h>
#include <linux/interrupt.h>
#include <linux/kthread.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <scsi/scsi.h>
#include <linux/delay.h>
#include <linux/io.h>
#include <linux/nubus.h>
+#include <linux/slab.h>
#include <asm/irq.h>
#include <asm/dma.h>
#include <linux/init.h>
#include <linux/dma-mapping.h>
#include <linux/smp_lock.h>
+#include <linux/slab.h>
#include <scsi/scsicam.h>
#include "scsi.h"
* For history of changes, see Documentation/ChangeLog.megaraid
*/
+#include <linux/slab.h>
#include "megaraid_mbox.h"
static int megaraid_init(void);
* Common management module
*/
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/smp_lock.h>
#include "megaraid_mm.h"
#include <linux/delay.h>
#include <linux/smp_lock.h>
#include <linux/uio.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <linux/fs.h>
#include <linux/compat.h>
#include <linux/delay.h>
#include <linux/types.h>
#include <linux/string.h>
-#include <linux/slab.h>
#include <linux/blkdev.h>
#include <linux/proc_fs.h>
#include <linux/stat.h>
#include <linux/workqueue.h>
#include <linux/delay.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include "mpt2sas_base.h"
#include <linux/pci.h>
#include <linux/interrupt.h>
#include <linux/raid_class.h>
+#include <linux/slab.h>
#include "mpt2sas_base.h"
#include <linux/workqueue.h>
#include <linux/delay.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <scsi/scsi.h>
#include <scsi/scsi_cmnd.h>
#include <linux/platform_device.h>
#include <linux/init.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <asm/mvme16xhw.h>
#include <scsi/scsi_host.h>
#include <scsi/scsi_device.h>
#include <linux/platform_device.h>
#include <linux/interrupt.h>
#include <linux/irq.h>
+#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <scsi/libsas.h>
#include <scsi/scsi_tcq.h>
#include <linux/delay.h>
#include <linux/dma-mapping.h>
#include <linux/errno.h>
+#include <linux/gfp.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/ioport.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/timer.h>
#include <linux/ioport.h>
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
+#include <linux/slab.h>
+
#include <scsi/osd_initiator.h>
#include <scsi/osd_sec.h>
#include <scsi/osd_attributes.h>
#include <linux/idr.h>
#include <linux/major.h>
#include <linux/file.h>
+#include <linux/slab.h>
#include <scsi/scsi.h>
#include <scsi/scsi_driver.h>
#include <linux/sched.h>
#include <linux/proc_fs.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/string.h>
#include <linux/errno.h>
*
*/
#include <linux/firmware.h>
+#include <linux/slab.h>
#include "pm8001_sas.h"
#include "pm8001_ctl.h"
* POSSIBILITY OF SUCH DAMAGES.
*
*/
+ #include <linux/slab.h>
#include "pm8001_sas.h"
#include "pm8001_hwi.h"
#include "pm8001_chips.h"
*
*/
+#include <linux/slab.h>
#include "pm8001_sas.h"
#include "pm8001_chips.h"
*
*/
+#include <linux/slab.h>
#include "pm8001_sas.h"
/**
#include <linux/hdreg.h>
#include <linux/version.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <asm/irq.h>
#include <asm/processor.h>
#include <linux/libata.h>
#include <linux/init.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/blkdev.h>
#include <linux/parport.h>
#include <linux/cdrom.h>
#include <linux/highmem.h>
+#include <linux/slab.h>
#include <scsi/scsi.h>
#include <scsi/scsi_cmnd.h>
* General Public License for more details.
*
******************************************************************************/
-#define QLA1280_VERSION "3.27"
+#define QLA1280_VERSION "3.27.1"
/*****************************************************************************
Revision History:
+ Rev 3.27.1, February 8, 2010, Michael Reed
+ - Retain firmware image for error recovery.
Rev 3.27, February 10, 2009, Michael Reed
- General code cleanup.
- Improve error recovery.
#include <linux/pci.h>
#include <linux/proc_fs.h>
#include <linux/stat.h>
-#include <linux/slab.h>
#include <linux/pci_ids.h>
#include <linux/interrupt.h>
#include <linux/init.h>
/*****************************************/
struct qla_boards {
- unsigned char name[9]; /* Board ID String */
+ char *name; /* Board ID String */
int numPorts; /* Number of SCSI ports */
- char *fwname; /* firmware name */
+ int fw_index; /* index into qla1280_fw_tbl for firmware */
};
/* NOTE: the last argument in each entry is used to index ql1280_board_tbl */
};
MODULE_DEVICE_TABLE(pci, qla1280_pci_tbl);
+DEFINE_MUTEX(qla1280_firmware_mutex);
+
+struct qla_fw {
+ char *fwname;
+ const struct firmware *fw;
+};
+
+#define QL_NUM_FW_IMAGES 3
+
+struct qla_fw qla1280_fw_tbl[QL_NUM_FW_IMAGES] = {
+ {"qlogic/1040.bin", NULL}, /* image 0 */
+ {"qlogic/1280.bin", NULL}, /* image 1 */
+ {"qlogic/12160.bin", NULL}, /* image 2 */
+};
+
+/* NOTE: Order of boards in this table must match order in qla1280_pci_tbl */
static struct qla_boards ql1280_board_tbl[] = {
- /* Name , Number of ports, FW details */
- {"QLA12160", 2, "qlogic/12160.bin"},
- {"QLA1040", 1, "qlogic/1040.bin"},
- {"QLA1080", 1, "qlogic/1280.bin"},
- {"QLA1240", 2, "qlogic/1280.bin"},
- {"QLA1280", 2, "qlogic/1280.bin"},
- {"QLA10160", 1, "qlogic/12160.bin"},
- {" ", 0, " "},
+ {.name = "QLA12160", .numPorts = 2, .fw_index = 2},
+ {.name = "QLA1040" , .numPorts = 1, .fw_index = 0},
+ {.name = "QLA1080" , .numPorts = 1, .fw_index = 1},
+ {.name = "QLA1240" , .numPorts = 2, .fw_index = 1},
+ {.name = "QLA1280" , .numPorts = 2, .fw_index = 1},
+ {.name = "QLA10160", .numPorts = 1, .fw_index = 2},
+ {.name = " ", .numPorts = 0, .fw_index = -1},
};
static int qla1280_verbose = 1;
return status;
}
+/*
+ * qla1280_request_firmware
+ * Acquire firmware for chip. Retain in memory
+ * for error recovery.
+ *
+ * Input:
+ * ha = adapter block pointer.
+ *
+ * Returns:
+ * Pointer to firmware image or an error code
+ * cast to pointer via ERR_PTR().
+ */
+static const struct firmware *
+qla1280_request_firmware(struct scsi_qla_host *ha)
+{
+ const struct firmware *fw;
+ int err;
+ int index;
+ char *fwname;
+
+ spin_unlock_irq(ha->host->host_lock);
+ mutex_lock(&qla1280_firmware_mutex);
+
+ index = ql1280_board_tbl[ha->devnum].fw_index;
+ fw = qla1280_fw_tbl[index].fw;
+ if (fw)
+ goto out;
+
+ fwname = qla1280_fw_tbl[index].fwname;
+ err = request_firmware(&fw, fwname, &ha->pdev->dev);
+
+ if (err) {
+ printk(KERN_ERR "Failed to load image \"%s\" err %d\n",
+ fwname, err);
+ fw = ERR_PTR(err);
+ goto unlock;
+ }
+ if ((fw->size % 2) || (fw->size < 6)) {
+ printk(KERN_ERR "Invalid firmware length %zu in image \"%s\"\n",
+ fw->size, fwname);
+ release_firmware(fw);
+ fw = ERR_PTR(-EINVAL);
+ goto unlock;
+ }
+
+ qla1280_fw_tbl[index].fw = fw;
+
+ out:
+ ha->fwver1 = fw->data[0];
+ ha->fwver2 = fw->data[1];
+ ha->fwver3 = fw->data[2];
+ unlock:
+ mutex_unlock(&qla1280_firmware_mutex);
+ spin_lock_irq(ha->host->host_lock);
+ return fw;
+}
+
/*
* Chip diagnostics
* Test chip for proper operation.
static int
qla1280_load_firmware_pio(struct scsi_qla_host *ha)
{
+ /* enter with host_lock acquired */
+
const struct firmware *fw;
const __le16 *fw_data;
uint16_t risc_address, risc_code_size;
uint16_t mb[MAILBOX_REGISTER_COUNT], i;
- int err;
+ int err = 0;
+
+ fw = qla1280_request_firmware(ha);
+ if (IS_ERR(fw))
+ return PTR_ERR(fw);
- spin_unlock_irq(ha->host->host_lock);
- err = request_firmware(&fw, ql1280_board_tbl[ha->devnum].fwname,
- &ha->pdev->dev);
- spin_lock_irq(ha->host->host_lock);
- if (err) {
- printk(KERN_ERR "Failed to load image \"%s\" err %d\n",
- ql1280_board_tbl[ha->devnum].fwname, err);
- return err;
- }
- if ((fw->size % 2) || (fw->size < 6)) {
- printk(KERN_ERR "Bogus length %zu in image \"%s\"\n",
- fw->size, ql1280_board_tbl[ha->devnum].fwname);
- err = -EINVAL;
- goto out;
- }
- ha->fwver1 = fw->data[0];
- ha->fwver2 = fw->data[1];
- ha->fwver3 = fw->data[2];
fw_data = (const __le16 *)&fw->data[0];
ha->fwstart = __le16_to_cpu(fw_data[2]);
if (err) {
printk(KERN_ERR "scsi(%li): Failed to load firmware\n",
ha->host_no);
- goto out;
+ break;
}
}
-out:
- release_firmware(fw);
+
return err;
}
static int
qla1280_load_firmware_dma(struct scsi_qla_host *ha)
{
+ /* enter with host_lock acquired */
const struct firmware *fw;
const __le16 *fw_data;
uint16_t risc_address, risc_code_size;
return -ENOMEM;
#endif
- spin_unlock_irq(ha->host->host_lock);
- err = request_firmware(&fw, ql1280_board_tbl[ha->devnum].fwname,
- &ha->pdev->dev);
- spin_lock_irq(ha->host->host_lock);
- if (err) {
- printk(KERN_ERR "Failed to load image \"%s\" err %d\n",
- ql1280_board_tbl[ha->devnum].fwname, err);
- return err;
- }
- if ((fw->size % 2) || (fw->size < 6)) {
- printk(KERN_ERR "Bogus length %zu in image \"%s\"\n",
- fw->size, ql1280_board_tbl[ha->devnum].fwname);
- err = -EINVAL;
- goto out;
- }
- ha->fwver1 = fw->data[0];
- ha->fwver2 = fw->data[1];
- ha->fwver3 = fw->data[2];
+ fw = qla1280_request_firmware(ha);
+ if (IS_ERR(fw))
+ return PTR_ERR(fw);
+
fw_data = (const __le16 *)&fw->data[0];
ha->fwstart = __le16_to_cpu(fw_data[2]);
#if DUMP_IT_BACK
pci_free_consistent(ha->pdev, 8000, tbuf, p_tbuf);
#endif
- release_firmware(fw);
return err;
}
static int
qla1280_load_firmware(struct scsi_qla_host *ha)
{
+ /* enter with host_lock taken */
int err;
err = qla1280_chip_diag(ha);
static void __exit
qla1280_exit(void)
{
+ int i;
+
pci_unregister_driver(&qla1280_pci_driver);
+ /* release any allocated firmware images */
+ for (i = 0; i < QL_NUM_FW_IMAGES; i++) {
+ if (qla1280_fw_tbl[i].fw) {
+ release_firmware(qla1280_fw_tbl[i].fw);
+ qla1280_fw_tbl[i].fw = NULL;
+ }
+ }
}
module_init(qla1280_init);
#include <linux/kthread.h>
#include <linux/vmalloc.h>
+#include <linux/slab.h>
#include <linux/delay.h>
static int qla24xx_vport_disable(struct fc_vport *, bool);
int rval = QLA_FUNCTION_FAILED;
uint16_t state[5];
- if (!vha->hw->flags.eeh_busy)
+ if (test_bit(ABORT_ISP_ACTIVE, &vha->dpc_flags) ||
+ test_bit(ISP_ABORT_NEEDED, &vha->dpc_flags))
+ DEBUG2_3_11(printk("%s(%ld): isp reset in progress.\n",
+ __func__, vha->host_no));
+ else if (!vha->hw->flags.eeh_busy)
rval = qla2x00_get_firmware_state(vha, state);
if (rval != QLA_SUCCESS)
memset(state, -1, sizeof(state));
/* Offset 384. */
uint8_t reserved_21[16];
- uint16_t reserved_22[8];
+ uint16_t reserved_22[3];
+
+ /*
+ * BIT 0 = Extended BB credits for LR
+ * BIT 1 = Virtual Fabric Enable
+ * BIT 2 = Enhanced Features Unused
+ * BIT 3-7 = Enhanced Features Reserved
+ */
+ /* Enhanced Features */
+ uint8_t enhanced_features;
+
+ uint8_t reserved_23;
+ uint16_t reserved_24[4];
/* Offset 416. */
- uint16_t reserved_23[32];
+ uint16_t reserved_25[32];
/* Offset 480. */
uint8_t model_name[16];
/* Offset 496. */
uint16_t feature_mask_l;
uint16_t feature_mask_h;
- uint16_t reserved_24[2];
+ uint16_t reserved_26[2];
uint16_t subsystem_vendor_id;
uint16_t subsystem_device_id;
#include "qla_gbl.h"
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/vmalloc.h>
#include "qla_devtbl.h"
#include "qla_def.h"
#include <linux/delay.h>
+#include <linux/slab.h>
#include <scsi/scsi_tcq.h>
#include <scsi/scsi_bsg_fc.h>
* vp_idx does not match
* Event is not global, vp_idx does not match
*/
- if ((mb[1] == 0xffff && (mb[3] & 0xff) != 0xff)
- || (mb[1] != 0xffff)) {
- if (vha->vp_idx != (mb[3] & 0xff))
- break;
- }
+ if (IS_QLA2XXX_MIDTYPE(ha) &&
+ ((mb[1] == 0xffff && (mb[3] & 0xff) != 0xff) ||
+ (mb[1] != 0xffff)) && vha->vp_idx != (mb[3] & 0xff))
+ break;
/* Global event -- port logout or port unavailable. */
if (mb[1] == 0xffff && mb[2] == 0x7) {
/* If possible, enable MSI-X. */
if (!IS_QLA2432(ha) && !IS_QLA2532(ha) &&
- !IS_QLA8432(ha) && !IS_QLA8001(ha))
- goto skip_msix;
+ !IS_QLA8432(ha) && !IS_QLA8001(ha))
+ goto skip_msi;
+
+ if (ha->pdev->subsystem_vendor == PCI_VENDOR_ID_HP &&
+ (ha->pdev->subsystem_device == 0x7040 ||
+ ha->pdev->subsystem_device == 0x7041 ||
+ ha->pdev->subsystem_device == 0x1705)) {
+ DEBUG2(qla_printk(KERN_WARNING, ha,
+ "MSI-X: Unsupported ISP2432 SSVID/SSDID (0x%X,0x%X).\n",
+ ha->pdev->subsystem_vendor,
+ ha->pdev->subsystem_device));
+ goto skip_msi;
+ }
if (IS_QLA2432(ha) && (ha->pdev->revision < QLA_MSIX_CHIP_REV_24XX ||
!QLA_MSIX_FW_MODE_1(ha->fw_attributes))) {
DEBUG2(qla_printk(KERN_WARNING, ha,
"MSI-X: Unsupported ISP2432 (0x%X, 0x%X).\n",
ha->pdev->revision, ha->fw_attributes));
-
goto skip_msix;
}
- if (ha->pdev->subsystem_vendor == PCI_VENDOR_ID_HP &&
- (ha->pdev->subsystem_device == 0x7040 ||
- ha->pdev->subsystem_device == 0x7041 ||
- ha->pdev->subsystem_device == 0x1705)) {
- DEBUG2(qla_printk(KERN_WARNING, ha,
- "MSI-X: Unsupported ISP2432 SSVID/SSDID (0x%X, 0x%X).\n",
- ha->pdev->subsystem_vendor,
- ha->pdev->subsystem_device));
-
- goto skip_msi;
- }
-
ret = qla24xx_enable_msix(ha, rsp);
if (!ret) {
DEBUG2(qla_printk(KERN_INFO, ha,
#include "qla_def.h"
#include <linux/delay.h>
+#include <linux/gfp.h>
/*
return rval;
}
+#define EXTENDED_BB_CREDITS BIT_0
/*
* qla2x00_execute_fw
* Start adapter firmware.
mcp->mb[1] = MSW(risc_addr);
mcp->mb[2] = LSW(risc_addr);
mcp->mb[3] = 0;
- mcp->mb[4] = 0;
+ if (IS_QLA81XX(ha)) {
+ struct nvram_81xx *nv = ha->nvram;
+ mcp->mb[4] = (nv->enhanced_features &
+ EXTENDED_BB_CREDITS);
+ } else
+ mcp->mb[4] = 0;
mcp->out_mb |= MBX_4|MBX_3|MBX_2|MBX_1;
mcp->in_mb |= MBX_1;
} else {
#include <linux/moduleparam.h>
#include <linux/vmalloc.h>
+#include <linux/slab.h>
#include <linux/list.h>
#include <scsi/scsi_tcq.h>
#include <linux/kthread.h>
#include <linux/mutex.h>
#include <linux/kobject.h>
+#include <linux/slab.h>
#include <scsi/scsi_tcq.h>
#include <scsi/scsicam.h>
/* Determine queue resources */
ha->max_req_queues = ha->max_rsp_queues = 1;
- if ((ql2xmaxqueues <= 1 || ql2xmultique_tag < 1) &&
+ if ((ql2xmaxqueues <= 1 && !ql2xmultique_tag) ||
+ (ql2xmaxqueues > 1 && ql2xmultique_tag) ||
(!IS_QLA25XX(ha) && !IS_QLA81XX(ha)))
goto mqiobase_exit;
+
ha->mqiobase = ioremap(pci_resource_start(ha->pdev, 3),
pci_resource_len(ha->pdev, 3));
if (ha->mqiobase) {
#include "qla_def.h"
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <asm/uaccess.h>
/*
* Driver version
*/
-#define QLA2XXX_VERSION "8.03.02-k1"
+#define QLA2XXX_VERSION "8.03.02-k2"
#define QLA_DRIVER_MAJOR_VER 8
#define QLA_DRIVER_MINOR_VER 3
#define QLA_DRIVER_PATCH_VER 2
-#define QLA_DRIVER_BETA_VER 1
+#define QLA_DRIVER_BETA_VER 2
* See LICENSE.qla4xxx for copyright and licensing details.
*/
#include <linux/moduleparam.h>
+#include <linux/slab.h>
#include <scsi/scsi_tcq.h>
#include <scsi/scsicam.h>
#include <linux/delay.h>
#include <linux/types.h>
#include <linux/string.h>
-#include <linux/slab.h>
+#include <linux/gfp.h>
#include <linux/blkdev.h>
#include <linux/proc_fs.h>
#include <linux/stat.h>
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/timer.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/string.h>
#include <linux/genhd.h>
#include <linux/moduleparam.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include <scsi/scsi_device.h>
#include <scsi/scsi_devinfo.h>
#include <linux/module.h>
#include <linux/sched.h>
+#include <linux/gfp.h>
#include <linux/timer.h>
#include <linux/string.h>
#include <linux/kernel.h>
#include <linux/jiffies.h>
#include <linux/security.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/netlink.h>
#include <linux/init.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/proc_fs.h>
#include <linux/errno.h>
#include <linux/blkdev.h>
#include <linux/seq_file.h>
#include <linux/mutex.h>
+#include <linux/gfp.h>
#include <asm/uaccess.h>
#include <scsi/scsi.h>
#include <linux/kthread.h>
#include <linux/spinlock.h>
#include <linux/async.h>
+#include <linux/slab.h>
#include <scsi/scsi.h>
#include <scsi/scsi_cmnd.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/blkdev.h>
#include <linux/device.h>
* 02110-1301 USA
*/
#include <linux/miscdevice.h>
+#include <linux/gfp.h>
#include <linux/file.h>
#include <linux/smp_lock.h>
#include <net/tcp.h>
#include <linux/hash.h>
#include <linux/module.h>
#include <linux/pagemap.h>
+#include <linux/slab.h>
#include <scsi/scsi.h>
#include <scsi/scsi_cmnd.h>
#include <scsi/scsi_device.h>
*/
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <scsi/scsi_device.h>
#include <scsi/scsi_host.h>
if (rport && (rport->port_state != FC_PORTSTATE_ONLINE)) {
req->errors = -ENXIO;
spin_unlock_irq(q->queue_lock);
- blk_end_request(req, -ENXIO, blk_rq_bytes(req));
+ blk_end_request_all(req, -ENXIO);
spin_lock_irq(q->queue_lock);
continue;
}
ret = fc_req_to_bsgjob(shost, rport, req);
if (ret) {
req->errors = ret;
- blk_end_request(req, ret, blk_rq_bytes(req));
+ blk_end_request_all(req, ret);
spin_lock_irq(q->queue_lock);
continue;
}
*/
#include <linux/module.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <net/tcp.h>
#include <scsi/scsi.h>
#include <scsi/scsi_host.h>
#include <linux/blkdev.h>
#include <linux/mutex.h>
#include <linux/sysfs.h>
+#include <linux/slab.h>
#include <scsi/scsi.h>
#include "scsi_priv.h"
#include <scsi/scsi_device.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/fs.h>
#include <linux/genhd.h>
#include <linux/kernel.h>
#include <linux/mutex.h>
#include <linux/string_helpers.h>
#include <linux/async.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/unaligned.h>
**-----------------------------------------------------------------------------
*/
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/enclosure.h>
#include <linux/errno.h>
#include <linux/mtio.h>
#include <linux/ioctl.h>
+#include <linux/slab.h>
#include <linux/fcntl.h>
#include <linux/init.h>
#include <linux/poll.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/blkdev.h>
#include <linux/device.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/stat.h>
#include <linux/mm.h>
#include <linux/blkdev.h>
#include <linux/init.h>
#include <linux/blkdev.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <scsi/scsi.h>
#include <linux/blkpg.h>
#include <linux/cdrom.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/uaccess.h>
#include <linux/string.h>
#include <linux/bcd.h>
#include <linux/blkdev.h>
+#include <linux/slab.h>
#include <scsi/scsi.h>
#include <scsi/scsi_cmnd.h>
#include <linux/mm.h>
#include <linux/init.h>
#include <linux/string.h>
+#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/mtio.h>
#include <linux/cdrom.h>
#include <linux/errno.h>
#include <linux/kernel.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/time.h>
#include <linux/pci.h>
#include <linux/blkdev.h>
* interrupt or bottom half.
*/
+#include <linux/gfp.h>
#include <linux/workqueue.h>
#include <linux/interrupt.h>
*/
#include <linux/kernel.h>
+#include <linux/gfp.h>
#include <linux/types.h>
#include <linux/delay.h>
#include <linux/module.h>
#include <linux/dma-mapping.h>
#include <linux/of.h>
#include <linux/of_device.h>
+#include <linux/gfp.h>
#include <asm/irq.h>
#include <asm/io.h>
#include <linux/interrupt.h>
#include <linux/init.h>
#include <linux/spinlock.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <scsi/scsi.h>
#include <linux/init.h>
#include <linux/ctype.h>
#include <linux/spinlock.h>
+#include <linux/slab.h>
#include <asm/dma.h>
#include <asm/irq.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <linux/workqueue.h>
#include <linux/pci.h>
#include <linux/kernel.h>
#include <linux/types.h>
#include <linux/string.h>
-#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/ioport.h>
#include <linux/proc_fs.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/zorro.h>
+#include <linux/slab.h>
#include <asm/amigahw.h>
#include <asm/amigaints.h>
#include <linux/pm.h>
#include <linux/bitops.h>
#include <linux/delay.h>
+#include <linux/gfp.h>
#include <asm/io.h>
#include <asm/irq.h>
#include <linux/serial_8250.h>
#include <linux/nmi.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/irq.h>
#include <linux/module.h>
#include <linux/serial_core.h>
#include <linux/signal.h>
-#include <linux/slab.h>
#include <linux/types.h>
#include <asm/hardware.h>
#include <linux/delay.h>
#include <linux/dio.h>
#include <linux/console.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include "8250.h"
#include <linux/amba/bus.h>
#include <linux/amba/serial.h>
#include <linux/clk.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <linux/amba/bus.h>
#include <linux/amba/serial.h>
#include <linux/clk.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/sizes.h>
#include <linux/module.h>
#include <linux/ioport.h>
+#include <linux/gfp.h>
#include <linux/io.h>
#include <linux/init.h>
#include <linux/console.h>
#include <linux/init.h>
#include <linux/console.h>
#include <linux/sysrq.h>
+#include <linux/slab.h>
#include <linux/platform_device.h>
#include <linux/tty.h>
#include <linux/tty_flip.h>
#include <linux/module.h>
#include <linux/tty.h>
+#include <linux/gfp.h>
#include <linux/ioport.h>
#include <linux/init.h>
#include <linux/serial.h>
#include <linux/module.h>
#include <linux/tty.h>
#include <linux/ioport.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/serial.h>
#include <linux/console.h>
void __iomem *pram;
unsigned long offset;
struct resource res;
- unsigned long len;
+ resource_size_t len;
/* Don't remap parameter RAM if it has already been initialized
* during console setup.
if (of_address_to_resource(np, 1, &res))
return NULL;
- len = 1 + res.end - res.start;
+ len = resource_size(&res);
pram = ioremap(res.start, len);
if (!pram)
return NULL;
#include <linux/clk.h>
#include <linux/delay.h>
#include <linux/rational.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/irq.h>
#include <linux/pci.h>
#include <linux/serial_core.h>
#include <linux/ioc3.h>
+#include <linux/slab.h>
/*
* Interesting things about the ioc3
#include <linux/pci.h>
#include <linux/ioc4.h>
#include <linux/serial_core.h>
+#include <linux/slab.h>
/*
* interesting things about the ioc4
***********************************************************************/
#include <linux/moduleparam.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include "jsm.h"
#include <linux/serial_reg.h>
#include <linux/delay.h> /* For udelay */
#include <linux/pci.h>
+#include <linux/slab.h>
#include "jsm.h"
#define MAX_MAX3100 4
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/device.h>
#include <linux/serial_core.h>
#include <linux/serial.h>
#include <linux/dma-mapping.h>
#include <linux/mv643xx.h>
#include <linux/platform_device.h>
+#include <linux/gfp.h>
#include <asm/io.h>
#include <asm/irq.h>
#include <linux/init.h>
#include <linux/serial.h>
#include <linux/console.h>
-#include <linux/slab.h>
#include <linux/delay.h> /* for udelay */
#include <linux/device.h>
#include <asm/io.h>
*/
#include <linux/init.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/serial_core.h>
#include <linux/serial_8250.h>
#include <linux/of_platform.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/console.h>
-#include <linux/slab.h>
#include <linux/adb.h>
#include <linux/pmu.h>
#include <linux/bitops.h>
#include <linux/serial_core.h>
#include <linux/clk.h>
#include <linux/io.h>
+#include <linux/slab.h>
struct uart_pxa_port {
struct uart_port port;
PCMCIA_PFC_DEVICE_PROD_ID13(1, "Xircom", "REM10", 0x2e3ee845, 0x76df1d29),
PCMCIA_PFC_DEVICE_PROD_ID13(1, "Xircom", "XEM5600", 0x2e3ee845, 0xf1403719),
PCMCIA_PFC_DEVICE_PROD_ID12(1, "AnyCom", "Fast Ethernet + 56K COMBO", 0x578ba6e7, 0xb0ac62c4),
+ PCMCIA_PFC_DEVICE_PROD_ID12(1, "ATKK", "LM33-PCM-T", 0xba9eb7e2, 0x077c174e),
PCMCIA_PFC_DEVICE_PROD_ID12(1, "D-Link", "DME336T", 0x1a424a1c, 0xb23897ff),
PCMCIA_PFC_DEVICE_PROD_ID12(1, "Gateway 2000", "XJEM3336", 0xdd9989be, 0x662c394c),
PCMCIA_PFC_DEVICE_PROD_ID12(1, "Grey Cell", "GCS3000", 0x2a151fac, 0x48b932ae),
#include <linux/list.h>
#include <linux/dmaengine.h>
#include <linux/scatterlist.h>
-#include <linux/timer.h>
+#include <linux/slab.h>
#ifdef CONFIG_SUPERH
#include <asm/sh_bios.h>
if ((ssr_status & SCxSR_BRK(port)) && err_enabled)
ret = sci_br_interrupt(irq, ptr);
- WARN_ONCE(ret == IRQ_NONE,
- "%s: %d IRQ %d, status %x, control %x\n", __func__,
- irq, port->line, ssr_status, scr_status);
-
return ret;
}
# define SCSCR_INIT(port) (port->mapbase == SCIF2) ? 0xF3 : 0xF0
#elif defined(CONFIG_CPU_SUBTYPE_SH7720) || \
defined(CONFIG_CPU_SUBTYPE_SH7721) || \
- defined(CONFIG_ARCH_SHMOBILE)
+ defined(CONFIG_ARCH_SH7367) || \
+ defined(CONFIG_ARCH_SH7377) || \
+ defined(CONFIG_ARCH_SH7372)
# define SCSCR_INIT(port) 0x0030 /* TIE=0,RIE=0,TE=1,RE=1 */
# define PORT_PTCR 0xA405011EUL
# define PORT_PVCR 0xA4050122UL
# define SCSCR_INIT(port) 0x0038 /* TIE=0,RIE=0,TE=1,RE=1,REIE=1 */
#elif defined(CONFIG_CPU_SUBTYPE_SH7724)
# define SCIF_ORER 0x0001 /* overrun error bit */
-# define SCSCR_INIT(port) 0x0038 /* TIE=0,RIE=0,TE=1,RE=1,REIE=1 */
+# define SCSCR_INIT(port) ((port)->type == PORT_SCIFA ? \
+ 0x30 /* TIE=0,RIE=0,TE=1,RE=1 */ : \
+ 0x38 /* TIE=0,RIE=0,TE=1,RE=1,REIE=1 */ )
#elif defined(CONFIG_CPU_SUBTYPE_SH4_202)
# define SCSPTR2 0xffe80020 /* 16 bit SCIF */
# define SCIF_ORER 0x0001 /* overrun error bit */
defined(CONFIG_CPU_SUBTYPE_SH7786) || \
defined(CONFIG_CPU_SUBTYPE_SHX3)
#define SCI_CTRL_FLAGS_REIE 0x08 /* 7750 SCIF */
+#elif defined(CONFIG_CPU_SUBTYPE_SH7724)
+#define SCI_CTRL_FLAGS_REIE ((port)->type == PORT_SCIFA ? 0 : 8)
#else
#define SCI_CTRL_FLAGS_REIE 0
#endif
#if defined(CONFIG_CPU_SUBTYPE_SH7705) || \
defined(CONFIG_CPU_SUBTYPE_SH7720) || \
defined(CONFIG_CPU_SUBTYPE_SH7721) || \
- defined(CONFIG_ARCH_SHMOBILE)
+ defined(CONFIG_ARCH_SH7367) || \
+ defined(CONFIG_ARCH_SH7377) || \
+ defined(CONFIG_ARCH_SH7372)
# define SCIF_ORER 0x0200
# define SCIF_ERRORS ( SCIF_PER | SCIF_FER | SCIF_ER | SCIF_BRK | SCIF_ORER)
# define SCIF_RFDC_MASK 0x007f
#if defined(CONFIG_CPU_SUBTYPE_SH7705) || \
defined(CONFIG_CPU_SUBTYPE_SH7720) || \
defined(CONFIG_CPU_SUBTYPE_SH7721) || \
- defined(CONFIG_ARCH_SHMOBILE)
+ defined(CONFIG_ARCH_SH7367) || \
+ defined(CONFIG_ARCH_SH7377) || \
+ defined(CONFIG_ARCH_SH7372)
# define SCxSR_RDxF_CLEAR(port) (sci_in(port, SCxSR) & 0xfffc)
# define SCxSR_ERROR_CLEAR(port) (sci_in(port, SCxSR) & 0xfd73)
# define SCxSR_TDxE_CLEAR(port) (sci_in(port, SCxSR) & 0xffdf)
SCI_OUT(sci_size, sci_offset, value); \
}
-#if defined(CONFIG_CPU_SH3) || defined(CONFIG_ARCH_SHMOBILE)
+#if defined(CONFIG_CPU_SH3) || \
+ defined(CONFIG_ARCH_SH7367) || \
+ defined(CONFIG_ARCH_SH7377) || \
+ defined(CONFIG_ARCH_SH7372)
#if defined(CONFIG_CPU_SUBTYPE_SH7710) || defined(CONFIG_CPU_SUBTYPE_SH7712)
#define SCIx_FNS(name, sh3_sci_offset, sh3_sci_size, sh4_sci_offset, sh4_sci_size, \
sh3_scif_offset, sh3_scif_size, sh4_scif_offset, sh4_scif_size, \
#elif defined(CONFIG_CPU_SUBTYPE_SH7705) || \
defined(CONFIG_CPU_SUBTYPE_SH7720) || \
defined(CONFIG_CPU_SUBTYPE_SH7721) || \
- defined(CONFIG_ARCH_SHMOBILE)
+ defined(CONFIG_ARCH_SH7367) || \
+ defined(CONFIG_ARCH_SH7377) || \
+ defined(CONFIG_ARCH_SH7372)
#define SCIF_FNS(name, scif_offset, scif_size) \
CPU_SCIF_FNS(name, scif_offset, scif_size)
#else
#if defined(CONFIG_CPU_SUBTYPE_SH7705) || \
defined(CONFIG_CPU_SUBTYPE_SH7720) || \
defined(CONFIG_CPU_SUBTYPE_SH7721) || \
- defined(CONFIG_ARCH_SHMOBILE)
+ defined(CONFIG_ARCH_SH7367) || \
+ defined(CONFIG_ARCH_SH7377) || \
+ defined(CONFIG_ARCH_SH7372)
SCIF_FNS(SCSMR, 0x00, 16)
SCIF_FNS(SCBRR, 0x04, 8)
#elif defined(CONFIG_CPU_SUBTYPE_SH7705) || \
defined(CONFIG_CPU_SUBTYPE_SH7720) || \
defined(CONFIG_CPU_SUBTYPE_SH7721) || \
- defined(CONFIG_ARCH_SHMOBILE)
+ defined(CONFIG_ARCH_SH7367) || \
+ defined(CONFIG_ARCH_SH7377) || \
+ defined(CONFIG_ARCH_SH7372)
#define SCBRR_VALUE(bps, clk) (((clk*2)+16*bps)/(32*bps)-1)
#elif defined(CONFIG_CPU_SUBTYPE_SH7723) ||\
defined(CONFIG_CPU_SUBTYPE_SH7724)
#include <linux/serial.h>
#include <linux/sysrq.h>
#include <linux/console.h>
+#include <linux/slab.h>
#ifdef CONFIG_SERIO
#include <linux/serio.h>
#endif
if (up->su_type == SU_PORT_KBD || up->su_type == SU_PORT_MS) {
err = sunsu_kbd_ms_init(up);
if (err) {
+ of_iounmap(&op->resource[0],
+ up->port.membase, up->reg_size);
kfree(up);
- goto out_unmap;
+ return err;
}
dev_set_drvdata(&op->dev, up);
#include <linux/kernel.h>
#include <linux/platform_device.h>
#include <linux/ioport.h>
+#include <linux/slab.h>
#include "timbuart.h"
#include <linux/module.h>
#include <linux/serial.h>
+#include <linux/slab.h>
#include <linux/serial_core.h>
#include <linux/io.h>
#include <linux/of_platform.h>
* Shared interrupt handling code for IPR and INTC2 types of IRQs.
*
* Copyright (C) 2007, 2008 Magnus Damm
- * Copyright (C) 2009 Paul Mundt
+ * Copyright (C) 2009, 2010 Paul Mundt
*
* Based on intc2.c and ipr.c
*
#include <linux/irq.h>
#include <linux/module.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/sh_intc.h>
#include <linux/sysdev.h>
#include <linux/list.h>
#include <linux/topology.h>
#include <linux/bitmap.h>
+#include <linux/cpumask.h>
#define _INTC_MK(fn, mode, addr_e, addr_d, width, shift) \
((shift) | ((width) << 5) | ((fn) << 9) | ((mode) << 13) | \
unsigned int cpu;
for (cpu = 0; cpu < SMP_NR(d, _INTC_ADDR_E(handle)); cpu++) {
+#ifdef CONFIG_SMP
+ if (!cpumask_test_cpu(cpu, irq_to_desc(irq)->affinity))
+ continue;
+#endif
addr = INTC_REG(d, _INTC_ADDR_E(handle), cpu);
intc_enable_fns[_INTC_MODE(handle)](addr, handle, intc_reg_fns\
[_INTC_FN(handle)], irq);
unsigned int cpu;
for (cpu = 0; cpu < SMP_NR(d, _INTC_ADDR_D(handle)); cpu++) {
+#ifdef CONFIG_SMP
+ if (!cpumask_test_cpu(cpu, irq_to_desc(irq)->affinity))
+ continue;
+#endif
addr = INTC_REG(d, _INTC_ADDR_D(handle), cpu);
intc_disable_fns[_INTC_MODE(handle)](addr, handle,intc_reg_fns\
[_INTC_FN(handle)], irq);
return 0; /* allow wakeup, but setup hardware in intc_suspend() */
}
+#ifdef CONFIG_SMP
+/*
+ * This is held with the irq desc lock held, so we don't require any
+ * additional locking here at the intc desc level. The affinity mask is
+ * later tested in the enable/disable paths.
+ */
+static int intc_set_affinity(unsigned int irq, const struct cpumask *cpumask)
+{
+ if (!cpumask_intersects(cpumask, cpu_online_mask))
+ return -1;
+
+ cpumask_copy(irq_to_desc(irq)->affinity, cpumask);
+
+ return 0;
+}
+#endif
+
static void intc_mask_ack(unsigned int irq)
{
struct intc_desc_int *d = get_intc_desc(irq);
d->chip.shutdown = intc_disable;
d->chip.set_type = intc_set_sense;
d->chip.set_wake = intc_set_wake;
+#ifdef CONFIG_SMP
+ d->chip.set_affinity = intc_set_affinity;
+#endif
if (hw->ack_regs) {
for (i = 0; i < hw->nr_ack_regs; i++)
#include <linux/delay.h>
#include <linux/ioc3.h>
#include <linux/rwsem.h>
+#include <linux/slab.h>
#define IOC3_PCI_SIZE 0x100000
#include <linux/amba/bus.h>
#include <linux/amba/pl022.h>
#include <linux/io.h>
+#include <linux/slab.h>
/*
* This macro is used to define some register default values.
#include <linux/err.h>
#include <linux/interrupt.h>
#include <linux/spi/spi.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <mach/board.h>
#include <linux/init.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/device.h>
#include <linux/platform_device.h>
#include <linux/dma-mapping.h>
#include <linux/spi/spi.h>
#include <linux/spi/spi_bitbang.h>
+#include <linux/slab.h>
#include <mach/spi.h>
#include <mach/edma.h>
#include <linux/interrupt.h>
#include <linux/highmem.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/spi/dw_spi.h>
#include <linux/spi/spi.h>
#include <linux/clk.h>
#include <linux/interrupt.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/spi/dw_spi.h>
#include <linux/spi/spi.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/spi/dw_spi.h>
#include <linux/spi/spi.h>
#include <linux/delay.h>
#include <linux/spi/spi.h>
#include <linux/fsl_devices.h>
+#include <linux/slab.h>
#include <asm/mpc52xx.h>
#include <asm/mpc52xx_psc.h>
#include <linux/of_spi.h>
#include <linux/io.h>
#include <linux/of_gpio.h>
+#include <linux/slab.h>
#include <asm/time.h>
#include <asm/mpc52xx.h>
#include <linux/err.h>
#include <linux/clk.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <linux/spi/spi.h>
#include <linux/clk.h>
#include <linux/io.h>
#include <linux/gpio.h>
+#include <linux/slab.h>
#include <linux/spi/spi.h>
#include <linux/interrupt.h>
#include <linux/err.h>
#include <linux/clk.h>
+#include <linux/slab.h>
#include <linux/spi/spi.h>
#include <linux/spi/spi_bitbang.h>
#include <linux/delay.h>
#include <linux/clk.h>
#include <linux/gpio.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/irq.h>
#include <linux/init.h>
#include <linux/cache.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <linux/mod_devicetable.h>
#include <linux/spi/spi.h>
#include <linux/module.h>
#include <linux/delay.h>
#include <linux/device.h>
+#include <linux/slab.h>
#include <linux/io.h>
#include <linux/ioport.h>
#include <linux/irq.h>
#include <linux/delay.h>
#include <linux/errno.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/spi/spi.h>
#include <linux/spi/spi_bitbang.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/spi/spi.h>
#include <linux/spi/spi_bitbang.h>
#include <linux/types.h>
#include <linux/gpio.h>
#include <linux/of_gpio.h>
#include <linux/of_spi.h>
+#include <linux/slab.h>
#include <sysdev/fsl_soc.h>
#include <asm/cpm.h>
#include <linux/platform_device.h>
#include <linux/gpio.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <linux/spi/spi.h>
#include <linux/spi/spi_bitbang.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/wait.h>
#include <linux/of_platform.h>
#include <linux/platform_device.h>
#include <linux/gpio.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <linux/spi/spi.h>
#include <linux/spi/spi_bitbang.h>
#include <linux/device.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/spi/spi.h>
#include <linux/spi/tle62x0.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <linux/of_platform.h>
#include <linux/of_device.h>
#include <linux/ssb/ssb_driver_gige.h>
#include <linux/pci.h>
#include <linux/pci_regs.h>
+#include <linux/slab.h>
/*
#include <linux/dma-mapping.h>
#include <linux/pci.h>
#include <linux/mmc/sdio_func.h>
+#include <linux/slab.h>
#include <pcmcia/cs_types.h>
#include <pcmcia/cs.h>
#include <linux/ssb/ssb.h>
#include <linux/ssb/ssb_regs.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/delay.h>
*/
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/ssb/ssb.h>
#include "ssb_private.h"
#include <linux/ctype.h>
+#include <linux/slab.h>
static const struct ssb_sprom *fallback_sprom;
*/
#include <linux/device.h>
+#include <linux/slab.h>
#include "main.h"
#include "device.h"
#include "send.h"
#include <linux/kthread.h> /* kernel threads */
#include <linux/pkt_sched.h> /* schedule types */
#include <linux/workqueue.h> /* workqueue */
+#include <linux/slab.h>
#include <net/sock.h> /* struct sock */
#include <linux/jiffies.h>
#include "types.h"
#include "translation-table.h"
#include "types.h"
#include "hash.h"
+#include <linux/slab.h>
#include <linux/ethtool.h>
#include <linux/etherdevice.h>
#include "../comedidev.h"
#include <linux/ioport.h>
+#include <linux/slab.h>
#define _8255_SIZE 4
#include <linux/module.h>
#include <linux/sched.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/ioport.h>
#include <linux/delay.h>
#include <linux/timex.h>
#include <linux/timer.h>
#include <linux/pci.h>
+#include <linux/gfp.h>
#include "../../comedidev.h"
#include <asm/io.h>
#if defined(CONFIG_APCI_1710) || defined(CONFIG_APCI_3200) || defined(CONFIG_APCI_3300)
#include "../pci_ids.h"
#include <linux/delay.h>
+#include <linux/gfp.h>
#include <linux/interrupt.h>
#include "amcc_s5933.h"
*/
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include "../comedidev.h"
*/
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include "../comedidev.h"
*/
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include "../comedidev.h"
#include <linux/delay.h>
#include <linux/pci.h>
#include "../comedilib.h"
#include "../comedidev.h"
#include <linux/string.h>
+#include <linux/slab.h>
/* The maxiumum number of channels per subdevice. */
#define MAX_CHANS 256
#include <linux/delay.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include "das08.h"
*/
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/interrupt.h>
#include <asm/dma.h>
#include "../comedidev.h"
*/
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include "../comedidev.h"
#include <linux/ioport.h>
#include "../comedidev.h"
+#include <linux/gfp.h>
#include <linux/ioport.h>
#include <linux/interrupt.h>
#include <asm/dma.h>
#include <linux/ctype.h>
#include <linux/firmware.h>
#include <linux/jiffies.h>
+#include <linux/slab.h>
#include <linux/timer.h>
#include "comedi_pci.h"
#include "jr3_pci.h"
#define DEBUG 1
#define DEBUG_FLAGS
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include "../comedidev.h"
#include "mite.h"
*/
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include "../comedidev.h"
#include "mite.h"
*/
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include "../comedidev.h"
#include <linux/ioport.h>
*/
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include "../comedidev.h"
#include <linux/ioport.h>
#undef LABPC_DEBUG
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include "../comedidev.h"
#include <linux/ioport.h>
/* #define LABPC_DEBUG enable debugging messages */
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include "../comedidev.h"
#include <linux/delay.h>
#include "../comedidev.h"
#include <linux/delay.h>
+#include <linux/slab.h>
#include "8253.h"
#include "8255.h"
*/
#include <linux/interrupt.h>
+#include <linux/gfp.h>
#include "../comedidev.h"
#include <linux/delay.h>
#include <linux/ioport.h>
#include <linux/mc146818rtc.h>
+#include <linux/gfp.h>
#include <linux/delay.h>
#include <asm/dma.h>
#include <linux/ioport.h>
#include <linux/mc146818rtc.h>
+#include <linux/gfp.h>
#include <linux/delay.h>
#include <asm/dma.h>
*/
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include "../comedidev.h"
#include "pcm_common.h"
#include <linux/pci.h> /* for PCI devices */
*/
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include "../comedidev.h"
#include "pcm_common.h"
#include <linux/delay.h>
#include <linux/ioport.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <asm/termios.h>
#include <asm/ioctls.h>
#include "../comedidev.h"
#include <linux/ioport.h>
+#include <linux/slab.h>
#define DRIVER_NAME "unioxx5"
#define UNIOXX5_SIZE 0x10
#include <linux/delay.h>
#include <linux/ioport.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <asm/io.h>
#include "../comedi.h"
#include <linux/delay.h>
#include <linux/ioport.h>
#include <linux/mm.h>
-#include <linux/slab.h>
/* functions specific to kcomedilib */
**********************************************************************/
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include "crystalhd_hw.h"
***************************************************************************/
#include <linux/version.h>
+#include <linux/slab.h>
#include "crystalhd_lnx.h"
* along with this driver. If not, see <http://www.gnu.org/licenses/>.
**********************************************************************/
+#include <linux/slab.h>
+
#include "crystalhd_misc.h"
#include "crystalhd_lnx.h"
#include <linux/vmalloc.h>
#include <linux/dma-mapping.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <asm/delay.h>
#include <sound/core.h>
#include <linux/file.h>
#include <linux/fcntl.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
MODULE_DESCRIPTION("v4l2 driver module for cx25821 based TV cards");
* Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
+#include <linux/slab.h>
+
#include "cx25821-video.h"
static void buffer_queue(struct videobuf_queue *vq, struct videobuf_buffer *vb)
*/
#include <linux/i2c.h>
+#include <linux/slab.h>
#include "cx25821.h"
#include "cx25821-sram.h"
#include "cx25821-video.h"
#include <linux/syscalls.h>
#include <linux/file.h>
#include <linux/fcntl.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
MODULE_DESCRIPTION("v4l2 driver module for cx25821 based TV cards");
#include <linux/syscalls.h>
#include <linux/file.h>
#include <linux/fcntl.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
MODULE_DESCRIPTION("v4l2 driver module for cx25821 based TV cards");
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/sched.h>
#include <mach/board.h>
#include <linux/spinlock.h>
#include <linux/videodev2.h>
#include <linux/proc_fs.h>
+#include <linux/slab.h>
#include <media/v4l2-dev.h>
#include <media/msm_camera.h>
#include <mach/camera.h>
#include <linux/fs.h>
#include <linux/sched.h>
#include <linux/android_pmem.h>
+#include <linux/slab.h>
#include <mach/msm_adsp.h>
#include <linux/delay.h>
#include <linux/wait.h>
/*
* Copyright (C) 2008-2009 QUALCOMM Incorporated.
*/
+#include <linux/slab.h>
#include <linux/uaccess.h>
#include <linux/interrupt.h>
#include <mach/irqs.h>
*/
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/i2c.h>
#include <linux/uaccess.h>
#include <linux/delay.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/i2c.h>
#include <linux/uaccess.h>
#include <linux/miscdevice.h>
#include <linux/delay.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/i2c.h>
#include <linux/uaccess.h>
#include <linux/miscdevice.h>
*/
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/i2c.h>
#include <linux/uaccess.h>
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/gpio.h>
#include <linux/gpio_event.h>
#include <linux/interrupt.h>
*/
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/input.h>
#include <linux/gpio_event.h>
#include <linux/hrtimer.h>
#include <linux/input.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
enum {
DEBOUNCE_UNSTABLE = BIT(0), /* Got irq, while debouncing */
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/gpio.h>
#include <linux/gpio_event.h>
#include <linux/hrtimer.h>
#include <linux/android_pmem.h>
#include <linux/mempolicy.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/uaccess.h>
#include <asm/cacheflush.h>
#include <linux/kernel.h>
#include <linux/kthread.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/uaccess.h>
#include <linux/wait.h>
#include <linux/list.h>
#include <linux/platform_device.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/uaccess.h>
#include "adsp.h"
#include <linux/kthread.h>
#include <linux/wait.h>
#include <linux/dma-mapping.h>
+#include <linux/gfp.h>
#include <linux/delay.h>
#include <linux/kthread.h>
#include <linux/wait.h>
#include <linux/dma-mapping.h>
+#include <linux/gfp.h>
#include <linux/delay.h>
#include <linux/wait.h>
#include <linux/dma-mapping.h>
#include <linux/delay.h>
+#include <linux/gfp.h>
#include <asm/atomic.h>
#include <asm/ioctls.h>
#include <linux/kthread.h>
#include <linux/wait.h>
#include <linux/dma-mapping.h>
+#include <linux/gfp.h>
#include <linux/delay.h>
#include <linux/kthread.h>
#include <linux/wait.h>
#include <linux/dma-mapping.h>
+#include <linux/gfp.h>
#include <linux/delay.h>
#include <linux/debugfs.h>
#include <linux/delay.h>
#include <linux/wakelock.h>
+#include <linux/gfp.h>
#include <linux/msm_audio.h>
#include <linux/sched.h>
#include <linux/wait.h>
#include <linux/dma-mapping.h>
+#include <linux/gfp.h>
#include <asm/ioctls.h>
#include <mach/msm_adsp.h>
#include <linux/module.h>
#include <linux/fs.h>
#include <linux/uaccess.h>
+#include <linux/slab.h>
#include <linux/kthread.h>
#include <linux/wait.h>
#include <linux/err.h>
#include <linux/sched.h>
#include <linux/poll.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/byteorder.h>
#include <linux/platform_device.h>
#include <linux/poll.h>
#include <linux/platform_device.h>
#include <linux/msm_rpcrouter.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/byteorder.h>
#include <linux/delay.h>
#include <linux/platform_device.h>
#include <linux/wakelock.h>
+#include <linux/slab.h>
#include <linux/msm_rpcrouter.h>
#include <linux/uaccess.h>
#include <linux/module.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#ifdef CONFIG_HAS_EARLYSUSPEND
#include <linux/earlysuspend.h>
#endif
#include <linux/types.h>
#include <linux/mm.h> /* PAGE_ALIGN() */
#include <linux/io.h>
+#include <linux/slab.h>
#include <asm/page.h>
*/
#include <asm/system.h>
-#include <linux/slab.h>
+#include <linux/gfp.h>
#include <linux/sched.h>
#include <linux/types.h>
#include <linux/sched.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/ctype.h>
#include <linux/string.h>
#include <linux/timer.h>
#include <linux/sched.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/ctype.h>
#include <linux/string.h>
#include <linux/timer.h>
}
/* Enable TXMAC */
- ctl |= 0x05; /* TX mac enable, FC disable */
+ ctl |= 0x09; /* TX mac enable, FC disable */
writel(ctl, &etdev->regs->txmac.ctl);
/* Ready to start the RXDMA/TXDMA engine */
#include <linux/sched.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/ctype.h>
#include <linux/string.h>
#include <linux/timer.h>
#include <linux/sched.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/ctype.h>
#include <linux/string.h>
#include <linux/timer.h>
#include <linux/sched.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/ctype.h>
#include <linux/string.h>
#include <linux/timer.h>
#include <linux/sched.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/ctype.h>
#include <linux/string.h>
#include <linux/timer.h>
#include <linux/sched.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/ctype.h>
#include <linux/string.h>
#include <linux/timer.h>
#include <linux/firmware.h>
#include <linux/mutex.h>
#include <linux/uaccess.h>
+#include <linux/slab.h>
#include <asm/system.h>
#include <linux/videodev2.h>
#include <media/tuner.h>
#include <linux/device.h>
#include <linux/i2c.h>
#include <linux/firmware.h>
+#include <linux/slab.h>
#include <asm/byteorder.h>
#include "go7007-priv.h"
#include <linux/delay.h>
#include <linux/sched.h>
#include <linux/spinlock.h>
+#include <linux/slab.h>
#include <linux/fs.h>
#include <linux/unistd.h>
#include <linux/time.h>
#include <linux/usb.h>
#include <linux/i2c.h>
#include <linux/videodev2.h>
+#include <linux/slab.h>
#include <media/v4l2-device.h>
#include <media/v4l2-common.h>
#include <media/v4l2-i2c-drv.h>
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/smp_lock.h>
#include <linux/usb.h>
#include <dvb-usb.h>
#include <linux/i2c.h>
#include <linux/mutex.h>
#include <linux/uaccess.h>
+#include <linux/slab.h>
#include <asm/system.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <linux/i2c.h>
#include <linux/videodev2.h>
#include <linux/ioctl.h>
+#include <linux/slab.h>
#include "wis-i2c.h"
#include <linux/i2c.h>
#include <linux/videodev2.h>
#include <linux/ioctl.h>
+#include <linux/slab.h>
#include "wis-i2c.h"
#include <linux/init.h>
#include <linux/i2c.h>
#include <linux/videodev2.h>
+#include <linux/slab.h>
#include <media/tuner.h>
#include <media/v4l2-common.h>
#include <media/v4l2-ioctl.h>
#include <linux/i2c.h>
#include <linux/videodev2.h>
#include <linux/ioctl.h>
+#include <linux/slab.h>
#include "wis-i2c.h"
#include <linux/i2c.h>
#include <linux/videodev2.h>
#include <linux/ioctl.h>
+#include <linux/slab.h>
#include "wis-i2c.h"
*/
#include <linux/kernel.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include "osd.h"
#include "logging.h"
#include "VmbusPrivate.h"
*/
#include <linux/kernel.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include <linux/list.h>
#include "osd.h"
#include "logging.h"
*/
#include <linux/kernel.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include <linux/vmalloc.h>
#include "osd.h"
#include "logging.h"
*/
#include <linux/kernel.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include <linux/vmalloc.h>
#include "osd.h"
#include "logging.h"
#include <linux/mm.h>
#include <linux/delay.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include "osd.h"
#include "logging.h"
#include "NetVsc.h"
*/
#include <linux/kernel.h>
#include <linux/highmem.h>
+#include <linux/slab.h>
#include <linux/io.h>
#include "osd.h"
#include "logging.h"
*/
#include <linux/kernel.h>
#include <linux/string.h>
+#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/delay.h>
#include "osd.h"
*/
#include <linux/kernel.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include "osd.h"
#include "logging.h"
#include "VersionInfo.h"
#include <linux/major.h>
#include <linux/delay.h>
#include <linux/hdreg.h>
+#include <linux/slab.h>
#include <scsi/scsi.h>
#include <scsi/scsi_cmnd.h>
#include <scsi/scsi_eh.h>
#include <linux/etherdevice.h>
#include <linux/skbuff.h>
#include <linux/in.h>
+#include <linux/slab.h>
#include <net/arp.h>
#include <net/route.h>
#include <net/sock.h>
#include <linux/time.h>
#include <linux/io.h>
#include <linux/bitops.h>
+#include <linux/slab.h>
#include "osd.h"
struct osd_callback_struct {
* Hank Janssen <hjanssen@microsoft.com>
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/device.h>
#include <linux/blkdev.h>
#include <linux/sysctl.h>
#include <linux/pci.h>
#include <linux/dmi.h>
+#include <linux/slab.h>
#include "VersionInfo.h"
#include "osd.h"
#include "logging.h"
#include <linux/sysfs.h>
#include <linux/rtc.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include "../iio.h"
#include "../sysfs.h"
#include <linux/device.h>
#include <linux/kernel.h>
#include <linux/spi/spi.h>
+#include <linux/slab.h>
#include <linux/sysfs.h>
#include <linux/list.h>
#include <linux/spi/spi.h>
#include <linux/sysfs.h>
#include <linux/list.h>
+#include <linux/slab.h>
#include "../iio.h"
#include "../sysfs.h"
#include <linux/gpio.h>
#include <linux/fs.h>
#include <linux/device.h>
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/spi/spi.h>
#include <linux/sysfs.h>
#include <linux/gpio.h>
#include <linux/fs.h>
#include <linux/device.h>
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/spi/spi.h>
#include <linux/sysfs.h>
#include <linux/i2c.h>
#include <linux/rtc.h>
#include <linux/regulator/consumer.h>
+#include <linux/slab.h>
#include "../iio.h"
#include "../sysfs.h"
#include <linux/gpio.h>
#include <linux/workqueue.h>
#include <linux/device.h>
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/sysfs.h>
#include <linux/list.h>
#include <linux/sched.h>
#include <linux/wait.h>
#include <linux/cdev.h>
+#include <linux/slab.h>
#include "iio.h"
#include "trigger_consumer.h"
#include <linux/module.h>
#include <linux/cdev.h>
#include <linux/idr.h>
+#include <linux/slab.h>
#include "iio.h"
#include "ring_generic.h"
#include <linux/device.h>
#include <linux/interrupt.h>
#include <linux/list.h>
+#include <linux/slab.h>
#include "iio.h"
#include "trigger.h"
#include <linux/pm.h>
#include <linux/hwmon.h>
#include <linux/err.h>
+#include <linux/slab.h>
#include "../iio.h"
#include "tsl2563.h"
* the Free Software Foundation.
*/
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/device.h>
#include <linux/platform_device.h>
#include <linux/interrupt.h>
#include <linux/gpio.h>
+#include <linux/slab.h>
#include "../iio.h"
#include "../trigger.h"
#include <linux/platform_device.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/rtc.h>
#include "../iio.h"
#include "../trigger.h"
#include "driver.h"
+#include <linux/slab.h>
+
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include "audio.h"
*/
#include "driver.h"
+
+#include <linux/slab.h>
+
#include "dumprequest.h"
#include "driver.h"
#include <linux/usb.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/rawmidi.h>
#include "driver.h"
+#include <linux/slab.h>
+
#include <sound/core.h>
#include <sound/control.h>
#include <sound/pcm.h>
#include "driver.h"
+#include <linux/slab.h>
+
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include "driver.h"
+#include <linux/slab.h>
+
#include "audio.h"
#include "capture.h"
#include "control.h"
#include "driver.h"
+#include <linux/slab.h>
+
#include "audio.h"
#include "control.h"
#include "variax.h"
#include <linux/ptrace.h>
#include <linux/ioport.h>
#include <linux/in.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/timer.h>
#include <linux/errno.h>
**********************************************************************/
#include <linux/kernel.h>
#include <linux/netdevice.h>
+#include <linux/slab.h>
#include <asm/octeon/octeon.h>
#include <linux/netdevice.h>
#include <linux/etherdevice.h>
#include <linux/phy.h>
+#include <linux/slab.h>
#include <net/dst.h>
/* Platform dependent. */
/* */
/************************************************************************/
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/if_arp.h>
#include <linux/uaccess.h>
#include "linux/netlink.h"
#include "linux/rtnetlink.h"
+#include "linux/slab.h"
#include <net/iw_handler.h>
#include <linux/uaccess.h>
#include <linux/wireless.h>
#include <linux/if_arp.h>
+#include <linux/slab.h>
#include <linux/io.h>
#include "zdcompat.h"
#include "usbdrv.h"
#include <linux/netlink.h>
+#include <linux/slab.h>
#include <net/iw_handler.h>
/* Memory management */
#include "usbdrv.h"
#include <linux/netlink.h>
+#include <linux/gfp.h>
#include <net/iw_handler.h>
#include "usbdrv.h"
#include <linux/netlink.h>
+#include <linux/slab.h>
#include <net/iw_handler.h>
extern void zfLnxInitUsbTxQ(zdev_t *dev);
#include "usbdrv.h"
#include <linux/netlink.h>
+#include <linux/slab.h>
#include <net/iw_handler.h>
extern void zfiRecv80211(zdev_t* dev, zbuf_t* buf, struct zsAdditionInfo* addInfo);
#endif
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include "usbdrv.h"
#include <linux/ioctl.h>
#include <linux/io.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include "poch.h"
#include <linux/mutex.h>
#include <linux/string.h>
#include <linux/in.h>
+#include <linux/slab.h>
#include "netfs.h"
#include <linux/fs.h>
#include <linux/jhash.h>
#include <linux/namei.h>
+#include <linux/slab.h>
#include <linux/pagemap.h>
#include "netfs.h"
#include <linux/backing-dev.h>
#include <linux/fs.h>
#include <linux/fsnotify.h>
-#include <linux/slab.h>
#include <linux/mempool.h>
#include "netfs.h"
#include <linux/kthread.h>
#include <linux/pagemap.h>
#include <linux/poll.h>
+#include <linux/slab.h>
#include <linux/swap.h>
#include <linux/syscalls.h>
#include <linux/vmalloc.h>
*/
#include <linux/module.h>
-#include <linux/slab.h>
#include <linux/fs.h>
#include <linux/ktime.h>
#include <linux/fs_struct.h>
#include <linux/device.h>
#include <linux/genhd.h>
#include <linux/highmem.h>
+#include <linux/slab.h>
#include <linux/lzo.h>
#include <linux/string.h>
#include <linux/swap.h>
#include "rt_config.h"
#include <linux/pci.h>
+#include <linux/slab.h>
/* Following information will be show when you run 'modinfo' */
/* *** If you have a solution for the bug in current version of driver, please mail to me. */
#include <linux/firmware.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include "rt_config.h"
unsigned long RTDebugLevel = RT_DEBUG_ERROR;
#include <linux/random.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/version.h>
#include <asm/uaccess.h>
******************************************************************************/
#include <linux/wireless.h>
#include <linux/kmod.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include "ieee80211.h"
#undef RX_DONT_PASS_UL
#undef DUMMY_RX
+#include <linux/slab.h>
#include <linux/syscalls.h>
#include <linux/eeprom_93cx6.h>
#include <linux/random.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/version.h>
#include <asm/uaccess.h>
#ifdef ENABLE_DOT11D
#include <linux/wireless.h>
#include <linux/version.h>
#include <linux/kmod.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include "ieee80211.h"
#include "ieee80211.h"
#include <linux/etherdevice.h>
+#include <linux/slab.h>
#include "rtl819x_TS.h"
#if LINUX_VERSION_CODE < KERNEL_VERSION(2,5,0)
//#define CONFIG_RTL8192_IO_MAP
#include <linux/vmalloc.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include "r8192E_hw.h"
#include "r8192E.h"
#include <linux/random.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/version.h>
#include <asm/uaccess.h>
#include "dot11d.h"
******************************************************************************/
#include <linux/wireless.h>
#include <linux/kmod.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include "ieee80211.h"
#include "ieee80211.h"
#include <linux/etherdevice.h>
+#include <linux/slab.h>
#include "rtl819x_TS.h"
void TsSetupTimeOut(unsigned long data)
*/
#include <linux/vmalloc.h>
+#include <linux/slab.h>
#undef LOOP_TEST
#undef DUMP_RX
#include <linux/random.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/version.h>
#include <asm/uaccess.h>
#ifdef ENABLE_DOT11D
#include <linux/wireless.h>
#include <linux/version.h>
#include <linux/kmod.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include "ieee80211.h"
#include "ieee80211.h"
#include <linux/etherdevice.h>
+#include <linux/slab.h>
#include "rtl819x_TS.h"
void TsSetupTimeOut(unsigned long data)
#include "r8192U_dm.h"
//#include "r8192xU_phyreg.h"
#include <linux/usb.h>
+#include <linux/slab.h>
// FIXME: check if 2.6.7 is ok
#ifdef CONFIG_RTL8192_PM
#include <linux/sched.h>
#include <linux/pci.h>
#include <linux/firmware.h>
+#include <linux/slab.h>
#include <asm/ioctl.h>
#include <linux/ioport.h>
#include <asm/io.h>
#include <linux/fb.h>
#include <linux/pci.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/uaccess.h>
#include <linux/console.h>
#include <linux/screen_info.h>
#include <linux/serialP.h>
#include <linux/rcupdate.h>
#include <linux/compat.h>
+#include <linux/slab.h>
#include <net/arp.h>
#include <net/net_namespace.h>
#include <linux/mm.h>
#include <linux/fb.h>
#include <linux/vmalloc.h>
+#include <linux/slab.h>
#include "udlfb.h"
* USA.
*/
+#include <linux/slab.h>
+
#include "usbip_common.h"
#include "stub.h"
* USA.
*/
+#include <linux/slab.h>
#include "usbip_common.h"
#include "stub.h"
* USA.
*/
+#include <linux/slab.h>
+
#include "usbip_common.h"
#include "stub.h"
#include "../../usb/core/hcd.h"
* USA.
*/
+#include <linux/slab.h>
+
#include "usbip_common.h"
#include "stub.h"
#include <linux/tcp.h>
#include <linux/in.h>
#include <linux/kthread.h>
+#include <linux/slab.h>
#include "usbip_common.h"
/* version information */
* USA.
*/
+#include <linux/slab.h>
#include "usbip_common.h"
#include "vhci.h"
* USA.
*/
+#include <linux/slab.h>
+
#include "usbip_common.h"
#include "vhci.h"
* USA.
*/
+#include <linux/slab.h>
+
#include "usbip_common.h"
#include "vhci.h"
#include <linux/interrupt.h>
#include <linux/spinlock.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <asm/time.h>
#include <asm/io.h>
#include <asm/uaccess.h>
#include <linux/interrupt.h>
#include <linux/spinlock.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <asm/time.h>
#include <asm/io.h>
#include <asm/uaccess.h>
#include <linux/pagemap.h>
#include <linux/pci.h>
#include <linux/semaphore.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/syscalls.h>
#include <linux/types.h>
#include <linux/syscalls.h>
#include <linux/mutex.h>
#include <linux/spinlock.h>
+#include <linux/slab.h>
#include "vme.h"
#include "vme_bridge.h"
#include "iowpa.h"
#include <linux/delay.h>
#include <linux/kthread.h>
+#include <linux/slab.h>
//#define DEBUG
/*--------------------- Static Definitions -------------------------*/
#include "wb35reg_f.h"
#include <linux/usb.h>
+#include <linux/slab.h>
extern void phy_calibration_winbond(struct hw_data *phw_data, u32 frequency);
//
//============================================================================
#include <linux/usb.h>
+#include <linux/slab.h>
#include "core.h"
#include "sysdef.h"
//
//============================================================================
#include <linux/usb.h>
+#include <linux/gfp.h>
#include "wb35tx_f.h"
#include "mds_f.h"
#include <linux/kernel.h>
#include <linux/sched.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/ctype.h>
#include <linux/string.h>
#include <linux/timer.h>
#include <wl_version.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/kernel.h>
// #include <linux/sched.h>
#include <linux/init.h>
#include <linux/sched.h>
#include <linux/ptrace.h>
-#include <linux/slab.h>
#include <linux/ctype.h>
#include <linux/string.h>
//#include <linux/timer.h>
#include <linux/if_arp.h>
#include <linux/ioport.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <asm/uaccess.h>
#include <linux/sched.h>
#include <linux/types.h>
#include <linux/skbuff.h>
-#include <linux/slab.h>
#include <linux/wireless.h>
#include <linux/netdevice.h>
#include <linux/etherdevice.h>
#include <linux/netdevice.h>
#include <linux/wireless.h>
-#include <linux/slab.h>
#include <linux/random.h>
#include <linux/kernel.h>
#include <linux/kernel.h>
#include <linux/sched.h>
#include <linux/types.h>
-#include <linux/slab.h>
#include <linux/netdevice.h>
#include <linux/etherdevice.h>
#include <linux/wireless.h>
/*================================================================*/
/* System Includes */
#include <linux/ihex.h>
+#include <linux/slab.h>
/*================================================================*/
/* Local Constants */
#include <linux/wait.h>
#include <linux/sched.h>
#include <linux/types.h>
-#include <linux/slab.h>
#include <linux/wireless.h>
#include <linux/netdevice.h>
#include <linux/delay.h>
#include <linux/kernel.h>
#include <linux/sched.h>
#include <linux/types.h>
-#include <linux/slab.h>
#include <linux/wireless.h>
#include <linux/netdevice.h>
#include <linux/io.h>
#include <linux/kernel.h>
#include <linux/list.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include <linux/tc.h>
#include <linux/types.h>
#include <linux/module.h>
#include <linux/device.h>
#include <linux/err.h>
+#include <linux/slab.h>
#include <linux/kdev_t.h>
#include <linux/idr.h>
#include <linux/thermal.h>
#include <linux/init.h>
#include <linux/poll.h>
#include <linux/device.h>
+#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/idr.h>
#include <linux/sched.h>
#include <linux/io.h>
#include <linux/uaccess.h>
#include <linux/uio_driver.h>
+#include <linux/slab.h>
#define PCI_VENDOR_ID_AEC 0xaecb
#define PCI_DEVICE_ID_AEC_VITCLTC 0x6250
#include <linux/device.h>
#include <linux/module.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/uio_driver.h>
#include <asm/io.h>
#include <linux/io.h>
#include <linux/module.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/uio_driver.h>
#define PCI_VENDOR_ID_HILSCHER 0x15CF
#include <linux/device.h>
#include <linux/module.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/uio_driver.h>
#include <linux/spinlock.h>
#include <linux/platform_device.h>
#include <linux/uio_driver.h>
#include <linux/stringify.h>
+#include <linux/slab.h>
#define DRIVER_NAME "uio_pdrv"
#include <linux/interrupt.h>
#include <linux/stringify.h>
#include <linux/pm_runtime.h>
+#include <linux/slab.h>
#define DRIVER_NAME "uio_pdrv_genirq"
#include <linux/pci.h>
#include <linux/uio_driver.h>
#include <linux/io.h>
+#include <linux/slab.h>
/* ID's for SERCOS III PCI card (PLX 9030) */
#define SERCOS_SUB_VENDOR_ID 0x1971
#include <linux/device.h>
#include <linux/errno.h>
#include <linux/firmware.h>
-#include <linux/gfp.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/kthread.h>
#include <linux/mutex.h>
#include <linux/freezer.h>
+#include <linux/slab.h>
#include <asm/unaligned.h>
#include <linux/device.h>
#include <linux/io.h>
#include <linux/list.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include <linux/usb/c67x00.h>
*/
#include <linux/kthread.h>
+#include <linux/slab.h>
#include "c67x00.h"
#include "c67x00-hcd.h"
wb = acm->delayed_wb;
acm->delayed_wb = NULL;
spin_unlock_irq(&acm->write_lock);
- acm_start_wb(acm, acm->delayed_wb);
+ acm_start_wb(acm, wb);
} else {
spin_unlock_irq(&acm->write_lock);
}
#define WDM_READ 4
#define WDM_INT_STALL 5
#define WDM_POLL_RUNNING 6
-
+#define WDM_RESPONDING 7
+#define WDM_SUSPENDING 8
#define WDM_MAX 16
int count;
dma_addr_t shandle;
dma_addr_t ihandle;
- struct mutex wlock;
- struct mutex rlock;
- struct mutex plock;
+ struct mutex lock;
wait_queue_head_t wait;
struct work_struct rxwork;
int werr;
int status = urb->status;
spin_lock(&desc->iuspin);
+ clear_bit(WDM_RESPONDING, &desc->flags);
if (status) {
switch (status) {
case -ENOENT:
dev_dbg(&desc->intf->dev,
"nonzero urb status received: -ENOENT");
- break;
+ goto skip_error;
case -ECONNRESET:
dev_dbg(&desc->intf->dev,
"nonzero urb status received: -ECONNRESET");
- break;
+ goto skip_error;
case -ESHUTDOWN:
dev_dbg(&desc->intf->dev,
"nonzero urb status received: -ESHUTDOWN");
- break;
+ goto skip_error;
case -EPIPE:
dev_err(&desc->intf->dev,
"nonzero urb status received: -EPIPE\n");
desc->reslength = urb->actual_length;
memmove(desc->ubuf + desc->length, desc->inbuf, desc->reslength);
desc->length += desc->reslength;
+skip_error:
wake_up(&desc->wait);
set_bit(WDM_READ, &desc->flags);
desc->response->transfer_flags |= URB_NO_TRANSFER_DMA_MAP;
spin_lock(&desc->iuspin);
clear_bit(WDM_READ, &desc->flags);
- if (!test_bit(WDM_DISCONNECTING, &desc->flags)) {
+ set_bit(WDM_RESPONDING, &desc->flags);
+ if (!test_bit(WDM_DISCONNECTING, &desc->flags)
+ && !test_bit(WDM_SUSPENDING, &desc->flags)) {
rv = usb_submit_urb(desc->response, GFP_ATOMIC);
dev_dbg(&desc->intf->dev, "%s: usb_submit_urb %d",
__func__, rv);
}
spin_unlock(&desc->iuspin);
if (rv < 0) {
+ clear_bit(WDM_RESPONDING, &desc->flags);
if (rv == -EPERM)
return;
if (rv == -ENOMEM) {
if (we < 0)
return -EIO;
- r = mutex_lock_interruptible(&desc->wlock); /* concurrent writes */
+ desc->outbuf = buf = kmalloc(count, GFP_KERNEL);
+ if (!buf) {
+ rv = -ENOMEM;
+ goto outnl;
+ }
+
+ r = copy_from_user(buf, buffer, count);
+ if (r > 0) {
+ kfree(buf);
+ rv = -EFAULT;
+ goto outnl;
+ }
+
+ /* concurrent writes and disconnect */
+ r = mutex_lock_interruptible(&desc->lock);
rv = -ERESTARTSYS;
- if (r)
+ if (r) {
+ kfree(buf);
goto outnl;
+ }
+
+ if (test_bit(WDM_DISCONNECTING, &desc->flags)) {
+ kfree(buf);
+ rv = -ENODEV;
+ goto outnp;
+ }
r = usb_autopm_get_interface(desc->intf);
- if (r < 0)
+ if (r < 0) {
+ kfree(buf);
goto outnp;
+ }
if (!file->f_flags && O_NONBLOCK)
r = wait_event_interruptible(desc->wait, !test_bit(WDM_IN_USE,
else
if (test_bit(WDM_IN_USE, &desc->flags))
r = -EAGAIN;
- if (r < 0)
- goto out;
-
- if (test_bit(WDM_DISCONNECTING, &desc->flags)) {
- rv = -ENODEV;
- goto out;
- }
-
- desc->outbuf = buf = kmalloc(count, GFP_KERNEL);
- if (!buf) {
- rv = -ENOMEM;
- goto out;
- }
-
- r = copy_from_user(buf, buffer, count);
- if (r > 0) {
+ if (r < 0) {
kfree(buf);
- rv = -EFAULT;
goto out;
}
out:
usb_autopm_put_interface(desc->intf);
outnp:
- mutex_unlock(&desc->wlock);
+ mutex_unlock(&desc->lock);
outnl:
return rv < 0 ? rv : count;
}
struct wdm_device *desc = file->private_data;
- rv = mutex_lock_interruptible(&desc->rlock); /*concurrent reads */
+ rv = mutex_lock_interruptible(&desc->lock); /*concurrent reads */
if (rv < 0)
return -ERESTARTSYS;
spin_lock_irq(&desc->iuspin);
if (desc->rerr) { /* read completed, error happened */
- int t = desc->rerr;
desc->rerr = 0;
spin_unlock_irq(&desc->iuspin);
- dev_err(&desc->intf->dev,
- "reading had resulted in %d\n", t);
rv = -EIO;
goto err;
}
rv = cntr;
err:
- mutex_unlock(&desc->rlock);
- if (rv < 0 && rv != -EAGAIN)
- dev_err(&desc->intf->dev, "wdm_read: exit error\n");
+ mutex_unlock(&desc->lock);
return rv;
}
}
intf->needs_remote_wakeup = 1;
- mutex_lock(&desc->plock);
+ mutex_lock(&desc->lock);
if (!desc->count++) {
rv = usb_submit_urb(desc->validity, GFP_KERNEL);
if (rv < 0) {
} else {
rv = 0;
}
- mutex_unlock(&desc->plock);
+ mutex_unlock(&desc->lock);
usb_autopm_put_interface(desc->intf);
out:
mutex_unlock(&wdm_mutex);
struct wdm_device *desc = file->private_data;
mutex_lock(&wdm_mutex);
- mutex_lock(&desc->plock);
+ mutex_lock(&desc->lock);
desc->count--;
- mutex_unlock(&desc->plock);
+ mutex_unlock(&desc->lock);
if (!desc->count) {
dev_dbg(&desc->intf->dev, "wdm_release: cleanup");
desc = kzalloc(sizeof(struct wdm_device), GFP_KERNEL);
if (!desc)
goto out;
- mutex_init(&desc->wlock);
- mutex_init(&desc->rlock);
- mutex_init(&desc->plock);
+ mutex_init(&desc->lock);
spin_lock_init(&desc->iuspin);
init_waitqueue_head(&desc->wait);
desc->wMaxCommand = maxcom;
/* to terminate pending flushes */
clear_bit(WDM_IN_USE, &desc->flags);
spin_unlock_irqrestore(&desc->iuspin, flags);
- cancel_work_sync(&desc->rxwork);
+ mutex_lock(&desc->lock);
kill_urbs(desc);
+ cancel_work_sync(&desc->rxwork);
+ mutex_unlock(&desc->lock);
wake_up_all(&desc->wait);
if (!desc->count)
cleanup(desc);
mutex_unlock(&wdm_mutex);
}
+#ifdef CONFIG_PM
static int wdm_suspend(struct usb_interface *intf, pm_message_t message)
{
struct wdm_device *desc = usb_get_intfdata(intf);
dev_dbg(&desc->intf->dev, "wdm%d_suspend\n", intf->minor);
- mutex_lock(&desc->plock);
-#ifdef CONFIG_PM
+ /* if this is an autosuspend the caller does the locking */
+ if (!(message.event & PM_EVENT_AUTO))
+ mutex_lock(&desc->lock);
+ spin_lock_irq(&desc->iuspin);
+
if ((message.event & PM_EVENT_AUTO) &&
- test_bit(WDM_IN_USE, &desc->flags)) {
+ (test_bit(WDM_IN_USE, &desc->flags)
+ || test_bit(WDM_RESPONDING, &desc->flags))) {
+ spin_unlock_irq(&desc->iuspin);
rv = -EBUSY;
} else {
-#endif
- cancel_work_sync(&desc->rxwork);
+
+ set_bit(WDM_SUSPENDING, &desc->flags);
+ spin_unlock_irq(&desc->iuspin);
+ /* callback submits work - order is essential */
kill_urbs(desc);
-#ifdef CONFIG_PM
+ cancel_work_sync(&desc->rxwork);
}
-#endif
- mutex_unlock(&desc->plock);
+ if (!(message.event & PM_EVENT_AUTO))
+ mutex_unlock(&desc->lock);
return rv;
}
+#endif
static int recover_from_urb_loss(struct wdm_device *desc)
{
}
return rv;
}
+
+#ifdef CONFIG_PM
static int wdm_resume(struct usb_interface *intf)
{
struct wdm_device *desc = usb_get_intfdata(intf);
int rv;
dev_dbg(&desc->intf->dev, "wdm%d_resume\n", intf->minor);
- mutex_lock(&desc->plock);
+
+ clear_bit(WDM_SUSPENDING, &desc->flags);
rv = recover_from_urb_loss(desc);
- mutex_unlock(&desc->plock);
+
return rv;
}
+#endif
static int wdm_pre_reset(struct usb_interface *intf)
{
struct wdm_device *desc = usb_get_intfdata(intf);
- mutex_lock(&desc->plock);
+ mutex_lock(&desc->lock);
return 0;
}
int rv;
rv = recover_from_urb_loss(desc);
- mutex_unlock(&desc->plock);
+ mutex_unlock(&desc->lock);
return 0;
}
.name = "cdc_wdm",
.probe = wdm_probe,
.disconnect = wdm_disconnect,
+#ifdef CONFIG_PM
.suspend = wdm_suspend,
.resume = wdm_resume,
.reset_resume = wdm_resume,
+#endif
.pre_reset = wdm_pre_reset,
.post_reset = wdm_post_reset,
.id_table = wdm_ids,
#include <linux/fs.h>
#include <linux/uaccess.h>
#include <linux/kref.h>
+#include <linux/slab.h>
#include <linux/mutex.h>
#include <linux/usb.h>
#include <linux/usb/tmc.h>
#include <linux/fs.h>
#include <linux/mm.h>
-#include <linux/slab.h>
+#include <linux/gfp.h>
#include <linux/poll.h>
#include <linux/usb.h>
#include <linux/smp_lock.h>
free_async(as);
return -ENOMEM;
}
+ /* Isochronous input data may end up being discontiguous
+ * if some of the packets are short. Clear the buffer so
+ * that the gaps don't leak kernel data to userspace.
+ */
+ if (is_in && uurb->type == USBDEVFS_URB_TYPE_ISO)
+ memset(as->urb->transfer_buffer, 0,
+ uurb->buffer_length);
}
as->urb->dev = ps->dev;
as->urb->pipe = (uurb->type << 30) |
void __user *addr = as->userurb;
unsigned int i;
- if (as->userbuffer && urb->actual_length)
- if (copy_to_user(as->userbuffer, urb->transfer_buffer,
- urb->actual_length))
+ if (as->userbuffer && urb->actual_length) {
+ if (urb->number_of_packets > 0) /* Isochronous */
+ i = urb->transfer_buffer_length;
+ else /* Non-Isoc */
+ i = urb->actual_length;
+ if (copy_to_user(as->userbuffer, urb->transfer_buffer, i))
goto err_out;
+ }
if (put_user(as->status, &userurb->status))
goto err_out;
if (put_user(urb->actual_length, &userurb->actual_length))
*/
#include <linux/device.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include <linux/usb/quirks.h>
#include <linux/pm_runtime.h>
#include <linux/kernel.h>
#include <linux/spinlock.h>
+#include <linux/slab.h>
#include <linux/idr.h>
#include <linux/usb.h>
#include "usb.h"
#include <linux/module.h>
#include <linux/errno.h>
#include <linux/rwsem.h>
+#include <linux/slab.h>
#include <linux/smp_lock.h>
#include <linux/usb.h>
if (urb->interval > (1 << 15))
return -EINVAL;
max = 1 << 15;
+ break;
case USB_SPEED_WIRELESS:
if (urb->interval > 16)
return -EINVAL;
which may be used with composite framework.
Say "y" to link the driver statically, or "m" to build
- a dynamically linked module called "g_file_storage". If unsure,
+ a dynamically linked module called "g_mass_storage". If unsure,
consider File-backed Storage Gadget.
config USB_G_SERIAL
{
struct at91_udc *udc = _udc;
u32 rescans = 5;
+ int disable_clock = 0;
+
+ if (!udc->clocked) {
+ clk_on(udc);
+ disable_clock = 1;
+ }
while (rescans--) {
u32 status;
}
}
+ if (disable_clock)
+ clk_off(udc);
+
return IRQ_HANDLED;
}
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <linux/device.h>
#include <linux/dma-mapping.h>
#include <linux/list.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/usb/ch9.h>
#include <linux/usb/gadget.h>
*/
#include <linux/errno.h>
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/list.h>
#include <linux/string.h>
}
#ifdef CONFIG_BLACKFIN
- } else if (gadget_is_musbhsfc(gadget) || gadget_is_musbhdrc(gadget)) {
+ } else if (gadget_is_musbhdrc(gadget)) {
if ((USB_ENDPOINT_XFER_BULK == type) ||
(USB_ENDPOINT_XFER_ISOC == type)) {
if (USB_DIR_IN & desc->bEndpointAddress)
/* #define VERBOSE_DEBUG */
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/device.h>
* Licensed under the GPL-2 or later.
*/
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/device.h>
#include <asm/atomic.h>
/* #define VERBOSE_DEBUG */
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/device.h>
#include <linux/etherdevice.h>
#include <linux/device.h>
#include <linux/etherdevice.h>
#include <linux/crc32.h>
+#include <linux/slab.h>
#include "u_ether.h"
/* #define VERBOSE_DEBUG */
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/device.h>
}
-static int fsg_bind(struct usb_configuration *c, struct usb_function *f)
+static int __init fsg_bind(struct usb_configuration *c, struct usb_function *f)
{
struct fsg_dev *fsg = fsg_from_func(f);
struct usb_gadget *gadget = c->cdev->gadget;
autoconf_fail:
ERROR(fsg, "unable to autoconfigure all endpoints\n");
rc = -ENOTSUPP;
- fsg_unbind(c, f);
return rc;
}
/* #define VERBOSE_DEBUG */
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/device.h>
* 02110-1301 USA
*/
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/device.h>
/* #define VERBOSE_DEBUG */
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/device.h>
#include <linux/etherdevice.h>
* either version 2 of that License or (at your option) any later version.
*/
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/device.h>
/* #define VERBOSE_DEBUG */
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/device.h>
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/device.h>
#include <linux/etherdevice.h>
#define gadget_is_r8a66597(g) 0
#endif
+#ifdef CONFIG_USB_S3C_HSOTG
+#define gadget_is_s3c_hsotg(g) (!strcmp("s3c-hsotg", (g)->name))
+#else
+#define gadget_is_s3c_hsotg(g) 0
+#endif
+
/**
* usb_gadget_controller_number - support bcdDevice id convention
return 0x24;
else if (gadget_is_r8a66597(gadget))
return 0x25;
+ else if (gadget_is_s3c_hsotg(gadget))
+ return 0x26;
return -ENOENT;
}
/* #define VERBOSE_DEBUG */
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/utsname.h>
#include <linux/device.h>
* usb_gadget_driver_{register,unregister}() must change.
*/
if (the_controller) {
- WARNING(dev, "ignoring %s\n", pci_name(pdev));
+ pr_warning("ignoring %s\n", pci_name(pdev));
return -EBUSY;
}
if (!pdev->irq) {
#include <linux/clk.h>
#include <linux/delay.h>
#include <linux/timer.h>
+#include <linux/slab.h>
#include <linux/usb/ch9.h>
#include <linux/usb/gadget.h>
*/
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include "lh7a40x_udc.h"
#include <linux/delay.h>
#include <linux/io.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/err.h>
#include <linux/usb/ch9.h>
#include <linux/usb/gadget.h>
ret = fsg_add(c->cdev, c, fsg_common);
if (ret < 0)
return ret;
- if (ret < 0)
- return ret;
return 0;
}
#include <linux/clk.h>
#include <linux/irq.h>
#include <linux/gpio.h>
+#include <linux/slab.h>
#include <asm/byteorder.h>
#include <mach/hardware.h>
#include <linux/platform_device.h>
#include <linux/clk.h>
#include <linux/err.h>
+#include <linux/slab.h>
#include <linux/usb/ch9.h>
#include <linux/usb/gadget.h>
#include <linux/init.h>
#include <linux/list.h>
#include <linux/proc_fs.h>
+#include <linux/slab.h>
#include <linux/seq_file.h>
#include <linux/netdevice.h>
#include <linux/seq_file.h>
#include <linux/delay.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <linux/usb/ch9.h>
#include <linux/usb/gadget.h>
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/device.h>
#include <linux/delay.h>
#include <linux/ctype.h>
/* #define VERBOSE_DEBUG */
#include <linux/kernel.h>
+#include <linux/gfp.h>
#include <linux/device.h>
#include <linux/ctype.h>
#include <linux/etherdevice.h>
#include <linux/delay.h>
#include <linux/tty.h>
#include <linux/tty_flip.h>
+#include <linux/slab.h>
#include "u_serial.h"
/* #define VERBOSE_DEBUG */
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/utsname.h>
#include <linux/device.h>
ifeq ($(CONFIG_FHCI_DEBUG),y)
fhci-objs += fhci-dbg.o
endif
-xhci-objs := xhci-hcd.o xhci-mem.o xhci-pci.o xhci-ring.o xhci-hub.o xhci-dbg.o
+xhci-hcd-objs := xhci.o xhci-mem.o xhci-pci.o xhci-ring.o xhci-hub.o xhci-dbg.o
obj-$(CONFIG_USB_WHCI_HCD) += whci/
obj-$(CONFIG_USB_OHCI_HCD) += ohci-hcd.o
obj-$(CONFIG_USB_UHCI_HCD) += uhci-hcd.o
obj-$(CONFIG_USB_FHCI_HCD) += fhci.o
-obj-$(CONFIG_USB_XHCI_HCD) += xhci.o
+obj-$(CONFIG_USB_XHCI_HCD) += xhci-hcd.o
obj-$(CONFIG_USB_SL811_HCD) += sl811-hcd.o
obj-$(CONFIG_USB_SL811_CS) += sl811_cs.o
obj-$(CONFIG_USB_U132_HCD) += u132-hcd.o
#include <linux/delay.h>
#include <linux/ioport.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/errno.h>
#include <linux/init.h>
#include <linux/moduleparam.h>
#include <linux/dma-mapping.h>
#include <linux/debugfs.h>
+#include <linux/slab.h>
#include "../core/hcd.h"
/* endpoints can be iso streams. for now, we don't
* accelerate iso completions ... so spin a while.
*/
- if (qh->hw->hw_info1 == 0) {
+ if (qh->hw == NULL) {
ehci_vdbg (ehci, "iso delay\n");
goto idle_timeout;
}
#include <linux/clk.h>
#include <linux/delay.h>
#include <linux/usb/otg.h>
+#include <linux/slab.h>
#include <mach/mxc_ehci.h>
#include <linux/clk.h>
#include <linux/gpio.h>
#include <linux/regulator/consumer.h>
+#include <linux/slab.h>
#include <plat/usb.h>
/*
urb->interval);
}
- /* if dev->ep [epnum] is a QH, info1.maxpacket is nonzero */
- } else if (unlikely (stream->hw_info1 != 0)) {
+ /* if dev->ep [epnum] is a QH, hw is set */
+ } else if (unlikely (stream->hw != NULL)) {
ehci_dbg (ehci, "dev %s ep%d%s, not iso??\n",
urb->dev->devpath, epnum,
usb_pipein(urb->pipe) ? "in" : "out");
static inline void
itd_link (struct ehci_hcd *ehci, unsigned frame, struct ehci_itd *itd)
{
- /* always prepend ITD/SITD ... only QH tree is order-sensitive */
- itd->itd_next = ehci->pshadow [frame];
- itd->hw_next = ehci->periodic [frame];
- ehci->pshadow [frame].itd = itd;
+ union ehci_shadow *prev = &ehci->pshadow[frame];
+ __hc32 *hw_p = &ehci->periodic[frame];
+ union ehci_shadow here = *prev;
+ __hc32 type = 0;
+
+ /* skip any iso nodes which might belong to previous microframes */
+ while (here.ptr) {
+ type = Q_NEXT_TYPE(ehci, *hw_p);
+ if (type == cpu_to_hc32(ehci, Q_TYPE_QH))
+ break;
+ prev = periodic_next_shadow(ehci, prev, type);
+ hw_p = shadow_next_periodic(ehci, &here, type);
+ here = *prev;
+ }
+
+ itd->itd_next = here;
+ itd->hw_next = *hw_p;
+ prev->itd = itd;
itd->frame = frame;
wmb ();
- ehci->periodic[frame] = cpu_to_hc32(ehci, itd->itd_dma | Q_TYPE_ITD);
+ *hw_p = cpu_to_hc32(ehci, itd->itd_dma | Q_TYPE_ITD);
}
/* fit urb's itds into the selected schedule slot; activate as needed */
* acts like a qh would, if EHCI had them for ISO.
*/
struct ehci_iso_stream {
- /* first two fields match QH, but info1 == 0 */
- __hc32 hw_next;
- __hc32 hw_info1;
+ /* first field matches ehci_hq, but is NULL */
+ struct ehci_qh_hw *hw;
u32 refcount;
u8 bEndpointAddress;
#include <linux/usb.h>
#include <linux/of_platform.h>
#include <linux/of_gpio.h>
+#include <linux/slab.h>
#include <asm/qe.h>
#include <asm/fsl_gtm.h>
#include "../core/hcd.h"
#include <linux/kernel.h>
#include <linux/types.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/list.h>
#include <linux/usb.h>
#include "../core/hcd.h"
#include <linux/types.h>
#include <linux/spinlock.h>
#include <linux/errno.h>
+#include <linux/slab.h>
#include <linux/list.h>
#include <linux/usb.h>
#include "../core/hcd.h"
#include <linux/kernel.h>
#include <linux/types.h>
#include <linux/errno.h>
+#include <linux/slab.h>
#include <linux/list.h>
#include <linux/io.h>
#include <linux/usb.h>
*/
#include <linux/kernel.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/workqueue.h>
#include <linux/wait.h>
#include <linux/kernel.h>
#include <linux/list.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include "../core/hcd.h"
#include <linux/errno.h>
#include <linux/init.h>
#include <linux/list.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include <linux/usb/isp116x.h>
#include <linux/platform_device.h>
*/
#include <linux/irq.h>
+#include <linux/slab.h>
static void urb_free_priv (struct ohci_hcd *hc, urb_priv_t *urb_priv)
{
#include <linux/io.h>
#include <linux/mm.h>
#include <linux/irq.h>
+#include <linux/slab.h>
#include <asm/cacheflush.h>
#include "../core/hcd.h"
/* this function must be called with interrupt disabled */
static void free_usb_address(struct r8a66597 *r8a66597,
- struct r8a66597_device *dev)
+ struct r8a66597_device *dev, int reset)
{
int port;
dev->state = USB_STATE_DEFAULT;
r8a66597->address_map &= ~(1 << dev->address);
dev->address = 0;
- dev_set_drvdata(&dev->udev->dev, NULL);
+ /*
+ * Only when resetting USB, it is necessary to erase drvdata. When
+ * a usb device with usb hub is disconnect, "dev->udev" is already
+ * freed on usb_desconnect(). So we cannot access the data.
+ */
+ if (reset)
+ dev_set_drvdata(&dev->udev->dev, NULL);
list_del(&dev->device_list);
kfree(dev);
struct r8a66597_device *dev = r8a66597->root_hub[port].dev;
disable_r8a66597_pipe_all(r8a66597, dev);
- free_usb_address(r8a66597, dev);
+ free_usb_address(r8a66597, dev, 0);
start_root_hub_sampling(r8a66597, port, 0);
}
spin_lock_irqsave(&r8a66597->lock, flags);
dev = get_r8a66597_device(r8a66597, addr);
disable_r8a66597_pipe_all(r8a66597, dev);
- free_usb_address(r8a66597, dev);
+ free_usb_address(r8a66597, dev, 0);
put_child_connect_map(r8a66597, addr);
spin_unlock_irqrestore(&r8a66597->lock, flags);
}
rh->port |= (1 << USB_PORT_FEAT_RESET);
disable_r8a66597_pipe_all(r8a66597, dev);
- free_usb_address(r8a66597, dev);
+ free_usb_address(r8a66597, dev, 1);
r8a66597_mdfy(r8a66597, USBRST, USBRST | UACT,
get_dvstctr_reg(port));
* (C) Copyright 1999-2001 Johannes Erdfelt
*/
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/debugfs.h>
#include <linux/smp_lock.h>
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#include <linux/kernel.h>
+#include <linux/gfp.h>
#include <linux/dma-mapping.h>
#include <linux/uwb/umc.h>
#include <linux/usb.h>
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/debugfs.h>
#include <linux/seq_file.h>
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#include <linux/kernel.h>
+#include <linux/gfp.h>
#include <linux/dma-mapping.h>
#include <linux/uwb/umc.h>
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#include <linux/kernel.h>
+#include <linux/gfp.h>
#include <linux/dma-mapping.h>
#include <linux/uwb/umc.h>
#include <linux/usb.h>
*/
#include <linux/kernel.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <linux/uwb/umc.h>
#include <linux/usb.h>
+++ /dev/null
-/*
- * xHCI host controller driver
- *
- * Copyright (C) 2008 Intel Corp.
- *
- * Author: Sarah Sharp
- * Some code borrowed from the Linux EHCI driver.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
- * or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
- * for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software Foundation,
- * Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
- */
-
-#include <linux/irq.h>
-#include <linux/module.h>
-#include <linux/moduleparam.h>
-
-#include "xhci.h"
-
-#define DRIVER_AUTHOR "Sarah Sharp"
-#define DRIVER_DESC "'eXtensible' Host Controller (xHC) Driver"
-
-/* Some 0.95 hardware can't handle the chain bit on a Link TRB being cleared */
-static int link_quirk;
-module_param(link_quirk, int, S_IRUGO | S_IWUSR);
-MODULE_PARM_DESC(link_quirk, "Don't clear the chain bit on a link TRB");
-
-/* TODO: copied from ehci-hcd.c - can this be refactored? */
-/*
- * handshake - spin reading hc until handshake completes or fails
- * @ptr: address of hc register to be read
- * @mask: bits to look at in result of read
- * @done: value of those bits when handshake succeeds
- * @usec: timeout in microseconds
- *
- * Returns negative errno, or zero on success
- *
- * Success happens when the "mask" bits have the specified value (hardware
- * handshake done). There are two failure modes: "usec" have passed (major
- * hardware flakeout), or the register reads as all-ones (hardware removed).
- */
-static int handshake(struct xhci_hcd *xhci, void __iomem *ptr,
- u32 mask, u32 done, int usec)
-{
- u32 result;
-
- do {
- result = xhci_readl(xhci, ptr);
- if (result == ~(u32)0) /* card removed */
- return -ENODEV;
- result &= mask;
- if (result == done)
- return 0;
- udelay(1);
- usec--;
- } while (usec > 0);
- return -ETIMEDOUT;
-}
-
-/*
- * Disable interrupts and begin the xHCI halting process.
- */
-void xhci_quiesce(struct xhci_hcd *xhci)
-{
- u32 halted;
- u32 cmd;
- u32 mask;
-
- mask = ~(XHCI_IRQS);
- halted = xhci_readl(xhci, &xhci->op_regs->status) & STS_HALT;
- if (!halted)
- mask &= ~CMD_RUN;
-
- cmd = xhci_readl(xhci, &xhci->op_regs->command);
- cmd &= mask;
- xhci_writel(xhci, cmd, &xhci->op_regs->command);
-}
-
-/*
- * Force HC into halt state.
- *
- * Disable any IRQs and clear the run/stop bit.
- * HC will complete any current and actively pipelined transactions, and
- * should halt within 16 microframes of the run/stop bit being cleared.
- * Read HC Halted bit in the status register to see when the HC is finished.
- * XXX: shouldn't we set HC_STATE_HALT here somewhere?
- */
-int xhci_halt(struct xhci_hcd *xhci)
-{
- xhci_dbg(xhci, "// Halt the HC\n");
- xhci_quiesce(xhci);
-
- return handshake(xhci, &xhci->op_regs->status,
- STS_HALT, STS_HALT, XHCI_MAX_HALT_USEC);
-}
-
-/*
- * Reset a halted HC, and set the internal HC state to HC_STATE_HALT.
- *
- * This resets pipelines, timers, counters, state machines, etc.
- * Transactions will be terminated immediately, and operational registers
- * will be set to their defaults.
- */
-int xhci_reset(struct xhci_hcd *xhci)
-{
- u32 command;
- u32 state;
-
- state = xhci_readl(xhci, &xhci->op_regs->status);
- if ((state & STS_HALT) == 0) {
- xhci_warn(xhci, "Host controller not halted, aborting reset.\n");
- return 0;
- }
-
- xhci_dbg(xhci, "// Reset the HC\n");
- command = xhci_readl(xhci, &xhci->op_regs->command);
- command |= CMD_RESET;
- xhci_writel(xhci, command, &xhci->op_regs->command);
- /* XXX: Why does EHCI set this here? Shouldn't other code do this? */
- xhci_to_hcd(xhci)->state = HC_STATE_HALT;
-
- return handshake(xhci, &xhci->op_regs->command, CMD_RESET, 0, 250 * 1000);
-}
-
-
-#if 0
-/* Set up MSI-X table for entry 0 (may claim other entries later) */
-static int xhci_setup_msix(struct xhci_hcd *xhci)
-{
- int ret;
- struct pci_dev *pdev = to_pci_dev(xhci_to_hcd(xhci)->self.controller);
-
- xhci->msix_count = 0;
- /* XXX: did I do this right? ixgbe does kcalloc for more than one */
- xhci->msix_entries = kmalloc(sizeof(struct msix_entry), GFP_KERNEL);
- if (!xhci->msix_entries) {
- xhci_err(xhci, "Failed to allocate MSI-X entries\n");
- return -ENOMEM;
- }
- xhci->msix_entries[0].entry = 0;
-
- ret = pci_enable_msix(pdev, xhci->msix_entries, xhci->msix_count);
- if (ret) {
- xhci_err(xhci, "Failed to enable MSI-X\n");
- goto free_entries;
- }
-
- /*
- * Pass the xhci pointer value as the request_irq "cookie".
- * If more irqs are added, this will need to be unique for each one.
- */
- ret = request_irq(xhci->msix_entries[0].vector, &xhci_irq, 0,
- "xHCI", xhci_to_hcd(xhci));
- if (ret) {
- xhci_err(xhci, "Failed to allocate MSI-X interrupt\n");
- goto disable_msix;
- }
- xhci_dbg(xhci, "Finished setting up MSI-X\n");
- return 0;
-
-disable_msix:
- pci_disable_msix(pdev);
-free_entries:
- kfree(xhci->msix_entries);
- xhci->msix_entries = NULL;
- return ret;
-}
-
-/* XXX: code duplication; can xhci_setup_msix call this? */
-/* Free any IRQs and disable MSI-X */
-static void xhci_cleanup_msix(struct xhci_hcd *xhci)
-{
- struct pci_dev *pdev = to_pci_dev(xhci_to_hcd(xhci)->self.controller);
- if (!xhci->msix_entries)
- return;
-
- free_irq(xhci->msix_entries[0].vector, xhci);
- pci_disable_msix(pdev);
- kfree(xhci->msix_entries);
- xhci->msix_entries = NULL;
- xhci_dbg(xhci, "Finished cleaning up MSI-X\n");
-}
-#endif
-
-/*
- * Initialize memory for HCD and xHC (one-time init).
- *
- * Program the PAGESIZE register, initialize the device context array, create
- * device contexts (?), set up a command ring segment (or two?), create event
- * ring (one for now).
- */
-int xhci_init(struct usb_hcd *hcd)
-{
- struct xhci_hcd *xhci = hcd_to_xhci(hcd);
- int retval = 0;
-
- xhci_dbg(xhci, "xhci_init\n");
- spin_lock_init(&xhci->lock);
- if (link_quirk) {
- xhci_dbg(xhci, "QUIRK: Not clearing Link TRB chain bits.\n");
- xhci->quirks |= XHCI_LINK_TRB_QUIRK;
- } else {
- xhci_dbg(xhci, "xHCI doesn't need link TRB QUIRK\n");
- }
- retval = xhci_mem_init(xhci, GFP_KERNEL);
- xhci_dbg(xhci, "Finished xhci_init\n");
-
- return retval;
-}
-
-/*
- * Called in interrupt context when there might be work
- * queued on the event ring
- *
- * xhci->lock must be held by caller.
- */
-static void xhci_work(struct xhci_hcd *xhci)
-{
- u32 temp;
- u64 temp_64;
-
- /*
- * Clear the op reg interrupt status first,
- * so we can receive interrupts from other MSI-X interrupters.
- * Write 1 to clear the interrupt status.
- */
- temp = xhci_readl(xhci, &xhci->op_regs->status);
- temp |= STS_EINT;
- xhci_writel(xhci, temp, &xhci->op_regs->status);
- /* FIXME when MSI-X is supported and there are multiple vectors */
- /* Clear the MSI-X event interrupt status */
-
- /* Acknowledge the interrupt */
- temp = xhci_readl(xhci, &xhci->ir_set->irq_pending);
- temp |= 0x3;
- xhci_writel(xhci, temp, &xhci->ir_set->irq_pending);
- /* Flush posted writes */
- xhci_readl(xhci, &xhci->ir_set->irq_pending);
-
- if (xhci->xhc_state & XHCI_STATE_DYING)
- xhci_dbg(xhci, "xHCI dying, ignoring interrupt. "
- "Shouldn't IRQs be disabled?\n");
- else
- /* FIXME this should be a delayed service routine
- * that clears the EHB.
- */
- xhci_handle_event(xhci);
-
- /* Clear the event handler busy flag (RW1C); the event ring should be empty. */
- temp_64 = xhci_read_64(xhci, &xhci->ir_set->erst_dequeue);
- xhci_write_64(xhci, temp_64 | ERST_EHB, &xhci->ir_set->erst_dequeue);
- /* Flush posted writes -- FIXME is this necessary? */
- xhci_readl(xhci, &xhci->ir_set->irq_pending);
-}
-
-/*-------------------------------------------------------------------------*/
-
-/*
- * xHCI spec says we can get an interrupt, and if the HC has an error condition,
- * we might get bad data out of the event ring. Section 4.10.2.7 has a list of
- * indicators of an event TRB error, but we check the status *first* to be safe.
- */
-irqreturn_t xhci_irq(struct usb_hcd *hcd)
-{
- struct xhci_hcd *xhci = hcd_to_xhci(hcd);
- u32 temp, temp2;
- union xhci_trb *trb;
-
- spin_lock(&xhci->lock);
- trb = xhci->event_ring->dequeue;
- /* Check if the xHC generated the interrupt, or the irq is shared */
- temp = xhci_readl(xhci, &xhci->op_regs->status);
- temp2 = xhci_readl(xhci, &xhci->ir_set->irq_pending);
- if (temp == 0xffffffff && temp2 == 0xffffffff)
- goto hw_died;
-
- if (!(temp & STS_EINT) && !ER_IRQ_PENDING(temp2)) {
- spin_unlock(&xhci->lock);
- return IRQ_NONE;
- }
- xhci_dbg(xhci, "op reg status = %08x\n", temp);
- xhci_dbg(xhci, "ir set irq_pending = %08x\n", temp2);
- xhci_dbg(xhci, "Event ring dequeue ptr:\n");
- xhci_dbg(xhci, "@%llx %08x %08x %08x %08x\n",
- (unsigned long long)xhci_trb_virt_to_dma(xhci->event_ring->deq_seg, trb),
- lower_32_bits(trb->link.segment_ptr),
- upper_32_bits(trb->link.segment_ptr),
- (unsigned int) trb->link.intr_target,
- (unsigned int) trb->link.control);
-
- if (temp & STS_FATAL) {
- xhci_warn(xhci, "WARNING: Host System Error\n");
- xhci_halt(xhci);
-hw_died:
- xhci_to_hcd(xhci)->state = HC_STATE_HALT;
- spin_unlock(&xhci->lock);
- return -ESHUTDOWN;
- }
-
- xhci_work(xhci);
- spin_unlock(&xhci->lock);
-
- return IRQ_HANDLED;
-}
-
-#ifdef CONFIG_USB_XHCI_HCD_DEBUGGING
-void xhci_event_ring_work(unsigned long arg)
-{
- unsigned long flags;
- int temp;
- u64 temp_64;
- struct xhci_hcd *xhci = (struct xhci_hcd *) arg;
- int i, j;
-
- xhci_dbg(xhci, "Poll event ring: %lu\n", jiffies);
-
- spin_lock_irqsave(&xhci->lock, flags);
- temp = xhci_readl(xhci, &xhci->op_regs->status);
- xhci_dbg(xhci, "op reg status = 0x%x\n", temp);
- if (temp == 0xffffffff || (xhci->xhc_state & XHCI_STATE_DYING)) {
- xhci_dbg(xhci, "HW died, polling stopped.\n");
- spin_unlock_irqrestore(&xhci->lock, flags);
- return;
- }
-
- temp = xhci_readl(xhci, &xhci->ir_set->irq_pending);
- xhci_dbg(xhci, "ir_set 0 pending = 0x%x\n", temp);
- xhci_dbg(xhci, "No-op commands handled = %d\n", xhci->noops_handled);
- xhci_dbg(xhci, "HC error bitmask = 0x%x\n", xhci->error_bitmask);
- xhci->error_bitmask = 0;
- xhci_dbg(xhci, "Event ring:\n");
- xhci_debug_segment(xhci, xhci->event_ring->deq_seg);
- xhci_dbg_ring_ptrs(xhci, xhci->event_ring);
- temp_64 = xhci_read_64(xhci, &xhci->ir_set->erst_dequeue);
- temp_64 &= ~ERST_PTR_MASK;
- xhci_dbg(xhci, "ERST deq = 64'h%0lx\n", (long unsigned int) temp_64);
- xhci_dbg(xhci, "Command ring:\n");
- xhci_debug_segment(xhci, xhci->cmd_ring->deq_seg);
- xhci_dbg_ring_ptrs(xhci, xhci->cmd_ring);
- xhci_dbg_cmd_ptrs(xhci);
- for (i = 0; i < MAX_HC_SLOTS; ++i) {
- if (!xhci->devs[i])
- continue;
- for (j = 0; j < 31; ++j) {
- struct xhci_ring *ring = xhci->devs[i]->eps[j].ring;
- if (!ring)
- continue;
- xhci_dbg(xhci, "Dev %d endpoint ring %d:\n", i, j);
- xhci_debug_segment(xhci, ring->deq_seg);
- }
- }
-
- if (xhci->noops_submitted != NUM_TEST_NOOPS)
- if (xhci_setup_one_noop(xhci))
- xhci_ring_cmd_db(xhci);
- spin_unlock_irqrestore(&xhci->lock, flags);
-
- if (!xhci->zombie)
- mod_timer(&xhci->event_ring_timer, jiffies + POLL_TIMEOUT * HZ);
- else
- xhci_dbg(xhci, "Quit polling the event ring.\n");
-}
-#endif
-
-/*
- * Start the HC after it was halted.
- *
- * This function is called by the USB core when the HC driver is added.
- * Its opposite is xhci_stop().
- *
- * xhci_init() must be called once before this function can be called.
- * Reset the HC, enable device slot contexts, program DCBAAP, and
- * set command ring pointer and event ring pointer.
- *
- * Setup MSI-X vectors and enable interrupts.
- */
-int xhci_run(struct usb_hcd *hcd)
-{
- u32 temp;
- u64 temp_64;
- struct xhci_hcd *xhci = hcd_to_xhci(hcd);
- void (*doorbell)(struct xhci_hcd *) = NULL;
-
- hcd->uses_new_polling = 1;
- hcd->poll_rh = 0;
-
- xhci_dbg(xhci, "xhci_run\n");
-#if 0 /* FIXME: MSI not setup yet */
- /* Do this at the very last minute */
- ret = xhci_setup_msix(xhci);
- if (!ret)
- return ret;
-
- return -ENOSYS;
-#endif
-#ifdef CONFIG_USB_XHCI_HCD_DEBUGGING
- init_timer(&xhci->event_ring_timer);
- xhci->event_ring_timer.data = (unsigned long) xhci;
- xhci->event_ring_timer.function = xhci_event_ring_work;
- /* Poll the event ring */
- xhci->event_ring_timer.expires = jiffies + POLL_TIMEOUT * HZ;
- xhci->zombie = 0;
- xhci_dbg(xhci, "Setting event ring polling timer\n");
- add_timer(&xhci->event_ring_timer);
-#endif
-
- xhci_dbg(xhci, "Command ring memory map follows:\n");
- xhci_debug_ring(xhci, xhci->cmd_ring);
- xhci_dbg_ring_ptrs(xhci, xhci->cmd_ring);
- xhci_dbg_cmd_ptrs(xhci);
-
- xhci_dbg(xhci, "ERST memory map follows:\n");
- xhci_dbg_erst(xhci, &xhci->erst);
- xhci_dbg(xhci, "Event ring:\n");
- xhci_debug_ring(xhci, xhci->event_ring);
- xhci_dbg_ring_ptrs(xhci, xhci->event_ring);
- temp_64 = xhci_read_64(xhci, &xhci->ir_set->erst_dequeue);
- temp_64 &= ~ERST_PTR_MASK;
- xhci_dbg(xhci, "ERST deq = 64'h%0lx\n", (long unsigned int) temp_64);
-
- xhci_dbg(xhci, "// Set the interrupt modulation register\n");
- temp = xhci_readl(xhci, &xhci->ir_set->irq_control);
- temp &= ~ER_IRQ_INTERVAL_MASK;
- temp |= (u32) 160;
- xhci_writel(xhci, temp, &xhci->ir_set->irq_control);
-
- /* Set the HCD state before we enable the irqs */
- hcd->state = HC_STATE_RUNNING;
- temp = xhci_readl(xhci, &xhci->op_regs->command);
- temp |= (CMD_EIE);
- xhci_dbg(xhci, "// Enable interrupts, cmd = 0x%x.\n",
- temp);
- xhci_writel(xhci, temp, &xhci->op_regs->command);
-
- temp = xhci_readl(xhci, &xhci->ir_set->irq_pending);
- xhci_dbg(xhci, "// Enabling event ring interrupter %p by writing 0x%x to irq_pending\n",
- xhci->ir_set, (unsigned int) ER_IRQ_ENABLE(temp));
- xhci_writel(xhci, ER_IRQ_ENABLE(temp),
- &xhci->ir_set->irq_pending);
- xhci_print_ir_set(xhci, xhci->ir_set, 0);
-
- if (NUM_TEST_NOOPS > 0)
- doorbell = xhci_setup_one_noop(xhci);
-
- temp = xhci_readl(xhci, &xhci->op_regs->command);
- temp |= (CMD_RUN);
- xhci_dbg(xhci, "// Turn on HC, cmd = 0x%x.\n",
- temp);
- xhci_writel(xhci, temp, &xhci->op_regs->command);
- /* Flush PCI posted writes */
- temp = xhci_readl(xhci, &xhci->op_regs->command);
- xhci_dbg(xhci, "// @%p = 0x%x\n", &xhci->op_regs->command, temp);
- if (doorbell)
- (*doorbell)(xhci);
-
- xhci_dbg(xhci, "Finished xhci_run\n");
- return 0;
-}
-
-/*
- * Stop xHCI driver.
- *
- * This function is called by the USB core when the HC driver is removed.
- * Its opposite is xhci_run().
- *
- * Disable device contexts, disable IRQs, and quiesce the HC.
- * Reset the HC, finish any completed transactions, and cleanup memory.
- */
-void xhci_stop(struct usb_hcd *hcd)
-{
- u32 temp;
- struct xhci_hcd *xhci = hcd_to_xhci(hcd);
-
- spin_lock_irq(&xhci->lock);
- xhci_halt(xhci);
- xhci_reset(xhci);
- spin_unlock_irq(&xhci->lock);
-
-#if 0 /* No MSI yet */
- xhci_cleanup_msix(xhci);
-#endif
-#ifdef CONFIG_USB_XHCI_HCD_DEBUGGING
- /* Tell the event ring poll function not to reschedule */
- xhci->zombie = 1;
- del_timer_sync(&xhci->event_ring_timer);
-#endif
-
- xhci_dbg(xhci, "// Disabling event ring interrupts\n");
- temp = xhci_readl(xhci, &xhci->op_regs->status);
- xhci_writel(xhci, temp & ~STS_EINT, &xhci->op_regs->status);
- temp = xhci_readl(xhci, &xhci->ir_set->irq_pending);
- xhci_writel(xhci, ER_IRQ_DISABLE(temp),
- &xhci->ir_set->irq_pending);
- xhci_print_ir_set(xhci, xhci->ir_set, 0);
-
- xhci_dbg(xhci, "cleaning up memory\n");
- xhci_mem_cleanup(xhci);
- xhci_dbg(xhci, "xhci_stop completed - status = %x\n",
- xhci_readl(xhci, &xhci->op_regs->status));
-}
-
-/*
- * Shutdown HC (not bus-specific)
- *
- * This is called when the machine is rebooting or halting. We assume that the
- * machine will be powered off, and the HC's internal state will be reset.
- * Don't bother to free memory.
- */
-void xhci_shutdown(struct usb_hcd *hcd)
-{
- struct xhci_hcd *xhci = hcd_to_xhci(hcd);
-
- spin_lock_irq(&xhci->lock);
- xhci_halt(xhci);
- spin_unlock_irq(&xhci->lock);
-
-#if 0
- xhci_cleanup_msix(xhci);
-#endif
-
- xhci_dbg(xhci, "xhci_shutdown completed - status = %x\n",
- xhci_readl(xhci, &xhci->op_regs->status));
-}
-
-/*-------------------------------------------------------------------------*/
-
-/**
- * xhci_get_endpoint_index - Used for passing endpoint bitmasks between the core and
- * HCDs. Find the index for an endpoint given its descriptor. Use the return
- * value to right shift 1 for the bitmask.
- *
- * Index = (epnum * 2) + direction - 1,
- * where direction = 0 for OUT, 1 for IN.
- * For control endpoints, the IN index is used (OUT index is unused), so
- * index = (epnum * 2) + direction - 1 = (epnum * 2) + 1 - 1 = (epnum * 2)
- */
-unsigned int xhci_get_endpoint_index(struct usb_endpoint_descriptor *desc)
-{
- unsigned int index;
- if (usb_endpoint_xfer_control(desc))
- index = (unsigned int) (usb_endpoint_num(desc)*2);
- else
- index = (unsigned int) (usb_endpoint_num(desc)*2) +
- (usb_endpoint_dir_in(desc) ? 1 : 0) - 1;
- return index;
-}
-
-/* Find the flag for this endpoint (for use in the control context). Use the
- * endpoint index to create a bitmask. The slot context is bit 0, endpoint 0 is
- * bit 1, etc.
- */
-unsigned int xhci_get_endpoint_flag(struct usb_endpoint_descriptor *desc)
-{
- return 1 << (xhci_get_endpoint_index(desc) + 1);
-}
-
-/* Find the flag for this endpoint (for use in the control context). Use the
- * endpoint index to create a bitmask. The slot context is bit 0, endpoint 0 is
- * bit 1, etc.
- */
-unsigned int xhci_get_endpoint_flag_from_index(unsigned int ep_index)
-{
- return 1 << (ep_index + 1);
-}
-
-/* Compute the last valid endpoint context index. Basically, this is the
- * endpoint index plus one. For slot contexts with more than valid endpoint,
- * we find the most significant bit set in the added contexts flags.
- * e.g. ep 1 IN (with epnum 0x81) => added_ctxs = 0b1000
- * fls(0b1000) = 4, but the endpoint context index is 3, so subtract one.
- */
-unsigned int xhci_last_valid_endpoint(u32 added_ctxs)
-{
- return fls(added_ctxs) - 1;
-}
-
-/* Returns 1 if the arguments are OK;
- * returns 0 this is a root hub; returns -EINVAL for NULL pointers.
- */
-int xhci_check_args(struct usb_hcd *hcd, struct usb_device *udev,
- struct usb_host_endpoint *ep, int check_ep, const char *func) {
- if (!hcd || (check_ep && !ep) || !udev) {
- printk(KERN_DEBUG "xHCI %s called with invalid args\n",
- func);
- return -EINVAL;
- }
- if (!udev->parent) {
- printk(KERN_DEBUG "xHCI %s called for root hub\n",
- func);
- return 0;
- }
- if (!udev->slot_id) {
- printk(KERN_DEBUG "xHCI %s called with unaddressed device\n",
- func);
- return -EINVAL;
- }
- return 1;
-}
-
-static int xhci_configure_endpoint(struct xhci_hcd *xhci,
- struct usb_device *udev, struct xhci_command *command,
- bool ctx_change, bool must_succeed);
-
-/*
- * Full speed devices may have a max packet size greater than 8 bytes, but the
- * USB core doesn't know that until it reads the first 8 bytes of the
- * descriptor. If the usb_device's max packet size changes after that point,
- * we need to issue an evaluate context command and wait on it.
- */
-static int xhci_check_maxpacket(struct xhci_hcd *xhci, unsigned int slot_id,
- unsigned int ep_index, struct urb *urb)
-{
- struct xhci_container_ctx *in_ctx;
- struct xhci_container_ctx *out_ctx;
- struct xhci_input_control_ctx *ctrl_ctx;
- struct xhci_ep_ctx *ep_ctx;
- int max_packet_size;
- int hw_max_packet_size;
- int ret = 0;
-
- out_ctx = xhci->devs[slot_id]->out_ctx;
- ep_ctx = xhci_get_ep_ctx(xhci, out_ctx, ep_index);
- hw_max_packet_size = MAX_PACKET_DECODED(ep_ctx->ep_info2);
- max_packet_size = urb->dev->ep0.desc.wMaxPacketSize;
- if (hw_max_packet_size != max_packet_size) {
- xhci_dbg(xhci, "Max Packet Size for ep 0 changed.\n");
- xhci_dbg(xhci, "Max packet size in usb_device = %d\n",
- max_packet_size);
- xhci_dbg(xhci, "Max packet size in xHCI HW = %d\n",
- hw_max_packet_size);
- xhci_dbg(xhci, "Issuing evaluate context command.\n");
-
- /* Set up the modified control endpoint 0 */
- xhci_endpoint_copy(xhci, xhci->devs[slot_id]->in_ctx,
- xhci->devs[slot_id]->out_ctx, ep_index);
- in_ctx = xhci->devs[slot_id]->in_ctx;
- ep_ctx = xhci_get_ep_ctx(xhci, in_ctx, ep_index);
- ep_ctx->ep_info2 &= ~MAX_PACKET_MASK;
- ep_ctx->ep_info2 |= MAX_PACKET(max_packet_size);
-
- /* Set up the input context flags for the command */
- /* FIXME: This won't work if a non-default control endpoint
- * changes max packet sizes.
- */
- ctrl_ctx = xhci_get_input_control_ctx(xhci, in_ctx);
- ctrl_ctx->add_flags = EP0_FLAG;
- ctrl_ctx->drop_flags = 0;
-
- xhci_dbg(xhci, "Slot %d input context\n", slot_id);
- xhci_dbg_ctx(xhci, in_ctx, ep_index);
- xhci_dbg(xhci, "Slot %d output context\n", slot_id);
- xhci_dbg_ctx(xhci, out_ctx, ep_index);
-
- ret = xhci_configure_endpoint(xhci, urb->dev, NULL,
- true, false);
-
- /* Clean up the input context for later use by bandwidth
- * functions.
- */
- ctrl_ctx->add_flags = SLOT_FLAG;
- }
- return ret;
-}
-
-/*
- * non-error returns are a promise to giveback() the urb later
- * we drop ownership so next owner (or urb unlink) can get it
- */
-int xhci_urb_enqueue(struct usb_hcd *hcd, struct urb *urb, gfp_t mem_flags)
-{
- struct xhci_hcd *xhci = hcd_to_xhci(hcd);
- unsigned long flags;
- int ret = 0;
- unsigned int slot_id, ep_index;
-
-
- if (!urb || xhci_check_args(hcd, urb->dev, urb->ep, true, __func__) <= 0)
- return -EINVAL;
-
- slot_id = urb->dev->slot_id;
- ep_index = xhci_get_endpoint_index(&urb->ep->desc);
-
- if (!xhci->devs || !xhci->devs[slot_id]) {
- if (!in_interrupt())
- dev_warn(&urb->dev->dev, "WARN: urb submitted for dev with no Slot ID\n");
- ret = -EINVAL;
- goto exit;
- }
- if (!test_bit(HCD_FLAG_HW_ACCESSIBLE, &hcd->flags)) {
- if (!in_interrupt())
- xhci_dbg(xhci, "urb submitted during PCI suspend\n");
- ret = -ESHUTDOWN;
- goto exit;
- }
- if (usb_endpoint_xfer_control(&urb->ep->desc)) {
- /* Check to see if the max packet size for the default control
- * endpoint changed during FS device enumeration
- */
- if (urb->dev->speed == USB_SPEED_FULL) {
- ret = xhci_check_maxpacket(xhci, slot_id,
- ep_index, urb);
- if (ret < 0)
- return ret;
- }
-
- /* We have a spinlock and interrupts disabled, so we must pass
- * atomic context to this function, which may allocate memory.
- */
- spin_lock_irqsave(&xhci->lock, flags);
- if (xhci->xhc_state & XHCI_STATE_DYING)
- goto dying;
- ret = xhci_queue_ctrl_tx(xhci, GFP_ATOMIC, urb,
- slot_id, ep_index);
- spin_unlock_irqrestore(&xhci->lock, flags);
- } else if (usb_endpoint_xfer_bulk(&urb->ep->desc)) {
- spin_lock_irqsave(&xhci->lock, flags);
- if (xhci->xhc_state & XHCI_STATE_DYING)
- goto dying;
- ret = xhci_queue_bulk_tx(xhci, GFP_ATOMIC, urb,
- slot_id, ep_index);
- spin_unlock_irqrestore(&xhci->lock, flags);
- } else if (usb_endpoint_xfer_int(&urb->ep->desc)) {
- spin_lock_irqsave(&xhci->lock, flags);
- if (xhci->xhc_state & XHCI_STATE_DYING)
- goto dying;
- ret = xhci_queue_intr_tx(xhci, GFP_ATOMIC, urb,
- slot_id, ep_index);
- spin_unlock_irqrestore(&xhci->lock, flags);
- } else {
- ret = -EINVAL;
- }
-exit:
- return ret;
-dying:
- xhci_dbg(xhci, "Ep 0x%x: URB %p submitted for "
- "non-responsive xHCI host.\n",
- urb->ep->desc.bEndpointAddress, urb);
- spin_unlock_irqrestore(&xhci->lock, flags);
- return -ESHUTDOWN;
-}
-
-/*
- * Remove the URB's TD from the endpoint ring. This may cause the HC to stop
- * USB transfers, potentially stopping in the middle of a TRB buffer. The HC
- * should pick up where it left off in the TD, unless a Set Transfer Ring
- * Dequeue Pointer is issued.
- *
- * The TRBs that make up the buffers for the canceled URB will be "removed" from
- * the ring. Since the ring is a contiguous structure, they can't be physically
- * removed. Instead, there are two options:
- *
- * 1) If the HC is in the middle of processing the URB to be canceled, we
- * simply move the ring's dequeue pointer past those TRBs using the Set
- * Transfer Ring Dequeue Pointer command. This will be the common case,
- * when drivers timeout on the last submitted URB and attempt to cancel.
- *
- * 2) If the HC is in the middle of a different TD, we turn the TRBs into a
- * series of 1-TRB transfer no-op TDs. (No-ops shouldn't be chained.) The
- * HC will need to invalidate the any TRBs it has cached after the stop
- * endpoint command, as noted in the xHCI 0.95 errata.
- *
- * 3) The TD may have completed by the time the Stop Endpoint Command
- * completes, so software needs to handle that case too.
- *
- * This function should protect against the TD enqueueing code ringing the
- * doorbell while this code is waiting for a Stop Endpoint command to complete.
- * It also needs to account for multiple cancellations on happening at the same
- * time for the same endpoint.
- *
- * Note that this function can be called in any context, or so says
- * usb_hcd_unlink_urb()
- */
-int xhci_urb_dequeue(struct usb_hcd *hcd, struct urb *urb, int status)
-{
- unsigned long flags;
- int ret;
- u32 temp;
- struct xhci_hcd *xhci;
- struct xhci_td *td;
- unsigned int ep_index;
- struct xhci_ring *ep_ring;
- struct xhci_virt_ep *ep;
-
- xhci = hcd_to_xhci(hcd);
- spin_lock_irqsave(&xhci->lock, flags);
- /* Make sure the URB hasn't completed or been unlinked already */
- ret = usb_hcd_check_unlink_urb(hcd, urb, status);
- if (ret || !urb->hcpriv)
- goto done;
- temp = xhci_readl(xhci, &xhci->op_regs->status);
- if (temp == 0xffffffff) {
- xhci_dbg(xhci, "HW died, freeing TD.\n");
- td = (struct xhci_td *) urb->hcpriv;
-
- usb_hcd_unlink_urb_from_ep(hcd, urb);
- spin_unlock_irqrestore(&xhci->lock, flags);
- usb_hcd_giveback_urb(xhci_to_hcd(xhci), urb, -ESHUTDOWN);
- kfree(td);
- return ret;
- }
- if (xhci->xhc_state & XHCI_STATE_DYING) {
- xhci_dbg(xhci, "Ep 0x%x: URB %p to be canceled on "
- "non-responsive xHCI host.\n",
- urb->ep->desc.bEndpointAddress, urb);
- /* Let the stop endpoint command watchdog timer (which set this
- * state) finish cleaning up the endpoint TD lists. We must
- * have caught it in the middle of dropping a lock and giving
- * back an URB.
- */
- goto done;
- }
-
- xhci_dbg(xhci, "Cancel URB %p\n", urb);
- xhci_dbg(xhci, "Event ring:\n");
- xhci_debug_ring(xhci, xhci->event_ring);
- ep_index = xhci_get_endpoint_index(&urb->ep->desc);
- ep = &xhci->devs[urb->dev->slot_id]->eps[ep_index];
- ep_ring = ep->ring;
- xhci_dbg(xhci, "Endpoint ring:\n");
- xhci_debug_ring(xhci, ep_ring);
- td = (struct xhci_td *) urb->hcpriv;
-
- list_add_tail(&td->cancelled_td_list, &ep->cancelled_td_list);
- /* Queue a stop endpoint command, but only if this is
- * the first cancellation to be handled.
- */
- if (!(ep->ep_state & EP_HALT_PENDING)) {
- ep->ep_state |= EP_HALT_PENDING;
- ep->stop_cmds_pending++;
- ep->stop_cmd_timer.expires = jiffies +
- XHCI_STOP_EP_CMD_TIMEOUT * HZ;
- add_timer(&ep->stop_cmd_timer);
- xhci_queue_stop_endpoint(xhci, urb->dev->slot_id, ep_index);
- xhci_ring_cmd_db(xhci);
- }
-done:
- spin_unlock_irqrestore(&xhci->lock, flags);
- return ret;
-}
-
-/* Drop an endpoint from a new bandwidth configuration for this device.
- * Only one call to this function is allowed per endpoint before
- * check_bandwidth() or reset_bandwidth() must be called.
- * A call to xhci_drop_endpoint() followed by a call to xhci_add_endpoint() will
- * add the endpoint to the schedule with possibly new parameters denoted by a
- * different endpoint descriptor in usb_host_endpoint.
- * A call to xhci_add_endpoint() followed by a call to xhci_drop_endpoint() is
- * not allowed.
- *
- * The USB core will not allow URBs to be queued to an endpoint that is being
- * disabled, so there's no need for mutual exclusion to protect
- * the xhci->devs[slot_id] structure.
- */
-int xhci_drop_endpoint(struct usb_hcd *hcd, struct usb_device *udev,
- struct usb_host_endpoint *ep)
-{
- struct xhci_hcd *xhci;
- struct xhci_container_ctx *in_ctx, *out_ctx;
- struct xhci_input_control_ctx *ctrl_ctx;
- struct xhci_slot_ctx *slot_ctx;
- unsigned int last_ctx;
- unsigned int ep_index;
- struct xhci_ep_ctx *ep_ctx;
- u32 drop_flag;
- u32 new_add_flags, new_drop_flags, new_slot_info;
- int ret;
-
- ret = xhci_check_args(hcd, udev, ep, 1, __func__);
- if (ret <= 0)
- return ret;
- xhci = hcd_to_xhci(hcd);
- xhci_dbg(xhci, "%s called for udev %p\n", __func__, udev);
-
- drop_flag = xhci_get_endpoint_flag(&ep->desc);
- if (drop_flag == SLOT_FLAG || drop_flag == EP0_FLAG) {
- xhci_dbg(xhci, "xHCI %s - can't drop slot or ep 0 %#x\n",
- __func__, drop_flag);
- return 0;
- }
-
- if (!xhci->devs || !xhci->devs[udev->slot_id]) {
- xhci_warn(xhci, "xHCI %s called with unaddressed device\n",
- __func__);
- return -EINVAL;
- }
-
- in_ctx = xhci->devs[udev->slot_id]->in_ctx;
- out_ctx = xhci->devs[udev->slot_id]->out_ctx;
- ctrl_ctx = xhci_get_input_control_ctx(xhci, in_ctx);
- ep_index = xhci_get_endpoint_index(&ep->desc);
- ep_ctx = xhci_get_ep_ctx(xhci, out_ctx, ep_index);
- /* If the HC already knows the endpoint is disabled,
- * or the HCD has noted it is disabled, ignore this request
- */
- if ((ep_ctx->ep_info & EP_STATE_MASK) == EP_STATE_DISABLED ||
- ctrl_ctx->drop_flags & xhci_get_endpoint_flag(&ep->desc)) {
- xhci_warn(xhci, "xHCI %s called with disabled ep %p\n",
- __func__, ep);
- return 0;
- }
-
- ctrl_ctx->drop_flags |= drop_flag;
- new_drop_flags = ctrl_ctx->drop_flags;
-
- ctrl_ctx->add_flags &= ~drop_flag;
- new_add_flags = ctrl_ctx->add_flags;
-
- last_ctx = xhci_last_valid_endpoint(ctrl_ctx->add_flags);
- slot_ctx = xhci_get_slot_ctx(xhci, in_ctx);
- /* Update the last valid endpoint context, if we deleted the last one */
- if ((slot_ctx->dev_info & LAST_CTX_MASK) > LAST_CTX(last_ctx)) {
- slot_ctx->dev_info &= ~LAST_CTX_MASK;
- slot_ctx->dev_info |= LAST_CTX(last_ctx);
- }
- new_slot_info = slot_ctx->dev_info;
-
- xhci_endpoint_zero(xhci, xhci->devs[udev->slot_id], ep);
-
- xhci_dbg(xhci, "drop ep 0x%x, slot id %d, new drop flags = %#x, new add flags = %#x, new slot info = %#x\n",
- (unsigned int) ep->desc.bEndpointAddress,
- udev->slot_id,
- (unsigned int) new_drop_flags,
- (unsigned int) new_add_flags,
- (unsigned int) new_slot_info);
- return 0;
-}
-
-/* Add an endpoint to a new possible bandwidth configuration for this device.
- * Only one call to this function is allowed per endpoint before
- * check_bandwidth() or reset_bandwidth() must be called.
- * A call to xhci_drop_endpoint() followed by a call to xhci_add_endpoint() will
- * add the endpoint to the schedule with possibly new parameters denoted by a
- * different endpoint descriptor in usb_host_endpoint.
- * A call to xhci_add_endpoint() followed by a call to xhci_drop_endpoint() is
- * not allowed.
- *
- * The USB core will not allow URBs to be queued to an endpoint until the
- * configuration or alt setting is installed in the device, so there's no need
- * for mutual exclusion to protect the xhci->devs[slot_id] structure.
- */
-int xhci_add_endpoint(struct usb_hcd *hcd, struct usb_device *udev,
- struct usb_host_endpoint *ep)
-{
- struct xhci_hcd *xhci;
- struct xhci_container_ctx *in_ctx, *out_ctx;
- unsigned int ep_index;
- struct xhci_ep_ctx *ep_ctx;
- struct xhci_slot_ctx *slot_ctx;
- struct xhci_input_control_ctx *ctrl_ctx;
- u32 added_ctxs;
- unsigned int last_ctx;
- u32 new_add_flags, new_drop_flags, new_slot_info;
- int ret = 0;
-
- ret = xhci_check_args(hcd, udev, ep, 1, __func__);
- if (ret <= 0) {
- /* So we won't queue a reset ep command for a root hub */
- ep->hcpriv = NULL;
- return ret;
- }
- xhci = hcd_to_xhci(hcd);
-
- added_ctxs = xhci_get_endpoint_flag(&ep->desc);
- last_ctx = xhci_last_valid_endpoint(added_ctxs);
- if (added_ctxs == SLOT_FLAG || added_ctxs == EP0_FLAG) {
- /* FIXME when we have to issue an evaluate endpoint command to
- * deal with ep0 max packet size changing once we get the
- * descriptors
- */
- xhci_dbg(xhci, "xHCI %s - can't add slot or ep 0 %#x\n",
- __func__, added_ctxs);
- return 0;
- }
-
- if (!xhci->devs || !xhci->devs[udev->slot_id]) {
- xhci_warn(xhci, "xHCI %s called with unaddressed device\n",
- __func__);
- return -EINVAL;
- }
-
- in_ctx = xhci->devs[udev->slot_id]->in_ctx;
- out_ctx = xhci->devs[udev->slot_id]->out_ctx;
- ctrl_ctx = xhci_get_input_control_ctx(xhci, in_ctx);
- ep_index = xhci_get_endpoint_index(&ep->desc);
- ep_ctx = xhci_get_ep_ctx(xhci, out_ctx, ep_index);
- /* If the HCD has already noted the endpoint is enabled,
- * ignore this request.
- */
- if (ctrl_ctx->add_flags & xhci_get_endpoint_flag(&ep->desc)) {
- xhci_warn(xhci, "xHCI %s called with enabled ep %p\n",
- __func__, ep);
- return 0;
- }
-
- /*
- * Configuration and alternate setting changes must be done in
- * process context, not interrupt context (or so documenation
- * for usb_set_interface() and usb_set_configuration() claim).
- */
- if (xhci_endpoint_init(xhci, xhci->devs[udev->slot_id],
- udev, ep, GFP_NOIO) < 0) {
- dev_dbg(&udev->dev, "%s - could not initialize ep %#x\n",
- __func__, ep->desc.bEndpointAddress);
- return -ENOMEM;
- }
-
- ctrl_ctx->add_flags |= added_ctxs;
- new_add_flags = ctrl_ctx->add_flags;
-
- /* If xhci_endpoint_disable() was called for this endpoint, but the
- * xHC hasn't been notified yet through the check_bandwidth() call,
- * this re-adds a new state for the endpoint from the new endpoint
- * descriptors. We must drop and re-add this endpoint, so we leave the
- * drop flags alone.
- */
- new_drop_flags = ctrl_ctx->drop_flags;
-
- slot_ctx = xhci_get_slot_ctx(xhci, in_ctx);
- /* Update the last valid endpoint context, if we just added one past */
- if ((slot_ctx->dev_info & LAST_CTX_MASK) < LAST_CTX(last_ctx)) {
- slot_ctx->dev_info &= ~LAST_CTX_MASK;
- slot_ctx->dev_info |= LAST_CTX(last_ctx);
- }
- new_slot_info = slot_ctx->dev_info;
-
- /* Store the usb_device pointer for later use */
- ep->hcpriv = udev;
-
- xhci_dbg(xhci, "add ep 0x%x, slot id %d, new drop flags = %#x, new add flags = %#x, new slot info = %#x\n",
- (unsigned int) ep->desc.bEndpointAddress,
- udev->slot_id,
- (unsigned int) new_drop_flags,
- (unsigned int) new_add_flags,
- (unsigned int) new_slot_info);
- return 0;
-}
-
-static void xhci_zero_in_ctx(struct xhci_hcd *xhci, struct xhci_virt_device *virt_dev)
-{
- struct xhci_input_control_ctx *ctrl_ctx;
- struct xhci_ep_ctx *ep_ctx;
- struct xhci_slot_ctx *slot_ctx;
- int i;
-
- /* When a device's add flag and drop flag are zero, any subsequent
- * configure endpoint command will leave that endpoint's state
- * untouched. Make sure we don't leave any old state in the input
- * endpoint contexts.
- */
- ctrl_ctx = xhci_get_input_control_ctx(xhci, virt_dev->in_ctx);
- ctrl_ctx->drop_flags = 0;
- ctrl_ctx->add_flags = 0;
- slot_ctx = xhci_get_slot_ctx(xhci, virt_dev->in_ctx);
- slot_ctx->dev_info &= ~LAST_CTX_MASK;
- /* Endpoint 0 is always valid */
- slot_ctx->dev_info |= LAST_CTX(1);
- for (i = 1; i < 31; ++i) {
- ep_ctx = xhci_get_ep_ctx(xhci, virt_dev->in_ctx, i);
- ep_ctx->ep_info = 0;
- ep_ctx->ep_info2 = 0;
- ep_ctx->deq = 0;
- ep_ctx->tx_info = 0;
- }
-}
-
-static int xhci_configure_endpoint_result(struct xhci_hcd *xhci,
- struct usb_device *udev, int *cmd_status)
-{
- int ret;
-
- switch (*cmd_status) {
- case COMP_ENOMEM:
- dev_warn(&udev->dev, "Not enough host controller resources "
- "for new device state.\n");
- ret = -ENOMEM;
- /* FIXME: can we allocate more resources for the HC? */
- break;
- case COMP_BW_ERR:
- dev_warn(&udev->dev, "Not enough bandwidth "
- "for new device state.\n");
- ret = -ENOSPC;
- /* FIXME: can we go back to the old state? */
- break;
- case COMP_TRB_ERR:
- /* the HCD set up something wrong */
- dev_warn(&udev->dev, "ERROR: Endpoint drop flag = 0, "
- "add flag = 1, "
- "and endpoint is not disabled.\n");
- ret = -EINVAL;
- break;
- case COMP_SUCCESS:
- dev_dbg(&udev->dev, "Successful Endpoint Configure command\n");
- ret = 0;
- break;
- default:
- xhci_err(xhci, "ERROR: unexpected command completion "
- "code 0x%x.\n", *cmd_status);
- ret = -EINVAL;
- break;
- }
- return ret;
-}
-
-static int xhci_evaluate_context_result(struct xhci_hcd *xhci,
- struct usb_device *udev, int *cmd_status)
-{
- int ret;
- struct xhci_virt_device *virt_dev = xhci->devs[udev->slot_id];
-
- switch (*cmd_status) {
- case COMP_EINVAL:
- dev_warn(&udev->dev, "WARN: xHCI driver setup invalid evaluate "
- "context command.\n");
- ret = -EINVAL;
- break;
- case COMP_EBADSLT:
- dev_warn(&udev->dev, "WARN: slot not enabled for"
- "evaluate context command.\n");
- case COMP_CTX_STATE:
- dev_warn(&udev->dev, "WARN: invalid context state for "
- "evaluate context command.\n");
- xhci_dbg_ctx(xhci, virt_dev->out_ctx, 1);
- ret = -EINVAL;
- break;
- case COMP_SUCCESS:
- dev_dbg(&udev->dev, "Successful evaluate context command\n");
- ret = 0;
- break;
- default:
- xhci_err(xhci, "ERROR: unexpected command completion "
- "code 0x%x.\n", *cmd_status);
- ret = -EINVAL;
- break;
- }
- return ret;
-}
-
-/* Issue a configure endpoint command or evaluate context command
- * and wait for it to finish.
- */
-static int xhci_configure_endpoint(struct xhci_hcd *xhci,
- struct usb_device *udev,
- struct xhci_command *command,
- bool ctx_change, bool must_succeed)
-{
- int ret;
- int timeleft;
- unsigned long flags;
- struct xhci_container_ctx *in_ctx;
- struct completion *cmd_completion;
- int *cmd_status;
- struct xhci_virt_device *virt_dev;
-
- spin_lock_irqsave(&xhci->lock, flags);
- virt_dev = xhci->devs[udev->slot_id];
- if (command) {
- in_ctx = command->in_ctx;
- cmd_completion = command->completion;
- cmd_status = &command->status;
- command->command_trb = xhci->cmd_ring->enqueue;
- list_add_tail(&command->cmd_list, &virt_dev->cmd_list);
- } else {
- in_ctx = virt_dev->in_ctx;
- cmd_completion = &virt_dev->cmd_completion;
- cmd_status = &virt_dev->cmd_status;
- }
-
- if (!ctx_change)
- ret = xhci_queue_configure_endpoint(xhci, in_ctx->dma,
- udev->slot_id, must_succeed);
- else
- ret = xhci_queue_evaluate_context(xhci, in_ctx->dma,
- udev->slot_id);
- if (ret < 0) {
- if (command)
- list_del(&command->cmd_list);
- spin_unlock_irqrestore(&xhci->lock, flags);
- xhci_dbg(xhci, "FIXME allocate a new ring segment\n");
- return -ENOMEM;
- }
- xhci_ring_cmd_db(xhci);
- spin_unlock_irqrestore(&xhci->lock, flags);
-
- /* Wait for the configure endpoint command to complete */
- timeleft = wait_for_completion_interruptible_timeout(
- cmd_completion,
- USB_CTRL_SET_TIMEOUT);
- if (timeleft <= 0) {
- xhci_warn(xhci, "%s while waiting for %s command\n",
- timeleft == 0 ? "Timeout" : "Signal",
- ctx_change == 0 ?
- "configure endpoint" :
- "evaluate context");
- /* FIXME cancel the configure endpoint command */
- return -ETIME;
- }
-
- if (!ctx_change)
- return xhci_configure_endpoint_result(xhci, udev, cmd_status);
- return xhci_evaluate_context_result(xhci, udev, cmd_status);
-}
-
-/* Called after one or more calls to xhci_add_endpoint() or
- * xhci_drop_endpoint(). If this call fails, the USB core is expected
- * to call xhci_reset_bandwidth().
- *
- * Since we are in the middle of changing either configuration or
- * installing a new alt setting, the USB core won't allow URBs to be
- * enqueued for any endpoint on the old config or interface. Nothing
- * else should be touching the xhci->devs[slot_id] structure, so we
- * don't need to take the xhci->lock for manipulating that.
- */
-int xhci_check_bandwidth(struct usb_hcd *hcd, struct usb_device *udev)
-{
- int i;
- int ret = 0;
- struct xhci_hcd *xhci;
- struct xhci_virt_device *virt_dev;
- struct xhci_input_control_ctx *ctrl_ctx;
- struct xhci_slot_ctx *slot_ctx;
-
- ret = xhci_check_args(hcd, udev, NULL, 0, __func__);
- if (ret <= 0)
- return ret;
- xhci = hcd_to_xhci(hcd);
-
- if (!udev->slot_id || !xhci->devs || !xhci->devs[udev->slot_id]) {
- xhci_warn(xhci, "xHCI %s called with unaddressed device\n",
- __func__);
- return -EINVAL;
- }
- xhci_dbg(xhci, "%s called for udev %p\n", __func__, udev);
- virt_dev = xhci->devs[udev->slot_id];
-
- /* See section 4.6.6 - A0 = 1; A1 = D0 = D1 = 0 */
- ctrl_ctx = xhci_get_input_control_ctx(xhci, virt_dev->in_ctx);
- ctrl_ctx->add_flags |= SLOT_FLAG;
- ctrl_ctx->add_flags &= ~EP0_FLAG;
- ctrl_ctx->drop_flags &= ~SLOT_FLAG;
- ctrl_ctx->drop_flags &= ~EP0_FLAG;
- xhci_dbg(xhci, "New Input Control Context:\n");
- slot_ctx = xhci_get_slot_ctx(xhci, virt_dev->in_ctx);
- xhci_dbg_ctx(xhci, virt_dev->in_ctx,
- LAST_CTX_TO_EP_NUM(slot_ctx->dev_info));
-
- ret = xhci_configure_endpoint(xhci, udev, NULL,
- false, false);
- if (ret) {
- /* Callee should call reset_bandwidth() */
- return ret;
- }
-
- xhci_dbg(xhci, "Output context after successful config ep cmd:\n");
- xhci_dbg_ctx(xhci, virt_dev->out_ctx,
- LAST_CTX_TO_EP_NUM(slot_ctx->dev_info));
-
- xhci_zero_in_ctx(xhci, virt_dev);
- /* Install new rings and free or cache any old rings */
- for (i = 1; i < 31; ++i) {
- if (!virt_dev->eps[i].new_ring)
- continue;
- /* Only cache or free the old ring if it exists.
- * It may not if this is the first add of an endpoint.
- */
- if (virt_dev->eps[i].ring) {
- xhci_free_or_cache_endpoint_ring(xhci, virt_dev, i);
- }
- virt_dev->eps[i].ring = virt_dev->eps[i].new_ring;
- virt_dev->eps[i].new_ring = NULL;
- }
-
- return ret;
-}
-
-void xhci_reset_bandwidth(struct usb_hcd *hcd, struct usb_device *udev)
-{
- struct xhci_hcd *xhci;
- struct xhci_virt_device *virt_dev;
- int i, ret;
-
- ret = xhci_check_args(hcd, udev, NULL, 0, __func__);
- if (ret <= 0)
- return;
- xhci = hcd_to_xhci(hcd);
-
- if (!xhci->devs || !xhci->devs[udev->slot_id]) {
- xhci_warn(xhci, "xHCI %s called with unaddressed device\n",
- __func__);
- return;
- }
- xhci_dbg(xhci, "%s called for udev %p\n", __func__, udev);
- virt_dev = xhci->devs[udev->slot_id];
- /* Free any rings allocated for added endpoints */
- for (i = 0; i < 31; ++i) {
- if (virt_dev->eps[i].new_ring) {
- xhci_ring_free(xhci, virt_dev->eps[i].new_ring);
- virt_dev->eps[i].new_ring = NULL;
- }
- }
- xhci_zero_in_ctx(xhci, virt_dev);
-}
-
-static void xhci_setup_input_ctx_for_config_ep(struct xhci_hcd *xhci,
- struct xhci_container_ctx *in_ctx,
- struct xhci_container_ctx *out_ctx,
- u32 add_flags, u32 drop_flags)
-{
- struct xhci_input_control_ctx *ctrl_ctx;
- ctrl_ctx = xhci_get_input_control_ctx(xhci, in_ctx);
- ctrl_ctx->add_flags = add_flags;
- ctrl_ctx->drop_flags = drop_flags;
- xhci_slot_copy(xhci, in_ctx, out_ctx);
- ctrl_ctx->add_flags |= SLOT_FLAG;
-
- xhci_dbg(xhci, "Input Context:\n");
- xhci_dbg_ctx(xhci, in_ctx, xhci_last_valid_endpoint(add_flags));
-}
-
-void xhci_setup_input_ctx_for_quirk(struct xhci_hcd *xhci,
- unsigned int slot_id, unsigned int ep_index,
- struct xhci_dequeue_state *deq_state)
-{
- struct xhci_container_ctx *in_ctx;
- struct xhci_ep_ctx *ep_ctx;
- u32 added_ctxs;
- dma_addr_t addr;
-
- xhci_endpoint_copy(xhci, xhci->devs[slot_id]->in_ctx,
- xhci->devs[slot_id]->out_ctx, ep_index);
- in_ctx = xhci->devs[slot_id]->in_ctx;
- ep_ctx = xhci_get_ep_ctx(xhci, in_ctx, ep_index);
- addr = xhci_trb_virt_to_dma(deq_state->new_deq_seg,
- deq_state->new_deq_ptr);
- if (addr == 0) {
- xhci_warn(xhci, "WARN Cannot submit config ep after "
- "reset ep command\n");
- xhci_warn(xhci, "WARN deq seg = %p, deq ptr = %p\n",
- deq_state->new_deq_seg,
- deq_state->new_deq_ptr);
- return;
- }
- ep_ctx->deq = addr | deq_state->new_cycle_state;
-
- added_ctxs = xhci_get_endpoint_flag_from_index(ep_index);
- xhci_setup_input_ctx_for_config_ep(xhci, xhci->devs[slot_id]->in_ctx,
- xhci->devs[slot_id]->out_ctx, added_ctxs, added_ctxs);
-}
-
-void xhci_cleanup_stalled_ring(struct xhci_hcd *xhci,
- struct usb_device *udev, unsigned int ep_index)
-{
- struct xhci_dequeue_state deq_state;
- struct xhci_virt_ep *ep;
-
- xhci_dbg(xhci, "Cleaning up stalled endpoint ring\n");
- ep = &xhci->devs[udev->slot_id]->eps[ep_index];
- /* We need to move the HW's dequeue pointer past this TD,
- * or it will attempt to resend it on the next doorbell ring.
- */
- xhci_find_new_dequeue_state(xhci, udev->slot_id,
- ep_index, ep->stopped_td,
- &deq_state);
-
- /* HW with the reset endpoint quirk will use the saved dequeue state to
- * issue a configure endpoint command later.
- */
- if (!(xhci->quirks & XHCI_RESET_EP_QUIRK)) {
- xhci_dbg(xhci, "Queueing new dequeue state\n");
- xhci_queue_new_dequeue_state(xhci, udev->slot_id,
- ep_index, &deq_state);
- } else {
- /* Better hope no one uses the input context between now and the
- * reset endpoint completion!
- */
- xhci_dbg(xhci, "Setting up input context for "
- "configure endpoint command\n");
- xhci_setup_input_ctx_for_quirk(xhci, udev->slot_id,
- ep_index, &deq_state);
- }
-}
-
-/* Deal with stalled endpoints. The core should have sent the control message
- * to clear the halt condition. However, we need to make the xHCI hardware
- * reset its sequence number, since a device will expect a sequence number of
- * zero after the halt condition is cleared.
- * Context: in_interrupt
- */
-void xhci_endpoint_reset(struct usb_hcd *hcd,
- struct usb_host_endpoint *ep)
-{
- struct xhci_hcd *xhci;
- struct usb_device *udev;
- unsigned int ep_index;
- unsigned long flags;
- int ret;
- struct xhci_virt_ep *virt_ep;
-
- xhci = hcd_to_xhci(hcd);
- udev = (struct usb_device *) ep->hcpriv;
- /* Called with a root hub endpoint (or an endpoint that wasn't added
- * with xhci_add_endpoint()
- */
- if (!ep->hcpriv)
- return;
- ep_index = xhci_get_endpoint_index(&ep->desc);
- virt_ep = &xhci->devs[udev->slot_id]->eps[ep_index];
- if (!virt_ep->stopped_td) {
- xhci_dbg(xhci, "Endpoint 0x%x not halted, refusing to reset.\n",
- ep->desc.bEndpointAddress);
- return;
- }
- if (usb_endpoint_xfer_control(&ep->desc)) {
- xhci_dbg(xhci, "Control endpoint stall already handled.\n");
- return;
- }
-
- xhci_dbg(xhci, "Queueing reset endpoint command\n");
- spin_lock_irqsave(&xhci->lock, flags);
- ret = xhci_queue_reset_ep(xhci, udev->slot_id, ep_index);
- /*
- * Can't change the ring dequeue pointer until it's transitioned to the
- * stopped state, which is only upon a successful reset endpoint
- * command. Better hope that last command worked!
- */
- if (!ret) {
- xhci_cleanup_stalled_ring(xhci, udev, ep_index);
- kfree(virt_ep->stopped_td);
- xhci_ring_cmd_db(xhci);
- }
- spin_unlock_irqrestore(&xhci->lock, flags);
-
- if (ret)
- xhci_warn(xhci, "FIXME allocate a new ring segment\n");
-}
-
-/*
- * This submits a Reset Device Command, which will set the device state to 0,
- * set the device address to 0, and disable all the endpoints except the default
- * control endpoint. The USB core should come back and call
- * xhci_address_device(), and then re-set up the configuration. If this is
- * called because of a usb_reset_and_verify_device(), then the old alternate
- * settings will be re-installed through the normal bandwidth allocation
- * functions.
- *
- * Wait for the Reset Device command to finish. Remove all structures
- * associated with the endpoints that were disabled. Clear the input device
- * structure? Cache the rings? Reset the control endpoint 0 max packet size?
- */
-int xhci_reset_device(struct usb_hcd *hcd, struct usb_device *udev)
-{
- int ret, i;
- unsigned long flags;
- struct xhci_hcd *xhci;
- unsigned int slot_id;
- struct xhci_virt_device *virt_dev;
- struct xhci_command *reset_device_cmd;
- int timeleft;
- int last_freed_endpoint;
-
- ret = xhci_check_args(hcd, udev, NULL, 0, __func__);
- if (ret <= 0)
- return ret;
- xhci = hcd_to_xhci(hcd);
- slot_id = udev->slot_id;
- virt_dev = xhci->devs[slot_id];
- if (!virt_dev) {
- xhci_dbg(xhci, "%s called with invalid slot ID %u\n",
- __func__, slot_id);
- return -EINVAL;
- }
-
- xhci_dbg(xhci, "Resetting device with slot ID %u\n", slot_id);
- /* Allocate the command structure that holds the struct completion.
- * Assume we're in process context, since the normal device reset
- * process has to wait for the device anyway. Storage devices are
- * reset as part of error handling, so use GFP_NOIO instead of
- * GFP_KERNEL.
- */
- reset_device_cmd = xhci_alloc_command(xhci, false, true, GFP_NOIO);
- if (!reset_device_cmd) {
- xhci_dbg(xhci, "Couldn't allocate command structure.\n");
- return -ENOMEM;
- }
-
- /* Attempt to submit the Reset Device command to the command ring */
- spin_lock_irqsave(&xhci->lock, flags);
- reset_device_cmd->command_trb = xhci->cmd_ring->enqueue;
- list_add_tail(&reset_device_cmd->cmd_list, &virt_dev->cmd_list);
- ret = xhci_queue_reset_device(xhci, slot_id);
- if (ret) {
- xhci_dbg(xhci, "FIXME: allocate a command ring segment\n");
- list_del(&reset_device_cmd->cmd_list);
- spin_unlock_irqrestore(&xhci->lock, flags);
- goto command_cleanup;
- }
- xhci_ring_cmd_db(xhci);
- spin_unlock_irqrestore(&xhci->lock, flags);
-
- /* Wait for the Reset Device command to finish */
- timeleft = wait_for_completion_interruptible_timeout(
- reset_device_cmd->completion,
- USB_CTRL_SET_TIMEOUT);
- if (timeleft <= 0) {
- xhci_warn(xhci, "%s while waiting for reset device command\n",
- timeleft == 0 ? "Timeout" : "Signal");
- spin_lock_irqsave(&xhci->lock, flags);
- /* The timeout might have raced with the event ring handler, so
- * only delete from the list if the item isn't poisoned.
- */
- if (reset_device_cmd->cmd_list.next != LIST_POISON1)
- list_del(&reset_device_cmd->cmd_list);
- spin_unlock_irqrestore(&xhci->lock, flags);
- ret = -ETIME;
- goto command_cleanup;
- }
-
- /* The Reset Device command can't fail, according to the 0.95/0.96 spec,
- * unless we tried to reset a slot ID that wasn't enabled,
- * or the device wasn't in the addressed or configured state.
- */
- ret = reset_device_cmd->status;
- switch (ret) {
- case COMP_EBADSLT: /* 0.95 completion code for bad slot ID */
- case COMP_CTX_STATE: /* 0.96 completion code for same thing */
- xhci_info(xhci, "Can't reset device (slot ID %u) in %s state\n",
- slot_id,
- xhci_get_slot_state(xhci, virt_dev->out_ctx));
- xhci_info(xhci, "Not freeing device rings.\n");
- /* Don't treat this as an error. May change my mind later. */
- ret = 0;
- goto command_cleanup;
- case COMP_SUCCESS:
- xhci_dbg(xhci, "Successful reset device command.\n");
- break;
- default:
- if (xhci_is_vendor_info_code(xhci, ret))
- break;
- xhci_warn(xhci, "Unknown completion code %u for "
- "reset device command.\n", ret);
- ret = -EINVAL;
- goto command_cleanup;
- }
-
- /* Everything but endpoint 0 is disabled, so free or cache the rings. */
- last_freed_endpoint = 1;
- for (i = 1; i < 31; ++i) {
- if (!virt_dev->eps[i].ring)
- continue;
- xhci_free_or_cache_endpoint_ring(xhci, virt_dev, i);
- last_freed_endpoint = i;
- }
- xhci_dbg(xhci, "Output context after successful reset device cmd:\n");
- xhci_dbg_ctx(xhci, virt_dev->out_ctx, last_freed_endpoint);
- ret = 0;
-
-command_cleanup:
- xhci_free_command(xhci, reset_device_cmd);
- return ret;
-}
-
-/*
- * At this point, the struct usb_device is about to go away, the device has
- * disconnected, and all traffic has been stopped and the endpoints have been
- * disabled. Free any HC data structures associated with that device.
- */
-void xhci_free_dev(struct usb_hcd *hcd, struct usb_device *udev)
-{
- struct xhci_hcd *xhci = hcd_to_xhci(hcd);
- struct xhci_virt_device *virt_dev;
- unsigned long flags;
- u32 state;
- int i;
-
- if (udev->slot_id == 0)
- return;
- virt_dev = xhci->devs[udev->slot_id];
- if (!virt_dev)
- return;
-
- /* Stop any wayward timer functions (which may grab the lock) */
- for (i = 0; i < 31; ++i) {
- virt_dev->eps[i].ep_state &= ~EP_HALT_PENDING;
- del_timer_sync(&virt_dev->eps[i].stop_cmd_timer);
- }
-
- spin_lock_irqsave(&xhci->lock, flags);
- /* Don't disable the slot if the host controller is dead. */
- state = xhci_readl(xhci, &xhci->op_regs->status);
- if (state == 0xffffffff || (xhci->xhc_state & XHCI_STATE_DYING)) {
- xhci_free_virt_device(xhci, udev->slot_id);
- spin_unlock_irqrestore(&xhci->lock, flags);
- return;
- }
-
- if (xhci_queue_slot_control(xhci, TRB_DISABLE_SLOT, udev->slot_id)) {
- spin_unlock_irqrestore(&xhci->lock, flags);
- xhci_dbg(xhci, "FIXME: allocate a command ring segment\n");
- return;
- }
- xhci_ring_cmd_db(xhci);
- spin_unlock_irqrestore(&xhci->lock, flags);
- /*
- * Event command completion handler will free any data structures
- * associated with the slot. XXX Can free sleep?
- */
-}
-
-/*
- * Returns 0 if the xHC ran out of device slots, the Enable Slot command
- * timed out, or allocating memory failed. Returns 1 on success.
- */
-int xhci_alloc_dev(struct usb_hcd *hcd, struct usb_device *udev)
-{
- struct xhci_hcd *xhci = hcd_to_xhci(hcd);
- unsigned long flags;
- int timeleft;
- int ret;
-
- spin_lock_irqsave(&xhci->lock, flags);
- ret = xhci_queue_slot_control(xhci, TRB_ENABLE_SLOT, 0);
- if (ret) {
- spin_unlock_irqrestore(&xhci->lock, flags);
- xhci_dbg(xhci, "FIXME: allocate a command ring segment\n");
- return 0;
- }
- xhci_ring_cmd_db(xhci);
- spin_unlock_irqrestore(&xhci->lock, flags);
-
- /* XXX: how much time for xHC slot assignment? */
- timeleft = wait_for_completion_interruptible_timeout(&xhci->addr_dev,
- USB_CTRL_SET_TIMEOUT);
- if (timeleft <= 0) {
- xhci_warn(xhci, "%s while waiting for a slot\n",
- timeleft == 0 ? "Timeout" : "Signal");
- /* FIXME cancel the enable slot request */
- return 0;
- }
-
- if (!xhci->slot_id) {
- xhci_err(xhci, "Error while assigning device slot ID\n");
- return 0;
- }
- /* xhci_alloc_virt_device() does not touch rings; no need to lock */
- if (!xhci_alloc_virt_device(xhci, xhci->slot_id, udev, GFP_KERNEL)) {
- /* Disable slot, if we can do it without mem alloc */
- xhci_warn(xhci, "Could not allocate xHCI USB device data structures\n");
- spin_lock_irqsave(&xhci->lock, flags);
- if (!xhci_queue_slot_control(xhci, TRB_DISABLE_SLOT, udev->slot_id))
- xhci_ring_cmd_db(xhci);
- spin_unlock_irqrestore(&xhci->lock, flags);
- return 0;
- }
- udev->slot_id = xhci->slot_id;
- /* Is this a LS or FS device under a HS hub? */
- /* Hub or peripherial? */
- return 1;
-}
-
-/*
- * Issue an Address Device command (which will issue a SetAddress request to
- * the device).
- * We should be protected by the usb_address0_mutex in khubd's hub_port_init, so
- * we should only issue and wait on one address command at the same time.
- *
- * We add one to the device address issued by the hardware because the USB core
- * uses address 1 for the root hubs (even though they're not really devices).
- */
-int xhci_address_device(struct usb_hcd *hcd, struct usb_device *udev)
-{
- unsigned long flags;
- int timeleft;
- struct xhci_virt_device *virt_dev;
- int ret = 0;
- struct xhci_hcd *xhci = hcd_to_xhci(hcd);
- struct xhci_slot_ctx *slot_ctx;
- struct xhci_input_control_ctx *ctrl_ctx;
- u64 temp_64;
-
- if (!udev->slot_id) {
- xhci_dbg(xhci, "Bad Slot ID %d\n", udev->slot_id);
- return -EINVAL;
- }
-
- virt_dev = xhci->devs[udev->slot_id];
-
- /* If this is a Set Address to an unconfigured device, setup ep 0 */
- if (!udev->config)
- xhci_setup_addressable_virt_dev(xhci, udev);
- /* Otherwise, assume the core has the device configured how it wants */
- xhci_dbg(xhci, "Slot ID %d Input Context:\n", udev->slot_id);
- xhci_dbg_ctx(xhci, virt_dev->in_ctx, 2);
-
- spin_lock_irqsave(&xhci->lock, flags);
- ret = xhci_queue_address_device(xhci, virt_dev->in_ctx->dma,
- udev->slot_id);
- if (ret) {
- spin_unlock_irqrestore(&xhci->lock, flags);
- xhci_dbg(xhci, "FIXME: allocate a command ring segment\n");
- return ret;
- }
- xhci_ring_cmd_db(xhci);
- spin_unlock_irqrestore(&xhci->lock, flags);
-
- /* ctrl tx can take up to 5 sec; XXX: need more time for xHC? */
- timeleft = wait_for_completion_interruptible_timeout(&xhci->addr_dev,
- USB_CTRL_SET_TIMEOUT);
- /* FIXME: From section 4.3.4: "Software shall be responsible for timing
- * the SetAddress() "recovery interval" required by USB and aborting the
- * command on a timeout.
- */
- if (timeleft <= 0) {
- xhci_warn(xhci, "%s while waiting for a slot\n",
- timeleft == 0 ? "Timeout" : "Signal");
- /* FIXME cancel the address device command */
- return -ETIME;
- }
-
- switch (virt_dev->cmd_status) {
- case COMP_CTX_STATE:
- case COMP_EBADSLT:
- xhci_err(xhci, "Setup ERROR: address device command for slot %d.\n",
- udev->slot_id);
- ret = -EINVAL;
- break;
- case COMP_TX_ERR:
- dev_warn(&udev->dev, "Device not responding to set address.\n");
- ret = -EPROTO;
- break;
- case COMP_SUCCESS:
- xhci_dbg(xhci, "Successful Address Device command\n");
- break;
- default:
- xhci_err(xhci, "ERROR: unexpected command completion "
- "code 0x%x.\n", virt_dev->cmd_status);
- xhci_dbg(xhci, "Slot ID %d Output Context:\n", udev->slot_id);
- xhci_dbg_ctx(xhci, virt_dev->out_ctx, 2);
- ret = -EINVAL;
- break;
- }
- if (ret) {
- return ret;
- }
- temp_64 = xhci_read_64(xhci, &xhci->op_regs->dcbaa_ptr);
- xhci_dbg(xhci, "Op regs DCBAA ptr = %#016llx\n", temp_64);
- xhci_dbg(xhci, "Slot ID %d dcbaa entry @%p = %#016llx\n",
- udev->slot_id,
- &xhci->dcbaa->dev_context_ptrs[udev->slot_id],
- (unsigned long long)
- xhci->dcbaa->dev_context_ptrs[udev->slot_id]);
- xhci_dbg(xhci, "Output Context DMA address = %#08llx\n",
- (unsigned long long)virt_dev->out_ctx->dma);
- xhci_dbg(xhci, "Slot ID %d Input Context:\n", udev->slot_id);
- xhci_dbg_ctx(xhci, virt_dev->in_ctx, 2);
- xhci_dbg(xhci, "Slot ID %d Output Context:\n", udev->slot_id);
- xhci_dbg_ctx(xhci, virt_dev->out_ctx, 2);
- /*
- * USB core uses address 1 for the roothubs, so we add one to the
- * address given back to us by the HC.
- */
- slot_ctx = xhci_get_slot_ctx(xhci, virt_dev->out_ctx);
- udev->devnum = (slot_ctx->dev_state & DEV_ADDR_MASK) + 1;
- /* Zero the input context control for later use */
- ctrl_ctx = xhci_get_input_control_ctx(xhci, virt_dev->in_ctx);
- ctrl_ctx->add_flags = 0;
- ctrl_ctx->drop_flags = 0;
-
- xhci_dbg(xhci, "Device address = %d\n", udev->devnum);
- /* XXX Meh, not sure if anyone else but choose_address uses this. */
- set_bit(udev->devnum, udev->bus->devmap.devicemap);
-
- return 0;
-}
-
-/* Once a hub descriptor is fetched for a device, we need to update the xHC's
- * internal data structures for the device.
- */
-int xhci_update_hub_device(struct usb_hcd *hcd, struct usb_device *hdev,
- struct usb_tt *tt, gfp_t mem_flags)
-{
- struct xhci_hcd *xhci = hcd_to_xhci(hcd);
- struct xhci_virt_device *vdev;
- struct xhci_command *config_cmd;
- struct xhci_input_control_ctx *ctrl_ctx;
- struct xhci_slot_ctx *slot_ctx;
- unsigned long flags;
- unsigned think_time;
- int ret;
-
- /* Ignore root hubs */
- if (!hdev->parent)
- return 0;
-
- vdev = xhci->devs[hdev->slot_id];
- if (!vdev) {
- xhci_warn(xhci, "Cannot update hub desc for unknown device.\n");
- return -EINVAL;
- }
- config_cmd = xhci_alloc_command(xhci, true, true, mem_flags);
- if (!config_cmd) {
- xhci_dbg(xhci, "Could not allocate xHCI command structure.\n");
- return -ENOMEM;
- }
-
- spin_lock_irqsave(&xhci->lock, flags);
- xhci_slot_copy(xhci, config_cmd->in_ctx, vdev->out_ctx);
- ctrl_ctx = xhci_get_input_control_ctx(xhci, config_cmd->in_ctx);
- ctrl_ctx->add_flags |= SLOT_FLAG;
- slot_ctx = xhci_get_slot_ctx(xhci, config_cmd->in_ctx);
- slot_ctx->dev_info |= DEV_HUB;
- if (tt->multi)
- slot_ctx->dev_info |= DEV_MTT;
- if (xhci->hci_version > 0x95) {
- xhci_dbg(xhci, "xHCI version %x needs hub "
- "TT think time and number of ports\n",
- (unsigned int) xhci->hci_version);
- slot_ctx->dev_info2 |= XHCI_MAX_PORTS(hdev->maxchild);
- /* Set TT think time - convert from ns to FS bit times.
- * 0 = 8 FS bit times, 1 = 16 FS bit times,
- * 2 = 24 FS bit times, 3 = 32 FS bit times.
- */
- think_time = tt->think_time;
- if (think_time != 0)
- think_time = (think_time / 666) - 1;
- slot_ctx->tt_info |= TT_THINK_TIME(think_time);
- } else {
- xhci_dbg(xhci, "xHCI version %x doesn't need hub "
- "TT think time or number of ports\n",
- (unsigned int) xhci->hci_version);
- }
- slot_ctx->dev_state = 0;
- spin_unlock_irqrestore(&xhci->lock, flags);
-
- xhci_dbg(xhci, "Set up %s for hub device.\n",
- (xhci->hci_version > 0x95) ?
- "configure endpoint" : "evaluate context");
- xhci_dbg(xhci, "Slot %u Input Context:\n", hdev->slot_id);
- xhci_dbg_ctx(xhci, config_cmd->in_ctx, 0);
-
- /* Issue and wait for the configure endpoint or
- * evaluate context command.
- */
- if (xhci->hci_version > 0x95)
- ret = xhci_configure_endpoint(xhci, hdev, config_cmd,
- false, false);
- else
- ret = xhci_configure_endpoint(xhci, hdev, config_cmd,
- true, false);
-
- xhci_dbg(xhci, "Slot %u Output Context:\n", hdev->slot_id);
- xhci_dbg_ctx(xhci, vdev->out_ctx, 0);
-
- xhci_free_command(xhci, config_cmd);
- return ret;
-}
-
-int xhci_get_frame(struct usb_hcd *hcd)
-{
- struct xhci_hcd *xhci = hcd_to_xhci(hcd);
- /* EHCI mods by the periodic size. Why? */
- return xhci_readl(xhci, &xhci->run_regs->microframe_index) >> 3;
-}
-
-MODULE_DESCRIPTION(DRIVER_DESC);
-MODULE_AUTHOR(DRIVER_AUTHOR);
-MODULE_LICENSE("GPL");
-
-static int __init xhci_hcd_init(void)
-{
-#ifdef CONFIG_PCI
- int retval = 0;
-
- retval = xhci_register_pci();
-
- if (retval < 0) {
- printk(KERN_DEBUG "Problem registering PCI driver.");
- return retval;
- }
-#endif
- /*
- * Check the compiler generated sizes of structures that must be laid
- * out in specific ways for hardware access.
- */
- BUILD_BUG_ON(sizeof(struct xhci_doorbell_array) != 256*32/8);
- BUILD_BUG_ON(sizeof(struct xhci_slot_ctx) != 8*32/8);
- BUILD_BUG_ON(sizeof(struct xhci_ep_ctx) != 8*32/8);
- /* xhci_device_control has eight fields, and also
- * embeds one xhci_slot_ctx and 31 xhci_ep_ctx
- */
- BUILD_BUG_ON(sizeof(struct xhci_stream_ctx) != 4*32/8);
- BUILD_BUG_ON(sizeof(union xhci_trb) != 4*32/8);
- BUILD_BUG_ON(sizeof(struct xhci_erst_entry) != 4*32/8);
- BUILD_BUG_ON(sizeof(struct xhci_cap_regs) != 7*32/8);
- BUILD_BUG_ON(sizeof(struct xhci_intr_reg) != 8*32/8);
- /* xhci_run_regs has eight fields and embeds 128 xhci_intr_regs */
- BUILD_BUG_ON(sizeof(struct xhci_run_regs) != (8+8*128)*32/8);
- BUILD_BUG_ON(sizeof(struct xhci_doorbell_array) != 256*32/8);
- return 0;
-}
-module_init(xhci_hcd_init);
-
-static void __exit xhci_hcd_cleanup(void)
-{
-#ifdef CONFIG_PCI
- xhci_unregister_pci();
-#endif
-}
-module_exit(xhci_hcd_cleanup);
#include <linux/usb.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/dmapool.h>
#include "xhci.h"
if (interval < 3)
interval = 3;
if ((1 << interval) != 8*ep->desc.bInterval)
- dev_warn(&udev->dev, "ep %#x - rounding interval to %d microframes\n",
- ep->desc.bEndpointAddress, 1 << interval);
+ dev_warn(&udev->dev,
+ "ep %#x - rounding interval"
+ " to %d microframes, "
+ "ep desc says %d microframes\n",
+ ep->desc.bEndpointAddress,
+ 1 << interval,
+ 8*ep->desc.bInterval);
}
break;
default:
*/
#include <linux/scatterlist.h>
+#include <linux/slab.h>
#include "xhci.h"
/*
--- /dev/null
+/*
+ * xHCI host controller driver
+ *
+ * Copyright (C) 2008 Intel Corp.
+ *
+ * Author: Sarah Sharp
+ * Some code borrowed from the Linux EHCI driver.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
+ * or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+ * for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software Foundation,
+ * Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#include <linux/irq.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/slab.h>
+
+#include "xhci.h"
+
+#define DRIVER_AUTHOR "Sarah Sharp"
+#define DRIVER_DESC "'eXtensible' Host Controller (xHC) Driver"
+
+/* Some 0.95 hardware can't handle the chain bit on a Link TRB being cleared */
+static int link_quirk;
+module_param(link_quirk, int, S_IRUGO | S_IWUSR);
+MODULE_PARM_DESC(link_quirk, "Don't clear the chain bit on a link TRB");
+
+/* TODO: copied from ehci-hcd.c - can this be refactored? */
+/*
+ * handshake - spin reading hc until handshake completes or fails
+ * @ptr: address of hc register to be read
+ * @mask: bits to look at in result of read
+ * @done: value of those bits when handshake succeeds
+ * @usec: timeout in microseconds
+ *
+ * Returns negative errno, or zero on success
+ *
+ * Success happens when the "mask" bits have the specified value (hardware
+ * handshake done). There are two failure modes: "usec" have passed (major
+ * hardware flakeout), or the register reads as all-ones (hardware removed).
+ */
+static int handshake(struct xhci_hcd *xhci, void __iomem *ptr,
+ u32 mask, u32 done, int usec)
+{
+ u32 result;
+
+ do {
+ result = xhci_readl(xhci, ptr);
+ if (result == ~(u32)0) /* card removed */
+ return -ENODEV;
+ result &= mask;
+ if (result == done)
+ return 0;
+ udelay(1);
+ usec--;
+ } while (usec > 0);
+ return -ETIMEDOUT;
+}
+
+/*
+ * Disable interrupts and begin the xHCI halting process.
+ */
+void xhci_quiesce(struct xhci_hcd *xhci)
+{
+ u32 halted;
+ u32 cmd;
+ u32 mask;
+
+ mask = ~(XHCI_IRQS);
+ halted = xhci_readl(xhci, &xhci->op_regs->status) & STS_HALT;
+ if (!halted)
+ mask &= ~CMD_RUN;
+
+ cmd = xhci_readl(xhci, &xhci->op_regs->command);
+ cmd &= mask;
+ xhci_writel(xhci, cmd, &xhci->op_regs->command);
+}
+
+/*
+ * Force HC into halt state.
+ *
+ * Disable any IRQs and clear the run/stop bit.
+ * HC will complete any current and actively pipelined transactions, and
+ * should halt within 16 microframes of the run/stop bit being cleared.
+ * Read HC Halted bit in the status register to see when the HC is finished.
+ * XXX: shouldn't we set HC_STATE_HALT here somewhere?
+ */
+int xhci_halt(struct xhci_hcd *xhci)
+{
+ xhci_dbg(xhci, "// Halt the HC\n");
+ xhci_quiesce(xhci);
+
+ return handshake(xhci, &xhci->op_regs->status,
+ STS_HALT, STS_HALT, XHCI_MAX_HALT_USEC);
+}
+
+/*
+ * Reset a halted HC, and set the internal HC state to HC_STATE_HALT.
+ *
+ * This resets pipelines, timers, counters, state machines, etc.
+ * Transactions will be terminated immediately, and operational registers
+ * will be set to their defaults.
+ */
+int xhci_reset(struct xhci_hcd *xhci)
+{
+ u32 command;
+ u32 state;
+
+ state = xhci_readl(xhci, &xhci->op_regs->status);
+ if ((state & STS_HALT) == 0) {
+ xhci_warn(xhci, "Host controller not halted, aborting reset.\n");
+ return 0;
+ }
+
+ xhci_dbg(xhci, "// Reset the HC\n");
+ command = xhci_readl(xhci, &xhci->op_regs->command);
+ command |= CMD_RESET;
+ xhci_writel(xhci, command, &xhci->op_regs->command);
+ /* XXX: Why does EHCI set this here? Shouldn't other code do this? */
+ xhci_to_hcd(xhci)->state = HC_STATE_HALT;
+
+ return handshake(xhci, &xhci->op_regs->command, CMD_RESET, 0, 250 * 1000);
+}
+
+
+#if 0
+/* Set up MSI-X table for entry 0 (may claim other entries later) */
+static int xhci_setup_msix(struct xhci_hcd *xhci)
+{
+ int ret;
+ struct pci_dev *pdev = to_pci_dev(xhci_to_hcd(xhci)->self.controller);
+
+ xhci->msix_count = 0;
+ /* XXX: did I do this right? ixgbe does kcalloc for more than one */
+ xhci->msix_entries = kmalloc(sizeof(struct msix_entry), GFP_KERNEL);
+ if (!xhci->msix_entries) {
+ xhci_err(xhci, "Failed to allocate MSI-X entries\n");
+ return -ENOMEM;
+ }
+ xhci->msix_entries[0].entry = 0;
+
+ ret = pci_enable_msix(pdev, xhci->msix_entries, xhci->msix_count);
+ if (ret) {
+ xhci_err(xhci, "Failed to enable MSI-X\n");
+ goto free_entries;
+ }
+
+ /*
+ * Pass the xhci pointer value as the request_irq "cookie".
+ * If more irqs are added, this will need to be unique for each one.
+ */
+ ret = request_irq(xhci->msix_entries[0].vector, &xhci_irq, 0,
+ "xHCI", xhci_to_hcd(xhci));
+ if (ret) {
+ xhci_err(xhci, "Failed to allocate MSI-X interrupt\n");
+ goto disable_msix;
+ }
+ xhci_dbg(xhci, "Finished setting up MSI-X\n");
+ return 0;
+
+disable_msix:
+ pci_disable_msix(pdev);
+free_entries:
+ kfree(xhci->msix_entries);
+ xhci->msix_entries = NULL;
+ return ret;
+}
+
+/* XXX: code duplication; can xhci_setup_msix call this? */
+/* Free any IRQs and disable MSI-X */
+static void xhci_cleanup_msix(struct xhci_hcd *xhci)
+{
+ struct pci_dev *pdev = to_pci_dev(xhci_to_hcd(xhci)->self.controller);
+ if (!xhci->msix_entries)
+ return;
+
+ free_irq(xhci->msix_entries[0].vector, xhci);
+ pci_disable_msix(pdev);
+ kfree(xhci->msix_entries);
+ xhci->msix_entries = NULL;
+ xhci_dbg(xhci, "Finished cleaning up MSI-X\n");
+}
+#endif
+
+/*
+ * Initialize memory for HCD and xHC (one-time init).
+ *
+ * Program the PAGESIZE register, initialize the device context array, create
+ * device contexts (?), set up a command ring segment (or two?), create event
+ * ring (one for now).
+ */
+int xhci_init(struct usb_hcd *hcd)
+{
+ struct xhci_hcd *xhci = hcd_to_xhci(hcd);
+ int retval = 0;
+
+ xhci_dbg(xhci, "xhci_init\n");
+ spin_lock_init(&xhci->lock);
+ if (link_quirk) {
+ xhci_dbg(xhci, "QUIRK: Not clearing Link TRB chain bits.\n");
+ xhci->quirks |= XHCI_LINK_TRB_QUIRK;
+ } else {
+ xhci_dbg(xhci, "xHCI doesn't need link TRB QUIRK\n");
+ }
+ retval = xhci_mem_init(xhci, GFP_KERNEL);
+ xhci_dbg(xhci, "Finished xhci_init\n");
+
+ return retval;
+}
+
+/*
+ * Called in interrupt context when there might be work
+ * queued on the event ring
+ *
+ * xhci->lock must be held by caller.
+ */
+static void xhci_work(struct xhci_hcd *xhci)
+{
+ u32 temp;
+ u64 temp_64;
+
+ /*
+ * Clear the op reg interrupt status first,
+ * so we can receive interrupts from other MSI-X interrupters.
+ * Write 1 to clear the interrupt status.
+ */
+ temp = xhci_readl(xhci, &xhci->op_regs->status);
+ temp |= STS_EINT;
+ xhci_writel(xhci, temp, &xhci->op_regs->status);
+ /* FIXME when MSI-X is supported and there are multiple vectors */
+ /* Clear the MSI-X event interrupt status */
+
+ /* Acknowledge the interrupt */
+ temp = xhci_readl(xhci, &xhci->ir_set->irq_pending);
+ temp |= 0x3;
+ xhci_writel(xhci, temp, &xhci->ir_set->irq_pending);
+ /* Flush posted writes */
+ xhci_readl(xhci, &xhci->ir_set->irq_pending);
+
+ if (xhci->xhc_state & XHCI_STATE_DYING)
+ xhci_dbg(xhci, "xHCI dying, ignoring interrupt. "
+ "Shouldn't IRQs be disabled?\n");
+ else
+ /* FIXME this should be a delayed service routine
+ * that clears the EHB.
+ */
+ xhci_handle_event(xhci);
+
+ /* Clear the event handler busy flag (RW1C); the event ring should be empty. */
+ temp_64 = xhci_read_64(xhci, &xhci->ir_set->erst_dequeue);
+ xhci_write_64(xhci, temp_64 | ERST_EHB, &xhci->ir_set->erst_dequeue);
+ /* Flush posted writes -- FIXME is this necessary? */
+ xhci_readl(xhci, &xhci->ir_set->irq_pending);
+}
+
+/*-------------------------------------------------------------------------*/
+
+/*
+ * xHCI spec says we can get an interrupt, and if the HC has an error condition,
+ * we might get bad data out of the event ring. Section 4.10.2.7 has a list of
+ * indicators of an event TRB error, but we check the status *first* to be safe.
+ */
+irqreturn_t xhci_irq(struct usb_hcd *hcd)
+{
+ struct xhci_hcd *xhci = hcd_to_xhci(hcd);
+ u32 temp, temp2;
+ union xhci_trb *trb;
+
+ spin_lock(&xhci->lock);
+ trb = xhci->event_ring->dequeue;
+ /* Check if the xHC generated the interrupt, or the irq is shared */
+ temp = xhci_readl(xhci, &xhci->op_regs->status);
+ temp2 = xhci_readl(xhci, &xhci->ir_set->irq_pending);
+ if (temp == 0xffffffff && temp2 == 0xffffffff)
+ goto hw_died;
+
+ if (!(temp & STS_EINT) && !ER_IRQ_PENDING(temp2)) {
+ spin_unlock(&xhci->lock);
+ return IRQ_NONE;
+ }
+ xhci_dbg(xhci, "op reg status = %08x\n", temp);
+ xhci_dbg(xhci, "ir set irq_pending = %08x\n", temp2);
+ xhci_dbg(xhci, "Event ring dequeue ptr:\n");
+ xhci_dbg(xhci, "@%llx %08x %08x %08x %08x\n",
+ (unsigned long long)xhci_trb_virt_to_dma(xhci->event_ring->deq_seg, trb),
+ lower_32_bits(trb->link.segment_ptr),
+ upper_32_bits(trb->link.segment_ptr),
+ (unsigned int) trb->link.intr_target,
+ (unsigned int) trb->link.control);
+
+ if (temp & STS_FATAL) {
+ xhci_warn(xhci, "WARNING: Host System Error\n");
+ xhci_halt(xhci);
+hw_died:
+ xhci_to_hcd(xhci)->state = HC_STATE_HALT;
+ spin_unlock(&xhci->lock);
+ return -ESHUTDOWN;
+ }
+
+ xhci_work(xhci);
+ spin_unlock(&xhci->lock);
+
+ return IRQ_HANDLED;
+}
+
+#ifdef CONFIG_USB_XHCI_HCD_DEBUGGING
+void xhci_event_ring_work(unsigned long arg)
+{
+ unsigned long flags;
+ int temp;
+ u64 temp_64;
+ struct xhci_hcd *xhci = (struct xhci_hcd *) arg;
+ int i, j;
+
+ xhci_dbg(xhci, "Poll event ring: %lu\n", jiffies);
+
+ spin_lock_irqsave(&xhci->lock, flags);
+ temp = xhci_readl(xhci, &xhci->op_regs->status);
+ xhci_dbg(xhci, "op reg status = 0x%x\n", temp);
+ if (temp == 0xffffffff || (xhci->xhc_state & XHCI_STATE_DYING)) {
+ xhci_dbg(xhci, "HW died, polling stopped.\n");
+ spin_unlock_irqrestore(&xhci->lock, flags);
+ return;
+ }
+
+ temp = xhci_readl(xhci, &xhci->ir_set->irq_pending);
+ xhci_dbg(xhci, "ir_set 0 pending = 0x%x\n", temp);
+ xhci_dbg(xhci, "No-op commands handled = %d\n", xhci->noops_handled);
+ xhci_dbg(xhci, "HC error bitmask = 0x%x\n", xhci->error_bitmask);
+ xhci->error_bitmask = 0;
+ xhci_dbg(xhci, "Event ring:\n");
+ xhci_debug_segment(xhci, xhci->event_ring->deq_seg);
+ xhci_dbg_ring_ptrs(xhci, xhci->event_ring);
+ temp_64 = xhci_read_64(xhci, &xhci->ir_set->erst_dequeue);
+ temp_64 &= ~ERST_PTR_MASK;
+ xhci_dbg(xhci, "ERST deq = 64'h%0lx\n", (long unsigned int) temp_64);
+ xhci_dbg(xhci, "Command ring:\n");
+ xhci_debug_segment(xhci, xhci->cmd_ring->deq_seg);
+ xhci_dbg_ring_ptrs(xhci, xhci->cmd_ring);
+ xhci_dbg_cmd_ptrs(xhci);
+ for (i = 0; i < MAX_HC_SLOTS; ++i) {
+ if (!xhci->devs[i])
+ continue;
+ for (j = 0; j < 31; ++j) {
+ struct xhci_ring *ring = xhci->devs[i]->eps[j].ring;
+ if (!ring)
+ continue;
+ xhci_dbg(xhci, "Dev %d endpoint ring %d:\n", i, j);
+ xhci_debug_segment(xhci, ring->deq_seg);
+ }
+ }
+
+ if (xhci->noops_submitted != NUM_TEST_NOOPS)
+ if (xhci_setup_one_noop(xhci))
+ xhci_ring_cmd_db(xhci);
+ spin_unlock_irqrestore(&xhci->lock, flags);
+
+ if (!xhci->zombie)
+ mod_timer(&xhci->event_ring_timer, jiffies + POLL_TIMEOUT * HZ);
+ else
+ xhci_dbg(xhci, "Quit polling the event ring.\n");
+}
+#endif
+
+/*
+ * Start the HC after it was halted.
+ *
+ * This function is called by the USB core when the HC driver is added.
+ * Its opposite is xhci_stop().
+ *
+ * xhci_init() must be called once before this function can be called.
+ * Reset the HC, enable device slot contexts, program DCBAAP, and
+ * set command ring pointer and event ring pointer.
+ *
+ * Setup MSI-X vectors and enable interrupts.
+ */
+int xhci_run(struct usb_hcd *hcd)
+{
+ u32 temp;
+ u64 temp_64;
+ struct xhci_hcd *xhci = hcd_to_xhci(hcd);
+ void (*doorbell)(struct xhci_hcd *) = NULL;
+
+ hcd->uses_new_polling = 1;
+ hcd->poll_rh = 0;
+
+ xhci_dbg(xhci, "xhci_run\n");
+#if 0 /* FIXME: MSI not setup yet */
+ /* Do this at the very last minute */
+ ret = xhci_setup_msix(xhci);
+ if (!ret)
+ return ret;
+
+ return -ENOSYS;
+#endif
+#ifdef CONFIG_USB_XHCI_HCD_DEBUGGING
+ init_timer(&xhci->event_ring_timer);
+ xhci->event_ring_timer.data = (unsigned long) xhci;
+ xhci->event_ring_timer.function = xhci_event_ring_work;
+ /* Poll the event ring */
+ xhci->event_ring_timer.expires = jiffies + POLL_TIMEOUT * HZ;
+ xhci->zombie = 0;
+ xhci_dbg(xhci, "Setting event ring polling timer\n");
+ add_timer(&xhci->event_ring_timer);
+#endif
+
+ xhci_dbg(xhci, "Command ring memory map follows:\n");
+ xhci_debug_ring(xhci, xhci->cmd_ring);
+ xhci_dbg_ring_ptrs(xhci, xhci->cmd_ring);
+ xhci_dbg_cmd_ptrs(xhci);
+
+ xhci_dbg(xhci, "ERST memory map follows:\n");
+ xhci_dbg_erst(xhci, &xhci->erst);
+ xhci_dbg(xhci, "Event ring:\n");
+ xhci_debug_ring(xhci, xhci->event_ring);
+ xhci_dbg_ring_ptrs(xhci, xhci->event_ring);
+ temp_64 = xhci_read_64(xhci, &xhci->ir_set->erst_dequeue);
+ temp_64 &= ~ERST_PTR_MASK;
+ xhci_dbg(xhci, "ERST deq = 64'h%0lx\n", (long unsigned int) temp_64);
+
+ xhci_dbg(xhci, "// Set the interrupt modulation register\n");
+ temp = xhci_readl(xhci, &xhci->ir_set->irq_control);
+ temp &= ~ER_IRQ_INTERVAL_MASK;
+ temp |= (u32) 160;
+ xhci_writel(xhci, temp, &xhci->ir_set->irq_control);
+
+ /* Set the HCD state before we enable the irqs */
+ hcd->state = HC_STATE_RUNNING;
+ temp = xhci_readl(xhci, &xhci->op_regs->command);
+ temp |= (CMD_EIE);
+ xhci_dbg(xhci, "// Enable interrupts, cmd = 0x%x.\n",
+ temp);
+ xhci_writel(xhci, temp, &xhci->op_regs->command);
+
+ temp = xhci_readl(xhci, &xhci->ir_set->irq_pending);
+ xhci_dbg(xhci, "// Enabling event ring interrupter %p by writing 0x%x to irq_pending\n",
+ xhci->ir_set, (unsigned int) ER_IRQ_ENABLE(temp));
+ xhci_writel(xhci, ER_IRQ_ENABLE(temp),
+ &xhci->ir_set->irq_pending);
+ xhci_print_ir_set(xhci, xhci->ir_set, 0);
+
+ if (NUM_TEST_NOOPS > 0)
+ doorbell = xhci_setup_one_noop(xhci);
+
+ temp = xhci_readl(xhci, &xhci->op_regs->command);
+ temp |= (CMD_RUN);
+ xhci_dbg(xhci, "// Turn on HC, cmd = 0x%x.\n",
+ temp);
+ xhci_writel(xhci, temp, &xhci->op_regs->command);
+ /* Flush PCI posted writes */
+ temp = xhci_readl(xhci, &xhci->op_regs->command);
+ xhci_dbg(xhci, "// @%p = 0x%x\n", &xhci->op_regs->command, temp);
+ if (doorbell)
+ (*doorbell)(xhci);
+
+ xhci_dbg(xhci, "Finished xhci_run\n");
+ return 0;
+}
+
+/*
+ * Stop xHCI driver.
+ *
+ * This function is called by the USB core when the HC driver is removed.
+ * Its opposite is xhci_run().
+ *
+ * Disable device contexts, disable IRQs, and quiesce the HC.
+ * Reset the HC, finish any completed transactions, and cleanup memory.
+ */
+void xhci_stop(struct usb_hcd *hcd)
+{
+ u32 temp;
+ struct xhci_hcd *xhci = hcd_to_xhci(hcd);
+
+ spin_lock_irq(&xhci->lock);
+ xhci_halt(xhci);
+ xhci_reset(xhci);
+ spin_unlock_irq(&xhci->lock);
+
+#if 0 /* No MSI yet */
+ xhci_cleanup_msix(xhci);
+#endif
+#ifdef CONFIG_USB_XHCI_HCD_DEBUGGING
+ /* Tell the event ring poll function not to reschedule */
+ xhci->zombie = 1;
+ del_timer_sync(&xhci->event_ring_timer);
+#endif
+
+ xhci_dbg(xhci, "// Disabling event ring interrupts\n");
+ temp = xhci_readl(xhci, &xhci->op_regs->status);
+ xhci_writel(xhci, temp & ~STS_EINT, &xhci->op_regs->status);
+ temp = xhci_readl(xhci, &xhci->ir_set->irq_pending);
+ xhci_writel(xhci, ER_IRQ_DISABLE(temp),
+ &xhci->ir_set->irq_pending);
+ xhci_print_ir_set(xhci, xhci->ir_set, 0);
+
+ xhci_dbg(xhci, "cleaning up memory\n");
+ xhci_mem_cleanup(xhci);
+ xhci_dbg(xhci, "xhci_stop completed - status = %x\n",
+ xhci_readl(xhci, &xhci->op_regs->status));
+}
+
+/*
+ * Shutdown HC (not bus-specific)
+ *
+ * This is called when the machine is rebooting or halting. We assume that the
+ * machine will be powered off, and the HC's internal state will be reset.
+ * Don't bother to free memory.
+ */
+void xhci_shutdown(struct usb_hcd *hcd)
+{
+ struct xhci_hcd *xhci = hcd_to_xhci(hcd);
+
+ spin_lock_irq(&xhci->lock);
+ xhci_halt(xhci);
+ spin_unlock_irq(&xhci->lock);
+
+#if 0
+ xhci_cleanup_msix(xhci);
+#endif
+
+ xhci_dbg(xhci, "xhci_shutdown completed - status = %x\n",
+ xhci_readl(xhci, &xhci->op_regs->status));
+}
+
+/*-------------------------------------------------------------------------*/
+
+/**
+ * xhci_get_endpoint_index - Used for passing endpoint bitmasks between the core and
+ * HCDs. Find the index for an endpoint given its descriptor. Use the return
+ * value to right shift 1 for the bitmask.
+ *
+ * Index = (epnum * 2) + direction - 1,
+ * where direction = 0 for OUT, 1 for IN.
+ * For control endpoints, the IN index is used (OUT index is unused), so
+ * index = (epnum * 2) + direction - 1 = (epnum * 2) + 1 - 1 = (epnum * 2)
+ */
+unsigned int xhci_get_endpoint_index(struct usb_endpoint_descriptor *desc)
+{
+ unsigned int index;
+ if (usb_endpoint_xfer_control(desc))
+ index = (unsigned int) (usb_endpoint_num(desc)*2);
+ else
+ index = (unsigned int) (usb_endpoint_num(desc)*2) +
+ (usb_endpoint_dir_in(desc) ? 1 : 0) - 1;
+ return index;
+}
+
+/* Find the flag for this endpoint (for use in the control context). Use the
+ * endpoint index to create a bitmask. The slot context is bit 0, endpoint 0 is
+ * bit 1, etc.
+ */
+unsigned int xhci_get_endpoint_flag(struct usb_endpoint_descriptor *desc)
+{
+ return 1 << (xhci_get_endpoint_index(desc) + 1);
+}
+
+/* Find the flag for this endpoint (for use in the control context). Use the
+ * endpoint index to create a bitmask. The slot context is bit 0, endpoint 0 is
+ * bit 1, etc.
+ */
+unsigned int xhci_get_endpoint_flag_from_index(unsigned int ep_index)
+{
+ return 1 << (ep_index + 1);
+}
+
+/* Compute the last valid endpoint context index. Basically, this is the
+ * endpoint index plus one. For slot contexts with more than valid endpoint,
+ * we find the most significant bit set in the added contexts flags.
+ * e.g. ep 1 IN (with epnum 0x81) => added_ctxs = 0b1000
+ * fls(0b1000) = 4, but the endpoint context index is 3, so subtract one.
+ */
+unsigned int xhci_last_valid_endpoint(u32 added_ctxs)
+{
+ return fls(added_ctxs) - 1;
+}
+
+/* Returns 1 if the arguments are OK;
+ * returns 0 this is a root hub; returns -EINVAL for NULL pointers.
+ */
+int xhci_check_args(struct usb_hcd *hcd, struct usb_device *udev,
+ struct usb_host_endpoint *ep, int check_ep, const char *func) {
+ if (!hcd || (check_ep && !ep) || !udev) {
+ printk(KERN_DEBUG "xHCI %s called with invalid args\n",
+ func);
+ return -EINVAL;
+ }
+ if (!udev->parent) {
+ printk(KERN_DEBUG "xHCI %s called for root hub\n",
+ func);
+ return 0;
+ }
+ if (!udev->slot_id) {
+ printk(KERN_DEBUG "xHCI %s called with unaddressed device\n",
+ func);
+ return -EINVAL;
+ }
+ return 1;
+}
+
+static int xhci_configure_endpoint(struct xhci_hcd *xhci,
+ struct usb_device *udev, struct xhci_command *command,
+ bool ctx_change, bool must_succeed);
+
+/*
+ * Full speed devices may have a max packet size greater than 8 bytes, but the
+ * USB core doesn't know that until it reads the first 8 bytes of the
+ * descriptor. If the usb_device's max packet size changes after that point,
+ * we need to issue an evaluate context command and wait on it.
+ */
+static int xhci_check_maxpacket(struct xhci_hcd *xhci, unsigned int slot_id,
+ unsigned int ep_index, struct urb *urb)
+{
+ struct xhci_container_ctx *in_ctx;
+ struct xhci_container_ctx *out_ctx;
+ struct xhci_input_control_ctx *ctrl_ctx;
+ struct xhci_ep_ctx *ep_ctx;
+ int max_packet_size;
+ int hw_max_packet_size;
+ int ret = 0;
+
+ out_ctx = xhci->devs[slot_id]->out_ctx;
+ ep_ctx = xhci_get_ep_ctx(xhci, out_ctx, ep_index);
+ hw_max_packet_size = MAX_PACKET_DECODED(ep_ctx->ep_info2);
+ max_packet_size = urb->dev->ep0.desc.wMaxPacketSize;
+ if (hw_max_packet_size != max_packet_size) {
+ xhci_dbg(xhci, "Max Packet Size for ep 0 changed.\n");
+ xhci_dbg(xhci, "Max packet size in usb_device = %d\n",
+ max_packet_size);
+ xhci_dbg(xhci, "Max packet size in xHCI HW = %d\n",
+ hw_max_packet_size);
+ xhci_dbg(xhci, "Issuing evaluate context command.\n");
+
+ /* Set up the modified control endpoint 0 */
+ xhci_endpoint_copy(xhci, xhci->devs[slot_id]->in_ctx,
+ xhci->devs[slot_id]->out_ctx, ep_index);
+ in_ctx = xhci->devs[slot_id]->in_ctx;
+ ep_ctx = xhci_get_ep_ctx(xhci, in_ctx, ep_index);
+ ep_ctx->ep_info2 &= ~MAX_PACKET_MASK;
+ ep_ctx->ep_info2 |= MAX_PACKET(max_packet_size);
+
+ /* Set up the input context flags for the command */
+ /* FIXME: This won't work if a non-default control endpoint
+ * changes max packet sizes.
+ */
+ ctrl_ctx = xhci_get_input_control_ctx(xhci, in_ctx);
+ ctrl_ctx->add_flags = EP0_FLAG;
+ ctrl_ctx->drop_flags = 0;
+
+ xhci_dbg(xhci, "Slot %d input context\n", slot_id);
+ xhci_dbg_ctx(xhci, in_ctx, ep_index);
+ xhci_dbg(xhci, "Slot %d output context\n", slot_id);
+ xhci_dbg_ctx(xhci, out_ctx, ep_index);
+
+ ret = xhci_configure_endpoint(xhci, urb->dev, NULL,
+ true, false);
+
+ /* Clean up the input context for later use by bandwidth
+ * functions.
+ */
+ ctrl_ctx->add_flags = SLOT_FLAG;
+ }
+ return ret;
+}
+
+/*
+ * non-error returns are a promise to giveback() the urb later
+ * we drop ownership so next owner (or urb unlink) can get it
+ */
+int xhci_urb_enqueue(struct usb_hcd *hcd, struct urb *urb, gfp_t mem_flags)
+{
+ struct xhci_hcd *xhci = hcd_to_xhci(hcd);
+ unsigned long flags;
+ int ret = 0;
+ unsigned int slot_id, ep_index;
+
+
+ if (!urb || xhci_check_args(hcd, urb->dev, urb->ep, true, __func__) <= 0)
+ return -EINVAL;
+
+ slot_id = urb->dev->slot_id;
+ ep_index = xhci_get_endpoint_index(&urb->ep->desc);
+
+ if (!xhci->devs || !xhci->devs[slot_id]) {
+ if (!in_interrupt())
+ dev_warn(&urb->dev->dev, "WARN: urb submitted for dev with no Slot ID\n");
+ ret = -EINVAL;
+ goto exit;
+ }
+ if (!test_bit(HCD_FLAG_HW_ACCESSIBLE, &hcd->flags)) {
+ if (!in_interrupt())
+ xhci_dbg(xhci, "urb submitted during PCI suspend\n");
+ ret = -ESHUTDOWN;
+ goto exit;
+ }
+ if (usb_endpoint_xfer_control(&urb->ep->desc)) {
+ /* Check to see if the max packet size for the default control
+ * endpoint changed during FS device enumeration
+ */
+ if (urb->dev->speed == USB_SPEED_FULL) {
+ ret = xhci_check_maxpacket(xhci, slot_id,
+ ep_index, urb);
+ if (ret < 0)
+ return ret;
+ }
+
+ /* We have a spinlock and interrupts disabled, so we must pass
+ * atomic context to this function, which may allocate memory.
+ */
+ spin_lock_irqsave(&xhci->lock, flags);
+ if (xhci->xhc_state & XHCI_STATE_DYING)
+ goto dying;
+ ret = xhci_queue_ctrl_tx(xhci, GFP_ATOMIC, urb,
+ slot_id, ep_index);
+ spin_unlock_irqrestore(&xhci->lock, flags);
+ } else if (usb_endpoint_xfer_bulk(&urb->ep->desc)) {
+ spin_lock_irqsave(&xhci->lock, flags);
+ if (xhci->xhc_state & XHCI_STATE_DYING)
+ goto dying;
+ ret = xhci_queue_bulk_tx(xhci, GFP_ATOMIC, urb,
+ slot_id, ep_index);
+ spin_unlock_irqrestore(&xhci->lock, flags);
+ } else if (usb_endpoint_xfer_int(&urb->ep->desc)) {
+ spin_lock_irqsave(&xhci->lock, flags);
+ if (xhci->xhc_state & XHCI_STATE_DYING)
+ goto dying;
+ ret = xhci_queue_intr_tx(xhci, GFP_ATOMIC, urb,
+ slot_id, ep_index);
+ spin_unlock_irqrestore(&xhci->lock, flags);
+ } else {
+ ret = -EINVAL;
+ }
+exit:
+ return ret;
+dying:
+ xhci_dbg(xhci, "Ep 0x%x: URB %p submitted for "
+ "non-responsive xHCI host.\n",
+ urb->ep->desc.bEndpointAddress, urb);
+ spin_unlock_irqrestore(&xhci->lock, flags);
+ return -ESHUTDOWN;
+}
+
+/*
+ * Remove the URB's TD from the endpoint ring. This may cause the HC to stop
+ * USB transfers, potentially stopping in the middle of a TRB buffer. The HC
+ * should pick up where it left off in the TD, unless a Set Transfer Ring
+ * Dequeue Pointer is issued.
+ *
+ * The TRBs that make up the buffers for the canceled URB will be "removed" from
+ * the ring. Since the ring is a contiguous structure, they can't be physically
+ * removed. Instead, there are two options:
+ *
+ * 1) If the HC is in the middle of processing the URB to be canceled, we
+ * simply move the ring's dequeue pointer past those TRBs using the Set
+ * Transfer Ring Dequeue Pointer command. This will be the common case,
+ * when drivers timeout on the last submitted URB and attempt to cancel.
+ *
+ * 2) If the HC is in the middle of a different TD, we turn the TRBs into a
+ * series of 1-TRB transfer no-op TDs. (No-ops shouldn't be chained.) The
+ * HC will need to invalidate the any TRBs it has cached after the stop
+ * endpoint command, as noted in the xHCI 0.95 errata.
+ *
+ * 3) The TD may have completed by the time the Stop Endpoint Command
+ * completes, so software needs to handle that case too.
+ *
+ * This function should protect against the TD enqueueing code ringing the
+ * doorbell while this code is waiting for a Stop Endpoint command to complete.
+ * It also needs to account for multiple cancellations on happening at the same
+ * time for the same endpoint.
+ *
+ * Note that this function can be called in any context, or so says
+ * usb_hcd_unlink_urb()
+ */
+int xhci_urb_dequeue(struct usb_hcd *hcd, struct urb *urb, int status)
+{
+ unsigned long flags;
+ int ret;
+ u32 temp;
+ struct xhci_hcd *xhci;
+ struct xhci_td *td;
+ unsigned int ep_index;
+ struct xhci_ring *ep_ring;
+ struct xhci_virt_ep *ep;
+
+ xhci = hcd_to_xhci(hcd);
+ spin_lock_irqsave(&xhci->lock, flags);
+ /* Make sure the URB hasn't completed or been unlinked already */
+ ret = usb_hcd_check_unlink_urb(hcd, urb, status);
+ if (ret || !urb->hcpriv)
+ goto done;
+ temp = xhci_readl(xhci, &xhci->op_regs->status);
+ if (temp == 0xffffffff) {
+ xhci_dbg(xhci, "HW died, freeing TD.\n");
+ td = (struct xhci_td *) urb->hcpriv;
+
+ usb_hcd_unlink_urb_from_ep(hcd, urb);
+ spin_unlock_irqrestore(&xhci->lock, flags);
+ usb_hcd_giveback_urb(xhci_to_hcd(xhci), urb, -ESHUTDOWN);
+ kfree(td);
+ return ret;
+ }
+ if (xhci->xhc_state & XHCI_STATE_DYING) {
+ xhci_dbg(xhci, "Ep 0x%x: URB %p to be canceled on "
+ "non-responsive xHCI host.\n",
+ urb->ep->desc.bEndpointAddress, urb);
+ /* Let the stop endpoint command watchdog timer (which set this
+ * state) finish cleaning up the endpoint TD lists. We must
+ * have caught it in the middle of dropping a lock and giving
+ * back an URB.
+ */
+ goto done;
+ }
+
+ xhci_dbg(xhci, "Cancel URB %p\n", urb);
+ xhci_dbg(xhci, "Event ring:\n");
+ xhci_debug_ring(xhci, xhci->event_ring);
+ ep_index = xhci_get_endpoint_index(&urb->ep->desc);
+ ep = &xhci->devs[urb->dev->slot_id]->eps[ep_index];
+ ep_ring = ep->ring;
+ xhci_dbg(xhci, "Endpoint ring:\n");
+ xhci_debug_ring(xhci, ep_ring);
+ td = (struct xhci_td *) urb->hcpriv;
+
+ list_add_tail(&td->cancelled_td_list, &ep->cancelled_td_list);
+ /* Queue a stop endpoint command, but only if this is
+ * the first cancellation to be handled.
+ */
+ if (!(ep->ep_state & EP_HALT_PENDING)) {
+ ep->ep_state |= EP_HALT_PENDING;
+ ep->stop_cmds_pending++;
+ ep->stop_cmd_timer.expires = jiffies +
+ XHCI_STOP_EP_CMD_TIMEOUT * HZ;
+ add_timer(&ep->stop_cmd_timer);
+ xhci_queue_stop_endpoint(xhci, urb->dev->slot_id, ep_index);
+ xhci_ring_cmd_db(xhci);
+ }
+done:
+ spin_unlock_irqrestore(&xhci->lock, flags);
+ return ret;
+}
+
+/* Drop an endpoint from a new bandwidth configuration for this device.
+ * Only one call to this function is allowed per endpoint before
+ * check_bandwidth() or reset_bandwidth() must be called.
+ * A call to xhci_drop_endpoint() followed by a call to xhci_add_endpoint() will
+ * add the endpoint to the schedule with possibly new parameters denoted by a
+ * different endpoint descriptor in usb_host_endpoint.
+ * A call to xhci_add_endpoint() followed by a call to xhci_drop_endpoint() is
+ * not allowed.
+ *
+ * The USB core will not allow URBs to be queued to an endpoint that is being
+ * disabled, so there's no need for mutual exclusion to protect
+ * the xhci->devs[slot_id] structure.
+ */
+int xhci_drop_endpoint(struct usb_hcd *hcd, struct usb_device *udev,
+ struct usb_host_endpoint *ep)
+{
+ struct xhci_hcd *xhci;
+ struct xhci_container_ctx *in_ctx, *out_ctx;
+ struct xhci_input_control_ctx *ctrl_ctx;
+ struct xhci_slot_ctx *slot_ctx;
+ unsigned int last_ctx;
+ unsigned int ep_index;
+ struct xhci_ep_ctx *ep_ctx;
+ u32 drop_flag;
+ u32 new_add_flags, new_drop_flags, new_slot_info;
+ int ret;
+
+ ret = xhci_check_args(hcd, udev, ep, 1, __func__);
+ if (ret <= 0)
+ return ret;
+ xhci = hcd_to_xhci(hcd);
+ xhci_dbg(xhci, "%s called for udev %p\n", __func__, udev);
+
+ drop_flag = xhci_get_endpoint_flag(&ep->desc);
+ if (drop_flag == SLOT_FLAG || drop_flag == EP0_FLAG) {
+ xhci_dbg(xhci, "xHCI %s - can't drop slot or ep 0 %#x\n",
+ __func__, drop_flag);
+ return 0;
+ }
+
+ if (!xhci->devs || !xhci->devs[udev->slot_id]) {
+ xhci_warn(xhci, "xHCI %s called with unaddressed device\n",
+ __func__);
+ return -EINVAL;
+ }
+
+ in_ctx = xhci->devs[udev->slot_id]->in_ctx;
+ out_ctx = xhci->devs[udev->slot_id]->out_ctx;
+ ctrl_ctx = xhci_get_input_control_ctx(xhci, in_ctx);
+ ep_index = xhci_get_endpoint_index(&ep->desc);
+ ep_ctx = xhci_get_ep_ctx(xhci, out_ctx, ep_index);
+ /* If the HC already knows the endpoint is disabled,
+ * or the HCD has noted it is disabled, ignore this request
+ */
+ if ((ep_ctx->ep_info & EP_STATE_MASK) == EP_STATE_DISABLED ||
+ ctrl_ctx->drop_flags & xhci_get_endpoint_flag(&ep->desc)) {
+ xhci_warn(xhci, "xHCI %s called with disabled ep %p\n",
+ __func__, ep);
+ return 0;
+ }
+
+ ctrl_ctx->drop_flags |= drop_flag;
+ new_drop_flags = ctrl_ctx->drop_flags;
+
+ ctrl_ctx->add_flags &= ~drop_flag;
+ new_add_flags = ctrl_ctx->add_flags;
+
+ last_ctx = xhci_last_valid_endpoint(ctrl_ctx->add_flags);
+ slot_ctx = xhci_get_slot_ctx(xhci, in_ctx);
+ /* Update the last valid endpoint context, if we deleted the last one */
+ if ((slot_ctx->dev_info & LAST_CTX_MASK) > LAST_CTX(last_ctx)) {
+ slot_ctx->dev_info &= ~LAST_CTX_MASK;
+ slot_ctx->dev_info |= LAST_CTX(last_ctx);
+ }
+ new_slot_info = slot_ctx->dev_info;
+
+ xhci_endpoint_zero(xhci, xhci->devs[udev->slot_id], ep);
+
+ xhci_dbg(xhci, "drop ep 0x%x, slot id %d, new drop flags = %#x, new add flags = %#x, new slot info = %#x\n",
+ (unsigned int) ep->desc.bEndpointAddress,
+ udev->slot_id,
+ (unsigned int) new_drop_flags,
+ (unsigned int) new_add_flags,
+ (unsigned int) new_slot_info);
+ return 0;
+}
+
+/* Add an endpoint to a new possible bandwidth configuration for this device.
+ * Only one call to this function is allowed per endpoint before
+ * check_bandwidth() or reset_bandwidth() must be called.
+ * A call to xhci_drop_endpoint() followed by a call to xhci_add_endpoint() will
+ * add the endpoint to the schedule with possibly new parameters denoted by a
+ * different endpoint descriptor in usb_host_endpoint.
+ * A call to xhci_add_endpoint() followed by a call to xhci_drop_endpoint() is
+ * not allowed.
+ *
+ * The USB core will not allow URBs to be queued to an endpoint until the
+ * configuration or alt setting is installed in the device, so there's no need
+ * for mutual exclusion to protect the xhci->devs[slot_id] structure.
+ */
+int xhci_add_endpoint(struct usb_hcd *hcd, struct usb_device *udev,
+ struct usb_host_endpoint *ep)
+{
+ struct xhci_hcd *xhci;
+ struct xhci_container_ctx *in_ctx, *out_ctx;
+ unsigned int ep_index;
+ struct xhci_ep_ctx *ep_ctx;
+ struct xhci_slot_ctx *slot_ctx;
+ struct xhci_input_control_ctx *ctrl_ctx;
+ u32 added_ctxs;
+ unsigned int last_ctx;
+ u32 new_add_flags, new_drop_flags, new_slot_info;
+ int ret = 0;
+
+ ret = xhci_check_args(hcd, udev, ep, 1, __func__);
+ if (ret <= 0) {
+ /* So we won't queue a reset ep command for a root hub */
+ ep->hcpriv = NULL;
+ return ret;
+ }
+ xhci = hcd_to_xhci(hcd);
+
+ added_ctxs = xhci_get_endpoint_flag(&ep->desc);
+ last_ctx = xhci_last_valid_endpoint(added_ctxs);
+ if (added_ctxs == SLOT_FLAG || added_ctxs == EP0_FLAG) {
+ /* FIXME when we have to issue an evaluate endpoint command to
+ * deal with ep0 max packet size changing once we get the
+ * descriptors
+ */
+ xhci_dbg(xhci, "xHCI %s - can't add slot or ep 0 %#x\n",
+ __func__, added_ctxs);
+ return 0;
+ }
+
+ if (!xhci->devs || !xhci->devs[udev->slot_id]) {
+ xhci_warn(xhci, "xHCI %s called with unaddressed device\n",
+ __func__);
+ return -EINVAL;
+ }
+
+ in_ctx = xhci->devs[udev->slot_id]->in_ctx;
+ out_ctx = xhci->devs[udev->slot_id]->out_ctx;
+ ctrl_ctx = xhci_get_input_control_ctx(xhci, in_ctx);
+ ep_index = xhci_get_endpoint_index(&ep->desc);
+ ep_ctx = xhci_get_ep_ctx(xhci, out_ctx, ep_index);
+ /* If the HCD has already noted the endpoint is enabled,
+ * ignore this request.
+ */
+ if (ctrl_ctx->add_flags & xhci_get_endpoint_flag(&ep->desc)) {
+ xhci_warn(xhci, "xHCI %s called with enabled ep %p\n",
+ __func__, ep);
+ return 0;
+ }
+
+ /*
+ * Configuration and alternate setting changes must be done in
+ * process context, not interrupt context (or so documenation
+ * for usb_set_interface() and usb_set_configuration() claim).
+ */
+ if (xhci_endpoint_init(xhci, xhci->devs[udev->slot_id],
+ udev, ep, GFP_NOIO) < 0) {
+ dev_dbg(&udev->dev, "%s - could not initialize ep %#x\n",
+ __func__, ep->desc.bEndpointAddress);
+ return -ENOMEM;
+ }
+
+ ctrl_ctx->add_flags |= added_ctxs;
+ new_add_flags = ctrl_ctx->add_flags;
+
+ /* If xhci_endpoint_disable() was called for this endpoint, but the
+ * xHC hasn't been notified yet through the check_bandwidth() call,
+ * this re-adds a new state for the endpoint from the new endpoint
+ * descriptors. We must drop and re-add this endpoint, so we leave the
+ * drop flags alone.
+ */
+ new_drop_flags = ctrl_ctx->drop_flags;
+
+ slot_ctx = xhci_get_slot_ctx(xhci, in_ctx);
+ /* Update the last valid endpoint context, if we just added one past */
+ if ((slot_ctx->dev_info & LAST_CTX_MASK) < LAST_CTX(last_ctx)) {
+ slot_ctx->dev_info &= ~LAST_CTX_MASK;
+ slot_ctx->dev_info |= LAST_CTX(last_ctx);
+ }
+ new_slot_info = slot_ctx->dev_info;
+
+ /* Store the usb_device pointer for later use */
+ ep->hcpriv = udev;
+
+ xhci_dbg(xhci, "add ep 0x%x, slot id %d, new drop flags = %#x, new add flags = %#x, new slot info = %#x\n",
+ (unsigned int) ep->desc.bEndpointAddress,
+ udev->slot_id,
+ (unsigned int) new_drop_flags,
+ (unsigned int) new_add_flags,
+ (unsigned int) new_slot_info);
+ return 0;
+}
+
+static void xhci_zero_in_ctx(struct xhci_hcd *xhci, struct xhci_virt_device *virt_dev)
+{
+ struct xhci_input_control_ctx *ctrl_ctx;
+ struct xhci_ep_ctx *ep_ctx;
+ struct xhci_slot_ctx *slot_ctx;
+ int i;
+
+ /* When a device's add flag and drop flag are zero, any subsequent
+ * configure endpoint command will leave that endpoint's state
+ * untouched. Make sure we don't leave any old state in the input
+ * endpoint contexts.
+ */
+ ctrl_ctx = xhci_get_input_control_ctx(xhci, virt_dev->in_ctx);
+ ctrl_ctx->drop_flags = 0;
+ ctrl_ctx->add_flags = 0;
+ slot_ctx = xhci_get_slot_ctx(xhci, virt_dev->in_ctx);
+ slot_ctx->dev_info &= ~LAST_CTX_MASK;
+ /* Endpoint 0 is always valid */
+ slot_ctx->dev_info |= LAST_CTX(1);
+ for (i = 1; i < 31; ++i) {
+ ep_ctx = xhci_get_ep_ctx(xhci, virt_dev->in_ctx, i);
+ ep_ctx->ep_info = 0;
+ ep_ctx->ep_info2 = 0;
+ ep_ctx->deq = 0;
+ ep_ctx->tx_info = 0;
+ }
+}
+
+static int xhci_configure_endpoint_result(struct xhci_hcd *xhci,
+ struct usb_device *udev, int *cmd_status)
+{
+ int ret;
+
+ switch (*cmd_status) {
+ case COMP_ENOMEM:
+ dev_warn(&udev->dev, "Not enough host controller resources "
+ "for new device state.\n");
+ ret = -ENOMEM;
+ /* FIXME: can we allocate more resources for the HC? */
+ break;
+ case COMP_BW_ERR:
+ dev_warn(&udev->dev, "Not enough bandwidth "
+ "for new device state.\n");
+ ret = -ENOSPC;
+ /* FIXME: can we go back to the old state? */
+ break;
+ case COMP_TRB_ERR:
+ /* the HCD set up something wrong */
+ dev_warn(&udev->dev, "ERROR: Endpoint drop flag = 0, "
+ "add flag = 1, "
+ "and endpoint is not disabled.\n");
+ ret = -EINVAL;
+ break;
+ case COMP_SUCCESS:
+ dev_dbg(&udev->dev, "Successful Endpoint Configure command\n");
+ ret = 0;
+ break;
+ default:
+ xhci_err(xhci, "ERROR: unexpected command completion "
+ "code 0x%x.\n", *cmd_status);
+ ret = -EINVAL;
+ break;
+ }
+ return ret;
+}
+
+static int xhci_evaluate_context_result(struct xhci_hcd *xhci,
+ struct usb_device *udev, int *cmd_status)
+{
+ int ret;
+ struct xhci_virt_device *virt_dev = xhci->devs[udev->slot_id];
+
+ switch (*cmd_status) {
+ case COMP_EINVAL:
+ dev_warn(&udev->dev, "WARN: xHCI driver setup invalid evaluate "
+ "context command.\n");
+ ret = -EINVAL;
+ break;
+ case COMP_EBADSLT:
+ dev_warn(&udev->dev, "WARN: slot not enabled for"
+ "evaluate context command.\n");
+ case COMP_CTX_STATE:
+ dev_warn(&udev->dev, "WARN: invalid context state for "
+ "evaluate context command.\n");
+ xhci_dbg_ctx(xhci, virt_dev->out_ctx, 1);
+ ret = -EINVAL;
+ break;
+ case COMP_SUCCESS:
+ dev_dbg(&udev->dev, "Successful evaluate context command\n");
+ ret = 0;
+ break;
+ default:
+ xhci_err(xhci, "ERROR: unexpected command completion "
+ "code 0x%x.\n", *cmd_status);
+ ret = -EINVAL;
+ break;
+ }
+ return ret;
+}
+
+/* Issue a configure endpoint command or evaluate context command
+ * and wait for it to finish.
+ */
+static int xhci_configure_endpoint(struct xhci_hcd *xhci,
+ struct usb_device *udev,
+ struct xhci_command *command,
+ bool ctx_change, bool must_succeed)
+{
+ int ret;
+ int timeleft;
+ unsigned long flags;
+ struct xhci_container_ctx *in_ctx;
+ struct completion *cmd_completion;
+ int *cmd_status;
+ struct xhci_virt_device *virt_dev;
+
+ spin_lock_irqsave(&xhci->lock, flags);
+ virt_dev = xhci->devs[udev->slot_id];
+ if (command) {
+ in_ctx = command->in_ctx;
+ cmd_completion = command->completion;
+ cmd_status = &command->status;
+ command->command_trb = xhci->cmd_ring->enqueue;
+ list_add_tail(&command->cmd_list, &virt_dev->cmd_list);
+ } else {
+ in_ctx = virt_dev->in_ctx;
+ cmd_completion = &virt_dev->cmd_completion;
+ cmd_status = &virt_dev->cmd_status;
+ }
+ init_completion(cmd_completion);
+
+ if (!ctx_change)
+ ret = xhci_queue_configure_endpoint(xhci, in_ctx->dma,
+ udev->slot_id, must_succeed);
+ else
+ ret = xhci_queue_evaluate_context(xhci, in_ctx->dma,
+ udev->slot_id);
+ if (ret < 0) {
+ if (command)
+ list_del(&command->cmd_list);
+ spin_unlock_irqrestore(&xhci->lock, flags);
+ xhci_dbg(xhci, "FIXME allocate a new ring segment\n");
+ return -ENOMEM;
+ }
+ xhci_ring_cmd_db(xhci);
+ spin_unlock_irqrestore(&xhci->lock, flags);
+
+ /* Wait for the configure endpoint command to complete */
+ timeleft = wait_for_completion_interruptible_timeout(
+ cmd_completion,
+ USB_CTRL_SET_TIMEOUT);
+ if (timeleft <= 0) {
+ xhci_warn(xhci, "%s while waiting for %s command\n",
+ timeleft == 0 ? "Timeout" : "Signal",
+ ctx_change == 0 ?
+ "configure endpoint" :
+ "evaluate context");
+ /* FIXME cancel the configure endpoint command */
+ return -ETIME;
+ }
+
+ if (!ctx_change)
+ return xhci_configure_endpoint_result(xhci, udev, cmd_status);
+ return xhci_evaluate_context_result(xhci, udev, cmd_status);
+}
+
+/* Called after one or more calls to xhci_add_endpoint() or
+ * xhci_drop_endpoint(). If this call fails, the USB core is expected
+ * to call xhci_reset_bandwidth().
+ *
+ * Since we are in the middle of changing either configuration or
+ * installing a new alt setting, the USB core won't allow URBs to be
+ * enqueued for any endpoint on the old config or interface. Nothing
+ * else should be touching the xhci->devs[slot_id] structure, so we
+ * don't need to take the xhci->lock for manipulating that.
+ */
+int xhci_check_bandwidth(struct usb_hcd *hcd, struct usb_device *udev)
+{
+ int i;
+ int ret = 0;
+ struct xhci_hcd *xhci;
+ struct xhci_virt_device *virt_dev;
+ struct xhci_input_control_ctx *ctrl_ctx;
+ struct xhci_slot_ctx *slot_ctx;
+
+ ret = xhci_check_args(hcd, udev, NULL, 0, __func__);
+ if (ret <= 0)
+ return ret;
+ xhci = hcd_to_xhci(hcd);
+
+ if (!udev->slot_id || !xhci->devs || !xhci->devs[udev->slot_id]) {
+ xhci_warn(xhci, "xHCI %s called with unaddressed device\n",
+ __func__);
+ return -EINVAL;
+ }
+ xhci_dbg(xhci, "%s called for udev %p\n", __func__, udev);
+ virt_dev = xhci->devs[udev->slot_id];
+
+ /* See section 4.6.6 - A0 = 1; A1 = D0 = D1 = 0 */
+ ctrl_ctx = xhci_get_input_control_ctx(xhci, virt_dev->in_ctx);
+ ctrl_ctx->add_flags |= SLOT_FLAG;
+ ctrl_ctx->add_flags &= ~EP0_FLAG;
+ ctrl_ctx->drop_flags &= ~SLOT_FLAG;
+ ctrl_ctx->drop_flags &= ~EP0_FLAG;
+ xhci_dbg(xhci, "New Input Control Context:\n");
+ slot_ctx = xhci_get_slot_ctx(xhci, virt_dev->in_ctx);
+ xhci_dbg_ctx(xhci, virt_dev->in_ctx,
+ LAST_CTX_TO_EP_NUM(slot_ctx->dev_info));
+
+ ret = xhci_configure_endpoint(xhci, udev, NULL,
+ false, false);
+ if (ret) {
+ /* Callee should call reset_bandwidth() */
+ return ret;
+ }
+
+ xhci_dbg(xhci, "Output context after successful config ep cmd:\n");
+ xhci_dbg_ctx(xhci, virt_dev->out_ctx,
+ LAST_CTX_TO_EP_NUM(slot_ctx->dev_info));
+
+ xhci_zero_in_ctx(xhci, virt_dev);
+ /* Install new rings and free or cache any old rings */
+ for (i = 1; i < 31; ++i) {
+ if (!virt_dev->eps[i].new_ring)
+ continue;
+ /* Only cache or free the old ring if it exists.
+ * It may not if this is the first add of an endpoint.
+ */
+ if (virt_dev->eps[i].ring) {
+ xhci_free_or_cache_endpoint_ring(xhci, virt_dev, i);
+ }
+ virt_dev->eps[i].ring = virt_dev->eps[i].new_ring;
+ virt_dev->eps[i].new_ring = NULL;
+ }
+
+ return ret;
+}
+
+void xhci_reset_bandwidth(struct usb_hcd *hcd, struct usb_device *udev)
+{
+ struct xhci_hcd *xhci;
+ struct xhci_virt_device *virt_dev;
+ int i, ret;
+
+ ret = xhci_check_args(hcd, udev, NULL, 0, __func__);
+ if (ret <= 0)
+ return;
+ xhci = hcd_to_xhci(hcd);
+
+ if (!xhci->devs || !xhci->devs[udev->slot_id]) {
+ xhci_warn(xhci, "xHCI %s called with unaddressed device\n",
+ __func__);
+ return;
+ }
+ xhci_dbg(xhci, "%s called for udev %p\n", __func__, udev);
+ virt_dev = xhci->devs[udev->slot_id];
+ /* Free any rings allocated for added endpoints */
+ for (i = 0; i < 31; ++i) {
+ if (virt_dev->eps[i].new_ring) {
+ xhci_ring_free(xhci, virt_dev->eps[i].new_ring);
+ virt_dev->eps[i].new_ring = NULL;
+ }
+ }
+ xhci_zero_in_ctx(xhci, virt_dev);
+}
+
+static void xhci_setup_input_ctx_for_config_ep(struct xhci_hcd *xhci,
+ struct xhci_container_ctx *in_ctx,
+ struct xhci_container_ctx *out_ctx,
+ u32 add_flags, u32 drop_flags)
+{
+ struct xhci_input_control_ctx *ctrl_ctx;
+ ctrl_ctx = xhci_get_input_control_ctx(xhci, in_ctx);
+ ctrl_ctx->add_flags = add_flags;
+ ctrl_ctx->drop_flags = drop_flags;
+ xhci_slot_copy(xhci, in_ctx, out_ctx);
+ ctrl_ctx->add_flags |= SLOT_FLAG;
+
+ xhci_dbg(xhci, "Input Context:\n");
+ xhci_dbg_ctx(xhci, in_ctx, xhci_last_valid_endpoint(add_flags));
+}
+
+void xhci_setup_input_ctx_for_quirk(struct xhci_hcd *xhci,
+ unsigned int slot_id, unsigned int ep_index,
+ struct xhci_dequeue_state *deq_state)
+{
+ struct xhci_container_ctx *in_ctx;
+ struct xhci_ep_ctx *ep_ctx;
+ u32 added_ctxs;
+ dma_addr_t addr;
+
+ xhci_endpoint_copy(xhci, xhci->devs[slot_id]->in_ctx,
+ xhci->devs[slot_id]->out_ctx, ep_index);
+ in_ctx = xhci->devs[slot_id]->in_ctx;
+ ep_ctx = xhci_get_ep_ctx(xhci, in_ctx, ep_index);
+ addr = xhci_trb_virt_to_dma(deq_state->new_deq_seg,
+ deq_state->new_deq_ptr);
+ if (addr == 0) {
+ xhci_warn(xhci, "WARN Cannot submit config ep after "
+ "reset ep command\n");
+ xhci_warn(xhci, "WARN deq seg = %p, deq ptr = %p\n",
+ deq_state->new_deq_seg,
+ deq_state->new_deq_ptr);
+ return;
+ }
+ ep_ctx->deq = addr | deq_state->new_cycle_state;
+
+ added_ctxs = xhci_get_endpoint_flag_from_index(ep_index);
+ xhci_setup_input_ctx_for_config_ep(xhci, xhci->devs[slot_id]->in_ctx,
+ xhci->devs[slot_id]->out_ctx, added_ctxs, added_ctxs);
+}
+
+void xhci_cleanup_stalled_ring(struct xhci_hcd *xhci,
+ struct usb_device *udev, unsigned int ep_index)
+{
+ struct xhci_dequeue_state deq_state;
+ struct xhci_virt_ep *ep;
+
+ xhci_dbg(xhci, "Cleaning up stalled endpoint ring\n");
+ ep = &xhci->devs[udev->slot_id]->eps[ep_index];
+ /* We need to move the HW's dequeue pointer past this TD,
+ * or it will attempt to resend it on the next doorbell ring.
+ */
+ xhci_find_new_dequeue_state(xhci, udev->slot_id,
+ ep_index, ep->stopped_td,
+ &deq_state);
+
+ /* HW with the reset endpoint quirk will use the saved dequeue state to
+ * issue a configure endpoint command later.
+ */
+ if (!(xhci->quirks & XHCI_RESET_EP_QUIRK)) {
+ xhci_dbg(xhci, "Queueing new dequeue state\n");
+ xhci_queue_new_dequeue_state(xhci, udev->slot_id,
+ ep_index, &deq_state);
+ } else {
+ /* Better hope no one uses the input context between now and the
+ * reset endpoint completion!
+ */
+ xhci_dbg(xhci, "Setting up input context for "
+ "configure endpoint command\n");
+ xhci_setup_input_ctx_for_quirk(xhci, udev->slot_id,
+ ep_index, &deq_state);
+ }
+}
+
+/* Deal with stalled endpoints. The core should have sent the control message
+ * to clear the halt condition. However, we need to make the xHCI hardware
+ * reset its sequence number, since a device will expect a sequence number of
+ * zero after the halt condition is cleared.
+ * Context: in_interrupt
+ */
+void xhci_endpoint_reset(struct usb_hcd *hcd,
+ struct usb_host_endpoint *ep)
+{
+ struct xhci_hcd *xhci;
+ struct usb_device *udev;
+ unsigned int ep_index;
+ unsigned long flags;
+ int ret;
+ struct xhci_virt_ep *virt_ep;
+
+ xhci = hcd_to_xhci(hcd);
+ udev = (struct usb_device *) ep->hcpriv;
+ /* Called with a root hub endpoint (or an endpoint that wasn't added
+ * with xhci_add_endpoint()
+ */
+ if (!ep->hcpriv)
+ return;
+ ep_index = xhci_get_endpoint_index(&ep->desc);
+ virt_ep = &xhci->devs[udev->slot_id]->eps[ep_index];
+ if (!virt_ep->stopped_td) {
+ xhci_dbg(xhci, "Endpoint 0x%x not halted, refusing to reset.\n",
+ ep->desc.bEndpointAddress);
+ return;
+ }
+ if (usb_endpoint_xfer_control(&ep->desc)) {
+ xhci_dbg(xhci, "Control endpoint stall already handled.\n");
+ return;
+ }
+
+ xhci_dbg(xhci, "Queueing reset endpoint command\n");
+ spin_lock_irqsave(&xhci->lock, flags);
+ ret = xhci_queue_reset_ep(xhci, udev->slot_id, ep_index);
+ /*
+ * Can't change the ring dequeue pointer until it's transitioned to the
+ * stopped state, which is only upon a successful reset endpoint
+ * command. Better hope that last command worked!
+ */
+ if (!ret) {
+ xhci_cleanup_stalled_ring(xhci, udev, ep_index);
+ kfree(virt_ep->stopped_td);
+ xhci_ring_cmd_db(xhci);
+ }
+ spin_unlock_irqrestore(&xhci->lock, flags);
+
+ if (ret)
+ xhci_warn(xhci, "FIXME allocate a new ring segment\n");
+}
+
+/*
+ * This submits a Reset Device Command, which will set the device state to 0,
+ * set the device address to 0, and disable all the endpoints except the default
+ * control endpoint. The USB core should come back and call
+ * xhci_address_device(), and then re-set up the configuration. If this is
+ * called because of a usb_reset_and_verify_device(), then the old alternate
+ * settings will be re-installed through the normal bandwidth allocation
+ * functions.
+ *
+ * Wait for the Reset Device command to finish. Remove all structures
+ * associated with the endpoints that were disabled. Clear the input device
+ * structure? Cache the rings? Reset the control endpoint 0 max packet size?
+ */
+int xhci_reset_device(struct usb_hcd *hcd, struct usb_device *udev)
+{
+ int ret, i;
+ unsigned long flags;
+ struct xhci_hcd *xhci;
+ unsigned int slot_id;
+ struct xhci_virt_device *virt_dev;
+ struct xhci_command *reset_device_cmd;
+ int timeleft;
+ int last_freed_endpoint;
+
+ ret = xhci_check_args(hcd, udev, NULL, 0, __func__);
+ if (ret <= 0)
+ return ret;
+ xhci = hcd_to_xhci(hcd);
+ slot_id = udev->slot_id;
+ virt_dev = xhci->devs[slot_id];
+ if (!virt_dev) {
+ xhci_dbg(xhci, "%s called with invalid slot ID %u\n",
+ __func__, slot_id);
+ return -EINVAL;
+ }
+
+ xhci_dbg(xhci, "Resetting device with slot ID %u\n", slot_id);
+ /* Allocate the command structure that holds the struct completion.
+ * Assume we're in process context, since the normal device reset
+ * process has to wait for the device anyway. Storage devices are
+ * reset as part of error handling, so use GFP_NOIO instead of
+ * GFP_KERNEL.
+ */
+ reset_device_cmd = xhci_alloc_command(xhci, false, true, GFP_NOIO);
+ if (!reset_device_cmd) {
+ xhci_dbg(xhci, "Couldn't allocate command structure.\n");
+ return -ENOMEM;
+ }
+
+ /* Attempt to submit the Reset Device command to the command ring */
+ spin_lock_irqsave(&xhci->lock, flags);
+ reset_device_cmd->command_trb = xhci->cmd_ring->enqueue;
+ list_add_tail(&reset_device_cmd->cmd_list, &virt_dev->cmd_list);
+ ret = xhci_queue_reset_device(xhci, slot_id);
+ if (ret) {
+ xhci_dbg(xhci, "FIXME: allocate a command ring segment\n");
+ list_del(&reset_device_cmd->cmd_list);
+ spin_unlock_irqrestore(&xhci->lock, flags);
+ goto command_cleanup;
+ }
+ xhci_ring_cmd_db(xhci);
+ spin_unlock_irqrestore(&xhci->lock, flags);
+
+ /* Wait for the Reset Device command to finish */
+ timeleft = wait_for_completion_interruptible_timeout(
+ reset_device_cmd->completion,
+ USB_CTRL_SET_TIMEOUT);
+ if (timeleft <= 0) {
+ xhci_warn(xhci, "%s while waiting for reset device command\n",
+ timeleft == 0 ? "Timeout" : "Signal");
+ spin_lock_irqsave(&xhci->lock, flags);
+ /* The timeout might have raced with the event ring handler, so
+ * only delete from the list if the item isn't poisoned.
+ */
+ if (reset_device_cmd->cmd_list.next != LIST_POISON1)
+ list_del(&reset_device_cmd->cmd_list);
+ spin_unlock_irqrestore(&xhci->lock, flags);
+ ret = -ETIME;
+ goto command_cleanup;
+ }
+
+ /* The Reset Device command can't fail, according to the 0.95/0.96 spec,
+ * unless we tried to reset a slot ID that wasn't enabled,
+ * or the device wasn't in the addressed or configured state.
+ */
+ ret = reset_device_cmd->status;
+ switch (ret) {
+ case COMP_EBADSLT: /* 0.95 completion code for bad slot ID */
+ case COMP_CTX_STATE: /* 0.96 completion code for same thing */
+ xhci_info(xhci, "Can't reset device (slot ID %u) in %s state\n",
+ slot_id,
+ xhci_get_slot_state(xhci, virt_dev->out_ctx));
+ xhci_info(xhci, "Not freeing device rings.\n");
+ /* Don't treat this as an error. May change my mind later. */
+ ret = 0;
+ goto command_cleanup;
+ case COMP_SUCCESS:
+ xhci_dbg(xhci, "Successful reset device command.\n");
+ break;
+ default:
+ if (xhci_is_vendor_info_code(xhci, ret))
+ break;
+ xhci_warn(xhci, "Unknown completion code %u for "
+ "reset device command.\n", ret);
+ ret = -EINVAL;
+ goto command_cleanup;
+ }
+
+ /* Everything but endpoint 0 is disabled, so free or cache the rings. */
+ last_freed_endpoint = 1;
+ for (i = 1; i < 31; ++i) {
+ if (!virt_dev->eps[i].ring)
+ continue;
+ xhci_free_or_cache_endpoint_ring(xhci, virt_dev, i);
+ last_freed_endpoint = i;
+ }
+ xhci_dbg(xhci, "Output context after successful reset device cmd:\n");
+ xhci_dbg_ctx(xhci, virt_dev->out_ctx, last_freed_endpoint);
+ ret = 0;
+
+command_cleanup:
+ xhci_free_command(xhci, reset_device_cmd);
+ return ret;
+}
+
+/*
+ * At this point, the struct usb_device is about to go away, the device has
+ * disconnected, and all traffic has been stopped and the endpoints have been
+ * disabled. Free any HC data structures associated with that device.
+ */
+void xhci_free_dev(struct usb_hcd *hcd, struct usb_device *udev)
+{
+ struct xhci_hcd *xhci = hcd_to_xhci(hcd);
+ struct xhci_virt_device *virt_dev;
+ unsigned long flags;
+ u32 state;
+ int i;
+
+ if (udev->slot_id == 0)
+ return;
+ virt_dev = xhci->devs[udev->slot_id];
+ if (!virt_dev)
+ return;
+
+ /* Stop any wayward timer functions (which may grab the lock) */
+ for (i = 0; i < 31; ++i) {
+ virt_dev->eps[i].ep_state &= ~EP_HALT_PENDING;
+ del_timer_sync(&virt_dev->eps[i].stop_cmd_timer);
+ }
+
+ spin_lock_irqsave(&xhci->lock, flags);
+ /* Don't disable the slot if the host controller is dead. */
+ state = xhci_readl(xhci, &xhci->op_regs->status);
+ if (state == 0xffffffff || (xhci->xhc_state & XHCI_STATE_DYING)) {
+ xhci_free_virt_device(xhci, udev->slot_id);
+ spin_unlock_irqrestore(&xhci->lock, flags);
+ return;
+ }
+
+ if (xhci_queue_slot_control(xhci, TRB_DISABLE_SLOT, udev->slot_id)) {
+ spin_unlock_irqrestore(&xhci->lock, flags);
+ xhci_dbg(xhci, "FIXME: allocate a command ring segment\n");
+ return;
+ }
+ xhci_ring_cmd_db(xhci);
+ spin_unlock_irqrestore(&xhci->lock, flags);
+ /*
+ * Event command completion handler will free any data structures
+ * associated with the slot. XXX Can free sleep?
+ */
+}
+
+/*
+ * Returns 0 if the xHC ran out of device slots, the Enable Slot command
+ * timed out, or allocating memory failed. Returns 1 on success.
+ */
+int xhci_alloc_dev(struct usb_hcd *hcd, struct usb_device *udev)
+{
+ struct xhci_hcd *xhci = hcd_to_xhci(hcd);
+ unsigned long flags;
+ int timeleft;
+ int ret;
+
+ spin_lock_irqsave(&xhci->lock, flags);
+ ret = xhci_queue_slot_control(xhci, TRB_ENABLE_SLOT, 0);
+ if (ret) {
+ spin_unlock_irqrestore(&xhci->lock, flags);
+ xhci_dbg(xhci, "FIXME: allocate a command ring segment\n");
+ return 0;
+ }
+ xhci_ring_cmd_db(xhci);
+ spin_unlock_irqrestore(&xhci->lock, flags);
+
+ /* XXX: how much time for xHC slot assignment? */
+ timeleft = wait_for_completion_interruptible_timeout(&xhci->addr_dev,
+ USB_CTRL_SET_TIMEOUT);
+ if (timeleft <= 0) {
+ xhci_warn(xhci, "%s while waiting for a slot\n",
+ timeleft == 0 ? "Timeout" : "Signal");
+ /* FIXME cancel the enable slot request */
+ return 0;
+ }
+
+ if (!xhci->slot_id) {
+ xhci_err(xhci, "Error while assigning device slot ID\n");
+ return 0;
+ }
+ /* xhci_alloc_virt_device() does not touch rings; no need to lock */
+ if (!xhci_alloc_virt_device(xhci, xhci->slot_id, udev, GFP_KERNEL)) {
+ /* Disable slot, if we can do it without mem alloc */
+ xhci_warn(xhci, "Could not allocate xHCI USB device data structures\n");
+ spin_lock_irqsave(&xhci->lock, flags);
+ if (!xhci_queue_slot_control(xhci, TRB_DISABLE_SLOT, udev->slot_id))
+ xhci_ring_cmd_db(xhci);
+ spin_unlock_irqrestore(&xhci->lock, flags);
+ return 0;
+ }
+ udev->slot_id = xhci->slot_id;
+ /* Is this a LS or FS device under a HS hub? */
+ /* Hub or peripherial? */
+ return 1;
+}
+
+/*
+ * Issue an Address Device command (which will issue a SetAddress request to
+ * the device).
+ * We should be protected by the usb_address0_mutex in khubd's hub_port_init, so
+ * we should only issue and wait on one address command at the same time.
+ *
+ * We add one to the device address issued by the hardware because the USB core
+ * uses address 1 for the root hubs (even though they're not really devices).
+ */
+int xhci_address_device(struct usb_hcd *hcd, struct usb_device *udev)
+{
+ unsigned long flags;
+ int timeleft;
+ struct xhci_virt_device *virt_dev;
+ int ret = 0;
+ struct xhci_hcd *xhci = hcd_to_xhci(hcd);
+ struct xhci_slot_ctx *slot_ctx;
+ struct xhci_input_control_ctx *ctrl_ctx;
+ u64 temp_64;
+
+ if (!udev->slot_id) {
+ xhci_dbg(xhci, "Bad Slot ID %d\n", udev->slot_id);
+ return -EINVAL;
+ }
+
+ virt_dev = xhci->devs[udev->slot_id];
+
+ /* If this is a Set Address to an unconfigured device, setup ep 0 */
+ if (!udev->config)
+ xhci_setup_addressable_virt_dev(xhci, udev);
+ /* Otherwise, assume the core has the device configured how it wants */
+ xhci_dbg(xhci, "Slot ID %d Input Context:\n", udev->slot_id);
+ xhci_dbg_ctx(xhci, virt_dev->in_ctx, 2);
+
+ spin_lock_irqsave(&xhci->lock, flags);
+ ret = xhci_queue_address_device(xhci, virt_dev->in_ctx->dma,
+ udev->slot_id);
+ if (ret) {
+ spin_unlock_irqrestore(&xhci->lock, flags);
+ xhci_dbg(xhci, "FIXME: allocate a command ring segment\n");
+ return ret;
+ }
+ xhci_ring_cmd_db(xhci);
+ spin_unlock_irqrestore(&xhci->lock, flags);
+
+ /* ctrl tx can take up to 5 sec; XXX: need more time for xHC? */
+ timeleft = wait_for_completion_interruptible_timeout(&xhci->addr_dev,
+ USB_CTRL_SET_TIMEOUT);
+ /* FIXME: From section 4.3.4: "Software shall be responsible for timing
+ * the SetAddress() "recovery interval" required by USB and aborting the
+ * command on a timeout.
+ */
+ if (timeleft <= 0) {
+ xhci_warn(xhci, "%s while waiting for a slot\n",
+ timeleft == 0 ? "Timeout" : "Signal");
+ /* FIXME cancel the address device command */
+ return -ETIME;
+ }
+
+ switch (virt_dev->cmd_status) {
+ case COMP_CTX_STATE:
+ case COMP_EBADSLT:
+ xhci_err(xhci, "Setup ERROR: address device command for slot %d.\n",
+ udev->slot_id);
+ ret = -EINVAL;
+ break;
+ case COMP_TX_ERR:
+ dev_warn(&udev->dev, "Device not responding to set address.\n");
+ ret = -EPROTO;
+ break;
+ case COMP_SUCCESS:
+ xhci_dbg(xhci, "Successful Address Device command\n");
+ break;
+ default:
+ xhci_err(xhci, "ERROR: unexpected command completion "
+ "code 0x%x.\n", virt_dev->cmd_status);
+ xhci_dbg(xhci, "Slot ID %d Output Context:\n", udev->slot_id);
+ xhci_dbg_ctx(xhci, virt_dev->out_ctx, 2);
+ ret = -EINVAL;
+ break;
+ }
+ if (ret) {
+ return ret;
+ }
+ temp_64 = xhci_read_64(xhci, &xhci->op_regs->dcbaa_ptr);
+ xhci_dbg(xhci, "Op regs DCBAA ptr = %#016llx\n", temp_64);
+ xhci_dbg(xhci, "Slot ID %d dcbaa entry @%p = %#016llx\n",
+ udev->slot_id,
+ &xhci->dcbaa->dev_context_ptrs[udev->slot_id],
+ (unsigned long long)
+ xhci->dcbaa->dev_context_ptrs[udev->slot_id]);
+ xhci_dbg(xhci, "Output Context DMA address = %#08llx\n",
+ (unsigned long long)virt_dev->out_ctx->dma);
+ xhci_dbg(xhci, "Slot ID %d Input Context:\n", udev->slot_id);
+ xhci_dbg_ctx(xhci, virt_dev->in_ctx, 2);
+ xhci_dbg(xhci, "Slot ID %d Output Context:\n", udev->slot_id);
+ xhci_dbg_ctx(xhci, virt_dev->out_ctx, 2);
+ /*
+ * USB core uses address 1 for the roothubs, so we add one to the
+ * address given back to us by the HC.
+ */
+ slot_ctx = xhci_get_slot_ctx(xhci, virt_dev->out_ctx);
+ udev->devnum = (slot_ctx->dev_state & DEV_ADDR_MASK) + 1;
+ /* Zero the input context control for later use */
+ ctrl_ctx = xhci_get_input_control_ctx(xhci, virt_dev->in_ctx);
+ ctrl_ctx->add_flags = 0;
+ ctrl_ctx->drop_flags = 0;
+
+ xhci_dbg(xhci, "Device address = %d\n", udev->devnum);
+ /* XXX Meh, not sure if anyone else but choose_address uses this. */
+ set_bit(udev->devnum, udev->bus->devmap.devicemap);
+
+ return 0;
+}
+
+/* Once a hub descriptor is fetched for a device, we need to update the xHC's
+ * internal data structures for the device.
+ */
+int xhci_update_hub_device(struct usb_hcd *hcd, struct usb_device *hdev,
+ struct usb_tt *tt, gfp_t mem_flags)
+{
+ struct xhci_hcd *xhci = hcd_to_xhci(hcd);
+ struct xhci_virt_device *vdev;
+ struct xhci_command *config_cmd;
+ struct xhci_input_control_ctx *ctrl_ctx;
+ struct xhci_slot_ctx *slot_ctx;
+ unsigned long flags;
+ unsigned think_time;
+ int ret;
+
+ /* Ignore root hubs */
+ if (!hdev->parent)
+ return 0;
+
+ vdev = xhci->devs[hdev->slot_id];
+ if (!vdev) {
+ xhci_warn(xhci, "Cannot update hub desc for unknown device.\n");
+ return -EINVAL;
+ }
+ config_cmd = xhci_alloc_command(xhci, true, true, mem_flags);
+ if (!config_cmd) {
+ xhci_dbg(xhci, "Could not allocate xHCI command structure.\n");
+ return -ENOMEM;
+ }
+
+ spin_lock_irqsave(&xhci->lock, flags);
+ xhci_slot_copy(xhci, config_cmd->in_ctx, vdev->out_ctx);
+ ctrl_ctx = xhci_get_input_control_ctx(xhci, config_cmd->in_ctx);
+ ctrl_ctx->add_flags |= SLOT_FLAG;
+ slot_ctx = xhci_get_slot_ctx(xhci, config_cmd->in_ctx);
+ slot_ctx->dev_info |= DEV_HUB;
+ if (tt->multi)
+ slot_ctx->dev_info |= DEV_MTT;
+ if (xhci->hci_version > 0x95) {
+ xhci_dbg(xhci, "xHCI version %x needs hub "
+ "TT think time and number of ports\n",
+ (unsigned int) xhci->hci_version);
+ slot_ctx->dev_info2 |= XHCI_MAX_PORTS(hdev->maxchild);
+ /* Set TT think time - convert from ns to FS bit times.
+ * 0 = 8 FS bit times, 1 = 16 FS bit times,
+ * 2 = 24 FS bit times, 3 = 32 FS bit times.
+ */
+ think_time = tt->think_time;
+ if (think_time != 0)
+ think_time = (think_time / 666) - 1;
+ slot_ctx->tt_info |= TT_THINK_TIME(think_time);
+ } else {
+ xhci_dbg(xhci, "xHCI version %x doesn't need hub "
+ "TT think time or number of ports\n",
+ (unsigned int) xhci->hci_version);
+ }
+ slot_ctx->dev_state = 0;
+ spin_unlock_irqrestore(&xhci->lock, flags);
+
+ xhci_dbg(xhci, "Set up %s for hub device.\n",
+ (xhci->hci_version > 0x95) ?
+ "configure endpoint" : "evaluate context");
+ xhci_dbg(xhci, "Slot %u Input Context:\n", hdev->slot_id);
+ xhci_dbg_ctx(xhci, config_cmd->in_ctx, 0);
+
+ /* Issue and wait for the configure endpoint or
+ * evaluate context command.
+ */
+ if (xhci->hci_version > 0x95)
+ ret = xhci_configure_endpoint(xhci, hdev, config_cmd,
+ false, false);
+ else
+ ret = xhci_configure_endpoint(xhci, hdev, config_cmd,
+ true, false);
+
+ xhci_dbg(xhci, "Slot %u Output Context:\n", hdev->slot_id);
+ xhci_dbg_ctx(xhci, vdev->out_ctx, 0);
+
+ xhci_free_command(xhci, config_cmd);
+ return ret;
+}
+
+int xhci_get_frame(struct usb_hcd *hcd)
+{
+ struct xhci_hcd *xhci = hcd_to_xhci(hcd);
+ /* EHCI mods by the periodic size. Why? */
+ return xhci_readl(xhci, &xhci->run_regs->microframe_index) >> 3;
+}
+
+MODULE_DESCRIPTION(DRIVER_DESC);
+MODULE_AUTHOR(DRIVER_AUTHOR);
+MODULE_LICENSE("GPL");
+
+static int __init xhci_hcd_init(void)
+{
+#ifdef CONFIG_PCI
+ int retval = 0;
+
+ retval = xhci_register_pci();
+
+ if (retval < 0) {
+ printk(KERN_DEBUG "Problem registering PCI driver.");
+ return retval;
+ }
+#endif
+ /*
+ * Check the compiler generated sizes of structures that must be laid
+ * out in specific ways for hardware access.
+ */
+ BUILD_BUG_ON(sizeof(struct xhci_doorbell_array) != 256*32/8);
+ BUILD_BUG_ON(sizeof(struct xhci_slot_ctx) != 8*32/8);
+ BUILD_BUG_ON(sizeof(struct xhci_ep_ctx) != 8*32/8);
+ /* xhci_device_control has eight fields, and also
+ * embeds one xhci_slot_ctx and 31 xhci_ep_ctx
+ */
+ BUILD_BUG_ON(sizeof(struct xhci_stream_ctx) != 4*32/8);
+ BUILD_BUG_ON(sizeof(union xhci_trb) != 4*32/8);
+ BUILD_BUG_ON(sizeof(struct xhci_erst_entry) != 4*32/8);
+ BUILD_BUG_ON(sizeof(struct xhci_cap_regs) != 7*32/8);
+ BUILD_BUG_ON(sizeof(struct xhci_intr_reg) != 8*32/8);
+ /* xhci_run_regs has eight fields and embeds 128 xhci_intr_regs */
+ BUILD_BUG_ON(sizeof(struct xhci_run_regs) != (8+8*128)*32/8);
+ BUILD_BUG_ON(sizeof(struct xhci_doorbell_array) != 256*32/8);
+ return 0;
+}
+module_init(xhci_hcd_init);
+
+static void __exit xhci_hcd_cleanup(void)
+{
+#ifdef CONFIG_PCI
+ xhci_unregister_pci();
+#endif
+}
+module_exit(xhci_hcd_cleanup);
#include <linux/errno.h>
#include <linux/init.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include <linux/backlight.h>
#include <linux/timer.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#define DRIVER_AUTHOR "Oliver Bock (bock@tfh-berlin.de)"
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/usb.h>
#include <linux/firmware.h>
#include <linux/errno.h>
#include <linux/module.h>
+#include <linux/slab.h>
static const struct usb_device_id id_table[] = {
{USB_DEVICE(0x05ac, 0x8300)},
#include <linux/string.h>
#include <linux/kd.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/vt_kern.h>
#include <linux/selection.h>
#include <linux/spinlock.h>
#include <linux/errno.h>
#include <linux/poll.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/spinlock.h>
#include "sisusb.h"
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/usb.h>
#include <linux/delay.h>
#include <linux/completion.h>
#include <linux/kref.h>
+#include <linux/slab.h>
/*
* Version Information
#include <linux/mm.h>
#include <linux/smp_lock.h>
#include <linux/scatterlist.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/usb.h>
+#include <linux/slab.h>
#include <linux/notifier.h>
#include <linux/mutex.h>
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include <linux/fs.h>
#include <asm/uaccess.h>
#include <linux/kernel.h>
#include <linux/list.h>
#include <linux/usb.h>
+#include <linux/slab.h>
#include <linux/time.h>
#include <linux/mutex.h>
#include <linux/debugfs.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/init.h>
#include <linux/list.h>
#include <linux/gpio.h>
*/
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include "musb_core.h"
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/init.h>
#include <linux/list.h>
#include <linux/delay.h>
u8 devctl, u8 power)
{
irqreturn_t handled = IRQ_NONE;
- void __iomem *mbase = musb->mregs;
DBG(3, "<== Power=%02x, DevCtl=%02x, int_usb=0x%x\n", power, devctl,
int_usb);
if (devctl & MUSB_DEVCTL_HM) {
#ifdef CONFIG_USB_MUSB_HDRC_HCD
+ void __iomem *mbase = musb->mregs;
+
switch (musb->xceiv->state) {
case OTG_STATE_A_SUSPEND:
/* remote wakeup? later, GetPortStatus
#ifdef CONFIG_USB_MUSB_HDRC_HCD
/* see manual for the order of the tests */
if (int_usb & MUSB_INTR_SESSREQ) {
+ void __iomem *mbase = musb->mregs;
+
DBG(1, "SESSION_REQUEST (%s)\n", otg_state_string(musb));
/* IRQ arrives from ID pin sense or (later, if VBUS power
case OTG_STATE_A_WAIT_BCON:
case OTG_STATE_A_WAIT_VRISE:
if (musb->vbuserr_retry) {
+ void __iomem *mbase = musb->mregs;
+
musb->vbuserr_retry--;
ignore = 1;
devctl |= MUSB_DEVCTL_SESSION;
if (int_usb & MUSB_INTR_CONNECT) {
struct usb_hcd *hcd = musb_to_hcd(musb);
+ void __iomem *mbase = musb->mregs;
handled = IRQ_HANDLED;
musb->is_active = 1;
/* host side needs more setup */
if (is_host_enabled(musb)) {
struct usb_hcd *hcd = musb_to_hcd(musb);
- u8 busctl;
otg_set_host(musb->xceiv, &hcd->self);
/* program PHY to use external vBus if required */
if (plat->extvbus) {
- busctl = musb_readb(musb->mregs, MUSB_ULPI_BUSCONTROL);
+ u8 busctl = musb_read_ulpi_buscontrol(musb->mregs);
busctl |= MUSB_ULPI_USE_EXTVBUS;
- musb_writeb(musb->mregs, MUSB_ULPI_BUSCONTROL, busctl);
+ musb_write_ulpi_buscontrol(musb->mregs, busctl);
}
}
struct musb_context_registers {
-#if defined(CONFIG_ARCH_OMAP34XX) || defined(CONFIG_ARCH_OMAP2430)
+#ifdef CONFIG_PM
u32 otg_sysconfig, otg_forcestandby;
#endif
u8 power;
struct musb_csr_regs index_regs[MUSB_C_NUM_EPS];
};
-#if defined(CONFIG_ARCH_OMAP34XX) || defined(CONFIG_ARCH_OMAP2430)
+#ifdef CONFIG_PM
extern void musb_platform_save_context(struct musb *musb,
struct musb_context_registers *musb_context);
extern void musb_platform_restore_context(struct musb *musb,
#include <linux/moduleparam.h>
#include <linux/stat.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include "musb_core.h"
dma->desired_mode = 1;
if (rx_count < hw_ep->max_packet_sz_rx) {
length = rx_count;
- dma->bDesiredMode = 0;
+ dma->desired_mode = 0;
} else {
length = urb->transfer_buffer_length;
}
musb_writew(mbase, MUSB_RXFIFOADD, c_off);
}
+static inline void musb_write_ulpi_buscontrol(void __iomem *mbase, u8 val)
+{
+ musb_writeb(mbase, MUSB_ULPI_BUSCONTROL, val);
+}
+
static inline u8 musb_read_txfifosz(void __iomem *mbase)
{
return musb_readb(mbase, MUSB_TXFIFOSZ);
return musb_readw(mbase, MUSB_RXFIFOADD);
}
+static inline u8 musb_read_ulpi_buscontrol(void __iomem *mbase)
+{
+ return musb_readb(mbase, MUSB_ULPI_BUSCONTROL);
+}
+
static inline u8 musb_read_configdata(void __iomem *mbase)
{
musb_writeb(mbase, MUSB_INDEX, 0);
{
}
+static inline void musb_write_ulpi_buscontrol(void __iomem *mbase, u8 val)
+{
+}
+
static inline u8 musb_read_txfifosz(void __iomem *mbase)
{
+ return 0;
}
static inline u16 musb_read_txfifoadd(void __iomem *mbase)
{
+ return 0;
}
static inline u8 musb_read_rxfifosz(void __iomem *mbase)
{
+ return 0;
}
static inline u16 musb_read_rxfifoadd(void __iomem *mbase)
{
+ return 0;
+}
+
+static inline u8 musb_read_ulpi_buscontrol(void __iomem *mbase)
+{
+ return 0;
}
static inline u8 musb_read_configdata(void __iomem *mbase)
static inline u8 musb_read_rxfunaddr(void __iomem *mbase, u8 epnum)
{
+ return 0;
}
static inline u8 musb_read_rxhubaddr(void __iomem *mbase, u8 epnum)
{
+ return 0;
}
static inline u8 musb_read_rxhubport(void __iomem *mbase, u8 epnum)
{
+ return 0;
}
static inline u8 musb_read_txfunaddr(void __iomem *mbase, u8 epnum)
{
+ return 0;
}
static inline u8 musb_read_txhubaddr(void __iomem *mbase, u8 epnum)
{
+ return 0;
}
static inline void musb_read_txhubport(void __iomem *mbase, u8 epnum)
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/init.h>
#include <linux/time.h>
#include <linux/device.h>
#include <linux/interrupt.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include "musb_core.h"
#include "musbhsdma.h"
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/init.h>
#include <linux/list.h>
#include <linux/clk.h>
#include <linux/usb.h>
#include <linux/platform_device.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <plat/dma.h>
#include <plat/mux.h>
#include <linux/kernel.h>
#include <linux/platform_device.h>
#include <linux/gpio.h>
+#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/usb.h>
#include <linux/workqueue.h>
#include <linux/platform_device.h>
#include <linux/dma-mapping.h>
#include <linux/usb/otg.h>
+#include <linux/slab.h>
struct nop_usb_xceiv {
struct otg_transceiver otg;
#include <linux/regulator/consumer.h>
#include <linux/err.h>
#include <linux/notifier.h>
+#include <linux/slab.h>
/* Register defines */
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include <linux/usb/otg.h>
#include <linux/usb/ulpi.h>
config USB_SERIAL_QCAUX
tristate "USB Qualcomm Auxiliary Serial Port Driver"
- ---help---
+ help
Say Y here if you want to use the auxiliary serial ports provided
by many modems based on Qualcomm chipsets. These ports often use
a proprietary protocol called DM and cannot be used for AT- or
PPP-based communication.
To compile this driver as a module, choose M here: the
- module will be called moto_modem. If unsure, choose N.
+ module will be called qcaux. If unsure, choose N.
config USB_SERIAL_QUALCOMM
tristate "USB Qualcomm Serial modem"
*/
#include <linux/tty.h>
+#include <linux/slab.h>
#include <linux/tty_flip.h>
#include <linux/circ_buf.h>
#include <linux/usb.h>
#include <linux/init.h>
#include <linux/ioctl.h>
#include <linux/tty.h>
+#include <linux/slab.h>
#include <linux/tty_flip.h>
#include <linux/module.h>
#include <linux/usb.h>
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/tty.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/usb.h>
#include <linux/usb/serial.h>
#include <linux/init.h>
#include <linux/tty.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include <linux/usb/serial.h>
#include <linux/serial.h>
/* The console is special in terms of closing the device so
* indicate this port is now acting as a system console. */
port->console = 1;
+ port->port.console = 1;
mutex_unlock(&serial->disc_mutex);
return retval;
return -EPROTO;
}
- /* Single data value */
- result = usb_control_msg(serial->dev,
- usb_sndctrlpipe(serial->dev, 0),
- request, REQTYPE_HOST_TO_DEVICE, data[0],
- 0, NULL, 0, 300);
return 0;
}
unsigned long tx_outstanding_bytes;
unsigned long tx_outstanding_urbs;
unsigned short max_packet_size;
- struct mutex cfg_lock; /* Avoid mess by parallel calls of config ioctl() */
+ struct mutex cfg_lock; /* Avoid mess by parallel calls of config ioctl() and change_speed() */
};
/* struct ftdi_sio_quirk is used by devices requiring special attention. */
{ USB_DEVICE(EVOLUTION_VID, EVOLUTION_ER1_PID) },
{ USB_DEVICE(EVOLUTION_VID, EVO_HYBRID_PID) },
{ USB_DEVICE(EVOLUTION_VID, EVO_RCM4_PID) },
+ { USB_DEVICE(CONTEC_VID, CONTEC_COM1USBH_PID) },
{ USB_DEVICE(FTDI_VID, FTDI_ARTEMIS_PID) },
{ USB_DEVICE(FTDI_VID, FTDI_ATIK_ATK16_PID) },
{ USB_DEVICE(FTDI_VID, FTDI_ATIK_ATK16C_PID) },
(priv->flags & ASYNC_SPD_MASK)) ||
(((priv->flags & ASYNC_SPD_MASK) == ASYNC_SPD_CUST) &&
(old_priv.custom_divisor != priv->custom_divisor))) {
- mutex_unlock(&priv->cfg_lock);
change_speed(tty, port);
+ mutex_unlock(&priv->cfg_lock);
}
else
mutex_unlock(&priv->cfg_lock);
clear_mctrl(port, TIOCM_DTR | TIOCM_RTS);
} else {
/* set the baudrate determined before */
+ mutex_lock(&priv->cfg_lock);
if (change_speed(tty, port))
dev_err(&port->dev, "%s urb failed to set baudrate\n",
__func__);
+ mutex_unlock(&priv->cfg_lock);
/* Ensure RTS and DTR are raised when baudrate changed from 0 */
if (!old_termios || (old_termios->c_cflag & CBAUD) == B0)
set_mctrl(port, TIOCM_DTR | TIOCM_RTS);
#define CONTEC_VID 0x06CE /* Vendor ID */
#define CONTEC_COM1USBH_PID 0x8311 /* COM-1(USB)H */
+/*
+ * Contec products (http://www.contec.com)
+ * Submitted by Daniel Sangorrin
+ */
+#define CONTEC_VID 0x06CE /* Vendor ID */
+#define CONTEC_COM1USBH_PID 0x8311 /* COM-1(USB)H */
+
/*
* Definitions for B&B Electronics products.
*/
spin_unlock_irqrestore(&port->lock, flags);
/* if we have a bulk endpoint, start reading from it */
- if (serial->num_bulk_in) {
+ if (port->bulk_in_size) {
/* Start reading from the device */
usb_fill_bulk_urb(port->read_urb, serial->dev,
usb_rcvbulkpipe(serial->dev,
dbg("%s - port %d", __func__, port->number);
if (serial->dev) {
- /* shutdown any bulk reads that might be going on */
- if (serial->num_bulk_out)
+ /* shutdown any bulk transfers that might be going on */
+ if (port->bulk_out_size)
usb_kill_urb(port->write_urb);
- if (serial->num_bulk_in)
+ if (port->bulk_in_size)
usb_kill_urb(port->read_urb);
}
}
dbg("%s - port %d", __func__, port->number);
+ /* only do something if we have a bulk out endpoint */
+ if (!port->bulk_out_size)
+ return -ENODEV;
+
if (count == 0) {
dbg("%s - write request of 0 bytes", __func__);
return 0;
}
- /* only do something if we have a bulk out endpoint */
- if (!serial->num_bulk_out)
- return 0;
-
if (serial->type->max_in_flight_urbs)
return usb_serial_multi_urb_write(tty, port,
buf, count);
int room = 0;
dbg("%s - port %d", __func__, port->number);
+
+ if (!port->bulk_out_size)
+ return 0;
+
spin_lock_irqsave(&port->lock, flags);
if (serial->type->max_in_flight_urbs) {
if (port->urbs_in_flight < serial->type->max_in_flight_urbs)
room = port->bulk_out_size *
(serial->type->max_in_flight_urbs -
port->urbs_in_flight);
- } else if (serial->num_bulk_out)
+ } else {
room = kfifo_avail(&port->write_fifo);
+ }
spin_unlock_irqrestore(&port->lock, flags);
dbg("%s - returns %d", __func__, room);
{
struct usb_serial_port *port = tty->driver_data;
struct usb_serial *serial = port->serial;
- int chars = 0;
unsigned long flags;
+ int chars;
dbg("%s - port %d", __func__, port->number);
+ if (!port->bulk_out_size)
+ return 0;
+
spin_lock_irqsave(&port->lock, flags);
if (serial->type->max_in_flight_urbs)
chars = port->tx_bytes_flight;
- else if (serial->num_bulk_out)
+ else
chars = kfifo_len(&port->write_fifo);
spin_unlock_irqrestore(&port->lock, flags);
((serial->type->read_bulk_callback) ?
serial->type->read_bulk_callback :
usb_serial_generic_read_bulk_callback), port);
+
result = usb_submit_urb(urb, mem_flags);
- if (result)
+ if (result && result != -EPERM) {
dev_err(&port->dev,
"%s - failed resubmitting read urb, error %d\n",
__func__, result);
+ }
}
EXPORT_SYMBOL_GPL(usb_serial_generic_resubmit_read_urb);
if (port->urbs_in_flight < 0)
port->urbs_in_flight = 0;
spin_unlock_irqrestore(&port->lock, flags);
-
- if (status) {
- dbg("%s - nonzero multi-urb write bulk status "
- "received: %d", __func__, status);
- return;
- }
} else {
port->write_urb_busy = 0;
- if (status) {
- dbg("%s - nonzero multi-urb write bulk status "
- "received: %d", __func__, status);
+ if (status)
kfifo_reset_out(&port->write_fifo);
- } else
+ else
usb_serial_generic_write_start(port);
}
+ if (status)
+ dbg("%s - non-zero urb status: %d", __func__, status);
+
usb_serial_port_softint(port);
}
EXPORT_SYMBOL_GPL(usb_serial_generic_write_bulk_callback);
* flags as the navman is rx only so cannot echo.
*/
+#include <linux/gfp.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/tty.h>
#include <linux/init.h>
#include <linux/tty.h>
#include <linux/tty_driver.h>
+#include <linux/slab.h>
#include <linux/tty_flip.h>
#include <linux/serial.h>
#include <linux/module.h>
#include <linux/errno.h>
#include <linux/tty.h>
#include <linux/tty_flip.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/bitops.h>
#include <linux/usb.h>
#define QUALCOMM_VENDOR_ID 0x05C6
-#define MAXON_VENDOR_ID 0x16d8
+#define CMOTECH_VENDOR_ID 0x16d8
+#define CMOTECH_PRODUCT_6008 0x6008
+#define CMOTECH_PRODUCT_6280 0x6280
#define TELIT_VENDOR_ID 0x1bc7
#define TELIT_PRODUCT_UC864E 0x1003
#define DLINK_VENDOR_ID 0x1186
#define DLINK_PRODUCT_DWM_652 0x3e04
#define DLINK_PRODUCT_DWM_652_U5 0xce16
+#define DLINK_PRODUCT_DWM_652_U5A 0xce1e
#define QISDA_VENDOR_ID 0x1da5
#define QISDA_PRODUCT_H21_4512 0x4512
#define ALCATEL_VENDOR_ID 0x1bbb
#define ALCATEL_PRODUCT_X060S 0x0000
+#define PIRELLI_VENDOR_ID 0x1266
+#define PIRELLI_PRODUCT_C100_1 0x1002
+#define PIRELLI_PRODUCT_C100_2 0x1003
+#define PIRELLI_PRODUCT_1004 0x1004
+#define PIRELLI_PRODUCT_1005 0x1005
+#define PIRELLI_PRODUCT_1006 0x1006
+#define PIRELLI_PRODUCT_1007 0x1007
+#define PIRELLI_PRODUCT_1008 0x1008
+#define PIRELLI_PRODUCT_1009 0x1009
+#define PIRELLI_PRODUCT_100A 0x100a
+#define PIRELLI_PRODUCT_100B 0x100b
+#define PIRELLI_PRODUCT_100C 0x100c
+#define PIRELLI_PRODUCT_100D 0x100d
+#define PIRELLI_PRODUCT_100E 0x100e
+#define PIRELLI_PRODUCT_100F 0x100f
+#define PIRELLI_PRODUCT_1011 0x1011
+#define PIRELLI_PRODUCT_1012 0x1012
+
/* Airplus products */
#define AIRPLUS_VENDOR_ID 0x1011
#define AIRPLUS_PRODUCT_MCD650 0x3198
{ USB_DEVICE(KYOCERA_VENDOR_ID, KYOCERA_PRODUCT_KPC680) },
{ USB_DEVICE(QUALCOMM_VENDOR_ID, 0x6000)}, /* ZTE AC8700 */
{ USB_DEVICE(QUALCOMM_VENDOR_ID, 0x6613)}, /* Onda H600/ZTE MF330 */
- { USB_DEVICE(MAXON_VENDOR_ID, 0x6280) }, /* BP3-USB & BP3-EXT HSDPA */
+ { USB_DEVICE(CMOTECH_VENDOR_ID, CMOTECH_PRODUCT_6280) }, /* BP3-USB & BP3-EXT HSDPA */
+ { USB_DEVICE(CMOTECH_VENDOR_ID, CMOTECH_PRODUCT_6008) },
{ USB_DEVICE(TELIT_VENDOR_ID, TELIT_PRODUCT_UC864E) },
{ USB_DEVICE(TELIT_VENDOR_ID, TELIT_PRODUCT_UC864G) },
{ USB_DEVICE_AND_INTERFACE_INFO(ZTE_VENDOR_ID, ZTE_PRODUCT_MF622, 0xff, 0xff, 0xff) }, /* ZTE WCDMA products */
{ USB_DEVICE(BENQ_VENDOR_ID, BENQ_PRODUCT_H10) },
{ USB_DEVICE(DLINK_VENDOR_ID, DLINK_PRODUCT_DWM_652) },
{ USB_DEVICE(ALINK_VENDOR_ID, DLINK_PRODUCT_DWM_652_U5) }, /* Yes, ALINK_VENDOR_ID */
+ { USB_DEVICE(ALINK_VENDOR_ID, DLINK_PRODUCT_DWM_652_U5A) },
{ USB_DEVICE(QISDA_VENDOR_ID, QISDA_PRODUCT_H21_4512) },
{ USB_DEVICE(QISDA_VENDOR_ID, QISDA_PRODUCT_H21_4523) },
{ USB_DEVICE(QISDA_VENDOR_ID, QISDA_PRODUCT_H20_4515) },
{ USB_DEVICE(TOSHIBA_VENDOR_ID, TOSHIBA_PRODUCT_G450) },
{ USB_DEVICE(TOSHIBA_VENDOR_ID, TOSHIBA_PRODUCT_HSDPA_MINICARD ) }, /* Toshiba 3G HSDPA == Novatel Expedite EU870D MiniCard */
{ USB_DEVICE(ALINK_VENDOR_ID, 0x9000) },
- { USB_DEVICE(ALINK_VENDOR_ID, 0xce16) },
{ USB_DEVICE_AND_INTERFACE_INFO(ALINK_VENDOR_ID, ALINK_PRODUCT_3GU, 0xff, 0xff, 0xff) },
{ USB_DEVICE(ALCATEL_VENDOR_ID, ALCATEL_PRODUCT_X060S) },
{ USB_DEVICE(AIRPLUS_VENDOR_ID, AIRPLUS_PRODUCT_MCD650) },
.driver_info = (kernel_ulong_t)&four_g_w14_blacklist
},
{ USB_DEVICE(HAIER_VENDOR_ID, HAIER_PRODUCT_CE100) },
+ /* Pirelli */
+ { USB_DEVICE(PIRELLI_VENDOR_ID, PIRELLI_PRODUCT_C100_1)},
+ { USB_DEVICE(PIRELLI_VENDOR_ID, PIRELLI_PRODUCT_C100_2)},
+ { USB_DEVICE(PIRELLI_VENDOR_ID, PIRELLI_PRODUCT_1004)},
+ { USB_DEVICE(PIRELLI_VENDOR_ID, PIRELLI_PRODUCT_1005)},
+ { USB_DEVICE(PIRELLI_VENDOR_ID, PIRELLI_PRODUCT_1006)},
+ { USB_DEVICE(PIRELLI_VENDOR_ID, PIRELLI_PRODUCT_1007)},
+ { USB_DEVICE(PIRELLI_VENDOR_ID, PIRELLI_PRODUCT_1008)},
+ { USB_DEVICE(PIRELLI_VENDOR_ID, PIRELLI_PRODUCT_1009)},
+ { USB_DEVICE(PIRELLI_VENDOR_ID, PIRELLI_PRODUCT_100A)},
+ { USB_DEVICE(PIRELLI_VENDOR_ID, PIRELLI_PRODUCT_100B) },
+ { USB_DEVICE(PIRELLI_VENDOR_ID, PIRELLI_PRODUCT_100C) },
+ { USB_DEVICE(PIRELLI_VENDOR_ID, PIRELLI_PRODUCT_100D) },
+ { USB_DEVICE(PIRELLI_VENDOR_ID, PIRELLI_PRODUCT_100E) },
+ { USB_DEVICE(PIRELLI_VENDOR_ID, PIRELLI_PRODUCT_100F) },
+ { USB_DEVICE(PIRELLI_VENDOR_ID, PIRELLI_PRODUCT_1011)},
+ { USB_DEVICE(PIRELLI_VENDOR_ID, PIRELLI_PRODUCT_1012)},
+
{ } /* Terminating entry */
};
MODULE_DEVICE_TABLE(usb, option_ids);
const struct usb_device_id *id)
{
struct option_intf_private *data;
+
/* D-Link DWM 652 still exposes CD-Rom emulation interface in modem mode */
if (serial->dev->descriptor.idVendor == DLINK_VENDOR_ID &&
serial->dev->descriptor.idProduct == DLINK_PRODUCT_DWM_652 &&
serial->interface->cur_altsetting->desc.bInterfaceClass == 0x8)
return -ENODEV;
+ /* Bandrich modem and AT command interface is 0xff */
+ if ((serial->dev->descriptor.idVendor == BANDRICH_VENDOR_ID ||
+ serial->dev->descriptor.idVendor == PIRELLI_VENDOR_ID) &&
+ serial->interface->cur_altsetting->desc.bInterfaceClass != 0xff)
+ return -ENODEV;
+
data = serial->private = kzalloc(sizeof(struct option_intf_private), GFP_KERNEL);
if (!data)
return -ENOMEM;
{USB_DEVICE(0x05c6, 0x9221)}, /* Generic Gobi QDL device */
{USB_DEVICE(0x05c6, 0x9231)}, /* Generic Gobi QDL device */
{USB_DEVICE(0x1f45, 0x0001)}, /* Unknown Gobi QDL device */
+ {USB_DEVICE(0x413c, 0x8185)}, /* Dell Gobi 2000 QDL device (N0218, VU936) */
+ {USB_DEVICE(0x413c, 0x8186)}, /* Dell Gobi 2000 Modem device (N0218, VU936) */
+ {USB_DEVICE(0x05c6, 0x9224)}, /* Sony Gobi 2000 QDL device (N0279, VU730) */
+ {USB_DEVICE(0x05c6, 0x9225)}, /* Sony Gobi 2000 Modem device (N0279, VU730) */
+ {USB_DEVICE(0x05c6, 0x9244)}, /* Samsung Gobi 2000 QDL device (VL176) */
+ {USB_DEVICE(0x05c6, 0x9245)}, /* Samsung Gobi 2000 Modem device (VL176) */
+ {USB_DEVICE(0x03f0, 0x241d)}, /* HP Gobi 2000 QDL device (VP412) */
+ {USB_DEVICE(0x03f0, 0x251d)}, /* HP Gobi 2000 Modem device (VP412) */
+ {USB_DEVICE(0x05c6, 0x9214)}, /* Acer Gobi 2000 QDL device (VP413) */
+ {USB_DEVICE(0x05c6, 0x9215)}, /* Acer Gobi 2000 Modem device (VP413) */
+ {USB_DEVICE(0x05c6, 0x9264)}, /* Asus Gobi 2000 QDL device (VR305) */
+ {USB_DEVICE(0x05c6, 0x9265)}, /* Asus Gobi 2000 Modem device (VR305) */
+ {USB_DEVICE(0x05c6, 0x9234)}, /* Top Global Gobi 2000 QDL device (VR306) */
+ {USB_DEVICE(0x05c6, 0x9235)}, /* Top Global Gobi 2000 Modem device (VR306) */
+ {USB_DEVICE(0x05c6, 0x9274)}, /* iRex Technologies Gobi 2000 QDL device (VR307) */
+ {USB_DEVICE(0x05c6, 0x9275)}, /* iRex Technologies Gobi 2000 Modem device (VR307) */
+ {USB_DEVICE(0x1199, 0x9000)}, /* Sierra Wireless Gobi 2000 QDL device (VT773) */
+ {USB_DEVICE(0x1199, 0x9001)}, /* Sierra Wireless Gobi 2000 Modem device (VT773) */
+ {USB_DEVICE(0x1199, 0x9002)}, /* Sierra Wireless Gobi 2000 Modem device (VT773) */
+ {USB_DEVICE(0x1199, 0x9003)}, /* Sierra Wireless Gobi 2000 Modem device (VT773) */
+ {USB_DEVICE(0x1199, 0x9004)}, /* Sierra Wireless Gobi 2000 Modem device (VT773) */
+ {USB_DEVICE(0x1199, 0x9005)}, /* Sierra Wireless Gobi 2000 Modem device (VT773) */
+ {USB_DEVICE(0x1199, 0x9006)}, /* Sierra Wireless Gobi 2000 Modem device (VT773) */
+ {USB_DEVICE(0x1199, 0x9007)}, /* Sierra Wireless Gobi 2000 Modem device (VT773) */
+ {USB_DEVICE(0x1199, 0x9008)}, /* Sierra Wireless Gobi 2000 Modem device (VT773) */
+ {USB_DEVICE(0x1199, 0x9009)}, /* Sierra Wireless Gobi 2000 Modem device (VT773) */
+ {USB_DEVICE(0x1199, 0x900a)}, /* Sierra Wireless Gobi 2000 Modem device (VT773) */
+ {USB_DEVICE(0x16d8, 0x8001)}, /* CMDTech Gobi 2000 QDL device (VU922) */
+ {USB_DEVICE(0x16d8, 0x8002)}, /* CMDTech Gobi 2000 Modem device (VU922) */
{ } /* Terminating entry */
};
MODULE_DEVICE_TABLE(usb, id_table);
#include <linux/kernel.h>
#include <linux/errno.h>
+#include <linux/gfp.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/tty.h>
#include <linux/tty_driver.h>
#include <linux/tty_flip.h>
#include <linux/jiffies.h>
#include <linux/errno.h>
#include <linux/tty.h>
+#include <linux/slab.h>
#include <linux/tty_flip.h>
#include <linux/module.h>
#include <linux/usb.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/tty.h>
+#include <linux/slab.h>
#include <linux/tty_driver.h>
#include <linux/tty_flip.h>
#include <linux/module.h>
* 2 as published by the Free Software Foundation.
*/
+#include <linux/gfp.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/tty.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <scsi/scsi.h>
#include <scsi/scsi_cmnd.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <scsi/scsi.h>
#include <scsi/scsi_cmnd.h>
*/
#include <linux/usb.h>
+#include <linux/slab.h>
#include "usb.h"
#include "transport.h"
* 675 Mass Ave, Cambridge, MA 02139, USA.
*/
-#include <linux/slab.h>
#include <linux/module.h>
#include <linux/mutex.h>
#include <scsi/scsi_cmnd.h>
#include <scsi/scsi_device.h>
#include <linux/usb.h>
+#include <linux/slab.h>
#include "usb.h"
#include "transport.h"
*/
#include <linux/sched.h>
+#include <linux/gfp.h>
#include <linux/errno.h>
-#include <linux/slab.h>
#include <linux/usb/quirks.h>
US_SC_DEVICE, US_PR_DEVICE, NULL,
US_FL_FIX_INQUIRY),
+/* Reported by Ondrej Zary <linux@rainbow-software.org>
+ * The device reports one sector more and breaks when that sector is accessed
+ */
+UNUSUAL_DEV( 0x04ce, 0x0002, 0x026c, 0x026c,
+ "ScanLogic",
+ "SL11R-IDE",
+ US_SC_DEVICE, US_PR_DEVICE, NULL,
+ US_FL_FIX_CAPACITY),
+
/* Reported by Kriston Fincher <kriston@airmail.net>
* Patch submitted by Sean Millichamp <sean@bruenor.org>
* This is to support the Panasonic PalmCam PV-SD4090
US_SC_DEVICE, US_PR_DEVICE, NULL,
US_FL_IGNORE_RESIDUE ),
-/* Jeremy Katz <katzj@redhat.com>:
- * The Blackberry Pearl can run in two modes; a usb-storage only mode
- * and a mode that allows access via mass storage and to its database.
- * The berry_charge module will set the device to dual mode and thus we
- * should ignore its native mode if that module is built
- */
-#ifdef CONFIG_USB_BERRY_CHARGE
-UNUSUAL_DEV( 0x0fca, 0x0006, 0x0001, 0x0001,
- "RIM",
- "Blackberry Pearl",
- US_SC_DEVICE, US_PR_DEVICE, NULL,
- US_FL_IGNORE_DEVICE ),
-#endif
-
/* Reported by Michael Stattmann <michael@stattmann.com> */
UNUSUAL_DEV( 0x0fce, 0xd008, 0x0000, 0x0000,
"Sony Ericsson",
#include <linux/interrupt.h>
#include <linux/delay.h>
#include <linux/random.h>
+#include <linux/slab.h>
#include <linux/mutex.h>
#include <linux/uwb.h>
#include <linux/usb/wusb.h>
#include <linux/module.h>
#include <linux/err.h>
#include <linux/uwb.h>
+#include <linux/slab.h>
#include <linux/usb/wusb.h>
#include <linux/scatterlist.h>
#include <linux/jiffies.h>
#include <linux/ctype.h>
+#include <linux/slab.h>
#include <linux/workqueue.h>
#include "wusbhc.h"
* - add timers that autoremove intervalled IEs?
*/
#include <linux/usb/wusb.h>
+#include <linux/slab.h>
#include "wusbhc.h"
/* Initialize the MMCIEs handling mechanism */
*
* wusbhc_rh_start_port_reset() ??? unimplemented
*/
+#include <linux/slab.h>
#include "wusbhc.h"
/*
* FIXME: docs
*/
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/usb/ch9.h>
#include <linux/random.h>
#include "wusbhc.h"
*
* FIXME: docs
*/
+#include <linux/slab.h>
#include "wusbhc.h"
#include "wa-hc.h"
*/
#include <linux/workqueue.h>
#include <linux/ctype.h>
+#include <linux/slab.h>
#include "wa-hc.h"
#include "wusbhc.h"
#include <linux/init.h>
#include <asm/atomic.h>
#include <linux/bitmap.h>
+#include <linux/slab.h>
#include "wusbhc.h"
#include "wa-hc.h"
*/
#include <linux/init.h>
#include <linux/spinlock.h>
+#include <linux/slab.h>
#include <linux/hash.h>
#include "wa-hc.h"
* FIXME: docs
*/
+#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/module.h>
#include <linux/device.h>
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/uwb.h>
#include "uwb-internal.h"
#include <linux/device.h>
#include <linux/err.h>
#include <linux/kdev_t.h>
+#include <linux/slab.h>
#include "uwb-internal.h"
*/
#include <linux/kernel.h>
#include <linux/random.h>
+#include <linux/slab.h>
#include <linux/uwb.h>
#include "uwb-internal.h"
*/
#include <linux/kthread.h>
#include <linux/freezer.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include "uwb-internal.h"
* uwb_est_get_size()
*/
#include <linux/spinlock.h>
+#include <linux/slab.h>
#include "uwb-internal.h"
*/
#include <linux/init.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include <linux/usb/wusb.h>
#include <linux/usb/wusb-wa.h>
}
/** USB device ID's that we handle */
-static struct usb_device_id hwarc_id_table[] = {
+static const struct usb_device_id hwarc_id_table[] = {
/* D-Link DUB-1210 */
{ USB_DEVICE_AND_INTERFACE_INFO(0x07d1, 0x3d02, 0xe0, 0x01, 0x02),
.driver_info = WUSB_QUIRK_WHCI_CMD_EVT },
*/
#include <linux/delay.h>
#include <linux/firmware.h>
+#include <linux/slab.h>
#include <linux/uwb.h>
#include "i1480-dfu.h"
#include <linux/module.h>
#include <linux/usb.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/uwb.h>
#include <linux/usb/wusb.h>
result = usb_control_msg(
i1480_usb->usb_dev, usb_sndctrlpipe(i1480_usb->usb_dev, 0),
0xf0, USB_DIR_OUT | USB_TYPE_VENDOR | USB_RECIP_DEVICE,
- cpu_to_le16(memory_address & 0xffff),
- cpu_to_le16((memory_address >> 16) & 0xffff),
+ memory_address, (memory_address >> 16),
i1480->cmd_buf, buffer_size, 100 /* FIXME: arbitrary */);
if (result < 0)
break;
result = usb_control_msg(
i1480_usb->usb_dev, usb_rcvctrlpipe(i1480_usb->usb_dev, 0),
0xf0, USB_DIR_IN | USB_TYPE_VENDOR | USB_RECIP_DEVICE,
- cpu_to_le16(itr_addr & 0xffff),
- cpu_to_le16((itr_addr >> 16) & 0xffff),
+ itr_addr, (itr_addr >> 16),
i1480->cmd_buf + itr, itr_size,
100 /* FIXME: arbitrary */);
if (result < 0) {
return result;
}
+MODULE_FIRMWARE("i1480-pre-phy-0.0.bin");
+MODULE_FIRMWARE("i1480-usb-0.0.bin");
+MODULE_FIRMWARE("i1480-phy-0.0.bin");
+
#define i1480_USB_DEV(v, p) \
{ \
.match_flags = USB_DEVICE_ID_MATCH_DEVICE \
/** USB device ID's that we handle */
-static struct usb_device_id i1480_usb_id_table[] = {
+static const struct usb_device_id i1480_usb_id_table[] = {
i1480_USB_DEV(0x8086, 0xdf3b),
i1480_USB_DEV(0x15a9, 0x0005),
i1480_USB_DEV(0x07d1, 0x3802),
* is being removed.
* i1480u_rm()
*/
+#include <linux/gfp.h>
#include <linux/if_arp.h>
#include <linux/etherdevice.h>
* i1480u_set_config():
*/
+#include <linux/slab.h>
#include <linux/if_arp.h>
#include <linux/etherdevice.h>
*
*/
+#include <linux/gfp.h>
#include <linux/netdevice.h>
#include <linux/etherdevice.h>
#include "i1480u-wlp.h"
* the times the MTU will be smaller than one page...
*/
+#include <linux/slab.h>
#include "i1480u-wlp.h"
enum {
* FIXME: docs
*/
+#include <linux/slab.h>
#include "uwb-internal.h"
/**
* FIXME: docs
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/device.h>
#include <linux/err.h>
#include <linux/kdev_t.h>
#include <linux/kdev_t.h>
#include <linux/etherdevice.h>
#include <linux/usb.h>
+#include <linux/slab.h>
#include "uwb-internal.h"
*/
#include <linux/kernel.h>
#include <linux/timer.h>
+#include <linux/slab.h>
#include <linux/err.h>
#include "uwb-internal.h"
*/
#include <linux/kernel.h>
#include <linux/err.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include "uwb-internal.h"
*/
#include <linux/kernel.h>
#include <linux/uwb.h>
+#include <linux/slab.h>
#include <linux/random.h>
#include "uwb-internal.h"
#include <linux/device.h>
#include <linux/err.h>
+#include <linux/slab.h>
#include "uwb-internal.h"
* This file is released under the GNU GPL v2.
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/uwb/umc.h>
static void umc_device_release(struct device *dev)
* Handler functions are called normally uwbd_evt_handle_*().
*/
#include <linux/kthread.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/freezer.h>
#include <linux/sched.h>
#include <linux/dma-mapping.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <linux/workqueue.h>
#include <linux/uwb.h>
#include <linux/uwb/whci.h>
#include <linux/kernel.h>
#include <linux/pci.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <linux/uwb/whci.h>
#include <linux/uwb/umc.h>
#include <linux/netdevice.h>
#include <linux/etherdevice.h>
+#include <linux/slab.h>
#include <linux/wlp.h>
#include "wlp-internal.h"
*/
#include <linux/wlp.h>
+#include <linux/slab.h>
#include "wlp-internal.h"
}
+static ssize_t wlp_get_attribute(struct wlp *wlp, u16 type_code,
+ struct wlp_attr_hdr *attr_hdr, void *value, ssize_t value_len,
+ ssize_t buflen)
+{
+ struct device *dev = &wlp->rc->uwb_dev.dev;
+ ssize_t attr_len = sizeof(*attr_hdr) + value_len;
+ if (buflen < 0)
+ return -EINVAL;
+ if (buflen < attr_len) {
+ dev_err(dev, "WLP: Not enough space in buffer to parse"
+ " attribute field. Need %d, received %zu\n",
+ (int)attr_len, buflen);
+ return -EIO;
+ }
+ if (wlp_check_attr_hdr(wlp, attr_hdr, type_code, value_len) < 0) {
+ dev_err(dev, "WLP: Header verification failed. \n");
+ return -EINVAL;
+ }
+ memcpy(value, (void *)attr_hdr + sizeof(*attr_hdr), value_len);
+ return attr_len;
+}
+
+static ssize_t wlp_vget_attribute(struct wlp *wlp, u16 type_code,
+ struct wlp_attr_hdr *attr_hdr, void *value, ssize_t max_value_len,
+ ssize_t buflen)
+{
+ struct device *dev = &wlp->rc->uwb_dev.dev;
+ size_t len;
+ if (buflen < 0)
+ return -EINVAL;
+ if (buflen < sizeof(*attr_hdr)) {
+ dev_err(dev, "WLP: Not enough space in buffer to parse"
+ " header.\n");
+ return -EIO;
+ }
+ if (le16_to_cpu(attr_hdr->type) != type_code) {
+ dev_err(dev, "WLP: Unexpected attribute type. Got %u, "
+ "expected %u.\n", le16_to_cpu(attr_hdr->type),
+ type_code);
+ return -EINVAL;
+ }
+ len = le16_to_cpu(attr_hdr->length);
+ if (len > max_value_len) {
+ dev_err(dev, "WLP: Attribute larger than maximum "
+ "allowed. Received %zu, max is %d.\n", len,
+ (int)max_value_len);
+ return -EFBIG;
+ }
+ if (buflen < sizeof(*attr_hdr) + len) {
+ dev_err(dev, "WLP: Not enough space in buffer to parse "
+ "variable data.\n");
+ return -EIO;
+ }
+ memcpy(value, (void *)attr_hdr + sizeof(*attr_hdr), len);
+ return sizeof(*attr_hdr) + len;
+}
+
/**
* Get value of attribute from fixed size attribute field.
*
ssize_t wlp_get_##name(struct wlp *wlp, struct wlp_attr_##name *attr, \
type *value, ssize_t buflen) \
{ \
- struct device *dev = &wlp->rc->uwb_dev.dev; \
- if (buflen < 0) \
- return -EINVAL; \
- if (buflen < sizeof(*attr)) { \
- dev_err(dev, "WLP: Not enough space in buffer to parse" \
- " attribute field. Need %d, received %zu\n", \
- (int)sizeof(*attr), buflen); \
- return -EIO; \
- } \
- if (wlp_check_attr_hdr(wlp, &attr->hdr, type_code, \
- sizeof(attr->name)) < 0) { \
- dev_err(dev, "WLP: Header verification failed. \n"); \
- return -EINVAL; \
- } \
- *value = attr->name; \
- return sizeof(*attr); \
+ return wlp_get_attribute(wlp, (type_code), &attr->hdr, \
+ value, sizeof(*value), buflen); \
}
#define wlp_get_sparse(type, type_code, name) \
struct wlp_attr_##name *attr, \
type_val *value, ssize_t buflen) \
{ \
- struct device *dev = &wlp->rc->uwb_dev.dev; \
- size_t len; \
- if (buflen < 0) \
- return -EINVAL; \
- if (buflen < sizeof(*attr)) { \
- dev_err(dev, "WLP: Not enough space in buffer to parse" \
- " header.\n"); \
- return -EIO; \
- } \
- if (le16_to_cpu(attr->hdr.type) != type_code) { \
- dev_err(dev, "WLP: Unexpected attribute type. Got %u, " \
- "expected %u.\n", le16_to_cpu(attr->hdr.type), \
- type_code); \
- return -EINVAL; \
- } \
- len = le16_to_cpu(attr->hdr.length); \
- if (len > max) { \
- dev_err(dev, "WLP: Attribute larger than maximum " \
- "allowed. Received %zu, max is %d.\n", len, \
- (int)max); \
- return -EFBIG; \
- } \
- if (buflen < sizeof(*attr) + len) { \
- dev_err(dev, "WLP: Not enough space in buffer to parse "\
- "variable data.\n"); \
- return -EIO; \
- } \
- memcpy(value, (void *) attr + sizeof(*attr), len); \
- return sizeof(*attr) + len; \
+ return wlp_vget_attribute(wlp, (type_code), &attr->hdr, \
+ value, (max), buflen); \
}
wlp_get(u8, WLP_ATTR_WLP_VER, version)
*/
#include <linux/etherdevice.h>
+#include <linux/slab.h>
#include <linux/wlp.h>
#include "wlp-internal.h"
* FIXME: docs
*/
#include <linux/wlp.h>
+#include <linux/slab.h>
#include "wlp-internal.h"
*/
#include <linux/etherdevice.h> /* for is_valid_ether_addr */
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <linux/wlp.h>
#include "wlp-internal.h"
#include <linux/workqueue.h>
#include <linux/rcupdate.h>
#include <linux/file.h>
+#include <linux/slab.h>
#include <linux/net.h>
#include <linux/if_packet.h>
mutex_lock(&vq->mutex);
vhost_disable_notify(vq);
- if (wmem < sock->sk->sk_sndbuf * 2)
+ if (wmem < sock->sk->sk_sndbuf / 2)
tx_poll_stop(net);
hdr_size = vq->hdr_size;
/* Verify that ring has been setup correctly. */
if (!vhost_vq_access_ok(vq)) {
r = -EFAULT;
- goto err;
+ goto err_vq;
}
sock = get_socket(fd);
if (IS_ERR(sock)) {
r = PTR_ERR(sock);
- goto err;
+ goto err_vq;
}
/* start polling new socket */
vhost_net_disable_vq(n, vq);
rcu_assign_pointer(vq->private_data, sock);
vhost_net_enable_vq(n, vq);
- mutex_unlock(&vq->mutex);
done:
if (oldsock) {
vhost_net_flush_vq(n, index);
fput(oldsock->file);
}
+
+err_vq:
+ mutex_unlock(&vq->mutex);
err:
mutex_unlock(&n->dev.mutex);
return r;
#include <linux/poll.h>
#include <linux/file.h>
#include <linux/highmem.h>
+#include <linux/slab.h>
#include <linux/net.h>
#include <linux/if_packet.h>
if (r < 0)
break;
eventfp = f.fd == -1 ? NULL : eventfd_fget(f.fd);
- if (IS_ERR(eventfp))
- return PTR_ERR(eventfp);
+ if (IS_ERR(eventfp)) {
+ r = PTR_ERR(eventfp);
+ break;
+ }
if (eventfp != vq->kick) {
pollstop = filep = vq->kick;
pollstart = vq->kick = eventfp;
if (r < 0)
break;
eventfp = f.fd == -1 ? NULL : eventfd_fget(f.fd);
- if (IS_ERR(eventfp))
- return PTR_ERR(eventfp);
+ if (IS_ERR(eventfp)) {
+ r = PTR_ERR(eventfp);
+ break;
+ }
if (eventfp != vq->call) {
filep = vq->call;
ctx = vq->call_ctx;
if (r < 0)
break;
eventfp = f.fd == -1 ? NULL : eventfd_fget(f.fd);
- if (IS_ERR(eventfp))
- return PTR_ERR(eventfp);
+ if (IS_ERR(eventfp)) {
+ r = PTR_ERR(eventfp);
+ break;
+ }
if (eventfp != vq->error) {
filep = vq->error;
vq->error = eventfp;
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/ctype.h>
-#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/init.h>
#include <linux/fb.h>
#include <linux/platform_device.h>
#include <linux/dma-mapping.h>
#include <linux/io.h>
+#include <linux/gfp.h>
#include <mach/hardware.h>
#include <asm/irq.h>
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/fb.h>
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/init.h>
#include <linux/delay.h>
#include <linux/backlight.h>
+#include <linux/gfp.h>
#include <mach/board.h>
#include <mach/cpu.h>
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
* ATI Mach64 CT/VT/GT/LT Cursor Support
*/
-#include <linux/slab.h>
#include <linux/fb.h>
#include <linux/init.h>
#include <linux/string.h>
#include "radeonfb.h"
#include <linux/backlight.h>
+#include <linux/slab.h>
#ifdef CONFIG_PMAC_BACKLIGHT
#include <asm/backlight.h>
#include "radeonfb.h"
+
+#include <linux/slab.h>
+
#include "../edid.h"
static struct fb_var_screeninfo radeonfb_default_var = {
#include <linux/ctype.h>
#include <linux/dma-mapping.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <asm/mach-au1x00/au1000.h>
#include <linux/interrupt.h>
#include <linux/ctype.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <asm/mach-au1x00/au1000.h>
#include "au1200fb.h"
#include <linux/i2c.h>
#include <linux/backlight.h>
#include <linux/mfd/88pm860x.h>
+#include <linux/slab.h>
#define MAX_BRIGHTNESS (0xFF)
#define MIN_BRIGHTNESS (0)
#include <linux/fb.h>
#include <linux/backlight.h>
#include <linux/mfd/adp5520.h>
+#include <linux/slab.h>
struct adp5520_bl {
struct device *master;
#include <linux/backlight.h>
#include <linux/fb.h>
+#include <linux/gfp.h>
#include <linux/io.h>
#include <linux/module.h>
#include <linux/platform_device.h>
#include <linux/backlight.h>
#include <linux/atmel_pwm.h>
#include <linux/atmel-pwm-bl.h>
+#include <linux/slab.h>
struct atmel_pwm_bl {
const struct atmel_pwm_bl_platform_data *pdata;
#include <linux/ctype.h>
#include <linux/err.h>
#include <linux/fb.h>
+#include <linux/slab.h>
#ifdef CONFIG_PMAC_BACKLIGHT
#include <asm/backlight.h>
#include <linux/lcd.h>
#include <linux/spi/spi.h>
#include <linux/spi/corgi_lcd.h>
+#include <linux/slab.h>
#include <asm/mach/sharpsl_param.h>
#define POWER_IS_ON(pwr) ((pwr) <= FB_BLANK_NORMAL)
#include <linux/backlight.h>
#include <linux/lcd.h>
#include <linux/pci.h>
+#include <linux/slab.h>
/* The LVDS- and panel power controls sits on the
* GPIO port of the ISA bridge.
#include <linux/fb.h>
#include <linux/backlight.h>
#include <linux/mfd/da903x.h>
+#include <linux/slab.h>
#define DA9030_WLED_CONTROL 0x25
#define DA9030_WLED_CP_EN (1 << 6)
#include <linux/init.h>
#include <linux/lcd.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/spi/spi.h>
#include <linux/delay.h>
#include <linux/gpio.h>
#include <linux/lcd.h>
+#include <linux/slab.h>
#include <linux/regulator/consumer.h>
#include <linux/spi/spi.h>
#include <linux/ctype.h>
#include <linux/err.h>
#include <linux/fb.h>
+#include <linux/slab.h>
#if defined(CONFIG_FB) || (defined(CONFIG_FB_MODULE) && \
defined(CONFIG_LCD_CLASS_DEVICE_MODULE))
#include <linux/device.h>
#include <linux/kernel.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/gpio.h>
#include <linux/lcd.h>
#include <linux/init.h>
#include <linux/lcd.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/spi/spi.h>
#include "ltv350qv.h"
#include <linux/i2c.h>
#include <linux/backlight.h>
#include <linux/mfd/max8925.h>
+#include <linux/slab.h>
#define MAX_BRIGHTNESS (0xff)
#define MIN_BRIGHTNESS (0)
#include <linux/platform_device.h>
#include <linux/fb.h>
#include <linux/backlight.h>
+#include <linux/slab.h>
#include <mach/hardware.h>
#include <plat/board.h>
#include <linux/fb.h>
#include <linux/backlight.h>
#include <linux/lcd.h>
+#include <linux/slab.h>
#include <video/platform_lcd.h>
#include <linux/err.h>
#include <linux/pwm.h>
#include <linux/pwm_backlight.h>
+#include <linux/slab.h>
struct pwm_bl_data {
struct pwm_device *pwm;
#include <linux/spi/tdo24m.h>
#include <linux/fb.h>
#include <linux/lcd.h>
+#include <linux/slab.h>
#define POWER_IS_ON(pwr) ((pwr) <= FB_BLANK_NORMAL)
#include <linux/gpio.h>
#include <linux/fb.h>
#include <linux/backlight.h>
+#include <linux/slab.h>
#include <asm/mach/sharpsl_param.h>
#include <linux/device.h>
#include <linux/spi/spi.h>
#include <linux/i2c.h>
+#include <linux/slab.h>
#include <linux/gpio.h>
#include <linux/delay.h>
#include <linux/lcd.h>
#include <linux/platform_device.h>
#include <linux/fb.h>
#include <linux/backlight.h>
+#include <linux/slab.h>
#include <linux/mfd/wm831x/core.h>
#include <linux/mfd/wm831x/pdata.h>
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/fb.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/types.h>
#include <linux/interrupt.h>
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/string.h>
+#include <linux/gfp.h>
#include <linux/fb.h>
#include <linux/init.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/string.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/fb.h>
#include <linux/fb.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include "carminefb.h"
#include "carminefb_regs.h"
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/fb.h>
-#include <linux/slab.h>
#include <asm/types.h>
#include <asm/io.h>
#include "fb_draw.h"
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/string.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/fb.h>
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/string.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/fb.h>
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/string.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/fb.h>
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/fb.h>
#include <linux/init.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include <linux/fb.h>
#include <linux/vt_kern.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include <linux/fb.h>
#include <linux/vt_kern.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include <linux/fb.h>
#include <linux/vt_kern.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include <linux/fb.h>
#include <linux/vt_kern.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include <linux/fb.h>
#include <linux/vt_kern.h>
#include <linux/console.h>
#include <linux/string.h>
#include <linux/kd.h>
-#include <linux/slab.h>
#include <linux/vt_kern.h>
#include <linux/vt_buffer.h>
#include <linux/selection.h>
#include <linux/clk.h>
#include <linux/cpufreq.h>
#include <linux/console.h>
+#include <linux/slab.h>
#include <video/da8xx-fb.h>
#define DRIVER_NAME "da8xx_lcdc"
#include <linux/idr.h>
#include <linux/err.h>
#include <linux/kdev_t.h>
+#include <linux/slab.h>
static ssize_t display_show_name(struct device *dev,
struct device_attribute *attr, char *buf)
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/platform_device.h>
#include <linux/platform_device.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <linux/clk.h>
#include <linux/fb.h>
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/fb.h>
#include <linux/init.h>
#include <linux/device.h>
#include <linux/fb.h>
#include <linux/i2c-algo-bit.h>
+#include <linux/slab.h>
#include "edid.h"
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
*
*/
#include <linux/fb.h>
+#include <linux/slab.h>
#define FB_CVT_CELLSIZE 8
#define FB_CVT_GTF_C 40
#include <linux/fb.h>
#include <linux/module.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <video/edid.h>
#ifdef CONFIG_PPC_OF
#include <asm/prom.h>
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/fb.h>
#include <linux/console.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/string.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/fb.h>
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/platform_device.h>
#include <linux/dma-mapping.h>
#include <linux/errno.h>
+#include <linux/gfp.h>
#include <linux/fb.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/fb.h>
#include <linux/init.h>
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/fb.h>
#include <linux/console.h>
FP_CRC, /* 0x458 */
};
+#define FP_PT2_HSP (1 << 22)
+#define FP_PT2_VSP (1 << 23)
#define FP_PT2_SCRC (1 << 27) /* shfclk free */
#define FP_PM_P (1 << 24) /* panel power ctl */
#include <linux/string.h>
#include <linux/console.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/suspend.h>
#include <linux/delay.h>
#include <linux/fb.h>
u32 msrlo, msrhi;
write_fp(par, FP_PT1, 0);
- write_fp(par, FP_PT2, FP_PT2_SCRC);
+ temp = FP_PT2_SCRC;
+
+ if (info->var.sync & FB_SYNC_HOR_HIGH_ACT)
+ temp |= FP_PT2_HSP;
+
+ if (info->var.sync & FB_SYNC_VERT_HIGH_ACT)
+ temp |= FP_PT2_VSP;
+
+ write_fp(par, FP_PT2, temp);
write_fp(par, FP_DFC, FP_DFC_BC);
msrlo = MSR_LX_MSR_PADSEL_TFT_SEL_LOW;
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/spinlock.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/fb.h>
#include <linux/init.h>
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/platform_device.h>
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/fb.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/delay.h>
+#include <linux/gfp.h>
#include <linux/pci.h>
#include <linux/fb.h>
#include "i810.h"
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/fb.h>
#include <linux/ioport.h>
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/string.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/fb.h>
#include "matroxfb_base.h"
#include "matroxfb_maven.h"
#include <linux/i2c.h>
+#include <linux/slab.h>
#include <linux/i2c-algo-bit.h>
/* MGA-TVO I2C for G200, G400 */
#include "matroxfb_g450.h"
#include <linux/matroxfb.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <linux/uaccess.h>
#ifdef CONFIG_PPC_PMAC
#include "matroxfb_misc.h"
#include "matroxfb_DAC1064.h"
#include <linux/matroxfb.h>
+#include <linux/slab.h>
#include <linux/uaccess.h>
/* **************************************************** */
#include "matroxfb_DAC1064.h"
#include <linux/i2c.h>
#include <linux/matroxfb.h>
+#include <linux/slab.h>
#include <asm/div64.h>
#define MGATVO_B 1
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/fb.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#if defined(CONFIG_OF)
#include <linux/of_platform.h>
#endif
#include <linux/debugfs.h>
+#include <linux/slab.h>
#define BIG_BUFFER_SIZE (1024)
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/fb.h>
#include <linux/kernel.h>
#include <linux/interrupt.h>
#include <linux/platform_device.h>
#include <linux/delay.h>
+#include <linux/gfp.h>
#include <linux/spinlock.h>
#include <linux/clk.h>
#include <linux/io.h>
* GNU General Public License for more details.
*/
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/platform_device.h>
#include <linux/interrupt.h>
#include <linux/sched.h>
#include <linux/gpio.h>
+#include <linux/slab.h>
#include <mach/msm_fb.h>
static DECLARE_WAIT_QUEUE_HEAD(nt35399_vsync_wait);
#include <linux/interrupt.h>
#include <linux/gpio.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <mach/msm_fb.h>
#include <linux/clk.h>
#include <linux/file.h>
#include <linux/major.h>
+#include <linux/slab.h>
#include <mach/msm_iomap.h>
#include <mach/msm_fb.h>
#include <linux/platform_device.h>
#include <linux/module.h>
#include <linux/fb.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/freezer.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/delay.h>
+#include <linux/gfp.h>
#include <linux/pci.h>
#include <linux/fb.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/delay.h>
+#include <linux/gfp.h>
#include <linux/pci.h>
#include <linux/fb.h>
#include <video/vga.h>
#include <linux/delay.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include "nv_type.h"
#include "nv_local.h"
#include "nv_proto.h"
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/clk.h>
#include <linux/io.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <plat/sram.h>
#include <plat/board.h>
*/
#include <linux/device.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/workqueue.h>
#include <linux/spi/spi.h>
#include <linux/dma-mapping.h>
#include <linux/vmalloc.h>
#include <linux/clk.h>
+#include <linux/gfp.h>
#include <mach/lcdc.h>
#include <plat/dma.h>
*/
#include <linux/platform_device.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include <linux/uaccess.h>
#include <plat/dma.h>
return 0;
}
+static void generic_panel_set_timings(struct omap_dss_device *dssdev,
+ struct omap_video_timings *timings)
+{
+ dpi_set_timings(dssdev, timings);
+}
+
+static void generic_panel_get_timings(struct omap_dss_device *dssdev,
+ struct omap_video_timings *timings)
+{
+ *timings = dssdev->panel.timings;
+}
+
+static int generic_panel_check_timings(struct omap_dss_device *dssdev,
+ struct omap_video_timings *timings)
+{
+ return dpi_check_timings(dssdev, timings);
+}
+
static struct omap_dss_driver generic_driver = {
.probe = generic_panel_probe,
.remove = generic_panel_remove,
.suspend = generic_panel_suspend,
.resume = generic_panel_resume,
+ .set_timings = generic_panel_set_timings,
+ .get_timings = generic_panel_get_timings,
+ .check_timings = generic_panel_check_timings,
+
.driver = {
.name = "generic_panel",
.owner = THIS_MODULE,
#include <linux/gpio.h>
#include <linux/completion.h>
#include <linux/workqueue.h>
+#include <linux/slab.h>
#include <plat/display.h>
#include <linux/regulator/consumer.h>
#include <linux/gpio.h>
#include <linux/err.h>
+#include <linux/slab.h>
#include <plat/display.h>
}
}
+ dss.dsi_clk_source = DSS_SRC_DSS1_ALWON_FCLK;
+ dss.dispc_clk_source = DSS_SRC_DSS1_ALWON_FCLK;
+
dss_save_context();
rev = dss_read_reg(DSS_REVISION);
#define DSS_SUBSYS_NAME "MANAGER"
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/platform_device.h>
#include <linux/spinlock.h>
#include <linux/kobject.h>
#include <linux/platform_device.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <plat/display.h>
#include <plat/cpu.h>
#include <linux/module.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/fb.h>
#include <linux/dma-mapping.h>
#include <linux/vmalloc.h>
#include <linux/kernel.h>
#include <linux/mm.h>
#include <linux/list.h>
+#include <linux/slab.h>
#include <linux/seq_file.h>
#include <linux/bootmem.h>
#include <linux/completion.h>
static u32 omap_vram_def_sdram_size __initdata;
static u32 omap_vram_def_sdram_start __initdata;
-static void __init omap_vram_early_vram(char **p)
+static int __init omap_vram_early_vram(char *p)
{
- omap_vram_def_sdram_size = memparse(*p, p);
- if (**p == ',')
- omap_vram_def_sdram_start = simple_strtoul((*p) + 1, p, 16);
+ omap_vram_def_sdram_size = memparse(p, &p);
+ if (*p == ',')
+ omap_vram_def_sdram_start = simple_strtoul(p + 1, &p, 16);
+ return 0;
}
-__early_param("vram=", omap_vram_early_vram);
+early_param("vram", omap_vram_early_vram);
/*
* Called from map_io. We need to call to this early enough so that we
*/
#include <linux/module.h>
#include <linux/video_output.h>
+#include <linux/slab.h>
#include <linux/err.h>
#include <linux/ctype.h>
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/string.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/fb.h>
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/string.h>
#include <linux/timer.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/fb.h>
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/string.h>
#include <linux/mm.h>
#include <linux/tty.h>
-#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/init.h>
#include <linux/dma-mapping.h>
#include <linux/clk.h>
+#include <linux/gfp.h>
#include <asm/uaccess.h>
#include <mach/gpio.h>
/*
* Map LCD controller registers.
*/
- fbi->reg_base = ioremap_nocache(res->start, res->end - res->start);
+ fbi->reg_base = ioremap_nocache(res->start, resource_size(res));
if (fbi->reg_base == NULL) {
ret = -ENOMEM;
goto failed;
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/platform_device.h>
#include <linux/fb.h>
#include <linux/spinlock_types.h>
#include <linux/spinlock.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <linux/module.h>
#include <linux/platform_device.h>
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <linux/init.h>
-#include <linux/gfp.h>
#include <linux/clk.h>
#include <linux/fb.h>
#include <linux/io.h>
#include <linux/string.h>
#include <linux/mm.h>
#include <linux/tty.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/fb.h>
#include <linux/svga.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/delay.h>
+#include <linux/gfp.h>
#include <linux/pci.h>
#include <linux/fb.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <asm/sh7760fb.h>
#include <linux/interrupt.h>
#include <linux/vmalloc.h>
#include <linux/ioctl.h>
+#include <linux/slab.h>
#include <video/sh_mobile_lcdc.h>
#include <asm/atomic.h>
#include <linux/pci.h>
#include <linux/delay.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <asm/io.h>
#include <linux/uaccess.h>
#include <video/sstfb.h>
#include <linux/module.h>
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/fb.h>
#include <linux/init.h>
#include <linux/of_device.h>
#include <linux/module.h>
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/fb.h>
#include <linux/pci.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/fb.h>
#include <linux/pci.h>
#include <linux/init.h>
static int __devinit e3d_pci_register(struct pci_dev *pdev,
const struct pci_device_id *ent)
{
+ struct device_node *of_node;
+ const char *device_type;
struct fb_info *info;
struct e3d_info *ep;
unsigned int line_length;
int err;
+ of_node = pci_device_to_OF_node(pdev);
+ if (!of_node) {
+ printk(KERN_ERR "e3d: Cannot find OF node of %s\n",
+ pci_name(pdev));
+ return -ENODEV;
+ }
+
+ device_type = of_get_property(of_node, "device_type", NULL);
+ if (!device_type) {
+ printk(KERN_INFO "e3d: Ignoring secondary output device "
+ "at %s\n", pci_name(pdev));
+ return -ENODEV;
+ }
+
err = pci_enable_device(pdev);
if (err < 0) {
printk(KERN_ERR "e3d: Cannot enable PCI device %s\n",
ep->info = info;
ep->pdev = pdev;
spin_lock_init(&ep->lock);
- ep->of_node = pci_device_to_OF_node(pdev);
- if (!ep->of_node) {
- printk(KERN_ERR "e3d: Cannot find OF node of %s\n",
- pci_name(pdev));
- err = -ENODEV;
- goto err_release_fb;
- }
+ ep->of_node = of_node;
/* Read the PCI base register of the frame buffer, which we
* need in order to interpret the RAMDAC_VID_*FB* values in
#include <linux/string.h>
#include <linux/fb.h>
#include <linux/svga.h>
-#include <linux/slab.h>
#include <asm/types.h>
#include <asm/io.h>
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/fb.h>
-#include <linux/slab.h>
#include <asm/types.h>
#include <asm/io.h>
#include "fb_draw.h"
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/string.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/fb.h>
#include <linux/module.h>
#include <linux/pci.h>
#include <linux/selection.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/tc.h>
#include <linux/fb.h>
#include <linux/init.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <video/vga.h>
#include <linux/fb.h>
#include <linux/io.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <video/edid.h>
#include <video/uvesafb.h>
#ifdef CONFIG_X86
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/fb.h>
#include <linux/pci.h>
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/fb.h>
#include <linux/ioport.h>
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/fb.h>
#include <linux/ioport.h>
#include <linux/module.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include <linux/stat.h>
#define _MASTER_FILE
#include <linux/string.h>
#include <linux/mm.h>
#include <linux/tty.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/fb.h>
#include <linux/svga.h>
#include <linux/kernel.h>
#include <linux/mm.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include <linux/vmalloc.h>
#include <asm/io.h>
#include <linux/errno.h>
#include <linux/fb.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/mm.h>
#include <linux/of_platform.h>
#include <linux/io.h>
#include <linux/xilinxfb.h>
+#include <linux/slab.h>
#include <asm/dcr.h>
#define DRIVER_NAME "xilinxfb"
#include <linux/kthread.h>
#include <linux/freezer.h>
#include <linux/delay.h>
+#include <linux/slab.h>
struct virtio_balloon
{
#include <linux/module.h>
#include <linux/list.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/virtio.h>
#include <linux/virtio_config.h>
#include <linux/virtio_ring.h>
#include <linux/virtio_config.h>
#include <linux/device.h>
+#include <linux/slab.h>
/* virtio guest is communicating with a virtual "device" that actually runs on
* a host processor. Memory barriers are used to control SMP effects. */
#include <linux/interrupt.h>
#include <linux/delay.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <linux/vlynq.h>
#include <linux/delay.h>
#include <linux/mfd/core.h>
#include <linux/mfd/ds1wm.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <linux/kernel.h>
#include <linux/mod_devicetable.h>
#include <linux/usb.h>
+#include <linux/slab.h>
#include "../w1_int.h"
#include "../w1.h"
#include <linux/interrupt.h>
#include <linux/platform_device.h>
#include <linux/clk.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/io.h>
#include <linux/module.h>
#include <linux/platform_device.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <linux/err.h>
#include <linux/clk.h>
#include <linux/io.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/w1-gpio.h>
#include "../w1.h"
#include <linux/device.h>
#include <linux/types.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#ifdef CONFIG_W1_SLAVE_DS2433_CRC
#include <linux/crc16.h>
#include <linux/platform_device.h>
#include <linux/mutex.h>
#include <linux/idr.h>
+#include <linux/gfp.h>
#include "../w1.h"
#include "../w1_int.h"
#include <linux/list.h>
#include <linux/delay.h>
#include <linux/kthread.h>
+#include <linux/slab.h>
#include "w1.h"
#include "w1_log.h"
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
+#include <linux/slab.h>
#include <linux/skbuff.h>
#include <linux/netlink.h>
#include <linux/connector.h>
*/
#include <linux/fs.h>
+#include <linux/gfp.h>
#include <linux/io.h>
#include <linux/miscdevice.h>
#include <linux/module.h>
#include <linux/uaccess.h>
#include <linux/io.h>
#include <linux/spinlock.h>
+#include <linux/slab.h>
#define TIMEOUT_MIN 1
#define TIMEOUT_MAX 2
#include <linux/interrupt.h>
#include <linux/ioport.h>
#include <linux/timer.h>
+#include <linux/slab.h>
#include <linux/smp_lock.h>
#include <linux/io.h>
#include <linux/of.h>
#include <linux/io.h>
#include <linux/device.h>
#include <linux/clk.h>
+#include <linux/slab.h>
#define MODULE_NAME "DAVINCI-WDT: "
#include <linux/efi.h>
#include <linux/string.h>
#include <linux/bootmem.h>
-#include <linux/slab.h>
#include <asm/desc.h>
#include <asm/cacheflush.h>
#include <linux/fs.h>
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/module.h>
#include <linux/pci.h>
#include <linux/timer.h>
#include <linux/uaccess.h>
#include <linux/io.h>
#include <linux/device.h>
+#include <linux/slab.h>
#define DEFAULT_HEARTBEAT 60
#define MAX_HEARTBEAT 60
#include <linux/interrupt.h>
#include <linux/platform_device.h>
#include <linux/uaccess.h>
+#include <linux/slab.h>
#include <asm/hardware/arm_twd.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/types.h>
#include <linux/watchdog.h>
#include <linux/bitops.h>
#include <linux/io.h>
#include <linux/uaccess.h>
+#include <linux/slab.h>
#include <mach/hardware.h>
#include <plat/prcm.h>
#include <linux/spinlock.h>
#include <linux/uaccess.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <mach/hardware.h>
#define MODULE_NAME "PNX4008-WDT: "
#include <linux/of_device.h>
#include <linux/io.h>
#include <linux/uaccess.h>
+#include <linux/slab.h>
/* RIO uses the NatSemi Super I/O power management logical device
#include <linux/uaccess.h>
#include <linux/io.h>
#include <linux/cpufreq.h>
+#include <linux/slab.h>
#include <mach/map.h>
#include <linux/miscdevice.h>
#include <linux/mutex.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <linux/watchdog.h>
#include <linux/uaccess.h>
#include <linux/module.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/fs.h>
#include <linux/watchdog.h>
#include <linux/mutex.h>
#include <linux/list.h>
#include <linux/sysdev.h>
+#include <linux/gfp.h>
#include <asm/page.h>
#include <asm/pgalloc.h>
#include <linux/module.h>
#include <linux/string.h>
#include <linux/bootmem.h>
+#include <linux/slab.h>
#include <asm/ptrace.h>
#include <asm/irq.h>
#include <linux/poll.h>
#include <linux/irq.h>
#include <linux/init.h>
-#include <linux/gfp.h>
#include <linux/mutex.h>
#include <linux/cpu.h>
#include <linux/module.h>
#include <linux/sched.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/uaccess.h>
*/
#include <linux/kernel.h>
#include <linux/err.h>
+#include <linux/slab.h>
#include <linux/reboot.h>
#include <linux/sysrq.h>
#include <linux/stop_machine.h>
* published by the Free Software Foundation.
*/
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/kobject.h>
* IN THE SOFTWARE.
*/
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/vmalloc.h>
#include <asm/xen/hypervisor.h>
#include <linux/kthread.h>
#include <linux/mutex.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <asm/page.h>
#include <asm/pgtable.h>
* Authors: Hollis Blanchard <hollisb@us.ibm.com>
*/
-#include <linux/gfp.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include <asm/page.h>
#include <xen/xencomm.h>
#include <xen/interface/xen.h>
#include <linux/init.h>
#include <linux/namei.h>
#include <linux/string.h>
+#include <linux/slab.h>
#include "xenfs.h"
#include "../xenbus/xenbus_comms.h"
#include <linux/jiffies.h>
#include <linux/file.h>
+#include <linux/slab.h>
#include <linux/stat.h>
#include <linux/sched.h>
#include <linux/fs.h>
#include <linux/module.h>
#include <linux/errno.h>
#include <linux/fs.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#include <linux/idr.h>
#include <net/9p/9p.h>
{
int i, n, l, clone, any, access;
u32 uid;
- struct p9_fid *fid;
+ struct p9_fid *fid, *old_fid = NULL;
struct dentry *d, *ds;
struct v9fs_session_info *v9ses;
char **wnames, *uname;
l = min(n - i, P9_MAXWELEM);
fid = p9_client_walk(fid, l, &wnames[i], clone);
if (IS_ERR(fid)) {
+ if (old_fid) {
+ /*
+ * If we fail, clunk fid which are mapping
+ * to path component and not the last component
+ * of the path.
+ */
+ p9_client_clunk(old_fid);
+ }
kfree(wnames);
return fid;
}
-
+ old_fid = fid;
i += l;
clone = 0;
}
#include <linux/sched.h>
#include <linux/parser.h>
#include <linux/idr.h>
+#include <linux/slab.h>
#include <net/9p/9p.h>
#include <net/9p/client.h>
#include <net/9p/transport.h>
list_add(&v9ses->slist, &v9fs_sessionlist);
spin_unlock(&v9fs_sessionlist_lock);
- v9ses->flags = V9FS_PROTO_2000U | V9FS_ACCESS_USER;
+ v9ses->flags = V9FS_ACCESS_USER;
strcpy(v9ses->uname, V9FS_DEFUSER);
strcpy(v9ses->aname, V9FS_DEFANAME);
v9ses->uid = ~0;
goto error;
}
- if (!p9_is_proto_dotu(v9ses->clnt))
- v9ses->flags &= ~V9FS_PROTO_2000U;
+ if (p9_is_proto_dotl(v9ses->clnt))
+ v9ses->flags |= V9FS_PROTO_2000L;
+ else if (p9_is_proto_dotu(v9ses->clnt))
+ v9ses->flags |= V9FS_PROTO_2000U;
v9ses->maxdata = v9ses->clnt->msize - P9_IOHDRSZ;
p9_client_disconnect(v9ses->clnt);
}
+/**
+ * v9fs_session_begin_cancel - Begin terminate of a session
+ * @v9ses: session to terminate
+ *
+ * After this call we don't allow any request other than clunk.
+ */
+
+void v9fs_session_begin_cancel(struct v9fs_session_info *v9ses)
+{
+ P9_DPRINTK(P9_DEBUG_ERROR, "begin cancel session %p\n", v9ses);
+ p9_client_begin_disconnect(v9ses->clnt);
+}
+
extern int v9fs_error_init(void);
static struct kobject *v9fs_kobj;
char *);
void v9fs_session_close(struct v9fs_session_info *v9ses);
void v9fs_session_cancel(struct v9fs_session_info *v9ses);
+void v9fs_session_begin_cancel(struct v9fs_session_info *v9ses);
#define V9FS_MAGIC 0x01021997
#include <linux/namei.h>
#include <linux/idr.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <net/9p/9p.h>
#include <net/9p/client.h>
#include <linux/sched.h>
#include <linux/inet.h>
#include <linux/idr.h>
+#include <linux/slab.h>
#include <net/9p/9p.h>
#include <net/9p/client.h>
rdir = (struct p9_rdir *) fid->rdir;
err = mutex_lock_interruptible(&rdir->mutex);
+ if (err)
+ return err;
while (err == 0) {
if (rdir->tail == rdir->head) {
err = v9fs_file_readn(filp, rdir->buf, NULL,
#include <linux/namei.h>
#include <linux/idr.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <net/9p/9p.h>
#include <net/9p/client.h>
static int v9fs_remove(struct inode *dir, struct dentry *file, int rmdir)
{
+ int retval;
struct inode *file_inode;
struct v9fs_session_info *v9ses;
struct p9_fid *v9fid;
if (IS_ERR(v9fid))
return PTR_ERR(v9fid);
- return p9_client_remove(v9fid);
+ retval = p9_client_remove(v9fid);
+ if (!retval)
+ drop_nlink(file_inode);
+ return retval;
}
static int
P9_DPRINTK(P9_DEBUG_VFS, "dir: %p dentry: (%s) %p nameidata: %p\n",
dir, dentry->d_name.name, dentry, nameidata);
+ if (dentry->d_name.len > NAME_MAX)
+ return ERR_PTR(-ENAMETOOLONG);
+
sb = dir->i_sb;
v9ses = v9fs_inode2v9ses(dir);
dfid = v9fs_fid_lookup(dentry->d_parent);
#include <linux/mount.h>
#include <linux/idr.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <net/9p/9p.h>
#include <net/9p/client.h>
kill_anon_super(s);
+ v9fs_session_cancel(v9ses);
v9fs_session_close(v9ses);
kfree(v9ses);
s->s_fs_info = NULL;
struct v9fs_session_info *v9ses;
v9ses = sb->s_fs_info;
- v9fs_session_cancel(v9ses);
+ v9fs_session_begin_cancel(v9ses);
}
static const struct super_operations v9fs_super_ops = {
source "net/sunrpc/Kconfig"
source "fs/smbfs/Kconfig"
+source "fs/ceph/Kconfig"
source "fs/cifs/Kconfig"
source "fs/ncpfs/Kconfig"
source "fs/coda/Kconfig"
obj-$(CONFIG_BTRFS_FS) += btrfs/
obj-$(CONFIG_GFS2_FS) += gfs2/
obj-$(CONFIG_EXOFS_FS) += exofs/
+obj-$(CONFIG_CEPH_FS) += ceph/
#include <linux/parser.h>
#include <linux/mount.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include <linux/smp_lock.h>
#include <linux/statfs.h>
#include "adfs.h"
* block allocation, deallocation, calculation of free space.
*/
+#include <linux/slab.h>
#include "affs.h"
/* This is, of course, shamelessly stolen from fs/minix */
* (C) 1991 Linus Torvalds - minix filesystem
*/
#include <linux/sched.h>
+#include <linux/gfp.h>
#include "affs.h"
extern const struct inode_operations affs_symlink_inode_operations;
#include <linux/magic.h>
#include <linux/sched.h>
#include <linux/smp_lock.h>
+#include <linux/slab.h>
#include "affs.h"
extern struct timezone sys_tz;
* 2 of the License, or (at your option) any later version.
*/
-#include <linux/slab.h>
#include <linux/sched.h>
#include "internal.h"
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#include <linux/ip.h>
#include "internal.h"
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/fs.h>
#include <linux/pagemap.h>
#include <linux/ctype.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/fs.h>
#include <linux/pagemap.h>
#include <linux/writeback.h>
+#include <linux/gfp.h>
#include "internal.h"
static int afs_readpage(struct file *file, struct page *page);
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#include <linux/circ_buf.h>
#include "internal.h"
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/fs.h>
#include <linux/pagemap.h>
#include <linux/sched.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/fs.h>
#include <linux/pagemap.h>
#include <linux/mount.h>
#include <linux/namei.h>
+#include <linux/gfp.h>
#include "internal.h"
* 2 of the License, or (at your option) any later version.
*/
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/af_rxrpc.h>
#include <rxrpc/packet.h>
if (!permits)
goto out_unlock;
- memcpy(permits->permits, xpermits->permits,
- count * sizeof(struct afs_permit));
+ if (xpermits)
+ memcpy(permits->permits, xpermits->permits,
+ count * sizeof(struct afs_permit));
_debug("key %x access %x",
key_serial(key), vnode->status.caller_access);
* 2 of the License, or (at your option) any later version.
*/
+#include <linux/gfp.h>
#include <linux/init.h>
#include <linux/sched.h>
#include "internal.h"
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/sched.h>
#include "internal.h"
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/fs.h>
#include <linux/sched.h>
#include "internal.h"
#include <linux/file.h>
#include <linux/poll.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/init.h>
#include <linux/fs.h>
#include <linux/mount.h>
#include <linux/capability.h>
#include <linux/errno.h>
#include <linux/stat.h>
+#include <linux/slab.h>
#include <linux/param.h>
#include <linux/time.h>
#include <linux/smp_lock.h>
#include <linux/magic.h>
#include <linux/dcache.h>
#include <linux/uaccess.h>
+#include <linux/slab.h>
#include "autofs_i.h"
#include <linux/capability.h>
#include <linux/errno.h>
#include <linux/stat.h>
+#include <linux/slab.h>
#include <linux/param.h>
#include <linux/time.h>
#include "autofs_i.h"
*/
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/buffer_head.h>
#include <linux/string.h>
#include <linux/fcntl.h>
#include <linux/ptrace.h>
#include <linux/user.h>
-#include <linux/slab.h>
#include <linux/binfmts.h>
#include <linux/personality.h>
#include <linux/init.h>
#include <linux/coredump.h>
+#include <linux/slab.h>
#include <asm/system.h>
#include <asm/uaccess.h>
struct file *file = cprm->file;
mm_segment_t fs;
int has_dumped = 0;
- unsigned long dump_start, dump_size;
+ void __user *dump_start;
+ int dump_size;
struct user dump;
#ifdef __alpha__
-# define START_DATA(u) (u.start_data)
+# define START_DATA(u) ((void __user *)u.start_data)
#else
-# define START_DATA(u) ((u.u_tsize << PAGE_SHIFT) + u.start_code)
+# define START_DATA(u) ((void __user *)((u.u_tsize << PAGE_SHIFT) + \
+ u.start_code))
#endif
-# define START_STACK(u) (u.start_stack)
+# define START_STACK(u) ((void __user *)u.start_stack)
fs = get_fs();
set_fs(KERNEL_DS);
/* make sure we actually have a data and stack area to dump */
set_fs(USER_DS);
- if (!access_ok(VERIFY_READ, (void __user *)START_DATA(dump), dump.u_dsize << PAGE_SHIFT))
+ if (!access_ok(VERIFY_READ, START_DATA(dump), dump.u_dsize << PAGE_SHIFT))
dump.u_dsize = 0;
- if (!access_ok(VERIFY_READ, (void __user *)START_STACK(dump), dump.u_ssize << PAGE_SHIFT))
+ if (!access_ok(VERIFY_READ, START_STACK(dump), dump.u_ssize << PAGE_SHIFT))
dump.u_ssize = 0;
set_fs(KERNEL_DS);
struct vm_area_struct *vma;
size_t size = 0;
- for (vma = current->mm->mmap; vma; vma->vm_next)
+ for (vma = current->mm->mmap; vma; vma = vma->vm_next)
if (maydump(vma, mm_flags))
size += vma->vm_end - vma->vm_start;
return size;
#include <linux/module.h>
#include <linux/string.h>
#include <linux/stat.h>
-#include <linux/slab.h>
#include <linux/binfmts.h>
#include <linux/elf.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/string.h>
#include <linux/stat.h>
-#include <linux/slab.h>
#include <linux/binfmts.h>
#include <linux/init.h>
#include <linux/file.h>
#include <linux/mempool.h>
#include <linux/bio.h>
#include <linux/workqueue.h>
+#include <linux/slab.h>
struct integrity_slab {
struct kmem_cache *slab;
#include <linux/posix_acl_xattr.h>
#include <linux/posix_acl.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include "ctree.h"
#include "btrfs_inode.h"
*/
#include <linux/kthread.h>
+#include <linux/slab.h>
#include <linux/list.h>
#include <linux/spinlock.h>
#include <linux/freezer.h>
#include <linux/swap.h>
#include <linux/writeback.h>
#include <linux/bit_spinlock.h>
-#include <linux/pagevec.h>
+#include <linux/slab.h>
#include "compat.h"
#include "ctree.h"
#include "disk-io.h"
unsigned long nr_pages = 0;
struct extent_map *em;
struct address_space *mapping = inode->i_mapping;
- struct pagevec pvec;
struct extent_map_tree *em_tree;
struct extent_io_tree *tree;
u64 end;
end_index = (i_size_read(inode) - 1) >> PAGE_CACHE_SHIFT;
- pagevec_init(&pvec, 0);
while (last_offset < compressed_end) {
page_index = last_offset >> PAGE_CACHE_SHIFT;
goto next;
}
- page = alloc_page(mapping_gfp_mask(mapping) & ~__GFP_FS);
+ page = __page_cache_alloc(mapping_gfp_mask(mapping) &
+ ~__GFP_FS);
if (!page)
break;
- page->index = page_index;
- /*
- * what we want to do here is call add_to_page_cache_lru,
- * but that isn't exported, so we reproduce it here
- */
- if (add_to_page_cache(page, mapping,
- page->index, GFP_NOFS)) {
+ if (add_to_page_cache_lru(page, mapping, page_index,
+ GFP_NOFS)) {
page_cache_release(page);
goto next;
}
- /* open coding of lru_cache_add, also not exported */
- page_cache_get(page);
- if (!pagevec_add(&pvec, page))
- __pagevec_lru_add_file(&pvec);
-
end = last_offset + PAGE_CACHE_SIZE - 1;
/*
* at this point, we have a locked page in the page cache
next:
last_offset += PAGE_CACHE_SIZE;
}
- if (pagevec_count(&pvec))
- __pagevec_lru_add_file(&pvec);
return 0;
}
*/
#include <linux/sched.h>
+#include <linux/slab.h>
#include "ctree.h"
#include "disk-io.h"
#include "transaction.h"
if (ret > 0 || item_size != btrfs_item_size_nr(leaf, path->slots[0]))
goto err;
+ /* the leaf has changed, it now has room. return now */
+ if (btrfs_leaf_free_space(root, path->nodes[0]) >= ins_len)
+ goto err;
+
if (key.type == BTRFS_EXTENT_DATA_KEY) {
fi = btrfs_item_ptr(leaf, path->slots[0],
struct btrfs_file_extent_item);
#include <linux/completion.h>
#include <linux/backing-dev.h>
#include <linux/wait.h>
+#include <linux/slab.h>
#include <asm/kmap_types.h>
#include "extent_io.h"
#include "extent_map.h"
u64 last_trans_log_full_commit;
u64 open_ioctl_trans;
unsigned long mount_opt;
- u64 max_extent;
u64 max_inline;
u64 alloc_start;
struct btrfs_transaction *running_transaction;
*/
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/sort.h>
#include "ctree.h"
#include "delayed-ref.h"
#include <linux/kthread.h>
#include <linux/freezer.h>
#include <linux/crc32c.h>
+#include <linux/slab.h>
#include "compat.h"
#include "ctree.h"
#include "disk-io.h"
atomic_set(&fs_info->async_submit_draining, 0);
atomic_set(&fs_info->nr_async_bios, 0);
fs_info->sb = sb;
- fs_info->max_extent = (u64)-1;
fs_info->max_inline = 8192 * 1024;
fs_info->metadata_ratio = 0;
csum_root->track_dirty = 1;
- btrfs_read_block_groups(extent_root);
+ ret = btrfs_read_block_groups(extent_root);
+ if (ret) {
+ printk(KERN_ERR "Failed to read block groups: %d\n", ret);
+ goto fail_block_groups;
+ }
fs_info->generation = generation;
fs_info->last_trans_committed = generation;
fs_info->cleaner_kthread = kthread_run(cleaner_kthread, tree_root,
"btrfs-cleaner");
if (IS_ERR(fs_info->cleaner_kthread))
- goto fail_csum_root;
+ goto fail_block_groups;
fs_info->transaction_kthread = kthread_run(transaction_kthread,
tree_root,
filemap_write_and_wait(fs_info->btree_inode->i_mapping);
invalidate_inode_pages2(fs_info->btree_inode->i_mapping);
-fail_csum_root:
+fail_block_groups:
+ btrfs_free_block_groups(fs_info);
free_extent_buffer(csum_root->node);
free_extent_buffer(csum_root->commit_root);
fail_dev_root:
#include <linux/sort.h>
#include <linux/rcupdate.h>
#include <linux/kthread.h>
+#include <linux/slab.h>
#include "compat.h"
#include "hash.h"
#include "ctree.h"
INIT_LIST_HEAD(&found->block_groups);
init_rwsem(&found->groups_sem);
+ init_waitqueue_head(&found->flush_wait);
+ init_waitqueue_head(&found->allocate_wait);
spin_lock_init(&found->lock);
found->flags = flags;
found->total_bytes = total_bytes;
}
spin_unlock(&BTRFS_I(inode)->accounting_lock);
- BTRFS_I(inode)->reserved_extents--;
+ BTRFS_I(inode)->reserved_extents -= num_items;
BUG_ON(BTRFS_I(inode)->reserved_extents < 0);
if (meta_sinfo->bytes_delalloc < num_bytes) {
spin_lock(&info->lock);
- if (!info->flushing) {
+ if (!info->flushing)
info->flushing = 1;
- init_waitqueue_head(&info->flush_wait);
- } else {
+ else
wait = true;
- }
spin_unlock(&info->lock);
if (!info->allocating_chunk) {
info->force_alloc = 1;
info->allocating_chunk = 1;
- init_waitqueue_head(&info->allocate_wait);
} else {
wait = true;
}
return -ENOSPC;
}
- BTRFS_I(inode)->reserved_extents++;
+ BTRFS_I(inode)->reserved_extents += num_items;
check_force_delalloc(meta_sinfo);
spin_unlock(&meta_sinfo->lock);
ins->offset = 0;
space_info = __find_space_info(root->fs_info, data);
+ if (!space_info) {
+ printk(KERN_ERR "No space info for %d\n", data);
+ return -ENOSPC;
+ }
if (orig_root->ref_cows || empty_size)
allowed_chunk_alloc = 1;
next = btrfs_find_tree_block(root, bytenr, blocksize);
if (!next) {
next = btrfs_find_create_tree_block(root, bytenr, blocksize);
+ if (!next)
+ return -ENOMEM;
reada = 1;
}
btrfs_tree_lock(next);
if (ret > 0) {
path->slots[level]++;
continue;
- }
+ } else if (ret < 0)
+ return ret;
level = wc->level;
}
return 0;
}
path->slots[0]++;
}
- ret = -ENOENT;
out:
return ret;
}
#include <linux/slab.h>
#include <linux/bio.h>
#include <linux/mm.h>
-#include <linux/gfp.h>
#include <linux/pagemap.h>
#include <linux/page-flags.h>
#include <linux/module.h>
{
struct bio *bio = NULL;
unsigned page_idx;
- struct pagevec pvec;
unsigned long bio_flags = 0;
- pagevec_init(&pvec, 0);
for (page_idx = 0; page_idx < nr_pages; page_idx++) {
struct page *page = list_entry(pages->prev, struct page, lru);
prefetchw(&page->flags);
list_del(&page->lru);
- /*
- * what we want to do here is call add_to_page_cache_lru,
- * but that isn't exported, so we reproduce it here
- */
- if (!add_to_page_cache(page, mapping,
+ if (!add_to_page_cache_lru(page, mapping,
page->index, GFP_KERNEL)) {
-
- /* open coding of lru_cache_add, also not exported */
- page_cache_get(page);
- if (!pagevec_add(&pvec, page))
- __pagevec_lru_add_file(&pvec);
__extent_read_full_page(tree, page, get_extent,
&bio, 0, &bio_flags);
}
page_cache_release(page);
}
- if (pagevec_count(&pvec))
- __pagevec_lru_add_file(&pvec);
BUG_ON(!list_empty(pages));
if (bio)
submit_one_bio(READ, bio, 0, bio_flags);
#include <linux/err.h>
-#include <linux/gfp.h>
#include <linux/slab.h>
#include <linux/module.h>
#include <linux/spinlock.h>
*/
#include <linux/bio.h>
+#include <linux/slab.h>
#include <linux/pagemap.h>
#include <linux/highmem.h>
#include "ctree.h"
#include <linux/writeback.h>
#include <linux/statfs.h>
#include <linux/compat.h>
+#include <linux/slab.h>
#include "ctree.h"
#include "disk-io.h"
#include "transaction.h"
#include <linux/pagemap.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/math64.h>
#include "ctree.h"
#include "free-space-cache.h"
#include <linux/xattr.h>
#include <linux/posix_acl.h>
#include <linux/falloc.h>
+#include <linux/slab.h>
#include "compat.h"
#include "ctree.h"
#include "disk-io.h"
while (disk_num_bytes > 0) {
unsigned long op;
- cur_alloc_size = min(disk_num_bytes, root->fs_info->max_extent);
+ cur_alloc_size = disk_num_bytes;
ret = btrfs_reserve_extent(trans, root, cur_alloc_size,
root->sectorsize, 0, alloc_hint,
(u64)-1, &ins, 1);
static int btrfs_split_extent_hook(struct inode *inode,
struct extent_state *orig, u64 split)
{
- struct btrfs_root *root = BTRFS_I(inode)->root;
- u64 size;
-
if (!(orig->state & EXTENT_DELALLOC))
return 0;
- size = orig->end - orig->start + 1;
- if (size > root->fs_info->max_extent) {
- u64 num_extents;
- u64 new_size;
-
- new_size = orig->end - split + 1;
- num_extents = div64_u64(size + root->fs_info->max_extent - 1,
- root->fs_info->max_extent);
-
- /*
- * if we break a large extent up then leave oustanding_extents
- * be, since we've already accounted for the large extent.
- */
- if (div64_u64(new_size + root->fs_info->max_extent - 1,
- root->fs_info->max_extent) < num_extents)
- return 0;
- }
-
spin_lock(&BTRFS_I(inode)->accounting_lock);
BTRFS_I(inode)->outstanding_extents++;
spin_unlock(&BTRFS_I(inode)->accounting_lock);
struct extent_state *new,
struct extent_state *other)
{
- struct btrfs_root *root = BTRFS_I(inode)->root;
- u64 new_size, old_size;
- u64 num_extents;
-
/* not delalloc, ignore it */
if (!(other->state & EXTENT_DELALLOC))
return 0;
- old_size = other->end - other->start + 1;
- if (new->start < other->start)
- new_size = other->end - new->start + 1;
- else
- new_size = new->end - other->start + 1;
-
- /* we're not bigger than the max, unreserve the space and go */
- if (new_size <= root->fs_info->max_extent) {
- spin_lock(&BTRFS_I(inode)->accounting_lock);
- BTRFS_I(inode)->outstanding_extents--;
- spin_unlock(&BTRFS_I(inode)->accounting_lock);
- return 0;
- }
-
- /*
- * If we grew by another max_extent, just return, we want to keep that
- * reserved amount.
- */
- num_extents = div64_u64(old_size + root->fs_info->max_extent - 1,
- root->fs_info->max_extent);
- if (div64_u64(new_size + root->fs_info->max_extent - 1,
- root->fs_info->max_extent) > num_extents)
- return 0;
-
spin_lock(&BTRFS_I(inode)->accounting_lock);
BTRFS_I(inode)->outstanding_extents--;
spin_unlock(&BTRFS_I(inode)->accounting_lock);
BTRFS_I(inode)->outstanding_extents++;
spin_unlock(&BTRFS_I(inode)->accounting_lock);
btrfs_delalloc_reserve_space(root, inode, end - start + 1);
+
spin_lock(&root->fs_info->delalloc_lock);
BTRFS_I(inode)->delalloc_bytes += end - start + 1;
root->fs_info->delalloc_bytes += end - start + 1;
if (bits & EXTENT_DO_ACCOUNTING) {
spin_lock(&BTRFS_I(inode)->accounting_lock);
+ WARN_ON(!BTRFS_I(inode)->outstanding_extents);
BTRFS_I(inode)->outstanding_extents--;
spin_unlock(&BTRFS_I(inode)->accounting_lock);
btrfs_unreserve_metadata_for_delalloc(root, inode, 1);
void btrfs_drop_inode(struct inode *inode)
{
struct btrfs_root *root = BTRFS_I(inode)->root;
-
if (inode->i_nlink > 0 && btrfs_root_refs(&root->root_item) == 0)
generic_delete_inode(inode);
else
struct btrfs_trans_handle *trans;
struct btrfs_root *root = BTRFS_I(inode)->root;
struct btrfs_key ins;
- u64 alloc_size;
u64 cur_offset = start;
u64 num_bytes = end - start;
int ret = 0;
u64 i_size;
while (num_bytes > 0) {
- alloc_size = min(num_bytes, root->fs_info->max_extent);
-
trans = btrfs_start_transaction(root, 1);
- ret = btrfs_reserve_extent(trans, root, alloc_size,
+ ret = btrfs_reserve_extent(trans, root, num_bytes,
root->sectorsize, 0, alloc_hint,
(u64)-1, &ins, 1);
if (ret) {
#include <linux/security.h>
#include <linux/xattr.h>
#include <linux/vmalloc.h>
+#include <linux/slab.h>
#include "compat.h"
#include "ctree.h"
#include "disk-io.h"
#include "print-tree.h"
#include "volumes.h"
#include "locking.h"
-#include "ctree.h"
/* Mask out flags that are inappropriate for the given type of inode. */
static inline __u32 btrfs_mask_flags(umode_t mode, __u32 flags)
em = btrfs_get_extent(inode, NULL, 0, start, len, 0);
unlock_extent(io_tree, start, start + len - 1, GFP_NOFS);
- if (!em)
+ if (IS_ERR(em))
return 0;
}
return -EPERM;
args = kmalloc(sizeof(*args), GFP_KERNEL);
+ if (!args)
+ return -ENOMEM;
+
if (copy_from_user(args, argp, sizeof(*args))) {
kfree(args);
return -EFAULT;
sizeof(*range))) {
ret = -EFAULT;
kfree(range);
+ goto out;
}
/* compression requires us to start the IO */
if ((range->flags & BTRFS_DEFRAG_RANGE_COMPRESS)) {
* Boston, MA 021110-1307, USA.
*/
#include <linux/sched.h>
-#include <linux/gfp.h>
#include <linux/pagemap.h>
#include <linux/spinlock.h>
#include <linux/page-flags.h>
* Boston, MA 021110-1307, USA.
*/
-#include <linux/gfp.h>
#include <linux/slab.h>
#include <linux/blkdev.h>
#include <linux/writeback.h>
struct btrfs_ordered_extent *entry)
{
struct btrfs_ordered_inode_tree *tree;
+ struct btrfs_root *root = BTRFS_I(inode)->root;
struct rb_node *node;
tree = &BTRFS_I(inode)->ordered_tree;
set_bit(BTRFS_ORDERED_COMPLETE, &entry->flags);
spin_lock(&BTRFS_I(inode)->accounting_lock);
+ WARN_ON(!BTRFS_I(inode)->outstanding_extents);
BTRFS_I(inode)->outstanding_extents--;
spin_unlock(&BTRFS_I(inode)->accounting_lock);
btrfs_unreserve_metadata_for_delalloc(BTRFS_I(inode)->root,
inode, 1);
- spin_lock(&BTRFS_I(inode)->root->fs_info->ordered_extent_lock);
+ spin_lock(&root->fs_info->ordered_extent_lock);
list_del_init(&entry->root_extent_list);
/*
!mapping_tagged(inode->i_mapping, PAGECACHE_TAG_DIRTY)) {
list_del_init(&BTRFS_I(inode)->ordered_operations);
}
- spin_unlock(&BTRFS_I(inode)->root->fs_info->ordered_extent_lock);
+ spin_unlock(&root->fs_info->ordered_extent_lock);
return 0;
}
*/
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/sort.h>
#include "ctree.h"
#include "ref-cache.h"
#include <linux/writeback.h>
#include <linux/blkdev.h>
#include <linux/rbtree.h>
+#include <linux/slab.h>
#include "ctree.h"
#include "disk-io.h"
#include "transaction.h"
#include <linux/namei.h>
#include <linux/miscdevice.h>
#include <linux/magic.h>
+#include <linux/slab.h>
#include "compat.h"
#include "ctree.h"
#include "disk-io.h"
enum {
Opt_degraded, Opt_subvol, Opt_subvolid, Opt_device, Opt_nodatasum,
- Opt_nodatacow, Opt_max_extent, Opt_max_inline, Opt_alloc_start,
- Opt_nobarrier, Opt_ssd, Opt_nossd, Opt_ssd_spread, Opt_thread_pool,
- Opt_noacl, Opt_compress, Opt_compress_force, Opt_notreelog, Opt_ratio,
- Opt_flushoncommit,
+ Opt_nodatacow, Opt_max_inline, Opt_alloc_start, Opt_nobarrier, Opt_ssd,
+ Opt_nossd, Opt_ssd_spread, Opt_thread_pool, Opt_noacl, Opt_compress,
+ Opt_compress_force, Opt_notreelog, Opt_ratio, Opt_flushoncommit,
Opt_discard, Opt_err,
};
{Opt_nodatasum, "nodatasum"},
{Opt_nodatacow, "nodatacow"},
{Opt_nobarrier, "nobarrier"},
- {Opt_max_extent, "max_extent=%s"},
{Opt_max_inline, "max_inline=%s"},
{Opt_alloc_start, "alloc_start=%s"},
{Opt_thread_pool, "thread_pool=%d"},
info->thread_pool_size);
}
break;
- case Opt_max_extent:
- num = match_strdup(&args[0]);
- if (num) {
- info->max_extent = memparse(num, NULL);
- kfree(num);
-
- info->max_extent = max_t(u64,
- info->max_extent, root->sectorsize);
- printk(KERN_INFO "btrfs: max_extent at %llu\n",
- (unsigned long long)info->max_extent);
- }
- break;
case Opt_max_inline:
num = match_strdup(&args[0]);
if (num) {
seq_puts(seq, ",nodatacow");
if (btrfs_test_opt(root, NOBARRIER))
seq_puts(seq, ",nobarrier");
- if (info->max_extent != (u64)-1)
- seq_printf(seq, ",max_extent=%llu",
- (unsigned long long)info->max_extent);
if (info->max_inline != 8192 * 1024)
seq_printf(seq, ",max_inline=%llu",
(unsigned long long)info->max_inline);
*/
#include <linux/fs.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#include <linux/writeback.h>
#include <linux/pagemap.h>
while (1) {
prepare_to_wait(&root->fs_info->transaction_wait, &wait,
TASK_UNINTERRUPTIBLE);
- if (cur_trans->blocked) {
- mutex_unlock(&root->fs_info->trans_mutex);
- schedule();
- mutex_lock(&root->fs_info->trans_mutex);
- finish_wait(&root->fs_info->transaction_wait,
- &wait);
- } else {
- finish_wait(&root->fs_info->transaction_wait,
- &wait);
+ if (!cur_trans->blocked)
break;
- }
+ mutex_unlock(&root->fs_info->trans_mutex);
+ schedule();
+ mutex_lock(&root->fs_info->trans_mutex);
}
+ finish_wait(&root->fs_info->transaction_wait, &wait);
put_transaction(cur_trans);
}
}
struct btrfs_root_item *new_root_item;
struct btrfs_root *tree_root = fs_info->tree_root;
struct btrfs_root *root = pending->root;
+ struct btrfs_root *parent_root;
+ struct inode *parent_inode;
struct extent_buffer *tmp;
struct extent_buffer *old;
int ret;
u64 objectid;
+ int namelen;
+ u64 index = 0;
+
+ parent_inode = pending->dentry->d_parent->d_inode;
+ parent_root = BTRFS_I(parent_inode)->root;
new_root_item = kmalloc(sizeof(*new_root_item), GFP_NOFS);
if (!new_root_item) {
if (ret)
goto fail;
- record_root_in_trans(trans, root);
- btrfs_set_root_last_snapshot(&root->root_item, trans->transid);
- memcpy(new_root_item, &root->root_item, sizeof(*new_root_item));
-
key.objectid = objectid;
/* record when the snapshot was created in key.offset */
key.offset = trans->transid;
btrfs_set_key_type(&key, BTRFS_ROOT_ITEM_KEY);
- old = btrfs_lock_root_node(root);
- btrfs_cow_block(trans, root, old, NULL, 0, &old);
- btrfs_set_lock_blocking(old);
-
- btrfs_copy_root(trans, root, old, &tmp, objectid);
- btrfs_tree_unlock(old);
- free_extent_buffer(old);
-
- btrfs_set_root_node(new_root_item, tmp);
- ret = btrfs_insert_root(trans, root->fs_info->tree_root, &key,
- new_root_item);
- btrfs_tree_unlock(tmp);
- free_extent_buffer(tmp);
- if (ret)
- goto fail;
-
- key.offset = (u64)-1;
memcpy(&pending->root_key, &key, sizeof(key));
-fail:
- kfree(new_root_item);
- return ret;
-}
-
-static noinline int finish_pending_snapshot(struct btrfs_fs_info *fs_info,
- struct btrfs_pending_snapshot *pending)
-{
- int ret;
- int namelen;
- u64 index = 0;
- struct btrfs_trans_handle *trans;
- struct inode *parent_inode;
- struct btrfs_root *parent_root;
-
- parent_inode = pending->dentry->d_parent->d_inode;
- parent_root = BTRFS_I(parent_inode)->root;
- trans = btrfs_join_transaction(parent_root, 1);
+ pending->root_key.offset = (u64)-1;
+ record_root_in_trans(trans, parent_root);
/*
* insert the directory item
*/
namelen = strlen(pending->name);
ret = btrfs_set_inode_index(parent_inode, &index);
+ BUG_ON(ret);
ret = btrfs_insert_dir_item(trans, parent_root,
pending->name, namelen,
parent_inode->i_ino,
&pending->root_key, BTRFS_FT_DIR, index);
-
- if (ret)
- goto fail;
+ BUG_ON(ret);
btrfs_i_size_write(parent_inode, parent_inode->i_size + namelen * 2);
ret = btrfs_update_inode(trans, parent_root, parent_inode);
BUG_ON(ret);
+ record_root_in_trans(trans, root);
+ btrfs_set_root_last_snapshot(&root->root_item, trans->transid);
+ memcpy(new_root_item, &root->root_item, sizeof(*new_root_item));
+
+ old = btrfs_lock_root_node(root);
+ btrfs_cow_block(trans, root, old, NULL, 0, &old);
+ btrfs_set_lock_blocking(old);
+
+ btrfs_copy_root(trans, root, old, &tmp, objectid);
+ btrfs_tree_unlock(old);
+ free_extent_buffer(old);
+
+ btrfs_set_root_node(new_root_item, tmp);
+ ret = btrfs_insert_root(trans, root->fs_info->tree_root, &key,
+ new_root_item);
+ BUG_ON(ret);
+ btrfs_tree_unlock(tmp);
+ free_extent_buffer(tmp);
+
ret = btrfs_add_root_ref(trans, parent_root->fs_info->tree_root,
pending->root_key.objectid,
parent_root->root_key.objectid,
parent_inode->i_ino, index, pending->name,
namelen);
-
BUG_ON(ret);
fail:
- btrfs_end_transaction(trans, fs_info->fs_root);
+ kfree(new_root_item);
return ret;
}
return 0;
}
-static noinline int finish_pending_snapshots(struct btrfs_trans_handle *trans,
- struct btrfs_fs_info *fs_info)
-{
- struct btrfs_pending_snapshot *pending;
- struct list_head *head = &trans->transaction->pending_snapshots;
- int ret;
-
- while (!list_empty(head)) {
- pending = list_entry(head->next,
- struct btrfs_pending_snapshot, list);
- ret = finish_pending_snapshot(fs_info, pending);
- BUG_ON(ret);
- list_del(&pending->list);
- kfree(pending->name);
- kfree(pending);
- }
- return 0;
-}
-
static void update_super_roots(struct btrfs_root *root)
{
struct btrfs_root_item *root_item;
btrfs_finish_extent_commit(trans, root);
- /* do the directory inserts of any pending snapshot creations */
- finish_pending_snapshots(trans, root->fs_info);
-
mutex_lock(&root->fs_info->trans_mutex);
cur_trans->commit_done = 1;
*/
#include <linux/sched.h>
+#include <linux/slab.h>
#include "ctree.h"
#include "transaction.h"
#include "disk-io.h"
*/
#include <linux/sched.h>
#include <linux/bio.h>
+#include <linux/slab.h>
#include <linux/buffer_head.h>
#include <linux/blkdev.h>
#include <linux/random.h>
min_stripes = 2;
}
if (type & (BTRFS_BLOCK_GROUP_RAID1)) {
- num_stripes = min_t(u64, 2, fs_devices->rw_devices);
- if (num_stripes < 2)
+ if (fs_devices->rw_devices < 2)
return -ENOSPC;
+ num_stripes = 2;
min_stripes = 2;
}
if (type & (BTRFS_BLOCK_GROUP_RAID10)) {
do_div(calc_size, stripe_len);
calc_size *= stripe_len;
}
+
/* we don't want tiny stripes */
- calc_size = max_t(u64, min_stripe_size, calc_size);
+ if (!looped)
+ calc_size = max_t(u64, min_stripe_size, calc_size);
do_div(calc_size, stripe_len);
calc_size *= stripe_len;
key.type = 0;
again:
ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
+ if (ret < 0)
+ goto error;
while (1) {
leaf = path->nodes[0];
slot = path->slots[0];
* 2 of the Licence, or (at your option) any later version.
*/
+#include <linux/slab.h>
#include <linux/mount.h>
#include <linux/buffer_head.h>
#include "internal.h"
#include <linux/mount.h>
#include <linux/namei.h>
#include <linux/security.h>
+#include <linux/slab.h>
#include "internal.h"
#define CACHEFILES_KEYBUF_SIZE 512
*/
#include <linux/mount.h>
+#include <linux/slab.h>
#include <linux/file.h>
#include "internal.h"
#include <linux/fsnotify.h>
#include <linux/quotaops.h>
#include <linux/xattr.h>
+#include <linux/slab.h>
#include "internal.h"
static const char cachefiles_xattr_cache[] =
--- /dev/null
+config CEPH_FS
+ tristate "Ceph distributed file system (EXPERIMENTAL)"
+ depends on INET && EXPERIMENTAL
+ select LIBCRC32C
+ select CONFIG_CRYPTO_AES
+ help
+ Choose Y or M here to include support for mounting the
+ experimental Ceph distributed file system. Ceph is an extremely
+ scalable file system designed to provide high performance,
+ reliable access to petabytes of storage.
+
+ More information at http://ceph.newdream.net/.
+
+ If unsure, say N.
+
+config CEPH_FS_PRETTYDEBUG
+ bool "Include file:line in ceph debug output"
+ depends on CEPH_FS
+ default n
+ help
+ If you say Y here, debug output will include a filename and
+ line to aid debugging. This icnreases kernel size and slows
+ execution slightly when debug call sites are enabled (e.g.,
+ via CONFIG_DYNAMIC_DEBUG).
+
+ If unsure, say N.
+
--- /dev/null
+#
+# Makefile for CEPH filesystem.
+#
+
+ifneq ($(KERNELRELEASE),)
+
+obj-$(CONFIG_CEPH_FS) += ceph.o
+
+ceph-objs := super.o inode.o dir.o file.o addr.o ioctl.o \
+ export.o caps.o snap.o xattr.o \
+ messenger.o msgpool.o buffer.o pagelist.o \
+ mds_client.o mdsmap.o \
+ mon_client.o \
+ osd_client.o osdmap.o crush/crush.o crush/mapper.o crush/hash.o \
+ debugfs.o \
+ auth.o auth_none.o \
+ crypto.o armor.o \
+ auth_x.o \
+ ceph_fs.o ceph_strings.o ceph_hash.o ceph_frag.o
+
+else
+#Otherwise we were called directly from the command
+# line; invoke the kernel build system.
+
+KERNELDIR ?= /lib/modules/$(shell uname -r)/build
+PWD := $(shell pwd)
+
+default: all
+
+all:
+ $(MAKE) -C $(KERNELDIR) M=$(PWD) CONFIG_CEPH_FS=m modules
+
+modules_install:
+ $(MAKE) -C $(KERNELDIR) M=$(PWD) CONFIG_CEPH_FS=m modules_install
+
+clean:
+ $(MAKE) -C $(KERNELDIR) M=$(PWD) clean
+
+endif
--- /dev/null
+#
+# The following files are shared by (and manually synchronized
+# between) the Ceph userland and kernel client.
+#
+# userland kernel
+src/include/ceph_fs.h fs/ceph/ceph_fs.h
+src/include/ceph_fs.cc fs/ceph/ceph_fs.c
+src/include/msgr.h fs/ceph/msgr.h
+src/include/rados.h fs/ceph/rados.h
+src/include/ceph_strings.cc fs/ceph/ceph_strings.c
+src/include/ceph_frag.h fs/ceph/ceph_frag.h
+src/include/ceph_frag.cc fs/ceph/ceph_frag.c
+src/include/ceph_hash.h fs/ceph/ceph_hash.h
+src/include/ceph_hash.cc fs/ceph/ceph_hash.c
+src/crush/crush.c fs/ceph/crush/crush.c
+src/crush/crush.h fs/ceph/crush/crush.h
+src/crush/mapper.c fs/ceph/crush/mapper.c
+src/crush/mapper.h fs/ceph/crush/mapper.h
+src/crush/hash.h fs/ceph/crush/hash.h
+src/crush/hash.c fs/ceph/crush/hash.c
--- /dev/null
+#include "ceph_debug.h"
+
+#include <linux/backing-dev.h>
+#include <linux/fs.h>
+#include <linux/mm.h>
+#include <linux/pagemap.h>
+#include <linux/writeback.h> /* generic_writepages */
+#include <linux/slab.h>
+#include <linux/pagevec.h>
+#include <linux/task_io_accounting_ops.h>
+
+#include "super.h"
+#include "osd_client.h"
+
+/*
+ * Ceph address space ops.
+ *
+ * There are a few funny things going on here.
+ *
+ * The page->private field is used to reference a struct
+ * ceph_snap_context for _every_ dirty page. This indicates which
+ * snapshot the page was logically dirtied in, and thus which snap
+ * context needs to be associated with the osd write during writeback.
+ *
+ * Similarly, struct ceph_inode_info maintains a set of counters to
+ * count dirty pages on the inode. In the absense of snapshots,
+ * i_wrbuffer_ref == i_wrbuffer_ref_head == the dirty page count.
+ *
+ * When a snapshot is taken (that is, when the client receives
+ * notification that a snapshot was taken), each inode with caps and
+ * with dirty pages (dirty pages implies there is a cap) gets a new
+ * ceph_cap_snap in the i_cap_snaps list (which is sorted in ascending
+ * order, new snaps go to the tail). The i_wrbuffer_ref_head count is
+ * moved to capsnap->dirty. (Unless a sync write is currently in
+ * progress. In that case, the capsnap is said to be "pending", new
+ * writes cannot start, and the capsnap isn't "finalized" until the
+ * write completes (or fails) and a final size/mtime for the inode for
+ * that snap can be settled upon.) i_wrbuffer_ref_head is reset to 0.
+ *
+ * On writeback, we must submit writes to the osd IN SNAP ORDER. So,
+ * we look for the first capsnap in i_cap_snaps and write out pages in
+ * that snap context _only_. Then we move on to the next capsnap,
+ * eventually reaching the "live" or "head" context (i.e., pages that
+ * are not yet snapped) and are writing the most recently dirtied
+ * pages.
+ *
+ * Invalidate and so forth must take care to ensure the dirty page
+ * accounting is preserved.
+ */
+
+#define CONGESTION_ON_THRESH(congestion_kb) (congestion_kb >> (PAGE_SHIFT-10))
+#define CONGESTION_OFF_THRESH(congestion_kb) \
+ (CONGESTION_ON_THRESH(congestion_kb) - \
+ (CONGESTION_ON_THRESH(congestion_kb) >> 2))
+
+
+
+/*
+ * Dirty a page. Optimistically adjust accounting, on the assumption
+ * that we won't race with invalidate. If we do, readjust.
+ */
+static int ceph_set_page_dirty(struct page *page)
+{
+ struct address_space *mapping = page->mapping;
+ struct inode *inode;
+ struct ceph_inode_info *ci;
+ int undo = 0;
+ struct ceph_snap_context *snapc;
+
+ if (unlikely(!mapping))
+ return !TestSetPageDirty(page);
+
+ if (TestSetPageDirty(page)) {
+ dout("%p set_page_dirty %p idx %lu -- already dirty\n",
+ mapping->host, page, page->index);
+ return 0;
+ }
+
+ inode = mapping->host;
+ ci = ceph_inode(inode);
+
+ /*
+ * Note that we're grabbing a snapc ref here without holding
+ * any locks!
+ */
+ snapc = ceph_get_snap_context(ci->i_snap_realm->cached_context);
+
+ /* dirty the head */
+ spin_lock(&inode->i_lock);
+ if (ci->i_wrbuffer_ref_head == 0)
+ ci->i_head_snapc = ceph_get_snap_context(snapc);
+ ++ci->i_wrbuffer_ref_head;
+ if (ci->i_wrbuffer_ref == 0)
+ igrab(inode);
+ ++ci->i_wrbuffer_ref;
+ dout("%p set_page_dirty %p idx %lu head %d/%d -> %d/%d "
+ "snapc %p seq %lld (%d snaps)\n",
+ mapping->host, page, page->index,
+ ci->i_wrbuffer_ref-1, ci->i_wrbuffer_ref_head-1,
+ ci->i_wrbuffer_ref, ci->i_wrbuffer_ref_head,
+ snapc, snapc->seq, snapc->num_snaps);
+ spin_unlock(&inode->i_lock);
+
+ /* now adjust page */
+ spin_lock_irq(&mapping->tree_lock);
+ if (page->mapping) { /* Race with truncate? */
+ WARN_ON_ONCE(!PageUptodate(page));
+
+ if (mapping_cap_account_dirty(mapping)) {
+ __inc_zone_page_state(page, NR_FILE_DIRTY);
+ __inc_bdi_stat(mapping->backing_dev_info,
+ BDI_RECLAIMABLE);
+ task_io_account_write(PAGE_CACHE_SIZE);
+ }
+ radix_tree_tag_set(&mapping->page_tree,
+ page_index(page), PAGECACHE_TAG_DIRTY);
+
+ /*
+ * Reference snap context in page->private. Also set
+ * PagePrivate so that we get invalidatepage callback.
+ */
+ page->private = (unsigned long)snapc;
+ SetPagePrivate(page);
+ } else {
+ dout("ANON set_page_dirty %p (raced truncate?)\n", page);
+ undo = 1;
+ }
+
+ spin_unlock_irq(&mapping->tree_lock);
+
+ if (undo)
+ /* whoops, we failed to dirty the page */
+ ceph_put_wrbuffer_cap_refs(ci, 1, snapc);
+
+ __mark_inode_dirty(mapping->host, I_DIRTY_PAGES);
+
+ BUG_ON(!PageDirty(page));
+ return 1;
+}
+
+/*
+ * If we are truncating the full page (i.e. offset == 0), adjust the
+ * dirty page counters appropriately. Only called if there is private
+ * data on the page.
+ */
+static void ceph_invalidatepage(struct page *page, unsigned long offset)
+{
+ struct inode *inode;
+ struct ceph_inode_info *ci;
+ struct ceph_snap_context *snapc = (void *)page->private;
+
+ BUG_ON(!PageLocked(page));
+ BUG_ON(!page->private);
+ BUG_ON(!PagePrivate(page));
+ BUG_ON(!page->mapping);
+
+ inode = page->mapping->host;
+
+ /*
+ * We can get non-dirty pages here due to races between
+ * set_page_dirty and truncate_complete_page; just spit out a
+ * warning, in case we end up with accounting problems later.
+ */
+ if (!PageDirty(page))
+ pr_err("%p invalidatepage %p page not dirty\n", inode, page);
+
+ if (offset == 0)
+ ClearPageChecked(page);
+
+ ci = ceph_inode(inode);
+ if (offset == 0) {
+ dout("%p invalidatepage %p idx %lu full dirty page %lu\n",
+ inode, page, page->index, offset);
+ ceph_put_wrbuffer_cap_refs(ci, 1, snapc);
+ ceph_put_snap_context(snapc);
+ page->private = 0;
+ ClearPagePrivate(page);
+ } else {
+ dout("%p invalidatepage %p idx %lu partial dirty page\n",
+ inode, page, page->index);
+ }
+}
+
+/* just a sanity check */
+static int ceph_releasepage(struct page *page, gfp_t g)
+{
+ struct inode *inode = page->mapping ? page->mapping->host : NULL;
+ dout("%p releasepage %p idx %lu\n", inode, page, page->index);
+ WARN_ON(PageDirty(page));
+ WARN_ON(page->private);
+ WARN_ON(PagePrivate(page));
+ return 0;
+}
+
+/*
+ * read a single page, without unlocking it.
+ */
+static int readpage_nounlock(struct file *filp, struct page *page)
+{
+ struct inode *inode = filp->f_dentry->d_inode;
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ struct ceph_osd_client *osdc = &ceph_inode_to_client(inode)->osdc;
+ int err = 0;
+ u64 len = PAGE_CACHE_SIZE;
+
+ dout("readpage inode %p file %p page %p index %lu\n",
+ inode, filp, page, page->index);
+ err = ceph_osdc_readpages(osdc, ceph_vino(inode), &ci->i_layout,
+ page->index << PAGE_CACHE_SHIFT, &len,
+ ci->i_truncate_seq, ci->i_truncate_size,
+ &page, 1);
+ if (err == -ENOENT)
+ err = 0;
+ if (err < 0) {
+ SetPageError(page);
+ goto out;
+ } else if (err < PAGE_CACHE_SIZE) {
+ /* zero fill remainder of page */
+ zero_user_segment(page, err, PAGE_CACHE_SIZE);
+ }
+ SetPageUptodate(page);
+
+out:
+ return err < 0 ? err : 0;
+}
+
+static int ceph_readpage(struct file *filp, struct page *page)
+{
+ int r = readpage_nounlock(filp, page);
+ unlock_page(page);
+ return r;
+}
+
+/*
+ * Build a vector of contiguous pages from the provided page list.
+ */
+static struct page **page_vector_from_list(struct list_head *page_list,
+ unsigned *nr_pages)
+{
+ struct page **pages;
+ struct page *page;
+ int next_index, contig_pages = 0;
+
+ /* build page vector */
+ pages = kmalloc(sizeof(*pages) * *nr_pages, GFP_NOFS);
+ if (!pages)
+ return ERR_PTR(-ENOMEM);
+
+ BUG_ON(list_empty(page_list));
+ next_index = list_entry(page_list->prev, struct page, lru)->index;
+ list_for_each_entry_reverse(page, page_list, lru) {
+ if (page->index == next_index) {
+ dout("readpages page %d %p\n", contig_pages, page);
+ pages[contig_pages] = page;
+ contig_pages++;
+ next_index++;
+ } else {
+ break;
+ }
+ }
+ *nr_pages = contig_pages;
+ return pages;
+}
+
+/*
+ * Read multiple pages. Leave pages we don't read + unlock in page_list;
+ * the caller (VM) cleans them up.
+ */
+static int ceph_readpages(struct file *file, struct address_space *mapping,
+ struct list_head *page_list, unsigned nr_pages)
+{
+ struct inode *inode = file->f_dentry->d_inode;
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ struct ceph_osd_client *osdc = &ceph_inode_to_client(inode)->osdc;
+ int rc = 0;
+ struct page **pages;
+ struct pagevec pvec;
+ loff_t offset;
+ u64 len;
+
+ dout("readpages %p file %p nr_pages %d\n",
+ inode, file, nr_pages);
+
+ pages = page_vector_from_list(page_list, &nr_pages);
+ if (IS_ERR(pages))
+ return PTR_ERR(pages);
+
+ /* guess read extent */
+ offset = pages[0]->index << PAGE_CACHE_SHIFT;
+ len = nr_pages << PAGE_CACHE_SHIFT;
+ rc = ceph_osdc_readpages(osdc, ceph_vino(inode), &ci->i_layout,
+ offset, &len,
+ ci->i_truncate_seq, ci->i_truncate_size,
+ pages, nr_pages);
+ if (rc == -ENOENT)
+ rc = 0;
+ if (rc < 0)
+ goto out;
+
+ /* set uptodate and add to lru in pagevec-sized chunks */
+ pagevec_init(&pvec, 0);
+ for (; !list_empty(page_list) && len > 0;
+ rc -= PAGE_CACHE_SIZE, len -= PAGE_CACHE_SIZE) {
+ struct page *page =
+ list_entry(page_list->prev, struct page, lru);
+
+ list_del(&page->lru);
+
+ if (rc < (int)PAGE_CACHE_SIZE) {
+ /* zero (remainder of) page */
+ int s = rc < 0 ? 0 : rc;
+ zero_user_segment(page, s, PAGE_CACHE_SIZE);
+ }
+
+ if (add_to_page_cache(page, mapping, page->index, GFP_NOFS)) {
+ page_cache_release(page);
+ dout("readpages %p add_to_page_cache failed %p\n",
+ inode, page);
+ continue;
+ }
+ dout("readpages %p adding %p idx %lu\n", inode, page,
+ page->index);
+ flush_dcache_page(page);
+ SetPageUptodate(page);
+ unlock_page(page);
+ if (pagevec_add(&pvec, page) == 0)
+ pagevec_lru_add_file(&pvec); /* add to lru */
+ }
+ pagevec_lru_add_file(&pvec);
+ rc = 0;
+
+out:
+ kfree(pages);
+ return rc;
+}
+
+/*
+ * Get ref for the oldest snapc for an inode with dirty data... that is, the
+ * only snap context we are allowed to write back.
+ *
+ * Caller holds i_lock.
+ */
+static struct ceph_snap_context *__get_oldest_context(struct inode *inode,
+ u64 *snap_size)
+{
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ struct ceph_snap_context *snapc = NULL;
+ struct ceph_cap_snap *capsnap = NULL;
+
+ list_for_each_entry(capsnap, &ci->i_cap_snaps, ci_item) {
+ dout(" cap_snap %p snapc %p has %d dirty pages\n", capsnap,
+ capsnap->context, capsnap->dirty_pages);
+ if (capsnap->dirty_pages) {
+ snapc = ceph_get_snap_context(capsnap->context);
+ if (snap_size)
+ *snap_size = capsnap->size;
+ break;
+ }
+ }
+ if (!snapc && ci->i_snap_realm) {
+ snapc = ceph_get_snap_context(ci->i_snap_realm->cached_context);
+ dout(" head snapc %p has %d dirty pages\n",
+ snapc, ci->i_wrbuffer_ref_head);
+ }
+ return snapc;
+}
+
+static struct ceph_snap_context *get_oldest_context(struct inode *inode,
+ u64 *snap_size)
+{
+ struct ceph_snap_context *snapc = NULL;
+
+ spin_lock(&inode->i_lock);
+ snapc = __get_oldest_context(inode, snap_size);
+ spin_unlock(&inode->i_lock);
+ return snapc;
+}
+
+/*
+ * Write a single page, but leave the page locked.
+ *
+ * If we get a write error, set the page error bit, but still adjust the
+ * dirty page accounting (i.e., page is no longer dirty).
+ */
+static int writepage_nounlock(struct page *page, struct writeback_control *wbc)
+{
+ struct inode *inode;
+ struct ceph_inode_info *ci;
+ struct ceph_client *client;
+ struct ceph_osd_client *osdc;
+ loff_t page_off = page->index << PAGE_CACHE_SHIFT;
+ int len = PAGE_CACHE_SIZE;
+ loff_t i_size;
+ int err = 0;
+ struct ceph_snap_context *snapc;
+ u64 snap_size = 0;
+ long writeback_stat;
+
+ dout("writepage %p idx %lu\n", page, page->index);
+
+ if (!page->mapping || !page->mapping->host) {
+ dout("writepage %p - no mapping\n", page);
+ return -EFAULT;
+ }
+ inode = page->mapping->host;
+ ci = ceph_inode(inode);
+ client = ceph_inode_to_client(inode);
+ osdc = &client->osdc;
+
+ /* verify this is a writeable snap context */
+ snapc = (void *)page->private;
+ if (snapc == NULL) {
+ dout("writepage %p page %p not dirty?\n", inode, page);
+ goto out;
+ }
+ if (snapc != get_oldest_context(inode, &snap_size)) {
+ dout("writepage %p page %p snapc %p not writeable - noop\n",
+ inode, page, (void *)page->private);
+ /* we should only noop if called by kswapd */
+ WARN_ON((current->flags & PF_MEMALLOC) == 0);
+ goto out;
+ }
+
+ /* is this a partial page at end of file? */
+ if (snap_size)
+ i_size = snap_size;
+ else
+ i_size = i_size_read(inode);
+ if (i_size < page_off + len)
+ len = i_size - page_off;
+
+ dout("writepage %p page %p index %lu on %llu~%u\n",
+ inode, page, page->index, page_off, len);
+
+ writeback_stat = atomic_long_inc_return(&client->writeback_count);
+ if (writeback_stat >
+ CONGESTION_ON_THRESH(client->mount_args->congestion_kb))
+ set_bdi_congested(&client->backing_dev_info, BLK_RW_ASYNC);
+
+ set_page_writeback(page);
+ err = ceph_osdc_writepages(osdc, ceph_vino(inode),
+ &ci->i_layout, snapc,
+ page_off, len,
+ ci->i_truncate_seq, ci->i_truncate_size,
+ &inode->i_mtime,
+ &page, 1, 0, 0, true);
+ if (err < 0) {
+ dout("writepage setting page/mapping error %d %p\n", err, page);
+ SetPageError(page);
+ mapping_set_error(&inode->i_data, err);
+ if (wbc)
+ wbc->pages_skipped++;
+ } else {
+ dout("writepage cleaned page %p\n", page);
+ err = 0; /* vfs expects us to return 0 */
+ }
+ page->private = 0;
+ ClearPagePrivate(page);
+ end_page_writeback(page);
+ ceph_put_wrbuffer_cap_refs(ci, 1, snapc);
+ ceph_put_snap_context(snapc);
+out:
+ return err;
+}
+
+static int ceph_writepage(struct page *page, struct writeback_control *wbc)
+{
+ int err;
+ struct inode *inode = page->mapping->host;
+ BUG_ON(!inode);
+ igrab(inode);
+ err = writepage_nounlock(page, wbc);
+ unlock_page(page);
+ iput(inode);
+ return err;
+}
+
+
+/*
+ * lame release_pages helper. release_pages() isn't exported to
+ * modules.
+ */
+static void ceph_release_pages(struct page **pages, int num)
+{
+ struct pagevec pvec;
+ int i;
+
+ pagevec_init(&pvec, 0);
+ for (i = 0; i < num; i++) {
+ if (pagevec_add(&pvec, pages[i]) == 0)
+ pagevec_release(&pvec);
+ }
+ pagevec_release(&pvec);
+}
+
+
+/*
+ * async writeback completion handler.
+ *
+ * If we get an error, set the mapping error bit, but not the individual
+ * page error bits.
+ */
+static void writepages_finish(struct ceph_osd_request *req,
+ struct ceph_msg *msg)
+{
+ struct inode *inode = req->r_inode;
+ struct ceph_osd_reply_head *replyhead;
+ struct ceph_osd_op *op;
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ unsigned wrote;
+ struct page *page;
+ int i;
+ struct ceph_snap_context *snapc = req->r_snapc;
+ struct address_space *mapping = inode->i_mapping;
+ struct writeback_control *wbc = req->r_wbc;
+ __s32 rc = -EIO;
+ u64 bytes = 0;
+ struct ceph_client *client = ceph_inode_to_client(inode);
+ long writeback_stat;
+ unsigned issued = __ceph_caps_issued(ci, NULL);
+
+ /* parse reply */
+ replyhead = msg->front.iov_base;
+ WARN_ON(le32_to_cpu(replyhead->num_ops) == 0);
+ op = (void *)(replyhead + 1);
+ rc = le32_to_cpu(replyhead->result);
+ bytes = le64_to_cpu(op->extent.length);
+
+ if (rc >= 0) {
+ /*
+ * Assume we wrote the pages we originally sent. The
+ * osd might reply with fewer pages if our writeback
+ * raced with a truncation and was adjusted at the osd,
+ * so don't believe the reply.
+ */
+ wrote = req->r_num_pages;
+ } else {
+ wrote = 0;
+ mapping_set_error(mapping, rc);
+ }
+ dout("writepages_finish %p rc %d bytes %llu wrote %d (pages)\n",
+ inode, rc, bytes, wrote);
+
+ /* clean all pages */
+ for (i = 0; i < req->r_num_pages; i++) {
+ page = req->r_pages[i];
+ BUG_ON(!page);
+ WARN_ON(!PageUptodate(page));
+
+ writeback_stat =
+ atomic_long_dec_return(&client->writeback_count);
+ if (writeback_stat <
+ CONGESTION_OFF_THRESH(client->mount_args->congestion_kb))
+ clear_bdi_congested(&client->backing_dev_info,
+ BLK_RW_ASYNC);
+
+ if (i >= wrote) {
+ dout("inode %p skipping page %p\n", inode, page);
+ wbc->pages_skipped++;
+ }
+ page->private = 0;
+ ClearPagePrivate(page);
+ ceph_put_snap_context(snapc);
+ dout("unlocking %d %p\n", i, page);
+ end_page_writeback(page);
+
+ /*
+ * We lost the cache cap, need to truncate the page before
+ * it is unlocked, otherwise we'd truncate it later in the
+ * page truncation thread, possibly losing some data that
+ * raced its way in
+ */
+ if ((issued & CEPH_CAP_FILE_CACHE) == 0)
+ generic_error_remove_page(inode->i_mapping, page);
+
+ unlock_page(page);
+ }
+ dout("%p wrote+cleaned %d pages\n", inode, wrote);
+ ceph_put_wrbuffer_cap_refs(ci, req->r_num_pages, snapc);
+
+ ceph_release_pages(req->r_pages, req->r_num_pages);
+ if (req->r_pages_from_pool)
+ mempool_free(req->r_pages,
+ ceph_client(inode->i_sb)->wb_pagevec_pool);
+ else
+ kfree(req->r_pages);
+ ceph_osdc_put_request(req);
+}
+
+/*
+ * allocate a page vec, either directly, or if necessary, via a the
+ * mempool. we avoid the mempool if we can because req->r_num_pages
+ * may be less than the maximum write size.
+ */
+static void alloc_page_vec(struct ceph_client *client,
+ struct ceph_osd_request *req)
+{
+ req->r_pages = kmalloc(sizeof(struct page *) * req->r_num_pages,
+ GFP_NOFS);
+ if (!req->r_pages) {
+ req->r_pages = mempool_alloc(client->wb_pagevec_pool, GFP_NOFS);
+ req->r_pages_from_pool = 1;
+ WARN_ON(!req->r_pages);
+ }
+}
+
+/*
+ * initiate async writeback
+ */
+static int ceph_writepages_start(struct address_space *mapping,
+ struct writeback_control *wbc)
+{
+ struct inode *inode = mapping->host;
+ struct backing_dev_info *bdi = mapping->backing_dev_info;
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ struct ceph_client *client;
+ pgoff_t index, start, end;
+ int range_whole = 0;
+ int should_loop = 1;
+ pgoff_t max_pages = 0, max_pages_ever = 0;
+ struct ceph_snap_context *snapc = NULL, *last_snapc = NULL;
+ struct pagevec pvec;
+ int done = 0;
+ int rc = 0;
+ unsigned wsize = 1 << inode->i_blkbits;
+ struct ceph_osd_request *req = NULL;
+ int do_sync;
+ u64 snap_size = 0;
+
+ /*
+ * Include a 'sync' in the OSD request if this is a data
+ * integrity write (e.g., O_SYNC write or fsync()), or if our
+ * cap is being revoked.
+ */
+ do_sync = wbc->sync_mode == WB_SYNC_ALL;
+ if (ceph_caps_revoking(ci, CEPH_CAP_FILE_BUFFER))
+ do_sync = 1;
+ dout("writepages_start %p dosync=%d (mode=%s)\n",
+ inode, do_sync,
+ wbc->sync_mode == WB_SYNC_NONE ? "NONE" :
+ (wbc->sync_mode == WB_SYNC_ALL ? "ALL" : "HOLD"));
+
+ client = ceph_inode_to_client(inode);
+ if (client->mount_state == CEPH_MOUNT_SHUTDOWN) {
+ pr_warning("writepage_start %p on forced umount\n", inode);
+ return -EIO; /* we're in a forced umount, don't write! */
+ }
+ if (client->mount_args->wsize && client->mount_args->wsize < wsize)
+ wsize = client->mount_args->wsize;
+ if (wsize < PAGE_CACHE_SIZE)
+ wsize = PAGE_CACHE_SIZE;
+ max_pages_ever = wsize >> PAGE_CACHE_SHIFT;
+
+ pagevec_init(&pvec, 0);
+
+ /* ?? */
+ if (wbc->nonblocking && bdi_write_congested(bdi)) {
+ dout(" writepages congested\n");
+ wbc->encountered_congestion = 1;
+ goto out_final;
+ }
+
+ /* where to start/end? */
+ if (wbc->range_cyclic) {
+ start = mapping->writeback_index; /* Start from prev offset */
+ end = -1;
+ dout(" cyclic, start at %lu\n", start);
+ } else {
+ start = wbc->range_start >> PAGE_CACHE_SHIFT;
+ end = wbc->range_end >> PAGE_CACHE_SHIFT;
+ if (wbc->range_start == 0 && wbc->range_end == LLONG_MAX)
+ range_whole = 1;
+ should_loop = 0;
+ dout(" not cyclic, %lu to %lu\n", start, end);
+ }
+ index = start;
+
+retry:
+ /* find oldest snap context with dirty data */
+ ceph_put_snap_context(snapc);
+ snapc = get_oldest_context(inode, &snap_size);
+ if (!snapc) {
+ /* hmm, why does writepages get called when there
+ is no dirty data? */
+ dout(" no snap context with dirty data?\n");
+ goto out;
+ }
+ dout(" oldest snapc is %p seq %lld (%d snaps)\n",
+ snapc, snapc->seq, snapc->num_snaps);
+ if (last_snapc && snapc != last_snapc) {
+ /* if we switched to a newer snapc, restart our scan at the
+ * start of the original file range. */
+ dout(" snapc differs from last pass, restarting at %lu\n",
+ index);
+ index = start;
+ }
+ last_snapc = snapc;
+
+ while (!done && index <= end) {
+ unsigned i;
+ int first;
+ pgoff_t next;
+ int pvec_pages, locked_pages;
+ struct page *page;
+ int want;
+ u64 offset, len;
+ struct ceph_osd_request_head *reqhead;
+ struct ceph_osd_op *op;
+ long writeback_stat;
+
+ next = 0;
+ locked_pages = 0;
+ max_pages = max_pages_ever;
+
+get_more_pages:
+ first = -1;
+ want = min(end - index,
+ min((pgoff_t)PAGEVEC_SIZE,
+ max_pages - (pgoff_t)locked_pages) - 1)
+ + 1;
+ pvec_pages = pagevec_lookup_tag(&pvec, mapping, &index,
+ PAGECACHE_TAG_DIRTY,
+ want);
+ dout("pagevec_lookup_tag got %d\n", pvec_pages);
+ if (!pvec_pages && !locked_pages)
+ break;
+ for (i = 0; i < pvec_pages && locked_pages < max_pages; i++) {
+ page = pvec.pages[i];
+ dout("? %p idx %lu\n", page, page->index);
+ if (locked_pages == 0)
+ lock_page(page); /* first page */
+ else if (!trylock_page(page))
+ break;
+
+ /* only dirty pages, or our accounting breaks */
+ if (unlikely(!PageDirty(page)) ||
+ unlikely(page->mapping != mapping)) {
+ dout("!dirty or !mapping %p\n", page);
+ unlock_page(page);
+ break;
+ }
+ if (!wbc->range_cyclic && page->index > end) {
+ dout("end of range %p\n", page);
+ done = 1;
+ unlock_page(page);
+ break;
+ }
+ if (next && (page->index != next)) {
+ dout("not consecutive %p\n", page);
+ unlock_page(page);
+ break;
+ }
+ if (wbc->sync_mode != WB_SYNC_NONE) {
+ dout("waiting on writeback %p\n", page);
+ wait_on_page_writeback(page);
+ }
+ if ((snap_size && page_offset(page) > snap_size) ||
+ (!snap_size &&
+ page_offset(page) > i_size_read(inode))) {
+ dout("%p page eof %llu\n", page, snap_size ?
+ snap_size : i_size_read(inode));
+ done = 1;
+ unlock_page(page);
+ break;
+ }
+ if (PageWriteback(page)) {
+ dout("%p under writeback\n", page);
+ unlock_page(page);
+ break;
+ }
+
+ /* only if matching snap context */
+ if (snapc != (void *)page->private) {
+ dout("page snapc %p != oldest %p\n",
+ (void *)page->private, snapc);
+ unlock_page(page);
+ if (!locked_pages)
+ continue; /* keep looking for snap */
+ break;
+ }
+
+ if (!clear_page_dirty_for_io(page)) {
+ dout("%p !clear_page_dirty_for_io\n", page);
+ unlock_page(page);
+ break;
+ }
+
+ /* ok */
+ if (locked_pages == 0) {
+ /* prepare async write request */
+ offset = page->index << PAGE_CACHE_SHIFT;
+ len = wsize;
+ req = ceph_osdc_new_request(&client->osdc,
+ &ci->i_layout,
+ ceph_vino(inode),
+ offset, &len,
+ CEPH_OSD_OP_WRITE,
+ CEPH_OSD_FLAG_WRITE |
+ CEPH_OSD_FLAG_ONDISK,
+ snapc, do_sync,
+ ci->i_truncate_seq,
+ ci->i_truncate_size,
+ &inode->i_mtime, true, 1);
+ max_pages = req->r_num_pages;
+
+ alloc_page_vec(client, req);
+ req->r_callback = writepages_finish;
+ req->r_inode = inode;
+ req->r_wbc = wbc;
+ }
+
+ /* note position of first page in pvec */
+ if (first < 0)
+ first = i;
+ dout("%p will write page %p idx %lu\n",
+ inode, page, page->index);
+
+ writeback_stat = atomic_long_inc_return(&client->writeback_count);
+ if (writeback_stat > CONGESTION_ON_THRESH(client->mount_args->congestion_kb)) {
+ set_bdi_congested(&client->backing_dev_info, BLK_RW_ASYNC);
+ }
+
+ set_page_writeback(page);
+ req->r_pages[locked_pages] = page;
+ locked_pages++;
+ next = page->index + 1;
+ }
+
+ /* did we get anything? */
+ if (!locked_pages)
+ goto release_pvec_pages;
+ if (i) {
+ int j;
+ BUG_ON(!locked_pages || first < 0);
+
+ if (pvec_pages && i == pvec_pages &&
+ locked_pages < max_pages) {
+ dout("reached end pvec, trying for more\n");
+ pagevec_reinit(&pvec);
+ goto get_more_pages;
+ }
+
+ /* shift unused pages over in the pvec... we
+ * will need to release them below. */
+ for (j = i; j < pvec_pages; j++) {
+ dout(" pvec leftover page %p\n",
+ pvec.pages[j]);
+ pvec.pages[j-i+first] = pvec.pages[j];
+ }
+ pvec.nr -= i-first;
+ }
+
+ /* submit the write */
+ offset = req->r_pages[0]->index << PAGE_CACHE_SHIFT;
+ len = min((snap_size ? snap_size : i_size_read(inode)) - offset,
+ (u64)locked_pages << PAGE_CACHE_SHIFT);
+ dout("writepages got %d pages at %llu~%llu\n",
+ locked_pages, offset, len);
+
+ /* revise final length, page count */
+ req->r_num_pages = locked_pages;
+ reqhead = req->r_request->front.iov_base;
+ op = (void *)(reqhead + 1);
+ op->extent.length = cpu_to_le64(len);
+ op->payload_len = cpu_to_le32(len);
+ req->r_request->hdr.data_len = cpu_to_le32(len);
+
+ ceph_osdc_start_request(&client->osdc, req, true);
+ req = NULL;
+
+ /* continue? */
+ index = next;
+ wbc->nr_to_write -= locked_pages;
+ if (wbc->nr_to_write <= 0)
+ done = 1;
+
+release_pvec_pages:
+ dout("pagevec_release on %d pages (%p)\n", (int)pvec.nr,
+ pvec.nr ? pvec.pages[0] : NULL);
+ pagevec_release(&pvec);
+
+ if (locked_pages && !done)
+ goto retry;
+ }
+
+ if (should_loop && !done) {
+ /* more to do; loop back to beginning of file */
+ dout("writepages looping back to beginning of file\n");
+ should_loop = 0;
+ index = 0;
+ goto retry;
+ }
+
+ if (wbc->range_cyclic || (range_whole && wbc->nr_to_write > 0))
+ mapping->writeback_index = index;
+
+out:
+ if (req)
+ ceph_osdc_put_request(req);
+ if (rc > 0)
+ rc = 0; /* vfs expects us to return 0 */
+ ceph_put_snap_context(snapc);
+ dout("writepages done, rc = %d\n", rc);
+out_final:
+ return rc;
+}
+
+
+
+/*
+ * See if a given @snapc is either writeable, or already written.
+ */
+static int context_is_writeable_or_written(struct inode *inode,
+ struct ceph_snap_context *snapc)
+{
+ struct ceph_snap_context *oldest = get_oldest_context(inode, NULL);
+ return !oldest || snapc->seq <= oldest->seq;
+}
+
+/*
+ * We are only allowed to write into/dirty the page if the page is
+ * clean, or already dirty within the same snap context.
+ *
+ * called with page locked.
+ * return success with page locked,
+ * or any failure (incl -EAGAIN) with page unlocked.
+ */
+static int ceph_update_writeable_page(struct file *file,
+ loff_t pos, unsigned len,
+ struct page *page)
+{
+ struct inode *inode = file->f_dentry->d_inode;
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ struct ceph_mds_client *mdsc = &ceph_inode_to_client(inode)->mdsc;
+ loff_t page_off = pos & PAGE_CACHE_MASK;
+ int pos_in_page = pos & ~PAGE_CACHE_MASK;
+ int end_in_page = pos_in_page + len;
+ loff_t i_size;
+ struct ceph_snap_context *snapc;
+ int r;
+
+retry_locked:
+ /* writepages currently holds page lock, but if we change that later, */
+ wait_on_page_writeback(page);
+
+ /* check snap context */
+ BUG_ON(!ci->i_snap_realm);
+ down_read(&mdsc->snap_rwsem);
+ BUG_ON(!ci->i_snap_realm->cached_context);
+ if (page->private &&
+ (void *)page->private != ci->i_snap_realm->cached_context) {
+ /*
+ * this page is already dirty in another (older) snap
+ * context! is it writeable now?
+ */
+ snapc = get_oldest_context(inode, NULL);
+ up_read(&mdsc->snap_rwsem);
+
+ if (snapc != (void *)page->private) {
+ dout(" page %p snapc %p not current or oldest\n",
+ page, (void *)page->private);
+ /*
+ * queue for writeback, and wait for snapc to
+ * be writeable or written
+ */
+ snapc = ceph_get_snap_context((void *)page->private);
+ unlock_page(page);
+ ceph_queue_writeback(inode);
+ r = wait_event_interruptible(ci->i_cap_wq,
+ context_is_writeable_or_written(inode, snapc));
+ ceph_put_snap_context(snapc);
+ if (r == -ERESTARTSYS)
+ return r;
+ return -EAGAIN;
+ }
+
+ /* yay, writeable, do it now (without dropping page lock) */
+ dout(" page %p snapc %p not current, but oldest\n",
+ page, snapc);
+ if (!clear_page_dirty_for_io(page))
+ goto retry_locked;
+ r = writepage_nounlock(page, NULL);
+ if (r < 0)
+ goto fail_nosnap;
+ goto retry_locked;
+ }
+
+ if (PageUptodate(page)) {
+ dout(" page %p already uptodate\n", page);
+ return 0;
+ }
+
+ /* full page? */
+ if (pos_in_page == 0 && len == PAGE_CACHE_SIZE)
+ return 0;
+
+ /* past end of file? */
+ i_size = inode->i_size; /* caller holds i_mutex */
+
+ if (i_size + len > inode->i_sb->s_maxbytes) {
+ /* file is too big */
+ r = -EINVAL;
+ goto fail;
+ }
+
+ if (page_off >= i_size ||
+ (pos_in_page == 0 && (pos+len) >= i_size &&
+ end_in_page - pos_in_page != PAGE_CACHE_SIZE)) {
+ dout(" zeroing %p 0 - %d and %d - %d\n",
+ page, pos_in_page, end_in_page, (int)PAGE_CACHE_SIZE);
+ zero_user_segments(page,
+ 0, pos_in_page,
+ end_in_page, PAGE_CACHE_SIZE);
+ return 0;
+ }
+
+ /* we need to read it. */
+ up_read(&mdsc->snap_rwsem);
+ r = readpage_nounlock(file, page);
+ if (r < 0)
+ goto fail_nosnap;
+ goto retry_locked;
+
+fail:
+ up_read(&mdsc->snap_rwsem);
+fail_nosnap:
+ unlock_page(page);
+ return r;
+}
+
+/*
+ * We are only allowed to write into/dirty the page if the page is
+ * clean, or already dirty within the same snap context.
+ */
+static int ceph_write_begin(struct file *file, struct address_space *mapping,
+ loff_t pos, unsigned len, unsigned flags,
+ struct page **pagep, void **fsdata)
+{
+ struct inode *inode = file->f_dentry->d_inode;
+ struct page *page;
+ pgoff_t index = pos >> PAGE_CACHE_SHIFT;
+ int r;
+
+ do {
+ /* get a page */
+ page = grab_cache_page_write_begin(mapping, index, 0);
+ if (!page)
+ return -ENOMEM;
+ *pagep = page;
+
+ dout("write_begin file %p inode %p page %p %d~%d\n", file,
+ inode, page, (int)pos, (int)len);
+
+ r = ceph_update_writeable_page(file, pos, len, page);
+ } while (r == -EAGAIN);
+
+ return r;
+}
+
+/*
+ * we don't do anything in here that simple_write_end doesn't do
+ * except adjust dirty page accounting and drop read lock on
+ * mdsc->snap_rwsem.
+ */
+static int ceph_write_end(struct file *file, struct address_space *mapping,
+ loff_t pos, unsigned len, unsigned copied,
+ struct page *page, void *fsdata)
+{
+ struct inode *inode = file->f_dentry->d_inode;
+ struct ceph_client *client = ceph_inode_to_client(inode);
+ struct ceph_mds_client *mdsc = &client->mdsc;
+ unsigned from = pos & (PAGE_CACHE_SIZE - 1);
+ int check_cap = 0;
+
+ dout("write_end file %p inode %p page %p %d~%d (%d)\n", file,
+ inode, page, (int)pos, (int)copied, (int)len);
+
+ /* zero the stale part of the page if we did a short copy */
+ if (copied < len)
+ zero_user_segment(page, from+copied, len);
+
+ /* did file size increase? */
+ /* (no need for i_size_read(); we caller holds i_mutex */
+ if (pos+copied > inode->i_size)
+ check_cap = ceph_inode_set_size(inode, pos+copied);
+
+ if (!PageUptodate(page))
+ SetPageUptodate(page);
+
+ set_page_dirty(page);
+
+ unlock_page(page);
+ up_read(&mdsc->snap_rwsem);
+ page_cache_release(page);
+
+ if (check_cap)
+ ceph_check_caps(ceph_inode(inode), CHECK_CAPS_AUTHONLY, NULL);
+
+ return copied;
+}
+
+/*
+ * we set .direct_IO to indicate direct io is supported, but since we
+ * intercept O_DIRECT reads and writes early, this function should
+ * never get called.
+ */
+static ssize_t ceph_direct_io(int rw, struct kiocb *iocb,
+ const struct iovec *iov,
+ loff_t pos, unsigned long nr_segs)
+{
+ WARN_ON(1);
+ return -EINVAL;
+}
+
+const struct address_space_operations ceph_aops = {
+ .readpage = ceph_readpage,
+ .readpages = ceph_readpages,
+ .writepage = ceph_writepage,
+ .writepages = ceph_writepages_start,
+ .write_begin = ceph_write_begin,
+ .write_end = ceph_write_end,
+ .set_page_dirty = ceph_set_page_dirty,
+ .invalidatepage = ceph_invalidatepage,
+ .releasepage = ceph_releasepage,
+ .direct_IO = ceph_direct_io,
+};
+
+
+/*
+ * vm ops
+ */
+
+/*
+ * Reuse write_begin here for simplicity.
+ */
+static int ceph_page_mkwrite(struct vm_area_struct *vma, struct vm_fault *vmf)
+{
+ struct inode *inode = vma->vm_file->f_dentry->d_inode;
+ struct page *page = vmf->page;
+ struct ceph_mds_client *mdsc = &ceph_inode_to_client(inode)->mdsc;
+ loff_t off = page->index << PAGE_CACHE_SHIFT;
+ loff_t size, len;
+ int ret;
+
+ size = i_size_read(inode);
+ if (off + PAGE_CACHE_SIZE <= size)
+ len = PAGE_CACHE_SIZE;
+ else
+ len = size & ~PAGE_CACHE_MASK;
+
+ dout("page_mkwrite %p %llu~%llu page %p idx %lu\n", inode,
+ off, len, page, page->index);
+
+ lock_page(page);
+
+ ret = VM_FAULT_NOPAGE;
+ if ((off > size) ||
+ (page->mapping != inode->i_mapping))
+ goto out;
+
+ ret = ceph_update_writeable_page(vma->vm_file, off, len, page);
+ if (ret == 0) {
+ /* success. we'll keep the page locked. */
+ set_page_dirty(page);
+ up_read(&mdsc->snap_rwsem);
+ ret = VM_FAULT_LOCKED;
+ } else {
+ if (ret == -ENOMEM)
+ ret = VM_FAULT_OOM;
+ else
+ ret = VM_FAULT_SIGBUS;
+ }
+out:
+ dout("page_mkwrite %p %llu~%llu = %d\n", inode, off, len, ret);
+ if (ret != VM_FAULT_LOCKED)
+ unlock_page(page);
+ return ret;
+}
+
+static struct vm_operations_struct ceph_vmops = {
+ .fault = filemap_fault,
+ .page_mkwrite = ceph_page_mkwrite,
+};
+
+int ceph_mmap(struct file *file, struct vm_area_struct *vma)
+{
+ struct address_space *mapping = file->f_mapping;
+
+ if (!mapping->a_ops->readpage)
+ return -ENOEXEC;
+ file_accessed(file);
+ vma->vm_ops = &ceph_vmops;
+ vma->vm_flags |= VM_CAN_NONLINEAR;
+ return 0;
+}
--- /dev/null
+
+#include <linux/errno.h>
+
+/*
+ * base64 encode/decode.
+ */
+
+const char *pem_key = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
+
+static int encode_bits(int c)
+{
+ return pem_key[c];
+}
+
+static int decode_bits(char c)
+{
+ if (c >= 'A' && c <= 'Z')
+ return c - 'A';
+ if (c >= 'a' && c <= 'z')
+ return c - 'a' + 26;
+ if (c >= '0' && c <= '9')
+ return c - '0' + 52;
+ if (c == '+')
+ return 62;
+ if (c == '/')
+ return 63;
+ if (c == '=')
+ return 0; /* just non-negative, please */
+ return -EINVAL;
+}
+
+int ceph_armor(char *dst, const char *src, const char *end)
+{
+ int olen = 0;
+ int line = 0;
+
+ while (src < end) {
+ unsigned char a, b, c;
+
+ a = *src++;
+ *dst++ = encode_bits(a >> 2);
+ if (src < end) {
+ b = *src++;
+ *dst++ = encode_bits(((a & 3) << 4) | (b >> 4));
+ if (src < end) {
+ c = *src++;
+ *dst++ = encode_bits(((b & 15) << 2) |
+ (c >> 6));
+ *dst++ = encode_bits(c & 63);
+ } else {
+ *dst++ = encode_bits((b & 15) << 2);
+ *dst++ = '=';
+ }
+ } else {
+ *dst++ = encode_bits(((a & 3) << 4));
+ *dst++ = '=';
+ *dst++ = '=';
+ }
+ olen += 4;
+ line += 4;
+ if (line == 64) {
+ line = 0;
+ *(dst++) = '\n';
+ olen++;
+ }
+ }
+ return olen;
+}
+
+int ceph_unarmor(char *dst, const char *src, const char *end)
+{
+ int olen = 0;
+
+ while (src < end) {
+ int a, b, c, d;
+
+ if (src < end && src[0] == '\n')
+ src++;
+ if (src + 4 > end)
+ return -EINVAL;
+ a = decode_bits(src[0]);
+ b = decode_bits(src[1]);
+ c = decode_bits(src[2]);
+ d = decode_bits(src[3]);
+ if (a < 0 || b < 0 || c < 0 || d < 0)
+ return -EINVAL;
+
+ *dst++ = (a << 2) | (b >> 4);
+ if (src[2] == '=')
+ return olen + 1;
+ *dst++ = ((b & 15) << 4) | (c >> 2);
+ if (src[3] == '=')
+ return olen + 2;
+ *dst++ = ((c & 3) << 6) | d;
+ olen += 3;
+ src += 4;
+ }
+ return olen;
+}
--- /dev/null
+#include "ceph_debug.h"
+
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/err.h>
+
+#include "types.h"
+#include "auth_none.h"
+#include "auth_x.h"
+#include "decode.h"
+#include "super.h"
+
+#include "messenger.h"
+
+/*
+ * get protocol handler
+ */
+static u32 supported_protocols[] = {
+ CEPH_AUTH_NONE,
+ CEPH_AUTH_CEPHX
+};
+
+int ceph_auth_init_protocol(struct ceph_auth_client *ac, int protocol)
+{
+ switch (protocol) {
+ case CEPH_AUTH_NONE:
+ return ceph_auth_none_init(ac);
+ case CEPH_AUTH_CEPHX:
+ return ceph_x_init(ac);
+ default:
+ return -ENOENT;
+ }
+}
+
+/*
+ * setup, teardown.
+ */
+struct ceph_auth_client *ceph_auth_init(const char *name, const char *secret)
+{
+ struct ceph_auth_client *ac;
+ int ret;
+
+ dout("auth_init name '%s' secret '%s'\n", name, secret);
+
+ ret = -ENOMEM;
+ ac = kzalloc(sizeof(*ac), GFP_NOFS);
+ if (!ac)
+ goto out;
+
+ ac->negotiating = true;
+ if (name)
+ ac->name = name;
+ else
+ ac->name = CEPH_AUTH_NAME_DEFAULT;
+ dout("auth_init name %s secret %s\n", ac->name, secret);
+ ac->secret = secret;
+ return ac;
+
+out:
+ return ERR_PTR(ret);
+}
+
+void ceph_auth_destroy(struct ceph_auth_client *ac)
+{
+ dout("auth_destroy %p\n", ac);
+ if (ac->ops)
+ ac->ops->destroy(ac);
+ kfree(ac);
+}
+
+/*
+ * Reset occurs when reconnecting to the monitor.
+ */
+void ceph_auth_reset(struct ceph_auth_client *ac)
+{
+ dout("auth_reset %p\n", ac);
+ if (ac->ops && !ac->negotiating)
+ ac->ops->reset(ac);
+ ac->negotiating = true;
+}
+
+int ceph_entity_name_encode(const char *name, void **p, void *end)
+{
+ int len = strlen(name);
+
+ if (*p + 2*sizeof(u32) + len > end)
+ return -ERANGE;
+ ceph_encode_32(p, CEPH_ENTITY_TYPE_CLIENT);
+ ceph_encode_32(p, len);
+ ceph_encode_copy(p, name, len);
+ return 0;
+}
+
+/*
+ * Initiate protocol negotiation with monitor. Include entity name
+ * and list supported protocols.
+ */
+int ceph_auth_build_hello(struct ceph_auth_client *ac, void *buf, size_t len)
+{
+ struct ceph_mon_request_header *monhdr = buf;
+ void *p = monhdr + 1, *end = buf + len, *lenp;
+ int i, num;
+ int ret;
+
+ dout("auth_build_hello\n");
+ monhdr->have_version = 0;
+ monhdr->session_mon = cpu_to_le16(-1);
+ monhdr->session_mon_tid = 0;
+
+ ceph_encode_32(&p, 0); /* no protocol, yet */
+
+ lenp = p;
+ p += sizeof(u32);
+
+ ceph_decode_need(&p, end, 1 + sizeof(u32), bad);
+ ceph_encode_8(&p, 1);
+ num = ARRAY_SIZE(supported_protocols);
+ ceph_encode_32(&p, num);
+ ceph_decode_need(&p, end, num * sizeof(u32), bad);
+ for (i = 0; i < num; i++)
+ ceph_encode_32(&p, supported_protocols[i]);
+
+ ret = ceph_entity_name_encode(ac->name, &p, end);
+ if (ret < 0)
+ return ret;
+ ceph_decode_need(&p, end, sizeof(u64), bad);
+ ceph_encode_64(&p, ac->global_id);
+
+ ceph_encode_32(&lenp, p - lenp - sizeof(u32));
+ return p - buf;
+
+bad:
+ return -ERANGE;
+}
+
+int ceph_build_auth_request(struct ceph_auth_client *ac,
+ void *msg_buf, size_t msg_len)
+{
+ struct ceph_mon_request_header *monhdr = msg_buf;
+ void *p = monhdr + 1;
+ void *end = msg_buf + msg_len;
+ int ret;
+
+ monhdr->have_version = 0;
+ monhdr->session_mon = cpu_to_le16(-1);
+ monhdr->session_mon_tid = 0;
+
+ ceph_encode_32(&p, ac->protocol);
+
+ ret = ac->ops->build_request(ac, p + sizeof(u32), end);
+ if (ret < 0) {
+ pr_err("error %d building request\n", ret);
+ return ret;
+ }
+ dout(" built request %d bytes\n", ret);
+ ceph_encode_32(&p, ret);
+ return p + ret - msg_buf;
+}
+
+/*
+ * Handle auth message from monitor.
+ */
+int ceph_handle_auth_reply(struct ceph_auth_client *ac,
+ void *buf, size_t len,
+ void *reply_buf, size_t reply_len)
+{
+ void *p = buf;
+ void *end = buf + len;
+ int protocol;
+ s32 result;
+ u64 global_id;
+ void *payload, *payload_end;
+ int payload_len;
+ char *result_msg;
+ int result_msg_len;
+ int ret = -EINVAL;
+
+ dout("handle_auth_reply %p %p\n", p, end);
+ ceph_decode_need(&p, end, sizeof(u32) * 3 + sizeof(u64), bad);
+ protocol = ceph_decode_32(&p);
+ result = ceph_decode_32(&p);
+ global_id = ceph_decode_64(&p);
+ payload_len = ceph_decode_32(&p);
+ payload = p;
+ p += payload_len;
+ ceph_decode_need(&p, end, sizeof(u32), bad);
+ result_msg_len = ceph_decode_32(&p);
+ result_msg = p;
+ p += result_msg_len;
+ if (p != end)
+ goto bad;
+
+ dout(" result %d '%.*s' gid %llu len %d\n", result, result_msg_len,
+ result_msg, global_id, payload_len);
+
+ payload_end = payload + payload_len;
+
+ if (global_id && ac->global_id != global_id) {
+ dout(" set global_id %lld -> %lld\n", ac->global_id, global_id);
+ ac->global_id = global_id;
+ }
+
+ if (ac->negotiating) {
+ /* server does not support our protocols? */
+ if (!protocol && result < 0) {
+ ret = result;
+ goto out;
+ }
+ /* set up (new) protocol handler? */
+ if (ac->protocol && ac->protocol != protocol) {
+ ac->ops->destroy(ac);
+ ac->protocol = 0;
+ ac->ops = NULL;
+ }
+ if (ac->protocol != protocol) {
+ ret = ceph_auth_init_protocol(ac, protocol);
+ if (ret) {
+ pr_err("error %d on auth protocol %d init\n",
+ ret, protocol);
+ goto out;
+ }
+ }
+
+ ac->negotiating = false;
+ }
+
+ ret = ac->ops->handle_reply(ac, result, payload, payload_end);
+ if (ret == -EAGAIN) {
+ return ceph_build_auth_request(ac, reply_buf, reply_len);
+ } else if (ret) {
+ pr_err("authentication error %d\n", ret);
+ return ret;
+ }
+ return 0;
+
+bad:
+ pr_err("failed to decode auth msg\n");
+out:
+ return ret;
+}
+
+int ceph_build_auth(struct ceph_auth_client *ac,
+ void *msg_buf, size_t msg_len)
+{
+ if (!ac->protocol)
+ return ceph_auth_build_hello(ac, msg_buf, msg_len);
+ BUG_ON(!ac->ops);
+ if (!ac->ops->is_authenticated(ac))
+ return ceph_build_auth_request(ac, msg_buf, msg_len);
+ return 0;
+}
+
+int ceph_auth_is_authenticated(struct ceph_auth_client *ac)
+{
+ if (!ac->ops)
+ return 0;
+ return ac->ops->is_authenticated(ac);
+}
--- /dev/null
+#ifndef _FS_CEPH_AUTH_H
+#define _FS_CEPH_AUTH_H
+
+#include "types.h"
+#include "buffer.h"
+
+/*
+ * Abstract interface for communicating with the authenticate module.
+ * There is some handshake that takes place between us and the monitor
+ * to acquire the necessary keys. These are used to generate an
+ * 'authorizer' that we use when connecting to a service (mds, osd).
+ */
+
+struct ceph_auth_client;
+struct ceph_authorizer;
+
+struct ceph_auth_client_ops {
+ /*
+ * true if we are authenticated and can connect to
+ * services.
+ */
+ int (*is_authenticated)(struct ceph_auth_client *ac);
+
+ /*
+ * build requests and process replies during monitor
+ * handshake. if handle_reply returns -EAGAIN, we build
+ * another request.
+ */
+ int (*build_request)(struct ceph_auth_client *ac, void *buf, void *end);
+ int (*handle_reply)(struct ceph_auth_client *ac, int result,
+ void *buf, void *end);
+
+ /*
+ * Create authorizer for connecting to a service, and verify
+ * the response to authenticate the service.
+ */
+ int (*create_authorizer)(struct ceph_auth_client *ac, int peer_type,
+ struct ceph_authorizer **a,
+ void **buf, size_t *len,
+ void **reply_buf, size_t *reply_len);
+ int (*verify_authorizer_reply)(struct ceph_auth_client *ac,
+ struct ceph_authorizer *a, size_t len);
+ void (*destroy_authorizer)(struct ceph_auth_client *ac,
+ struct ceph_authorizer *a);
+ void (*invalidate_authorizer)(struct ceph_auth_client *ac,
+ int peer_type);
+
+ /* reset when we (re)connect to a monitor */
+ void (*reset)(struct ceph_auth_client *ac);
+
+ void (*destroy)(struct ceph_auth_client *ac);
+};
+
+struct ceph_auth_client {
+ u32 protocol; /* CEPH_AUTH_* */
+ void *private; /* for use by protocol implementation */
+ const struct ceph_auth_client_ops *ops; /* null iff protocol==0 */
+
+ bool negotiating; /* true if negotiating protocol */
+ const char *name; /* entity name */
+ u64 global_id; /* our unique id in system */
+ const char *secret; /* our secret key */
+ unsigned want_keys; /* which services we want */
+};
+
+extern struct ceph_auth_client *ceph_auth_init(const char *name,
+ const char *secret);
+extern void ceph_auth_destroy(struct ceph_auth_client *ac);
+
+extern void ceph_auth_reset(struct ceph_auth_client *ac);
+
+extern int ceph_auth_build_hello(struct ceph_auth_client *ac,
+ void *buf, size_t len);
+extern int ceph_handle_auth_reply(struct ceph_auth_client *ac,
+ void *buf, size_t len,
+ void *reply_buf, size_t reply_len);
+extern int ceph_entity_name_encode(const char *name, void **p, void *end);
+
+extern int ceph_build_auth(struct ceph_auth_client *ac,
+ void *msg_buf, size_t msg_len);
+
+extern int ceph_auth_is_authenticated(struct ceph_auth_client *ac);
+
+#endif
--- /dev/null
+
+#include "ceph_debug.h"
+
+#include <linux/err.h>
+#include <linux/module.h>
+#include <linux/random.h>
+#include <linux/slab.h>
+
+#include "auth_none.h"
+#include "auth.h"
+#include "decode.h"
+
+static void reset(struct ceph_auth_client *ac)
+{
+ struct ceph_auth_none_info *xi = ac->private;
+
+ xi->starting = true;
+ xi->built_authorizer = false;
+}
+
+static void destroy(struct ceph_auth_client *ac)
+{
+ kfree(ac->private);
+ ac->private = NULL;
+}
+
+static int is_authenticated(struct ceph_auth_client *ac)
+{
+ struct ceph_auth_none_info *xi = ac->private;
+
+ return !xi->starting;
+}
+
+/*
+ * the generic auth code decode the global_id, and we carry no actual
+ * authenticate state, so nothing happens here.
+ */
+static int handle_reply(struct ceph_auth_client *ac, int result,
+ void *buf, void *end)
+{
+ struct ceph_auth_none_info *xi = ac->private;
+
+ xi->starting = false;
+ return result;
+}
+
+/*
+ * build an 'authorizer' with our entity_name and global_id. we can
+ * reuse a single static copy since it is identical for all services
+ * we connect to.
+ */
+static int ceph_auth_none_create_authorizer(
+ struct ceph_auth_client *ac, int peer_type,
+ struct ceph_authorizer **a,
+ void **buf, size_t *len,
+ void **reply_buf, size_t *reply_len)
+{
+ struct ceph_auth_none_info *ai = ac->private;
+ struct ceph_none_authorizer *au = &ai->au;
+ void *p, *end;
+ int ret;
+
+ if (!ai->built_authorizer) {
+ p = au->buf;
+ end = p + sizeof(au->buf);
+ ceph_encode_8(&p, 1);
+ ret = ceph_entity_name_encode(ac->name, &p, end - 8);
+ if (ret < 0)
+ goto bad;
+ ceph_decode_need(&p, end, sizeof(u64), bad2);
+ ceph_encode_64(&p, ac->global_id);
+ au->buf_len = p - (void *)au->buf;
+ ai->built_authorizer = true;
+ dout("built authorizer len %d\n", au->buf_len);
+ }
+
+ *a = (struct ceph_authorizer *)au;
+ *buf = au->buf;
+ *len = au->buf_len;
+ *reply_buf = au->reply_buf;
+ *reply_len = sizeof(au->reply_buf);
+ return 0;
+
+bad2:
+ ret = -ERANGE;
+bad:
+ return ret;
+}
+
+static void ceph_auth_none_destroy_authorizer(struct ceph_auth_client *ac,
+ struct ceph_authorizer *a)
+{
+ /* nothing to do */
+}
+
+static const struct ceph_auth_client_ops ceph_auth_none_ops = {
+ .reset = reset,
+ .destroy = destroy,
+ .is_authenticated = is_authenticated,
+ .handle_reply = handle_reply,
+ .create_authorizer = ceph_auth_none_create_authorizer,
+ .destroy_authorizer = ceph_auth_none_destroy_authorizer,
+};
+
+int ceph_auth_none_init(struct ceph_auth_client *ac)
+{
+ struct ceph_auth_none_info *xi;
+
+ dout("ceph_auth_none_init %p\n", ac);
+ xi = kzalloc(sizeof(*xi), GFP_NOFS);
+ if (!xi)
+ return -ENOMEM;
+
+ xi->starting = true;
+ xi->built_authorizer = false;
+
+ ac->protocol = CEPH_AUTH_NONE;
+ ac->private = xi;
+ ac->ops = &ceph_auth_none_ops;
+ return 0;
+}
+
--- /dev/null
+#ifndef _FS_CEPH_AUTH_NONE_H
+#define _FS_CEPH_AUTH_NONE_H
+
+#include "auth.h"
+
+/*
+ * null security mode.
+ *
+ * we use a single static authorizer that simply encodes our entity name
+ * and global id.
+ */
+
+struct ceph_none_authorizer {
+ char buf[128];
+ int buf_len;
+ char reply_buf[0];
+};
+
+struct ceph_auth_none_info {
+ bool starting;
+ bool built_authorizer;
+ struct ceph_none_authorizer au; /* we only need one; it's static */
+};
+
+extern int ceph_auth_none_init(struct ceph_auth_client *ac);
+
+#endif
+
--- /dev/null
+
+#include "ceph_debug.h"
+
+#include <linux/err.h>
+#include <linux/module.h>
+#include <linux/random.h>
+#include <linux/slab.h>
+
+#include "auth_x.h"
+#include "auth_x_protocol.h"
+#include "crypto.h"
+#include "auth.h"
+#include "decode.h"
+
+struct kmem_cache *ceph_x_ticketbuf_cachep;
+
+#define TEMP_TICKET_BUF_LEN 256
+
+static void ceph_x_validate_tickets(struct ceph_auth_client *ac, int *pneed);
+
+static int ceph_x_is_authenticated(struct ceph_auth_client *ac)
+{
+ struct ceph_x_info *xi = ac->private;
+ int need;
+
+ ceph_x_validate_tickets(ac, &need);
+ dout("ceph_x_is_authenticated want=%d need=%d have=%d\n",
+ ac->want_keys, need, xi->have_keys);
+ return (ac->want_keys & xi->have_keys) == ac->want_keys;
+}
+
+static int ceph_x_encrypt_buflen(int ilen)
+{
+ return sizeof(struct ceph_x_encrypt_header) + ilen + 16 +
+ sizeof(u32);
+}
+
+static int ceph_x_encrypt(struct ceph_crypto_key *secret,
+ void *ibuf, int ilen, void *obuf, size_t olen)
+{
+ struct ceph_x_encrypt_header head = {
+ .struct_v = 1,
+ .magic = cpu_to_le64(CEPHX_ENC_MAGIC)
+ };
+ size_t len = olen - sizeof(u32);
+ int ret;
+
+ ret = ceph_encrypt2(secret, obuf + sizeof(u32), &len,
+ &head, sizeof(head), ibuf, ilen);
+ if (ret)
+ return ret;
+ ceph_encode_32(&obuf, len);
+ return len + sizeof(u32);
+}
+
+static int ceph_x_decrypt(struct ceph_crypto_key *secret,
+ void **p, void *end, void *obuf, size_t olen)
+{
+ struct ceph_x_encrypt_header head;
+ size_t head_len = sizeof(head);
+ int len, ret;
+
+ len = ceph_decode_32(p);
+ if (*p + len > end)
+ return -EINVAL;
+
+ dout("ceph_x_decrypt len %d\n", len);
+ ret = ceph_decrypt2(secret, &head, &head_len, obuf, &olen,
+ *p, len);
+ if (ret)
+ return ret;
+ if (head.struct_v != 1 || le64_to_cpu(head.magic) != CEPHX_ENC_MAGIC)
+ return -EPERM;
+ *p += len;
+ return olen;
+}
+
+/*
+ * get existing (or insert new) ticket handler
+ */
+struct ceph_x_ticket_handler *get_ticket_handler(struct ceph_auth_client *ac,
+ int service)
+{
+ struct ceph_x_ticket_handler *th;
+ struct ceph_x_info *xi = ac->private;
+ struct rb_node *parent = NULL, **p = &xi->ticket_handlers.rb_node;
+
+ while (*p) {
+ parent = *p;
+ th = rb_entry(parent, struct ceph_x_ticket_handler, node);
+ if (service < th->service)
+ p = &(*p)->rb_left;
+ else if (service > th->service)
+ p = &(*p)->rb_right;
+ else
+ return th;
+ }
+
+ /* add it */
+ th = kzalloc(sizeof(*th), GFP_NOFS);
+ if (!th)
+ return ERR_PTR(-ENOMEM);
+ th->service = service;
+ rb_link_node(&th->node, parent, p);
+ rb_insert_color(&th->node, &xi->ticket_handlers);
+ return th;
+}
+
+static void remove_ticket_handler(struct ceph_auth_client *ac,
+ struct ceph_x_ticket_handler *th)
+{
+ struct ceph_x_info *xi = ac->private;
+
+ dout("remove_ticket_handler %p %d\n", th, th->service);
+ rb_erase(&th->node, &xi->ticket_handlers);
+ ceph_crypto_key_destroy(&th->session_key);
+ if (th->ticket_blob)
+ ceph_buffer_put(th->ticket_blob);
+ kfree(th);
+}
+
+static int ceph_x_proc_ticket_reply(struct ceph_auth_client *ac,
+ struct ceph_crypto_key *secret,
+ void *buf, void *end)
+{
+ struct ceph_x_info *xi = ac->private;
+ int num;
+ void *p = buf;
+ int ret;
+ char *dbuf;
+ char *ticket_buf;
+ u8 struct_v;
+
+ dbuf = kmem_cache_alloc(ceph_x_ticketbuf_cachep, GFP_NOFS | GFP_ATOMIC);
+ if (!dbuf)
+ return -ENOMEM;
+
+ ret = -ENOMEM;
+ ticket_buf = kmem_cache_alloc(ceph_x_ticketbuf_cachep,
+ GFP_NOFS | GFP_ATOMIC);
+ if (!ticket_buf)
+ goto out_dbuf;
+
+ ceph_decode_need(&p, end, 1 + sizeof(u32), bad);
+ struct_v = ceph_decode_8(&p);
+ if (struct_v != 1)
+ goto bad;
+ num = ceph_decode_32(&p);
+ dout("%d tickets\n", num);
+ while (num--) {
+ int type;
+ u8 struct_v;
+ struct ceph_x_ticket_handler *th;
+ void *dp, *dend;
+ int dlen;
+ char is_enc;
+ struct timespec validity;
+ struct ceph_crypto_key old_key;
+ void *tp, *tpend;
+ struct ceph_timespec new_validity;
+ struct ceph_crypto_key new_session_key;
+ struct ceph_buffer *new_ticket_blob;
+ unsigned long new_expires, new_renew_after;
+ u64 new_secret_id;
+
+ ceph_decode_need(&p, end, sizeof(u32) + 1, bad);
+
+ type = ceph_decode_32(&p);
+ dout(" ticket type %d %s\n", type, ceph_entity_type_name(type));
+
+ struct_v = ceph_decode_8(&p);
+ if (struct_v != 1)
+ goto bad;
+
+ th = get_ticket_handler(ac, type);
+ if (IS_ERR(th)) {
+ ret = PTR_ERR(th);
+ goto out;
+ }
+
+ /* blob for me */
+ dlen = ceph_x_decrypt(secret, &p, end, dbuf,
+ TEMP_TICKET_BUF_LEN);
+ if (dlen <= 0) {
+ ret = dlen;
+ goto out;
+ }
+ dout(" decrypted %d bytes\n", dlen);
+ dend = dbuf + dlen;
+ dp = dbuf;
+
+ struct_v = ceph_decode_8(&dp);
+ if (struct_v != 1)
+ goto bad;
+
+ memcpy(&old_key, &th->session_key, sizeof(old_key));
+ ret = ceph_crypto_key_decode(&new_session_key, &dp, dend);
+ if (ret)
+ goto out;
+
+ ceph_decode_copy(&dp, &new_validity, sizeof(new_validity));
+ ceph_decode_timespec(&validity, &new_validity);
+ new_expires = get_seconds() + validity.tv_sec;
+ new_renew_after = new_expires - (validity.tv_sec / 4);
+ dout(" expires=%lu renew_after=%lu\n", new_expires,
+ new_renew_after);
+
+ /* ticket blob for service */
+ ceph_decode_8_safe(&p, end, is_enc, bad);
+ tp = ticket_buf;
+ if (is_enc) {
+ /* encrypted */
+ dout(" encrypted ticket\n");
+ dlen = ceph_x_decrypt(&old_key, &p, end, ticket_buf,
+ TEMP_TICKET_BUF_LEN);
+ if (dlen < 0) {
+ ret = dlen;
+ goto out;
+ }
+ dlen = ceph_decode_32(&tp);
+ } else {
+ /* unencrypted */
+ ceph_decode_32_safe(&p, end, dlen, bad);
+ ceph_decode_need(&p, end, dlen, bad);
+ ceph_decode_copy(&p, ticket_buf, dlen);
+ }
+ tpend = tp + dlen;
+ dout(" ticket blob is %d bytes\n", dlen);
+ ceph_decode_need(&tp, tpend, 1 + sizeof(u64), bad);
+ struct_v = ceph_decode_8(&tp);
+ new_secret_id = ceph_decode_64(&tp);
+ ret = ceph_decode_buffer(&new_ticket_blob, &tp, tpend);
+ if (ret)
+ goto out;
+
+ /* all is well, update our ticket */
+ ceph_crypto_key_destroy(&th->session_key);
+ if (th->ticket_blob)
+ ceph_buffer_put(th->ticket_blob);
+ th->session_key = new_session_key;
+ th->ticket_blob = new_ticket_blob;
+ th->validity = new_validity;
+ th->secret_id = new_secret_id;
+ th->expires = new_expires;
+ th->renew_after = new_renew_after;
+ dout(" got ticket service %d (%s) secret_id %lld len %d\n",
+ type, ceph_entity_type_name(type), th->secret_id,
+ (int)th->ticket_blob->vec.iov_len);
+ xi->have_keys |= th->service;
+ }
+
+ ret = 0;
+out:
+ kmem_cache_free(ceph_x_ticketbuf_cachep, ticket_buf);
+out_dbuf:
+ kmem_cache_free(ceph_x_ticketbuf_cachep, dbuf);
+ return ret;
+
+bad:
+ ret = -EINVAL;
+ goto out;
+}
+
+static int ceph_x_build_authorizer(struct ceph_auth_client *ac,
+ struct ceph_x_ticket_handler *th,
+ struct ceph_x_authorizer *au)
+{
+ int maxlen;
+ struct ceph_x_authorize_a *msg_a;
+ struct ceph_x_authorize_b msg_b;
+ void *p, *end;
+ int ret;
+ int ticket_blob_len =
+ (th->ticket_blob ? th->ticket_blob->vec.iov_len : 0);
+
+ dout("build_authorizer for %s %p\n",
+ ceph_entity_type_name(th->service), au);
+
+ maxlen = sizeof(*msg_a) + sizeof(msg_b) +
+ ceph_x_encrypt_buflen(ticket_blob_len);
+ dout(" need len %d\n", maxlen);
+ if (au->buf && au->buf->alloc_len < maxlen) {
+ ceph_buffer_put(au->buf);
+ au->buf = NULL;
+ }
+ if (!au->buf) {
+ au->buf = ceph_buffer_new(maxlen, GFP_NOFS);
+ if (!au->buf)
+ return -ENOMEM;
+ }
+ au->service = th->service;
+
+ msg_a = au->buf->vec.iov_base;
+ msg_a->struct_v = 1;
+ msg_a->global_id = cpu_to_le64(ac->global_id);
+ msg_a->service_id = cpu_to_le32(th->service);
+ msg_a->ticket_blob.struct_v = 1;
+ msg_a->ticket_blob.secret_id = cpu_to_le64(th->secret_id);
+ msg_a->ticket_blob.blob_len = cpu_to_le32(ticket_blob_len);
+ if (ticket_blob_len) {
+ memcpy(msg_a->ticket_blob.blob, th->ticket_blob->vec.iov_base,
+ th->ticket_blob->vec.iov_len);
+ }
+ dout(" th %p secret_id %lld %lld\n", th, th->secret_id,
+ le64_to_cpu(msg_a->ticket_blob.secret_id));
+
+ p = msg_a + 1;
+ p += ticket_blob_len;
+ end = au->buf->vec.iov_base + au->buf->vec.iov_len;
+
+ get_random_bytes(&au->nonce, sizeof(au->nonce));
+ msg_b.struct_v = 1;
+ msg_b.nonce = cpu_to_le64(au->nonce);
+ ret = ceph_x_encrypt(&th->session_key, &msg_b, sizeof(msg_b),
+ p, end - p);
+ if (ret < 0)
+ goto out_buf;
+ p += ret;
+ au->buf->vec.iov_len = p - au->buf->vec.iov_base;
+ dout(" built authorizer nonce %llx len %d\n", au->nonce,
+ (int)au->buf->vec.iov_len);
+ BUG_ON(au->buf->vec.iov_len > maxlen);
+ return 0;
+
+out_buf:
+ ceph_buffer_put(au->buf);
+ au->buf = NULL;
+ return ret;
+}
+
+static int ceph_x_encode_ticket(struct ceph_x_ticket_handler *th,
+ void **p, void *end)
+{
+ ceph_decode_need(p, end, 1 + sizeof(u64), bad);
+ ceph_encode_8(p, 1);
+ ceph_encode_64(p, th->secret_id);
+ if (th->ticket_blob) {
+ const char *buf = th->ticket_blob->vec.iov_base;
+ u32 len = th->ticket_blob->vec.iov_len;
+
+ ceph_encode_32_safe(p, end, len, bad);
+ ceph_encode_copy_safe(p, end, buf, len, bad);
+ } else {
+ ceph_encode_32_safe(p, end, 0, bad);
+ }
+
+ return 0;
+bad:
+ return -ERANGE;
+}
+
+static void ceph_x_validate_tickets(struct ceph_auth_client *ac, int *pneed)
+{
+ int want = ac->want_keys;
+ struct ceph_x_info *xi = ac->private;
+ int service;
+
+ *pneed = ac->want_keys & ~(xi->have_keys);
+
+ for (service = 1; service <= want; service <<= 1) {
+ struct ceph_x_ticket_handler *th;
+
+ if (!(ac->want_keys & service))
+ continue;
+
+ if (*pneed & service)
+ continue;
+
+ th = get_ticket_handler(ac, service);
+
+ if (!th) {
+ *pneed |= service;
+ continue;
+ }
+
+ if (get_seconds() >= th->renew_after)
+ *pneed |= service;
+ if (get_seconds() >= th->expires)
+ xi->have_keys &= ~service;
+ }
+}
+
+
+static int ceph_x_build_request(struct ceph_auth_client *ac,
+ void *buf, void *end)
+{
+ struct ceph_x_info *xi = ac->private;
+ int need;
+ struct ceph_x_request_header *head = buf;
+ int ret;
+ struct ceph_x_ticket_handler *th =
+ get_ticket_handler(ac, CEPH_ENTITY_TYPE_AUTH);
+
+ ceph_x_validate_tickets(ac, &need);
+
+ dout("build_request want %x have %x need %x\n",
+ ac->want_keys, xi->have_keys, need);
+
+ if (need & CEPH_ENTITY_TYPE_AUTH) {
+ struct ceph_x_authenticate *auth = (void *)(head + 1);
+ void *p = auth + 1;
+ struct ceph_x_challenge_blob tmp;
+ char tmp_enc[40];
+ u64 *u;
+
+ if (p > end)
+ return -ERANGE;
+
+ dout(" get_auth_session_key\n");
+ head->op = cpu_to_le16(CEPHX_GET_AUTH_SESSION_KEY);
+
+ /* encrypt and hash */
+ get_random_bytes(&auth->client_challenge, sizeof(u64));
+ tmp.client_challenge = auth->client_challenge;
+ tmp.server_challenge = cpu_to_le64(xi->server_challenge);
+ ret = ceph_x_encrypt(&xi->secret, &tmp, sizeof(tmp),
+ tmp_enc, sizeof(tmp_enc));
+ if (ret < 0)
+ return ret;
+
+ auth->struct_v = 1;
+ auth->key = 0;
+ for (u = (u64 *)tmp_enc; u + 1 <= (u64 *)(tmp_enc + ret); u++)
+ auth->key ^= *u;
+ dout(" server_challenge %llx client_challenge %llx key %llx\n",
+ xi->server_challenge, le64_to_cpu(auth->client_challenge),
+ le64_to_cpu(auth->key));
+
+ /* now encode the old ticket if exists */
+ ret = ceph_x_encode_ticket(th, &p, end);
+ if (ret < 0)
+ return ret;
+
+ return p - buf;
+ }
+
+ if (need) {
+ void *p = head + 1;
+ struct ceph_x_service_ticket_request *req;
+
+ if (p > end)
+ return -ERANGE;
+ head->op = cpu_to_le16(CEPHX_GET_PRINCIPAL_SESSION_KEY);
+
+ BUG_ON(!th);
+ ret = ceph_x_build_authorizer(ac, th, &xi->auth_authorizer);
+ if (ret)
+ return ret;
+ ceph_encode_copy(&p, xi->auth_authorizer.buf->vec.iov_base,
+ xi->auth_authorizer.buf->vec.iov_len);
+
+ req = p;
+ req->keys = cpu_to_le32(need);
+ p += sizeof(*req);
+ return p - buf;
+ }
+
+ return 0;
+}
+
+static int ceph_x_handle_reply(struct ceph_auth_client *ac, int result,
+ void *buf, void *end)
+{
+ struct ceph_x_info *xi = ac->private;
+ struct ceph_x_reply_header *head = buf;
+ struct ceph_x_ticket_handler *th;
+ int len = end - buf;
+ int op;
+ int ret;
+
+ if (result)
+ return result; /* XXX hmm? */
+
+ if (xi->starting) {
+ /* it's a hello */
+ struct ceph_x_server_challenge *sc = buf;
+
+ if (len != sizeof(*sc))
+ return -EINVAL;
+ xi->server_challenge = le64_to_cpu(sc->server_challenge);
+ dout("handle_reply got server challenge %llx\n",
+ xi->server_challenge);
+ xi->starting = false;
+ xi->have_keys &= ~CEPH_ENTITY_TYPE_AUTH;
+ return -EAGAIN;
+ }
+
+ op = le32_to_cpu(head->op);
+ result = le32_to_cpu(head->result);
+ dout("handle_reply op %d result %d\n", op, result);
+ switch (op) {
+ case CEPHX_GET_AUTH_SESSION_KEY:
+ /* verify auth key */
+ ret = ceph_x_proc_ticket_reply(ac, &xi->secret,
+ buf + sizeof(*head), end);
+ break;
+
+ case CEPHX_GET_PRINCIPAL_SESSION_KEY:
+ th = get_ticket_handler(ac, CEPH_ENTITY_TYPE_AUTH);
+ BUG_ON(!th);
+ ret = ceph_x_proc_ticket_reply(ac, &th->session_key,
+ buf + sizeof(*head), end);
+ break;
+
+ default:
+ return -EINVAL;
+ }
+ if (ret)
+ return ret;
+ if (ac->want_keys == xi->have_keys)
+ return 0;
+ return -EAGAIN;
+}
+
+static int ceph_x_create_authorizer(
+ struct ceph_auth_client *ac, int peer_type,
+ struct ceph_authorizer **a,
+ void **buf, size_t *len,
+ void **reply_buf, size_t *reply_len)
+{
+ struct ceph_x_authorizer *au;
+ struct ceph_x_ticket_handler *th;
+ int ret;
+
+ th = get_ticket_handler(ac, peer_type);
+ if (IS_ERR(th))
+ return PTR_ERR(th);
+
+ au = kzalloc(sizeof(*au), GFP_NOFS);
+ if (!au)
+ return -ENOMEM;
+
+ ret = ceph_x_build_authorizer(ac, th, au);
+ if (ret) {
+ kfree(au);
+ return ret;
+ }
+
+ *a = (struct ceph_authorizer *)au;
+ *buf = au->buf->vec.iov_base;
+ *len = au->buf->vec.iov_len;
+ *reply_buf = au->reply_buf;
+ *reply_len = sizeof(au->reply_buf);
+ return 0;
+}
+
+static int ceph_x_verify_authorizer_reply(struct ceph_auth_client *ac,
+ struct ceph_authorizer *a, size_t len)
+{
+ struct ceph_x_authorizer *au = (void *)a;
+ struct ceph_x_ticket_handler *th;
+ int ret = 0;
+ struct ceph_x_authorize_reply reply;
+ void *p = au->reply_buf;
+ void *end = p + sizeof(au->reply_buf);
+
+ th = get_ticket_handler(ac, au->service);
+ if (!th)
+ return -EIO; /* hrm! */
+ ret = ceph_x_decrypt(&th->session_key, &p, end, &reply, sizeof(reply));
+ if (ret < 0)
+ return ret;
+ if (ret != sizeof(reply))
+ return -EPERM;
+
+ if (au->nonce + 1 != le64_to_cpu(reply.nonce_plus_one))
+ ret = -EPERM;
+ else
+ ret = 0;
+ dout("verify_authorizer_reply nonce %llx got %llx ret %d\n",
+ au->nonce, le64_to_cpu(reply.nonce_plus_one), ret);
+ return ret;
+}
+
+static void ceph_x_destroy_authorizer(struct ceph_auth_client *ac,
+ struct ceph_authorizer *a)
+{
+ struct ceph_x_authorizer *au = (void *)a;
+
+ ceph_buffer_put(au->buf);
+ kfree(au);
+}
+
+
+static void ceph_x_reset(struct ceph_auth_client *ac)
+{
+ struct ceph_x_info *xi = ac->private;
+
+ dout("reset\n");
+ xi->starting = true;
+ xi->server_challenge = 0;
+}
+
+static void ceph_x_destroy(struct ceph_auth_client *ac)
+{
+ struct ceph_x_info *xi = ac->private;
+ struct rb_node *p;
+
+ dout("ceph_x_destroy %p\n", ac);
+ ceph_crypto_key_destroy(&xi->secret);
+
+ while ((p = rb_first(&xi->ticket_handlers)) != NULL) {
+ struct ceph_x_ticket_handler *th =
+ rb_entry(p, struct ceph_x_ticket_handler, node);
+ remove_ticket_handler(ac, th);
+ }
+
+ kmem_cache_destroy(ceph_x_ticketbuf_cachep);
+
+ kfree(ac->private);
+ ac->private = NULL;
+}
+
+static void ceph_x_invalidate_authorizer(struct ceph_auth_client *ac,
+ int peer_type)
+{
+ struct ceph_x_ticket_handler *th;
+
+ th = get_ticket_handler(ac, peer_type);
+ if (th && !IS_ERR(th))
+ remove_ticket_handler(ac, th);
+}
+
+
+static const struct ceph_auth_client_ops ceph_x_ops = {
+ .is_authenticated = ceph_x_is_authenticated,
+ .build_request = ceph_x_build_request,
+ .handle_reply = ceph_x_handle_reply,
+ .create_authorizer = ceph_x_create_authorizer,
+ .verify_authorizer_reply = ceph_x_verify_authorizer_reply,
+ .destroy_authorizer = ceph_x_destroy_authorizer,
+ .invalidate_authorizer = ceph_x_invalidate_authorizer,
+ .reset = ceph_x_reset,
+ .destroy = ceph_x_destroy,
+};
+
+
+int ceph_x_init(struct ceph_auth_client *ac)
+{
+ struct ceph_x_info *xi;
+ int ret;
+
+ dout("ceph_x_init %p\n", ac);
+ xi = kzalloc(sizeof(*xi), GFP_NOFS);
+ if (!xi)
+ return -ENOMEM;
+
+ ret = -ENOMEM;
+ ceph_x_ticketbuf_cachep = kmem_cache_create("ceph_x_ticketbuf",
+ TEMP_TICKET_BUF_LEN, 8,
+ (SLAB_RECLAIM_ACCOUNT|SLAB_MEM_SPREAD),
+ NULL);
+ if (!ceph_x_ticketbuf_cachep)
+ goto done_nomem;
+ ret = -EINVAL;
+ if (!ac->secret) {
+ pr_err("no secret set (for auth_x protocol)\n");
+ goto done_nomem;
+ }
+
+ ret = ceph_crypto_key_unarmor(&xi->secret, ac->secret);
+ if (ret)
+ goto done_nomem;
+
+ xi->starting = true;
+ xi->ticket_handlers = RB_ROOT;
+
+ ac->protocol = CEPH_AUTH_CEPHX;
+ ac->private = xi;
+ ac->ops = &ceph_x_ops;
+ return 0;
+
+done_nomem:
+ kfree(xi);
+ if (ceph_x_ticketbuf_cachep)
+ kmem_cache_destroy(ceph_x_ticketbuf_cachep);
+ return ret;
+}
+
+
--- /dev/null
+#ifndef _FS_CEPH_AUTH_X_H
+#define _FS_CEPH_AUTH_X_H
+
+#include <linux/rbtree.h>
+
+#include "crypto.h"
+#include "auth.h"
+#include "auth_x_protocol.h"
+
+/*
+ * Handle ticket for a single service.
+ */
+struct ceph_x_ticket_handler {
+ struct rb_node node;
+ unsigned service;
+
+ struct ceph_crypto_key session_key;
+ struct ceph_timespec validity;
+
+ u64 secret_id;
+ struct ceph_buffer *ticket_blob;
+
+ unsigned long renew_after, expires;
+};
+
+
+struct ceph_x_authorizer {
+ struct ceph_buffer *buf;
+ unsigned service;
+ u64 nonce;
+ char reply_buf[128]; /* big enough for encrypted blob */
+};
+
+struct ceph_x_info {
+ struct ceph_crypto_key secret;
+
+ bool starting;
+ u64 server_challenge;
+
+ unsigned have_keys;
+ struct rb_root ticket_handlers;
+
+ struct ceph_x_authorizer auth_authorizer;
+};
+
+extern int ceph_x_init(struct ceph_auth_client *ac);
+
+#endif
+
--- /dev/null
+#ifndef __FS_CEPH_AUTH_X_PROTOCOL
+#define __FS_CEPH_AUTH_X_PROTOCOL
+
+#define CEPHX_GET_AUTH_SESSION_KEY 0x0100
+#define CEPHX_GET_PRINCIPAL_SESSION_KEY 0x0200
+#define CEPHX_GET_ROTATING_KEY 0x0400
+
+/* common bits */
+struct ceph_x_ticket_blob {
+ __u8 struct_v;
+ __le64 secret_id;
+ __le32 blob_len;
+ char blob[];
+} __attribute__ ((packed));
+
+
+/* common request/reply headers */
+struct ceph_x_request_header {
+ __le16 op;
+} __attribute__ ((packed));
+
+struct ceph_x_reply_header {
+ __le16 op;
+ __le32 result;
+} __attribute__ ((packed));
+
+
+/* authenticate handshake */
+
+/* initial hello (no reply header) */
+struct ceph_x_server_challenge {
+ __u8 struct_v;
+ __le64 server_challenge;
+} __attribute__ ((packed));
+
+struct ceph_x_authenticate {
+ __u8 struct_v;
+ __le64 client_challenge;
+ __le64 key;
+ /* ticket blob */
+} __attribute__ ((packed));
+
+struct ceph_x_service_ticket_request {
+ __u8 struct_v;
+ __le32 keys;
+} __attribute__ ((packed));
+
+struct ceph_x_challenge_blob {
+ __le64 server_challenge;
+ __le64 client_challenge;
+} __attribute__ ((packed));
+
+
+
+/* authorize handshake */
+
+/*
+ * The authorizer consists of two pieces:
+ * a - service id, ticket blob
+ * b - encrypted with session key
+ */
+struct ceph_x_authorize_a {
+ __u8 struct_v;
+ __le64 global_id;
+ __le32 service_id;
+ struct ceph_x_ticket_blob ticket_blob;
+} __attribute__ ((packed));
+
+struct ceph_x_authorize_b {
+ __u8 struct_v;
+ __le64 nonce;
+} __attribute__ ((packed));
+
+struct ceph_x_authorize_reply {
+ __u8 struct_v;
+ __le64 nonce_plus_one;
+} __attribute__ ((packed));
+
+
+/*
+ * encyption bundle
+ */
+#define CEPHX_ENC_MAGIC 0xff009cad8826aa55ull
+
+struct ceph_x_encrypt_header {
+ __u8 struct_v;
+ __le64 magic;
+} __attribute__ ((packed));
+
+#endif
--- /dev/null
+
+#include "ceph_debug.h"
+
+#include <linux/slab.h>
+
+#include "buffer.h"
+#include "decode.h"
+
+struct ceph_buffer *ceph_buffer_new(size_t len, gfp_t gfp)
+{
+ struct ceph_buffer *b;
+
+ b = kmalloc(sizeof(*b), gfp);
+ if (!b)
+ return NULL;
+
+ b->vec.iov_base = kmalloc(len, gfp | __GFP_NOWARN);
+ if (b->vec.iov_base) {
+ b->is_vmalloc = false;
+ } else {
+ b->vec.iov_base = __vmalloc(len, gfp, PAGE_KERNEL);
+ if (!b->vec.iov_base) {
+ kfree(b);
+ return NULL;
+ }
+ b->is_vmalloc = true;
+ }
+
+ kref_init(&b->kref);
+ b->alloc_len = len;
+ b->vec.iov_len = len;
+ dout("buffer_new %p\n", b);
+ return b;
+}
+
+void ceph_buffer_release(struct kref *kref)
+{
+ struct ceph_buffer *b = container_of(kref, struct ceph_buffer, kref);
+
+ dout("buffer_release %p\n", b);
+ if (b->vec.iov_base) {
+ if (b->is_vmalloc)
+ vfree(b->vec.iov_base);
+ else
+ kfree(b->vec.iov_base);
+ }
+ kfree(b);
+}
+
+int ceph_buffer_alloc(struct ceph_buffer *b, int len, gfp_t gfp)
+{
+ b->vec.iov_base = kmalloc(len, gfp | __GFP_NOWARN);
+ if (b->vec.iov_base) {
+ b->is_vmalloc = false;
+ } else {
+ b->vec.iov_base = __vmalloc(len, gfp, PAGE_KERNEL);
+ b->is_vmalloc = true;
+ }
+ if (!b->vec.iov_base)
+ return -ENOMEM;
+ b->alloc_len = len;
+ b->vec.iov_len = len;
+ return 0;
+}
+
+int ceph_decode_buffer(struct ceph_buffer **b, void **p, void *end)
+{
+ size_t len;
+
+ ceph_decode_need(p, end, sizeof(u32), bad);
+ len = ceph_decode_32(p);
+ dout("decode_buffer len %d\n", (int)len);
+ ceph_decode_need(p, end, len, bad);
+ *b = ceph_buffer_new(len, GFP_NOFS);
+ if (!*b)
+ return -ENOMEM;
+ ceph_decode_copy(p, (*b)->vec.iov_base, len);
+ return 0;
+bad:
+ return -EINVAL;
+}
--- /dev/null
+#ifndef __FS_CEPH_BUFFER_H
+#define __FS_CEPH_BUFFER_H
+
+#include <linux/kref.h>
+#include <linux/mm.h>
+#include <linux/vmalloc.h>
+#include <linux/types.h>
+#include <linux/uio.h>
+
+/*
+ * a simple reference counted buffer.
+ *
+ * use kmalloc for small sizes (<= one page), vmalloc for larger
+ * sizes.
+ */
+struct ceph_buffer {
+ struct kref kref;
+ struct kvec vec;
+ size_t alloc_len;
+ bool is_vmalloc;
+};
+
+extern struct ceph_buffer *ceph_buffer_new(size_t len, gfp_t gfp);
+extern void ceph_buffer_release(struct kref *kref);
+
+static inline struct ceph_buffer *ceph_buffer_get(struct ceph_buffer *b)
+{
+ kref_get(&b->kref);
+ return b;
+}
+
+static inline void ceph_buffer_put(struct ceph_buffer *b)
+{
+ kref_put(&b->kref, ceph_buffer_release);
+}
+
+extern int ceph_decode_buffer(struct ceph_buffer **b, void **p, void *end);
+
+#endif
--- /dev/null
+#include "ceph_debug.h"
+
+#include <linux/fs.h>
+#include <linux/kernel.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/vmalloc.h>
+#include <linux/wait.h>
+#include <linux/writeback.h>
+
+#include "super.h"
+#include "decode.h"
+#include "messenger.h"
+
+/*
+ * Capability management
+ *
+ * The Ceph metadata servers control client access to inode metadata
+ * and file data by issuing capabilities, granting clients permission
+ * to read and/or write both inode field and file data to OSDs
+ * (storage nodes). Each capability consists of a set of bits
+ * indicating which operations are allowed.
+ *
+ * If the client holds a *_SHARED cap, the client has a coherent value
+ * that can be safely read from the cached inode.
+ *
+ * In the case of a *_EXCL (exclusive) or FILE_WR capabilities, the
+ * client is allowed to change inode attributes (e.g., file size,
+ * mtime), note its dirty state in the ceph_cap, and asynchronously
+ * flush that metadata change to the MDS.
+ *
+ * In the event of a conflicting operation (perhaps by another
+ * client), the MDS will revoke the conflicting client capabilities.
+ *
+ * In order for a client to cache an inode, it must hold a capability
+ * with at least one MDS server. When inodes are released, release
+ * notifications are batched and periodically sent en masse to the MDS
+ * cluster to release server state.
+ */
+
+
+/*
+ * Generate readable cap strings for debugging output.
+ */
+#define MAX_CAP_STR 20
+static char cap_str[MAX_CAP_STR][40];
+static DEFINE_SPINLOCK(cap_str_lock);
+static int last_cap_str;
+
+static char *gcap_string(char *s, int c)
+{
+ if (c & CEPH_CAP_GSHARED)
+ *s++ = 's';
+ if (c & CEPH_CAP_GEXCL)
+ *s++ = 'x';
+ if (c & CEPH_CAP_GCACHE)
+ *s++ = 'c';
+ if (c & CEPH_CAP_GRD)
+ *s++ = 'r';
+ if (c & CEPH_CAP_GWR)
+ *s++ = 'w';
+ if (c & CEPH_CAP_GBUFFER)
+ *s++ = 'b';
+ if (c & CEPH_CAP_GLAZYIO)
+ *s++ = 'l';
+ return s;
+}
+
+const char *ceph_cap_string(int caps)
+{
+ int i;
+ char *s;
+ int c;
+
+ spin_lock(&cap_str_lock);
+ i = last_cap_str++;
+ if (last_cap_str == MAX_CAP_STR)
+ last_cap_str = 0;
+ spin_unlock(&cap_str_lock);
+
+ s = cap_str[i];
+
+ if (caps & CEPH_CAP_PIN)
+ *s++ = 'p';
+
+ c = (caps >> CEPH_CAP_SAUTH) & 3;
+ if (c) {
+ *s++ = 'A';
+ s = gcap_string(s, c);
+ }
+
+ c = (caps >> CEPH_CAP_SLINK) & 3;
+ if (c) {
+ *s++ = 'L';
+ s = gcap_string(s, c);
+ }
+
+ c = (caps >> CEPH_CAP_SXATTR) & 3;
+ if (c) {
+ *s++ = 'X';
+ s = gcap_string(s, c);
+ }
+
+ c = caps >> CEPH_CAP_SFILE;
+ if (c) {
+ *s++ = 'F';
+ s = gcap_string(s, c);
+ }
+
+ if (s == cap_str[i])
+ *s++ = '-';
+ *s = 0;
+ return cap_str[i];
+}
+
+/*
+ * Cap reservations
+ *
+ * Maintain a global pool of preallocated struct ceph_caps, referenced
+ * by struct ceph_caps_reservations. This ensures that we preallocate
+ * memory needed to successfully process an MDS response. (If an MDS
+ * sends us cap information and we fail to process it, we will have
+ * problems due to the client and MDS being out of sync.)
+ *
+ * Reservations are 'owned' by a ceph_cap_reservation context.
+ */
+static spinlock_t caps_list_lock;
+static struct list_head caps_list; /* unused (reserved or unreserved) */
+static int caps_total_count; /* total caps allocated */
+static int caps_use_count; /* in use */
+static int caps_reserve_count; /* unused, reserved */
+static int caps_avail_count; /* unused, unreserved */
+static int caps_min_count; /* keep at least this many (unreserved) */
+
+void __init ceph_caps_init(void)
+{
+ INIT_LIST_HEAD(&caps_list);
+ spin_lock_init(&caps_list_lock);
+}
+
+void ceph_caps_finalize(void)
+{
+ struct ceph_cap *cap;
+
+ spin_lock(&caps_list_lock);
+ while (!list_empty(&caps_list)) {
+ cap = list_first_entry(&caps_list, struct ceph_cap, caps_item);
+ list_del(&cap->caps_item);
+ kmem_cache_free(ceph_cap_cachep, cap);
+ }
+ caps_total_count = 0;
+ caps_avail_count = 0;
+ caps_use_count = 0;
+ caps_reserve_count = 0;
+ caps_min_count = 0;
+ spin_unlock(&caps_list_lock);
+}
+
+void ceph_adjust_min_caps(int delta)
+{
+ spin_lock(&caps_list_lock);
+ caps_min_count += delta;
+ BUG_ON(caps_min_count < 0);
+ spin_unlock(&caps_list_lock);
+}
+
+int ceph_reserve_caps(struct ceph_cap_reservation *ctx, int need)
+{
+ int i;
+ struct ceph_cap *cap;
+ int have;
+ int alloc = 0;
+ LIST_HEAD(newcaps);
+ int ret = 0;
+
+ dout("reserve caps ctx=%p need=%d\n", ctx, need);
+
+ /* first reserve any caps that are already allocated */
+ spin_lock(&caps_list_lock);
+ if (caps_avail_count >= need)
+ have = need;
+ else
+ have = caps_avail_count;
+ caps_avail_count -= have;
+ caps_reserve_count += have;
+ BUG_ON(caps_total_count != caps_use_count + caps_reserve_count +
+ caps_avail_count);
+ spin_unlock(&caps_list_lock);
+
+ for (i = have; i < need; i++) {
+ cap = kmem_cache_alloc(ceph_cap_cachep, GFP_NOFS);
+ if (!cap) {
+ ret = -ENOMEM;
+ goto out_alloc_count;
+ }
+ list_add(&cap->caps_item, &newcaps);
+ alloc++;
+ }
+ BUG_ON(have + alloc != need);
+
+ spin_lock(&caps_list_lock);
+ caps_total_count += alloc;
+ caps_reserve_count += alloc;
+ list_splice(&newcaps, &caps_list);
+
+ BUG_ON(caps_total_count != caps_use_count + caps_reserve_count +
+ caps_avail_count);
+ spin_unlock(&caps_list_lock);
+
+ ctx->count = need;
+ dout("reserve caps ctx=%p %d = %d used + %d resv + %d avail\n",
+ ctx, caps_total_count, caps_use_count, caps_reserve_count,
+ caps_avail_count);
+ return 0;
+
+out_alloc_count:
+ /* we didn't manage to reserve as much as we needed */
+ pr_warning("reserve caps ctx=%p ENOMEM need=%d got=%d\n",
+ ctx, need, have);
+ return ret;
+}
+
+int ceph_unreserve_caps(struct ceph_cap_reservation *ctx)
+{
+ dout("unreserve caps ctx=%p count=%d\n", ctx, ctx->count);
+ if (ctx->count) {
+ spin_lock(&caps_list_lock);
+ BUG_ON(caps_reserve_count < ctx->count);
+ caps_reserve_count -= ctx->count;
+ caps_avail_count += ctx->count;
+ ctx->count = 0;
+ dout("unreserve caps %d = %d used + %d resv + %d avail\n",
+ caps_total_count, caps_use_count, caps_reserve_count,
+ caps_avail_count);
+ BUG_ON(caps_total_count != caps_use_count + caps_reserve_count +
+ caps_avail_count);
+ spin_unlock(&caps_list_lock);
+ }
+ return 0;
+}
+
+static struct ceph_cap *get_cap(struct ceph_cap_reservation *ctx)
+{
+ struct ceph_cap *cap = NULL;
+
+ /* temporary, until we do something about cap import/export */
+ if (!ctx)
+ return kmem_cache_alloc(ceph_cap_cachep, GFP_NOFS);
+
+ spin_lock(&caps_list_lock);
+ dout("get_cap ctx=%p (%d) %d = %d used + %d resv + %d avail\n",
+ ctx, ctx->count, caps_total_count, caps_use_count,
+ caps_reserve_count, caps_avail_count);
+ BUG_ON(!ctx->count);
+ BUG_ON(ctx->count > caps_reserve_count);
+ BUG_ON(list_empty(&caps_list));
+
+ ctx->count--;
+ caps_reserve_count--;
+ caps_use_count++;
+
+ cap = list_first_entry(&caps_list, struct ceph_cap, caps_item);
+ list_del(&cap->caps_item);
+
+ BUG_ON(caps_total_count != caps_use_count + caps_reserve_count +
+ caps_avail_count);
+ spin_unlock(&caps_list_lock);
+ return cap;
+}
+
+void ceph_put_cap(struct ceph_cap *cap)
+{
+ spin_lock(&caps_list_lock);
+ dout("put_cap %p %d = %d used + %d resv + %d avail\n",
+ cap, caps_total_count, caps_use_count,
+ caps_reserve_count, caps_avail_count);
+ caps_use_count--;
+ /*
+ * Keep some preallocated caps around (ceph_min_count), to
+ * avoid lots of free/alloc churn.
+ */
+ if (caps_avail_count >= caps_reserve_count + caps_min_count) {
+ caps_total_count--;
+ kmem_cache_free(ceph_cap_cachep, cap);
+ } else {
+ caps_avail_count++;
+ list_add(&cap->caps_item, &caps_list);
+ }
+
+ BUG_ON(caps_total_count != caps_use_count + caps_reserve_count +
+ caps_avail_count);
+ spin_unlock(&caps_list_lock);
+}
+
+void ceph_reservation_status(struct ceph_client *client,
+ int *total, int *avail, int *used, int *reserved,
+ int *min)
+{
+ if (total)
+ *total = caps_total_count;
+ if (avail)
+ *avail = caps_avail_count;
+ if (used)
+ *used = caps_use_count;
+ if (reserved)
+ *reserved = caps_reserve_count;
+ if (min)
+ *min = caps_min_count;
+}
+
+/*
+ * Find ceph_cap for given mds, if any.
+ *
+ * Called with i_lock held.
+ */
+static struct ceph_cap *__get_cap_for_mds(struct ceph_inode_info *ci, int mds)
+{
+ struct ceph_cap *cap;
+ struct rb_node *n = ci->i_caps.rb_node;
+
+ while (n) {
+ cap = rb_entry(n, struct ceph_cap, ci_node);
+ if (mds < cap->mds)
+ n = n->rb_left;
+ else if (mds > cap->mds)
+ n = n->rb_right;
+ else
+ return cap;
+ }
+ return NULL;
+}
+
+/*
+ * Return id of any MDS with a cap, preferably FILE_WR|WRBUFFER|EXCL, else
+ * -1.
+ */
+static int __ceph_get_cap_mds(struct ceph_inode_info *ci, u32 *mseq)
+{
+ struct ceph_cap *cap;
+ int mds = -1;
+ struct rb_node *p;
+
+ /* prefer mds with WR|WRBUFFER|EXCL caps */
+ for (p = rb_first(&ci->i_caps); p; p = rb_next(p)) {
+ cap = rb_entry(p, struct ceph_cap, ci_node);
+ mds = cap->mds;
+ if (mseq)
+ *mseq = cap->mseq;
+ if (cap->issued & (CEPH_CAP_FILE_WR |
+ CEPH_CAP_FILE_BUFFER |
+ CEPH_CAP_FILE_EXCL))
+ break;
+ }
+ return mds;
+}
+
+int ceph_get_cap_mds(struct inode *inode)
+{
+ int mds;
+ spin_lock(&inode->i_lock);
+ mds = __ceph_get_cap_mds(ceph_inode(inode), NULL);
+ spin_unlock(&inode->i_lock);
+ return mds;
+}
+
+/*
+ * Called under i_lock.
+ */
+static void __insert_cap_node(struct ceph_inode_info *ci,
+ struct ceph_cap *new)
+{
+ struct rb_node **p = &ci->i_caps.rb_node;
+ struct rb_node *parent = NULL;
+ struct ceph_cap *cap = NULL;
+
+ while (*p) {
+ parent = *p;
+ cap = rb_entry(parent, struct ceph_cap, ci_node);
+ if (new->mds < cap->mds)
+ p = &(*p)->rb_left;
+ else if (new->mds > cap->mds)
+ p = &(*p)->rb_right;
+ else
+ BUG();
+ }
+
+ rb_link_node(&new->ci_node, parent, p);
+ rb_insert_color(&new->ci_node, &ci->i_caps);
+}
+
+/*
+ * (re)set cap hold timeouts, which control the delayed release
+ * of unused caps back to the MDS. Should be called on cap use.
+ */
+static void __cap_set_timeouts(struct ceph_mds_client *mdsc,
+ struct ceph_inode_info *ci)
+{
+ struct ceph_mount_args *ma = mdsc->client->mount_args;
+
+ ci->i_hold_caps_min = round_jiffies(jiffies +
+ ma->caps_wanted_delay_min * HZ);
+ ci->i_hold_caps_max = round_jiffies(jiffies +
+ ma->caps_wanted_delay_max * HZ);
+ dout("__cap_set_timeouts %p min %lu max %lu\n", &ci->vfs_inode,
+ ci->i_hold_caps_min - jiffies, ci->i_hold_caps_max - jiffies);
+}
+
+/*
+ * (Re)queue cap at the end of the delayed cap release list.
+ *
+ * If I_FLUSH is set, leave the inode at the front of the list.
+ *
+ * Caller holds i_lock
+ * -> we take mdsc->cap_delay_lock
+ */
+static void __cap_delay_requeue(struct ceph_mds_client *mdsc,
+ struct ceph_inode_info *ci)
+{
+ __cap_set_timeouts(mdsc, ci);
+ dout("__cap_delay_requeue %p flags %d at %lu\n", &ci->vfs_inode,
+ ci->i_ceph_flags, ci->i_hold_caps_max);
+ if (!mdsc->stopping) {
+ spin_lock(&mdsc->cap_delay_lock);
+ if (!list_empty(&ci->i_cap_delay_list)) {
+ if (ci->i_ceph_flags & CEPH_I_FLUSH)
+ goto no_change;
+ list_del_init(&ci->i_cap_delay_list);
+ }
+ list_add_tail(&ci->i_cap_delay_list, &mdsc->cap_delay_list);
+no_change:
+ spin_unlock(&mdsc->cap_delay_lock);
+ }
+}
+
+/*
+ * Queue an inode for immediate writeback. Mark inode with I_FLUSH,
+ * indicating we should send a cap message to flush dirty metadata
+ * asap, and move to the front of the delayed cap list.
+ */
+static void __cap_delay_requeue_front(struct ceph_mds_client *mdsc,
+ struct ceph_inode_info *ci)
+{
+ dout("__cap_delay_requeue_front %p\n", &ci->vfs_inode);
+ spin_lock(&mdsc->cap_delay_lock);
+ ci->i_ceph_flags |= CEPH_I_FLUSH;
+ if (!list_empty(&ci->i_cap_delay_list))
+ list_del_init(&ci->i_cap_delay_list);
+ list_add(&ci->i_cap_delay_list, &mdsc->cap_delay_list);
+ spin_unlock(&mdsc->cap_delay_lock);
+}
+
+/*
+ * Cancel delayed work on cap.
+ *
+ * Caller must hold i_lock.
+ */
+static void __cap_delay_cancel(struct ceph_mds_client *mdsc,
+ struct ceph_inode_info *ci)
+{
+ dout("__cap_delay_cancel %p\n", &ci->vfs_inode);
+ if (list_empty(&ci->i_cap_delay_list))
+ return;
+ spin_lock(&mdsc->cap_delay_lock);
+ list_del_init(&ci->i_cap_delay_list);
+ spin_unlock(&mdsc->cap_delay_lock);
+}
+
+/*
+ * Common issue checks for add_cap, handle_cap_grant.
+ */
+static void __check_cap_issue(struct ceph_inode_info *ci, struct ceph_cap *cap,
+ unsigned issued)
+{
+ unsigned had = __ceph_caps_issued(ci, NULL);
+
+ /*
+ * Each time we receive FILE_CACHE anew, we increment
+ * i_rdcache_gen.
+ */
+ if ((issued & CEPH_CAP_FILE_CACHE) &&
+ (had & CEPH_CAP_FILE_CACHE) == 0)
+ ci->i_rdcache_gen++;
+
+ /*
+ * if we are newly issued FILE_SHARED, clear I_COMPLETE; we
+ * don't know what happened to this directory while we didn't
+ * have the cap.
+ */
+ if ((issued & CEPH_CAP_FILE_SHARED) &&
+ (had & CEPH_CAP_FILE_SHARED) == 0) {
+ ci->i_shared_gen++;
+ if (S_ISDIR(ci->vfs_inode.i_mode)) {
+ dout(" marking %p NOT complete\n", &ci->vfs_inode);
+ ci->i_ceph_flags &= ~CEPH_I_COMPLETE;
+ }
+ }
+}
+
+/*
+ * Add a capability under the given MDS session.
+ *
+ * Caller should hold session snap_rwsem (read) and s_mutex.
+ *
+ * @fmode is the open file mode, if we are opening a file, otherwise
+ * it is < 0. (This is so we can atomically add the cap and add an
+ * open file reference to it.)
+ */
+int ceph_add_cap(struct inode *inode,
+ struct ceph_mds_session *session, u64 cap_id,
+ int fmode, unsigned issued, unsigned wanted,
+ unsigned seq, unsigned mseq, u64 realmino, int flags,
+ struct ceph_cap_reservation *caps_reservation)
+{
+ struct ceph_mds_client *mdsc = &ceph_inode_to_client(inode)->mdsc;
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ struct ceph_cap *new_cap = NULL;
+ struct ceph_cap *cap;
+ int mds = session->s_mds;
+ int actual_wanted;
+
+ dout("add_cap %p mds%d cap %llx %s seq %d\n", inode,
+ session->s_mds, cap_id, ceph_cap_string(issued), seq);
+
+ /*
+ * If we are opening the file, include file mode wanted bits
+ * in wanted.
+ */
+ if (fmode >= 0)
+ wanted |= ceph_caps_for_mode(fmode);
+
+retry:
+ spin_lock(&inode->i_lock);
+ cap = __get_cap_for_mds(ci, mds);
+ if (!cap) {
+ if (new_cap) {
+ cap = new_cap;
+ new_cap = NULL;
+ } else {
+ spin_unlock(&inode->i_lock);
+ new_cap = get_cap(caps_reservation);
+ if (new_cap == NULL)
+ return -ENOMEM;
+ goto retry;
+ }
+
+ cap->issued = 0;
+ cap->implemented = 0;
+ cap->mds = mds;
+ cap->mds_wanted = 0;
+
+ cap->ci = ci;
+ __insert_cap_node(ci, cap);
+
+ /* clear out old exporting info? (i.e. on cap import) */
+ if (ci->i_cap_exporting_mds == mds) {
+ ci->i_cap_exporting_issued = 0;
+ ci->i_cap_exporting_mseq = 0;
+ ci->i_cap_exporting_mds = -1;
+ }
+
+ /* add to session cap list */
+ cap->session = session;
+ spin_lock(&session->s_cap_lock);
+ list_add_tail(&cap->session_caps, &session->s_caps);
+ session->s_nr_caps++;
+ spin_unlock(&session->s_cap_lock);
+ }
+
+ if (!ci->i_snap_realm) {
+ /*
+ * add this inode to the appropriate snap realm
+ */
+ struct ceph_snap_realm *realm = ceph_lookup_snap_realm(mdsc,
+ realmino);
+ if (realm) {
+ ceph_get_snap_realm(mdsc, realm);
+ spin_lock(&realm->inodes_with_caps_lock);
+ ci->i_snap_realm = realm;
+ list_add(&ci->i_snap_realm_item,
+ &realm->inodes_with_caps);
+ spin_unlock(&realm->inodes_with_caps_lock);
+ } else {
+ pr_err("ceph_add_cap: couldn't find snap realm %llx\n",
+ realmino);
+ }
+ }
+
+ __check_cap_issue(ci, cap, issued);
+
+ /*
+ * If we are issued caps we don't want, or the mds' wanted
+ * value appears to be off, queue a check so we'll release
+ * later and/or update the mds wanted value.
+ */
+ actual_wanted = __ceph_caps_wanted(ci);
+ if ((wanted & ~actual_wanted) ||
+ (issued & ~actual_wanted & CEPH_CAP_ANY_WR)) {
+ dout(" issued %s, mds wanted %s, actual %s, queueing\n",
+ ceph_cap_string(issued), ceph_cap_string(wanted),
+ ceph_cap_string(actual_wanted));
+ __cap_delay_requeue(mdsc, ci);
+ }
+
+ if (flags & CEPH_CAP_FLAG_AUTH)
+ ci->i_auth_cap = cap;
+ else if (ci->i_auth_cap == cap)
+ ci->i_auth_cap = NULL;
+
+ dout("add_cap inode %p (%llx.%llx) cap %p %s now %s seq %d mds%d\n",
+ inode, ceph_vinop(inode), cap, ceph_cap_string(issued),
+ ceph_cap_string(issued|cap->issued), seq, mds);
+ cap->cap_id = cap_id;
+ cap->issued = issued;
+ cap->implemented |= issued;
+ cap->mds_wanted |= wanted;
+ cap->seq = seq;
+ cap->issue_seq = seq;
+ cap->mseq = mseq;
+ cap->cap_gen = session->s_cap_gen;
+
+ if (fmode >= 0)
+ __ceph_get_fmode(ci, fmode);
+ spin_unlock(&inode->i_lock);
+ wake_up(&ci->i_cap_wq);
+ return 0;
+}
+
+/*
+ * Return true if cap has not timed out and belongs to the current
+ * generation of the MDS session (i.e. has not gone 'stale' due to
+ * us losing touch with the mds).
+ */
+static int __cap_is_valid(struct ceph_cap *cap)
+{
+ unsigned long ttl;
+ u32 gen;
+
+ spin_lock(&cap->session->s_cap_lock);
+ gen = cap->session->s_cap_gen;
+ ttl = cap->session->s_cap_ttl;
+ spin_unlock(&cap->session->s_cap_lock);
+
+ if (cap->cap_gen < gen || time_after_eq(jiffies, ttl)) {
+ dout("__cap_is_valid %p cap %p issued %s "
+ "but STALE (gen %u vs %u)\n", &cap->ci->vfs_inode,
+ cap, ceph_cap_string(cap->issued), cap->cap_gen, gen);
+ return 0;
+ }
+
+ return 1;
+}
+
+/*
+ * Return set of valid cap bits issued to us. Note that caps time
+ * out, and may be invalidated in bulk if the client session times out
+ * and session->s_cap_gen is bumped.
+ */
+int __ceph_caps_issued(struct ceph_inode_info *ci, int *implemented)
+{
+ int have = ci->i_snap_caps | ci->i_cap_exporting_issued;
+ struct ceph_cap *cap;
+ struct rb_node *p;
+
+ if (implemented)
+ *implemented = 0;
+ for (p = rb_first(&ci->i_caps); p; p = rb_next(p)) {
+ cap = rb_entry(p, struct ceph_cap, ci_node);
+ if (!__cap_is_valid(cap))
+ continue;
+ dout("__ceph_caps_issued %p cap %p issued %s\n",
+ &ci->vfs_inode, cap, ceph_cap_string(cap->issued));
+ have |= cap->issued;
+ if (implemented)
+ *implemented |= cap->implemented;
+ }
+ return have;
+}
+
+/*
+ * Get cap bits issued by caps other than @ocap
+ */
+int __ceph_caps_issued_other(struct ceph_inode_info *ci, struct ceph_cap *ocap)
+{
+ int have = ci->i_snap_caps;
+ struct ceph_cap *cap;
+ struct rb_node *p;
+
+ for (p = rb_first(&ci->i_caps); p; p = rb_next(p)) {
+ cap = rb_entry(p, struct ceph_cap, ci_node);
+ if (cap == ocap)
+ continue;
+ if (!__cap_is_valid(cap))
+ continue;
+ have |= cap->issued;
+ }
+ return have;
+}
+
+/*
+ * Move a cap to the end of the LRU (oldest caps at list head, newest
+ * at list tail).
+ */
+static void __touch_cap(struct ceph_cap *cap)
+{
+ struct ceph_mds_session *s = cap->session;
+
+ spin_lock(&s->s_cap_lock);
+ if (s->s_cap_iterator == NULL) {
+ dout("__touch_cap %p cap %p mds%d\n", &cap->ci->vfs_inode, cap,
+ s->s_mds);
+ list_move_tail(&cap->session_caps, &s->s_caps);
+ } else {
+ dout("__touch_cap %p cap %p mds%d NOP, iterating over caps\n",
+ &cap->ci->vfs_inode, cap, s->s_mds);
+ }
+ spin_unlock(&s->s_cap_lock);
+}
+
+/*
+ * Check if we hold the given mask. If so, move the cap(s) to the
+ * front of their respective LRUs. (This is the preferred way for
+ * callers to check for caps they want.)
+ */
+int __ceph_caps_issued_mask(struct ceph_inode_info *ci, int mask, int touch)
+{
+ struct ceph_cap *cap;
+ struct rb_node *p;
+ int have = ci->i_snap_caps;
+
+ if ((have & mask) == mask) {
+ dout("__ceph_caps_issued_mask %p snap issued %s"
+ " (mask %s)\n", &ci->vfs_inode,
+ ceph_cap_string(have),
+ ceph_cap_string(mask));
+ return 1;
+ }
+
+ for (p = rb_first(&ci->i_caps); p; p = rb_next(p)) {
+ cap = rb_entry(p, struct ceph_cap, ci_node);
+ if (!__cap_is_valid(cap))
+ continue;
+ if ((cap->issued & mask) == mask) {
+ dout("__ceph_caps_issued_mask %p cap %p issued %s"
+ " (mask %s)\n", &ci->vfs_inode, cap,
+ ceph_cap_string(cap->issued),
+ ceph_cap_string(mask));
+ if (touch)
+ __touch_cap(cap);
+ return 1;
+ }
+
+ /* does a combination of caps satisfy mask? */
+ have |= cap->issued;
+ if ((have & mask) == mask) {
+ dout("__ceph_caps_issued_mask %p combo issued %s"
+ " (mask %s)\n", &ci->vfs_inode,
+ ceph_cap_string(cap->issued),
+ ceph_cap_string(mask));
+ if (touch) {
+ struct rb_node *q;
+
+ /* touch this + preceeding caps */
+ __touch_cap(cap);
+ for (q = rb_first(&ci->i_caps); q != p;
+ q = rb_next(q)) {
+ cap = rb_entry(q, struct ceph_cap,
+ ci_node);
+ if (!__cap_is_valid(cap))
+ continue;
+ __touch_cap(cap);
+ }
+ }
+ return 1;
+ }
+ }
+
+ return 0;
+}
+
+/*
+ * Return true if mask caps are currently being revoked by an MDS.
+ */
+int ceph_caps_revoking(struct ceph_inode_info *ci, int mask)
+{
+ struct inode *inode = &ci->vfs_inode;
+ struct ceph_cap *cap;
+ struct rb_node *p;
+ int ret = 0;
+
+ spin_lock(&inode->i_lock);
+ for (p = rb_first(&ci->i_caps); p; p = rb_next(p)) {
+ cap = rb_entry(p, struct ceph_cap, ci_node);
+ if (__cap_is_valid(cap) &&
+ (cap->implemented & ~cap->issued & mask)) {
+ ret = 1;
+ break;
+ }
+ }
+ spin_unlock(&inode->i_lock);
+ dout("ceph_caps_revoking %p %s = %d\n", inode,
+ ceph_cap_string(mask), ret);
+ return ret;
+}
+
+int __ceph_caps_used(struct ceph_inode_info *ci)
+{
+ int used = 0;
+ if (ci->i_pin_ref)
+ used |= CEPH_CAP_PIN;
+ if (ci->i_rd_ref)
+ used |= CEPH_CAP_FILE_RD;
+ if (ci->i_rdcache_ref || ci->i_rdcache_gen)
+ used |= CEPH_CAP_FILE_CACHE;
+ if (ci->i_wr_ref)
+ used |= CEPH_CAP_FILE_WR;
+ if (ci->i_wrbuffer_ref)
+ used |= CEPH_CAP_FILE_BUFFER;
+ return used;
+}
+
+/*
+ * wanted, by virtue of open file modes
+ */
+int __ceph_caps_file_wanted(struct ceph_inode_info *ci)
+{
+ int want = 0;
+ int mode;
+ for (mode = 0; mode < 4; mode++)
+ if (ci->i_nr_by_mode[mode])
+ want |= ceph_caps_for_mode(mode);
+ return want;
+}
+
+/*
+ * Return caps we have registered with the MDS(s) as 'wanted'.
+ */
+int __ceph_caps_mds_wanted(struct ceph_inode_info *ci)
+{
+ struct ceph_cap *cap;
+ struct rb_node *p;
+ int mds_wanted = 0;
+
+ for (p = rb_first(&ci->i_caps); p; p = rb_next(p)) {
+ cap = rb_entry(p, struct ceph_cap, ci_node);
+ if (!__cap_is_valid(cap))
+ continue;
+ mds_wanted |= cap->mds_wanted;
+ }
+ return mds_wanted;
+}
+
+/*
+ * called under i_lock
+ */
+static int __ceph_is_any_caps(struct ceph_inode_info *ci)
+{
+ return !RB_EMPTY_ROOT(&ci->i_caps) || ci->i_cap_exporting_mds >= 0;
+}
+
+/*
+ * caller should hold i_lock.
+ * caller will not hold session s_mutex if called from destroy_inode.
+ */
+void __ceph_remove_cap(struct ceph_cap *cap)
+{
+ struct ceph_mds_session *session = cap->session;
+ struct ceph_inode_info *ci = cap->ci;
+ struct ceph_mds_client *mdsc = &ceph_client(ci->vfs_inode.i_sb)->mdsc;
+
+ dout("__ceph_remove_cap %p from %p\n", cap, &ci->vfs_inode);
+
+ /* remove from inode list */
+ rb_erase(&cap->ci_node, &ci->i_caps);
+ cap->ci = NULL;
+ if (ci->i_auth_cap == cap)
+ ci->i_auth_cap = NULL;
+
+ /* remove from session list */
+ spin_lock(&session->s_cap_lock);
+ if (session->s_cap_iterator == cap) {
+ /* not yet, we are iterating over this very cap */
+ dout("__ceph_remove_cap delaying %p removal from session %p\n",
+ cap, cap->session);
+ } else {
+ list_del_init(&cap->session_caps);
+ session->s_nr_caps--;
+ cap->session = NULL;
+ }
+ spin_unlock(&session->s_cap_lock);
+
+ if (cap->session == NULL)
+ ceph_put_cap(cap);
+
+ if (!__ceph_is_any_caps(ci) && ci->i_snap_realm) {
+ struct ceph_snap_realm *realm = ci->i_snap_realm;
+ spin_lock(&realm->inodes_with_caps_lock);
+ list_del_init(&ci->i_snap_realm_item);
+ ci->i_snap_realm_counter++;
+ ci->i_snap_realm = NULL;
+ spin_unlock(&realm->inodes_with_caps_lock);
+ ceph_put_snap_realm(mdsc, realm);
+ }
+ if (!__ceph_is_any_real_caps(ci))
+ __cap_delay_cancel(mdsc, ci);
+}
+
+/*
+ * Build and send a cap message to the given MDS.
+ *
+ * Caller should be holding s_mutex.
+ */
+static int send_cap_msg(struct ceph_mds_session *session,
+ u64 ino, u64 cid, int op,
+ int caps, int wanted, int dirty,
+ u32 seq, u64 flush_tid, u32 issue_seq, u32 mseq,
+ u64 size, u64 max_size,
+ struct timespec *mtime, struct timespec *atime,
+ u64 time_warp_seq,
+ uid_t uid, gid_t gid, mode_t mode,
+ u64 xattr_version,
+ struct ceph_buffer *xattrs_buf,
+ u64 follows)
+{
+ struct ceph_mds_caps *fc;
+ struct ceph_msg *msg;
+
+ dout("send_cap_msg %s %llx %llx caps %s wanted %s dirty %s"
+ " seq %u/%u mseq %u follows %lld size %llu/%llu"
+ " xattr_ver %llu xattr_len %d\n", ceph_cap_op_name(op),
+ cid, ino, ceph_cap_string(caps), ceph_cap_string(wanted),
+ ceph_cap_string(dirty),
+ seq, issue_seq, mseq, follows, size, max_size,
+ xattr_version, xattrs_buf ? (int)xattrs_buf->vec.iov_len : 0);
+
+ msg = ceph_msg_new(CEPH_MSG_CLIENT_CAPS, sizeof(*fc), 0, 0, NULL);
+ if (IS_ERR(msg))
+ return PTR_ERR(msg);
+
+ msg->hdr.tid = cpu_to_le64(flush_tid);
+
+ fc = msg->front.iov_base;
+ memset(fc, 0, sizeof(*fc));
+
+ fc->cap_id = cpu_to_le64(cid);
+ fc->op = cpu_to_le32(op);
+ fc->seq = cpu_to_le32(seq);
+ fc->issue_seq = cpu_to_le32(issue_seq);
+ fc->migrate_seq = cpu_to_le32(mseq);
+ fc->caps = cpu_to_le32(caps);
+ fc->wanted = cpu_to_le32(wanted);
+ fc->dirty = cpu_to_le32(dirty);
+ fc->ino = cpu_to_le64(ino);
+ fc->snap_follows = cpu_to_le64(follows);
+
+ fc->size = cpu_to_le64(size);
+ fc->max_size = cpu_to_le64(max_size);
+ if (mtime)
+ ceph_encode_timespec(&fc->mtime, mtime);
+ if (atime)
+ ceph_encode_timespec(&fc->atime, atime);
+ fc->time_warp_seq = cpu_to_le32(time_warp_seq);
+
+ fc->uid = cpu_to_le32(uid);
+ fc->gid = cpu_to_le32(gid);
+ fc->mode = cpu_to_le32(mode);
+
+ fc->xattr_version = cpu_to_le64(xattr_version);
+ if (xattrs_buf) {
+ msg->middle = ceph_buffer_get(xattrs_buf);
+ fc->xattr_len = cpu_to_le32(xattrs_buf->vec.iov_len);
+ msg->hdr.middle_len = cpu_to_le32(xattrs_buf->vec.iov_len);
+ }
+
+ ceph_con_send(&session->s_con, msg);
+ return 0;
+}
+
+/*
+ * Queue cap releases when an inode is dropped from our cache. Since
+ * inode is about to be destroyed, there is no need for i_lock.
+ */
+void ceph_queue_caps_release(struct inode *inode)
+{
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ struct rb_node *p;
+
+ p = rb_first(&ci->i_caps);
+ while (p) {
+ struct ceph_cap *cap = rb_entry(p, struct ceph_cap, ci_node);
+ struct ceph_mds_session *session = cap->session;
+ struct ceph_msg *msg;
+ struct ceph_mds_cap_release *head;
+ struct ceph_mds_cap_item *item;
+
+ spin_lock(&session->s_cap_lock);
+ BUG_ON(!session->s_num_cap_releases);
+ msg = list_first_entry(&session->s_cap_releases,
+ struct ceph_msg, list_head);
+
+ dout(" adding %p release to mds%d msg %p (%d left)\n",
+ inode, session->s_mds, msg, session->s_num_cap_releases);
+
+ BUG_ON(msg->front.iov_len + sizeof(*item) > PAGE_CACHE_SIZE);
+ head = msg->front.iov_base;
+ head->num = cpu_to_le32(le32_to_cpu(head->num) + 1);
+ item = msg->front.iov_base + msg->front.iov_len;
+ item->ino = cpu_to_le64(ceph_ino(inode));
+ item->cap_id = cpu_to_le64(cap->cap_id);
+ item->migrate_seq = cpu_to_le32(cap->mseq);
+ item->seq = cpu_to_le32(cap->issue_seq);
+
+ session->s_num_cap_releases--;
+
+ msg->front.iov_len += sizeof(*item);
+ if (le32_to_cpu(head->num) == CEPH_CAPS_PER_RELEASE) {
+ dout(" release msg %p full\n", msg);
+ list_move_tail(&msg->list_head,
+ &session->s_cap_releases_done);
+ } else {
+ dout(" release msg %p at %d/%d (%d)\n", msg,
+ (int)le32_to_cpu(head->num),
+ (int)CEPH_CAPS_PER_RELEASE,
+ (int)msg->front.iov_len);
+ }
+ spin_unlock(&session->s_cap_lock);
+ p = rb_next(p);
+ __ceph_remove_cap(cap);
+ }
+}
+
+/*
+ * Send a cap msg on the given inode. Update our caps state, then
+ * drop i_lock and send the message.
+ *
+ * Make note of max_size reported/requested from mds, revoked caps
+ * that have now been implemented.
+ *
+ * Make half-hearted attempt ot to invalidate page cache if we are
+ * dropping RDCACHE. Note that this will leave behind locked pages
+ * that we'll then need to deal with elsewhere.
+ *
+ * Return non-zero if delayed release, or we experienced an error
+ * such that the caller should requeue + retry later.
+ *
+ * called with i_lock, then drops it.
+ * caller should hold snap_rwsem (read), s_mutex.
+ */
+static int __send_cap(struct ceph_mds_client *mdsc, struct ceph_cap *cap,
+ int op, int used, int want, int retain, int flushing,
+ unsigned *pflush_tid)
+ __releases(cap->ci->vfs_inode->i_lock)
+{
+ struct ceph_inode_info *ci = cap->ci;
+ struct inode *inode = &ci->vfs_inode;
+ u64 cap_id = cap->cap_id;
+ int held, revoking, dropping, keep;
+ u64 seq, issue_seq, mseq, time_warp_seq, follows;
+ u64 size, max_size;
+ struct timespec mtime, atime;
+ int wake = 0;
+ mode_t mode;
+ uid_t uid;
+ gid_t gid;
+ struct ceph_mds_session *session;
+ u64 xattr_version = 0;
+ int delayed = 0;
+ u64 flush_tid = 0;
+ int i;
+ int ret;
+
+ held = cap->issued | cap->implemented;
+ revoking = cap->implemented & ~cap->issued;
+ retain &= ~revoking;
+ dropping = cap->issued & ~retain;
+
+ dout("__send_cap %p cap %p session %p %s -> %s (revoking %s)\n",
+ inode, cap, cap->session,
+ ceph_cap_string(held), ceph_cap_string(held & retain),
+ ceph_cap_string(revoking));
+ BUG_ON((retain & CEPH_CAP_PIN) == 0);
+
+ session = cap->session;
+
+ /* don't release wanted unless we've waited a bit. */
+ if ((ci->i_ceph_flags & CEPH_I_NODELAY) == 0 &&
+ time_before(jiffies, ci->i_hold_caps_min)) {
+ dout(" delaying issued %s -> %s, wanted %s -> %s on send\n",
+ ceph_cap_string(cap->issued),
+ ceph_cap_string(cap->issued & retain),
+ ceph_cap_string(cap->mds_wanted),
+ ceph_cap_string(want));
+ want |= cap->mds_wanted;
+ retain |= cap->issued;
+ delayed = 1;
+ }
+ ci->i_ceph_flags &= ~(CEPH_I_NODELAY | CEPH_I_FLUSH);
+
+ cap->issued &= retain; /* drop bits we don't want */
+ if (cap->implemented & ~cap->issued) {
+ /*
+ * Wake up any waiters on wanted -> needed transition.
+ * This is due to the weird transition from buffered
+ * to sync IO... we need to flush dirty pages _before_
+ * allowing sync writes to avoid reordering.
+ */
+ wake = 1;
+ }
+ cap->implemented &= cap->issued | used;
+ cap->mds_wanted = want;
+
+ if (flushing) {
+ /*
+ * assign a tid for flush operations so we can avoid
+ * flush1 -> dirty1 -> flush2 -> flushack1 -> mark
+ * clean type races. track latest tid for every bit
+ * so we can handle flush AxFw, flush Fw, and have the
+ * first ack clean Ax.
+ */
+ flush_tid = ++ci->i_cap_flush_last_tid;
+ if (pflush_tid)
+ *pflush_tid = flush_tid;
+ dout(" cap_flush_tid %d\n", (int)flush_tid);
+ for (i = 0; i < CEPH_CAP_BITS; i++)
+ if (flushing & (1 << i))
+ ci->i_cap_flush_tid[i] = flush_tid;
+ }
+
+ keep = cap->implemented;
+ seq = cap->seq;
+ issue_seq = cap->issue_seq;
+ mseq = cap->mseq;
+ size = inode->i_size;
+ ci->i_reported_size = size;
+ max_size = ci->i_wanted_max_size;
+ ci->i_requested_max_size = max_size;
+ mtime = inode->i_mtime;
+ atime = inode->i_atime;
+ time_warp_seq = ci->i_time_warp_seq;
+ follows = ci->i_snap_realm->cached_context->seq;
+ uid = inode->i_uid;
+ gid = inode->i_gid;
+ mode = inode->i_mode;
+
+ if (dropping & CEPH_CAP_XATTR_EXCL) {
+ __ceph_build_xattrs_blob(ci);
+ xattr_version = ci->i_xattrs.version + 1;
+ }
+
+ spin_unlock(&inode->i_lock);
+
+ ret = send_cap_msg(session, ceph_vino(inode).ino, cap_id,
+ op, keep, want, flushing, seq, flush_tid, issue_seq, mseq,
+ size, max_size, &mtime, &atime, time_warp_seq,
+ uid, gid, mode,
+ xattr_version,
+ (flushing & CEPH_CAP_XATTR_EXCL) ? ci->i_xattrs.blob : NULL,
+ follows);
+ if (ret < 0) {
+ dout("error sending cap msg, must requeue %p\n", inode);
+ delayed = 1;
+ }
+
+ if (wake)
+ wake_up(&ci->i_cap_wq);
+
+ return delayed;
+}
+
+/*
+ * When a snapshot is taken, clients accumulate dirty metadata on
+ * inodes with capabilities in ceph_cap_snaps to describe the file
+ * state at the time the snapshot was taken. This must be flushed
+ * asynchronously back to the MDS once sync writes complete and dirty
+ * data is written out.
+ *
+ * Called under i_lock. Takes s_mutex as needed.
+ */
+void __ceph_flush_snaps(struct ceph_inode_info *ci,
+ struct ceph_mds_session **psession)
+{
+ struct inode *inode = &ci->vfs_inode;
+ int mds;
+ struct ceph_cap_snap *capsnap;
+ u32 mseq;
+ struct ceph_mds_client *mdsc = &ceph_inode_to_client(inode)->mdsc;
+ struct ceph_mds_session *session = NULL; /* if session != NULL, we hold
+ session->s_mutex */
+ u64 next_follows = 0; /* keep track of how far we've gotten through the
+ i_cap_snaps list, and skip these entries next time
+ around to avoid an infinite loop */
+
+ if (psession)
+ session = *psession;
+
+ dout("__flush_snaps %p\n", inode);
+retry:
+ list_for_each_entry(capsnap, &ci->i_cap_snaps, ci_item) {
+ /* avoid an infiniute loop after retry */
+ if (capsnap->follows < next_follows)
+ continue;
+ /*
+ * we need to wait for sync writes to complete and for dirty
+ * pages to be written out.
+ */
+ if (capsnap->dirty_pages || capsnap->writing)
+ continue;
+
+ /* pick mds, take s_mutex */
+ mds = __ceph_get_cap_mds(ci, &mseq);
+ if (session && session->s_mds != mds) {
+ dout("oops, wrong session %p mutex\n", session);
+ mutex_unlock(&session->s_mutex);
+ ceph_put_mds_session(session);
+ session = NULL;
+ }
+ if (!session) {
+ spin_unlock(&inode->i_lock);
+ mutex_lock(&mdsc->mutex);
+ session = __ceph_lookup_mds_session(mdsc, mds);
+ mutex_unlock(&mdsc->mutex);
+ if (session) {
+ dout("inverting session/ino locks on %p\n",
+ session);
+ mutex_lock(&session->s_mutex);
+ }
+ /*
+ * if session == NULL, we raced against a cap
+ * deletion. retry, and we'll get a better
+ * @mds value next time.
+ */
+ spin_lock(&inode->i_lock);
+ goto retry;
+ }
+
+ capsnap->flush_tid = ++ci->i_cap_flush_last_tid;
+ atomic_inc(&capsnap->nref);
+ if (!list_empty(&capsnap->flushing_item))
+ list_del_init(&capsnap->flushing_item);
+ list_add_tail(&capsnap->flushing_item,
+ &session->s_cap_snaps_flushing);
+ spin_unlock(&inode->i_lock);
+
+ dout("flush_snaps %p cap_snap %p follows %lld size %llu\n",
+ inode, capsnap, next_follows, capsnap->size);
+ send_cap_msg(session, ceph_vino(inode).ino, 0,
+ CEPH_CAP_OP_FLUSHSNAP, capsnap->issued, 0,
+ capsnap->dirty, 0, capsnap->flush_tid, 0, mseq,
+ capsnap->size, 0,
+ &capsnap->mtime, &capsnap->atime,
+ capsnap->time_warp_seq,
+ capsnap->uid, capsnap->gid, capsnap->mode,
+ 0, NULL,
+ capsnap->follows);
+
+ next_follows = capsnap->follows + 1;
+ ceph_put_cap_snap(capsnap);
+
+ spin_lock(&inode->i_lock);
+ goto retry;
+ }
+
+ /* we flushed them all; remove this inode from the queue */
+ spin_lock(&mdsc->snap_flush_lock);
+ list_del_init(&ci->i_snap_flush_item);
+ spin_unlock(&mdsc->snap_flush_lock);
+
+ if (psession)
+ *psession = session;
+ else if (session) {
+ mutex_unlock(&session->s_mutex);
+ ceph_put_mds_session(session);
+ }
+}
+
+static void ceph_flush_snaps(struct ceph_inode_info *ci)
+{
+ struct inode *inode = &ci->vfs_inode;
+
+ spin_lock(&inode->i_lock);
+ __ceph_flush_snaps(ci, NULL);
+ spin_unlock(&inode->i_lock);
+}
+
+/*
+ * Mark caps dirty. If inode is newly dirty, add to the global dirty
+ * list.
+ */
+void __ceph_mark_dirty_caps(struct ceph_inode_info *ci, int mask)
+{
+ struct ceph_mds_client *mdsc = &ceph_client(ci->vfs_inode.i_sb)->mdsc;
+ struct inode *inode = &ci->vfs_inode;
+ int was = ci->i_dirty_caps;
+ int dirty = 0;
+
+ dout("__mark_dirty_caps %p %s dirty %s -> %s\n", &ci->vfs_inode,
+ ceph_cap_string(mask), ceph_cap_string(was),
+ ceph_cap_string(was | mask));
+ ci->i_dirty_caps |= mask;
+ if (was == 0) {
+ dout(" inode %p now dirty\n", &ci->vfs_inode);
+ BUG_ON(!list_empty(&ci->i_dirty_item));
+ spin_lock(&mdsc->cap_dirty_lock);
+ list_add(&ci->i_dirty_item, &mdsc->cap_dirty);
+ spin_unlock(&mdsc->cap_dirty_lock);
+ if (ci->i_flushing_caps == 0) {
+ igrab(inode);
+ dirty |= I_DIRTY_SYNC;
+ }
+ }
+ BUG_ON(list_empty(&ci->i_dirty_item));
+ if (((was | ci->i_flushing_caps) & CEPH_CAP_FILE_BUFFER) &&
+ (mask & CEPH_CAP_FILE_BUFFER))
+ dirty |= I_DIRTY_DATASYNC;
+ if (dirty)
+ __mark_inode_dirty(inode, dirty);
+ __cap_delay_requeue(mdsc, ci);
+}
+
+/*
+ * Add dirty inode to the flushing list. Assigned a seq number so we
+ * can wait for caps to flush without starving.
+ *
+ * Called under i_lock.
+ */
+static int __mark_caps_flushing(struct inode *inode,
+ struct ceph_mds_session *session)
+{
+ struct ceph_mds_client *mdsc = &ceph_client(inode->i_sb)->mdsc;
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ int flushing;
+
+ BUG_ON(ci->i_dirty_caps == 0);
+ BUG_ON(list_empty(&ci->i_dirty_item));
+
+ flushing = ci->i_dirty_caps;
+ dout("__mark_caps_flushing flushing %s, flushing_caps %s -> %s\n",
+ ceph_cap_string(flushing),
+ ceph_cap_string(ci->i_flushing_caps),
+ ceph_cap_string(ci->i_flushing_caps | flushing));
+ ci->i_flushing_caps |= flushing;
+ ci->i_dirty_caps = 0;
+ dout(" inode %p now !dirty\n", inode);
+
+ spin_lock(&mdsc->cap_dirty_lock);
+ list_del_init(&ci->i_dirty_item);
+
+ ci->i_cap_flush_seq = ++mdsc->cap_flush_seq;
+ if (list_empty(&ci->i_flushing_item)) {
+ list_add_tail(&ci->i_flushing_item, &session->s_cap_flushing);
+ mdsc->num_cap_flushing++;
+ dout(" inode %p now flushing seq %lld\n", inode,
+ ci->i_cap_flush_seq);
+ } else {
+ list_move_tail(&ci->i_flushing_item, &session->s_cap_flushing);
+ dout(" inode %p now flushing (more) seq %lld\n", inode,
+ ci->i_cap_flush_seq);
+ }
+ spin_unlock(&mdsc->cap_dirty_lock);
+
+ return flushing;
+}
+
+/*
+ * try to invalidate mapping pages without blocking.
+ */
+static int mapping_is_empty(struct address_space *mapping)
+{
+ struct page *page = find_get_page(mapping, 0);
+
+ if (!page)
+ return 1;
+
+ put_page(page);
+ return 0;
+}
+
+static int try_nonblocking_invalidate(struct inode *inode)
+{
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ u32 invalidating_gen = ci->i_rdcache_gen;
+
+ spin_unlock(&inode->i_lock);
+ invalidate_mapping_pages(&inode->i_data, 0, -1);
+ spin_lock(&inode->i_lock);
+
+ if (mapping_is_empty(&inode->i_data) &&
+ invalidating_gen == ci->i_rdcache_gen) {
+ /* success. */
+ dout("try_nonblocking_invalidate %p success\n", inode);
+ ci->i_rdcache_gen = 0;
+ ci->i_rdcache_revoking = 0;
+ return 0;
+ }
+ dout("try_nonblocking_invalidate %p failed\n", inode);
+ return -1;
+}
+
+/*
+ * Swiss army knife function to examine currently used and wanted
+ * versus held caps. Release, flush, ack revoked caps to mds as
+ * appropriate.
+ *
+ * CHECK_CAPS_NODELAY - caller is delayed work and we should not delay
+ * cap release further.
+ * CHECK_CAPS_AUTHONLY - we should only check the auth cap
+ * CHECK_CAPS_FLUSH - we should flush any dirty caps immediately, without
+ * further delay.
+ */
+void ceph_check_caps(struct ceph_inode_info *ci, int flags,
+ struct ceph_mds_session *session)
+ __releases(session->s_mutex)
+{
+ struct ceph_client *client = ceph_inode_to_client(&ci->vfs_inode);
+ struct ceph_mds_client *mdsc = &client->mdsc;
+ struct inode *inode = &ci->vfs_inode;
+ struct ceph_cap *cap;
+ int file_wanted, used;
+ int took_snap_rwsem = 0; /* true if mdsc->snap_rwsem held */
+ int issued, implemented, want, retain, revoking, flushing = 0;
+ int mds = -1; /* keep track of how far we've gone through i_caps list
+ to avoid an infinite loop on retry */
+ struct rb_node *p;
+ int tried_invalidate = 0;
+ int delayed = 0, sent = 0, force_requeue = 0, num;
+ int queue_invalidate = 0;
+ int is_delayed = flags & CHECK_CAPS_NODELAY;
+
+ /* if we are unmounting, flush any unused caps immediately. */
+ if (mdsc->stopping)
+ is_delayed = 1;
+
+ spin_lock(&inode->i_lock);
+
+ if (ci->i_ceph_flags & CEPH_I_FLUSH)
+ flags |= CHECK_CAPS_FLUSH;
+
+ /* flush snaps first time around only */
+ if (!list_empty(&ci->i_cap_snaps))
+ __ceph_flush_snaps(ci, &session);
+ goto retry_locked;
+retry:
+ spin_lock(&inode->i_lock);
+retry_locked:
+ file_wanted = __ceph_caps_file_wanted(ci);
+ used = __ceph_caps_used(ci);
+ want = file_wanted | used;
+ issued = __ceph_caps_issued(ci, &implemented);
+ revoking = implemented & ~issued;
+
+ retain = want | CEPH_CAP_PIN;
+ if (!mdsc->stopping && inode->i_nlink > 0) {
+ if (want) {
+ retain |= CEPH_CAP_ANY; /* be greedy */
+ } else {
+ retain |= CEPH_CAP_ANY_SHARED;
+ /*
+ * keep RD only if we didn't have the file open RW,
+ * because then the mds would revoke it anyway to
+ * journal max_size=0.
+ */
+ if (ci->i_max_size == 0)
+ retain |= CEPH_CAP_ANY_RD;
+ }
+ }
+
+ dout("check_caps %p file_want %s used %s dirty %s flushing %s"
+ " issued %s revoking %s retain %s %s%s%s\n", inode,
+ ceph_cap_string(file_wanted),
+ ceph_cap_string(used), ceph_cap_string(ci->i_dirty_caps),
+ ceph_cap_string(ci->i_flushing_caps),
+ ceph_cap_string(issued), ceph_cap_string(revoking),
+ ceph_cap_string(retain),
+ (flags & CHECK_CAPS_AUTHONLY) ? " AUTHONLY" : "",
+ (flags & CHECK_CAPS_NODELAY) ? " NODELAY" : "",
+ (flags & CHECK_CAPS_FLUSH) ? " FLUSH" : "");
+
+ /*
+ * If we no longer need to hold onto old our caps, and we may
+ * have cached pages, but don't want them, then try to invalidate.
+ * If we fail, it's because pages are locked.... try again later.
+ */
+ if ((!is_delayed || mdsc->stopping) &&
+ ci->i_wrbuffer_ref == 0 && /* no dirty pages... */
+ ci->i_rdcache_gen && /* may have cached pages */
+ (file_wanted == 0 || /* no open files */
+ (revoking & CEPH_CAP_FILE_CACHE)) && /* or revoking cache */
+ !tried_invalidate) {
+ dout("check_caps trying to invalidate on %p\n", inode);
+ if (try_nonblocking_invalidate(inode) < 0) {
+ if (revoking & CEPH_CAP_FILE_CACHE) {
+ dout("check_caps queuing invalidate\n");
+ queue_invalidate = 1;
+ ci->i_rdcache_revoking = ci->i_rdcache_gen;
+ } else {
+ dout("check_caps failed to invalidate pages\n");
+ /* we failed to invalidate pages. check these
+ caps again later. */
+ force_requeue = 1;
+ __cap_set_timeouts(mdsc, ci);
+ }
+ }
+ tried_invalidate = 1;
+ goto retry_locked;
+ }
+
+ num = 0;
+ for (p = rb_first(&ci->i_caps); p; p = rb_next(p)) {
+ cap = rb_entry(p, struct ceph_cap, ci_node);
+ num++;
+
+ /* avoid looping forever */
+ if (mds >= cap->mds ||
+ ((flags & CHECK_CAPS_AUTHONLY) && cap != ci->i_auth_cap))
+ continue;
+
+ /* NOTE: no side-effects allowed, until we take s_mutex */
+
+ revoking = cap->implemented & ~cap->issued;
+ if (revoking)
+ dout(" mds%d revoking %s\n", cap->mds,
+ ceph_cap_string(revoking));
+
+ if (cap == ci->i_auth_cap &&
+ (cap->issued & CEPH_CAP_FILE_WR)) {
+ /* request larger max_size from MDS? */
+ if (ci->i_wanted_max_size > ci->i_max_size &&
+ ci->i_wanted_max_size > ci->i_requested_max_size) {
+ dout("requesting new max_size\n");
+ goto ack;
+ }
+
+ /* approaching file_max? */
+ if ((inode->i_size << 1) >= ci->i_max_size &&
+ (ci->i_reported_size << 1) < ci->i_max_size) {
+ dout("i_size approaching max_size\n");
+ goto ack;
+ }
+ }
+ /* flush anything dirty? */
+ if (cap == ci->i_auth_cap && (flags & CHECK_CAPS_FLUSH) &&
+ ci->i_dirty_caps) {
+ dout("flushing dirty caps\n");
+ goto ack;
+ }
+
+ /* completed revocation? going down and there are no caps? */
+ if (revoking && (revoking & used) == 0) {
+ dout("completed revocation of %s\n",
+ ceph_cap_string(cap->implemented & ~cap->issued));
+ goto ack;
+ }
+
+ /* want more caps from mds? */
+ if (want & ~(cap->mds_wanted | cap->issued))
+ goto ack;
+
+ /* things we might delay */
+ if ((cap->issued & ~retain) == 0 &&
+ cap->mds_wanted == want)
+ continue; /* nope, all good */
+
+ if (is_delayed)
+ goto ack;
+
+ /* delay? */
+ if ((ci->i_ceph_flags & CEPH_I_NODELAY) == 0 &&
+ time_before(jiffies, ci->i_hold_caps_max)) {
+ dout(" delaying issued %s -> %s, wanted %s -> %s\n",
+ ceph_cap_string(cap->issued),
+ ceph_cap_string(cap->issued & retain),
+ ceph_cap_string(cap->mds_wanted),
+ ceph_cap_string(want));
+ delayed++;
+ continue;
+ }
+
+ack:
+ if (ci->i_ceph_flags & CEPH_I_NOFLUSH) {
+ dout(" skipping %p I_NOFLUSH set\n", inode);
+ continue;
+ }
+
+ if (session && session != cap->session) {
+ dout("oops, wrong session %p mutex\n", session);
+ mutex_unlock(&session->s_mutex);
+ session = NULL;
+ }
+ if (!session) {
+ session = cap->session;
+ if (mutex_trylock(&session->s_mutex) == 0) {
+ dout("inverting session/ino locks on %p\n",
+ session);
+ spin_unlock(&inode->i_lock);
+ if (took_snap_rwsem) {
+ up_read(&mdsc->snap_rwsem);
+ took_snap_rwsem = 0;
+ }
+ mutex_lock(&session->s_mutex);
+ goto retry;
+ }
+ }
+ /* take snap_rwsem after session mutex */
+ if (!took_snap_rwsem) {
+ if (down_read_trylock(&mdsc->snap_rwsem) == 0) {
+ dout("inverting snap/in locks on %p\n",
+ inode);
+ spin_unlock(&inode->i_lock);
+ down_read(&mdsc->snap_rwsem);
+ took_snap_rwsem = 1;
+ goto retry;
+ }
+ took_snap_rwsem = 1;
+ }
+
+ if (cap == ci->i_auth_cap && ci->i_dirty_caps)
+ flushing = __mark_caps_flushing(inode, session);
+
+ mds = cap->mds; /* remember mds, so we don't repeat */
+ sent++;
+
+ /* __send_cap drops i_lock */
+ delayed += __send_cap(mdsc, cap, CEPH_CAP_OP_UPDATE, used, want,
+ retain, flushing, NULL);
+ goto retry; /* retake i_lock and restart our cap scan. */
+ }
+
+ /*
+ * Reschedule delayed caps release if we delayed anything,
+ * otherwise cancel.
+ */
+ if (delayed && is_delayed)
+ force_requeue = 1; /* __send_cap delayed release; requeue */
+ if (!delayed && !is_delayed)
+ __cap_delay_cancel(mdsc, ci);
+ else if (!is_delayed || force_requeue)
+ __cap_delay_requeue(mdsc, ci);
+
+ spin_unlock(&inode->i_lock);
+
+ if (queue_invalidate)
+ ceph_queue_invalidate(inode);
+
+ if (session)
+ mutex_unlock(&session->s_mutex);
+ if (took_snap_rwsem)
+ up_read(&mdsc->snap_rwsem);
+}
+
+/*
+ * Try to flush dirty caps back to the auth mds.
+ */
+static int try_flush_caps(struct inode *inode, struct ceph_mds_session *session,
+ unsigned *flush_tid)
+{
+ struct ceph_mds_client *mdsc = &ceph_client(inode->i_sb)->mdsc;
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ int unlock_session = session ? 0 : 1;
+ int flushing = 0;
+
+retry:
+ spin_lock(&inode->i_lock);
+ if (ci->i_ceph_flags & CEPH_I_NOFLUSH) {
+ dout("try_flush_caps skipping %p I_NOFLUSH set\n", inode);
+ goto out;
+ }
+ if (ci->i_dirty_caps && ci->i_auth_cap) {
+ struct ceph_cap *cap = ci->i_auth_cap;
+ int used = __ceph_caps_used(ci);
+ int want = __ceph_caps_wanted(ci);
+ int delayed;
+
+ if (!session) {
+ spin_unlock(&inode->i_lock);
+ session = cap->session;
+ mutex_lock(&session->s_mutex);
+ goto retry;
+ }
+ BUG_ON(session != cap->session);
+ if (cap->session->s_state < CEPH_MDS_SESSION_OPEN)
+ goto out;
+
+ flushing = __mark_caps_flushing(inode, session);
+
+ /* __send_cap drops i_lock */
+ delayed = __send_cap(mdsc, cap, CEPH_CAP_OP_FLUSH, used, want,
+ cap->issued | cap->implemented, flushing,
+ flush_tid);
+ if (!delayed)
+ goto out_unlocked;
+
+ spin_lock(&inode->i_lock);
+ __cap_delay_requeue(mdsc, ci);
+ }
+out:
+ spin_unlock(&inode->i_lock);
+out_unlocked:
+ if (session && unlock_session)
+ mutex_unlock(&session->s_mutex);
+ return flushing;
+}
+
+/*
+ * Return true if we've flushed caps through the given flush_tid.
+ */
+static int caps_are_flushed(struct inode *inode, unsigned tid)
+{
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ int dirty, i, ret = 1;
+
+ spin_lock(&inode->i_lock);
+ dirty = __ceph_caps_dirty(ci);
+ for (i = 0; i < CEPH_CAP_BITS; i++)
+ if ((ci->i_flushing_caps & (1 << i)) &&
+ ci->i_cap_flush_tid[i] <= tid) {
+ /* still flushing this bit */
+ ret = 0;
+ break;
+ }
+ spin_unlock(&inode->i_lock);
+ return ret;
+}
+
+/*
+ * Wait on any unsafe replies for the given inode. First wait on the
+ * newest request, and make that the upper bound. Then, if there are
+ * more requests, keep waiting on the oldest as long as it is still older
+ * than the original request.
+ */
+static void sync_write_wait(struct inode *inode)
+{
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ struct list_head *head = &ci->i_unsafe_writes;
+ struct ceph_osd_request *req;
+ u64 last_tid;
+
+ spin_lock(&ci->i_unsafe_lock);
+ if (list_empty(head))
+ goto out;
+
+ /* set upper bound as _last_ entry in chain */
+ req = list_entry(head->prev, struct ceph_osd_request,
+ r_unsafe_item);
+ last_tid = req->r_tid;
+
+ do {
+ ceph_osdc_get_request(req);
+ spin_unlock(&ci->i_unsafe_lock);
+ dout("sync_write_wait on tid %llu (until %llu)\n",
+ req->r_tid, last_tid);
+ wait_for_completion(&req->r_safe_completion);
+ spin_lock(&ci->i_unsafe_lock);
+ ceph_osdc_put_request(req);
+
+ /*
+ * from here on look at first entry in chain, since we
+ * only want to wait for anything older than last_tid
+ */
+ if (list_empty(head))
+ break;
+ req = list_entry(head->next, struct ceph_osd_request,
+ r_unsafe_item);
+ } while (req->r_tid < last_tid);
+out:
+ spin_unlock(&ci->i_unsafe_lock);
+}
+
+int ceph_fsync(struct file *file, struct dentry *dentry, int datasync)
+{
+ struct inode *inode = dentry->d_inode;
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ unsigned flush_tid;
+ int ret;
+ int dirty;
+
+ dout("fsync %p%s\n", inode, datasync ? " datasync" : "");
+ sync_write_wait(inode);
+
+ ret = filemap_write_and_wait(inode->i_mapping);
+ if (ret < 0)
+ return ret;
+
+ dirty = try_flush_caps(inode, NULL, &flush_tid);
+ dout("fsync dirty caps are %s\n", ceph_cap_string(dirty));
+
+ /*
+ * only wait on non-file metadata writeback (the mds
+ * can recover size and mtime, so we don't need to
+ * wait for that)
+ */
+ if (!datasync && (dirty & ~CEPH_CAP_ANY_FILE_WR)) {
+ dout("fsync waiting for flush_tid %u\n", flush_tid);
+ ret = wait_event_interruptible(ci->i_cap_wq,
+ caps_are_flushed(inode, flush_tid));
+ }
+
+ dout("fsync %p%s done\n", inode, datasync ? " datasync" : "");
+ return ret;
+}
+
+/*
+ * Flush any dirty caps back to the mds. If we aren't asked to wait,
+ * queue inode for flush but don't do so immediately, because we can
+ * get by with fewer MDS messages if we wait for data writeback to
+ * complete first.
+ */
+int ceph_write_inode(struct inode *inode, struct writeback_control *wbc)
+{
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ unsigned flush_tid;
+ int err = 0;
+ int dirty;
+ int wait = wbc->sync_mode == WB_SYNC_ALL;
+
+ dout("write_inode %p wait=%d\n", inode, wait);
+ if (wait) {
+ dirty = try_flush_caps(inode, NULL, &flush_tid);
+ if (dirty)
+ err = wait_event_interruptible(ci->i_cap_wq,
+ caps_are_flushed(inode, flush_tid));
+ } else {
+ struct ceph_mds_client *mdsc = &ceph_client(inode->i_sb)->mdsc;
+
+ spin_lock(&inode->i_lock);
+ if (__ceph_caps_dirty(ci))
+ __cap_delay_requeue_front(mdsc, ci);
+ spin_unlock(&inode->i_lock);
+ }
+ return err;
+}
+
+/*
+ * After a recovering MDS goes active, we need to resend any caps
+ * we were flushing.
+ *
+ * Caller holds session->s_mutex.
+ */
+static void kick_flushing_capsnaps(struct ceph_mds_client *mdsc,
+ struct ceph_mds_session *session)
+{
+ struct ceph_cap_snap *capsnap;
+
+ dout("kick_flushing_capsnaps mds%d\n", session->s_mds);
+ list_for_each_entry(capsnap, &session->s_cap_snaps_flushing,
+ flushing_item) {
+ struct ceph_inode_info *ci = capsnap->ci;
+ struct inode *inode = &ci->vfs_inode;
+ struct ceph_cap *cap;
+
+ spin_lock(&inode->i_lock);
+ cap = ci->i_auth_cap;
+ if (cap && cap->session == session) {
+ dout("kick_flushing_caps %p cap %p capsnap %p\n", inode,
+ cap, capsnap);
+ __ceph_flush_snaps(ci, &session);
+ } else {
+ pr_err("%p auth cap %p not mds%d ???\n", inode,
+ cap, session->s_mds);
+ spin_unlock(&inode->i_lock);
+ }
+ }
+}
+
+void ceph_kick_flushing_caps(struct ceph_mds_client *mdsc,
+ struct ceph_mds_session *session)
+{
+ struct ceph_inode_info *ci;
+
+ kick_flushing_capsnaps(mdsc, session);
+
+ dout("kick_flushing_caps mds%d\n", session->s_mds);
+ list_for_each_entry(ci, &session->s_cap_flushing, i_flushing_item) {
+ struct inode *inode = &ci->vfs_inode;
+ struct ceph_cap *cap;
+ int delayed = 0;
+
+ spin_lock(&inode->i_lock);
+ cap = ci->i_auth_cap;
+ if (cap && cap->session == session) {
+ dout("kick_flushing_caps %p cap %p %s\n", inode,
+ cap, ceph_cap_string(ci->i_flushing_caps));
+ delayed = __send_cap(mdsc, cap, CEPH_CAP_OP_FLUSH,
+ __ceph_caps_used(ci),
+ __ceph_caps_wanted(ci),
+ cap->issued | cap->implemented,
+ ci->i_flushing_caps, NULL);
+ if (delayed) {
+ spin_lock(&inode->i_lock);
+ __cap_delay_requeue(mdsc, ci);
+ spin_unlock(&inode->i_lock);
+ }
+ } else {
+ pr_err("%p auth cap %p not mds%d ???\n", inode,
+ cap, session->s_mds);
+ spin_unlock(&inode->i_lock);
+ }
+ }
+}
+
+
+/*
+ * Take references to capabilities we hold, so that we don't release
+ * them to the MDS prematurely.
+ *
+ * Protected by i_lock.
+ */
+static void __take_cap_refs(struct ceph_inode_info *ci, int got)
+{
+ if (got & CEPH_CAP_PIN)
+ ci->i_pin_ref++;
+ if (got & CEPH_CAP_FILE_RD)
+ ci->i_rd_ref++;
+ if (got & CEPH_CAP_FILE_CACHE)
+ ci->i_rdcache_ref++;
+ if (got & CEPH_CAP_FILE_WR)
+ ci->i_wr_ref++;
+ if (got & CEPH_CAP_FILE_BUFFER) {
+ if (ci->i_wrbuffer_ref == 0)
+ igrab(&ci->vfs_inode);
+ ci->i_wrbuffer_ref++;
+ dout("__take_cap_refs %p wrbuffer %d -> %d (?)\n",
+ &ci->vfs_inode, ci->i_wrbuffer_ref-1, ci->i_wrbuffer_ref);
+ }
+}
+
+/*
+ * Try to grab cap references. Specify those refs we @want, and the
+ * minimal set we @need. Also include the larger offset we are writing
+ * to (when applicable), and check against max_size here as well.
+ * Note that caller is responsible for ensuring max_size increases are
+ * requested from the MDS.
+ */
+static int try_get_cap_refs(struct ceph_inode_info *ci, int need, int want,
+ int *got, loff_t endoff, int *check_max, int *err)
+{
+ struct inode *inode = &ci->vfs_inode;
+ int ret = 0;
+ int have, implemented;
+ int file_wanted;
+
+ dout("get_cap_refs %p need %s want %s\n", inode,
+ ceph_cap_string(need), ceph_cap_string(want));
+ spin_lock(&inode->i_lock);
+
+ /* make sure file is actually open */
+ file_wanted = __ceph_caps_file_wanted(ci);
+ if ((file_wanted & need) == 0) {
+ dout("try_get_cap_refs need %s file_wanted %s, EBADF\n",
+ ceph_cap_string(need), ceph_cap_string(file_wanted));
+ *err = -EBADF;
+ ret = 1;
+ goto out;
+ }
+
+ if (need & CEPH_CAP_FILE_WR) {
+ if (endoff >= 0 && endoff > (loff_t)ci->i_max_size) {
+ dout("get_cap_refs %p endoff %llu > maxsize %llu\n",
+ inode, endoff, ci->i_max_size);
+ if (endoff > ci->i_wanted_max_size) {
+ *check_max = 1;
+ ret = 1;
+ }
+ goto out;
+ }
+ /*
+ * If a sync write is in progress, we must wait, so that we
+ * can get a final snapshot value for size+mtime.
+ */
+ if (__ceph_have_pending_cap_snap(ci)) {
+ dout("get_cap_refs %p cap_snap_pending\n", inode);
+ goto out;
+ }
+ }
+ have = __ceph_caps_issued(ci, &implemented);
+
+ /*
+ * disallow writes while a truncate is pending
+ */
+ if (ci->i_truncate_pending)
+ have &= ~CEPH_CAP_FILE_WR;
+
+ if ((have & need) == need) {
+ /*
+ * Look at (implemented & ~have & not) so that we keep waiting
+ * on transition from wanted -> needed caps. This is needed
+ * for WRBUFFER|WR -> WR to avoid a new WR sync write from
+ * going before a prior buffered writeback happens.
+ */
+ int not = want & ~(have & need);
+ int revoking = implemented & ~have;
+ dout("get_cap_refs %p have %s but not %s (revoking %s)\n",
+ inode, ceph_cap_string(have), ceph_cap_string(not),
+ ceph_cap_string(revoking));
+ if ((revoking & not) == 0) {
+ *got = need | (have & want);
+ __take_cap_refs(ci, *got);
+ ret = 1;
+ }
+ } else {
+ dout("get_cap_refs %p have %s needed %s\n", inode,
+ ceph_cap_string(have), ceph_cap_string(need));
+ }
+out:
+ spin_unlock(&inode->i_lock);
+ dout("get_cap_refs %p ret %d got %s\n", inode,
+ ret, ceph_cap_string(*got));
+ return ret;
+}
+
+/*
+ * Check the offset we are writing up to against our current
+ * max_size. If necessary, tell the MDS we want to write to
+ * a larger offset.
+ */
+static void check_max_size(struct inode *inode, loff_t endoff)
+{
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ int check = 0;
+
+ /* do we need to explicitly request a larger max_size? */
+ spin_lock(&inode->i_lock);
+ if ((endoff >= ci->i_max_size ||
+ endoff > (inode->i_size << 1)) &&
+ endoff > ci->i_wanted_max_size) {
+ dout("write %p at large endoff %llu, req max_size\n",
+ inode, endoff);
+ ci->i_wanted_max_size = endoff;
+ check = 1;
+ }
+ spin_unlock(&inode->i_lock);
+ if (check)
+ ceph_check_caps(ci, CHECK_CAPS_AUTHONLY, NULL);
+}
+
+/*
+ * Wait for caps, and take cap references. If we can't get a WR cap
+ * due to a small max_size, make sure we check_max_size (and possibly
+ * ask the mds) so we don't get hung up indefinitely.
+ */
+int ceph_get_caps(struct ceph_inode_info *ci, int need, int want, int *got,
+ loff_t endoff)
+{
+ int check_max, ret, err;
+
+retry:
+ if (endoff > 0)
+ check_max_size(&ci->vfs_inode, endoff);
+ check_max = 0;
+ err = 0;
+ ret = wait_event_interruptible(ci->i_cap_wq,
+ try_get_cap_refs(ci, need, want,
+ got, endoff,
+ &check_max, &err));
+ if (err)
+ ret = err;
+ if (check_max)
+ goto retry;
+ return ret;
+}
+
+/*
+ * Take cap refs. Caller must already know we hold at least one ref
+ * on the caps in question or we don't know this is safe.
+ */
+void ceph_get_cap_refs(struct ceph_inode_info *ci, int caps)
+{
+ spin_lock(&ci->vfs_inode.i_lock);
+ __take_cap_refs(ci, caps);
+ spin_unlock(&ci->vfs_inode.i_lock);
+}
+
+/*
+ * Release cap refs.
+ *
+ * If we released the last ref on any given cap, call ceph_check_caps
+ * to release (or schedule a release).
+ *
+ * If we are releasing a WR cap (from a sync write), finalize any affected
+ * cap_snap, and wake up any waiters.
+ */
+void ceph_put_cap_refs(struct ceph_inode_info *ci, int had)
+{
+ struct inode *inode = &ci->vfs_inode;
+ int last = 0, put = 0, flushsnaps = 0, wake = 0;
+ struct ceph_cap_snap *capsnap;
+
+ spin_lock(&inode->i_lock);
+ if (had & CEPH_CAP_PIN)
+ --ci->i_pin_ref;
+ if (had & CEPH_CAP_FILE_RD)
+ if (--ci->i_rd_ref == 0)
+ last++;
+ if (had & CEPH_CAP_FILE_CACHE)
+ if (--ci->i_rdcache_ref == 0)
+ last++;
+ if (had & CEPH_CAP_FILE_BUFFER) {
+ if (--ci->i_wrbuffer_ref == 0) {
+ last++;
+ put++;
+ }
+ dout("put_cap_refs %p wrbuffer %d -> %d (?)\n",
+ inode, ci->i_wrbuffer_ref+1, ci->i_wrbuffer_ref);
+ }
+ if (had & CEPH_CAP_FILE_WR)
+ if (--ci->i_wr_ref == 0) {
+ last++;
+ if (!list_empty(&ci->i_cap_snaps)) {
+ capsnap = list_first_entry(&ci->i_cap_snaps,
+ struct ceph_cap_snap,
+ ci_item);
+ if (capsnap->writing) {
+ capsnap->writing = 0;
+ flushsnaps =
+ __ceph_finish_cap_snap(ci,
+ capsnap);
+ wake = 1;
+ }
+ }
+ }
+ spin_unlock(&inode->i_lock);
+
+ dout("put_cap_refs %p had %s %s\n", inode, ceph_cap_string(had),
+ last ? "last" : "");
+
+ if (last && !flushsnaps)
+ ceph_check_caps(ci, 0, NULL);
+ else if (flushsnaps)
+ ceph_flush_snaps(ci);
+ if (wake)
+ wake_up(&ci->i_cap_wq);
+ if (put)
+ iput(inode);
+}
+
+/*
+ * Release @nr WRBUFFER refs on dirty pages for the given @snapc snap
+ * context. Adjust per-snap dirty page accounting as appropriate.
+ * Once all dirty data for a cap_snap is flushed, flush snapped file
+ * metadata back to the MDS. If we dropped the last ref, call
+ * ceph_check_caps.
+ */
+void ceph_put_wrbuffer_cap_refs(struct ceph_inode_info *ci, int nr,
+ struct ceph_snap_context *snapc)
+{
+ struct inode *inode = &ci->vfs_inode;
+ int last = 0;
+ int last_snap = 0;
+ int found = 0;
+ struct ceph_cap_snap *capsnap = NULL;
+
+ spin_lock(&inode->i_lock);
+ ci->i_wrbuffer_ref -= nr;
+ last = !ci->i_wrbuffer_ref;
+
+ if (ci->i_head_snapc == snapc) {
+ ci->i_wrbuffer_ref_head -= nr;
+ if (!ci->i_wrbuffer_ref_head) {
+ ceph_put_snap_context(ci->i_head_snapc);
+ ci->i_head_snapc = NULL;
+ }
+ dout("put_wrbuffer_cap_refs on %p head %d/%d -> %d/%d %s\n",
+ inode,
+ ci->i_wrbuffer_ref+nr, ci->i_wrbuffer_ref_head+nr,
+ ci->i_wrbuffer_ref, ci->i_wrbuffer_ref_head,
+ last ? " LAST" : "");
+ } else {
+ list_for_each_entry(capsnap, &ci->i_cap_snaps, ci_item) {
+ if (capsnap->context == snapc) {
+ found = 1;
+ capsnap->dirty_pages -= nr;
+ last_snap = !capsnap->dirty_pages;
+ break;
+ }
+ }
+ BUG_ON(!found);
+ dout("put_wrbuffer_cap_refs on %p cap_snap %p "
+ " snap %lld %d/%d -> %d/%d %s%s\n",
+ inode, capsnap, capsnap->context->seq,
+ ci->i_wrbuffer_ref+nr, capsnap->dirty_pages + nr,
+ ci->i_wrbuffer_ref, capsnap->dirty_pages,
+ last ? " (wrbuffer last)" : "",
+ last_snap ? " (capsnap last)" : "");
+ }
+
+ spin_unlock(&inode->i_lock);
+
+ if (last) {
+ ceph_check_caps(ci, CHECK_CAPS_AUTHONLY, NULL);
+ iput(inode);
+ } else if (last_snap) {
+ ceph_flush_snaps(ci);
+ wake_up(&ci->i_cap_wq);
+ }
+}
+
+/*
+ * Handle a cap GRANT message from the MDS. (Note that a GRANT may
+ * actually be a revocation if it specifies a smaller cap set.)
+ *
+ * caller holds s_mutex and i_lock, we drop both.
+ *
+ * return value:
+ * 0 - ok
+ * 1 - check_caps on auth cap only (writeback)
+ * 2 - check_caps (ack revoke)
+ */
+static void handle_cap_grant(struct inode *inode, struct ceph_mds_caps *grant,
+ struct ceph_mds_session *session,
+ struct ceph_cap *cap,
+ struct ceph_buffer *xattr_buf)
+ __releases(inode->i_lock)
+ __releases(session->s_mutex)
+{
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ int mds = session->s_mds;
+ int seq = le32_to_cpu(grant->seq);
+ int newcaps = le32_to_cpu(grant->caps);
+ int issued, implemented, used, wanted, dirty;
+ u64 size = le64_to_cpu(grant->size);
+ u64 max_size = le64_to_cpu(grant->max_size);
+ struct timespec mtime, atime, ctime;
+ int check_caps = 0;
+ int wake = 0;
+ int writeback = 0;
+ int revoked_rdcache = 0;
+ int queue_invalidate = 0;
+
+ dout("handle_cap_grant inode %p cap %p mds%d seq %d %s\n",
+ inode, cap, mds, seq, ceph_cap_string(newcaps));
+ dout(" size %llu max_size %llu, i_size %llu\n", size, max_size,
+ inode->i_size);
+
+ /*
+ * If CACHE is being revoked, and we have no dirty buffers,
+ * try to invalidate (once). (If there are dirty buffers, we
+ * will invalidate _after_ writeback.)
+ */
+ if (((cap->issued & ~newcaps) & CEPH_CAP_FILE_CACHE) &&
+ !ci->i_wrbuffer_ref) {
+ if (try_nonblocking_invalidate(inode) == 0) {
+ revoked_rdcache = 1;
+ } else {
+ /* there were locked pages.. invalidate later
+ in a separate thread. */
+ if (ci->i_rdcache_revoking != ci->i_rdcache_gen) {
+ queue_invalidate = 1;
+ ci->i_rdcache_revoking = ci->i_rdcache_gen;
+ }
+ }
+ }
+
+ /* side effects now are allowed */
+
+ issued = __ceph_caps_issued(ci, &implemented);
+ issued |= implemented | __ceph_caps_dirty(ci);
+
+ cap->cap_gen = session->s_cap_gen;
+
+ __check_cap_issue(ci, cap, newcaps);
+
+ if ((issued & CEPH_CAP_AUTH_EXCL) == 0) {
+ inode->i_mode = le32_to_cpu(grant->mode);
+ inode->i_uid = le32_to_cpu(grant->uid);
+ inode->i_gid = le32_to_cpu(grant->gid);
+ dout("%p mode 0%o uid.gid %d.%d\n", inode, inode->i_mode,
+ inode->i_uid, inode->i_gid);
+ }
+
+ if ((issued & CEPH_CAP_LINK_EXCL) == 0)
+ inode->i_nlink = le32_to_cpu(grant->nlink);
+
+ if ((issued & CEPH_CAP_XATTR_EXCL) == 0 && grant->xattr_len) {
+ int len = le32_to_cpu(grant->xattr_len);
+ u64 version = le64_to_cpu(grant->xattr_version);
+
+ if (version > ci->i_xattrs.version) {
+ dout(" got new xattrs v%llu on %p len %d\n",
+ version, inode, len);
+ if (ci->i_xattrs.blob)
+ ceph_buffer_put(ci->i_xattrs.blob);
+ ci->i_xattrs.blob = ceph_buffer_get(xattr_buf);
+ ci->i_xattrs.version = version;
+ }
+ }
+
+ /* size/ctime/mtime/atime? */
+ ceph_fill_file_size(inode, issued,
+ le32_to_cpu(grant->truncate_seq),
+ le64_to_cpu(grant->truncate_size), size);
+ ceph_decode_timespec(&mtime, &grant->mtime);
+ ceph_decode_timespec(&atime, &grant->atime);
+ ceph_decode_timespec(&ctime, &grant->ctime);
+ ceph_fill_file_time(inode, issued,
+ le32_to_cpu(grant->time_warp_seq), &ctime, &mtime,
+ &atime);
+
+ /* max size increase? */
+ if (max_size != ci->i_max_size) {
+ dout("max_size %lld -> %llu\n", ci->i_max_size, max_size);
+ ci->i_max_size = max_size;
+ if (max_size >= ci->i_wanted_max_size) {
+ ci->i_wanted_max_size = 0; /* reset */
+ ci->i_requested_max_size = 0;
+ }
+ wake = 1;
+ }
+
+ /* check cap bits */
+ wanted = __ceph_caps_wanted(ci);
+ used = __ceph_caps_used(ci);
+ dirty = __ceph_caps_dirty(ci);
+ dout(" my wanted = %s, used = %s, dirty %s\n",
+ ceph_cap_string(wanted),
+ ceph_cap_string(used),
+ ceph_cap_string(dirty));
+ if (wanted != le32_to_cpu(grant->wanted)) {
+ dout("mds wanted %s -> %s\n",
+ ceph_cap_string(le32_to_cpu(grant->wanted)),
+ ceph_cap_string(wanted));
+ grant->wanted = cpu_to_le32(wanted);
+ }
+
+ cap->seq = seq;
+
+ /* file layout may have changed */
+ ci->i_layout = grant->layout;
+
+ /* revocation, grant, or no-op? */
+ if (cap->issued & ~newcaps) {
+ dout("revocation: %s -> %s\n", ceph_cap_string(cap->issued),
+ ceph_cap_string(newcaps));
+ if ((used & ~newcaps) & CEPH_CAP_FILE_BUFFER)
+ writeback = 1; /* will delay ack */
+ else if (dirty & ~newcaps)
+ check_caps = 1; /* initiate writeback in check_caps */
+ else if (((used & ~newcaps) & CEPH_CAP_FILE_CACHE) == 0 ||
+ revoked_rdcache)
+ check_caps = 2; /* send revoke ack in check_caps */
+ cap->issued = newcaps;
+ cap->implemented |= newcaps;
+ } else if (cap->issued == newcaps) {
+ dout("caps unchanged: %s -> %s\n",
+ ceph_cap_string(cap->issued), ceph_cap_string(newcaps));
+ } else {
+ dout("grant: %s -> %s\n", ceph_cap_string(cap->issued),
+ ceph_cap_string(newcaps));
+ cap->issued = newcaps;
+ cap->implemented |= newcaps; /* add bits only, to
+ * avoid stepping on a
+ * pending revocation */
+ wake = 1;
+ }
+ BUG_ON(cap->issued & ~cap->implemented);
+
+ spin_unlock(&inode->i_lock);
+ if (writeback)
+ /*
+ * queue inode for writeback: we can't actually call
+ * filemap_write_and_wait, etc. from message handler
+ * context.
+ */
+ ceph_queue_writeback(inode);
+ if (queue_invalidate)
+ ceph_queue_invalidate(inode);
+ if (wake)
+ wake_up(&ci->i_cap_wq);
+
+ if (check_caps == 1)
+ ceph_check_caps(ci, CHECK_CAPS_NODELAY|CHECK_CAPS_AUTHONLY,
+ session);
+ else if (check_caps == 2)
+ ceph_check_caps(ci, CHECK_CAPS_NODELAY, session);
+ else
+ mutex_unlock(&session->s_mutex);
+}
+
+/*
+ * Handle FLUSH_ACK from MDS, indicating that metadata we sent to the
+ * MDS has been safely committed.
+ */
+static void handle_cap_flush_ack(struct inode *inode, u64 flush_tid,
+ struct ceph_mds_caps *m,
+ struct ceph_mds_session *session,
+ struct ceph_cap *cap)
+ __releases(inode->i_lock)
+{
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ struct ceph_mds_client *mdsc = &ceph_client(inode->i_sb)->mdsc;
+ unsigned seq = le32_to_cpu(m->seq);
+ int dirty = le32_to_cpu(m->dirty);
+ int cleaned = 0;
+ int drop = 0;
+ int i;
+
+ for (i = 0; i < CEPH_CAP_BITS; i++)
+ if ((dirty & (1 << i)) &&
+ flush_tid == ci->i_cap_flush_tid[i])
+ cleaned |= 1 << i;
+
+ dout("handle_cap_flush_ack inode %p mds%d seq %d on %s cleaned %s,"
+ " flushing %s -> %s\n",
+ inode, session->s_mds, seq, ceph_cap_string(dirty),
+ ceph_cap_string(cleaned), ceph_cap_string(ci->i_flushing_caps),
+ ceph_cap_string(ci->i_flushing_caps & ~cleaned));
+
+ if (ci->i_flushing_caps == (ci->i_flushing_caps & ~cleaned))
+ goto out;
+
+ ci->i_flushing_caps &= ~cleaned;
+
+ spin_lock(&mdsc->cap_dirty_lock);
+ if (ci->i_flushing_caps == 0) {
+ list_del_init(&ci->i_flushing_item);
+ if (!list_empty(&session->s_cap_flushing))
+ dout(" mds%d still flushing cap on %p\n",
+ session->s_mds,
+ &list_entry(session->s_cap_flushing.next,
+ struct ceph_inode_info,
+ i_flushing_item)->vfs_inode);
+ mdsc->num_cap_flushing--;
+ wake_up(&mdsc->cap_flushing_wq);
+ dout(" inode %p now !flushing\n", inode);
+
+ if (ci->i_dirty_caps == 0) {
+ dout(" inode %p now clean\n", inode);
+ BUG_ON(!list_empty(&ci->i_dirty_item));
+ drop = 1;
+ } else {
+ BUG_ON(list_empty(&ci->i_dirty_item));
+ }
+ }
+ spin_unlock(&mdsc->cap_dirty_lock);
+ wake_up(&ci->i_cap_wq);
+
+out:
+ spin_unlock(&inode->i_lock);
+ if (drop)
+ iput(inode);
+}
+
+/*
+ * Handle FLUSHSNAP_ACK. MDS has flushed snap data to disk and we can
+ * throw away our cap_snap.
+ *
+ * Caller hold s_mutex.
+ */
+static void handle_cap_flushsnap_ack(struct inode *inode, u64 flush_tid,
+ struct ceph_mds_caps *m,
+ struct ceph_mds_session *session)
+{
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ u64 follows = le64_to_cpu(m->snap_follows);
+ struct ceph_cap_snap *capsnap;
+ int drop = 0;
+
+ dout("handle_cap_flushsnap_ack inode %p ci %p mds%d follows %lld\n",
+ inode, ci, session->s_mds, follows);
+
+ spin_lock(&inode->i_lock);
+ list_for_each_entry(capsnap, &ci->i_cap_snaps, ci_item) {
+ if (capsnap->follows == follows) {
+ if (capsnap->flush_tid != flush_tid) {
+ dout(" cap_snap %p follows %lld tid %lld !="
+ " %lld\n", capsnap, follows,
+ flush_tid, capsnap->flush_tid);
+ break;
+ }
+ WARN_ON(capsnap->dirty_pages || capsnap->writing);
+ dout(" removing cap_snap %p follows %lld\n",
+ capsnap, follows);
+ ceph_put_snap_context(capsnap->context);
+ list_del(&capsnap->ci_item);
+ list_del(&capsnap->flushing_item);
+ ceph_put_cap_snap(capsnap);
+ drop = 1;
+ break;
+ } else {
+ dout(" skipping cap_snap %p follows %lld\n",
+ capsnap, capsnap->follows);
+ }
+ }
+ spin_unlock(&inode->i_lock);
+ if (drop)
+ iput(inode);
+}
+
+/*
+ * Handle TRUNC from MDS, indicating file truncation.
+ *
+ * caller hold s_mutex.
+ */
+static void handle_cap_trunc(struct inode *inode,
+ struct ceph_mds_caps *trunc,
+ struct ceph_mds_session *session)
+ __releases(inode->i_lock)
+{
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ int mds = session->s_mds;
+ int seq = le32_to_cpu(trunc->seq);
+ u32 truncate_seq = le32_to_cpu(trunc->truncate_seq);
+ u64 truncate_size = le64_to_cpu(trunc->truncate_size);
+ u64 size = le64_to_cpu(trunc->size);
+ int implemented = 0;
+ int dirty = __ceph_caps_dirty(ci);
+ int issued = __ceph_caps_issued(ceph_inode(inode), &implemented);
+ int queue_trunc = 0;
+
+ issued |= implemented | dirty;
+
+ dout("handle_cap_trunc inode %p mds%d seq %d to %lld seq %d\n",
+ inode, mds, seq, truncate_size, truncate_seq);
+ queue_trunc = ceph_fill_file_size(inode, issued,
+ truncate_seq, truncate_size, size);
+ spin_unlock(&inode->i_lock);
+
+ if (queue_trunc)
+ ceph_queue_vmtruncate(inode);
+}
+
+/*
+ * Handle EXPORT from MDS. Cap is being migrated _from_ this mds to a
+ * different one. If we are the most recent migration we've seen (as
+ * indicated by mseq), make note of the migrating cap bits for the
+ * duration (until we see the corresponding IMPORT).
+ *
+ * caller holds s_mutex
+ */
+static void handle_cap_export(struct inode *inode, struct ceph_mds_caps *ex,
+ struct ceph_mds_session *session)
+{
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ int mds = session->s_mds;
+ unsigned mseq = le32_to_cpu(ex->migrate_seq);
+ struct ceph_cap *cap = NULL, *t;
+ struct rb_node *p;
+ int remember = 1;
+
+ dout("handle_cap_export inode %p ci %p mds%d mseq %d\n",
+ inode, ci, mds, mseq);
+
+ spin_lock(&inode->i_lock);
+
+ /* make sure we haven't seen a higher mseq */
+ for (p = rb_first(&ci->i_caps); p; p = rb_next(p)) {
+ t = rb_entry(p, struct ceph_cap, ci_node);
+ if (ceph_seq_cmp(t->mseq, mseq) > 0) {
+ dout(" higher mseq on cap from mds%d\n",
+ t->session->s_mds);
+ remember = 0;
+ }
+ if (t->session->s_mds == mds)
+ cap = t;
+ }
+
+ if (cap) {
+ if (remember) {
+ /* make note */
+ ci->i_cap_exporting_mds = mds;
+ ci->i_cap_exporting_mseq = mseq;
+ ci->i_cap_exporting_issued = cap->issued;
+ }
+ __ceph_remove_cap(cap);
+ }
+ /* else, we already released it */
+
+ spin_unlock(&inode->i_lock);
+}
+
+/*
+ * Handle cap IMPORT. If there are temp bits from an older EXPORT,
+ * clean them up.
+ *
+ * caller holds s_mutex.
+ */
+static void handle_cap_import(struct ceph_mds_client *mdsc,
+ struct inode *inode, struct ceph_mds_caps *im,
+ struct ceph_mds_session *session,
+ void *snaptrace, int snaptrace_len)
+{
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ int mds = session->s_mds;
+ unsigned issued = le32_to_cpu(im->caps);
+ unsigned wanted = le32_to_cpu(im->wanted);
+ unsigned seq = le32_to_cpu(im->seq);
+ unsigned mseq = le32_to_cpu(im->migrate_seq);
+ u64 realmino = le64_to_cpu(im->realm);
+ u64 cap_id = le64_to_cpu(im->cap_id);
+
+ if (ci->i_cap_exporting_mds >= 0 &&
+ ceph_seq_cmp(ci->i_cap_exporting_mseq, mseq) < 0) {
+ dout("handle_cap_import inode %p ci %p mds%d mseq %d"
+ " - cleared exporting from mds%d\n",
+ inode, ci, mds, mseq,
+ ci->i_cap_exporting_mds);
+ ci->i_cap_exporting_issued = 0;
+ ci->i_cap_exporting_mseq = 0;
+ ci->i_cap_exporting_mds = -1;
+ } else {
+ dout("handle_cap_import inode %p ci %p mds%d mseq %d\n",
+ inode, ci, mds, mseq);
+ }
+
+ down_write(&mdsc->snap_rwsem);
+ ceph_update_snap_trace(mdsc, snaptrace, snaptrace+snaptrace_len,
+ false);
+ downgrade_write(&mdsc->snap_rwsem);
+ ceph_add_cap(inode, session, cap_id, -1,
+ issued, wanted, seq, mseq, realmino, CEPH_CAP_FLAG_AUTH,
+ NULL /* no caps context */);
+ try_flush_caps(inode, session, NULL);
+ up_read(&mdsc->snap_rwsem);
+}
+
+/*
+ * Handle a caps message from the MDS.
+ *
+ * Identify the appropriate session, inode, and call the right handler
+ * based on the cap op.
+ */
+void ceph_handle_caps(struct ceph_mds_session *session,
+ struct ceph_msg *msg)
+{
+ struct ceph_mds_client *mdsc = session->s_mdsc;
+ struct super_block *sb = mdsc->client->sb;
+ struct inode *inode;
+ struct ceph_cap *cap;
+ struct ceph_mds_caps *h;
+ int mds = session->s_mds;
+ int op;
+ u32 seq;
+ struct ceph_vino vino;
+ u64 cap_id;
+ u64 size, max_size;
+ u64 tid;
+ void *snaptrace;
+
+ dout("handle_caps from mds%d\n", mds);
+
+ /* decode */
+ tid = le64_to_cpu(msg->hdr.tid);
+ if (msg->front.iov_len < sizeof(*h))
+ goto bad;
+ h = msg->front.iov_base;
+ snaptrace = h + 1;
+ op = le32_to_cpu(h->op);
+ vino.ino = le64_to_cpu(h->ino);
+ vino.snap = CEPH_NOSNAP;
+ cap_id = le64_to_cpu(h->cap_id);
+ seq = le32_to_cpu(h->seq);
+ size = le64_to_cpu(h->size);
+ max_size = le64_to_cpu(h->max_size);
+
+ mutex_lock(&session->s_mutex);
+ session->s_seq++;
+ dout(" mds%d seq %lld cap seq %u\n", session->s_mds, session->s_seq,
+ (unsigned)seq);
+
+ /* lookup ino */
+ inode = ceph_find_inode(sb, vino);
+ dout(" op %s ino %llx.%llx inode %p\n", ceph_cap_op_name(op), vino.ino,
+ vino.snap, inode);
+ if (!inode) {
+ dout(" i don't have ino %llx\n", vino.ino);
+ goto done;
+ }
+
+ /* these will work even if we don't have a cap yet */
+ switch (op) {
+ case CEPH_CAP_OP_FLUSHSNAP_ACK:
+ handle_cap_flushsnap_ack(inode, tid, h, session);
+ goto done;
+
+ case CEPH_CAP_OP_EXPORT:
+ handle_cap_export(inode, h, session);
+ goto done;
+
+ case CEPH_CAP_OP_IMPORT:
+ handle_cap_import(mdsc, inode, h, session,
+ snaptrace, le32_to_cpu(h->snap_trace_len));
+ ceph_check_caps(ceph_inode(inode), CHECK_CAPS_NODELAY,
+ session);
+ goto done_unlocked;
+ }
+
+ /* the rest require a cap */
+ spin_lock(&inode->i_lock);
+ cap = __get_cap_for_mds(ceph_inode(inode), mds);
+ if (!cap) {
+ dout("no cap on %p ino %llx.%llx from mds%d, releasing\n",
+ inode, ceph_ino(inode), ceph_snap(inode), mds);
+ spin_unlock(&inode->i_lock);
+ goto done;
+ }
+
+ /* note that each of these drops i_lock for us */
+ switch (op) {
+ case CEPH_CAP_OP_REVOKE:
+ case CEPH_CAP_OP_GRANT:
+ handle_cap_grant(inode, h, session, cap, msg->middle);
+ goto done_unlocked;
+
+ case CEPH_CAP_OP_FLUSH_ACK:
+ handle_cap_flush_ack(inode, tid, h, session, cap);
+ break;
+
+ case CEPH_CAP_OP_TRUNC:
+ handle_cap_trunc(inode, h, session);
+ break;
+
+ default:
+ spin_unlock(&inode->i_lock);
+ pr_err("ceph_handle_caps: unknown cap op %d %s\n", op,
+ ceph_cap_op_name(op));
+ }
+
+done:
+ mutex_unlock(&session->s_mutex);
+done_unlocked:
+ if (inode)
+ iput(inode);
+ return;
+
+bad:
+ pr_err("ceph_handle_caps: corrupt message\n");
+ ceph_msg_dump(msg);
+ return;
+}
+
+/*
+ * Delayed work handler to process end of delayed cap release LRU list.
+ */
+void ceph_check_delayed_caps(struct ceph_mds_client *mdsc)
+{
+ struct ceph_inode_info *ci;
+ int flags = CHECK_CAPS_NODELAY;
+
+ dout("check_delayed_caps\n");
+ while (1) {
+ spin_lock(&mdsc->cap_delay_lock);
+ if (list_empty(&mdsc->cap_delay_list))
+ break;
+ ci = list_first_entry(&mdsc->cap_delay_list,
+ struct ceph_inode_info,
+ i_cap_delay_list);
+ if ((ci->i_ceph_flags & CEPH_I_FLUSH) == 0 &&
+ time_before(jiffies, ci->i_hold_caps_max))
+ break;
+ list_del_init(&ci->i_cap_delay_list);
+ spin_unlock(&mdsc->cap_delay_lock);
+ dout("check_delayed_caps on %p\n", &ci->vfs_inode);
+ ceph_check_caps(ci, flags, NULL);
+ }
+ spin_unlock(&mdsc->cap_delay_lock);
+}
+
+/*
+ * Flush all dirty caps to the mds
+ */
+void ceph_flush_dirty_caps(struct ceph_mds_client *mdsc)
+{
+ struct ceph_inode_info *ci, *nci = NULL;
+ struct inode *inode, *ninode = NULL;
+ struct list_head *p, *n;
+
+ dout("flush_dirty_caps\n");
+ spin_lock(&mdsc->cap_dirty_lock);
+ list_for_each_safe(p, n, &mdsc->cap_dirty) {
+ if (nci) {
+ ci = nci;
+ inode = ninode;
+ ci->i_ceph_flags &= ~CEPH_I_NOFLUSH;
+ dout("flush_dirty_caps inode %p (was next inode)\n",
+ inode);
+ } else {
+ ci = list_entry(p, struct ceph_inode_info,
+ i_dirty_item);
+ inode = igrab(&ci->vfs_inode);
+ BUG_ON(!inode);
+ dout("flush_dirty_caps inode %p\n", inode);
+ }
+ if (n != &mdsc->cap_dirty) {
+ nci = list_entry(n, struct ceph_inode_info,
+ i_dirty_item);
+ ninode = igrab(&nci->vfs_inode);
+ BUG_ON(!ninode);
+ nci->i_ceph_flags |= CEPH_I_NOFLUSH;
+ dout("flush_dirty_caps next inode %p, noflush\n",
+ ninode);
+ } else {
+ nci = NULL;
+ ninode = NULL;
+ }
+ spin_unlock(&mdsc->cap_dirty_lock);
+ if (inode) {
+ ceph_check_caps(ci, CHECK_CAPS_NODELAY|CHECK_CAPS_FLUSH,
+ NULL);
+ iput(inode);
+ }
+ spin_lock(&mdsc->cap_dirty_lock);
+ }
+ spin_unlock(&mdsc->cap_dirty_lock);
+}
+
+/*
+ * Drop open file reference. If we were the last open file,
+ * we may need to release capabilities to the MDS (or schedule
+ * their delayed release).
+ */
+void ceph_put_fmode(struct ceph_inode_info *ci, int fmode)
+{
+ struct inode *inode = &ci->vfs_inode;
+ int last = 0;
+
+ spin_lock(&inode->i_lock);
+ dout("put_fmode %p fmode %d %d -> %d\n", inode, fmode,
+ ci->i_nr_by_mode[fmode], ci->i_nr_by_mode[fmode]-1);
+ BUG_ON(ci->i_nr_by_mode[fmode] == 0);
+ if (--ci->i_nr_by_mode[fmode] == 0)
+ last++;
+ spin_unlock(&inode->i_lock);
+
+ if (last && ci->i_vino.snap == CEPH_NOSNAP)
+ ceph_check_caps(ci, 0, NULL);
+}
+
+/*
+ * Helpers for embedding cap and dentry lease releases into mds
+ * requests.
+ *
+ * @force is used by dentry_release (below) to force inclusion of a
+ * record for the directory inode, even when there aren't any caps to
+ * drop.
+ */
+int ceph_encode_inode_release(void **p, struct inode *inode,
+ int mds, int drop, int unless, int force)
+{
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ struct ceph_cap *cap;
+ struct ceph_mds_request_release *rel = *p;
+ int ret = 0;
+ int used = 0;
+
+ spin_lock(&inode->i_lock);
+ used = __ceph_caps_used(ci);
+
+ dout("encode_inode_release %p mds%d used %s drop %s unless %s\n", inode,
+ mds, ceph_cap_string(used), ceph_cap_string(drop),
+ ceph_cap_string(unless));
+
+ /* only drop unused caps */
+ drop &= ~used;
+
+ cap = __get_cap_for_mds(ci, mds);
+ if (cap && __cap_is_valid(cap)) {
+ if (force ||
+ ((cap->issued & drop) &&
+ (cap->issued & unless) == 0)) {
+ if ((cap->issued & drop) &&
+ (cap->issued & unless) == 0) {
+ dout("encode_inode_release %p cap %p %s -> "
+ "%s\n", inode, cap,
+ ceph_cap_string(cap->issued),
+ ceph_cap_string(cap->issued & ~drop));
+ cap->issued &= ~drop;
+ cap->implemented &= ~drop;
+ if (ci->i_ceph_flags & CEPH_I_NODELAY) {
+ int wanted = __ceph_caps_wanted(ci);
+ dout(" wanted %s -> %s (act %s)\n",
+ ceph_cap_string(cap->mds_wanted),
+ ceph_cap_string(cap->mds_wanted &
+ ~wanted),
+ ceph_cap_string(wanted));
+ cap->mds_wanted &= wanted;
+ }
+ } else {
+ dout("encode_inode_release %p cap %p %s"
+ " (force)\n", inode, cap,
+ ceph_cap_string(cap->issued));
+ }
+
+ rel->ino = cpu_to_le64(ceph_ino(inode));
+ rel->cap_id = cpu_to_le64(cap->cap_id);
+ rel->seq = cpu_to_le32(cap->seq);
+ rel->issue_seq = cpu_to_le32(cap->issue_seq),
+ rel->mseq = cpu_to_le32(cap->mseq);
+ rel->caps = cpu_to_le32(cap->issued);
+ rel->wanted = cpu_to_le32(cap->mds_wanted);
+ rel->dname_len = 0;
+ rel->dname_seq = 0;
+ *p += sizeof(*rel);
+ ret = 1;
+ } else {
+ dout("encode_inode_release %p cap %p %s\n",
+ inode, cap, ceph_cap_string(cap->issued));
+ }
+ }
+ spin_unlock(&inode->i_lock);
+ return ret;
+}
+
+int ceph_encode_dentry_release(void **p, struct dentry *dentry,
+ int mds, int drop, int unless)
+{
+ struct inode *dir = dentry->d_parent->d_inode;
+ struct ceph_mds_request_release *rel = *p;
+ struct ceph_dentry_info *di = ceph_dentry(dentry);
+ int force = 0;
+ int ret;
+
+ /*
+ * force an record for the directory caps if we have a dentry lease.
+ * this is racy (can't take i_lock and d_lock together), but it
+ * doesn't have to be perfect; the mds will revoke anything we don't
+ * release.
+ */
+ spin_lock(&dentry->d_lock);
+ if (di->lease_session && di->lease_session->s_mds == mds)
+ force = 1;
+ spin_unlock(&dentry->d_lock);
+
+ ret = ceph_encode_inode_release(p, dir, mds, drop, unless, force);
+
+ spin_lock(&dentry->d_lock);
+ if (ret && di->lease_session && di->lease_session->s_mds == mds) {
+ dout("encode_dentry_release %p mds%d seq %d\n",
+ dentry, mds, (int)di->lease_seq);
+ rel->dname_len = cpu_to_le32(dentry->d_name.len);
+ memcpy(*p, dentry->d_name.name, dentry->d_name.len);
+ *p += dentry->d_name.len;
+ rel->dname_seq = cpu_to_le32(di->lease_seq);
+ }
+ spin_unlock(&dentry->d_lock);
+ return ret;
+}
--- /dev/null
+#ifndef _FS_CEPH_DEBUG_H
+#define _FS_CEPH_DEBUG_H
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#ifdef CONFIG_CEPH_FS_PRETTYDEBUG
+
+/*
+ * wrap pr_debug to include a filename:lineno prefix on each line.
+ * this incurs some overhead (kernel size and execution time) due to
+ * the extra function call at each call site.
+ */
+
+# if defined(DEBUG) || defined(CONFIG_DYNAMIC_DEBUG)
+extern const char *ceph_file_part(const char *s, int len);
+# define dout(fmt, ...) \
+ pr_debug(" %12.12s:%-4d : " fmt, \
+ ceph_file_part(__FILE__, sizeof(__FILE__)), \
+ __LINE__, ##__VA_ARGS__)
+# else
+/* faux printk call just to see any compiler warnings. */
+# define dout(fmt, ...) do { \
+ if (0) \
+ printk(KERN_DEBUG fmt, ##__VA_ARGS__); \
+ } while (0)
+# endif
+
+#else
+
+/*
+ * or, just wrap pr_debug
+ */
+# define dout(fmt, ...) pr_debug(" " fmt, ##__VA_ARGS__)
+
+#endif
+
+#endif
--- /dev/null
+/*
+ * Ceph 'frag' type
+ */
+#include "types.h"
+
+int ceph_frag_compare(__u32 a, __u32 b)
+{
+ unsigned va = ceph_frag_value(a);
+ unsigned vb = ceph_frag_value(b);
+ if (va < vb)
+ return -1;
+ if (va > vb)
+ return 1;
+ va = ceph_frag_bits(a);
+ vb = ceph_frag_bits(b);
+ if (va < vb)
+ return -1;
+ if (va > vb)
+ return 1;
+ return 0;
+}
--- /dev/null
+#ifndef _FS_CEPH_FRAG_H
+#define _FS_CEPH_FRAG_H
+
+/*
+ * "Frags" are a way to describe a subset of a 32-bit number space,
+ * using a mask and a value to match against that mask. Any given frag
+ * (subset of the number space) can be partitioned into 2^n sub-frags.
+ *
+ * Frags are encoded into a 32-bit word:
+ * 8 upper bits = "bits"
+ * 24 lower bits = "value"
+ * (We could go to 5+27 bits, but who cares.)
+ *
+ * We use the _most_ significant bits of the 24 bit value. This makes
+ * values logically sort.
+ *
+ * Unfortunately, because the "bits" field is still in the high bits, we
+ * can't sort encoded frags numerically. However, it does allow you
+ * to feed encoded frags as values into frag_contains_value.
+ */
+static inline __u32 ceph_frag_make(__u32 b, __u32 v)
+{
+ return (b << 24) |
+ (v & (0xffffffu << (24-b)) & 0xffffffu);
+}
+static inline __u32 ceph_frag_bits(__u32 f)
+{
+ return f >> 24;
+}
+static inline __u32 ceph_frag_value(__u32 f)
+{
+ return f & 0xffffffu;
+}
+static inline __u32 ceph_frag_mask(__u32 f)
+{
+ return (0xffffffu << (24-ceph_frag_bits(f))) & 0xffffffu;
+}
+static inline __u32 ceph_frag_mask_shift(__u32 f)
+{
+ return 24 - ceph_frag_bits(f);
+}
+
+static inline int ceph_frag_contains_value(__u32 f, __u32 v)
+{
+ return (v & ceph_frag_mask(f)) == ceph_frag_value(f);
+}
+static inline int ceph_frag_contains_frag(__u32 f, __u32 sub)
+{
+ /* is sub as specific as us, and contained by us? */
+ return ceph_frag_bits(sub) >= ceph_frag_bits(f) &&
+ (ceph_frag_value(sub) & ceph_frag_mask(f)) == ceph_frag_value(f);
+}
+
+static inline __u32 ceph_frag_parent(__u32 f)
+{
+ return ceph_frag_make(ceph_frag_bits(f) - 1,
+ ceph_frag_value(f) & (ceph_frag_mask(f) << 1));
+}
+static inline int ceph_frag_is_left_child(__u32 f)
+{
+ return ceph_frag_bits(f) > 0 &&
+ (ceph_frag_value(f) & (0x1000000 >> ceph_frag_bits(f))) == 0;
+}
+static inline int ceph_frag_is_right_child(__u32 f)
+{
+ return ceph_frag_bits(f) > 0 &&
+ (ceph_frag_value(f) & (0x1000000 >> ceph_frag_bits(f))) == 1;
+}
+static inline __u32 ceph_frag_sibling(__u32 f)
+{
+ return ceph_frag_make(ceph_frag_bits(f),
+ ceph_frag_value(f) ^ (0x1000000 >> ceph_frag_bits(f)));
+}
+static inline __u32 ceph_frag_left_child(__u32 f)
+{
+ return ceph_frag_make(ceph_frag_bits(f)+1, ceph_frag_value(f));
+}
+static inline __u32 ceph_frag_right_child(__u32 f)
+{
+ return ceph_frag_make(ceph_frag_bits(f)+1,
+ ceph_frag_value(f) | (0x1000000 >> (1+ceph_frag_bits(f))));
+}
+static inline __u32 ceph_frag_make_child(__u32 f, int by, int i)
+{
+ int newbits = ceph_frag_bits(f) + by;
+ return ceph_frag_make(newbits,
+ ceph_frag_value(f) | (i << (24 - newbits)));
+}
+static inline int ceph_frag_is_leftmost(__u32 f)
+{
+ return ceph_frag_value(f) == 0;
+}
+static inline int ceph_frag_is_rightmost(__u32 f)
+{
+ return ceph_frag_value(f) == ceph_frag_mask(f);
+}
+static inline __u32 ceph_frag_next(__u32 f)
+{
+ return ceph_frag_make(ceph_frag_bits(f),
+ ceph_frag_value(f) + (0x1000000 >> ceph_frag_bits(f)));
+}
+
+/*
+ * comparator to sort frags logically, as when traversing the
+ * number space in ascending order...
+ */
+int ceph_frag_compare(__u32 a, __u32 b);
+
+#endif
--- /dev/null
+/*
+ * Some non-inline ceph helpers
+ */
+#include "types.h"
+
+/*
+ * return true if @layout appears to be valid
+ */
+int ceph_file_layout_is_valid(const struct ceph_file_layout *layout)
+{
+ __u32 su = le32_to_cpu(layout->fl_stripe_unit);
+ __u32 sc = le32_to_cpu(layout->fl_stripe_count);
+ __u32 os = le32_to_cpu(layout->fl_object_size);
+
+ /* stripe unit, object size must be non-zero, 64k increment */
+ if (!su || (su & (CEPH_MIN_STRIPE_UNIT-1)))
+ return 0;
+ if (!os || (os & (CEPH_MIN_STRIPE_UNIT-1)))
+ return 0;
+ /* object size must be a multiple of stripe unit */
+ if (os < su || os % su)
+ return 0;
+ /* stripe count must be non-zero */
+ if (!sc)
+ return 0;
+ return 1;
+}
+
+
+int ceph_flags_to_mode(int flags)
+{
+#ifdef O_DIRECTORY /* fixme */
+ if ((flags & O_DIRECTORY) == O_DIRECTORY)
+ return CEPH_FILE_MODE_PIN;
+#endif
+#ifdef O_LAZY
+ if (flags & O_LAZY)
+ return CEPH_FILE_MODE_LAZY;
+#endif
+ if ((flags & O_APPEND) == O_APPEND)
+ flags |= O_WRONLY;
+
+ flags &= O_ACCMODE;
+ if ((flags & O_RDWR) == O_RDWR)
+ return CEPH_FILE_MODE_RDWR;
+ if ((flags & O_WRONLY) == O_WRONLY)
+ return CEPH_FILE_MODE_WR;
+ return CEPH_FILE_MODE_RD;
+}
+
+int ceph_caps_for_mode(int mode)
+{
+ switch (mode) {
+ case CEPH_FILE_MODE_PIN:
+ return CEPH_CAP_PIN;
+ case CEPH_FILE_MODE_RD:
+ return CEPH_CAP_PIN | CEPH_CAP_FILE_SHARED |
+ CEPH_CAP_FILE_RD | CEPH_CAP_FILE_CACHE;
+ case CEPH_FILE_MODE_RDWR:
+ return CEPH_CAP_PIN | CEPH_CAP_FILE_SHARED |
+ CEPH_CAP_FILE_EXCL |
+ CEPH_CAP_FILE_RD | CEPH_CAP_FILE_CACHE |
+ CEPH_CAP_FILE_WR | CEPH_CAP_FILE_BUFFER |
+ CEPH_CAP_AUTH_SHARED | CEPH_CAP_AUTH_EXCL |
+ CEPH_CAP_XATTR_SHARED | CEPH_CAP_XATTR_EXCL;
+ case CEPH_FILE_MODE_WR:
+ return CEPH_CAP_PIN | CEPH_CAP_FILE_SHARED |
+ CEPH_CAP_FILE_EXCL |
+ CEPH_CAP_FILE_WR | CEPH_CAP_FILE_BUFFER |
+ CEPH_CAP_AUTH_SHARED | CEPH_CAP_AUTH_EXCL |
+ CEPH_CAP_XATTR_SHARED | CEPH_CAP_XATTR_EXCL;
+ }
+ return 0;
+}
--- /dev/null
+/*
+ * ceph_fs.h - Ceph constants and data types to share between kernel and
+ * user space.
+ *
+ * Most types in this file are defined as little-endian, and are
+ * primarily intended to describe data structures that pass over the
+ * wire or that are stored on disk.
+ *
+ * LGPL2
+ */
+
+#ifndef _FS_CEPH_CEPH_FS_H
+#define _FS_CEPH_CEPH_FS_H
+
+#include "msgr.h"
+#include "rados.h"
+
+/*
+ * Ceph release version
+ */
+#define CEPH_VERSION_MAJOR 0
+#define CEPH_VERSION_MINOR 19
+#define CEPH_VERSION_PATCH 0
+
+#define _CEPH_STRINGIFY(x) #x
+#define CEPH_STRINGIFY(x) _CEPH_STRINGIFY(x)
+#define CEPH_MAKE_VERSION(x, y, z) CEPH_STRINGIFY(x) "." CEPH_STRINGIFY(y) \
+ "." CEPH_STRINGIFY(z)
+#define CEPH_VERSION CEPH_MAKE_VERSION(CEPH_VERSION_MAJOR, \
+ CEPH_VERSION_MINOR, CEPH_VERSION_PATCH)
+
+/*
+ * subprotocol versions. when specific messages types or high-level
+ * protocols change, bump the affected components. we keep rev
+ * internal cluster protocols separately from the public,
+ * client-facing protocol.
+ */
+#define CEPH_OSD_PROTOCOL 8 /* cluster internal */
+#define CEPH_MDS_PROTOCOL 9 /* cluster internal */
+#define CEPH_MON_PROTOCOL 5 /* cluster internal */
+#define CEPH_OSDC_PROTOCOL 24 /* server/client */
+#define CEPH_MDSC_PROTOCOL 32 /* server/client */
+#define CEPH_MONC_PROTOCOL 15 /* server/client */
+
+
+#define CEPH_INO_ROOT 1
+#define CEPH_INO_CEPH 2 /* hidden .ceph dir */
+
+/* arbitrary limit on max # of monitors (cluster of 3 is typical) */
+#define CEPH_MAX_MON 31
+
+
+/*
+ * feature bits
+ */
+#define CEPH_FEATURE_SUPPORTED 0
+#define CEPH_FEATURE_REQUIRED 0
+
+
+/*
+ * ceph_file_layout - describe data layout for a file/inode
+ */
+struct ceph_file_layout {
+ /* file -> object mapping */
+ __le32 fl_stripe_unit; /* stripe unit, in bytes. must be multiple
+ of page size. */
+ __le32 fl_stripe_count; /* over this many objects */
+ __le32 fl_object_size; /* until objects are this big, then move to
+ new objects */
+ __le32 fl_cas_hash; /* 0 = none; 1 = sha256 */
+
+ /* pg -> disk layout */
+ __le32 fl_object_stripe_unit; /* for per-object parity, if any */
+
+ /* object -> pg layout */
+ __le32 fl_pg_preferred; /* preferred primary for pg (-1 for none) */
+ __le32 fl_pg_pool; /* namespace, crush ruleset, rep level */
+} __attribute__ ((packed));
+
+#define CEPH_MIN_STRIPE_UNIT 65536
+
+int ceph_file_layout_is_valid(const struct ceph_file_layout *layout);
+
+
+/* crypto algorithms */
+#define CEPH_CRYPTO_NONE 0x0
+#define CEPH_CRYPTO_AES 0x1
+
+/* security/authentication protocols */
+#define CEPH_AUTH_UNKNOWN 0x0
+#define CEPH_AUTH_NONE 0x1
+#define CEPH_AUTH_CEPHX 0x2
+
+
+/*********************************************
+ * message layer
+ */
+
+/*
+ * message types
+ */
+
+/* misc */
+#define CEPH_MSG_SHUTDOWN 1
+#define CEPH_MSG_PING 2
+
+/* client <-> monitor */
+#define CEPH_MSG_MON_MAP 4
+#define CEPH_MSG_MON_GET_MAP 5
+#define CEPH_MSG_STATFS 13
+#define CEPH_MSG_STATFS_REPLY 14
+#define CEPH_MSG_MON_SUBSCRIBE 15
+#define CEPH_MSG_MON_SUBSCRIBE_ACK 16
+#define CEPH_MSG_AUTH 17
+#define CEPH_MSG_AUTH_REPLY 18
+
+/* client <-> mds */
+#define CEPH_MSG_MDS_MAP 21
+
+#define CEPH_MSG_CLIENT_SESSION 22
+#define CEPH_MSG_CLIENT_RECONNECT 23
+
+#define CEPH_MSG_CLIENT_REQUEST 24
+#define CEPH_MSG_CLIENT_REQUEST_FORWARD 25
+#define CEPH_MSG_CLIENT_REPLY 26
+#define CEPH_MSG_CLIENT_CAPS 0x310
+#define CEPH_MSG_CLIENT_LEASE 0x311
+#define CEPH_MSG_CLIENT_SNAP 0x312
+#define CEPH_MSG_CLIENT_CAPRELEASE 0x313
+
+/* osd */
+#define CEPH_MSG_OSD_MAP 41
+#define CEPH_MSG_OSD_OP 42
+#define CEPH_MSG_OSD_OPREPLY 43
+
+struct ceph_mon_request_header {
+ __le64 have_version;
+ __le16 session_mon;
+ __le64 session_mon_tid;
+} __attribute__ ((packed));
+
+struct ceph_mon_statfs {
+ struct ceph_mon_request_header monhdr;
+ struct ceph_fsid fsid;
+} __attribute__ ((packed));
+
+struct ceph_statfs {
+ __le64 kb, kb_used, kb_avail;
+ __le64 num_objects;
+} __attribute__ ((packed));
+
+struct ceph_mon_statfs_reply {
+ struct ceph_fsid fsid;
+ __le64 version;
+ struct ceph_statfs st;
+} __attribute__ ((packed));
+
+struct ceph_osd_getmap {
+ struct ceph_mon_request_header monhdr;
+ struct ceph_fsid fsid;
+ __le32 start;
+} __attribute__ ((packed));
+
+struct ceph_mds_getmap {
+ struct ceph_mon_request_header monhdr;
+ struct ceph_fsid fsid;
+} __attribute__ ((packed));
+
+struct ceph_client_mount {
+ struct ceph_mon_request_header monhdr;
+} __attribute__ ((packed));
+
+struct ceph_mon_subscribe_item {
+ __le64 have_version; __le64 have;
+ __u8 onetime;
+} __attribute__ ((packed));
+
+struct ceph_mon_subscribe_ack {
+ __le32 duration; /* seconds */
+ struct ceph_fsid fsid;
+} __attribute__ ((packed));
+
+/*
+ * mds states
+ * > 0 -> in
+ * <= 0 -> out
+ */
+#define CEPH_MDS_STATE_DNE 0 /* down, does not exist. */
+#define CEPH_MDS_STATE_STOPPED -1 /* down, once existed, but no subtrees.
+ empty log. */
+#define CEPH_MDS_STATE_BOOT -4 /* up, boot announcement. */
+#define CEPH_MDS_STATE_STANDBY -5 /* up, idle. waiting for assignment. */
+#define CEPH_MDS_STATE_CREATING -6 /* up, creating MDS instance. */
+#define CEPH_MDS_STATE_STARTING -7 /* up, starting previously stopped mds */
+#define CEPH_MDS_STATE_STANDBY_REPLAY -8 /* up, tailing active node's journal */
+
+#define CEPH_MDS_STATE_REPLAY 8 /* up, replaying journal. */
+#define CEPH_MDS_STATE_RESOLVE 9 /* up, disambiguating distributed
+ operations (import, rename, etc.) */
+#define CEPH_MDS_STATE_RECONNECT 10 /* up, reconnect to clients */
+#define CEPH_MDS_STATE_REJOIN 11 /* up, rejoining distributed cache */
+#define CEPH_MDS_STATE_CLIENTREPLAY 12 /* up, replaying client operations */
+#define CEPH_MDS_STATE_ACTIVE 13 /* up, active */
+#define CEPH_MDS_STATE_STOPPING 14 /* up, but exporting metadata */
+
+extern const char *ceph_mds_state_name(int s);
+
+
+/*
+ * metadata lock types.
+ * - these are bitmasks.. we can compose them
+ * - they also define the lock ordering by the MDS
+ * - a few of these are internal to the mds
+ */
+#define CEPH_LOCK_DN 1
+#define CEPH_LOCK_ISNAP 2
+#define CEPH_LOCK_IVERSION 4 /* mds internal */
+#define CEPH_LOCK_IFILE 8 /* mds internal */
+#define CEPH_LOCK_IAUTH 32
+#define CEPH_LOCK_ILINK 64
+#define CEPH_LOCK_IDFT 128 /* dir frag tree */
+#define CEPH_LOCK_INEST 256 /* mds internal */
+#define CEPH_LOCK_IXATTR 512
+#define CEPH_LOCK_INO 2048 /* immutable inode bits; not a lock */
+
+/* client_session ops */
+enum {
+ CEPH_SESSION_REQUEST_OPEN,
+ CEPH_SESSION_OPEN,
+ CEPH_SESSION_REQUEST_CLOSE,
+ CEPH_SESSION_CLOSE,
+ CEPH_SESSION_REQUEST_RENEWCAPS,
+ CEPH_SESSION_RENEWCAPS,
+ CEPH_SESSION_STALE,
+ CEPH_SESSION_RECALL_STATE,
+};
+
+extern const char *ceph_session_op_name(int op);
+
+struct ceph_mds_session_head {
+ __le32 op;
+ __le64 seq;
+ struct ceph_timespec stamp;
+ __le32 max_caps, max_leases;
+} __attribute__ ((packed));
+
+/* client_request */
+/*
+ * metadata ops.
+ * & 0x001000 -> write op
+ * & 0x010000 -> follow symlink (e.g. stat(), not lstat()).
+ & & 0x100000 -> use weird ino/path trace
+ */
+#define CEPH_MDS_OP_WRITE 0x001000
+enum {
+ CEPH_MDS_OP_LOOKUP = 0x00100,
+ CEPH_MDS_OP_GETATTR = 0x00101,
+ CEPH_MDS_OP_LOOKUPHASH = 0x00102,
+ CEPH_MDS_OP_LOOKUPPARENT = 0x00103,
+
+ CEPH_MDS_OP_SETXATTR = 0x01105,
+ CEPH_MDS_OP_RMXATTR = 0x01106,
+ CEPH_MDS_OP_SETLAYOUT = 0x01107,
+ CEPH_MDS_OP_SETATTR = 0x01108,
+
+ CEPH_MDS_OP_MKNOD = 0x01201,
+ CEPH_MDS_OP_LINK = 0x01202,
+ CEPH_MDS_OP_UNLINK = 0x01203,
+ CEPH_MDS_OP_RENAME = 0x01204,
+ CEPH_MDS_OP_MKDIR = 0x01220,
+ CEPH_MDS_OP_RMDIR = 0x01221,
+ CEPH_MDS_OP_SYMLINK = 0x01222,
+
+ CEPH_MDS_OP_CREATE = 0x01301,
+ CEPH_MDS_OP_OPEN = 0x00302,
+ CEPH_MDS_OP_READDIR = 0x00305,
+
+ CEPH_MDS_OP_LOOKUPSNAP = 0x00400,
+ CEPH_MDS_OP_MKSNAP = 0x01400,
+ CEPH_MDS_OP_RMSNAP = 0x01401,
+ CEPH_MDS_OP_LSSNAP = 0x00402,
+};
+
+extern const char *ceph_mds_op_name(int op);
+
+
+#define CEPH_SETATTR_MODE 1
+#define CEPH_SETATTR_UID 2
+#define CEPH_SETATTR_GID 4
+#define CEPH_SETATTR_MTIME 8
+#define CEPH_SETATTR_ATIME 16
+#define CEPH_SETATTR_SIZE 32
+#define CEPH_SETATTR_CTIME 64
+
+union ceph_mds_request_args {
+ struct {
+ __le32 mask; /* CEPH_CAP_* */
+ } __attribute__ ((packed)) getattr;
+ struct {
+ __le32 mode;
+ __le32 uid;
+ __le32 gid;
+ struct ceph_timespec mtime;
+ struct ceph_timespec atime;
+ __le64 size, old_size; /* old_size needed by truncate */
+ __le32 mask; /* CEPH_SETATTR_* */
+ } __attribute__ ((packed)) setattr;
+ struct {
+ __le32 frag; /* which dir fragment */
+ __le32 max_entries; /* how many dentries to grab */
+ } __attribute__ ((packed)) readdir;
+ struct {
+ __le32 mode;
+ __le32 rdev;
+ } __attribute__ ((packed)) mknod;
+ struct {
+ __le32 mode;
+ } __attribute__ ((packed)) mkdir;
+ struct {
+ __le32 flags;
+ __le32 mode;
+ __le32 stripe_unit; /* layout for newly created file */
+ __le32 stripe_count; /* ... */
+ __le32 object_size;
+ __le32 file_replication;
+ __le32 preferred;
+ } __attribute__ ((packed)) open;
+ struct {
+ __le32 flags;
+ } __attribute__ ((packed)) setxattr;
+ struct {
+ struct ceph_file_layout layout;
+ } __attribute__ ((packed)) setlayout;
+} __attribute__ ((packed));
+
+#define CEPH_MDS_FLAG_REPLAY 1 /* this is a replayed op */
+#define CEPH_MDS_FLAG_WANT_DENTRY 2 /* want dentry in reply */
+
+struct ceph_mds_request_head {
+ __le64 oldest_client_tid;
+ __le32 mdsmap_epoch; /* on client */
+ __le32 flags; /* CEPH_MDS_FLAG_* */
+ __u8 num_retry, num_fwd; /* count retry, fwd attempts */
+ __le16 num_releases; /* # include cap/lease release records */
+ __le32 op; /* mds op code */
+ __le32 caller_uid, caller_gid;
+ __le64 ino; /* use this ino for openc, mkdir, mknod,
+ etc. (if replaying) */
+ union ceph_mds_request_args args;
+} __attribute__ ((packed));
+
+/* cap/lease release record */
+struct ceph_mds_request_release {
+ __le64 ino, cap_id; /* ino and unique cap id */
+ __le32 caps, wanted; /* new issued, wanted */
+ __le32 seq, issue_seq, mseq;
+ __le32 dname_seq; /* if releasing a dentry lease, a */
+ __le32 dname_len; /* string follows. */
+} __attribute__ ((packed));
+
+/* client reply */
+struct ceph_mds_reply_head {
+ __le32 op;
+ __le32 result;
+ __le32 mdsmap_epoch;
+ __u8 safe; /* true if committed to disk */
+ __u8 is_dentry, is_target; /* true if dentry, target inode records
+ are included with reply */
+} __attribute__ ((packed));
+
+/* one for each node split */
+struct ceph_frag_tree_split {
+ __le32 frag; /* this frag splits... */
+ __le32 by; /* ...by this many bits */
+} __attribute__ ((packed));
+
+struct ceph_frag_tree_head {
+ __le32 nsplits; /* num ceph_frag_tree_split records */
+ struct ceph_frag_tree_split splits[];
+} __attribute__ ((packed));
+
+/* capability issue, for bundling with mds reply */
+struct ceph_mds_reply_cap {
+ __le32 caps, wanted; /* caps issued, wanted */
+ __le64 cap_id;
+ __le32 seq, mseq;
+ __le64 realm; /* snap realm */
+ __u8 flags; /* CEPH_CAP_FLAG_* */
+} __attribute__ ((packed));
+
+#define CEPH_CAP_FLAG_AUTH 1 /* cap is issued by auth mds */
+
+/* inode record, for bundling with mds reply */
+struct ceph_mds_reply_inode {
+ __le64 ino;
+ __le64 snapid;
+ __le32 rdev;
+ __le64 version; /* inode version */
+ __le64 xattr_version; /* version for xattr blob */
+ struct ceph_mds_reply_cap cap; /* caps issued for this inode */
+ struct ceph_file_layout layout;
+ struct ceph_timespec ctime, mtime, atime;
+ __le32 time_warp_seq;
+ __le64 size, max_size, truncate_size;
+ __le32 truncate_seq;
+ __le32 mode, uid, gid;
+ __le32 nlink;
+ __le64 files, subdirs, rbytes, rfiles, rsubdirs; /* dir stats */
+ struct ceph_timespec rctime;
+ struct ceph_frag_tree_head fragtree; /* (must be at end of struct) */
+} __attribute__ ((packed));
+/* followed by frag array, then symlink string, then xattr blob */
+
+/* reply_lease follows dname, and reply_inode */
+struct ceph_mds_reply_lease {
+ __le16 mask; /* lease type(s) */
+ __le32 duration_ms; /* lease duration */
+ __le32 seq;
+} __attribute__ ((packed));
+
+struct ceph_mds_reply_dirfrag {
+ __le32 frag; /* fragment */
+ __le32 auth; /* auth mds, if this is a delegation point */
+ __le32 ndist; /* number of mds' this is replicated on */
+ __le32 dist[];
+} __attribute__ ((packed));
+
+/* file access modes */
+#define CEPH_FILE_MODE_PIN 0
+#define CEPH_FILE_MODE_RD 1
+#define CEPH_FILE_MODE_WR 2
+#define CEPH_FILE_MODE_RDWR 3 /* RD | WR */
+#define CEPH_FILE_MODE_LAZY 4 /* lazy io */
+#define CEPH_FILE_MODE_NUM 8 /* bc these are bit fields.. mostly */
+
+int ceph_flags_to_mode(int flags);
+
+
+/* capability bits */
+#define CEPH_CAP_PIN 1 /* no specific capabilities beyond the pin */
+
+/* generic cap bits */
+#define CEPH_CAP_GSHARED 1 /* client can reads */
+#define CEPH_CAP_GEXCL 2 /* client can read and update */
+#define CEPH_CAP_GCACHE 4 /* (file) client can cache reads */
+#define CEPH_CAP_GRD 8 /* (file) client can read */
+#define CEPH_CAP_GWR 16 /* (file) client can write */
+#define CEPH_CAP_GBUFFER 32 /* (file) client can buffer writes */
+#define CEPH_CAP_GWREXTEND 64 /* (file) client can extend EOF */
+#define CEPH_CAP_GLAZYIO 128 /* (file) client can perform lazy io */
+
+/* per-lock shift */
+#define CEPH_CAP_SAUTH 2
+#define CEPH_CAP_SLINK 4
+#define CEPH_CAP_SXATTR 6
+#define CEPH_CAP_SFILE 8 /* goes at the end (uses >2 cap bits) */
+
+#define CEPH_CAP_BITS 16
+
+/* composed values */
+#define CEPH_CAP_AUTH_SHARED (CEPH_CAP_GSHARED << CEPH_CAP_SAUTH)
+#define CEPH_CAP_AUTH_EXCL (CEPH_CAP_GEXCL << CEPH_CAP_SAUTH)
+#define CEPH_CAP_LINK_SHARED (CEPH_CAP_GSHARED << CEPH_CAP_SLINK)
+#define CEPH_CAP_LINK_EXCL (CEPH_CAP_GEXCL << CEPH_CAP_SLINK)
+#define CEPH_CAP_XATTR_SHARED (CEPH_CAP_GSHARED << CEPH_CAP_SXATTR)
+#define CEPH_CAP_XATTR_EXCL (CEPH_CAP_GEXCL << CEPH_CAP_SXATTR)
+#define CEPH_CAP_FILE(x) (x << CEPH_CAP_SFILE)
+#define CEPH_CAP_FILE_SHARED (CEPH_CAP_GSHARED << CEPH_CAP_SFILE)
+#define CEPH_CAP_FILE_EXCL (CEPH_CAP_GEXCL << CEPH_CAP_SFILE)
+#define CEPH_CAP_FILE_CACHE (CEPH_CAP_GCACHE << CEPH_CAP_SFILE)
+#define CEPH_CAP_FILE_RD (CEPH_CAP_GRD << CEPH_CAP_SFILE)
+#define CEPH_CAP_FILE_WR (CEPH_CAP_GWR << CEPH_CAP_SFILE)
+#define CEPH_CAP_FILE_BUFFER (CEPH_CAP_GBUFFER << CEPH_CAP_SFILE)
+#define CEPH_CAP_FILE_WREXTEND (CEPH_CAP_GWREXTEND << CEPH_CAP_SFILE)
+#define CEPH_CAP_FILE_LAZYIO (CEPH_CAP_GLAZYIO << CEPH_CAP_SFILE)
+
+/* cap masks (for getattr) */
+#define CEPH_STAT_CAP_INODE CEPH_CAP_PIN
+#define CEPH_STAT_CAP_TYPE CEPH_CAP_PIN /* mode >> 12 */
+#define CEPH_STAT_CAP_SYMLINK CEPH_CAP_PIN
+#define CEPH_STAT_CAP_UID CEPH_CAP_AUTH_SHARED
+#define CEPH_STAT_CAP_GID CEPH_CAP_AUTH_SHARED
+#define CEPH_STAT_CAP_MODE CEPH_CAP_AUTH_SHARED
+#define CEPH_STAT_CAP_NLINK CEPH_CAP_LINK_SHARED
+#define CEPH_STAT_CAP_LAYOUT CEPH_CAP_FILE_SHARED
+#define CEPH_STAT_CAP_MTIME CEPH_CAP_FILE_SHARED
+#define CEPH_STAT_CAP_SIZE CEPH_CAP_FILE_SHARED
+#define CEPH_STAT_CAP_ATIME CEPH_CAP_FILE_SHARED /* fixme */
+#define CEPH_STAT_CAP_XATTR CEPH_CAP_XATTR_SHARED
+#define CEPH_STAT_CAP_INODE_ALL (CEPH_CAP_PIN | \
+ CEPH_CAP_AUTH_SHARED | \
+ CEPH_CAP_LINK_SHARED | \
+ CEPH_CAP_FILE_SHARED | \
+ CEPH_CAP_XATTR_SHARED)
+
+#define CEPH_CAP_ANY_SHARED (CEPH_CAP_AUTH_SHARED | \
+ CEPH_CAP_LINK_SHARED | \
+ CEPH_CAP_XATTR_SHARED | \
+ CEPH_CAP_FILE_SHARED)
+#define CEPH_CAP_ANY_RD (CEPH_CAP_ANY_SHARED | CEPH_CAP_FILE_RD | \
+ CEPH_CAP_FILE_CACHE)
+
+#define CEPH_CAP_ANY_EXCL (CEPH_CAP_AUTH_EXCL | \
+ CEPH_CAP_LINK_EXCL | \
+ CEPH_CAP_XATTR_EXCL | \
+ CEPH_CAP_FILE_EXCL)
+#define CEPH_CAP_ANY_FILE_WR (CEPH_CAP_FILE_WR | CEPH_CAP_FILE_BUFFER | \
+ CEPH_CAP_FILE_EXCL)
+#define CEPH_CAP_ANY_WR (CEPH_CAP_ANY_EXCL | CEPH_CAP_ANY_FILE_WR)
+#define CEPH_CAP_ANY (CEPH_CAP_ANY_RD | CEPH_CAP_ANY_EXCL | \
+ CEPH_CAP_ANY_FILE_WR | CEPH_CAP_PIN)
+
+#define CEPH_CAP_LOCKS (CEPH_LOCK_IFILE | CEPH_LOCK_IAUTH | CEPH_LOCK_ILINK | \
+ CEPH_LOCK_IXATTR)
+
+int ceph_caps_for_mode(int mode);
+
+enum {
+ CEPH_CAP_OP_GRANT, /* mds->client grant */
+ CEPH_CAP_OP_REVOKE, /* mds->client revoke */
+ CEPH_CAP_OP_TRUNC, /* mds->client trunc notify */
+ CEPH_CAP_OP_EXPORT, /* mds has exported the cap */
+ CEPH_CAP_OP_IMPORT, /* mds has imported the cap */
+ CEPH_CAP_OP_UPDATE, /* client->mds update */
+ CEPH_CAP_OP_DROP, /* client->mds drop cap bits */
+ CEPH_CAP_OP_FLUSH, /* client->mds cap writeback */
+ CEPH_CAP_OP_FLUSH_ACK, /* mds->client flushed */
+ CEPH_CAP_OP_FLUSHSNAP, /* client->mds flush snapped metadata */
+ CEPH_CAP_OP_FLUSHSNAP_ACK, /* mds->client flushed snapped metadata */
+ CEPH_CAP_OP_RELEASE, /* client->mds release (clean) cap */
+ CEPH_CAP_OP_RENEW, /* client->mds renewal request */
+};
+
+extern const char *ceph_cap_op_name(int op);
+
+/*
+ * caps message, used for capability callbacks, acks, requests, etc.
+ */
+struct ceph_mds_caps {
+ __le32 op; /* CEPH_CAP_OP_* */
+ __le64 ino, realm;
+ __le64 cap_id;
+ __le32 seq, issue_seq;
+ __le32 caps, wanted, dirty; /* latest issued/wanted/dirty */
+ __le32 migrate_seq;
+ __le64 snap_follows;
+ __le32 snap_trace_len;
+
+ /* authlock */
+ __le32 uid, gid, mode;
+
+ /* linklock */
+ __le32 nlink;
+
+ /* xattrlock */
+ __le32 xattr_len;
+ __le64 xattr_version;
+
+ /* filelock */
+ __le64 size, max_size, truncate_size;
+ __le32 truncate_seq;
+ struct ceph_timespec mtime, atime, ctime;
+ struct ceph_file_layout layout;
+ __le32 time_warp_seq;
+} __attribute__ ((packed));
+
+/* cap release msg head */
+struct ceph_mds_cap_release {
+ __le32 num; /* number of cap_items that follow */
+} __attribute__ ((packed));
+
+struct ceph_mds_cap_item {
+ __le64 ino;
+ __le64 cap_id;
+ __le32 migrate_seq, seq;
+} __attribute__ ((packed));
+
+#define CEPH_MDS_LEASE_REVOKE 1 /* mds -> client */
+#define CEPH_MDS_LEASE_RELEASE 2 /* client -> mds */
+#define CEPH_MDS_LEASE_RENEW 3 /* client <-> mds */
+#define CEPH_MDS_LEASE_REVOKE_ACK 4 /* client -> mds */
+
+extern const char *ceph_lease_op_name(int o);
+
+/* lease msg header */
+struct ceph_mds_lease {
+ __u8 action; /* CEPH_MDS_LEASE_* */
+ __le16 mask; /* which lease */
+ __le64 ino;
+ __le64 first, last; /* snap range */
+ __le32 seq;
+ __le32 duration_ms; /* duration of renewal */
+} __attribute__ ((packed));
+/* followed by a __le32+string for dname */
+
+/* client reconnect */
+struct ceph_mds_cap_reconnect {
+ __le64 cap_id;
+ __le32 wanted;
+ __le32 issued;
+ __le64 size;
+ struct ceph_timespec mtime, atime;
+ __le64 snaprealm;
+ __le64 pathbase; /* base ino for our path to this ino */
+} __attribute__ ((packed));
+/* followed by encoded string */
+
+struct ceph_mds_snaprealm_reconnect {
+ __le64 ino; /* snap realm base */
+ __le64 seq; /* snap seq for this snap realm */
+ __le64 parent; /* parent realm */
+} __attribute__ ((packed));
+
+/*
+ * snaps
+ */
+enum {
+ CEPH_SNAP_OP_UPDATE, /* CREATE or DESTROY */
+ CEPH_SNAP_OP_CREATE,
+ CEPH_SNAP_OP_DESTROY,
+ CEPH_SNAP_OP_SPLIT,
+};
+
+extern const char *ceph_snap_op_name(int o);
+
+/* snap msg header */
+struct ceph_mds_snap_head {
+ __le32 op; /* CEPH_SNAP_OP_* */
+ __le64 split; /* ino to split off, if any */
+ __le32 num_split_inos; /* # inos belonging to new child realm */
+ __le32 num_split_realms; /* # child realms udner new child realm */
+ __le32 trace_len; /* size of snap trace blob */
+} __attribute__ ((packed));
+/* followed by split ino list, then split realms, then the trace blob */
+
+/*
+ * encode info about a snaprealm, as viewed by a client
+ */
+struct ceph_mds_snap_realm {
+ __le64 ino; /* ino */
+ __le64 created; /* snap: when created */
+ __le64 parent; /* ino: parent realm */
+ __le64 parent_since; /* snap: same parent since */
+ __le64 seq; /* snap: version */
+ __le32 num_snaps;
+ __le32 num_prior_parent_snaps;
+} __attribute__ ((packed));
+/* followed by my snap list, then prior parent snap list */
+
+#endif
--- /dev/null
+
+#include "types.h"
+
+/*
+ * Robert Jenkin's hash function.
+ * http://burtleburtle.net/bob/hash/evahash.html
+ * This is in the public domain.
+ */
+#define mix(a, b, c) \
+ do { \
+ a = a - b; a = a - c; a = a ^ (c >> 13); \
+ b = b - c; b = b - a; b = b ^ (a << 8); \
+ c = c - a; c = c - b; c = c ^ (b >> 13); \
+ a = a - b; a = a - c; a = a ^ (c >> 12); \
+ b = b - c; b = b - a; b = b ^ (a << 16); \
+ c = c - a; c = c - b; c = c ^ (b >> 5); \
+ a = a - b; a = a - c; a = a ^ (c >> 3); \
+ b = b - c; b = b - a; b = b ^ (a << 10); \
+ c = c - a; c = c - b; c = c ^ (b >> 15); \
+ } while (0)
+
+unsigned ceph_str_hash_rjenkins(const char *str, unsigned length)
+{
+ const unsigned char *k = (const unsigned char *)str;
+ __u32 a, b, c; /* the internal state */
+ __u32 len; /* how many key bytes still need mixing */
+
+ /* Set up the internal state */
+ len = length;
+ a = 0x9e3779b9; /* the golden ratio; an arbitrary value */
+ b = a;
+ c = 0; /* variable initialization of internal state */
+
+ /* handle most of the key */
+ while (len >= 12) {
+ a = a + (k[0] + ((__u32)k[1] << 8) + ((__u32)k[2] << 16) +
+ ((__u32)k[3] << 24));
+ b = b + (k[4] + ((__u32)k[5] << 8) + ((__u32)k[6] << 16) +
+ ((__u32)k[7] << 24));
+ c = c + (k[8] + ((__u32)k[9] << 8) + ((__u32)k[10] << 16) +
+ ((__u32)k[11] << 24));
+ mix(a, b, c);
+ k = k + 12;
+ len = len - 12;
+ }
+
+ /* handle the last 11 bytes */
+ c = c + length;
+ switch (len) { /* all the case statements fall through */
+ case 11:
+ c = c + ((__u32)k[10] << 24);
+ case 10:
+ c = c + ((__u32)k[9] << 16);
+ case 9:
+ c = c + ((__u32)k[8] << 8);
+ /* the first byte of c is reserved for the length */
+ case 8:
+ b = b + ((__u32)k[7] << 24);
+ case 7:
+ b = b + ((__u32)k[6] << 16);
+ case 6:
+ b = b + ((__u32)k[5] << 8);
+ case 5:
+ b = b + k[4];
+ case 4:
+ a = a + ((__u32)k[3] << 24);
+ case 3:
+ a = a + ((__u32)k[2] << 16);
+ case 2:
+ a = a + ((__u32)k[1] << 8);
+ case 1:
+ a = a + k[0];
+ /* case 0: nothing left to add */
+ }
+ mix(a, b, c);
+
+ return c;
+}
+
+/*
+ * linux dcache hash
+ */
+unsigned ceph_str_hash_linux(const char *str, unsigned length)
+{
+ unsigned long hash = 0;
+ unsigned char c;
+
+ while (length--) {
+ c = *str++;
+ hash = (hash + (c << 4) + (c >> 4)) * 11;
+ }
+ return hash;
+}
+
+
+unsigned ceph_str_hash(int type, const char *s, unsigned len)
+{
+ switch (type) {
+ case CEPH_STR_HASH_LINUX:
+ return ceph_str_hash_linux(s, len);
+ case CEPH_STR_HASH_RJENKINS:
+ return ceph_str_hash_rjenkins(s, len);
+ default:
+ return -1;
+ }
+}
+
+const char *ceph_str_hash_name(int type)
+{
+ switch (type) {
+ case CEPH_STR_HASH_LINUX:
+ return "linux";
+ case CEPH_STR_HASH_RJENKINS:
+ return "rjenkins";
+ default:
+ return "unknown";
+ }
+}
--- /dev/null
+#ifndef _FS_CEPH_HASH_H
+#define _FS_CEPH_HASH_H
+
+#define CEPH_STR_HASH_LINUX 0x1 /* linux dcache hash */
+#define CEPH_STR_HASH_RJENKINS 0x2 /* robert jenkins' */
+
+extern unsigned ceph_str_hash_linux(const char *s, unsigned len);
+extern unsigned ceph_str_hash_rjenkins(const char *s, unsigned len);
+
+extern unsigned ceph_str_hash(int type, const char *s, unsigned len);
+extern const char *ceph_str_hash_name(int type);
+
+#endif
--- /dev/null
+/*
+ * Ceph string constants
+ */
+#include "types.h"
+
+const char *ceph_entity_type_name(int type)
+{
+ switch (type) {
+ case CEPH_ENTITY_TYPE_MDS: return "mds";
+ case CEPH_ENTITY_TYPE_OSD: return "osd";
+ case CEPH_ENTITY_TYPE_MON: return "mon";
+ case CEPH_ENTITY_TYPE_CLIENT: return "client";
+ case CEPH_ENTITY_TYPE_ADMIN: return "admin";
+ case CEPH_ENTITY_TYPE_AUTH: return "auth";
+ default: return "unknown";
+ }
+}
+
+const char *ceph_osd_op_name(int op)
+{
+ switch (op) {
+ case CEPH_OSD_OP_READ: return "read";
+ case CEPH_OSD_OP_STAT: return "stat";
+
+ case CEPH_OSD_OP_MASKTRUNC: return "masktrunc";
+
+ case CEPH_OSD_OP_WRITE: return "write";
+ case CEPH_OSD_OP_DELETE: return "delete";
+ case CEPH_OSD_OP_TRUNCATE: return "truncate";
+ case CEPH_OSD_OP_ZERO: return "zero";
+ case CEPH_OSD_OP_WRITEFULL: return "writefull";
+
+ case CEPH_OSD_OP_APPEND: return "append";
+ case CEPH_OSD_OP_STARTSYNC: return "startsync";
+ case CEPH_OSD_OP_SETTRUNC: return "settrunc";
+ case CEPH_OSD_OP_TRIMTRUNC: return "trimtrunc";
+
+ case CEPH_OSD_OP_TMAPUP: return "tmapup";
+ case CEPH_OSD_OP_TMAPGET: return "tmapget";
+ case CEPH_OSD_OP_TMAPPUT: return "tmapput";
+
+ case CEPH_OSD_OP_GETXATTR: return "getxattr";
+ case CEPH_OSD_OP_GETXATTRS: return "getxattrs";
+ case CEPH_OSD_OP_SETXATTR: return "setxattr";
+ case CEPH_OSD_OP_SETXATTRS: return "setxattrs";
+ case CEPH_OSD_OP_RESETXATTRS: return "resetxattrs";
+ case CEPH_OSD_OP_RMXATTR: return "rmxattr";
+
+ case CEPH_OSD_OP_PULL: return "pull";
+ case CEPH_OSD_OP_PUSH: return "push";
+ case CEPH_OSD_OP_BALANCEREADS: return "balance-reads";
+ case CEPH_OSD_OP_UNBALANCEREADS: return "unbalance-reads";
+ case CEPH_OSD_OP_SCRUB: return "scrub";
+
+ case CEPH_OSD_OP_WRLOCK: return "wrlock";
+ case CEPH_OSD_OP_WRUNLOCK: return "wrunlock";
+ case CEPH_OSD_OP_RDLOCK: return "rdlock";
+ case CEPH_OSD_OP_RDUNLOCK: return "rdunlock";
+ case CEPH_OSD_OP_UPLOCK: return "uplock";
+ case CEPH_OSD_OP_DNLOCK: return "dnlock";
+
+ case CEPH_OSD_OP_CALL: return "call";
+
+ case CEPH_OSD_OP_PGLS: return "pgls";
+ }
+ return "???";
+}
+
+const char *ceph_mds_state_name(int s)
+{
+ switch (s) {
+ /* down and out */
+ case CEPH_MDS_STATE_DNE: return "down:dne";
+ case CEPH_MDS_STATE_STOPPED: return "down:stopped";
+ /* up and out */
+ case CEPH_MDS_STATE_BOOT: return "up:boot";
+ case CEPH_MDS_STATE_STANDBY: return "up:standby";
+ case CEPH_MDS_STATE_STANDBY_REPLAY: return "up:standby-replay";
+ case CEPH_MDS_STATE_CREATING: return "up:creating";
+ case CEPH_MDS_STATE_STARTING: return "up:starting";
+ /* up and in */
+ case CEPH_MDS_STATE_REPLAY: return "up:replay";
+ case CEPH_MDS_STATE_RESOLVE: return "up:resolve";
+ case CEPH_MDS_STATE_RECONNECT: return "up:reconnect";
+ case CEPH_MDS_STATE_REJOIN: return "up:rejoin";
+ case CEPH_MDS_STATE_CLIENTREPLAY: return "up:clientreplay";
+ case CEPH_MDS_STATE_ACTIVE: return "up:active";
+ case CEPH_MDS_STATE_STOPPING: return "up:stopping";
+ }
+ return "???";
+}
+
+const char *ceph_session_op_name(int op)
+{
+ switch (op) {
+ case CEPH_SESSION_REQUEST_OPEN: return "request_open";
+ case CEPH_SESSION_OPEN: return "open";
+ case CEPH_SESSION_REQUEST_CLOSE: return "request_close";
+ case CEPH_SESSION_CLOSE: return "close";
+ case CEPH_SESSION_REQUEST_RENEWCAPS: return "request_renewcaps";
+ case CEPH_SESSION_RENEWCAPS: return "renewcaps";
+ case CEPH_SESSION_STALE: return "stale";
+ case CEPH_SESSION_RECALL_STATE: return "recall_state";
+ }
+ return "???";
+}
+
+const char *ceph_mds_op_name(int op)
+{
+ switch (op) {
+ case CEPH_MDS_OP_LOOKUP: return "lookup";
+ case CEPH_MDS_OP_LOOKUPHASH: return "lookuphash";
+ case CEPH_MDS_OP_LOOKUPPARENT: return "lookupparent";
+ case CEPH_MDS_OP_GETATTR: return "getattr";
+ case CEPH_MDS_OP_SETXATTR: return "setxattr";
+ case CEPH_MDS_OP_SETATTR: return "setattr";
+ case CEPH_MDS_OP_RMXATTR: return "rmxattr";
+ case CEPH_MDS_OP_READDIR: return "readdir";
+ case CEPH_MDS_OP_MKNOD: return "mknod";
+ case CEPH_MDS_OP_LINK: return "link";
+ case CEPH_MDS_OP_UNLINK: return "unlink";
+ case CEPH_MDS_OP_RENAME: return "rename";
+ case CEPH_MDS_OP_MKDIR: return "mkdir";
+ case CEPH_MDS_OP_RMDIR: return "rmdir";
+ case CEPH_MDS_OP_SYMLINK: return "symlink";
+ case CEPH_MDS_OP_CREATE: return "create";
+ case CEPH_MDS_OP_OPEN: return "open";
+ case CEPH_MDS_OP_LOOKUPSNAP: return "lookupsnap";
+ case CEPH_MDS_OP_LSSNAP: return "lssnap";
+ case CEPH_MDS_OP_MKSNAP: return "mksnap";
+ case CEPH_MDS_OP_RMSNAP: return "rmsnap";
+ }
+ return "???";
+}
+
+const char *ceph_cap_op_name(int op)
+{
+ switch (op) {
+ case CEPH_CAP_OP_GRANT: return "grant";
+ case CEPH_CAP_OP_REVOKE: return "revoke";
+ case CEPH_CAP_OP_TRUNC: return "trunc";
+ case CEPH_CAP_OP_EXPORT: return "export";
+ case CEPH_CAP_OP_IMPORT: return "import";
+ case CEPH_CAP_OP_UPDATE: return "update";
+ case CEPH_CAP_OP_DROP: return "drop";
+ case CEPH_CAP_OP_FLUSH: return "flush";
+ case CEPH_CAP_OP_FLUSH_ACK: return "flush_ack";
+ case CEPH_CAP_OP_FLUSHSNAP: return "flushsnap";
+ case CEPH_CAP_OP_FLUSHSNAP_ACK: return "flushsnap_ack";
+ case CEPH_CAP_OP_RELEASE: return "release";
+ case CEPH_CAP_OP_RENEW: return "renew";
+ }
+ return "???";
+}
+
+const char *ceph_lease_op_name(int o)
+{
+ switch (o) {
+ case CEPH_MDS_LEASE_REVOKE: return "revoke";
+ case CEPH_MDS_LEASE_RELEASE: return "release";
+ case CEPH_MDS_LEASE_RENEW: return "renew";
+ case CEPH_MDS_LEASE_REVOKE_ACK: return "revoke_ack";
+ }
+ return "???";
+}
+
+const char *ceph_snap_op_name(int o)
+{
+ switch (o) {
+ case CEPH_SNAP_OP_UPDATE: return "update";
+ case CEPH_SNAP_OP_CREATE: return "create";
+ case CEPH_SNAP_OP_DESTROY: return "destroy";
+ case CEPH_SNAP_OP_SPLIT: return "split";
+ }
+ return "???";
+}
--- /dev/null
+
+#ifdef __KERNEL__
+# include <linux/slab.h>
+#else
+# include <stdlib.h>
+# include <assert.h>
+# define kfree(x) do { if (x) free(x); } while (0)
+# define BUG_ON(x) assert(!(x))
+#endif
+
+#include "crush.h"
+
+const char *crush_bucket_alg_name(int alg)
+{
+ switch (alg) {
+ case CRUSH_BUCKET_UNIFORM: return "uniform";
+ case CRUSH_BUCKET_LIST: return "list";
+ case CRUSH_BUCKET_TREE: return "tree";
+ case CRUSH_BUCKET_STRAW: return "straw";
+ default: return "unknown";
+ }
+}
+
+/**
+ * crush_get_bucket_item_weight - Get weight of an item in given bucket
+ * @b: bucket pointer
+ * @p: item index in bucket
+ */
+int crush_get_bucket_item_weight(struct crush_bucket *b, int p)
+{
+ if (p >= b->size)
+ return 0;
+
+ switch (b->alg) {
+ case CRUSH_BUCKET_UNIFORM:
+ return ((struct crush_bucket_uniform *)b)->item_weight;
+ case CRUSH_BUCKET_LIST:
+ return ((struct crush_bucket_list *)b)->item_weights[p];
+ case CRUSH_BUCKET_TREE:
+ if (p & 1)
+ return ((struct crush_bucket_tree *)b)->node_weights[p];
+ return 0;
+ case CRUSH_BUCKET_STRAW:
+ return ((struct crush_bucket_straw *)b)->item_weights[p];
+ }
+ return 0;
+}
+
+/**
+ * crush_calc_parents - Calculate parent vectors for the given crush map.
+ * @map: crush_map pointer
+ */
+void crush_calc_parents(struct crush_map *map)
+{
+ int i, b, c;
+
+ for (b = 0; b < map->max_buckets; b++) {
+ if (map->buckets[b] == NULL)
+ continue;
+ for (i = 0; i < map->buckets[b]->size; i++) {
+ c = map->buckets[b]->items[i];
+ BUG_ON(c >= map->max_devices ||
+ c < -map->max_buckets);
+ if (c >= 0)
+ map->device_parents[c] = map->buckets[b]->id;
+ else
+ map->bucket_parents[-1-c] = map->buckets[b]->id;
+ }
+ }
+}
+
+void crush_destroy_bucket_uniform(struct crush_bucket_uniform *b)
+{
+ kfree(b->h.perm);
+ kfree(b->h.items);
+ kfree(b);
+}
+
+void crush_destroy_bucket_list(struct crush_bucket_list *b)
+{
+ kfree(b->item_weights);
+ kfree(b->sum_weights);
+ kfree(b->h.perm);
+ kfree(b->h.items);
+ kfree(b);
+}
+
+void crush_destroy_bucket_tree(struct crush_bucket_tree *b)
+{
+ kfree(b->node_weights);
+ kfree(b);
+}
+
+void crush_destroy_bucket_straw(struct crush_bucket_straw *b)
+{
+ kfree(b->straws);
+ kfree(b->item_weights);
+ kfree(b->h.perm);
+ kfree(b->h.items);
+ kfree(b);
+}
+
+void crush_destroy_bucket(struct crush_bucket *b)
+{
+ switch (b->alg) {
+ case CRUSH_BUCKET_UNIFORM:
+ crush_destroy_bucket_uniform((struct crush_bucket_uniform *)b);
+ break;
+ case CRUSH_BUCKET_LIST:
+ crush_destroy_bucket_list((struct crush_bucket_list *)b);
+ break;
+ case CRUSH_BUCKET_TREE:
+ crush_destroy_bucket_tree((struct crush_bucket_tree *)b);
+ break;
+ case CRUSH_BUCKET_STRAW:
+ crush_destroy_bucket_straw((struct crush_bucket_straw *)b);
+ break;
+ }
+}
+
+/**
+ * crush_destroy - Destroy a crush_map
+ * @map: crush_map pointer
+ */
+void crush_destroy(struct crush_map *map)
+{
+ int b;
+
+ /* buckets */
+ if (map->buckets) {
+ for (b = 0; b < map->max_buckets; b++) {
+ if (map->buckets[b] == NULL)
+ continue;
+ crush_destroy_bucket(map->buckets[b]);
+ }
+ kfree(map->buckets);
+ }
+
+ /* rules */
+ if (map->rules) {
+ for (b = 0; b < map->max_rules; b++)
+ kfree(map->rules[b]);
+ kfree(map->rules);
+ }
+
+ kfree(map->bucket_parents);
+ kfree(map->device_parents);
+ kfree(map);
+}
+
+
--- /dev/null
+#ifndef _CRUSH_CRUSH_H
+#define _CRUSH_CRUSH_H
+
+#include <linux/types.h>
+
+/*
+ * CRUSH is a pseudo-random data distribution algorithm that
+ * efficiently distributes input values (typically, data objects)
+ * across a heterogeneous, structured storage cluster.
+ *
+ * The algorithm was originally described in detail in this paper
+ * (although the algorithm has evolved somewhat since then):
+ *
+ * http://www.ssrc.ucsc.edu/Papers/weil-sc06.pdf
+ *
+ * LGPL2
+ */
+
+
+#define CRUSH_MAGIC 0x00010000ul /* for detecting algorithm revisions */
+
+
+#define CRUSH_MAX_DEPTH 10 /* max crush hierarchy depth */
+#define CRUSH_MAX_SET 10 /* max size of a mapping result */
+
+
+/*
+ * CRUSH uses user-defined "rules" to describe how inputs should be
+ * mapped to devices. A rule consists of sequence of steps to perform
+ * to generate the set of output devices.
+ */
+struct crush_rule_step {
+ __u32 op;
+ __s32 arg1;
+ __s32 arg2;
+};
+
+/* step op codes */
+enum {
+ CRUSH_RULE_NOOP = 0,
+ CRUSH_RULE_TAKE = 1, /* arg1 = value to start with */
+ CRUSH_RULE_CHOOSE_FIRSTN = 2, /* arg1 = num items to pick */
+ /* arg2 = type */
+ CRUSH_RULE_CHOOSE_INDEP = 3, /* same */
+ CRUSH_RULE_EMIT = 4, /* no args */
+ CRUSH_RULE_CHOOSE_LEAF_FIRSTN = 6,
+ CRUSH_RULE_CHOOSE_LEAF_INDEP = 7,
+};
+
+/*
+ * for specifying choose num (arg1) relative to the max parameter
+ * passed to do_rule
+ */
+#define CRUSH_CHOOSE_N 0
+#define CRUSH_CHOOSE_N_MINUS(x) (-(x))
+
+/*
+ * The rule mask is used to describe what the rule is intended for.
+ * Given a ruleset and size of output set, we search through the
+ * rule list for a matching rule_mask.
+ */
+struct crush_rule_mask {
+ __u8 ruleset;
+ __u8 type;
+ __u8 min_size;
+ __u8 max_size;
+};
+
+struct crush_rule {
+ __u32 len;
+ struct crush_rule_mask mask;
+ struct crush_rule_step steps[0];
+};
+
+#define crush_rule_size(len) (sizeof(struct crush_rule) + \
+ (len)*sizeof(struct crush_rule_step))
+
+
+
+/*
+ * A bucket is a named container of other items (either devices or
+ * other buckets). Items within a bucket are chosen using one of a
+ * few different algorithms. The table summarizes how the speed of
+ * each option measures up against mapping stability when items are
+ * added or removed.
+ *
+ * Bucket Alg Speed Additions Removals
+ * ------------------------------------------------
+ * uniform O(1) poor poor
+ * list O(n) optimal poor
+ * tree O(log n) good good
+ * straw O(n) optimal optimal
+ */
+enum {
+ CRUSH_BUCKET_UNIFORM = 1,
+ CRUSH_BUCKET_LIST = 2,
+ CRUSH_BUCKET_TREE = 3,
+ CRUSH_BUCKET_STRAW = 4
+};
+extern const char *crush_bucket_alg_name(int alg);
+
+struct crush_bucket {
+ __s32 id; /* this'll be negative */
+ __u16 type; /* non-zero; type=0 is reserved for devices */
+ __u8 alg; /* one of CRUSH_BUCKET_* */
+ __u8 hash; /* which hash function to use, CRUSH_HASH_* */
+ __u32 weight; /* 16-bit fixed point */
+ __u32 size; /* num items */
+ __s32 *items;
+
+ /*
+ * cached random permutation: used for uniform bucket and for
+ * the linear search fallback for the other bucket types.
+ */
+ __u32 perm_x; /* @x for which *perm is defined */
+ __u32 perm_n; /* num elements of *perm that are permuted/defined */
+ __u32 *perm;
+};
+
+struct crush_bucket_uniform {
+ struct crush_bucket h;
+ __u32 item_weight; /* 16-bit fixed point; all items equally weighted */
+};
+
+struct crush_bucket_list {
+ struct crush_bucket h;
+ __u32 *item_weights; /* 16-bit fixed point */
+ __u32 *sum_weights; /* 16-bit fixed point. element i is sum
+ of weights 0..i, inclusive */
+};
+
+struct crush_bucket_tree {
+ struct crush_bucket h; /* note: h.size is _tree_ size, not number of
+ actual items */
+ __u8 num_nodes;
+ __u32 *node_weights;
+};
+
+struct crush_bucket_straw {
+ struct crush_bucket h;
+ __u32 *item_weights; /* 16-bit fixed point */
+ __u32 *straws; /* 16-bit fixed point */
+};
+
+
+
+/*
+ * CRUSH map includes all buckets, rules, etc.
+ */
+struct crush_map {
+ struct crush_bucket **buckets;
+ struct crush_rule **rules;
+
+ /*
+ * Parent pointers to identify the parent bucket a device or
+ * bucket in the hierarchy. If an item appears more than
+ * once, this is the _last_ time it appeared (where buckets
+ * are processed in bucket id order, from -1 on down to
+ * -max_buckets.
+ */
+ __u32 *bucket_parents;
+ __u32 *device_parents;
+
+ __s32 max_buckets;
+ __u32 max_rules;
+ __s32 max_devices;
+};
+
+
+/* crush.c */
+extern int crush_get_bucket_item_weight(struct crush_bucket *b, int pos);
+extern void crush_calc_parents(struct crush_map *map);
+extern void crush_destroy_bucket_uniform(struct crush_bucket_uniform *b);
+extern void crush_destroy_bucket_list(struct crush_bucket_list *b);
+extern void crush_destroy_bucket_tree(struct crush_bucket_tree *b);
+extern void crush_destroy_bucket_straw(struct crush_bucket_straw *b);
+extern void crush_destroy_bucket(struct crush_bucket *b);
+extern void crush_destroy(struct crush_map *map);
+
+#endif
--- /dev/null
+
+#include <linux/types.h>
+#include "hash.h"
+
+/*
+ * Robert Jenkins' function for mixing 32-bit values
+ * http://burtleburtle.net/bob/hash/evahash.html
+ * a, b = random bits, c = input and output
+ */
+#define crush_hashmix(a, b, c) do { \
+ a = a-b; a = a-c; a = a^(c>>13); \
+ b = b-c; b = b-a; b = b^(a<<8); \
+ c = c-a; c = c-b; c = c^(b>>13); \
+ a = a-b; a = a-c; a = a^(c>>12); \
+ b = b-c; b = b-a; b = b^(a<<16); \
+ c = c-a; c = c-b; c = c^(b>>5); \
+ a = a-b; a = a-c; a = a^(c>>3); \
+ b = b-c; b = b-a; b = b^(a<<10); \
+ c = c-a; c = c-b; c = c^(b>>15); \
+ } while (0)
+
+#define crush_hash_seed 1315423911
+
+static __u32 crush_hash32_rjenkins1(__u32 a)
+{
+ __u32 hash = crush_hash_seed ^ a;
+ __u32 b = a;
+ __u32 x = 231232;
+ __u32 y = 1232;
+ crush_hashmix(b, x, hash);
+ crush_hashmix(y, a, hash);
+ return hash;
+}
+
+static __u32 crush_hash32_rjenkins1_2(__u32 a, __u32 b)
+{
+ __u32 hash = crush_hash_seed ^ a ^ b;
+ __u32 x = 231232;
+ __u32 y = 1232;
+ crush_hashmix(a, b, hash);
+ crush_hashmix(x, a, hash);
+ crush_hashmix(b, y, hash);
+ return hash;
+}
+
+static __u32 crush_hash32_rjenkins1_3(__u32 a, __u32 b, __u32 c)
+{
+ __u32 hash = crush_hash_seed ^ a ^ b ^ c;
+ __u32 x = 231232;
+ __u32 y = 1232;
+ crush_hashmix(a, b, hash);
+ crush_hashmix(c, x, hash);
+ crush_hashmix(y, a, hash);
+ crush_hashmix(b, x, hash);
+ crush_hashmix(y, c, hash);
+ return hash;
+}
+
+static __u32 crush_hash32_rjenkins1_4(__u32 a, __u32 b, __u32 c, __u32 d)
+{
+ __u32 hash = crush_hash_seed ^ a ^ b ^ c ^ d;
+ __u32 x = 231232;
+ __u32 y = 1232;
+ crush_hashmix(a, b, hash);
+ crush_hashmix(c, d, hash);
+ crush_hashmix(a, x, hash);
+ crush_hashmix(y, b, hash);
+ crush_hashmix(c, x, hash);
+ crush_hashmix(y, d, hash);
+ return hash;
+}
+
+static __u32 crush_hash32_rjenkins1_5(__u32 a, __u32 b, __u32 c, __u32 d,
+ __u32 e)
+{
+ __u32 hash = crush_hash_seed ^ a ^ b ^ c ^ d ^ e;
+ __u32 x = 231232;
+ __u32 y = 1232;
+ crush_hashmix(a, b, hash);
+ crush_hashmix(c, d, hash);
+ crush_hashmix(e, x, hash);
+ crush_hashmix(y, a, hash);
+ crush_hashmix(b, x, hash);
+ crush_hashmix(y, c, hash);
+ crush_hashmix(d, x, hash);
+ crush_hashmix(y, e, hash);
+ return hash;
+}
+
+
+__u32 crush_hash32(int type, __u32 a)
+{
+ switch (type) {
+ case CRUSH_HASH_RJENKINS1:
+ return crush_hash32_rjenkins1(a);
+ default:
+ return 0;
+ }
+}
+
+__u32 crush_hash32_2(int type, __u32 a, __u32 b)
+{
+ switch (type) {
+ case CRUSH_HASH_RJENKINS1:
+ return crush_hash32_rjenkins1_2(a, b);
+ default:
+ return 0;
+ }
+}
+
+__u32 crush_hash32_3(int type, __u32 a, __u32 b, __u32 c)
+{
+ switch (type) {
+ case CRUSH_HASH_RJENKINS1:
+ return crush_hash32_rjenkins1_3(a, b, c);
+ default:
+ return 0;
+ }
+}
+
+__u32 crush_hash32_4(int type, __u32 a, __u32 b, __u32 c, __u32 d)
+{
+ switch (type) {
+ case CRUSH_HASH_RJENKINS1:
+ return crush_hash32_rjenkins1_4(a, b, c, d);
+ default:
+ return 0;
+ }
+}
+
+__u32 crush_hash32_5(int type, __u32 a, __u32 b, __u32 c, __u32 d, __u32 e)
+{
+ switch (type) {
+ case CRUSH_HASH_RJENKINS1:
+ return crush_hash32_rjenkins1_5(a, b, c, d, e);
+ default:
+ return 0;
+ }
+}
+
+const char *crush_hash_name(int type)
+{
+ switch (type) {
+ case CRUSH_HASH_RJENKINS1:
+ return "rjenkins1";
+ default:
+ return "unknown";
+ }
+}
--- /dev/null
+#ifndef _CRUSH_HASH_H
+#define _CRUSH_HASH_H
+
+#define CRUSH_HASH_RJENKINS1 0
+
+#define CRUSH_HASH_DEFAULT CRUSH_HASH_RJENKINS1
+
+extern const char *crush_hash_name(int type);
+
+extern __u32 crush_hash32(int type, __u32 a);
+extern __u32 crush_hash32_2(int type, __u32 a, __u32 b);
+extern __u32 crush_hash32_3(int type, __u32 a, __u32 b, __u32 c);
+extern __u32 crush_hash32_4(int type, __u32 a, __u32 b, __u32 c, __u32 d);
+extern __u32 crush_hash32_5(int type, __u32 a, __u32 b, __u32 c, __u32 d,
+ __u32 e);
+
+#endif
--- /dev/null
+
+#ifdef __KERNEL__
+# include <linux/string.h>
+# include <linux/slab.h>
+# include <linux/bug.h>
+# include <linux/kernel.h>
+# ifndef dprintk
+# define dprintk(args...)
+# endif
+#else
+# include <string.h>
+# include <stdio.h>
+# include <stdlib.h>
+# include <assert.h>
+# define BUG_ON(x) assert(!(x))
+# define dprintk(args...) /* printf(args) */
+# define kmalloc(x, f) malloc(x)
+# define kfree(x) free(x)
+#endif
+
+#include "crush.h"
+#include "hash.h"
+
+/*
+ * Implement the core CRUSH mapping algorithm.
+ */
+
+/**
+ * crush_find_rule - find a crush_rule id for a given ruleset, type, and size.
+ * @map: the crush_map
+ * @ruleset: the storage ruleset id (user defined)
+ * @type: storage ruleset type (user defined)
+ * @size: output set size
+ */
+int crush_find_rule(struct crush_map *map, int ruleset, int type, int size)
+{
+ int i;
+
+ for (i = 0; i < map->max_rules; i++) {
+ if (map->rules[i] &&
+ map->rules[i]->mask.ruleset == ruleset &&
+ map->rules[i]->mask.type == type &&
+ map->rules[i]->mask.min_size <= size &&
+ map->rules[i]->mask.max_size >= size)
+ return i;
+ }
+ return -1;
+}
+
+
+/*
+ * bucket choose methods
+ *
+ * For each bucket algorithm, we have a "choose" method that, given a
+ * crush input @x and replica position (usually, position in output set) @r,
+ * will produce an item in the bucket.
+ */
+
+/*
+ * Choose based on a random permutation of the bucket.
+ *
+ * We used to use some prime number arithmetic to do this, but it
+ * wasn't very random, and had some other bad behaviors. Instead, we
+ * calculate an actual random permutation of the bucket members.
+ * Since this is expensive, we optimize for the r=0 case, which
+ * captures the vast majority of calls.
+ */
+static int bucket_perm_choose(struct crush_bucket *bucket,
+ int x, int r)
+{
+ unsigned pr = r % bucket->size;
+ unsigned i, s;
+
+ /* start a new permutation if @x has changed */
+ if (bucket->perm_x != x || bucket->perm_n == 0) {
+ dprintk("bucket %d new x=%d\n", bucket->id, x);
+ bucket->perm_x = x;
+
+ /* optimize common r=0 case */
+ if (pr == 0) {
+ s = crush_hash32_3(bucket->hash, x, bucket->id, 0) %
+ bucket->size;
+ bucket->perm[0] = s;
+ bucket->perm_n = 0xffff; /* magic value, see below */
+ goto out;
+ }
+
+ for (i = 0; i < bucket->size; i++)
+ bucket->perm[i] = i;
+ bucket->perm_n = 0;
+ } else if (bucket->perm_n == 0xffff) {
+ /* clean up after the r=0 case above */
+ for (i = 1; i < bucket->size; i++)
+ bucket->perm[i] = i;
+ bucket->perm[bucket->perm[0]] = 0;
+ bucket->perm_n = 1;
+ }
+
+ /* calculate permutation up to pr */
+ for (i = 0; i < bucket->perm_n; i++)
+ dprintk(" perm_choose have %d: %d\n", i, bucket->perm[i]);
+ while (bucket->perm_n <= pr) {
+ unsigned p = bucket->perm_n;
+ /* no point in swapping the final entry */
+ if (p < bucket->size - 1) {
+ i = crush_hash32_3(bucket->hash, x, bucket->id, p) %
+ (bucket->size - p);
+ if (i) {
+ unsigned t = bucket->perm[p + i];
+ bucket->perm[p + i] = bucket->perm[p];
+ bucket->perm[p] = t;
+ }
+ dprintk(" perm_choose swap %d with %d\n", p, p+i);
+ }
+ bucket->perm_n++;
+ }
+ for (i = 0; i < bucket->size; i++)
+ dprintk(" perm_choose %d: %d\n", i, bucket->perm[i]);
+
+ s = bucket->perm[pr];
+out:
+ dprintk(" perm_choose %d sz=%d x=%d r=%d (%d) s=%d\n", bucket->id,
+ bucket->size, x, r, pr, s);
+ return bucket->items[s];
+}
+
+/* uniform */
+static int bucket_uniform_choose(struct crush_bucket_uniform *bucket,
+ int x, int r)
+{
+ return bucket_perm_choose(&bucket->h, x, r);
+}
+
+/* list */
+static int bucket_list_choose(struct crush_bucket_list *bucket,
+ int x, int r)
+{
+ int i;
+
+ for (i = bucket->h.size-1; i >= 0; i--) {
+ __u64 w = crush_hash32_4(bucket->h.hash,x, bucket->h.items[i],
+ r, bucket->h.id);
+ w &= 0xffff;
+ dprintk("list_choose i=%d x=%d r=%d item %d weight %x "
+ "sw %x rand %llx",
+ i, x, r, bucket->h.items[i], bucket->item_weights[i],
+ bucket->sum_weights[i], w);
+ w *= bucket->sum_weights[i];
+ w = w >> 16;
+ /*dprintk(" scaled %llx\n", w);*/
+ if (w < bucket->item_weights[i])
+ return bucket->h.items[i];
+ }
+
+ BUG_ON(1);
+ return 0;
+}
+
+
+/* (binary) tree */
+static int height(int n)
+{
+ int h = 0;
+ while ((n & 1) == 0) {
+ h++;
+ n = n >> 1;
+ }
+ return h;
+}
+
+static int left(int x)
+{
+ int h = height(x);
+ return x - (1 << (h-1));
+}
+
+static int right(int x)
+{
+ int h = height(x);
+ return x + (1 << (h-1));
+}
+
+static int terminal(int x)
+{
+ return x & 1;
+}
+
+static int bucket_tree_choose(struct crush_bucket_tree *bucket,
+ int x, int r)
+{
+ int n, l;
+ __u32 w;
+ __u64 t;
+
+ /* start at root */
+ n = bucket->num_nodes >> 1;
+
+ while (!terminal(n)) {
+ /* pick point in [0, w) */
+ w = bucket->node_weights[n];
+ t = (__u64)crush_hash32_4(bucket->h.hash, x, n, r,
+ bucket->h.id) * (__u64)w;
+ t = t >> 32;
+
+ /* descend to the left or right? */
+ l = left(n);
+ if (t < bucket->node_weights[l])
+ n = l;
+ else
+ n = right(n);
+ }
+
+ return bucket->h.items[n >> 1];
+}
+
+
+/* straw */
+
+static int bucket_straw_choose(struct crush_bucket_straw *bucket,
+ int x, int r)
+{
+ int i;
+ int high = 0;
+ __u64 high_draw = 0;
+ __u64 draw;
+
+ for (i = 0; i < bucket->h.size; i++) {
+ draw = crush_hash32_3(bucket->h.hash, x, bucket->h.items[i], r);
+ draw &= 0xffff;
+ draw *= bucket->straws[i];
+ if (i == 0 || draw > high_draw) {
+ high = i;
+ high_draw = draw;
+ }
+ }
+ return bucket->h.items[high];
+}
+
+static int crush_bucket_choose(struct crush_bucket *in, int x, int r)
+{
+ dprintk("choose %d x=%d r=%d\n", in->id, x, r);
+ switch (in->alg) {
+ case CRUSH_BUCKET_UNIFORM:
+ return bucket_uniform_choose((struct crush_bucket_uniform *)in,
+ x, r);
+ case CRUSH_BUCKET_LIST:
+ return bucket_list_choose((struct crush_bucket_list *)in,
+ x, r);
+ case CRUSH_BUCKET_TREE:
+ return bucket_tree_choose((struct crush_bucket_tree *)in,
+ x, r);
+ case CRUSH_BUCKET_STRAW:
+ return bucket_straw_choose((struct crush_bucket_straw *)in,
+ x, r);
+ default:
+ BUG_ON(1);
+ return in->items[0];
+ }
+}
+
+/*
+ * true if device is marked "out" (failed, fully offloaded)
+ * of the cluster
+ */
+static int is_out(struct crush_map *map, __u32 *weight, int item, int x)
+{
+ if (weight[item] >= 0x1000)
+ return 0;
+ if (weight[item] == 0)
+ return 1;
+ if ((crush_hash32_2(CRUSH_HASH_RJENKINS1, x, item) & 0xffff)
+ < weight[item])
+ return 0;
+ return 1;
+}
+
+/**
+ * crush_choose - choose numrep distinct items of given type
+ * @map: the crush_map
+ * @bucket: the bucket we are choose an item from
+ * @x: crush input value
+ * @numrep: the number of items to choose
+ * @type: the type of item to choose
+ * @out: pointer to output vector
+ * @outpos: our position in that vector
+ * @firstn: true if choosing "first n" items, false if choosing "indep"
+ * @recurse_to_leaf: true if we want one device under each item of given type
+ * @out2: second output vector for leaf items (if @recurse_to_leaf)
+ */
+static int crush_choose(struct crush_map *map,
+ struct crush_bucket *bucket,
+ __u32 *weight,
+ int x, int numrep, int type,
+ int *out, int outpos,
+ int firstn, int recurse_to_leaf,
+ int *out2)
+{
+ int rep;
+ int ftotal, flocal;
+ int retry_descent, retry_bucket, skip_rep;
+ struct crush_bucket *in = bucket;
+ int r;
+ int i;
+ int item = 0;
+ int itemtype;
+ int collide, reject;
+ const int orig_tries = 5; /* attempts before we fall back to search */
+ dprintk("choose bucket %d x %d outpos %d\n", bucket->id, x, outpos);
+
+ for (rep = outpos; rep < numrep; rep++) {
+ /* keep trying until we get a non-out, non-colliding item */
+ ftotal = 0;
+ skip_rep = 0;
+ do {
+ retry_descent = 0;
+ in = bucket; /* initial bucket */
+
+ /* choose through intervening buckets */
+ flocal = 0;
+ do {
+ collide = 0;
+ retry_bucket = 0;
+ r = rep;
+ if (in->alg == CRUSH_BUCKET_UNIFORM) {
+ /* be careful */
+ if (firstn || numrep >= in->size)
+ /* r' = r + f_total */
+ r += ftotal;
+ else if (in->size % numrep == 0)
+ /* r'=r+(n+1)*f_local */
+ r += (numrep+1) *
+ (flocal+ftotal);
+ else
+ /* r' = r + n*f_local */
+ r += numrep * (flocal+ftotal);
+ } else {
+ if (firstn)
+ /* r' = r + f_total */
+ r += ftotal;
+ else
+ /* r' = r + n*f_local */
+ r += numrep * (flocal+ftotal);
+ }
+
+ /* bucket choose */
+ if (in->size == 0) {
+ reject = 1;
+ goto reject;
+ }
+ if (flocal >= (in->size>>1) &&
+ flocal > orig_tries)
+ item = bucket_perm_choose(in, x, r);
+ else
+ item = crush_bucket_choose(in, x, r);
+ BUG_ON(item >= map->max_devices);
+
+ /* desired type? */
+ if (item < 0)
+ itemtype = map->buckets[-1-item]->type;
+ else
+ itemtype = 0;
+ dprintk(" item %d type %d\n", item, itemtype);
+
+ /* keep going? */
+ if (itemtype != type) {
+ BUG_ON(item >= 0 ||
+ (-1-item) >= map->max_buckets);
+ in = map->buckets[-1-item];
+ continue;
+ }
+
+ /* collision? */
+ for (i = 0; i < outpos; i++) {
+ if (out[i] == item) {
+ collide = 1;
+ break;
+ }
+ }
+
+ if (recurse_to_leaf &&
+ item < 0 &&
+ crush_choose(map, map->buckets[-1-item],
+ weight,
+ x, outpos+1, 0,
+ out2, outpos,
+ firstn, 0, NULL) <= outpos) {
+ reject = 1;
+ } else {
+ /* out? */
+ if (itemtype == 0)
+ reject = is_out(map, weight,
+ item, x);
+ else
+ reject = 0;
+ }
+
+reject:
+ if (reject || collide) {
+ ftotal++;
+ flocal++;
+
+ if (collide && flocal < 3)
+ /* retry locally a few times */
+ retry_bucket = 1;
+ else if (flocal < in->size + orig_tries)
+ /* exhaustive bucket search */
+ retry_bucket = 1;
+ else if (ftotal < 20)
+ /* then retry descent */
+ retry_descent = 1;
+ else
+ /* else give up */
+ skip_rep = 1;
+ dprintk(" reject %d collide %d "
+ "ftotal %d flocal %d\n",
+ reject, collide, ftotal,
+ flocal);
+ }
+ } while (retry_bucket);
+ } while (retry_descent);
+
+ if (skip_rep) {
+ dprintk("skip rep\n");
+ continue;
+ }
+
+ dprintk("choose got %d\n", item);
+ out[outpos] = item;
+ outpos++;
+ }
+
+ dprintk("choose returns %d\n", outpos);
+ return outpos;
+}
+
+
+/**
+ * crush_do_rule - calculate a mapping with the given input and rule
+ * @map: the crush_map
+ * @ruleno: the rule id
+ * @x: hash input
+ * @result: pointer to result vector
+ * @result_max: maximum result size
+ * @force: force initial replica choice; -1 for none
+ */
+int crush_do_rule(struct crush_map *map,
+ int ruleno, int x, int *result, int result_max,
+ int force, __u32 *weight)
+{
+ int result_len;
+ int force_context[CRUSH_MAX_DEPTH];
+ int force_pos = -1;
+ int a[CRUSH_MAX_SET];
+ int b[CRUSH_MAX_SET];
+ int c[CRUSH_MAX_SET];
+ int recurse_to_leaf;
+ int *w;
+ int wsize = 0;
+ int *o;
+ int osize;
+ int *tmp;
+ struct crush_rule *rule;
+ int step;
+ int i, j;
+ int numrep;
+ int firstn;
+ int rc = -1;
+
+ BUG_ON(ruleno >= map->max_rules);
+
+ rule = map->rules[ruleno];
+ result_len = 0;
+ w = a;
+ o = b;
+
+ /*
+ * determine hierarchical context of force, if any. note
+ * that this may or may not correspond to the specific types
+ * referenced by the crush rule.
+ */
+ if (force >= 0) {
+ if (force >= map->max_devices ||
+ map->device_parents[force] == 0) {
+ /*dprintk("CRUSH: forcefed device dne\n");*/
+ rc = -1; /* force fed device dne */
+ goto out;
+ }
+ if (!is_out(map, weight, force, x)) {
+ while (1) {
+ force_context[++force_pos] = force;
+ if (force >= 0)
+ force = map->device_parents[force];
+ else
+ force = map->bucket_parents[-1-force];
+ if (force == 0)
+ break;
+ }
+ }
+ }
+
+ for (step = 0; step < rule->len; step++) {
+ firstn = 0;
+ switch (rule->steps[step].op) {
+ case CRUSH_RULE_TAKE:
+ w[0] = rule->steps[step].arg1;
+ if (force_pos >= 0) {
+ BUG_ON(force_context[force_pos] != w[0]);
+ force_pos--;
+ }
+ wsize = 1;
+ break;
+
+ case CRUSH_RULE_CHOOSE_LEAF_FIRSTN:
+ case CRUSH_RULE_CHOOSE_FIRSTN:
+ firstn = 1;
+ case CRUSH_RULE_CHOOSE_LEAF_INDEP:
+ case CRUSH_RULE_CHOOSE_INDEP:
+ BUG_ON(wsize == 0);
+
+ recurse_to_leaf =
+ rule->steps[step].op ==
+ CRUSH_RULE_CHOOSE_LEAF_FIRSTN ||
+ rule->steps[step].op ==
+ CRUSH_RULE_CHOOSE_LEAF_INDEP;
+
+ /* reset output */
+ osize = 0;
+
+ for (i = 0; i < wsize; i++) {
+ /*
+ * see CRUSH_N, CRUSH_N_MINUS macros.
+ * basically, numrep <= 0 means relative to
+ * the provided result_max
+ */
+ numrep = rule->steps[step].arg1;
+ if (numrep <= 0) {
+ numrep += result_max;
+ if (numrep <= 0)
+ continue;
+ }
+ j = 0;
+ if (osize == 0 && force_pos >= 0) {
+ /* skip any intermediate types */
+ while (force_pos &&
+ force_context[force_pos] < 0 &&
+ rule->steps[step].arg2 !=
+ map->buckets[-1 -
+ force_context[force_pos]]->type)
+ force_pos--;
+ o[osize] = force_context[force_pos];
+ if (recurse_to_leaf)
+ c[osize] = force_context[0];
+ j++;
+ force_pos--;
+ }
+ osize += crush_choose(map,
+ map->buckets[-1-w[i]],
+ weight,
+ x, numrep,
+ rule->steps[step].arg2,
+ o+osize, j,
+ firstn,
+ recurse_to_leaf, c+osize);
+ }
+
+ if (recurse_to_leaf)
+ /* copy final _leaf_ values to output set */
+ memcpy(o, c, osize*sizeof(*o));
+
+ /* swap t and w arrays */
+ tmp = o;
+ o = w;
+ w = tmp;
+ wsize = osize;
+ break;
+
+
+ case CRUSH_RULE_EMIT:
+ for (i = 0; i < wsize && result_len < result_max; i++) {
+ result[result_len] = w[i];
+ result_len++;
+ }
+ wsize = 0;
+ break;
+
+ default:
+ BUG_ON(1);
+ }
+ }
+ rc = result_len;
+
+out:
+ return rc;
+}
+
+
--- /dev/null
+#ifndef _CRUSH_MAPPER_H
+#define _CRUSH_MAPPER_H
+
+/*
+ * CRUSH functions for find rules and then mapping an input to an
+ * output set.
+ *
+ * LGPL2
+ */
+
+#include "crush.h"
+
+extern int crush_find_rule(struct crush_map *map, int pool, int type, int size);
+extern int crush_do_rule(struct crush_map *map,
+ int ruleno,
+ int x, int *result, int result_max,
+ int forcefeed, /* -1 for none */
+ __u32 *weights);
+
+#endif
--- /dev/null
+
+#include "ceph_debug.h"
+
+#include <linux/err.h>
+#include <linux/scatterlist.h>
+#include <linux/slab.h>
+#include <crypto/hash.h>
+
+#include "crypto.h"
+#include "decode.h"
+
+int ceph_crypto_key_encode(struct ceph_crypto_key *key, void **p, void *end)
+{
+ if (*p + sizeof(u16) + sizeof(key->created) +
+ sizeof(u16) + key->len > end)
+ return -ERANGE;
+ ceph_encode_16(p, key->type);
+ ceph_encode_copy(p, &key->created, sizeof(key->created));
+ ceph_encode_16(p, key->len);
+ ceph_encode_copy(p, key->key, key->len);
+ return 0;
+}
+
+int ceph_crypto_key_decode(struct ceph_crypto_key *key, void **p, void *end)
+{
+ ceph_decode_need(p, end, 2*sizeof(u16) + sizeof(key->created), bad);
+ key->type = ceph_decode_16(p);
+ ceph_decode_copy(p, &key->created, sizeof(key->created));
+ key->len = ceph_decode_16(p);
+ ceph_decode_need(p, end, key->len, bad);
+ key->key = kmalloc(key->len, GFP_NOFS);
+ if (!key->key)
+ return -ENOMEM;
+ ceph_decode_copy(p, key->key, key->len);
+ return 0;
+
+bad:
+ dout("failed to decode crypto key\n");
+ return -EINVAL;
+}
+
+int ceph_crypto_key_unarmor(struct ceph_crypto_key *key, const char *inkey)
+{
+ int inlen = strlen(inkey);
+ int blen = inlen * 3 / 4;
+ void *buf, *p;
+ int ret;
+
+ dout("crypto_key_unarmor %s\n", inkey);
+ buf = kmalloc(blen, GFP_NOFS);
+ if (!buf)
+ return -ENOMEM;
+ blen = ceph_unarmor(buf, inkey, inkey+inlen);
+ if (blen < 0) {
+ kfree(buf);
+ return blen;
+ }
+
+ p = buf;
+ ret = ceph_crypto_key_decode(key, &p, p + blen);
+ kfree(buf);
+ if (ret)
+ return ret;
+ dout("crypto_key_unarmor key %p type %d len %d\n", key,
+ key->type, key->len);
+ return 0;
+}
+
+
+
+#define AES_KEY_SIZE 16
+
+static struct crypto_blkcipher *ceph_crypto_alloc_cipher(void)
+{
+ return crypto_alloc_blkcipher("cbc(aes)", 0, CRYPTO_ALG_ASYNC);
+}
+
+const u8 *aes_iv = "cephsageyudagreg";
+
+int ceph_aes_encrypt(const void *key, int key_len, void *dst, size_t *dst_len,
+ const void *src, size_t src_len)
+{
+ struct scatterlist sg_in[2], sg_out[1];
+ struct crypto_blkcipher *tfm = ceph_crypto_alloc_cipher();
+ struct blkcipher_desc desc = { .tfm = tfm, .flags = 0 };
+ int ret;
+ void *iv;
+ int ivsize;
+ size_t zero_padding = (0x10 - (src_len & 0x0f));
+ char pad[16];
+
+ if (IS_ERR(tfm))
+ return PTR_ERR(tfm);
+
+ memset(pad, zero_padding, zero_padding);
+
+ *dst_len = src_len + zero_padding;
+
+ crypto_blkcipher_setkey((void *)tfm, key, key_len);
+ sg_init_table(sg_in, 2);
+ sg_set_buf(&sg_in[0], src, src_len);
+ sg_set_buf(&sg_in[1], pad, zero_padding);
+ sg_init_table(sg_out, 1);
+ sg_set_buf(sg_out, dst, *dst_len);
+ iv = crypto_blkcipher_crt(tfm)->iv;
+ ivsize = crypto_blkcipher_ivsize(tfm);
+
+ memcpy(iv, aes_iv, ivsize);
+ /*
+ print_hex_dump(KERN_ERR, "enc key: ", DUMP_PREFIX_NONE, 16, 1,
+ key, key_len, 1);
+ print_hex_dump(KERN_ERR, "enc src: ", DUMP_PREFIX_NONE, 16, 1,
+ src, src_len, 1);
+ print_hex_dump(KERN_ERR, "enc pad: ", DUMP_PREFIX_NONE, 16, 1,
+ pad, zero_padding, 1);
+ */
+ ret = crypto_blkcipher_encrypt(&desc, sg_out, sg_in,
+ src_len + zero_padding);
+ crypto_free_blkcipher(tfm);
+ if (ret < 0)
+ pr_err("ceph_aes_crypt failed %d\n", ret);
+ /*
+ print_hex_dump(KERN_ERR, "enc out: ", DUMP_PREFIX_NONE, 16, 1,
+ dst, *dst_len, 1);
+ */
+ return 0;
+}
+
+int ceph_aes_encrypt2(const void *key, int key_len, void *dst, size_t *dst_len,
+ const void *src1, size_t src1_len,
+ const void *src2, size_t src2_len)
+{
+ struct scatterlist sg_in[3], sg_out[1];
+ struct crypto_blkcipher *tfm = ceph_crypto_alloc_cipher();
+ struct blkcipher_desc desc = { .tfm = tfm, .flags = 0 };
+ int ret;
+ void *iv;
+ int ivsize;
+ size_t zero_padding = (0x10 - ((src1_len + src2_len) & 0x0f));
+ char pad[16];
+
+ if (IS_ERR(tfm))
+ return PTR_ERR(tfm);
+
+ memset(pad, zero_padding, zero_padding);
+
+ *dst_len = src1_len + src2_len + zero_padding;
+
+ crypto_blkcipher_setkey((void *)tfm, key, key_len);
+ sg_init_table(sg_in, 3);
+ sg_set_buf(&sg_in[0], src1, src1_len);
+ sg_set_buf(&sg_in[1], src2, src2_len);
+ sg_set_buf(&sg_in[2], pad, zero_padding);
+ sg_init_table(sg_out, 1);
+ sg_set_buf(sg_out, dst, *dst_len);
+ iv = crypto_blkcipher_crt(tfm)->iv;
+ ivsize = crypto_blkcipher_ivsize(tfm);
+
+ memcpy(iv, aes_iv, ivsize);
+ /*
+ print_hex_dump(KERN_ERR, "enc key: ", DUMP_PREFIX_NONE, 16, 1,
+ key, key_len, 1);
+ print_hex_dump(KERN_ERR, "enc src1: ", DUMP_PREFIX_NONE, 16, 1,
+ src1, src1_len, 1);
+ print_hex_dump(KERN_ERR, "enc src2: ", DUMP_PREFIX_NONE, 16, 1,
+ src2, src2_len, 1);
+ print_hex_dump(KERN_ERR, "enc pad: ", DUMP_PREFIX_NONE, 16, 1,
+ pad, zero_padding, 1);
+ */
+ ret = crypto_blkcipher_encrypt(&desc, sg_out, sg_in,
+ src1_len + src2_len + zero_padding);
+ crypto_free_blkcipher(tfm);
+ if (ret < 0)
+ pr_err("ceph_aes_crypt2 failed %d\n", ret);
+ /*
+ print_hex_dump(KERN_ERR, "enc out: ", DUMP_PREFIX_NONE, 16, 1,
+ dst, *dst_len, 1);
+ */
+ return 0;
+}
+
+int ceph_aes_decrypt(const void *key, int key_len, void *dst, size_t *dst_len,
+ const void *src, size_t src_len)
+{
+ struct scatterlist sg_in[1], sg_out[2];
+ struct crypto_blkcipher *tfm = ceph_crypto_alloc_cipher();
+ struct blkcipher_desc desc = { .tfm = tfm };
+ char pad[16];
+ void *iv;
+ int ivsize;
+ int ret;
+ int last_byte;
+
+ if (IS_ERR(tfm))
+ return PTR_ERR(tfm);
+
+ crypto_blkcipher_setkey((void *)tfm, key, key_len);
+ sg_init_table(sg_in, 1);
+ sg_init_table(sg_out, 2);
+ sg_set_buf(sg_in, src, src_len);
+ sg_set_buf(&sg_out[0], dst, *dst_len);
+ sg_set_buf(&sg_out[1], pad, sizeof(pad));
+
+ iv = crypto_blkcipher_crt(tfm)->iv;
+ ivsize = crypto_blkcipher_ivsize(tfm);
+
+ memcpy(iv, aes_iv, ivsize);
+
+ /*
+ print_hex_dump(KERN_ERR, "dec key: ", DUMP_PREFIX_NONE, 16, 1,
+ key, key_len, 1);
+ print_hex_dump(KERN_ERR, "dec in: ", DUMP_PREFIX_NONE, 16, 1,
+ src, src_len, 1);
+ */
+
+ ret = crypto_blkcipher_decrypt(&desc, sg_out, sg_in, src_len);
+ crypto_free_blkcipher(tfm);
+ if (ret < 0) {
+ pr_err("ceph_aes_decrypt failed %d\n", ret);
+ return ret;
+ }
+
+ if (src_len <= *dst_len)
+ last_byte = ((char *)dst)[src_len - 1];
+ else
+ last_byte = pad[src_len - *dst_len - 1];
+ if (last_byte <= 16 && src_len >= last_byte) {
+ *dst_len = src_len - last_byte;
+ } else {
+ pr_err("ceph_aes_decrypt got bad padding %d on src len %d\n",
+ last_byte, (int)src_len);
+ return -EPERM; /* bad padding */
+ }
+ /*
+ print_hex_dump(KERN_ERR, "dec out: ", DUMP_PREFIX_NONE, 16, 1,
+ dst, *dst_len, 1);
+ */
+ return 0;
+}
+
+int ceph_aes_decrypt2(const void *key, int key_len,
+ void *dst1, size_t *dst1_len,
+ void *dst2, size_t *dst2_len,
+ const void *src, size_t src_len)
+{
+ struct scatterlist sg_in[1], sg_out[3];
+ struct crypto_blkcipher *tfm = ceph_crypto_alloc_cipher();
+ struct blkcipher_desc desc = { .tfm = tfm };
+ char pad[16];
+ void *iv;
+ int ivsize;
+ int ret;
+ int last_byte;
+
+ if (IS_ERR(tfm))
+ return PTR_ERR(tfm);
+
+ sg_init_table(sg_in, 1);
+ sg_set_buf(sg_in, src, src_len);
+ sg_init_table(sg_out, 3);
+ sg_set_buf(&sg_out[0], dst1, *dst1_len);
+ sg_set_buf(&sg_out[1], dst2, *dst2_len);
+ sg_set_buf(&sg_out[2], pad, sizeof(pad));
+
+ crypto_blkcipher_setkey((void *)tfm, key, key_len);
+ iv = crypto_blkcipher_crt(tfm)->iv;
+ ivsize = crypto_blkcipher_ivsize(tfm);
+
+ memcpy(iv, aes_iv, ivsize);
+
+ /*
+ print_hex_dump(KERN_ERR, "dec key: ", DUMP_PREFIX_NONE, 16, 1,
+ key, key_len, 1);
+ print_hex_dump(KERN_ERR, "dec in: ", DUMP_PREFIX_NONE, 16, 1,
+ src, src_len, 1);
+ */
+
+ ret = crypto_blkcipher_decrypt(&desc, sg_out, sg_in, src_len);
+ crypto_free_blkcipher(tfm);
+ if (ret < 0) {
+ pr_err("ceph_aes_decrypt failed %d\n", ret);
+ return ret;
+ }
+
+ if (src_len <= *dst1_len)
+ last_byte = ((char *)dst1)[src_len - 1];
+ else if (src_len <= *dst1_len + *dst2_len)
+ last_byte = ((char *)dst2)[src_len - *dst1_len - 1];
+ else
+ last_byte = pad[src_len - *dst1_len - *dst2_len - 1];
+ if (last_byte <= 16 && src_len >= last_byte) {
+ src_len -= last_byte;
+ } else {
+ pr_err("ceph_aes_decrypt got bad padding %d on src len %d\n",
+ last_byte, (int)src_len);
+ return -EPERM; /* bad padding */
+ }
+
+ if (src_len < *dst1_len) {
+ *dst1_len = src_len;
+ *dst2_len = 0;
+ } else {
+ *dst2_len = src_len - *dst1_len;
+ }
+ /*
+ print_hex_dump(KERN_ERR, "dec out1: ", DUMP_PREFIX_NONE, 16, 1,
+ dst1, *dst1_len, 1);
+ print_hex_dump(KERN_ERR, "dec out2: ", DUMP_PREFIX_NONE, 16, 1,
+ dst2, *dst2_len, 1);
+ */
+
+ return 0;
+}
+
+
+int ceph_decrypt(struct ceph_crypto_key *secret, void *dst, size_t *dst_len,
+ const void *src, size_t src_len)
+{
+ switch (secret->type) {
+ case CEPH_CRYPTO_NONE:
+ if (*dst_len < src_len)
+ return -ERANGE;
+ memcpy(dst, src, src_len);
+ *dst_len = src_len;
+ return 0;
+
+ case CEPH_CRYPTO_AES:
+ return ceph_aes_decrypt(secret->key, secret->len, dst,
+ dst_len, src, src_len);
+
+ default:
+ return -EINVAL;
+ }
+}
+
+int ceph_decrypt2(struct ceph_crypto_key *secret,
+ void *dst1, size_t *dst1_len,
+ void *dst2, size_t *dst2_len,
+ const void *src, size_t src_len)
+{
+ size_t t;
+
+ switch (secret->type) {
+ case CEPH_CRYPTO_NONE:
+ if (*dst1_len + *dst2_len < src_len)
+ return -ERANGE;
+ t = min(*dst1_len, src_len);
+ memcpy(dst1, src, t);
+ *dst1_len = t;
+ src += t;
+ src_len -= t;
+ if (src_len) {
+ t = min(*dst2_len, src_len);
+ memcpy(dst2, src, t);
+ *dst2_len = t;
+ }
+ return 0;
+
+ case CEPH_CRYPTO_AES:
+ return ceph_aes_decrypt2(secret->key, secret->len,
+ dst1, dst1_len, dst2, dst2_len,
+ src, src_len);
+
+ default:
+ return -EINVAL;
+ }
+}
+
+int ceph_encrypt(struct ceph_crypto_key *secret, void *dst, size_t *dst_len,
+ const void *src, size_t src_len)
+{
+ switch (secret->type) {
+ case CEPH_CRYPTO_NONE:
+ if (*dst_len < src_len)
+ return -ERANGE;
+ memcpy(dst, src, src_len);
+ *dst_len = src_len;
+ return 0;
+
+ case CEPH_CRYPTO_AES:
+ return ceph_aes_encrypt(secret->key, secret->len, dst,
+ dst_len, src, src_len);
+
+ default:
+ return -EINVAL;
+ }
+}
+
+int ceph_encrypt2(struct ceph_crypto_key *secret, void *dst, size_t *dst_len,
+ const void *src1, size_t src1_len,
+ const void *src2, size_t src2_len)
+{
+ switch (secret->type) {
+ case CEPH_CRYPTO_NONE:
+ if (*dst_len < src1_len + src2_len)
+ return -ERANGE;
+ memcpy(dst, src1, src1_len);
+ memcpy(dst + src1_len, src2, src2_len);
+ *dst_len = src1_len + src2_len;
+ return 0;
+
+ case CEPH_CRYPTO_AES:
+ return ceph_aes_encrypt2(secret->key, secret->len, dst, dst_len,
+ src1, src1_len, src2, src2_len);
+
+ default:
+ return -EINVAL;
+ }
+}
--- /dev/null
+#ifndef _FS_CEPH_CRYPTO_H
+#define _FS_CEPH_CRYPTO_H
+
+#include "types.h"
+#include "buffer.h"
+
+/*
+ * cryptographic secret
+ */
+struct ceph_crypto_key {
+ int type;
+ struct ceph_timespec created;
+ int len;
+ void *key;
+};
+
+static inline void ceph_crypto_key_destroy(struct ceph_crypto_key *key)
+{
+ kfree(key->key);
+}
+
+extern int ceph_crypto_key_encode(struct ceph_crypto_key *key,
+ void **p, void *end);
+extern int ceph_crypto_key_decode(struct ceph_crypto_key *key,
+ void **p, void *end);
+extern int ceph_crypto_key_unarmor(struct ceph_crypto_key *key, const char *in);
+
+/* crypto.c */
+extern int ceph_decrypt(struct ceph_crypto_key *secret,
+ void *dst, size_t *dst_len,
+ const void *src, size_t src_len);
+extern int ceph_encrypt(struct ceph_crypto_key *secret,
+ void *dst, size_t *dst_len,
+ const void *src, size_t src_len);
+extern int ceph_decrypt2(struct ceph_crypto_key *secret,
+ void *dst1, size_t *dst1_len,
+ void *dst2, size_t *dst2_len,
+ const void *src, size_t src_len);
+extern int ceph_encrypt2(struct ceph_crypto_key *secret,
+ void *dst, size_t *dst_len,
+ const void *src1, size_t src1_len,
+ const void *src2, size_t src2_len);
+
+/* armor.c */
+extern int ceph_armor(char *dst, const void *src, const void *end);
+extern int ceph_unarmor(void *dst, const char *src, const char *end);
+
+#endif
--- /dev/null
+#include "ceph_debug.h"
+
+#include <linux/device.h>
+#include <linux/slab.h>
+#include <linux/module.h>
+#include <linux/ctype.h>
+#include <linux/debugfs.h>
+#include <linux/seq_file.h>
+
+#include "super.h"
+#include "mds_client.h"
+#include "mon_client.h"
+#include "auth.h"
+
+#ifdef CONFIG_DEBUG_FS
+
+/*
+ * Implement /sys/kernel/debug/ceph fun
+ *
+ * /sys/kernel/debug/ceph/client* - an instance of the ceph client
+ * .../osdmap - current osdmap
+ * .../mdsmap - current mdsmap
+ * .../monmap - current monmap
+ * .../osdc - active osd requests
+ * .../mdsc - active mds requests
+ * .../monc - mon client state
+ * .../dentry_lru - dump contents of dentry lru
+ * .../caps - expose cap (reservation) stats
+ * .../bdi - symlink to ../../bdi/something
+ */
+
+static struct dentry *ceph_debugfs_dir;
+
+static int monmap_show(struct seq_file *s, void *p)
+{
+ int i;
+ struct ceph_client *client = s->private;
+
+ if (client->monc.monmap == NULL)
+ return 0;
+
+ seq_printf(s, "epoch %d\n", client->monc.monmap->epoch);
+ for (i = 0; i < client->monc.monmap->num_mon; i++) {
+ struct ceph_entity_inst *inst =
+ &client->monc.monmap->mon_inst[i];
+
+ seq_printf(s, "\t%s%lld\t%s\n",
+ ENTITY_NAME(inst->name),
+ pr_addr(&inst->addr.in_addr));
+ }
+ return 0;
+}
+
+static int mdsmap_show(struct seq_file *s, void *p)
+{
+ int i;
+ struct ceph_client *client = s->private;
+
+ if (client->mdsc.mdsmap == NULL)
+ return 0;
+ seq_printf(s, "epoch %d\n", client->mdsc.mdsmap->m_epoch);
+ seq_printf(s, "root %d\n", client->mdsc.mdsmap->m_root);
+ seq_printf(s, "session_timeout %d\n",
+ client->mdsc.mdsmap->m_session_timeout);
+ seq_printf(s, "session_autoclose %d\n",
+ client->mdsc.mdsmap->m_session_autoclose);
+ for (i = 0; i < client->mdsc.mdsmap->m_max_mds; i++) {
+ struct ceph_entity_addr *addr =
+ &client->mdsc.mdsmap->m_info[i].addr;
+ int state = client->mdsc.mdsmap->m_info[i].state;
+
+ seq_printf(s, "\tmds%d\t%s\t(%s)\n", i, pr_addr(&addr->in_addr),
+ ceph_mds_state_name(state));
+ }
+ return 0;
+}
+
+static int osdmap_show(struct seq_file *s, void *p)
+{
+ int i;
+ struct ceph_client *client = s->private;
+ struct rb_node *n;
+
+ if (client->osdc.osdmap == NULL)
+ return 0;
+ seq_printf(s, "epoch %d\n", client->osdc.osdmap->epoch);
+ seq_printf(s, "flags%s%s\n",
+ (client->osdc.osdmap->flags & CEPH_OSDMAP_NEARFULL) ?
+ " NEARFULL" : "",
+ (client->osdc.osdmap->flags & CEPH_OSDMAP_FULL) ?
+ " FULL" : "");
+ for (n = rb_first(&client->osdc.osdmap->pg_pools); n; n = rb_next(n)) {
+ struct ceph_pg_pool_info *pool =
+ rb_entry(n, struct ceph_pg_pool_info, node);
+ seq_printf(s, "pg_pool %d pg_num %d / %d, lpg_num %d / %d\n",
+ pool->id, pool->v.pg_num, pool->pg_num_mask,
+ pool->v.lpg_num, pool->lpg_num_mask);
+ }
+ for (i = 0; i < client->osdc.osdmap->max_osd; i++) {
+ struct ceph_entity_addr *addr =
+ &client->osdc.osdmap->osd_addr[i];
+ int state = client->osdc.osdmap->osd_state[i];
+ char sb[64];
+
+ seq_printf(s, "\tosd%d\t%s\t%3d%%\t(%s)\n",
+ i, pr_addr(&addr->in_addr),
+ ((client->osdc.osdmap->osd_weight[i]*100) >> 16),
+ ceph_osdmap_state_str(sb, sizeof(sb), state));
+ }
+ return 0;
+}
+
+static int monc_show(struct seq_file *s, void *p)
+{
+ struct ceph_client *client = s->private;
+ struct ceph_mon_statfs_request *req;
+ struct ceph_mon_client *monc = &client->monc;
+ struct rb_node *rp;
+
+ mutex_lock(&monc->mutex);
+
+ if (monc->have_mdsmap)
+ seq_printf(s, "have mdsmap %u\n", (unsigned)monc->have_mdsmap);
+ if (monc->have_osdmap)
+ seq_printf(s, "have osdmap %u\n", (unsigned)monc->have_osdmap);
+ if (monc->want_next_osdmap)
+ seq_printf(s, "want next osdmap\n");
+
+ for (rp = rb_first(&monc->statfs_request_tree); rp; rp = rb_next(rp)) {
+ req = rb_entry(rp, struct ceph_mon_statfs_request, node);
+ seq_printf(s, "%lld statfs\n", req->tid);
+ }
+
+ mutex_unlock(&monc->mutex);
+ return 0;
+}
+
+static int mdsc_show(struct seq_file *s, void *p)
+{
+ struct ceph_client *client = s->private;
+ struct ceph_mds_client *mdsc = &client->mdsc;
+ struct ceph_mds_request *req;
+ struct rb_node *rp;
+ int pathlen;
+ u64 pathbase;
+ char *path;
+
+ mutex_lock(&mdsc->mutex);
+ for (rp = rb_first(&mdsc->request_tree); rp; rp = rb_next(rp)) {
+ req = rb_entry(rp, struct ceph_mds_request, r_node);
+
+ if (req->r_request)
+ seq_printf(s, "%lld\tmds%d\t", req->r_tid, req->r_mds);
+ else
+ seq_printf(s, "%lld\t(no request)\t", req->r_tid);
+
+ seq_printf(s, "%s", ceph_mds_op_name(req->r_op));
+
+ if (req->r_got_unsafe)
+ seq_printf(s, "\t(unsafe)");
+ else
+ seq_printf(s, "\t");
+
+ if (req->r_inode) {
+ seq_printf(s, " #%llx", ceph_ino(req->r_inode));
+ } else if (req->r_dentry) {
+ path = ceph_mdsc_build_path(req->r_dentry, &pathlen,
+ &pathbase, 0);
+ spin_lock(&req->r_dentry->d_lock);
+ seq_printf(s, " #%llx/%.*s (%s)",
+ ceph_ino(req->r_dentry->d_parent->d_inode),
+ req->r_dentry->d_name.len,
+ req->r_dentry->d_name.name,
+ path ? path : "");
+ spin_unlock(&req->r_dentry->d_lock);
+ kfree(path);
+ } else if (req->r_path1) {
+ seq_printf(s, " #%llx/%s", req->r_ino1.ino,
+ req->r_path1);
+ }
+
+ if (req->r_old_dentry) {
+ path = ceph_mdsc_build_path(req->r_old_dentry, &pathlen,
+ &pathbase, 0);
+ spin_lock(&req->r_old_dentry->d_lock);
+ seq_printf(s, " #%llx/%.*s (%s)",
+ ceph_ino(req->r_old_dentry->d_parent->d_inode),
+ req->r_old_dentry->d_name.len,
+ req->r_old_dentry->d_name.name,
+ path ? path : "");
+ spin_unlock(&req->r_old_dentry->d_lock);
+ kfree(path);
+ } else if (req->r_path2) {
+ if (req->r_ino2.ino)
+ seq_printf(s, " #%llx/%s", req->r_ino2.ino,
+ req->r_path2);
+ else
+ seq_printf(s, " %s", req->r_path2);
+ }
+
+ seq_printf(s, "\n");
+ }
+ mutex_unlock(&mdsc->mutex);
+
+ return 0;
+}
+
+static int osdc_show(struct seq_file *s, void *pp)
+{
+ struct ceph_client *client = s->private;
+ struct ceph_osd_client *osdc = &client->osdc;
+ struct rb_node *p;
+
+ mutex_lock(&osdc->request_mutex);
+ for (p = rb_first(&osdc->requests); p; p = rb_next(p)) {
+ struct ceph_osd_request *req;
+ struct ceph_osd_request_head *head;
+ struct ceph_osd_op *op;
+ int num_ops;
+ int opcode, olen;
+ int i;
+
+ req = rb_entry(p, struct ceph_osd_request, r_node);
+
+ seq_printf(s, "%lld\tosd%d\t%d.%x\t", req->r_tid,
+ req->r_osd ? req->r_osd->o_osd : -1,
+ le32_to_cpu(req->r_pgid.pool),
+ le16_to_cpu(req->r_pgid.ps));
+
+ head = req->r_request->front.iov_base;
+ op = (void *)(head + 1);
+
+ num_ops = le16_to_cpu(head->num_ops);
+ olen = le32_to_cpu(head->object_len);
+ seq_printf(s, "%.*s", olen,
+ (const char *)(head->ops + num_ops));
+
+ if (req->r_reassert_version.epoch)
+ seq_printf(s, "\t%u'%llu",
+ (unsigned)le32_to_cpu(req->r_reassert_version.epoch),
+ le64_to_cpu(req->r_reassert_version.version));
+ else
+ seq_printf(s, "\t");
+
+ for (i = 0; i < num_ops; i++) {
+ opcode = le16_to_cpu(op->op);
+ seq_printf(s, "\t%s", ceph_osd_op_name(opcode));
+ op++;
+ }
+
+ seq_printf(s, "\n");
+ }
+ mutex_unlock(&osdc->request_mutex);
+ return 0;
+}
+
+static int caps_show(struct seq_file *s, void *p)
+{
+ struct ceph_client *client = p;
+ int total, avail, used, reserved, min;
+
+ ceph_reservation_status(client, &total, &avail, &used, &reserved, &min);
+ seq_printf(s, "total\t\t%d\n"
+ "avail\t\t%d\n"
+ "used\t\t%d\n"
+ "reserved\t%d\n"
+ "min\t%d\n",
+ total, avail, used, reserved, min);
+ return 0;
+}
+
+static int dentry_lru_show(struct seq_file *s, void *ptr)
+{
+ struct ceph_client *client = s->private;
+ struct ceph_mds_client *mdsc = &client->mdsc;
+ struct ceph_dentry_info *di;
+
+ spin_lock(&mdsc->dentry_lru_lock);
+ list_for_each_entry(di, &mdsc->dentry_lru, lru) {
+ struct dentry *dentry = di->dentry;
+ seq_printf(s, "%p %p\t%.*s\n",
+ di, dentry, dentry->d_name.len, dentry->d_name.name);
+ }
+ spin_unlock(&mdsc->dentry_lru_lock);
+
+ return 0;
+}
+
+#define DEFINE_SHOW_FUNC(name) \
+static int name##_open(struct inode *inode, struct file *file) \
+{ \
+ struct seq_file *sf; \
+ int ret; \
+ \
+ ret = single_open(file, name, NULL); \
+ sf = file->private_data; \
+ sf->private = inode->i_private; \
+ return ret; \
+} \
+ \
+static const struct file_operations name##_fops = { \
+ .open = name##_open, \
+ .read = seq_read, \
+ .llseek = seq_lseek, \
+ .release = single_release, \
+};
+
+DEFINE_SHOW_FUNC(monmap_show)
+DEFINE_SHOW_FUNC(mdsmap_show)
+DEFINE_SHOW_FUNC(osdmap_show)
+DEFINE_SHOW_FUNC(monc_show)
+DEFINE_SHOW_FUNC(mdsc_show)
+DEFINE_SHOW_FUNC(osdc_show)
+DEFINE_SHOW_FUNC(dentry_lru_show)
+DEFINE_SHOW_FUNC(caps_show)
+
+static int congestion_kb_set(void *data, u64 val)
+{
+ struct ceph_client *client = (struct ceph_client *)data;
+
+ if (client)
+ client->mount_args->congestion_kb = (int)val;
+
+ return 0;
+}
+
+static int congestion_kb_get(void *data, u64 *val)
+{
+ struct ceph_client *client = (struct ceph_client *)data;
+
+ if (client)
+ *val = (u64)client->mount_args->congestion_kb;
+
+ return 0;
+}
+
+
+DEFINE_SIMPLE_ATTRIBUTE(congestion_kb_fops, congestion_kb_get,
+ congestion_kb_set, "%llu\n");
+
+int __init ceph_debugfs_init(void)
+{
+ ceph_debugfs_dir = debugfs_create_dir("ceph", NULL);
+ if (!ceph_debugfs_dir)
+ return -ENOMEM;
+ return 0;
+}
+
+void ceph_debugfs_cleanup(void)
+{
+ debugfs_remove(ceph_debugfs_dir);
+}
+
+int ceph_debugfs_client_init(struct ceph_client *client)
+{
+ int ret = 0;
+ char name[80];
+
+ snprintf(name, sizeof(name), FSID_FORMAT ".client%lld",
+ PR_FSID(&client->fsid), client->monc.auth->global_id);
+
+ client->debugfs_dir = debugfs_create_dir(name, ceph_debugfs_dir);
+ if (!client->debugfs_dir)
+ goto out;
+
+ client->monc.debugfs_file = debugfs_create_file("monc",
+ 0600,
+ client->debugfs_dir,
+ client,
+ &monc_show_fops);
+ if (!client->monc.debugfs_file)
+ goto out;
+
+ client->mdsc.debugfs_file = debugfs_create_file("mdsc",
+ 0600,
+ client->debugfs_dir,
+ client,
+ &mdsc_show_fops);
+ if (!client->mdsc.debugfs_file)
+ goto out;
+
+ client->osdc.debugfs_file = debugfs_create_file("osdc",
+ 0600,
+ client->debugfs_dir,
+ client,
+ &osdc_show_fops);
+ if (!client->osdc.debugfs_file)
+ goto out;
+
+ client->debugfs_monmap = debugfs_create_file("monmap",
+ 0600,
+ client->debugfs_dir,
+ client,
+ &monmap_show_fops);
+ if (!client->debugfs_monmap)
+ goto out;
+
+ client->debugfs_mdsmap = debugfs_create_file("mdsmap",
+ 0600,
+ client->debugfs_dir,
+ client,
+ &mdsmap_show_fops);
+ if (!client->debugfs_mdsmap)
+ goto out;
+
+ client->debugfs_osdmap = debugfs_create_file("osdmap",
+ 0600,
+ client->debugfs_dir,
+ client,
+ &osdmap_show_fops);
+ if (!client->debugfs_osdmap)
+ goto out;
+
+ client->debugfs_dentry_lru = debugfs_create_file("dentry_lru",
+ 0600,
+ client->debugfs_dir,
+ client,
+ &dentry_lru_show_fops);
+ if (!client->debugfs_dentry_lru)
+ goto out;
+
+ client->debugfs_caps = debugfs_create_file("caps",
+ 0400,
+ client->debugfs_dir,
+ client,
+ &caps_show_fops);
+ if (!client->debugfs_caps)
+ goto out;
+
+ client->debugfs_congestion_kb = debugfs_create_file("writeback_congestion_kb",
+ 0600,
+ client->debugfs_dir,
+ client,
+ &congestion_kb_fops);
+ if (!client->debugfs_congestion_kb)
+ goto out;
+
+ sprintf(name, "../../bdi/%s", dev_name(client->sb->s_bdi->dev));
+ client->debugfs_bdi = debugfs_create_symlink("bdi", client->debugfs_dir,
+ name);
+
+ return 0;
+
+out:
+ ceph_debugfs_client_cleanup(client);
+ return ret;
+}
+
+void ceph_debugfs_client_cleanup(struct ceph_client *client)
+{
+ debugfs_remove(client->debugfs_bdi);
+ debugfs_remove(client->debugfs_caps);
+ debugfs_remove(client->debugfs_dentry_lru);
+ debugfs_remove(client->debugfs_osdmap);
+ debugfs_remove(client->debugfs_mdsmap);
+ debugfs_remove(client->debugfs_monmap);
+ debugfs_remove(client->osdc.debugfs_file);
+ debugfs_remove(client->mdsc.debugfs_file);
+ debugfs_remove(client->monc.debugfs_file);
+ debugfs_remove(client->debugfs_congestion_kb);
+ debugfs_remove(client->debugfs_dir);
+}
+
+#else // CONFIG_DEBUG_FS
+
+int __init ceph_debugfs_init(void)
+{
+ return 0;
+}
+
+void ceph_debugfs_cleanup(void)
+{
+}
+
+int ceph_debugfs_client_init(struct ceph_client *client)
+{
+ return 0;
+}
+
+void ceph_debugfs_client_cleanup(struct ceph_client *client)
+{
+}
+
+#endif // CONFIG_DEBUG_FS
--- /dev/null
+#ifndef __CEPH_DECODE_H
+#define __CEPH_DECODE_H
+
+#include <asm/unaligned.h>
+#include <linux/time.h>
+
+#include "types.h"
+
+/*
+ * in all cases,
+ * void **p pointer to position pointer
+ * void *end pointer to end of buffer (last byte + 1)
+ */
+
+static inline u64 ceph_decode_64(void **p)
+{
+ u64 v = get_unaligned_le64(*p);
+ *p += sizeof(u64);
+ return v;
+}
+static inline u32 ceph_decode_32(void **p)
+{
+ u32 v = get_unaligned_le32(*p);
+ *p += sizeof(u32);
+ return v;
+}
+static inline u16 ceph_decode_16(void **p)
+{
+ u16 v = get_unaligned_le16(*p);
+ *p += sizeof(u16);
+ return v;
+}
+static inline u8 ceph_decode_8(void **p)
+{
+ u8 v = *(u8 *)*p;
+ (*p)++;
+ return v;
+}
+static inline void ceph_decode_copy(void **p, void *pv, size_t n)
+{
+ memcpy(pv, *p, n);
+ *p += n;
+}
+
+/*
+ * bounds check input.
+ */
+#define ceph_decode_need(p, end, n, bad) \
+ do { \
+ if (unlikely(*(p) + (n) > (end))) \
+ goto bad; \
+ } while (0)
+
+#define ceph_decode_64_safe(p, end, v, bad) \
+ do { \
+ ceph_decode_need(p, end, sizeof(u64), bad); \
+ v = ceph_decode_64(p); \
+ } while (0)
+#define ceph_decode_32_safe(p, end, v, bad) \
+ do { \
+ ceph_decode_need(p, end, sizeof(u32), bad); \
+ v = ceph_decode_32(p); \
+ } while (0)
+#define ceph_decode_16_safe(p, end, v, bad) \
+ do { \
+ ceph_decode_need(p, end, sizeof(u16), bad); \
+ v = ceph_decode_16(p); \
+ } while (0)
+#define ceph_decode_8_safe(p, end, v, bad) \
+ do { \
+ ceph_decode_need(p, end, sizeof(u8), bad); \
+ v = ceph_decode_8(p); \
+ } while (0)
+
+#define ceph_decode_copy_safe(p, end, pv, n, bad) \
+ do { \
+ ceph_decode_need(p, end, n, bad); \
+ ceph_decode_copy(p, pv, n); \
+ } while (0)
+
+/*
+ * struct ceph_timespec <-> struct timespec
+ */
+static inline void ceph_decode_timespec(struct timespec *ts,
+ const struct ceph_timespec *tv)
+{
+ ts->tv_sec = le32_to_cpu(tv->tv_sec);
+ ts->tv_nsec = le32_to_cpu(tv->tv_nsec);
+}
+static inline void ceph_encode_timespec(struct ceph_timespec *tv,
+ const struct timespec *ts)
+{
+ tv->tv_sec = cpu_to_le32(ts->tv_sec);
+ tv->tv_nsec = cpu_to_le32(ts->tv_nsec);
+}
+
+/*
+ * sockaddr_storage <-> ceph_sockaddr
+ */
+static inline void ceph_encode_addr(struct ceph_entity_addr *a)
+{
+ a->in_addr.ss_family = htons(a->in_addr.ss_family);
+}
+static inline void ceph_decode_addr(struct ceph_entity_addr *a)
+{
+ a->in_addr.ss_family = ntohs(a->in_addr.ss_family);
+ WARN_ON(a->in_addr.ss_family == 512);
+}
+
+/*
+ * encoders
+ */
+static inline void ceph_encode_64(void **p, u64 v)
+{
+ put_unaligned_le64(v, (__le64 *)*p);
+ *p += sizeof(u64);
+}
+static inline void ceph_encode_32(void **p, u32 v)
+{
+ put_unaligned_le32(v, (__le32 *)*p);
+ *p += sizeof(u32);
+}
+static inline void ceph_encode_16(void **p, u16 v)
+{
+ put_unaligned_le16(v, (__le16 *)*p);
+ *p += sizeof(u16);
+}
+static inline void ceph_encode_8(void **p, u8 v)
+{
+ *(u8 *)*p = v;
+ (*p)++;
+}
+static inline void ceph_encode_copy(void **p, const void *s, int len)
+{
+ memcpy(*p, s, len);
+ *p += len;
+}
+
+/*
+ * filepath, string encoders
+ */
+static inline void ceph_encode_filepath(void **p, void *end,
+ u64 ino, const char *path)
+{
+ u32 len = path ? strlen(path) : 0;
+ BUG_ON(*p + sizeof(ino) + sizeof(len) + len > end);
+ ceph_encode_8(p, 1);
+ ceph_encode_64(p, ino);
+ ceph_encode_32(p, len);
+ if (len)
+ memcpy(*p, path, len);
+ *p += len;
+}
+
+static inline void ceph_encode_string(void **p, void *end,
+ const char *s, u32 len)
+{
+ BUG_ON(*p + sizeof(len) + len > end);
+ ceph_encode_32(p, len);
+ if (len)
+ memcpy(*p, s, len);
+ *p += len;
+}
+
+#define ceph_encode_need(p, end, n, bad) \
+ do { \
+ if (unlikely(*(p) + (n) > (end))) \
+ goto bad; \
+ } while (0)
+
+#define ceph_encode_64_safe(p, end, v, bad) \
+ do { \
+ ceph_encode_need(p, end, sizeof(u64), bad); \
+ ceph_encode_64(p, v); \
+ } while (0)
+#define ceph_encode_32_safe(p, end, v, bad) \
+ do { \
+ ceph_encode_need(p, end, sizeof(u32), bad); \
+ ceph_encode_32(p, v); \
+ } while (0)
+#define ceph_encode_16_safe(p, end, v, bad) \
+ do { \
+ ceph_encode_need(p, end, sizeof(u16), bad); \
+ ceph_encode_16(p, v); \
+ } while (0)
+
+#define ceph_encode_copy_safe(p, end, pv, n, bad) \
+ do { \
+ ceph_encode_need(p, end, n, bad); \
+ ceph_encode_copy(p, pv, n); \
+ } while (0)
+
+
+#endif
--- /dev/null
+#include "ceph_debug.h"
+
+#include <linux/spinlock.h>
+#include <linux/fs_struct.h>
+#include <linux/namei.h>
+#include <linux/slab.h>
+#include <linux/sched.h>
+
+#include "super.h"
+
+/*
+ * Directory operations: readdir, lookup, create, link, unlink,
+ * rename, etc.
+ */
+
+/*
+ * Ceph MDS operations are specified in terms of a base ino and
+ * relative path. Thus, the client can specify an operation on a
+ * specific inode (e.g., a getattr due to fstat(2)), or as a path
+ * relative to, say, the root directory.
+ *
+ * Normally, we limit ourselves to strict inode ops (no path component)
+ * or dentry operations (a single path component relative to an ino). The
+ * exception to this is open_root_dentry(), which will open the mount
+ * point by name.
+ */
+
+const struct inode_operations ceph_dir_iops;
+const struct file_operations ceph_dir_fops;
+struct dentry_operations ceph_dentry_ops;
+
+/*
+ * Initialize ceph dentry state.
+ */
+int ceph_init_dentry(struct dentry *dentry)
+{
+ struct ceph_dentry_info *di;
+
+ if (dentry->d_fsdata)
+ return 0;
+
+ if (ceph_snap(dentry->d_parent->d_inode) == CEPH_NOSNAP)
+ dentry->d_op = &ceph_dentry_ops;
+ else if (ceph_snap(dentry->d_parent->d_inode) == CEPH_SNAPDIR)
+ dentry->d_op = &ceph_snapdir_dentry_ops;
+ else
+ dentry->d_op = &ceph_snap_dentry_ops;
+
+ di = kmem_cache_alloc(ceph_dentry_cachep, GFP_NOFS);
+ if (!di)
+ return -ENOMEM; /* oh well */
+
+ spin_lock(&dentry->d_lock);
+ if (dentry->d_fsdata) /* lost a race */
+ goto out_unlock;
+ di->dentry = dentry;
+ di->lease_session = NULL;
+ dentry->d_fsdata = di;
+ dentry->d_time = jiffies;
+ ceph_dentry_lru_add(dentry);
+out_unlock:
+ spin_unlock(&dentry->d_lock);
+ return 0;
+}
+
+
+
+/*
+ * for readdir, we encode the directory frag and offset within that
+ * frag into f_pos.
+ */
+static unsigned fpos_frag(loff_t p)
+{
+ return p >> 32;
+}
+static unsigned fpos_off(loff_t p)
+{
+ return p & 0xffffffff;
+}
+
+/*
+ * When possible, we try to satisfy a readdir by peeking at the
+ * dcache. We make this work by carefully ordering dentries on
+ * d_u.d_child when we initially get results back from the MDS, and
+ * falling back to a "normal" sync readdir if any dentries in the dir
+ * are dropped.
+ *
+ * I_COMPLETE tells indicates we have all dentries in the dir. It is
+ * defined IFF we hold CEPH_CAP_FILE_SHARED (which will be revoked by
+ * the MDS if/when the directory is modified).
+ */
+static int __dcache_readdir(struct file *filp,
+ void *dirent, filldir_t filldir)
+{
+ struct inode *inode = filp->f_dentry->d_inode;
+ struct ceph_file_info *fi = filp->private_data;
+ struct dentry *parent = filp->f_dentry;
+ struct inode *dir = parent->d_inode;
+ struct list_head *p;
+ struct dentry *dentry, *last;
+ struct ceph_dentry_info *di;
+ int err = 0;
+
+ /* claim ref on last dentry we returned */
+ last = fi->dentry;
+ fi->dentry = NULL;
+
+ dout("__dcache_readdir %p at %llu (last %p)\n", dir, filp->f_pos,
+ last);
+
+ spin_lock(&dcache_lock);
+
+ /* start at beginning? */
+ if (filp->f_pos == 2 || (last &&
+ filp->f_pos < ceph_dentry(last)->offset)) {
+ if (list_empty(&parent->d_subdirs))
+ goto out_unlock;
+ p = parent->d_subdirs.prev;
+ dout(" initial p %p/%p\n", p->prev, p->next);
+ } else {
+ p = last->d_u.d_child.prev;
+ }
+
+more:
+ dentry = list_entry(p, struct dentry, d_u.d_child);
+ di = ceph_dentry(dentry);
+ while (1) {
+ dout(" p %p/%p d_subdirs %p/%p\n", p->prev, p->next,
+ parent->d_subdirs.prev, parent->d_subdirs.next);
+ if (p == &parent->d_subdirs) {
+ fi->at_end = 1;
+ goto out_unlock;
+ }
+ if (!d_unhashed(dentry) && dentry->d_inode &&
+ ceph_snap(dentry->d_inode) != CEPH_SNAPDIR &&
+ ceph_ino(dentry->d_inode) != CEPH_INO_CEPH &&
+ filp->f_pos <= di->offset)
+ break;
+ dout(" skipping %p %.*s at %llu (%llu)%s%s\n", dentry,
+ dentry->d_name.len, dentry->d_name.name, di->offset,
+ filp->f_pos, d_unhashed(dentry) ? " unhashed" : "",
+ !dentry->d_inode ? " null" : "");
+ p = p->prev;
+ dentry = list_entry(p, struct dentry, d_u.d_child);
+ di = ceph_dentry(dentry);
+ }
+
+ atomic_inc(&dentry->d_count);
+ spin_unlock(&dcache_lock);
+ spin_unlock(&inode->i_lock);
+
+ dout(" %llu (%llu) dentry %p %.*s %p\n", di->offset, filp->f_pos,
+ dentry, dentry->d_name.len, dentry->d_name.name, dentry->d_inode);
+ filp->f_pos = di->offset;
+ err = filldir(dirent, dentry->d_name.name,
+ dentry->d_name.len, di->offset,
+ dentry->d_inode->i_ino,
+ dentry->d_inode->i_mode >> 12);
+
+ if (last) {
+ if (err < 0) {
+ /* remember our position */
+ fi->dentry = last;
+ fi->next_offset = di->offset;
+ } else {
+ dput(last);
+ }
+ last = NULL;
+ }
+
+ spin_lock(&inode->i_lock);
+ spin_lock(&dcache_lock);
+
+ if (err < 0)
+ goto out_unlock;
+
+ last = dentry;
+
+ p = p->prev;
+ filp->f_pos++;
+
+ /* make sure a dentry wasn't dropped while we didn't have dcache_lock */
+ if ((ceph_inode(dir)->i_ceph_flags & CEPH_I_COMPLETE))
+ goto more;
+ dout(" lost I_COMPLETE on %p; falling back to mds\n", dir);
+ err = -EAGAIN;
+
+out_unlock:
+ spin_unlock(&dcache_lock);
+
+ if (last) {
+ spin_unlock(&inode->i_lock);
+ dput(last);
+ spin_lock(&inode->i_lock);
+ }
+
+ return err;
+}
+
+/*
+ * make note of the last dentry we read, so we can
+ * continue at the same lexicographical point,
+ * regardless of what dir changes take place on the
+ * server.
+ */
+static int note_last_dentry(struct ceph_file_info *fi, const char *name,
+ int len)
+{
+ kfree(fi->last_name);
+ fi->last_name = kmalloc(len+1, GFP_NOFS);
+ if (!fi->last_name)
+ return -ENOMEM;
+ memcpy(fi->last_name, name, len);
+ fi->last_name[len] = 0;
+ dout("note_last_dentry '%s'\n", fi->last_name);
+ return 0;
+}
+
+static int ceph_readdir(struct file *filp, void *dirent, filldir_t filldir)
+{
+ struct ceph_file_info *fi = filp->private_data;
+ struct inode *inode = filp->f_dentry->d_inode;
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ struct ceph_client *client = ceph_inode_to_client(inode);
+ struct ceph_mds_client *mdsc = &client->mdsc;
+ unsigned frag = fpos_frag(filp->f_pos);
+ int off = fpos_off(filp->f_pos);
+ int err;
+ u32 ftype;
+ struct ceph_mds_reply_info_parsed *rinfo;
+ const int max_entries = client->mount_args->max_readdir;
+
+ dout("readdir %p filp %p frag %u off %u\n", inode, filp, frag, off);
+ if (fi->at_end)
+ return 0;
+
+ /* always start with . and .. */
+ if (filp->f_pos == 0) {
+ /* note dir version at start of readdir so we can tell
+ * if any dentries get dropped */
+ fi->dir_release_count = ci->i_release_count;
+
+ dout("readdir off 0 -> '.'\n");
+ if (filldir(dirent, ".", 1, ceph_make_fpos(0, 0),
+ inode->i_ino, inode->i_mode >> 12) < 0)
+ return 0;
+ filp->f_pos = 1;
+ off = 1;
+ }
+ if (filp->f_pos == 1) {
+ dout("readdir off 1 -> '..'\n");
+ if (filldir(dirent, "..", 2, ceph_make_fpos(0, 1),
+ filp->f_dentry->d_parent->d_inode->i_ino,
+ inode->i_mode >> 12) < 0)
+ return 0;
+ filp->f_pos = 2;
+ off = 2;
+ }
+
+ /* can we use the dcache? */
+ spin_lock(&inode->i_lock);
+ if ((filp->f_pos == 2 || fi->dentry) &&
+ !ceph_test_opt(client, NOASYNCREADDIR) &&
+ (ci->i_ceph_flags & CEPH_I_COMPLETE) &&
+ __ceph_caps_issued_mask(ci, CEPH_CAP_FILE_SHARED, 1)) {
+ err = __dcache_readdir(filp, dirent, filldir);
+ if (err != -EAGAIN) {
+ spin_unlock(&inode->i_lock);
+ return err;
+ }
+ }
+ spin_unlock(&inode->i_lock);
+ if (fi->dentry) {
+ err = note_last_dentry(fi, fi->dentry->d_name.name,
+ fi->dentry->d_name.len);
+ if (err)
+ return err;
+ dput(fi->dentry);
+ fi->dentry = NULL;
+ }
+
+ /* proceed with a normal readdir */
+
+more:
+ /* do we have the correct frag content buffered? */
+ if (fi->frag != frag || fi->last_readdir == NULL) {
+ struct ceph_mds_request *req;
+ int op = ceph_snap(inode) == CEPH_SNAPDIR ?
+ CEPH_MDS_OP_LSSNAP : CEPH_MDS_OP_READDIR;
+
+ /* discard old result, if any */
+ if (fi->last_readdir) {
+ ceph_mdsc_put_request(fi->last_readdir);
+ fi->last_readdir = NULL;
+ }
+
+ /* requery frag tree, as the frag topology may have changed */
+ frag = ceph_choose_frag(ceph_inode(inode), frag, NULL, NULL);
+
+ dout("readdir fetching %llx.%llx frag %x offset '%s'\n",
+ ceph_vinop(inode), frag, fi->last_name);
+ req = ceph_mdsc_create_request(mdsc, op, USE_AUTH_MDS);
+ if (IS_ERR(req))
+ return PTR_ERR(req);
+ req->r_inode = igrab(inode);
+ req->r_dentry = dget(filp->f_dentry);
+ /* hints to request -> mds selection code */
+ req->r_direct_mode = USE_AUTH_MDS;
+ req->r_direct_hash = ceph_frag_value(frag);
+ req->r_direct_is_hash = true;
+ req->r_path2 = kstrdup(fi->last_name, GFP_NOFS);
+ req->r_readdir_offset = fi->next_offset;
+ req->r_args.readdir.frag = cpu_to_le32(frag);
+ req->r_args.readdir.max_entries = cpu_to_le32(max_entries);
+ req->r_num_caps = max_entries;
+ err = ceph_mdsc_do_request(mdsc, NULL, req);
+ if (err < 0) {
+ ceph_mdsc_put_request(req);
+ return err;
+ }
+ dout("readdir got and parsed readdir result=%d"
+ " on frag %x, end=%d, complete=%d\n", err, frag,
+ (int)req->r_reply_info.dir_end,
+ (int)req->r_reply_info.dir_complete);
+
+ if (!req->r_did_prepopulate) {
+ dout("readdir !did_prepopulate");
+ fi->dir_release_count--; /* preclude I_COMPLETE */
+ }
+
+ /* note next offset and last dentry name */
+ fi->offset = fi->next_offset;
+ fi->last_readdir = req;
+
+ if (req->r_reply_info.dir_end) {
+ kfree(fi->last_name);
+ fi->last_name = NULL;
+ fi->next_offset = 0;
+ } else {
+ rinfo = &req->r_reply_info;
+ err = note_last_dentry(fi,
+ rinfo->dir_dname[rinfo->dir_nr-1],
+ rinfo->dir_dname_len[rinfo->dir_nr-1]);
+ if (err)
+ return err;
+ fi->next_offset += rinfo->dir_nr;
+ }
+ }
+
+ rinfo = &fi->last_readdir->r_reply_info;
+ dout("readdir frag %x num %d off %d chunkoff %d\n", frag,
+ rinfo->dir_nr, off, fi->offset);
+ while (off - fi->offset >= 0 && off - fi->offset < rinfo->dir_nr) {
+ u64 pos = ceph_make_fpos(frag, off);
+ struct ceph_mds_reply_inode *in =
+ rinfo->dir_in[off - fi->offset].in;
+ dout("readdir off %d (%d/%d) -> %lld '%.*s' %p\n",
+ off, off - fi->offset, rinfo->dir_nr, pos,
+ rinfo->dir_dname_len[off - fi->offset],
+ rinfo->dir_dname[off - fi->offset], in);
+ BUG_ON(!in);
+ ftype = le32_to_cpu(in->mode) >> 12;
+ if (filldir(dirent,
+ rinfo->dir_dname[off - fi->offset],
+ rinfo->dir_dname_len[off - fi->offset],
+ pos,
+ le64_to_cpu(in->ino),
+ ftype) < 0) {
+ dout("filldir stopping us...\n");
+ return 0;
+ }
+ off++;
+ filp->f_pos = pos + 1;
+ }
+
+ if (fi->last_name) {
+ ceph_mdsc_put_request(fi->last_readdir);
+ fi->last_readdir = NULL;
+ goto more;
+ }
+
+ /* more frags? */
+ if (!ceph_frag_is_rightmost(frag)) {
+ frag = ceph_frag_next(frag);
+ off = 0;
+ filp->f_pos = ceph_make_fpos(frag, off);
+ dout("readdir next frag is %x\n", frag);
+ goto more;
+ }
+ fi->at_end = 1;
+
+ /*
+ * if dir_release_count still matches the dir, no dentries
+ * were released during the whole readdir, and we should have
+ * the complete dir contents in our cache.
+ */
+ spin_lock(&inode->i_lock);
+ if (ci->i_release_count == fi->dir_release_count) {
+ dout(" marking %p complete\n", inode);
+ ci->i_ceph_flags |= CEPH_I_COMPLETE;
+ ci->i_max_offset = filp->f_pos;
+ }
+ spin_unlock(&inode->i_lock);
+
+ dout("readdir %p filp %p done.\n", inode, filp);
+ return 0;
+}
+
+static void reset_readdir(struct ceph_file_info *fi)
+{
+ if (fi->last_readdir) {
+ ceph_mdsc_put_request(fi->last_readdir);
+ fi->last_readdir = NULL;
+ }
+ kfree(fi->last_name);
+ fi->next_offset = 2; /* compensate for . and .. */
+ if (fi->dentry) {
+ dput(fi->dentry);
+ fi->dentry = NULL;
+ }
+ fi->at_end = 0;
+}
+
+static loff_t ceph_dir_llseek(struct file *file, loff_t offset, int origin)
+{
+ struct ceph_file_info *fi = file->private_data;
+ struct inode *inode = file->f_mapping->host;
+ loff_t old_offset = offset;
+ loff_t retval;
+
+ mutex_lock(&inode->i_mutex);
+ switch (origin) {
+ case SEEK_END:
+ offset += inode->i_size + 2; /* FIXME */
+ break;
+ case SEEK_CUR:
+ offset += file->f_pos;
+ }
+ retval = -EINVAL;
+ if (offset >= 0 && offset <= inode->i_sb->s_maxbytes) {
+ if (offset != file->f_pos) {
+ file->f_pos = offset;
+ file->f_version = 0;
+ fi->at_end = 0;
+ }
+ retval = offset;
+
+ /*
+ * discard buffered readdir content on seekdir(0), or
+ * seek to new frag, or seek prior to current chunk.
+ */
+ if (offset == 0 ||
+ fpos_frag(offset) != fpos_frag(old_offset) ||
+ fpos_off(offset) < fi->offset) {
+ dout("dir_llseek dropping %p content\n", file);
+ reset_readdir(fi);
+ }
+
+ /* bump dir_release_count if we did a forward seek */
+ if (offset > old_offset)
+ fi->dir_release_count--;
+ }
+ mutex_unlock(&inode->i_mutex);
+ return retval;
+}
+
+/*
+ * Process result of a lookup/open request.
+ *
+ * Mainly, make sure we return the final req->r_dentry (if it already
+ * existed) in place of the original VFS-provided dentry when they
+ * differ.
+ *
+ * Gracefully handle the case where the MDS replies with -ENOENT and
+ * no trace (which it may do, at its discretion, e.g., if it doesn't
+ * care to issue a lease on the negative dentry).
+ */
+struct dentry *ceph_finish_lookup(struct ceph_mds_request *req,
+ struct dentry *dentry, int err)
+{
+ struct ceph_client *client = ceph_client(dentry->d_sb);
+ struct inode *parent = dentry->d_parent->d_inode;
+
+ /* .snap dir? */
+ if (err == -ENOENT &&
+ ceph_vino(parent).ino != CEPH_INO_ROOT && /* no .snap in root dir */
+ strcmp(dentry->d_name.name,
+ client->mount_args->snapdir_name) == 0) {
+ struct inode *inode = ceph_get_snapdir(parent);
+ dout("ENOENT on snapdir %p '%.*s', linking to snapdir %p\n",
+ dentry, dentry->d_name.len, dentry->d_name.name, inode);
+ d_add(dentry, inode);
+ err = 0;
+ }
+
+ if (err == -ENOENT) {
+ /* no trace? */
+ err = 0;
+ if (!req->r_reply_info.head->is_dentry) {
+ dout("ENOENT and no trace, dentry %p inode %p\n",
+ dentry, dentry->d_inode);
+ if (dentry->d_inode) {
+ d_drop(dentry);
+ err = -ENOENT;
+ } else {
+ d_add(dentry, NULL);
+ }
+ }
+ }
+ if (err)
+ dentry = ERR_PTR(err);
+ else if (dentry != req->r_dentry)
+ dentry = dget(req->r_dentry); /* we got spliced */
+ else
+ dentry = NULL;
+ return dentry;
+}
+
+static int is_root_ceph_dentry(struct inode *inode, struct dentry *dentry)
+{
+ return ceph_ino(inode) == CEPH_INO_ROOT &&
+ strncmp(dentry->d_name.name, ".ceph", 5) == 0;
+}
+
+/*
+ * Look up a single dir entry. If there is a lookup intent, inform
+ * the MDS so that it gets our 'caps wanted' value in a single op.
+ */
+static struct dentry *ceph_lookup(struct inode *dir, struct dentry *dentry,
+ struct nameidata *nd)
+{
+ struct ceph_client *client = ceph_sb_to_client(dir->i_sb);
+ struct ceph_mds_client *mdsc = &client->mdsc;
+ struct ceph_mds_request *req;
+ int op;
+ int err;
+
+ dout("lookup %p dentry %p '%.*s'\n",
+ dir, dentry, dentry->d_name.len, dentry->d_name.name);
+
+ if (dentry->d_name.len > NAME_MAX)
+ return ERR_PTR(-ENAMETOOLONG);
+
+ err = ceph_init_dentry(dentry);
+ if (err < 0)
+ return ERR_PTR(err);
+
+ /* open (but not create!) intent? */
+ if (nd &&
+ (nd->flags & LOOKUP_OPEN) &&
+ (nd->flags & LOOKUP_CONTINUE) == 0 && /* only open last component */
+ !(nd->intent.open.flags & O_CREAT)) {
+ int mode = nd->intent.open.create_mode & ~current->fs->umask;
+ return ceph_lookup_open(dir, dentry, nd, mode, 1);
+ }
+
+ /* can we conclude ENOENT locally? */
+ if (dentry->d_inode == NULL) {
+ struct ceph_inode_info *ci = ceph_inode(dir);
+ struct ceph_dentry_info *di = ceph_dentry(dentry);
+
+ spin_lock(&dir->i_lock);
+ dout(" dir %p flags are %d\n", dir, ci->i_ceph_flags);
+ if (strncmp(dentry->d_name.name,
+ client->mount_args->snapdir_name,
+ dentry->d_name.len) &&
+ !is_root_ceph_dentry(dir, dentry) &&
+ (ci->i_ceph_flags & CEPH_I_COMPLETE) &&
+ (__ceph_caps_issued_mask(ci, CEPH_CAP_FILE_SHARED, 1))) {
+ di->offset = ci->i_max_offset++;
+ spin_unlock(&dir->i_lock);
+ dout(" dir %p complete, -ENOENT\n", dir);
+ d_add(dentry, NULL);
+ di->lease_shared_gen = ci->i_shared_gen;
+ return NULL;
+ }
+ spin_unlock(&dir->i_lock);
+ }
+
+ op = ceph_snap(dir) == CEPH_SNAPDIR ?
+ CEPH_MDS_OP_LOOKUPSNAP : CEPH_MDS_OP_LOOKUP;
+ req = ceph_mdsc_create_request(mdsc, op, USE_ANY_MDS);
+ if (IS_ERR(req))
+ return ERR_PTR(PTR_ERR(req));
+ req->r_dentry = dget(dentry);
+ req->r_num_caps = 2;
+ /* we only need inode linkage */
+ req->r_args.getattr.mask = cpu_to_le32(CEPH_STAT_CAP_INODE);
+ req->r_locked_dir = dir;
+ err = ceph_mdsc_do_request(mdsc, NULL, req);
+ dentry = ceph_finish_lookup(req, dentry, err);
+ ceph_mdsc_put_request(req); /* will dput(dentry) */
+ dout("lookup result=%p\n", dentry);
+ return dentry;
+}
+
+/*
+ * If we do a create but get no trace back from the MDS, follow up with
+ * a lookup (the VFS expects us to link up the provided dentry).
+ */
+int ceph_handle_notrace_create(struct inode *dir, struct dentry *dentry)
+{
+ struct dentry *result = ceph_lookup(dir, dentry, NULL);
+
+ if (result && !IS_ERR(result)) {
+ /*
+ * We created the item, then did a lookup, and found
+ * it was already linked to another inode we already
+ * had in our cache (and thus got spliced). Link our
+ * dentry to that inode, but don't hash it, just in
+ * case the VFS wants to dereference it.
+ */
+ BUG_ON(!result->d_inode);
+ d_instantiate(dentry, result->d_inode);
+ return 0;
+ }
+ return PTR_ERR(result);
+}
+
+static int ceph_mknod(struct inode *dir, struct dentry *dentry,
+ int mode, dev_t rdev)
+{
+ struct ceph_client *client = ceph_sb_to_client(dir->i_sb);
+ struct ceph_mds_client *mdsc = &client->mdsc;
+ struct ceph_mds_request *req;
+ int err;
+
+ if (ceph_snap(dir) != CEPH_NOSNAP)
+ return -EROFS;
+
+ dout("mknod in dir %p dentry %p mode 0%o rdev %d\n",
+ dir, dentry, mode, rdev);
+ req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_MKNOD, USE_AUTH_MDS);
+ if (IS_ERR(req)) {
+ d_drop(dentry);
+ return PTR_ERR(req);
+ }
+ req->r_dentry = dget(dentry);
+ req->r_num_caps = 2;
+ req->r_locked_dir = dir;
+ req->r_args.mknod.mode = cpu_to_le32(mode);
+ req->r_args.mknod.rdev = cpu_to_le32(rdev);
+ req->r_dentry_drop = CEPH_CAP_FILE_SHARED;
+ req->r_dentry_unless = CEPH_CAP_FILE_EXCL;
+ err = ceph_mdsc_do_request(mdsc, dir, req);
+ if (!err && !req->r_reply_info.head->is_dentry)
+ err = ceph_handle_notrace_create(dir, dentry);
+ ceph_mdsc_put_request(req);
+ if (err)
+ d_drop(dentry);
+ return err;
+}
+
+static int ceph_create(struct inode *dir, struct dentry *dentry, int mode,
+ struct nameidata *nd)
+{
+ dout("create in dir %p dentry %p name '%.*s'\n",
+ dir, dentry, dentry->d_name.len, dentry->d_name.name);
+
+ if (ceph_snap(dir) != CEPH_NOSNAP)
+ return -EROFS;
+
+ if (nd) {
+ BUG_ON((nd->flags & LOOKUP_OPEN) == 0);
+ dentry = ceph_lookup_open(dir, dentry, nd, mode, 0);
+ /* hrm, what should i do here if we get aliased? */
+ if (IS_ERR(dentry))
+ return PTR_ERR(dentry);
+ return 0;
+ }
+
+ /* fall back to mknod */
+ return ceph_mknod(dir, dentry, (mode & ~S_IFMT) | S_IFREG, 0);
+}
+
+static int ceph_symlink(struct inode *dir, struct dentry *dentry,
+ const char *dest)
+{
+ struct ceph_client *client = ceph_sb_to_client(dir->i_sb);
+ struct ceph_mds_client *mdsc = &client->mdsc;
+ struct ceph_mds_request *req;
+ int err;
+
+ if (ceph_snap(dir) != CEPH_NOSNAP)
+ return -EROFS;
+
+ dout("symlink in dir %p dentry %p to '%s'\n", dir, dentry, dest);
+ req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_SYMLINK, USE_AUTH_MDS);
+ if (IS_ERR(req)) {
+ d_drop(dentry);
+ return PTR_ERR(req);
+ }
+ req->r_dentry = dget(dentry);
+ req->r_num_caps = 2;
+ req->r_path2 = kstrdup(dest, GFP_NOFS);
+ req->r_locked_dir = dir;
+ req->r_dentry_drop = CEPH_CAP_FILE_SHARED;
+ req->r_dentry_unless = CEPH_CAP_FILE_EXCL;
+ err = ceph_mdsc_do_request(mdsc, dir, req);
+ if (!err && !req->r_reply_info.head->is_dentry)
+ err = ceph_handle_notrace_create(dir, dentry);
+ ceph_mdsc_put_request(req);
+ if (err)
+ d_drop(dentry);
+ return err;
+}
+
+static int ceph_mkdir(struct inode *dir, struct dentry *dentry, int mode)
+{
+ struct ceph_client *client = ceph_sb_to_client(dir->i_sb);
+ struct ceph_mds_client *mdsc = &client->mdsc;
+ struct ceph_mds_request *req;
+ int err = -EROFS;
+ int op;
+
+ if (ceph_snap(dir) == CEPH_SNAPDIR) {
+ /* mkdir .snap/foo is a MKSNAP */
+ op = CEPH_MDS_OP_MKSNAP;
+ dout("mksnap dir %p snap '%.*s' dn %p\n", dir,
+ dentry->d_name.len, dentry->d_name.name, dentry);
+ } else if (ceph_snap(dir) == CEPH_NOSNAP) {
+ dout("mkdir dir %p dn %p mode 0%o\n", dir, dentry, mode);
+ op = CEPH_MDS_OP_MKDIR;
+ } else {
+ goto out;
+ }
+ req = ceph_mdsc_create_request(mdsc, op, USE_AUTH_MDS);
+ if (IS_ERR(req)) {
+ err = PTR_ERR(req);
+ goto out;
+ }
+
+ req->r_dentry = dget(dentry);
+ req->r_num_caps = 2;
+ req->r_locked_dir = dir;
+ req->r_args.mkdir.mode = cpu_to_le32(mode);
+ req->r_dentry_drop = CEPH_CAP_FILE_SHARED;
+ req->r_dentry_unless = CEPH_CAP_FILE_EXCL;
+ err = ceph_mdsc_do_request(mdsc, dir, req);
+ if (!err && !req->r_reply_info.head->is_dentry)
+ err = ceph_handle_notrace_create(dir, dentry);
+ ceph_mdsc_put_request(req);
+out:
+ if (err < 0)
+ d_drop(dentry);
+ return err;
+}
+
+static int ceph_link(struct dentry *old_dentry, struct inode *dir,
+ struct dentry *dentry)
+{
+ struct ceph_client *client = ceph_sb_to_client(dir->i_sb);
+ struct ceph_mds_client *mdsc = &client->mdsc;
+ struct ceph_mds_request *req;
+ int err;
+
+ if (ceph_snap(dir) != CEPH_NOSNAP)
+ return -EROFS;
+
+ dout("link in dir %p old_dentry %p dentry %p\n", dir,
+ old_dentry, dentry);
+ req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_LINK, USE_AUTH_MDS);
+ if (IS_ERR(req)) {
+ d_drop(dentry);
+ return PTR_ERR(req);
+ }
+ req->r_dentry = dget(dentry);
+ req->r_num_caps = 2;
+ req->r_old_dentry = dget(old_dentry); /* or inode? hrm. */
+ req->r_locked_dir = dir;
+ req->r_dentry_drop = CEPH_CAP_FILE_SHARED;
+ req->r_dentry_unless = CEPH_CAP_FILE_EXCL;
+ err = ceph_mdsc_do_request(mdsc, dir, req);
+ if (err)
+ d_drop(dentry);
+ else if (!req->r_reply_info.head->is_dentry)
+ d_instantiate(dentry, igrab(old_dentry->d_inode));
+ ceph_mdsc_put_request(req);
+ return err;
+}
+
+/*
+ * For a soon-to-be unlinked file, drop the AUTH_RDCACHE caps. If it
+ * looks like the link count will hit 0, drop any other caps (other
+ * than PIN) we don't specifically want (due to the file still being
+ * open).
+ */
+static int drop_caps_for_unlink(struct inode *inode)
+{
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ int drop = CEPH_CAP_LINK_SHARED | CEPH_CAP_LINK_EXCL;
+
+ spin_lock(&inode->i_lock);
+ if (inode->i_nlink == 1) {
+ drop |= ~(__ceph_caps_wanted(ci) | CEPH_CAP_PIN);
+ ci->i_ceph_flags |= CEPH_I_NODELAY;
+ }
+ spin_unlock(&inode->i_lock);
+ return drop;
+}
+
+/*
+ * rmdir and unlink are differ only by the metadata op code
+ */
+static int ceph_unlink(struct inode *dir, struct dentry *dentry)
+{
+ struct ceph_client *client = ceph_sb_to_client(dir->i_sb);
+ struct ceph_mds_client *mdsc = &client->mdsc;
+ struct inode *inode = dentry->d_inode;
+ struct ceph_mds_request *req;
+ int err = -EROFS;
+ int op;
+
+ if (ceph_snap(dir) == CEPH_SNAPDIR) {
+ /* rmdir .snap/foo is RMSNAP */
+ dout("rmsnap dir %p '%.*s' dn %p\n", dir, dentry->d_name.len,
+ dentry->d_name.name, dentry);
+ op = CEPH_MDS_OP_RMSNAP;
+ } else if (ceph_snap(dir) == CEPH_NOSNAP) {
+ dout("unlink/rmdir dir %p dn %p inode %p\n",
+ dir, dentry, inode);
+ op = ((dentry->d_inode->i_mode & S_IFMT) == S_IFDIR) ?
+ CEPH_MDS_OP_RMDIR : CEPH_MDS_OP_UNLINK;
+ } else
+ goto out;
+ req = ceph_mdsc_create_request(mdsc, op, USE_AUTH_MDS);
+ if (IS_ERR(req)) {
+ err = PTR_ERR(req);
+ goto out;
+ }
+ req->r_dentry = dget(dentry);
+ req->r_num_caps = 2;
+ req->r_locked_dir = dir;
+ req->r_dentry_drop = CEPH_CAP_FILE_SHARED;
+ req->r_dentry_unless = CEPH_CAP_FILE_EXCL;
+ req->r_inode_drop = drop_caps_for_unlink(inode);
+ err = ceph_mdsc_do_request(mdsc, dir, req);
+ if (!err && !req->r_reply_info.head->is_dentry)
+ d_delete(dentry);
+ ceph_mdsc_put_request(req);
+out:
+ return err;
+}
+
+static int ceph_rename(struct inode *old_dir, struct dentry *old_dentry,
+ struct inode *new_dir, struct dentry *new_dentry)
+{
+ struct ceph_client *client = ceph_sb_to_client(old_dir->i_sb);
+ struct ceph_mds_client *mdsc = &client->mdsc;
+ struct ceph_mds_request *req;
+ int err;
+
+ if (ceph_snap(old_dir) != ceph_snap(new_dir))
+ return -EXDEV;
+ if (ceph_snap(old_dir) != CEPH_NOSNAP ||
+ ceph_snap(new_dir) != CEPH_NOSNAP)
+ return -EROFS;
+ dout("rename dir %p dentry %p to dir %p dentry %p\n",
+ old_dir, old_dentry, new_dir, new_dentry);
+ req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_RENAME, USE_AUTH_MDS);
+ if (IS_ERR(req))
+ return PTR_ERR(req);
+ req->r_dentry = dget(new_dentry);
+ req->r_num_caps = 2;
+ req->r_old_dentry = dget(old_dentry);
+ req->r_locked_dir = new_dir;
+ req->r_old_dentry_drop = CEPH_CAP_FILE_SHARED;
+ req->r_old_dentry_unless = CEPH_CAP_FILE_EXCL;
+ req->r_dentry_drop = CEPH_CAP_FILE_SHARED;
+ req->r_dentry_unless = CEPH_CAP_FILE_EXCL;
+ /* release LINK_RDCACHE on source inode (mds will lock it) */
+ req->r_old_inode_drop = CEPH_CAP_LINK_SHARED;
+ if (new_dentry->d_inode)
+ req->r_inode_drop = drop_caps_for_unlink(new_dentry->d_inode);
+ err = ceph_mdsc_do_request(mdsc, old_dir, req);
+ if (!err && !req->r_reply_info.head->is_dentry) {
+ /*
+ * Normally d_move() is done by fill_trace (called by
+ * do_request, above). If there is no trace, we need
+ * to do it here.
+ */
+ d_move(old_dentry, new_dentry);
+ }
+ ceph_mdsc_put_request(req);
+ return err;
+}
+
+
+/*
+ * Check if dentry lease is valid. If not, delete the lease. Try to
+ * renew if the least is more than half up.
+ */
+static int dentry_lease_is_valid(struct dentry *dentry)
+{
+ struct ceph_dentry_info *di;
+ struct ceph_mds_session *s;
+ int valid = 0;
+ u32 gen;
+ unsigned long ttl;
+ struct ceph_mds_session *session = NULL;
+ struct inode *dir = NULL;
+ u32 seq = 0;
+
+ spin_lock(&dentry->d_lock);
+ di = ceph_dentry(dentry);
+ if (di && di->lease_session) {
+ s = di->lease_session;
+ spin_lock(&s->s_cap_lock);
+ gen = s->s_cap_gen;
+ ttl = s->s_cap_ttl;
+ spin_unlock(&s->s_cap_lock);
+
+ if (di->lease_gen == gen &&
+ time_before(jiffies, dentry->d_time) &&
+ time_before(jiffies, ttl)) {
+ valid = 1;
+ if (di->lease_renew_after &&
+ time_after(jiffies, di->lease_renew_after)) {
+ /* we should renew */
+ dir = dentry->d_parent->d_inode;
+ session = ceph_get_mds_session(s);
+ seq = di->lease_seq;
+ di->lease_renew_after = 0;
+ di->lease_renew_from = jiffies;
+ }
+ }
+ }
+ spin_unlock(&dentry->d_lock);
+
+ if (session) {
+ ceph_mdsc_lease_send_msg(session, dir, dentry,
+ CEPH_MDS_LEASE_RENEW, seq);
+ ceph_put_mds_session(session);
+ }
+ dout("dentry_lease_is_valid - dentry %p = %d\n", dentry, valid);
+ return valid;
+}
+
+/*
+ * Check if directory-wide content lease/cap is valid.
+ */
+static int dir_lease_is_valid(struct inode *dir, struct dentry *dentry)
+{
+ struct ceph_inode_info *ci = ceph_inode(dir);
+ struct ceph_dentry_info *di = ceph_dentry(dentry);
+ int valid = 0;
+
+ spin_lock(&dir->i_lock);
+ if (ci->i_shared_gen == di->lease_shared_gen)
+ valid = __ceph_caps_issued_mask(ci, CEPH_CAP_FILE_SHARED, 1);
+ spin_unlock(&dir->i_lock);
+ dout("dir_lease_is_valid dir %p v%u dentry %p v%u = %d\n",
+ dir, (unsigned)ci->i_shared_gen, dentry,
+ (unsigned)di->lease_shared_gen, valid);
+ return valid;
+}
+
+/*
+ * Check if cached dentry can be trusted.
+ */
+static int ceph_d_revalidate(struct dentry *dentry, struct nameidata *nd)
+{
+ struct inode *dir = dentry->d_parent->d_inode;
+
+ dout("d_revalidate %p '%.*s' inode %p\n", dentry,
+ dentry->d_name.len, dentry->d_name.name, dentry->d_inode);
+
+ /* always trust cached snapped dentries, snapdir dentry */
+ if (ceph_snap(dir) != CEPH_NOSNAP) {
+ dout("d_revalidate %p '%.*s' inode %p is SNAPPED\n", dentry,
+ dentry->d_name.len, dentry->d_name.name, dentry->d_inode);
+ goto out_touch;
+ }
+ if (dentry->d_inode && ceph_snap(dentry->d_inode) == CEPH_SNAPDIR)
+ goto out_touch;
+
+ if (dentry_lease_is_valid(dentry) ||
+ dir_lease_is_valid(dir, dentry))
+ goto out_touch;
+
+ dout("d_revalidate %p invalid\n", dentry);
+ d_drop(dentry);
+ return 0;
+out_touch:
+ ceph_dentry_lru_touch(dentry);
+ return 1;
+}
+
+/*
+ * When a dentry is released, clear the dir I_COMPLETE if it was part
+ * of the current dir gen.
+ */
+static void ceph_dentry_release(struct dentry *dentry)
+{
+ struct ceph_dentry_info *di = ceph_dentry(dentry);
+ struct inode *parent_inode = dentry->d_parent->d_inode;
+
+ if (parent_inode) {
+ struct ceph_inode_info *ci = ceph_inode(parent_inode);
+
+ spin_lock(&parent_inode->i_lock);
+ if (ci->i_shared_gen == di->lease_shared_gen) {
+ dout(" clearing %p complete (d_release)\n",
+ parent_inode);
+ ci->i_ceph_flags &= ~CEPH_I_COMPLETE;
+ ci->i_release_count++;
+ }
+ spin_unlock(&parent_inode->i_lock);
+ }
+ if (di) {
+ ceph_dentry_lru_del(dentry);
+ if (di->lease_session)
+ ceph_put_mds_session(di->lease_session);
+ kmem_cache_free(ceph_dentry_cachep, di);
+ dentry->d_fsdata = NULL;
+ }
+}
+
+static int ceph_snapdir_d_revalidate(struct dentry *dentry,
+ struct nameidata *nd)
+{
+ /*
+ * Eventually, we'll want to revalidate snapped metadata
+ * too... probably...
+ */
+ return 1;
+}
+
+
+
+/*
+ * read() on a dir. This weird interface hack only works if mounted
+ * with '-o dirstat'.
+ */
+static ssize_t ceph_read_dir(struct file *file, char __user *buf, size_t size,
+ loff_t *ppos)
+{
+ struct ceph_file_info *cf = file->private_data;
+ struct inode *inode = file->f_dentry->d_inode;
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ int left;
+
+ if (!ceph_test_opt(ceph_client(inode->i_sb), DIRSTAT))
+ return -EISDIR;
+
+ if (!cf->dir_info) {
+ cf->dir_info = kmalloc(1024, GFP_NOFS);
+ if (!cf->dir_info)
+ return -ENOMEM;
+ cf->dir_info_len =
+ sprintf(cf->dir_info,
+ "entries: %20lld\n"
+ " files: %20lld\n"
+ " subdirs: %20lld\n"
+ "rentries: %20lld\n"
+ " rfiles: %20lld\n"
+ " rsubdirs: %20lld\n"
+ "rbytes: %20lld\n"
+ "rctime: %10ld.%09ld\n",
+ ci->i_files + ci->i_subdirs,
+ ci->i_files,
+ ci->i_subdirs,
+ ci->i_rfiles + ci->i_rsubdirs,
+ ci->i_rfiles,
+ ci->i_rsubdirs,
+ ci->i_rbytes,
+ (long)ci->i_rctime.tv_sec,
+ (long)ci->i_rctime.tv_nsec);
+ }
+
+ if (*ppos >= cf->dir_info_len)
+ return 0;
+ size = min_t(unsigned, size, cf->dir_info_len-*ppos);
+ left = copy_to_user(buf, cf->dir_info + *ppos, size);
+ if (left == size)
+ return -EFAULT;
+ *ppos += (size - left);
+ return size - left;
+}
+
+/*
+ * an fsync() on a dir will wait for any uncommitted directory
+ * operations to commit.
+ */
+static int ceph_dir_fsync(struct file *file, struct dentry *dentry,
+ int datasync)
+{
+ struct inode *inode = dentry->d_inode;
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ struct list_head *head = &ci->i_unsafe_dirops;
+ struct ceph_mds_request *req;
+ u64 last_tid;
+ int ret = 0;
+
+ dout("dir_fsync %p\n", inode);
+ spin_lock(&ci->i_unsafe_lock);
+ if (list_empty(head))
+ goto out;
+
+ req = list_entry(head->prev,
+ struct ceph_mds_request, r_unsafe_dir_item);
+ last_tid = req->r_tid;
+
+ do {
+ ceph_mdsc_get_request(req);
+ spin_unlock(&ci->i_unsafe_lock);
+ dout("dir_fsync %p wait on tid %llu (until %llu)\n",
+ inode, req->r_tid, last_tid);
+ if (req->r_timeout) {
+ ret = wait_for_completion_timeout(
+ &req->r_safe_completion, req->r_timeout);
+ if (ret > 0)
+ ret = 0;
+ else if (ret == 0)
+ ret = -EIO; /* timed out */
+ } else {
+ wait_for_completion(&req->r_safe_completion);
+ }
+ spin_lock(&ci->i_unsafe_lock);
+ ceph_mdsc_put_request(req);
+
+ if (ret || list_empty(head))
+ break;
+ req = list_entry(head->next,
+ struct ceph_mds_request, r_unsafe_dir_item);
+ } while (req->r_tid < last_tid);
+out:
+ spin_unlock(&ci->i_unsafe_lock);
+ return ret;
+}
+
+/*
+ * We maintain a private dentry LRU.
+ *
+ * FIXME: this needs to be changed to a per-mds lru to be useful.
+ */
+void ceph_dentry_lru_add(struct dentry *dn)
+{
+ struct ceph_dentry_info *di = ceph_dentry(dn);
+ struct ceph_mds_client *mdsc;
+
+ dout("dentry_lru_add %p %p '%.*s'\n", di, dn,
+ dn->d_name.len, dn->d_name.name);
+ if (di) {
+ mdsc = &ceph_client(dn->d_sb)->mdsc;
+ spin_lock(&mdsc->dentry_lru_lock);
+ list_add_tail(&di->lru, &mdsc->dentry_lru);
+ mdsc->num_dentry++;
+ spin_unlock(&mdsc->dentry_lru_lock);
+ }
+}
+
+void ceph_dentry_lru_touch(struct dentry *dn)
+{
+ struct ceph_dentry_info *di = ceph_dentry(dn);
+ struct ceph_mds_client *mdsc;
+
+ dout("dentry_lru_touch %p %p '%.*s'\n", di, dn,
+ dn->d_name.len, dn->d_name.name);
+ if (di) {
+ mdsc = &ceph_client(dn->d_sb)->mdsc;
+ spin_lock(&mdsc->dentry_lru_lock);
+ list_move_tail(&di->lru, &mdsc->dentry_lru);
+ spin_unlock(&mdsc->dentry_lru_lock);
+ }
+}
+
+void ceph_dentry_lru_del(struct dentry *dn)
+{
+ struct ceph_dentry_info *di = ceph_dentry(dn);
+ struct ceph_mds_client *mdsc;
+
+ dout("dentry_lru_del %p %p '%.*s'\n", di, dn,
+ dn->d_name.len, dn->d_name.name);
+ if (di) {
+ mdsc = &ceph_client(dn->d_sb)->mdsc;
+ spin_lock(&mdsc->dentry_lru_lock);
+ list_del_init(&di->lru);
+ mdsc->num_dentry--;
+ spin_unlock(&mdsc->dentry_lru_lock);
+ }
+}
+
+const struct file_operations ceph_dir_fops = {
+ .read = ceph_read_dir,
+ .readdir = ceph_readdir,
+ .llseek = ceph_dir_llseek,
+ .open = ceph_open,
+ .release = ceph_release,
+ .unlocked_ioctl = ceph_ioctl,
+ .fsync = ceph_dir_fsync,
+};
+
+const struct inode_operations ceph_dir_iops = {
+ .lookup = ceph_lookup,
+ .permission = ceph_permission,
+ .getattr = ceph_getattr,
+ .setattr = ceph_setattr,
+ .setxattr = ceph_setxattr,
+ .getxattr = ceph_getxattr,
+ .listxattr = ceph_listxattr,
+ .removexattr = ceph_removexattr,
+ .mknod = ceph_mknod,
+ .symlink = ceph_symlink,
+ .mkdir = ceph_mkdir,
+ .link = ceph_link,
+ .unlink = ceph_unlink,
+ .rmdir = ceph_unlink,
+ .rename = ceph_rename,
+ .create = ceph_create,
+};
+
+struct dentry_operations ceph_dentry_ops = {
+ .d_revalidate = ceph_d_revalidate,
+ .d_release = ceph_dentry_release,
+};
+
+struct dentry_operations ceph_snapdir_dentry_ops = {
+ .d_revalidate = ceph_snapdir_d_revalidate,
+};
+
+struct dentry_operations ceph_snap_dentry_ops = {
+};
--- /dev/null
+#include "ceph_debug.h"
+
+#include <linux/exportfs.h>
+#include <linux/slab.h>
+#include <asm/unaligned.h>
+
+#include "super.h"
+
+/*
+ * NFS export support
+ *
+ * NFS re-export of a ceph mount is, at present, only semireliable.
+ * The basic issue is that the Ceph architectures doesn't lend itself
+ * well to generating filehandles that will remain valid forever.
+ *
+ * So, we do our best. If you're lucky, your inode will be in the
+ * client's cache. If it's not, and you have a connectable fh, then
+ * the MDS server may be able to find it for you. Otherwise, you get
+ * ESTALE.
+ *
+ * There are ways to this more reliable, but in the non-connectable fh
+ * case, we won't every work perfectly, and in the connectable case,
+ * some changes are needed on the MDS side to work better.
+ */
+
+/*
+ * Basic fh
+ */
+struct ceph_nfs_fh {
+ u64 ino;
+} __attribute__ ((packed));
+
+/*
+ * Larger 'connectable' fh that includes parent ino and name hash.
+ * Use this whenever possible, as it works more reliably.
+ */
+struct ceph_nfs_confh {
+ u64 ino, parent_ino;
+ u32 parent_name_hash;
+} __attribute__ ((packed));
+
+static int ceph_encode_fh(struct dentry *dentry, u32 *rawfh, int *max_len,
+ int connectable)
+{
+ struct ceph_nfs_fh *fh = (void *)rawfh;
+ struct ceph_nfs_confh *cfh = (void *)rawfh;
+ struct dentry *parent = dentry->d_parent;
+ struct inode *inode = dentry->d_inode;
+ int type;
+
+ /* don't re-export snaps */
+ if (ceph_snap(inode) != CEPH_NOSNAP)
+ return -EINVAL;
+
+ if (*max_len >= sizeof(*cfh)) {
+ dout("encode_fh %p connectable\n", dentry);
+ cfh->ino = ceph_ino(dentry->d_inode);
+ cfh->parent_ino = ceph_ino(parent->d_inode);
+ cfh->parent_name_hash = parent->d_name.hash;
+ *max_len = sizeof(*cfh);
+ type = 2;
+ } else if (*max_len > sizeof(*fh)) {
+ if (connectable)
+ return -ENOSPC;
+ dout("encode_fh %p\n", dentry);
+ fh->ino = ceph_ino(dentry->d_inode);
+ *max_len = sizeof(*fh);
+ type = 1;
+ } else {
+ return -ENOSPC;
+ }
+ return type;
+}
+
+/*
+ * convert regular fh to dentry
+ *
+ * FIXME: we should try harder by querying the mds for the ino.
+ */
+static struct dentry *__fh_to_dentry(struct super_block *sb,
+ struct ceph_nfs_fh *fh)
+{
+ struct inode *inode;
+ struct dentry *dentry;
+ struct ceph_vino vino;
+ int err;
+
+ dout("__fh_to_dentry %llx\n", fh->ino);
+ vino.ino = fh->ino;
+ vino.snap = CEPH_NOSNAP;
+ inode = ceph_find_inode(sb, vino);
+ if (!inode)
+ return ERR_PTR(-ESTALE);
+
+ dentry = d_obtain_alias(inode);
+ if (!dentry) {
+ pr_err("fh_to_dentry %llx -- inode %p but ENOMEM\n",
+ fh->ino, inode);
+ iput(inode);
+ return ERR_PTR(-ENOMEM);
+ }
+ err = ceph_init_dentry(dentry);
+
+ if (err < 0) {
+ iput(inode);
+ return ERR_PTR(err);
+ }
+ dout("__fh_to_dentry %llx %p dentry %p\n", fh->ino, inode, dentry);
+ return dentry;
+}
+
+/*
+ * convert connectable fh to dentry
+ */
+static struct dentry *__cfh_to_dentry(struct super_block *sb,
+ struct ceph_nfs_confh *cfh)
+{
+ struct ceph_mds_client *mdsc = &ceph_client(sb)->mdsc;
+ struct inode *inode;
+ struct dentry *dentry;
+ struct ceph_vino vino;
+ int err;
+
+ dout("__cfh_to_dentry %llx (%llx/%x)\n",
+ cfh->ino, cfh->parent_ino, cfh->parent_name_hash);
+
+ vino.ino = cfh->ino;
+ vino.snap = CEPH_NOSNAP;
+ inode = ceph_find_inode(sb, vino);
+ if (!inode) {
+ struct ceph_mds_request *req;
+
+ req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_LOOKUPHASH,
+ USE_ANY_MDS);
+ if (IS_ERR(req))
+ return ERR_PTR(PTR_ERR(req));
+
+ req->r_ino1 = vino;
+ req->r_ino2.ino = cfh->parent_ino;
+ req->r_ino2.snap = CEPH_NOSNAP;
+ req->r_path2 = kmalloc(16, GFP_NOFS);
+ snprintf(req->r_path2, 16, "%d", cfh->parent_name_hash);
+ req->r_num_caps = 1;
+ err = ceph_mdsc_do_request(mdsc, NULL, req);
+ ceph_mdsc_put_request(req);
+ inode = ceph_find_inode(sb, vino);
+ if (!inode)
+ return ERR_PTR(err ? err : -ESTALE);
+ }
+
+ dentry = d_obtain_alias(inode);
+ if (!dentry) {
+ pr_err("cfh_to_dentry %llx -- inode %p but ENOMEM\n",
+ cfh->ino, inode);
+ iput(inode);
+ return ERR_PTR(-ENOMEM);
+ }
+ err = ceph_init_dentry(dentry);
+ if (err < 0) {
+ iput(inode);
+ return ERR_PTR(err);
+ }
+ dout("__cfh_to_dentry %llx %p dentry %p\n", cfh->ino, inode, dentry);
+ return dentry;
+}
+
+static struct dentry *ceph_fh_to_dentry(struct super_block *sb, struct fid *fid,
+ int fh_len, int fh_type)
+{
+ if (fh_type == 1)
+ return __fh_to_dentry(sb, (struct ceph_nfs_fh *)fid->raw);
+ else
+ return __cfh_to_dentry(sb, (struct ceph_nfs_confh *)fid->raw);
+}
+
+/*
+ * get parent, if possible.
+ *
+ * FIXME: we could do better by querying the mds to discover the
+ * parent.
+ */
+static struct dentry *ceph_fh_to_parent(struct super_block *sb,
+ struct fid *fid,
+ int fh_len, int fh_type)
+{
+ struct ceph_nfs_confh *cfh = (void *)fid->raw;
+ struct ceph_vino vino;
+ struct inode *inode;
+ struct dentry *dentry;
+ int err;
+
+ if (fh_type == 1)
+ return ERR_PTR(-ESTALE);
+
+ pr_debug("fh_to_parent %llx/%d\n", cfh->parent_ino,
+ cfh->parent_name_hash);
+
+ vino.ino = cfh->ino;
+ vino.snap = CEPH_NOSNAP;
+ inode = ceph_find_inode(sb, vino);
+ if (!inode)
+ return ERR_PTR(-ESTALE);
+
+ dentry = d_obtain_alias(inode);
+ if (!dentry) {
+ pr_err("fh_to_parent %llx -- inode %p but ENOMEM\n",
+ cfh->ino, inode);
+ iput(inode);
+ return ERR_PTR(-ENOMEM);
+ }
+ err = ceph_init_dentry(dentry);
+ if (err < 0) {
+ iput(inode);
+ return ERR_PTR(err);
+ }
+ dout("fh_to_parent %llx %p dentry %p\n", cfh->ino, inode, dentry);
+ return dentry;
+}
+
+const struct export_operations ceph_export_ops = {
+ .encode_fh = ceph_encode_fh,
+ .fh_to_dentry = ceph_fh_to_dentry,
+ .fh_to_parent = ceph_fh_to_parent,
+};
--- /dev/null
+#include "ceph_debug.h"
+
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/file.h>
+#include <linux/namei.h>
+#include <linux/writeback.h>
+
+#include "super.h"
+#include "mds_client.h"
+
+/*
+ * Ceph file operations
+ *
+ * Implement basic open/close functionality, and implement
+ * read/write.
+ *
+ * We implement three modes of file I/O:
+ * - buffered uses the generic_file_aio_{read,write} helpers
+ *
+ * - synchronous is used when there is multi-client read/write
+ * sharing, avoids the page cache, and synchronously waits for an
+ * ack from the OSD.
+ *
+ * - direct io takes the variant of the sync path that references
+ * user pages directly.
+ *
+ * fsync() flushes and waits on dirty pages, but just queues metadata
+ * for writeback: since the MDS can recover size and mtime there is no
+ * need to wait for MDS acknowledgement.
+ */
+
+
+/*
+ * Prepare an open request. Preallocate ceph_cap to avoid an
+ * inopportune ENOMEM later.
+ */
+static struct ceph_mds_request *
+prepare_open_request(struct super_block *sb, int flags, int create_mode)
+{
+ struct ceph_client *client = ceph_sb_to_client(sb);
+ struct ceph_mds_client *mdsc = &client->mdsc;
+ struct ceph_mds_request *req;
+ int want_auth = USE_ANY_MDS;
+ int op = (flags & O_CREAT) ? CEPH_MDS_OP_CREATE : CEPH_MDS_OP_OPEN;
+
+ if (flags & (O_WRONLY|O_RDWR|O_CREAT|O_TRUNC))
+ want_auth = USE_AUTH_MDS;
+
+ req = ceph_mdsc_create_request(mdsc, op, want_auth);
+ if (IS_ERR(req))
+ goto out;
+ req->r_fmode = ceph_flags_to_mode(flags);
+ req->r_args.open.flags = cpu_to_le32(flags);
+ req->r_args.open.mode = cpu_to_le32(create_mode);
+ req->r_args.open.preferred = cpu_to_le32(-1);
+out:
+ return req;
+}
+
+/*
+ * initialize private struct file data.
+ * if we fail, clean up by dropping fmode reference on the ceph_inode
+ */
+static int ceph_init_file(struct inode *inode, struct file *file, int fmode)
+{
+ struct ceph_file_info *cf;
+ int ret = 0;
+
+ switch (inode->i_mode & S_IFMT) {
+ case S_IFREG:
+ case S_IFDIR:
+ dout("init_file %p %p 0%o (regular)\n", inode, file,
+ inode->i_mode);
+ cf = kmem_cache_alloc(ceph_file_cachep, GFP_NOFS | __GFP_ZERO);
+ if (cf == NULL) {
+ ceph_put_fmode(ceph_inode(inode), fmode); /* clean up */
+ return -ENOMEM;
+ }
+ cf->fmode = fmode;
+ cf->next_offset = 2;
+ file->private_data = cf;
+ BUG_ON(inode->i_fop->release != ceph_release);
+ break;
+
+ case S_IFLNK:
+ dout("init_file %p %p 0%o (symlink)\n", inode, file,
+ inode->i_mode);
+ ceph_put_fmode(ceph_inode(inode), fmode); /* clean up */
+ break;
+
+ default:
+ dout("init_file %p %p 0%o (special)\n", inode, file,
+ inode->i_mode);
+ /*
+ * we need to drop the open ref now, since we don't
+ * have .release set to ceph_release.
+ */
+ ceph_put_fmode(ceph_inode(inode), fmode); /* clean up */
+ BUG_ON(inode->i_fop->release == ceph_release);
+
+ /* call the proper open fop */
+ ret = inode->i_fop->open(inode, file);
+ }
+ return ret;
+}
+
+/*
+ * If the filp already has private_data, that means the file was
+ * already opened by intent during lookup, and we do nothing.
+ *
+ * If we already have the requisite capabilities, we can satisfy
+ * the open request locally (no need to request new caps from the
+ * MDS). We do, however, need to inform the MDS (asynchronously)
+ * if our wanted caps set expands.
+ */
+int ceph_open(struct inode *inode, struct file *file)
+{
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ struct ceph_client *client = ceph_sb_to_client(inode->i_sb);
+ struct ceph_mds_client *mdsc = &client->mdsc;
+ struct ceph_mds_request *req;
+ struct ceph_file_info *cf = file->private_data;
+ struct inode *parent_inode = file->f_dentry->d_parent->d_inode;
+ int err;
+ int flags, fmode, wanted;
+
+ if (cf) {
+ dout("open file %p is already opened\n", file);
+ return 0;
+ }
+
+ /* filter out O_CREAT|O_EXCL; vfs did that already. yuck. */
+ flags = file->f_flags & ~(O_CREAT|O_EXCL);
+ if (S_ISDIR(inode->i_mode))
+ flags = O_DIRECTORY; /* mds likes to know */
+
+ dout("open inode %p ino %llx.%llx file %p flags %d (%d)\n", inode,
+ ceph_vinop(inode), file, flags, file->f_flags);
+ fmode = ceph_flags_to_mode(flags);
+ wanted = ceph_caps_for_mode(fmode);
+
+ /* snapped files are read-only */
+ if (ceph_snap(inode) != CEPH_NOSNAP && (file->f_mode & FMODE_WRITE))
+ return -EROFS;
+
+ /* trivially open snapdir */
+ if (ceph_snap(inode) == CEPH_SNAPDIR) {
+ spin_lock(&inode->i_lock);
+ __ceph_get_fmode(ci, fmode);
+ spin_unlock(&inode->i_lock);
+ return ceph_init_file(inode, file, fmode);
+ }
+
+ /*
+ * No need to block if we have any caps. Update wanted set
+ * asynchronously.
+ */
+ spin_lock(&inode->i_lock);
+ if (__ceph_is_any_real_caps(ci)) {
+ int mds_wanted = __ceph_caps_mds_wanted(ci);
+ int issued = __ceph_caps_issued(ci, NULL);
+
+ dout("open %p fmode %d want %s issued %s using existing\n",
+ inode, fmode, ceph_cap_string(wanted),
+ ceph_cap_string(issued));
+ __ceph_get_fmode(ci, fmode);
+ spin_unlock(&inode->i_lock);
+
+ /* adjust wanted? */
+ if ((issued & wanted) != wanted &&
+ (mds_wanted & wanted) != wanted &&
+ ceph_snap(inode) != CEPH_SNAPDIR)
+ ceph_check_caps(ci, 0, NULL);
+
+ return ceph_init_file(inode, file, fmode);
+ } else if (ceph_snap(inode) != CEPH_NOSNAP &&
+ (ci->i_snap_caps & wanted) == wanted) {
+ __ceph_get_fmode(ci, fmode);
+ spin_unlock(&inode->i_lock);
+ return ceph_init_file(inode, file, fmode);
+ }
+ spin_unlock(&inode->i_lock);
+
+ dout("open fmode %d wants %s\n", fmode, ceph_cap_string(wanted));
+ req = prepare_open_request(inode->i_sb, flags, 0);
+ if (IS_ERR(req)) {
+ err = PTR_ERR(req);
+ goto out;
+ }
+ req->r_inode = igrab(inode);
+ req->r_num_caps = 1;
+ err = ceph_mdsc_do_request(mdsc, parent_inode, req);
+ if (!err)
+ err = ceph_init_file(inode, file, req->r_fmode);
+ ceph_mdsc_put_request(req);
+ dout("open result=%d on %llx.%llx\n", err, ceph_vinop(inode));
+out:
+ return err;
+}
+
+
+/*
+ * Do a lookup + open with a single request.
+ *
+ * If this succeeds, but some subsequent check in the vfs
+ * may_open() fails, the struct *file gets cleaned up (i.e.
+ * ceph_release gets called). So fear not!
+ */
+/*
+ * flags
+ * path_lookup_open -> LOOKUP_OPEN
+ * path_lookup_create -> LOOKUP_OPEN|LOOKUP_CREATE
+ */
+struct dentry *ceph_lookup_open(struct inode *dir, struct dentry *dentry,
+ struct nameidata *nd, int mode,
+ int locked_dir)
+{
+ struct ceph_client *client = ceph_sb_to_client(dir->i_sb);
+ struct ceph_mds_client *mdsc = &client->mdsc;
+ struct file *file = nd->intent.open.file;
+ struct inode *parent_inode = get_dentry_parent_inode(file->f_dentry);
+ struct ceph_mds_request *req;
+ int err;
+ int flags = nd->intent.open.flags - 1; /* silly vfs! */
+
+ dout("ceph_lookup_open dentry %p '%.*s' flags %d mode 0%o\n",
+ dentry, dentry->d_name.len, dentry->d_name.name, flags, mode);
+
+ /* do the open */
+ req = prepare_open_request(dir->i_sb, flags, mode);
+ if (IS_ERR(req))
+ return ERR_PTR(PTR_ERR(req));
+ req->r_dentry = dget(dentry);
+ req->r_num_caps = 2;
+ if (flags & O_CREAT) {
+ req->r_dentry_drop = CEPH_CAP_FILE_SHARED;
+ req->r_dentry_unless = CEPH_CAP_FILE_EXCL;
+ }
+ req->r_locked_dir = dir; /* caller holds dir->i_mutex */
+ err = ceph_mdsc_do_request(mdsc, parent_inode, req);
+ dentry = ceph_finish_lookup(req, dentry, err);
+ if (!err && (flags & O_CREAT) && !req->r_reply_info.head->is_dentry)
+ err = ceph_handle_notrace_create(dir, dentry);
+ if (!err)
+ err = ceph_init_file(req->r_dentry->d_inode, file,
+ req->r_fmode);
+ ceph_mdsc_put_request(req);
+ dout("ceph_lookup_open result=%p\n", dentry);
+ return dentry;
+}
+
+int ceph_release(struct inode *inode, struct file *file)
+{
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ struct ceph_file_info *cf = file->private_data;
+
+ dout("release inode %p file %p\n", inode, file);
+ ceph_put_fmode(ci, cf->fmode);
+ if (cf->last_readdir)
+ ceph_mdsc_put_request(cf->last_readdir);
+ kfree(cf->last_name);
+ kfree(cf->dir_info);
+ dput(cf->dentry);
+ kmem_cache_free(ceph_file_cachep, cf);
+
+ /* wake up anyone waiting for caps on this inode */
+ wake_up(&ci->i_cap_wq);
+ return 0;
+}
+
+/*
+ * build a vector of user pages
+ */
+static struct page **get_direct_page_vector(const char __user *data,
+ int num_pages,
+ loff_t off, size_t len)
+{
+ struct page **pages;
+ int rc;
+
+ pages = kmalloc(sizeof(*pages) * num_pages, GFP_NOFS);
+ if (!pages)
+ return ERR_PTR(-ENOMEM);
+
+ down_read(¤t->mm->mmap_sem);
+ rc = get_user_pages(current, current->mm, (unsigned long)data,
+ num_pages, 0, 0, pages, NULL);
+ up_read(¤t->mm->mmap_sem);
+ if (rc < 0)
+ goto fail;
+ return pages;
+
+fail:
+ kfree(pages);
+ return ERR_PTR(rc);
+}
+
+static void put_page_vector(struct page **pages, int num_pages)
+{
+ int i;
+
+ for (i = 0; i < num_pages; i++)
+ put_page(pages[i]);
+ kfree(pages);
+}
+
+void ceph_release_page_vector(struct page **pages, int num_pages)
+{
+ int i;
+
+ for (i = 0; i < num_pages; i++)
+ __free_pages(pages[i], 0);
+ kfree(pages);
+}
+
+/*
+ * allocate a vector new pages
+ */
+static struct page **alloc_page_vector(int num_pages)
+{
+ struct page **pages;
+ int i;
+
+ pages = kmalloc(sizeof(*pages) * num_pages, GFP_NOFS);
+ if (!pages)
+ return ERR_PTR(-ENOMEM);
+ for (i = 0; i < num_pages; i++) {
+ pages[i] = alloc_page(GFP_NOFS);
+ if (pages[i] == NULL) {
+ ceph_release_page_vector(pages, i);
+ return ERR_PTR(-ENOMEM);
+ }
+ }
+ return pages;
+}
+
+/*
+ * copy user data into a page vector
+ */
+static int copy_user_to_page_vector(struct page **pages,
+ const char __user *data,
+ loff_t off, size_t len)
+{
+ int i = 0;
+ int po = off & ~PAGE_CACHE_MASK;
+ int left = len;
+ int l, bad;
+
+ while (left > 0) {
+ l = min_t(int, PAGE_CACHE_SIZE-po, left);
+ bad = copy_from_user(page_address(pages[i]) + po, data, l);
+ if (bad == l)
+ return -EFAULT;
+ data += l - bad;
+ left -= l - bad;
+ po += l - bad;
+ if (po == PAGE_CACHE_SIZE) {
+ po = 0;
+ i++;
+ }
+ }
+ return len;
+}
+
+/*
+ * copy user data from a page vector into a user pointer
+ */
+static int copy_page_vector_to_user(struct page **pages, char __user *data,
+ loff_t off, size_t len)
+{
+ int i = 0;
+ int po = off & ~PAGE_CACHE_MASK;
+ int left = len;
+ int l, bad;
+
+ while (left > 0) {
+ l = min_t(int, left, PAGE_CACHE_SIZE-po);
+ bad = copy_to_user(data, page_address(pages[i]) + po, l);
+ if (bad == l)
+ return -EFAULT;
+ data += l - bad;
+ left -= l - bad;
+ if (po) {
+ po += l - bad;
+ if (po == PAGE_CACHE_SIZE)
+ po = 0;
+ }
+ i++;
+ }
+ return len;
+}
+
+/*
+ * Zero an extent within a page vector. Offset is relative to the
+ * start of the first page.
+ */
+static void zero_page_vector_range(int off, int len, struct page **pages)
+{
+ int i = off >> PAGE_CACHE_SHIFT;
+
+ off &= ~PAGE_CACHE_MASK;
+
+ dout("zero_page_vector_page %u~%u\n", off, len);
+
+ /* leading partial page? */
+ if (off) {
+ int end = min((int)PAGE_CACHE_SIZE, off + len);
+ dout("zeroing %d %p head from %d\n", i, pages[i],
+ (int)off);
+ zero_user_segment(pages[i], off, end);
+ len -= (end - off);
+ i++;
+ }
+ while (len >= PAGE_CACHE_SIZE) {
+ dout("zeroing %d %p len=%d\n", i, pages[i], len);
+ zero_user_segment(pages[i], 0, PAGE_CACHE_SIZE);
+ len -= PAGE_CACHE_SIZE;
+ i++;
+ }
+ /* trailing partial page? */
+ if (len) {
+ dout("zeroing %d %p tail to %d\n", i, pages[i], (int)len);
+ zero_user_segment(pages[i], 0, len);
+ }
+}
+
+
+/*
+ * Read a range of bytes striped over one or more objects. Iterate over
+ * objects we stripe over. (That's not atomic, but good enough for now.)
+ *
+ * If we get a short result from the OSD, check against i_size; we need to
+ * only return a short read to the caller if we hit EOF.
+ */
+static int striped_read(struct inode *inode,
+ u64 off, u64 len,
+ struct page **pages, int num_pages,
+ int *checkeof)
+{
+ struct ceph_client *client = ceph_inode_to_client(inode);
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ u64 pos, this_len;
+ int page_off = off & ~PAGE_CACHE_MASK; /* first byte's offset in page */
+ int left, pages_left;
+ int read;
+ struct page **page_pos;
+ int ret;
+ bool hit_stripe, was_short;
+
+ /*
+ * we may need to do multiple reads. not atomic, unfortunately.
+ */
+ pos = off;
+ left = len;
+ page_pos = pages;
+ pages_left = num_pages;
+ read = 0;
+
+more:
+ this_len = left;
+ ret = ceph_osdc_readpages(&client->osdc, ceph_vino(inode),
+ &ci->i_layout, pos, &this_len,
+ ci->i_truncate_seq,
+ ci->i_truncate_size,
+ page_pos, pages_left);
+ hit_stripe = this_len < left;
+ was_short = ret >= 0 && ret < this_len;
+ if (ret == -ENOENT)
+ ret = 0;
+ dout("striped_read %llu~%u (read %u) got %d%s%s\n", pos, left, read,
+ ret, hit_stripe ? " HITSTRIPE" : "", was_short ? " SHORT" : "");
+
+ if (ret > 0) {
+ int didpages =
+ ((pos & ~PAGE_CACHE_MASK) + ret) >> PAGE_CACHE_SHIFT;
+
+ if (read < pos - off) {
+ dout(" zero gap %llu to %llu\n", off + read, pos);
+ zero_page_vector_range(page_off + read,
+ pos - off - read, pages);
+ }
+ pos += ret;
+ read = pos - off;
+ left -= ret;
+ page_pos += didpages;
+ pages_left -= didpages;
+
+ /* hit stripe? */
+ if (left && hit_stripe)
+ goto more;
+ }
+
+ if (was_short) {
+ /* was original extent fully inside i_size? */
+ if (pos + left <= inode->i_size) {
+ dout("zero tail\n");
+ zero_page_vector_range(page_off + read, len - read,
+ pages);
+ read = len;
+ goto out;
+ }
+
+ /* check i_size */
+ *checkeof = 1;
+ }
+
+out:
+ if (ret >= 0)
+ ret = read;
+ dout("striped_read returns %d\n", ret);
+ return ret;
+}
+
+/*
+ * Completely synchronous read and write methods. Direct from __user
+ * buffer to osd, or directly to user pages (if O_DIRECT).
+ *
+ * If the read spans object boundary, just do multiple reads.
+ */
+static ssize_t ceph_sync_read(struct file *file, char __user *data,
+ unsigned len, loff_t *poff, int *checkeof)
+{
+ struct inode *inode = file->f_dentry->d_inode;
+ struct page **pages;
+ u64 off = *poff;
+ int num_pages = calc_pages_for(off, len);
+ int ret;
+
+ dout("sync_read on file %p %llu~%u %s\n", file, off, len,
+ (file->f_flags & O_DIRECT) ? "O_DIRECT" : "");
+
+ if (file->f_flags & O_DIRECT) {
+ pages = get_direct_page_vector(data, num_pages, off, len);
+
+ /*
+ * flush any page cache pages in this range. this
+ * will make concurrent normal and O_DIRECT io slow,
+ * but it will at least behave sensibly when they are
+ * in sequence.
+ */
+ } else {
+ pages = alloc_page_vector(num_pages);
+ }
+ if (IS_ERR(pages))
+ return PTR_ERR(pages);
+
+ ret = filemap_write_and_wait(inode->i_mapping);
+ if (ret < 0)
+ goto done;
+
+ ret = striped_read(inode, off, len, pages, num_pages, checkeof);
+
+ if (ret >= 0 && (file->f_flags & O_DIRECT) == 0)
+ ret = copy_page_vector_to_user(pages, data, off, ret);
+ if (ret >= 0)
+ *poff = off + ret;
+
+done:
+ if (file->f_flags & O_DIRECT)
+ put_page_vector(pages, num_pages);
+ else
+ ceph_release_page_vector(pages, num_pages);
+ dout("sync_read result %d\n", ret);
+ return ret;
+}
+
+/*
+ * Write commit callback, called if we requested both an ACK and
+ * ONDISK commit reply from the OSD.
+ */
+static void sync_write_commit(struct ceph_osd_request *req,
+ struct ceph_msg *msg)
+{
+ struct ceph_inode_info *ci = ceph_inode(req->r_inode);
+
+ dout("sync_write_commit %p tid %llu\n", req, req->r_tid);
+ spin_lock(&ci->i_unsafe_lock);
+ list_del_init(&req->r_unsafe_item);
+ spin_unlock(&ci->i_unsafe_lock);
+ ceph_put_cap_refs(ci, CEPH_CAP_FILE_WR);
+}
+
+/*
+ * Synchronous write, straight from __user pointer or user pages (if
+ * O_DIRECT).
+ *
+ * If write spans object boundary, just do multiple writes. (For a
+ * correct atomic write, we should e.g. take write locks on all
+ * objects, rollback on failure, etc.)
+ */
+static ssize_t ceph_sync_write(struct file *file, const char __user *data,
+ size_t left, loff_t *offset)
+{
+ struct inode *inode = file->f_dentry->d_inode;
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ struct ceph_client *client = ceph_inode_to_client(inode);
+ struct ceph_osd_request *req;
+ struct page **pages;
+ int num_pages;
+ long long unsigned pos;
+ u64 len;
+ int written = 0;
+ int flags;
+ int do_sync = 0;
+ int check_caps = 0;
+ int ret;
+ struct timespec mtime = CURRENT_TIME;
+
+ if (ceph_snap(file->f_dentry->d_inode) != CEPH_NOSNAP)
+ return -EROFS;
+
+ dout("sync_write on file %p %lld~%u %s\n", file, *offset,
+ (unsigned)left, (file->f_flags & O_DIRECT) ? "O_DIRECT" : "");
+
+ if (file->f_flags & O_APPEND)
+ pos = i_size_read(inode);
+ else
+ pos = *offset;
+
+ ret = filemap_write_and_wait_range(inode->i_mapping, pos, pos + left);
+ if (ret < 0)
+ return ret;
+
+ ret = invalidate_inode_pages2_range(inode->i_mapping,
+ pos >> PAGE_CACHE_SHIFT,
+ (pos + left) >> PAGE_CACHE_SHIFT);
+ if (ret < 0)
+ dout("invalidate_inode_pages2_range returned %d\n", ret);
+
+ flags = CEPH_OSD_FLAG_ORDERSNAP |
+ CEPH_OSD_FLAG_ONDISK |
+ CEPH_OSD_FLAG_WRITE;
+ if ((file->f_flags & (O_SYNC|O_DIRECT)) == 0)
+ flags |= CEPH_OSD_FLAG_ACK;
+ else
+ do_sync = 1;
+
+ /*
+ * we may need to do multiple writes here if we span an object
+ * boundary. this isn't atomic, unfortunately. :(
+ */
+more:
+ len = left;
+ req = ceph_osdc_new_request(&client->osdc, &ci->i_layout,
+ ceph_vino(inode), pos, &len,
+ CEPH_OSD_OP_WRITE, flags,
+ ci->i_snap_realm->cached_context,
+ do_sync,
+ ci->i_truncate_seq, ci->i_truncate_size,
+ &mtime, false, 2);
+ if (IS_ERR(req))
+ return PTR_ERR(req);
+
+ num_pages = calc_pages_for(pos, len);
+
+ if (file->f_flags & O_DIRECT) {
+ pages = get_direct_page_vector(data, num_pages, pos, len);
+ if (IS_ERR(pages)) {
+ ret = PTR_ERR(pages);
+ goto out;
+ }
+
+ /*
+ * throw out any page cache pages in this range. this
+ * may block.
+ */
+ truncate_inode_pages_range(inode->i_mapping, pos, pos+len);
+ } else {
+ pages = alloc_page_vector(num_pages);
+ if (IS_ERR(pages)) {
+ ret = PTR_ERR(pages);
+ goto out;
+ }
+ ret = copy_user_to_page_vector(pages, data, pos, len);
+ if (ret < 0) {
+ ceph_release_page_vector(pages, num_pages);
+ goto out;
+ }
+
+ if ((file->f_flags & O_SYNC) == 0) {
+ /* get a second commit callback */
+ req->r_safe_callback = sync_write_commit;
+ req->r_own_pages = 1;
+ }
+ }
+ req->r_pages = pages;
+ req->r_num_pages = num_pages;
+ req->r_inode = inode;
+
+ ret = ceph_osdc_start_request(&client->osdc, req, false);
+ if (!ret) {
+ if (req->r_safe_callback) {
+ /*
+ * Add to inode unsafe list only after we
+ * start_request so that a tid has been assigned.
+ */
+ spin_lock(&ci->i_unsafe_lock);
+ list_add(&ci->i_unsafe_writes, &req->r_unsafe_item);
+ spin_unlock(&ci->i_unsafe_lock);
+ ceph_get_cap_refs(ci, CEPH_CAP_FILE_WR);
+ }
+ ret = ceph_osdc_wait_request(&client->osdc, req);
+ }
+
+ if (file->f_flags & O_DIRECT)
+ put_page_vector(pages, num_pages);
+ else if (file->f_flags & O_SYNC)
+ ceph_release_page_vector(pages, num_pages);
+
+out:
+ ceph_osdc_put_request(req);
+ if (ret == 0) {
+ pos += len;
+ written += len;
+ left -= len;
+ if (left)
+ goto more;
+
+ ret = written;
+ *offset = pos;
+ if (pos > i_size_read(inode))
+ check_caps = ceph_inode_set_size(inode, pos);
+ if (check_caps)
+ ceph_check_caps(ceph_inode(inode), CHECK_CAPS_AUTHONLY,
+ NULL);
+ }
+ return ret;
+}
+
+/*
+ * Wrap generic_file_aio_read with checks for cap bits on the inode.
+ * Atomically grab references, so that those bits are not released
+ * back to the MDS mid-read.
+ *
+ * Hmm, the sync read case isn't actually async... should it be?
+ */
+static ssize_t ceph_aio_read(struct kiocb *iocb, const struct iovec *iov,
+ unsigned long nr_segs, loff_t pos)
+{
+ struct file *filp = iocb->ki_filp;
+ loff_t *ppos = &iocb->ki_pos;
+ size_t len = iov->iov_len;
+ struct inode *inode = filp->f_dentry->d_inode;
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ void *base = iov->iov_base;
+ ssize_t ret;
+ int got = 0;
+ int checkeof = 0, read = 0;
+
+ dout("aio_read %p %llx.%llx %llu~%u trying to get caps on %p\n",
+ inode, ceph_vinop(inode), pos, (unsigned)len, inode);
+again:
+ __ceph_do_pending_vmtruncate(inode);
+ ret = ceph_get_caps(ci, CEPH_CAP_FILE_RD, CEPH_CAP_FILE_CACHE,
+ &got, -1);
+ if (ret < 0)
+ goto out;
+ dout("aio_read %p %llx.%llx %llu~%u got cap refs on %s\n",
+ inode, ceph_vinop(inode), pos, (unsigned)len,
+ ceph_cap_string(got));
+
+ if ((got & CEPH_CAP_FILE_CACHE) == 0 ||
+ (iocb->ki_filp->f_flags & O_DIRECT) ||
+ (inode->i_sb->s_flags & MS_SYNCHRONOUS))
+ /* hmm, this isn't really async... */
+ ret = ceph_sync_read(filp, base, len, ppos, &checkeof);
+ else
+ ret = generic_file_aio_read(iocb, iov, nr_segs, pos);
+
+out:
+ dout("aio_read %p %llx.%llx dropping cap refs on %s = %d\n",
+ inode, ceph_vinop(inode), ceph_cap_string(got), (int)ret);
+ ceph_put_cap_refs(ci, got);
+
+ if (checkeof && ret >= 0) {
+ int statret = ceph_do_getattr(inode, CEPH_STAT_CAP_SIZE);
+
+ /* hit EOF or hole? */
+ if (statret == 0 && *ppos < inode->i_size) {
+ dout("aio_read sync_read hit hole, reading more\n");
+ read += ret;
+ base += ret;
+ len -= ret;
+ checkeof = 0;
+ goto again;
+ }
+ }
+ if (ret >= 0)
+ ret += read;
+
+ return ret;
+}
+
+/*
+ * Take cap references to avoid releasing caps to MDS mid-write.
+ *
+ * If we are synchronous, and write with an old snap context, the OSD
+ * may return EOLDSNAPC. In that case, retry the write.. _after_
+ * dropping our cap refs and allowing the pending snap to logically
+ * complete _before_ this write occurs.
+ *
+ * If we are near ENOSPC, write synchronously.
+ */
+static ssize_t ceph_aio_write(struct kiocb *iocb, const struct iovec *iov,
+ unsigned long nr_segs, loff_t pos)
+{
+ struct file *file = iocb->ki_filp;
+ struct inode *inode = file->f_dentry->d_inode;
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ struct ceph_osd_client *osdc = &ceph_client(inode->i_sb)->osdc;
+ loff_t endoff = pos + iov->iov_len;
+ int got = 0;
+ int ret, err;
+
+ if (ceph_snap(inode) != CEPH_NOSNAP)
+ return -EROFS;
+
+retry_snap:
+ if (ceph_osdmap_flag(osdc->osdmap, CEPH_OSDMAP_FULL))
+ return -ENOSPC;
+ __ceph_do_pending_vmtruncate(inode);
+ dout("aio_write %p %llx.%llx %llu~%u getting caps. i_size %llu\n",
+ inode, ceph_vinop(inode), pos, (unsigned)iov->iov_len,
+ inode->i_size);
+ ret = ceph_get_caps(ci, CEPH_CAP_FILE_WR, CEPH_CAP_FILE_BUFFER,
+ &got, endoff);
+ if (ret < 0)
+ goto out;
+
+ dout("aio_write %p %llx.%llx %llu~%u got cap refs on %s\n",
+ inode, ceph_vinop(inode), pos, (unsigned)iov->iov_len,
+ ceph_cap_string(got));
+
+ if ((got & CEPH_CAP_FILE_BUFFER) == 0 ||
+ (iocb->ki_filp->f_flags & O_DIRECT) ||
+ (inode->i_sb->s_flags & MS_SYNCHRONOUS)) {
+ ret = ceph_sync_write(file, iov->iov_base, iov->iov_len,
+ &iocb->ki_pos);
+ } else {
+ ret = generic_file_aio_write(iocb, iov, nr_segs, pos);
+
+ if ((ret >= 0 || ret == -EIOCBQUEUED) &&
+ ((file->f_flags & O_SYNC) || IS_SYNC(file->f_mapping->host)
+ || ceph_osdmap_flag(osdc->osdmap, CEPH_OSDMAP_NEARFULL))) {
+ err = vfs_fsync_range(file, file->f_path.dentry,
+ pos, pos + ret - 1, 1);
+ if (err < 0)
+ ret = err;
+ }
+ }
+ if (ret >= 0) {
+ spin_lock(&inode->i_lock);
+ __ceph_mark_dirty_caps(ci, CEPH_CAP_FILE_WR);
+ spin_unlock(&inode->i_lock);
+ }
+
+out:
+ dout("aio_write %p %llx.%llx %llu~%u dropping cap refs on %s\n",
+ inode, ceph_vinop(inode), pos, (unsigned)iov->iov_len,
+ ceph_cap_string(got));
+ ceph_put_cap_refs(ci, got);
+
+ if (ret == -EOLDSNAPC) {
+ dout("aio_write %p %llx.%llx %llu~%u got EOLDSNAPC, retrying\n",
+ inode, ceph_vinop(inode), pos, (unsigned)iov->iov_len);
+ goto retry_snap;
+ }
+
+ return ret;
+}
+
+/*
+ * llseek. be sure to verify file size on SEEK_END.
+ */
+static loff_t ceph_llseek(struct file *file, loff_t offset, int origin)
+{
+ struct inode *inode = file->f_mapping->host;
+ int ret;
+
+ mutex_lock(&inode->i_mutex);
+ __ceph_do_pending_vmtruncate(inode);
+ switch (origin) {
+ case SEEK_END:
+ ret = ceph_do_getattr(inode, CEPH_STAT_CAP_SIZE);
+ if (ret < 0) {
+ offset = ret;
+ goto out;
+ }
+ offset += inode->i_size;
+ break;
+ case SEEK_CUR:
+ /*
+ * Here we special-case the lseek(fd, 0, SEEK_CUR)
+ * position-querying operation. Avoid rewriting the "same"
+ * f_pos value back to the file because a concurrent read(),
+ * write() or lseek() might have altered it
+ */
+ if (offset == 0) {
+ offset = file->f_pos;
+ goto out;
+ }
+ offset += file->f_pos;
+ break;
+ }
+
+ if (offset < 0 || offset > inode->i_sb->s_maxbytes) {
+ offset = -EINVAL;
+ goto out;
+ }
+
+ /* Special lock needed here? */
+ if (offset != file->f_pos) {
+ file->f_pos = offset;
+ file->f_version = 0;
+ }
+
+out:
+ mutex_unlock(&inode->i_mutex);
+ return offset;
+}
+
+const struct file_operations ceph_file_fops = {
+ .open = ceph_open,
+ .release = ceph_release,
+ .llseek = ceph_llseek,
+ .read = do_sync_read,
+ .write = do_sync_write,
+ .aio_read = ceph_aio_read,
+ .aio_write = ceph_aio_write,
+ .mmap = ceph_mmap,
+ .fsync = ceph_fsync,
+ .splice_read = generic_file_splice_read,
+ .splice_write = generic_file_splice_write,
+ .unlocked_ioctl = ceph_ioctl,
+ .compat_ioctl = ceph_ioctl,
+};
+
--- /dev/null
+#include "ceph_debug.h"
+
+#include <linux/module.h>
+#include <linux/fs.h>
+#include <linux/smp_lock.h>
+#include <linux/slab.h>
+#include <linux/string.h>
+#include <linux/uaccess.h>
+#include <linux/kernel.h>
+#include <linux/namei.h>
+#include <linux/writeback.h>
+#include <linux/vmalloc.h>
+#include <linux/pagevec.h>
+
+#include "super.h"
+#include "decode.h"
+
+/*
+ * Ceph inode operations
+ *
+ * Implement basic inode helpers (get, alloc) and inode ops (getattr,
+ * setattr, etc.), xattr helpers, and helpers for assimilating
+ * metadata returned by the MDS into our cache.
+ *
+ * Also define helpers for doing asynchronous writeback, invalidation,
+ * and truncation for the benefit of those who can't afford to block
+ * (typically because they are in the message handler path).
+ */
+
+static const struct inode_operations ceph_symlink_iops;
+
+static void ceph_invalidate_work(struct work_struct *work);
+static void ceph_writeback_work(struct work_struct *work);
+static void ceph_vmtruncate_work(struct work_struct *work);
+
+/*
+ * find or create an inode, given the ceph ino number
+ */
+struct inode *ceph_get_inode(struct super_block *sb, struct ceph_vino vino)
+{
+ struct inode *inode;
+ ino_t t = ceph_vino_to_ino(vino);
+
+ inode = iget5_locked(sb, t, ceph_ino_compare, ceph_set_ino_cb, &vino);
+ if (inode == NULL)
+ return ERR_PTR(-ENOMEM);
+ if (inode->i_state & I_NEW) {
+ dout("get_inode created new inode %p %llx.%llx ino %llx\n",
+ inode, ceph_vinop(inode), (u64)inode->i_ino);
+ unlock_new_inode(inode);
+ }
+
+ dout("get_inode on %lu=%llx.%llx got %p\n", inode->i_ino, vino.ino,
+ vino.snap, inode);
+ return inode;
+}
+
+/*
+ * get/constuct snapdir inode for a given directory
+ */
+struct inode *ceph_get_snapdir(struct inode *parent)
+{
+ struct ceph_vino vino = {
+ .ino = ceph_ino(parent),
+ .snap = CEPH_SNAPDIR,
+ };
+ struct inode *inode = ceph_get_inode(parent->i_sb, vino);
+ struct ceph_inode_info *ci = ceph_inode(inode);
+
+ BUG_ON(!S_ISDIR(parent->i_mode));
+ if (IS_ERR(inode))
+ return ERR_PTR(PTR_ERR(inode));
+ inode->i_mode = parent->i_mode;
+ inode->i_uid = parent->i_uid;
+ inode->i_gid = parent->i_gid;
+ inode->i_op = &ceph_dir_iops;
+ inode->i_fop = &ceph_dir_fops;
+ ci->i_snap_caps = CEPH_CAP_PIN; /* so we can open */
+ ci->i_rbytes = 0;
+ return inode;
+}
+
+const struct inode_operations ceph_file_iops = {
+ .permission = ceph_permission,
+ .setattr = ceph_setattr,
+ .getattr = ceph_getattr,
+ .setxattr = ceph_setxattr,
+ .getxattr = ceph_getxattr,
+ .listxattr = ceph_listxattr,
+ .removexattr = ceph_removexattr,
+};
+
+
+/*
+ * We use a 'frag tree' to keep track of the MDS's directory fragments
+ * for a given inode (usually there is just a single fragment). We
+ * need to know when a child frag is delegated to a new MDS, or when
+ * it is flagged as replicated, so we can direct our requests
+ * accordingly.
+ */
+
+/*
+ * find/create a frag in the tree
+ */
+static struct ceph_inode_frag *__get_or_create_frag(struct ceph_inode_info *ci,
+ u32 f)
+{
+ struct rb_node **p;
+ struct rb_node *parent = NULL;
+ struct ceph_inode_frag *frag;
+ int c;
+
+ p = &ci->i_fragtree.rb_node;
+ while (*p) {
+ parent = *p;
+ frag = rb_entry(parent, struct ceph_inode_frag, node);
+ c = ceph_frag_compare(f, frag->frag);
+ if (c < 0)
+ p = &(*p)->rb_left;
+ else if (c > 0)
+ p = &(*p)->rb_right;
+ else
+ return frag;
+ }
+
+ frag = kmalloc(sizeof(*frag), GFP_NOFS);
+ if (!frag) {
+ pr_err("__get_or_create_frag ENOMEM on %p %llx.%llx "
+ "frag %x\n", &ci->vfs_inode,
+ ceph_vinop(&ci->vfs_inode), f);
+ return ERR_PTR(-ENOMEM);
+ }
+ frag->frag = f;
+ frag->split_by = 0;
+ frag->mds = -1;
+ frag->ndist = 0;
+
+ rb_link_node(&frag->node, parent, p);
+ rb_insert_color(&frag->node, &ci->i_fragtree);
+
+ dout("get_or_create_frag added %llx.%llx frag %x\n",
+ ceph_vinop(&ci->vfs_inode), f);
+ return frag;
+}
+
+/*
+ * find a specific frag @f
+ */
+struct ceph_inode_frag *__ceph_find_frag(struct ceph_inode_info *ci, u32 f)
+{
+ struct rb_node *n = ci->i_fragtree.rb_node;
+
+ while (n) {
+ struct ceph_inode_frag *frag =
+ rb_entry(n, struct ceph_inode_frag, node);
+ int c = ceph_frag_compare(f, frag->frag);
+ if (c < 0)
+ n = n->rb_left;
+ else if (c > 0)
+ n = n->rb_right;
+ else
+ return frag;
+ }
+ return NULL;
+}
+
+/*
+ * Choose frag containing the given value @v. If @pfrag is
+ * specified, copy the frag delegation info to the caller if
+ * it is present.
+ */
+u32 ceph_choose_frag(struct ceph_inode_info *ci, u32 v,
+ struct ceph_inode_frag *pfrag,
+ int *found)
+{
+ u32 t = ceph_frag_make(0, 0);
+ struct ceph_inode_frag *frag;
+ unsigned nway, i;
+ u32 n;
+
+ if (found)
+ *found = 0;
+
+ mutex_lock(&ci->i_fragtree_mutex);
+ while (1) {
+ WARN_ON(!ceph_frag_contains_value(t, v));
+ frag = __ceph_find_frag(ci, t);
+ if (!frag)
+ break; /* t is a leaf */
+ if (frag->split_by == 0) {
+ if (pfrag)
+ memcpy(pfrag, frag, sizeof(*pfrag));
+ if (found)
+ *found = 1;
+ break;
+ }
+
+ /* choose child */
+ nway = 1 << frag->split_by;
+ dout("choose_frag(%x) %x splits by %d (%d ways)\n", v, t,
+ frag->split_by, nway);
+ for (i = 0; i < nway; i++) {
+ n = ceph_frag_make_child(t, frag->split_by, i);
+ if (ceph_frag_contains_value(n, v)) {
+ t = n;
+ break;
+ }
+ }
+ BUG_ON(i == nway);
+ }
+ dout("choose_frag(%x) = %x\n", v, t);
+
+ mutex_unlock(&ci->i_fragtree_mutex);
+ return t;
+}
+
+/*
+ * Process dirfrag (delegation) info from the mds. Include leaf
+ * fragment in tree ONLY if ndist > 0. Otherwise, only
+ * branches/splits are included in i_fragtree)
+ */
+static int ceph_fill_dirfrag(struct inode *inode,
+ struct ceph_mds_reply_dirfrag *dirinfo)
+{
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ struct ceph_inode_frag *frag;
+ u32 id = le32_to_cpu(dirinfo->frag);
+ int mds = le32_to_cpu(dirinfo->auth);
+ int ndist = le32_to_cpu(dirinfo->ndist);
+ int i;
+ int err = 0;
+
+ mutex_lock(&ci->i_fragtree_mutex);
+ if (ndist == 0) {
+ /* no delegation info needed. */
+ frag = __ceph_find_frag(ci, id);
+ if (!frag)
+ goto out;
+ if (frag->split_by == 0) {
+ /* tree leaf, remove */
+ dout("fill_dirfrag removed %llx.%llx frag %x"
+ " (no ref)\n", ceph_vinop(inode), id);
+ rb_erase(&frag->node, &ci->i_fragtree);
+ kfree(frag);
+ } else {
+ /* tree branch, keep and clear */
+ dout("fill_dirfrag cleared %llx.%llx frag %x"
+ " referral\n", ceph_vinop(inode), id);
+ frag->mds = -1;
+ frag->ndist = 0;
+ }
+ goto out;
+ }
+
+
+ /* find/add this frag to store mds delegation info */
+ frag = __get_or_create_frag(ci, id);
+ if (IS_ERR(frag)) {
+ /* this is not the end of the world; we can continue
+ with bad/inaccurate delegation info */
+ pr_err("fill_dirfrag ENOMEM on mds ref %llx.%llx fg %x\n",
+ ceph_vinop(inode), le32_to_cpu(dirinfo->frag));
+ err = -ENOMEM;
+ goto out;
+ }
+
+ frag->mds = mds;
+ frag->ndist = min_t(u32, ndist, CEPH_MAX_DIRFRAG_REP);
+ for (i = 0; i < frag->ndist; i++)
+ frag->dist[i] = le32_to_cpu(dirinfo->dist[i]);
+ dout("fill_dirfrag %llx.%llx frag %x ndist=%d\n",
+ ceph_vinop(inode), frag->frag, frag->ndist);
+
+out:
+ mutex_unlock(&ci->i_fragtree_mutex);
+ return err;
+}
+
+
+/*
+ * initialize a newly allocated inode.
+ */
+struct inode *ceph_alloc_inode(struct super_block *sb)
+{
+ struct ceph_inode_info *ci;
+ int i;
+
+ ci = kmem_cache_alloc(ceph_inode_cachep, GFP_NOFS);
+ if (!ci)
+ return NULL;
+
+ dout("alloc_inode %p\n", &ci->vfs_inode);
+
+ ci->i_version = 0;
+ ci->i_time_warp_seq = 0;
+ ci->i_ceph_flags = 0;
+ ci->i_release_count = 0;
+ ci->i_symlink = NULL;
+
+ ci->i_fragtree = RB_ROOT;
+ mutex_init(&ci->i_fragtree_mutex);
+
+ ci->i_xattrs.blob = NULL;
+ ci->i_xattrs.prealloc_blob = NULL;
+ ci->i_xattrs.dirty = false;
+ ci->i_xattrs.index = RB_ROOT;
+ ci->i_xattrs.count = 0;
+ ci->i_xattrs.names_size = 0;
+ ci->i_xattrs.vals_size = 0;
+ ci->i_xattrs.version = 0;
+ ci->i_xattrs.index_version = 0;
+
+ ci->i_caps = RB_ROOT;
+ ci->i_auth_cap = NULL;
+ ci->i_dirty_caps = 0;
+ ci->i_flushing_caps = 0;
+ INIT_LIST_HEAD(&ci->i_dirty_item);
+ INIT_LIST_HEAD(&ci->i_flushing_item);
+ ci->i_cap_flush_seq = 0;
+ ci->i_cap_flush_last_tid = 0;
+ memset(&ci->i_cap_flush_tid, 0, sizeof(ci->i_cap_flush_tid));
+ init_waitqueue_head(&ci->i_cap_wq);
+ ci->i_hold_caps_min = 0;
+ ci->i_hold_caps_max = 0;
+ INIT_LIST_HEAD(&ci->i_cap_delay_list);
+ ci->i_cap_exporting_mds = 0;
+ ci->i_cap_exporting_mseq = 0;
+ ci->i_cap_exporting_issued = 0;
+ INIT_LIST_HEAD(&ci->i_cap_snaps);
+ ci->i_head_snapc = NULL;
+ ci->i_snap_caps = 0;
+
+ for (i = 0; i < CEPH_FILE_MODE_NUM; i++)
+ ci->i_nr_by_mode[i] = 0;
+
+ ci->i_truncate_seq = 0;
+ ci->i_truncate_size = 0;
+ ci->i_truncate_pending = 0;
+
+ ci->i_max_size = 0;
+ ci->i_reported_size = 0;
+ ci->i_wanted_max_size = 0;
+ ci->i_requested_max_size = 0;
+
+ ci->i_pin_ref = 0;
+ ci->i_rd_ref = 0;
+ ci->i_rdcache_ref = 0;
+ ci->i_wr_ref = 0;
+ ci->i_wrbuffer_ref = 0;
+ ci->i_wrbuffer_ref_head = 0;
+ ci->i_shared_gen = 0;
+ ci->i_rdcache_gen = 0;
+ ci->i_rdcache_revoking = 0;
+
+ INIT_LIST_HEAD(&ci->i_unsafe_writes);
+ INIT_LIST_HEAD(&ci->i_unsafe_dirops);
+ spin_lock_init(&ci->i_unsafe_lock);
+
+ ci->i_snap_realm = NULL;
+ INIT_LIST_HEAD(&ci->i_snap_realm_item);
+ INIT_LIST_HEAD(&ci->i_snap_flush_item);
+
+ INIT_WORK(&ci->i_wb_work, ceph_writeback_work);
+ INIT_WORK(&ci->i_pg_inv_work, ceph_invalidate_work);
+
+ INIT_WORK(&ci->i_vmtruncate_work, ceph_vmtruncate_work);
+
+ return &ci->vfs_inode;
+}
+
+void ceph_destroy_inode(struct inode *inode)
+{
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ struct ceph_inode_frag *frag;
+ struct rb_node *n;
+
+ dout("destroy_inode %p ino %llx.%llx\n", inode, ceph_vinop(inode));
+
+ ceph_queue_caps_release(inode);
+
+ /*
+ * we may still have a snap_realm reference if there are stray
+ * caps in i_cap_exporting_issued or i_snap_caps.
+ */
+ if (ci->i_snap_realm) {
+ struct ceph_mds_client *mdsc =
+ &ceph_client(ci->vfs_inode.i_sb)->mdsc;
+ struct ceph_snap_realm *realm = ci->i_snap_realm;
+
+ dout(" dropping residual ref to snap realm %p\n", realm);
+ spin_lock(&realm->inodes_with_caps_lock);
+ list_del_init(&ci->i_snap_realm_item);
+ spin_unlock(&realm->inodes_with_caps_lock);
+ ceph_put_snap_realm(mdsc, realm);
+ }
+
+ kfree(ci->i_symlink);
+ while ((n = rb_first(&ci->i_fragtree)) != NULL) {
+ frag = rb_entry(n, struct ceph_inode_frag, node);
+ rb_erase(n, &ci->i_fragtree);
+ kfree(frag);
+ }
+
+ __ceph_destroy_xattrs(ci);
+ if (ci->i_xattrs.blob)
+ ceph_buffer_put(ci->i_xattrs.blob);
+ if (ci->i_xattrs.prealloc_blob)
+ ceph_buffer_put(ci->i_xattrs.prealloc_blob);
+
+ kmem_cache_free(ceph_inode_cachep, ci);
+}
+
+
+/*
+ * Helpers to fill in size, ctime, mtime, and atime. We have to be
+ * careful because either the client or MDS may have more up to date
+ * info, depending on which capabilities are held, and whether
+ * time_warp_seq or truncate_seq have increased. (Ordinarily, mtime
+ * and size are monotonically increasing, except when utimes() or
+ * truncate() increments the corresponding _seq values.)
+ */
+int ceph_fill_file_size(struct inode *inode, int issued,
+ u32 truncate_seq, u64 truncate_size, u64 size)
+{
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ int queue_trunc = 0;
+
+ if (ceph_seq_cmp(truncate_seq, ci->i_truncate_seq) > 0 ||
+ (truncate_seq == ci->i_truncate_seq && size > inode->i_size)) {
+ dout("size %lld -> %llu\n", inode->i_size, size);
+ inode->i_size = size;
+ inode->i_blocks = (size + (1<<9) - 1) >> 9;
+ ci->i_reported_size = size;
+ if (truncate_seq != ci->i_truncate_seq) {
+ dout("truncate_seq %u -> %u\n",
+ ci->i_truncate_seq, truncate_seq);
+ ci->i_truncate_seq = truncate_seq;
+ /*
+ * If we hold relevant caps, or in the case where we're
+ * not the only client referencing this file and we
+ * don't hold those caps, then we need to check whether
+ * the file is either opened or mmaped
+ */
+ if ((issued & (CEPH_CAP_FILE_CACHE|CEPH_CAP_FILE_RD|
+ CEPH_CAP_FILE_WR|CEPH_CAP_FILE_BUFFER|
+ CEPH_CAP_FILE_EXCL)) ||
+ mapping_mapped(inode->i_mapping) ||
+ __ceph_caps_file_wanted(ci)) {
+ ci->i_truncate_pending++;
+ queue_trunc = 1;
+ }
+ }
+ }
+ if (ceph_seq_cmp(truncate_seq, ci->i_truncate_seq) >= 0 &&
+ ci->i_truncate_size != truncate_size) {
+ dout("truncate_size %lld -> %llu\n", ci->i_truncate_size,
+ truncate_size);
+ ci->i_truncate_size = truncate_size;
+ }
+ return queue_trunc;
+}
+
+void ceph_fill_file_time(struct inode *inode, int issued,
+ u64 time_warp_seq, struct timespec *ctime,
+ struct timespec *mtime, struct timespec *atime)
+{
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ int warn = 0;
+
+ if (issued & (CEPH_CAP_FILE_EXCL|
+ CEPH_CAP_FILE_WR|
+ CEPH_CAP_FILE_BUFFER)) {
+ if (timespec_compare(ctime, &inode->i_ctime) > 0) {
+ dout("ctime %ld.%09ld -> %ld.%09ld inc w/ cap\n",
+ inode->i_ctime.tv_sec, inode->i_ctime.tv_nsec,
+ ctime->tv_sec, ctime->tv_nsec);
+ inode->i_ctime = *ctime;
+ }
+ if (ceph_seq_cmp(time_warp_seq, ci->i_time_warp_seq) > 0) {
+ /* the MDS did a utimes() */
+ dout("mtime %ld.%09ld -> %ld.%09ld "
+ "tw %d -> %d\n",
+ inode->i_mtime.tv_sec, inode->i_mtime.tv_nsec,
+ mtime->tv_sec, mtime->tv_nsec,
+ ci->i_time_warp_seq, (int)time_warp_seq);
+
+ inode->i_mtime = *mtime;
+ inode->i_atime = *atime;
+ ci->i_time_warp_seq = time_warp_seq;
+ } else if (time_warp_seq == ci->i_time_warp_seq) {
+ /* nobody did utimes(); take the max */
+ if (timespec_compare(mtime, &inode->i_mtime) > 0) {
+ dout("mtime %ld.%09ld -> %ld.%09ld inc\n",
+ inode->i_mtime.tv_sec,
+ inode->i_mtime.tv_nsec,
+ mtime->tv_sec, mtime->tv_nsec);
+ inode->i_mtime = *mtime;
+ }
+ if (timespec_compare(atime, &inode->i_atime) > 0) {
+ dout("atime %ld.%09ld -> %ld.%09ld inc\n",
+ inode->i_atime.tv_sec,
+ inode->i_atime.tv_nsec,
+ atime->tv_sec, atime->tv_nsec);
+ inode->i_atime = *atime;
+ }
+ } else if (issued & CEPH_CAP_FILE_EXCL) {
+ /* we did a utimes(); ignore mds values */
+ } else {
+ warn = 1;
+ }
+ } else {
+ /* we have no write caps; whatever the MDS says is true */
+ if (ceph_seq_cmp(time_warp_seq, ci->i_time_warp_seq) >= 0) {
+ inode->i_ctime = *ctime;
+ inode->i_mtime = *mtime;
+ inode->i_atime = *atime;
+ ci->i_time_warp_seq = time_warp_seq;
+ } else {
+ warn = 1;
+ }
+ }
+ if (warn) /* time_warp_seq shouldn't go backwards */
+ dout("%p mds time_warp_seq %llu < %u\n",
+ inode, time_warp_seq, ci->i_time_warp_seq);
+}
+
+/*
+ * Populate an inode based on info from mds. May be called on new or
+ * existing inodes.
+ */
+static int fill_inode(struct inode *inode,
+ struct ceph_mds_reply_info_in *iinfo,
+ struct ceph_mds_reply_dirfrag *dirinfo,
+ struct ceph_mds_session *session,
+ unsigned long ttl_from, int cap_fmode,
+ struct ceph_cap_reservation *caps_reservation)
+{
+ struct ceph_mds_reply_inode *info = iinfo->in;
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ int i;
+ int issued, implemented;
+ struct timespec mtime, atime, ctime;
+ u32 nsplits;
+ struct ceph_buffer *xattr_blob = NULL;
+ int err = 0;
+ int queue_trunc = 0;
+
+ dout("fill_inode %p ino %llx.%llx v %llu had %llu\n",
+ inode, ceph_vinop(inode), le64_to_cpu(info->version),
+ ci->i_version);
+
+ /*
+ * prealloc xattr data, if it looks like we'll need it. only
+ * if len > 4 (meaning there are actually xattrs; the first 4
+ * bytes are the xattr count).
+ */
+ if (iinfo->xattr_len > 4) {
+ xattr_blob = ceph_buffer_new(iinfo->xattr_len, GFP_NOFS);
+ if (!xattr_blob)
+ pr_err("fill_inode ENOMEM xattr blob %d bytes\n",
+ iinfo->xattr_len);
+ }
+
+ spin_lock(&inode->i_lock);
+
+ /*
+ * provided version will be odd if inode value is projected,
+ * even if stable. skip the update if we have a newer info
+ * (e.g., due to inode info racing form multiple MDSs), or if
+ * we are getting projected (unstable) inode info.
+ */
+ if (le64_to_cpu(info->version) > 0 &&
+ (ci->i_version & ~1) > le64_to_cpu(info->version))
+ goto no_change;
+
+ issued = __ceph_caps_issued(ci, &implemented);
+ issued |= implemented | __ceph_caps_dirty(ci);
+
+ /* update inode */
+ ci->i_version = le64_to_cpu(info->version);
+ inode->i_version++;
+ inode->i_rdev = le32_to_cpu(info->rdev);
+
+ if ((issued & CEPH_CAP_AUTH_EXCL) == 0) {
+ inode->i_mode = le32_to_cpu(info->mode);
+ inode->i_uid = le32_to_cpu(info->uid);
+ inode->i_gid = le32_to_cpu(info->gid);
+ dout("%p mode 0%o uid.gid %d.%d\n", inode, inode->i_mode,
+ inode->i_uid, inode->i_gid);
+ }
+
+ if ((issued & CEPH_CAP_LINK_EXCL) == 0)
+ inode->i_nlink = le32_to_cpu(info->nlink);
+
+ /* be careful with mtime, atime, size */
+ ceph_decode_timespec(&atime, &info->atime);
+ ceph_decode_timespec(&mtime, &info->mtime);
+ ceph_decode_timespec(&ctime, &info->ctime);
+ queue_trunc = ceph_fill_file_size(inode, issued,
+ le32_to_cpu(info->truncate_seq),
+ le64_to_cpu(info->truncate_size),
+ le64_to_cpu(info->size));
+ ceph_fill_file_time(inode, issued,
+ le32_to_cpu(info->time_warp_seq),
+ &ctime, &mtime, &atime);
+
+ ci->i_max_size = le64_to_cpu(info->max_size);
+ ci->i_layout = info->layout;
+ inode->i_blkbits = fls(le32_to_cpu(info->layout.fl_stripe_unit)) - 1;
+
+ /* xattrs */
+ /* note that if i_xattrs.len <= 4, i_xattrs.data will still be NULL. */
+ if ((issued & CEPH_CAP_XATTR_EXCL) == 0 &&
+ le64_to_cpu(info->xattr_version) > ci->i_xattrs.version) {
+ if (ci->i_xattrs.blob)
+ ceph_buffer_put(ci->i_xattrs.blob);
+ ci->i_xattrs.blob = xattr_blob;
+ if (xattr_blob)
+ memcpy(ci->i_xattrs.blob->vec.iov_base,
+ iinfo->xattr_data, iinfo->xattr_len);
+ ci->i_xattrs.version = le64_to_cpu(info->xattr_version);
+ }
+
+ inode->i_mapping->a_ops = &ceph_aops;
+ inode->i_mapping->backing_dev_info =
+ &ceph_client(inode->i_sb)->backing_dev_info;
+
+ switch (inode->i_mode & S_IFMT) {
+ case S_IFIFO:
+ case S_IFBLK:
+ case S_IFCHR:
+ case S_IFSOCK:
+ init_special_inode(inode, inode->i_mode, inode->i_rdev);
+ inode->i_op = &ceph_file_iops;
+ break;
+ case S_IFREG:
+ inode->i_op = &ceph_file_iops;
+ inode->i_fop = &ceph_file_fops;
+ break;
+ case S_IFLNK:
+ inode->i_op = &ceph_symlink_iops;
+ if (!ci->i_symlink) {
+ int symlen = iinfo->symlink_len;
+ char *sym;
+
+ BUG_ON(symlen != inode->i_size);
+ spin_unlock(&inode->i_lock);
+
+ err = -ENOMEM;
+ sym = kmalloc(symlen+1, GFP_NOFS);
+ if (!sym)
+ goto out;
+ memcpy(sym, iinfo->symlink, symlen);
+ sym[symlen] = 0;
+
+ spin_lock(&inode->i_lock);
+ if (!ci->i_symlink)
+ ci->i_symlink = sym;
+ else
+ kfree(sym); /* lost a race */
+ }
+ break;
+ case S_IFDIR:
+ inode->i_op = &ceph_dir_iops;
+ inode->i_fop = &ceph_dir_fops;
+
+ ci->i_files = le64_to_cpu(info->files);
+ ci->i_subdirs = le64_to_cpu(info->subdirs);
+ ci->i_rbytes = le64_to_cpu(info->rbytes);
+ ci->i_rfiles = le64_to_cpu(info->rfiles);
+ ci->i_rsubdirs = le64_to_cpu(info->rsubdirs);
+ ceph_decode_timespec(&ci->i_rctime, &info->rctime);
+
+ /* set dir completion flag? */
+ if (ci->i_files == 0 && ci->i_subdirs == 0 &&
+ ceph_snap(inode) == CEPH_NOSNAP &&
+ (le32_to_cpu(info->cap.caps) & CEPH_CAP_FILE_SHARED)) {
+ dout(" marking %p complete (empty)\n", inode);
+ ci->i_ceph_flags |= CEPH_I_COMPLETE;
+ ci->i_max_offset = 2;
+ }
+
+ /* it may be better to set st_size in getattr instead? */
+ if (ceph_test_opt(ceph_client(inode->i_sb), RBYTES))
+ inode->i_size = ci->i_rbytes;
+ break;
+ default:
+ pr_err("fill_inode %llx.%llx BAD mode 0%o\n",
+ ceph_vinop(inode), inode->i_mode);
+ }
+
+no_change:
+ spin_unlock(&inode->i_lock);
+
+ /* queue truncate if we saw i_size decrease */
+ if (queue_trunc)
+ ceph_queue_vmtruncate(inode);
+
+ /* populate frag tree */
+ /* FIXME: move me up, if/when version reflects fragtree changes */
+ nsplits = le32_to_cpu(info->fragtree.nsplits);
+ mutex_lock(&ci->i_fragtree_mutex);
+ for (i = 0; i < nsplits; i++) {
+ u32 id = le32_to_cpu(info->fragtree.splits[i].frag);
+ struct ceph_inode_frag *frag = __get_or_create_frag(ci, id);
+
+ if (IS_ERR(frag))
+ continue;
+ frag->split_by = le32_to_cpu(info->fragtree.splits[i].by);
+ dout(" frag %x split by %d\n", frag->frag, frag->split_by);
+ }
+ mutex_unlock(&ci->i_fragtree_mutex);
+
+ /* were we issued a capability? */
+ if (info->cap.caps) {
+ if (ceph_snap(inode) == CEPH_NOSNAP) {
+ ceph_add_cap(inode, session,
+ le64_to_cpu(info->cap.cap_id),
+ cap_fmode,
+ le32_to_cpu(info->cap.caps),
+ le32_to_cpu(info->cap.wanted),
+ le32_to_cpu(info->cap.seq),
+ le32_to_cpu(info->cap.mseq),
+ le64_to_cpu(info->cap.realm),
+ info->cap.flags,
+ caps_reservation);
+ } else {
+ spin_lock(&inode->i_lock);
+ dout(" %p got snap_caps %s\n", inode,
+ ceph_cap_string(le32_to_cpu(info->cap.caps)));
+ ci->i_snap_caps |= le32_to_cpu(info->cap.caps);
+ if (cap_fmode >= 0)
+ __ceph_get_fmode(ci, cap_fmode);
+ spin_unlock(&inode->i_lock);
+ }
+ }
+
+ /* update delegation info? */
+ if (dirinfo)
+ ceph_fill_dirfrag(inode, dirinfo);
+
+ err = 0;
+
+out:
+ if (xattr_blob)
+ ceph_buffer_put(xattr_blob);
+ return err;
+}
+
+/*
+ * caller should hold session s_mutex.
+ */
+static void update_dentry_lease(struct dentry *dentry,
+ struct ceph_mds_reply_lease *lease,
+ struct ceph_mds_session *session,
+ unsigned long from_time)
+{
+ struct ceph_dentry_info *di = ceph_dentry(dentry);
+ long unsigned duration = le32_to_cpu(lease->duration_ms);
+ long unsigned ttl = from_time + (duration * HZ) / 1000;
+ long unsigned half_ttl = from_time + (duration * HZ / 2) / 1000;
+ struct inode *dir;
+
+ /* only track leases on regular dentries */
+ if (dentry->d_op != &ceph_dentry_ops)
+ return;
+
+ spin_lock(&dentry->d_lock);
+ dout("update_dentry_lease %p mask %d duration %lu ms ttl %lu\n",
+ dentry, le16_to_cpu(lease->mask), duration, ttl);
+
+ /* make lease_rdcache_gen match directory */
+ dir = dentry->d_parent->d_inode;
+ di->lease_shared_gen = ceph_inode(dir)->i_shared_gen;
+
+ if (lease->mask == 0)
+ goto out_unlock;
+
+ if (di->lease_gen == session->s_cap_gen &&
+ time_before(ttl, dentry->d_time))
+ goto out_unlock; /* we already have a newer lease. */
+
+ if (di->lease_session && di->lease_session != session)
+ goto out_unlock;
+
+ ceph_dentry_lru_touch(dentry);
+
+ if (!di->lease_session)
+ di->lease_session = ceph_get_mds_session(session);
+ di->lease_gen = session->s_cap_gen;
+ di->lease_seq = le32_to_cpu(lease->seq);
+ di->lease_renew_after = half_ttl;
+ di->lease_renew_from = 0;
+ dentry->d_time = ttl;
+out_unlock:
+ spin_unlock(&dentry->d_lock);
+ return;
+}
+
+/*
+ * splice a dentry to an inode.
+ * caller must hold directory i_mutex for this to be safe.
+ *
+ * we will only rehash the resulting dentry if @prehash is
+ * true; @prehash will be set to false (for the benefit of
+ * the caller) if we fail.
+ */
+static struct dentry *splice_dentry(struct dentry *dn, struct inode *in,
+ bool *prehash)
+{
+ struct dentry *realdn;
+
+ /* dn must be unhashed */
+ if (!d_unhashed(dn))
+ d_drop(dn);
+ realdn = d_materialise_unique(dn, in);
+ if (IS_ERR(realdn)) {
+ pr_err("splice_dentry error %p inode %p ino %llx.%llx\n",
+ dn, in, ceph_vinop(in));
+ if (prehash)
+ *prehash = false; /* don't rehash on error */
+ dn = realdn; /* note realdn contains the error */
+ goto out;
+ } else if (realdn) {
+ dout("dn %p (%d) spliced with %p (%d) "
+ "inode %p ino %llx.%llx\n",
+ dn, atomic_read(&dn->d_count),
+ realdn, atomic_read(&realdn->d_count),
+ realdn->d_inode, ceph_vinop(realdn->d_inode));
+ dput(dn);
+ dn = realdn;
+ } else {
+ BUG_ON(!ceph_dentry(dn));
+
+ dout("dn %p attached to %p ino %llx.%llx\n",
+ dn, dn->d_inode, ceph_vinop(dn->d_inode));
+ }
+ if ((!prehash || *prehash) && d_unhashed(dn))
+ d_rehash(dn);
+out:
+ return dn;
+}
+
+/*
+ * Set dentry's directory position based on the current dir's max, and
+ * order it in d_subdirs, so that dcache_readdir behaves.
+ */
+static void ceph_set_dentry_offset(struct dentry *dn)
+{
+ struct dentry *dir = dn->d_parent;
+ struct inode *inode = dn->d_parent->d_inode;
+ struct ceph_dentry_info *di;
+
+ BUG_ON(!inode);
+
+ di = ceph_dentry(dn);
+
+ spin_lock(&inode->i_lock);
+ di->offset = ceph_inode(inode)->i_max_offset++;
+ spin_unlock(&inode->i_lock);
+
+ spin_lock(&dcache_lock);
+ spin_lock(&dn->d_lock);
+ list_move_tail(&dir->d_subdirs, &dn->d_u.d_child);
+ dout("set_dentry_offset %p %lld (%p %p)\n", dn, di->offset,
+ dn->d_u.d_child.prev, dn->d_u.d_child.next);
+ spin_unlock(&dn->d_lock);
+ spin_unlock(&dcache_lock);
+}
+
+/*
+ * Incorporate results into the local cache. This is either just
+ * one inode, or a directory, dentry, and possibly linked-to inode (e.g.,
+ * after a lookup).
+ *
+ * A reply may contain
+ * a directory inode along with a dentry.
+ * and/or a target inode
+ *
+ * Called with snap_rwsem (read).
+ */
+int ceph_fill_trace(struct super_block *sb, struct ceph_mds_request *req,
+ struct ceph_mds_session *session)
+{
+ struct ceph_mds_reply_info_parsed *rinfo = &req->r_reply_info;
+ struct inode *in = NULL;
+ struct ceph_mds_reply_inode *ininfo;
+ struct ceph_vino vino;
+ int i = 0;
+ int err = 0;
+
+ dout("fill_trace %p is_dentry %d is_target %d\n", req,
+ rinfo->head->is_dentry, rinfo->head->is_target);
+
+#if 0
+ /*
+ * Debugging hook:
+ *
+ * If we resend completed ops to a recovering mds, we get no
+ * trace. Since that is very rare, pretend this is the case
+ * to ensure the 'no trace' handlers in the callers behave.
+ *
+ * Fill in inodes unconditionally to avoid breaking cap
+ * invariants.
+ */
+ if (rinfo->head->op & CEPH_MDS_OP_WRITE) {
+ pr_info("fill_trace faking empty trace on %lld %s\n",
+ req->r_tid, ceph_mds_op_name(rinfo->head->op));
+ if (rinfo->head->is_dentry) {
+ rinfo->head->is_dentry = 0;
+ err = fill_inode(req->r_locked_dir,
+ &rinfo->diri, rinfo->dirfrag,
+ session, req->r_request_started, -1);
+ }
+ if (rinfo->head->is_target) {
+ rinfo->head->is_target = 0;
+ ininfo = rinfo->targeti.in;
+ vino.ino = le64_to_cpu(ininfo->ino);
+ vino.snap = le64_to_cpu(ininfo->snapid);
+ in = ceph_get_inode(sb, vino);
+ err = fill_inode(in, &rinfo->targeti, NULL,
+ session, req->r_request_started,
+ req->r_fmode);
+ iput(in);
+ }
+ }
+#endif
+
+ if (!rinfo->head->is_target && !rinfo->head->is_dentry) {
+ dout("fill_trace reply is empty!\n");
+ if (rinfo->head->result == 0 && req->r_locked_dir) {
+ struct ceph_inode_info *ci =
+ ceph_inode(req->r_locked_dir);
+ dout(" clearing %p complete (empty trace)\n",
+ req->r_locked_dir);
+ ci->i_ceph_flags &= ~CEPH_I_COMPLETE;
+ ci->i_release_count++;
+ }
+ return 0;
+ }
+
+ if (rinfo->head->is_dentry) {
+ struct inode *dir = req->r_locked_dir;
+
+ err = fill_inode(dir, &rinfo->diri, rinfo->dirfrag,
+ session, req->r_request_started, -1,
+ &req->r_caps_reservation);
+ if (err < 0)
+ return err;
+ }
+
+ if (rinfo->head->is_dentry && !req->r_aborted) {
+ /*
+ * lookup link rename : null -> possibly existing inode
+ * mknod symlink mkdir : null -> new inode
+ * unlink : linked -> null
+ */
+ struct inode *dir = req->r_locked_dir;
+ struct dentry *dn = req->r_dentry;
+ bool have_dir_cap, have_lease;
+
+ BUG_ON(!dn);
+ BUG_ON(!dir);
+ BUG_ON(dn->d_parent->d_inode != dir);
+ BUG_ON(ceph_ino(dir) !=
+ le64_to_cpu(rinfo->diri.in->ino));
+ BUG_ON(ceph_snap(dir) !=
+ le64_to_cpu(rinfo->diri.in->snapid));
+
+ /* do we have a lease on the whole dir? */
+ have_dir_cap =
+ (le32_to_cpu(rinfo->diri.in->cap.caps) &
+ CEPH_CAP_FILE_SHARED);
+
+ /* do we have a dn lease? */
+ have_lease = have_dir_cap ||
+ (le16_to_cpu(rinfo->dlease->mask) &
+ CEPH_LOCK_DN);
+
+ if (!have_lease)
+ dout("fill_trace no dentry lease or dir cap\n");
+
+ /* rename? */
+ if (req->r_old_dentry && req->r_op == CEPH_MDS_OP_RENAME) {
+ dout(" src %p '%.*s' dst %p '%.*s'\n",
+ req->r_old_dentry,
+ req->r_old_dentry->d_name.len,
+ req->r_old_dentry->d_name.name,
+ dn, dn->d_name.len, dn->d_name.name);
+ dout("fill_trace doing d_move %p -> %p\n",
+ req->r_old_dentry, dn);
+ d_move(req->r_old_dentry, dn);
+ dout(" src %p '%.*s' dst %p '%.*s'\n",
+ req->r_old_dentry,
+ req->r_old_dentry->d_name.len,
+ req->r_old_dentry->d_name.name,
+ dn, dn->d_name.len, dn->d_name.name);
+ /* ensure target dentry is invalidated, despite
+ rehashing bug in vfs_rename_dir */
+ dn->d_time = jiffies;
+ ceph_dentry(dn)->lease_shared_gen = 0;
+ /* take overwritten dentry's readdir offset */
+ ceph_dentry(req->r_old_dentry)->offset =
+ ceph_dentry(dn)->offset;
+ dn = req->r_old_dentry; /* use old_dentry */
+ in = dn->d_inode;
+ }
+
+ /* null dentry? */
+ if (!rinfo->head->is_target) {
+ dout("fill_trace null dentry\n");
+ if (dn->d_inode) {
+ dout("d_delete %p\n", dn);
+ d_delete(dn);
+ } else {
+ dout("d_instantiate %p NULL\n", dn);
+ d_instantiate(dn, NULL);
+ if (have_lease && d_unhashed(dn))
+ d_rehash(dn);
+ update_dentry_lease(dn, rinfo->dlease,
+ session,
+ req->r_request_started);
+ }
+ goto done;
+ }
+
+ /* attach proper inode */
+ ininfo = rinfo->targeti.in;
+ vino.ino = le64_to_cpu(ininfo->ino);
+ vino.snap = le64_to_cpu(ininfo->snapid);
+ if (!dn->d_inode) {
+ in = ceph_get_inode(sb, vino);
+ if (IS_ERR(in)) {
+ pr_err("fill_trace bad get_inode "
+ "%llx.%llx\n", vino.ino, vino.snap);
+ err = PTR_ERR(in);
+ d_delete(dn);
+ goto done;
+ }
+ dn = splice_dentry(dn, in, &have_lease);
+ if (IS_ERR(dn)) {
+ err = PTR_ERR(dn);
+ goto done;
+ }
+ req->r_dentry = dn; /* may have spliced */
+ ceph_set_dentry_offset(dn);
+ igrab(in);
+ } else if (ceph_ino(in) == vino.ino &&
+ ceph_snap(in) == vino.snap) {
+ igrab(in);
+ } else {
+ dout(" %p links to %p %llx.%llx, not %llx.%llx\n",
+ dn, in, ceph_ino(in), ceph_snap(in),
+ vino.ino, vino.snap);
+ have_lease = false;
+ in = NULL;
+ }
+
+ if (have_lease)
+ update_dentry_lease(dn, rinfo->dlease, session,
+ req->r_request_started);
+ dout(" final dn %p\n", dn);
+ i++;
+ } else if (req->r_op == CEPH_MDS_OP_LOOKUPSNAP ||
+ req->r_op == CEPH_MDS_OP_MKSNAP) {
+ struct dentry *dn = req->r_dentry;
+
+ /* fill out a snapdir LOOKUPSNAP dentry */
+ BUG_ON(!dn);
+ BUG_ON(!req->r_locked_dir);
+ BUG_ON(ceph_snap(req->r_locked_dir) != CEPH_SNAPDIR);
+ ininfo = rinfo->targeti.in;
+ vino.ino = le64_to_cpu(ininfo->ino);
+ vino.snap = le64_to_cpu(ininfo->snapid);
+ in = ceph_get_inode(sb, vino);
+ if (IS_ERR(in)) {
+ pr_err("fill_inode get_inode badness %llx.%llx\n",
+ vino.ino, vino.snap);
+ err = PTR_ERR(in);
+ d_delete(dn);
+ goto done;
+ }
+ dout(" linking snapped dir %p to dn %p\n", in, dn);
+ dn = splice_dentry(dn, in, NULL);
+ if (IS_ERR(dn)) {
+ err = PTR_ERR(dn);
+ goto done;
+ }
+ ceph_set_dentry_offset(dn);
+ req->r_dentry = dn; /* may have spliced */
+ igrab(in);
+ rinfo->head->is_dentry = 1; /* fool notrace handlers */
+ }
+
+ if (rinfo->head->is_target) {
+ vino.ino = le64_to_cpu(rinfo->targeti.in->ino);
+ vino.snap = le64_to_cpu(rinfo->targeti.in->snapid);
+
+ if (in == NULL || ceph_ino(in) != vino.ino ||
+ ceph_snap(in) != vino.snap) {
+ in = ceph_get_inode(sb, vino);
+ if (IS_ERR(in)) {
+ err = PTR_ERR(in);
+ goto done;
+ }
+ }
+ req->r_target_inode = in;
+
+ err = fill_inode(in,
+ &rinfo->targeti, NULL,
+ session, req->r_request_started,
+ (le32_to_cpu(rinfo->head->result) == 0) ?
+ req->r_fmode : -1,
+ &req->r_caps_reservation);
+ if (err < 0) {
+ pr_err("fill_inode badness %p %llx.%llx\n",
+ in, ceph_vinop(in));
+ goto done;
+ }
+ }
+
+done:
+ dout("fill_trace done err=%d\n", err);
+ return err;
+}
+
+/*
+ * Prepopulate our cache with readdir results, leases, etc.
+ */
+int ceph_readdir_prepopulate(struct ceph_mds_request *req,
+ struct ceph_mds_session *session)
+{
+ struct dentry *parent = req->r_dentry;
+ struct ceph_mds_reply_info_parsed *rinfo = &req->r_reply_info;
+ struct qstr dname;
+ struct dentry *dn;
+ struct inode *in;
+ int err = 0, i;
+ struct inode *snapdir = NULL;
+ struct ceph_mds_request_head *rhead = req->r_request->front.iov_base;
+ u64 frag = le32_to_cpu(rhead->args.readdir.frag);
+ struct ceph_dentry_info *di;
+
+ if (le32_to_cpu(rinfo->head->op) == CEPH_MDS_OP_LSSNAP) {
+ snapdir = ceph_get_snapdir(parent->d_inode);
+ parent = d_find_alias(snapdir);
+ dout("readdir_prepopulate %d items under SNAPDIR dn %p\n",
+ rinfo->dir_nr, parent);
+ } else {
+ dout("readdir_prepopulate %d items under dn %p\n",
+ rinfo->dir_nr, parent);
+ if (rinfo->dir_dir)
+ ceph_fill_dirfrag(parent->d_inode, rinfo->dir_dir);
+ }
+
+ for (i = 0; i < rinfo->dir_nr; i++) {
+ struct ceph_vino vino;
+
+ dname.name = rinfo->dir_dname[i];
+ dname.len = rinfo->dir_dname_len[i];
+ dname.hash = full_name_hash(dname.name, dname.len);
+
+ vino.ino = le64_to_cpu(rinfo->dir_in[i].in->ino);
+ vino.snap = le64_to_cpu(rinfo->dir_in[i].in->snapid);
+
+retry_lookup:
+ dn = d_lookup(parent, &dname);
+ dout("d_lookup on parent=%p name=%.*s got %p\n",
+ parent, dname.len, dname.name, dn);
+
+ if (!dn) {
+ dn = d_alloc(parent, &dname);
+ dout("d_alloc %p '%.*s' = %p\n", parent,
+ dname.len, dname.name, dn);
+ if (dn == NULL) {
+ dout("d_alloc badness\n");
+ err = -ENOMEM;
+ goto out;
+ }
+ err = ceph_init_dentry(dn);
+ if (err < 0)
+ goto out;
+ } else if (dn->d_inode &&
+ (ceph_ino(dn->d_inode) != vino.ino ||
+ ceph_snap(dn->d_inode) != vino.snap)) {
+ dout(" dn %p points to wrong inode %p\n",
+ dn, dn->d_inode);
+ d_delete(dn);
+ dput(dn);
+ goto retry_lookup;
+ } else {
+ /* reorder parent's d_subdirs */
+ spin_lock(&dcache_lock);
+ spin_lock(&dn->d_lock);
+ list_move(&dn->d_u.d_child, &parent->d_subdirs);
+ spin_unlock(&dn->d_lock);
+ spin_unlock(&dcache_lock);
+ }
+
+ di = dn->d_fsdata;
+ di->offset = ceph_make_fpos(frag, i + req->r_readdir_offset);
+
+ /* inode */
+ if (dn->d_inode) {
+ in = dn->d_inode;
+ } else {
+ in = ceph_get_inode(parent->d_sb, vino);
+ if (in == NULL) {
+ dout("new_inode badness\n");
+ d_delete(dn);
+ dput(dn);
+ err = -ENOMEM;
+ goto out;
+ }
+ dn = splice_dentry(dn, in, NULL);
+ }
+
+ if (fill_inode(in, &rinfo->dir_in[i], NULL, session,
+ req->r_request_started, -1,
+ &req->r_caps_reservation) < 0) {
+ pr_err("fill_inode badness on %p\n", in);
+ dput(dn);
+ continue;
+ }
+ update_dentry_lease(dn, rinfo->dir_dlease[i],
+ req->r_session, req->r_request_started);
+ dput(dn);
+ }
+ req->r_did_prepopulate = true;
+
+out:
+ if (snapdir) {
+ iput(snapdir);
+ dput(parent);
+ }
+ dout("readdir_prepopulate done\n");
+ return err;
+}
+
+int ceph_inode_set_size(struct inode *inode, loff_t size)
+{
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ int ret = 0;
+
+ spin_lock(&inode->i_lock);
+ dout("set_size %p %llu -> %llu\n", inode, inode->i_size, size);
+ inode->i_size = size;
+ inode->i_blocks = (size + (1 << 9) - 1) >> 9;
+
+ /* tell the MDS if we are approaching max_size */
+ if ((size << 1) >= ci->i_max_size &&
+ (ci->i_reported_size << 1) < ci->i_max_size)
+ ret = 1;
+
+ spin_unlock(&inode->i_lock);
+ return ret;
+}
+
+/*
+ * Write back inode data in a worker thread. (This can't be done
+ * in the message handler context.)
+ */
+void ceph_queue_writeback(struct inode *inode)
+{
+ if (queue_work(ceph_inode_to_client(inode)->wb_wq,
+ &ceph_inode(inode)->i_wb_work)) {
+ dout("ceph_queue_writeback %p\n", inode);
+ igrab(inode);
+ } else {
+ dout("ceph_queue_writeback %p failed\n", inode);
+ }
+}
+
+static void ceph_writeback_work(struct work_struct *work)
+{
+ struct ceph_inode_info *ci = container_of(work, struct ceph_inode_info,
+ i_wb_work);
+ struct inode *inode = &ci->vfs_inode;
+
+ dout("writeback %p\n", inode);
+ filemap_fdatawrite(&inode->i_data);
+ iput(inode);
+}
+
+/*
+ * queue an async invalidation
+ */
+void ceph_queue_invalidate(struct inode *inode)
+{
+ if (queue_work(ceph_inode_to_client(inode)->pg_inv_wq,
+ &ceph_inode(inode)->i_pg_inv_work)) {
+ dout("ceph_queue_invalidate %p\n", inode);
+ igrab(inode);
+ } else {
+ dout("ceph_queue_invalidate %p failed\n", inode);
+ }
+}
+
+/*
+ * invalidate any pages that are not dirty or under writeback. this
+ * includes pages that are clean and mapped.
+ */
+static void ceph_invalidate_nondirty_pages(struct address_space *mapping)
+{
+ struct pagevec pvec;
+ pgoff_t next = 0;
+ int i;
+
+ pagevec_init(&pvec, 0);
+ while (pagevec_lookup(&pvec, mapping, next, PAGEVEC_SIZE)) {
+ for (i = 0; i < pagevec_count(&pvec); i++) {
+ struct page *page = pvec.pages[i];
+ pgoff_t index;
+ int skip_page =
+ (PageDirty(page) || PageWriteback(page));
+
+ if (!skip_page)
+ skip_page = !trylock_page(page);
+
+ /*
+ * We really shouldn't be looking at the ->index of an
+ * unlocked page. But we're not allowed to lock these
+ * pages. So we rely upon nobody altering the ->index
+ * of this (pinned-by-us) page.
+ */
+ index = page->index;
+ if (index > next)
+ next = index;
+ next++;
+
+ if (skip_page)
+ continue;
+
+ generic_error_remove_page(mapping, page);
+ unlock_page(page);
+ }
+ pagevec_release(&pvec);
+ cond_resched();
+ }
+}
+
+/*
+ * Invalidate inode pages in a worker thread. (This can't be done
+ * in the message handler context.)
+ */
+static void ceph_invalidate_work(struct work_struct *work)
+{
+ struct ceph_inode_info *ci = container_of(work, struct ceph_inode_info,
+ i_pg_inv_work);
+ struct inode *inode = &ci->vfs_inode;
+ u32 orig_gen;
+ int check = 0;
+
+ spin_lock(&inode->i_lock);
+ dout("invalidate_pages %p gen %d revoking %d\n", inode,
+ ci->i_rdcache_gen, ci->i_rdcache_revoking);
+ if (ci->i_rdcache_gen == 0 ||
+ ci->i_rdcache_revoking != ci->i_rdcache_gen) {
+ BUG_ON(ci->i_rdcache_revoking > ci->i_rdcache_gen);
+ /* nevermind! */
+ ci->i_rdcache_revoking = 0;
+ spin_unlock(&inode->i_lock);
+ goto out;
+ }
+ orig_gen = ci->i_rdcache_gen;
+ spin_unlock(&inode->i_lock);
+
+ ceph_invalidate_nondirty_pages(inode->i_mapping);
+
+ spin_lock(&inode->i_lock);
+ if (orig_gen == ci->i_rdcache_gen) {
+ dout("invalidate_pages %p gen %d successful\n", inode,
+ ci->i_rdcache_gen);
+ ci->i_rdcache_gen = 0;
+ ci->i_rdcache_revoking = 0;
+ check = 1;
+ } else {
+ dout("invalidate_pages %p gen %d raced, gen now %d\n",
+ inode, orig_gen, ci->i_rdcache_gen);
+ }
+ spin_unlock(&inode->i_lock);
+
+ if (check)
+ ceph_check_caps(ci, 0, NULL);
+out:
+ iput(inode);
+}
+
+
+/*
+ * called by trunc_wq; take i_mutex ourselves
+ *
+ * We also truncate in a separate thread as well.
+ */
+static void ceph_vmtruncate_work(struct work_struct *work)
+{
+ struct ceph_inode_info *ci = container_of(work, struct ceph_inode_info,
+ i_vmtruncate_work);
+ struct inode *inode = &ci->vfs_inode;
+
+ dout("vmtruncate_work %p\n", inode);
+ mutex_lock(&inode->i_mutex);
+ __ceph_do_pending_vmtruncate(inode);
+ mutex_unlock(&inode->i_mutex);
+ iput(inode);
+}
+
+/*
+ * Queue an async vmtruncate. If we fail to queue work, we will handle
+ * the truncation the next time we call __ceph_do_pending_vmtruncate.
+ */
+void ceph_queue_vmtruncate(struct inode *inode)
+{
+ struct ceph_inode_info *ci = ceph_inode(inode);
+
+ if (queue_work(ceph_client(inode->i_sb)->trunc_wq,
+ &ci->i_vmtruncate_work)) {
+ dout("ceph_queue_vmtruncate %p\n", inode);
+ igrab(inode);
+ } else {
+ dout("ceph_queue_vmtruncate %p failed, pending=%d\n",
+ inode, ci->i_truncate_pending);
+ }
+}
+
+/*
+ * called with i_mutex held.
+ *
+ * Make sure any pending truncation is applied before doing anything
+ * that may depend on it.
+ */
+void __ceph_do_pending_vmtruncate(struct inode *inode)
+{
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ u64 to;
+ int wrbuffer_refs, wake = 0;
+
+retry:
+ spin_lock(&inode->i_lock);
+ if (ci->i_truncate_pending == 0) {
+ dout("__do_pending_vmtruncate %p none pending\n", inode);
+ spin_unlock(&inode->i_lock);
+ return;
+ }
+
+ /*
+ * make sure any dirty snapped pages are flushed before we
+ * possibly truncate them.. so write AND block!
+ */
+ if (ci->i_wrbuffer_ref_head < ci->i_wrbuffer_ref) {
+ dout("__do_pending_vmtruncate %p flushing snaps first\n",
+ inode);
+ spin_unlock(&inode->i_lock);
+ filemap_write_and_wait_range(&inode->i_data, 0,
+ inode->i_sb->s_maxbytes);
+ goto retry;
+ }
+
+ to = ci->i_truncate_size;
+ wrbuffer_refs = ci->i_wrbuffer_ref;
+ dout("__do_pending_vmtruncate %p (%d) to %lld\n", inode,
+ ci->i_truncate_pending, to);
+ spin_unlock(&inode->i_lock);
+
+ truncate_inode_pages(inode->i_mapping, to);
+
+ spin_lock(&inode->i_lock);
+ ci->i_truncate_pending--;
+ if (ci->i_truncate_pending == 0)
+ wake = 1;
+ spin_unlock(&inode->i_lock);
+
+ if (wrbuffer_refs == 0)
+ ceph_check_caps(ci, CHECK_CAPS_AUTHONLY, NULL);
+ if (wake)
+ wake_up(&ci->i_cap_wq);
+}
+
+
+/*
+ * symlinks
+ */
+static void *ceph_sym_follow_link(struct dentry *dentry, struct nameidata *nd)
+{
+ struct ceph_inode_info *ci = ceph_inode(dentry->d_inode);
+ nd_set_link(nd, ci->i_symlink);
+ return NULL;
+}
+
+static const struct inode_operations ceph_symlink_iops = {
+ .readlink = generic_readlink,
+ .follow_link = ceph_sym_follow_link,
+};
+
+/*
+ * setattr
+ */
+int ceph_setattr(struct dentry *dentry, struct iattr *attr)
+{
+ struct inode *inode = dentry->d_inode;
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ struct inode *parent_inode = dentry->d_parent->d_inode;
+ const unsigned int ia_valid = attr->ia_valid;
+ struct ceph_mds_request *req;
+ struct ceph_mds_client *mdsc = &ceph_client(dentry->d_sb)->mdsc;
+ int issued;
+ int release = 0, dirtied = 0;
+ int mask = 0;
+ int err = 0;
+
+ if (ceph_snap(inode) != CEPH_NOSNAP)
+ return -EROFS;
+
+ __ceph_do_pending_vmtruncate(inode);
+
+ err = inode_change_ok(inode, attr);
+ if (err != 0)
+ return err;
+
+ req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_SETATTR,
+ USE_AUTH_MDS);
+ if (IS_ERR(req))
+ return PTR_ERR(req);
+
+ spin_lock(&inode->i_lock);
+ issued = __ceph_caps_issued(ci, NULL);
+ dout("setattr %p issued %s\n", inode, ceph_cap_string(issued));
+
+ if (ia_valid & ATTR_UID) {
+ dout("setattr %p uid %d -> %d\n", inode,
+ inode->i_uid, attr->ia_uid);
+ if (issued & CEPH_CAP_AUTH_EXCL) {
+ inode->i_uid = attr->ia_uid;
+ dirtied |= CEPH_CAP_AUTH_EXCL;
+ } else if ((issued & CEPH_CAP_AUTH_SHARED) == 0 ||
+ attr->ia_uid != inode->i_uid) {
+ req->r_args.setattr.uid = cpu_to_le32(attr->ia_uid);
+ mask |= CEPH_SETATTR_UID;
+ release |= CEPH_CAP_AUTH_SHARED;
+ }
+ }
+ if (ia_valid & ATTR_GID) {
+ dout("setattr %p gid %d -> %d\n", inode,
+ inode->i_gid, attr->ia_gid);
+ if (issued & CEPH_CAP_AUTH_EXCL) {
+ inode->i_gid = attr->ia_gid;
+ dirtied |= CEPH_CAP_AUTH_EXCL;
+ } else if ((issued & CEPH_CAP_AUTH_SHARED) == 0 ||
+ attr->ia_gid != inode->i_gid) {
+ req->r_args.setattr.gid = cpu_to_le32(attr->ia_gid);
+ mask |= CEPH_SETATTR_GID;
+ release |= CEPH_CAP_AUTH_SHARED;
+ }
+ }
+ if (ia_valid & ATTR_MODE) {
+ dout("setattr %p mode 0%o -> 0%o\n", inode, inode->i_mode,
+ attr->ia_mode);
+ if (issued & CEPH_CAP_AUTH_EXCL) {
+ inode->i_mode = attr->ia_mode;
+ dirtied |= CEPH_CAP_AUTH_EXCL;
+ } else if ((issued & CEPH_CAP_AUTH_SHARED) == 0 ||
+ attr->ia_mode != inode->i_mode) {
+ req->r_args.setattr.mode = cpu_to_le32(attr->ia_mode);
+ mask |= CEPH_SETATTR_MODE;
+ release |= CEPH_CAP_AUTH_SHARED;
+ }
+ }
+
+ if (ia_valid & ATTR_ATIME) {
+ dout("setattr %p atime %ld.%ld -> %ld.%ld\n", inode,
+ inode->i_atime.tv_sec, inode->i_atime.tv_nsec,
+ attr->ia_atime.tv_sec, attr->ia_atime.tv_nsec);
+ if (issued & CEPH_CAP_FILE_EXCL) {
+ ci->i_time_warp_seq++;
+ inode->i_atime = attr->ia_atime;
+ dirtied |= CEPH_CAP_FILE_EXCL;
+ } else if ((issued & CEPH_CAP_FILE_WR) &&
+ timespec_compare(&inode->i_atime,
+ &attr->ia_atime) < 0) {
+ inode->i_atime = attr->ia_atime;
+ dirtied |= CEPH_CAP_FILE_WR;
+ } else if ((issued & CEPH_CAP_FILE_SHARED) == 0 ||
+ !timespec_equal(&inode->i_atime, &attr->ia_atime)) {
+ ceph_encode_timespec(&req->r_args.setattr.atime,
+ &attr->ia_atime);
+ mask |= CEPH_SETATTR_ATIME;
+ release |= CEPH_CAP_FILE_CACHE | CEPH_CAP_FILE_RD |
+ CEPH_CAP_FILE_WR;
+ }
+ }
+ if (ia_valid & ATTR_MTIME) {
+ dout("setattr %p mtime %ld.%ld -> %ld.%ld\n", inode,
+ inode->i_mtime.tv_sec, inode->i_mtime.tv_nsec,
+ attr->ia_mtime.tv_sec, attr->ia_mtime.tv_nsec);
+ if (issued & CEPH_CAP_FILE_EXCL) {
+ ci->i_time_warp_seq++;
+ inode->i_mtime = attr->ia_mtime;
+ dirtied |= CEPH_CAP_FILE_EXCL;
+ } else if ((issued & CEPH_CAP_FILE_WR) &&
+ timespec_compare(&inode->i_mtime,
+ &attr->ia_mtime) < 0) {
+ inode->i_mtime = attr->ia_mtime;
+ dirtied |= CEPH_CAP_FILE_WR;
+ } else if ((issued & CEPH_CAP_FILE_SHARED) == 0 ||
+ !timespec_equal(&inode->i_mtime, &attr->ia_mtime)) {
+ ceph_encode_timespec(&req->r_args.setattr.mtime,
+ &attr->ia_mtime);
+ mask |= CEPH_SETATTR_MTIME;
+ release |= CEPH_CAP_FILE_SHARED | CEPH_CAP_FILE_RD |
+ CEPH_CAP_FILE_WR;
+ }
+ }
+ if (ia_valid & ATTR_SIZE) {
+ dout("setattr %p size %lld -> %lld\n", inode,
+ inode->i_size, attr->ia_size);
+ if (attr->ia_size > inode->i_sb->s_maxbytes) {
+ err = -EINVAL;
+ goto out;
+ }
+ if ((issued & CEPH_CAP_FILE_EXCL) &&
+ attr->ia_size > inode->i_size) {
+ inode->i_size = attr->ia_size;
+ inode->i_blocks =
+ (attr->ia_size + (1 << 9) - 1) >> 9;
+ inode->i_ctime = attr->ia_ctime;
+ ci->i_reported_size = attr->ia_size;
+ dirtied |= CEPH_CAP_FILE_EXCL;
+ } else if ((issued & CEPH_CAP_FILE_SHARED) == 0 ||
+ attr->ia_size != inode->i_size) {
+ req->r_args.setattr.size = cpu_to_le64(attr->ia_size);
+ req->r_args.setattr.old_size =
+ cpu_to_le64(inode->i_size);
+ mask |= CEPH_SETATTR_SIZE;
+ release |= CEPH_CAP_FILE_SHARED | CEPH_CAP_FILE_RD |
+ CEPH_CAP_FILE_WR;
+ }
+ }
+
+ /* these do nothing */
+ if (ia_valid & ATTR_CTIME) {
+ bool only = (ia_valid & (ATTR_SIZE|ATTR_MTIME|ATTR_ATIME|
+ ATTR_MODE|ATTR_UID|ATTR_GID)) == 0;
+ dout("setattr %p ctime %ld.%ld -> %ld.%ld (%s)\n", inode,
+ inode->i_ctime.tv_sec, inode->i_ctime.tv_nsec,
+ attr->ia_ctime.tv_sec, attr->ia_ctime.tv_nsec,
+ only ? "ctime only" : "ignored");
+ inode->i_ctime = attr->ia_ctime;
+ if (only) {
+ /*
+ * if kernel wants to dirty ctime but nothing else,
+ * we need to choose a cap to dirty under, or do
+ * a almost-no-op setattr
+ */
+ if (issued & CEPH_CAP_AUTH_EXCL)
+ dirtied |= CEPH_CAP_AUTH_EXCL;
+ else if (issued & CEPH_CAP_FILE_EXCL)
+ dirtied |= CEPH_CAP_FILE_EXCL;
+ else if (issued & CEPH_CAP_XATTR_EXCL)
+ dirtied |= CEPH_CAP_XATTR_EXCL;
+ else
+ mask |= CEPH_SETATTR_CTIME;
+ }
+ }
+ if (ia_valid & ATTR_FILE)
+ dout("setattr %p ATTR_FILE ... hrm!\n", inode);
+
+ if (dirtied) {
+ __ceph_mark_dirty_caps(ci, dirtied);
+ inode->i_ctime = CURRENT_TIME;
+ }
+
+ release &= issued;
+ spin_unlock(&inode->i_lock);
+
+ if (mask) {
+ req->r_inode = igrab(inode);
+ req->r_inode_drop = release;
+ req->r_args.setattr.mask = cpu_to_le32(mask);
+ req->r_num_caps = 1;
+ err = ceph_mdsc_do_request(mdsc, parent_inode, req);
+ }
+ dout("setattr %p result=%d (%s locally, %d remote)\n", inode, err,
+ ceph_cap_string(dirtied), mask);
+
+ ceph_mdsc_put_request(req);
+ __ceph_do_pending_vmtruncate(inode);
+ return err;
+out:
+ spin_unlock(&inode->i_lock);
+ ceph_mdsc_put_request(req);
+ return err;
+}
+
+/*
+ * Verify that we have a lease on the given mask. If not,
+ * do a getattr against an mds.
+ */
+int ceph_do_getattr(struct inode *inode, int mask)
+{
+ struct ceph_client *client = ceph_sb_to_client(inode->i_sb);
+ struct ceph_mds_client *mdsc = &client->mdsc;
+ struct ceph_mds_request *req;
+ int err;
+
+ if (ceph_snap(inode) == CEPH_SNAPDIR) {
+ dout("do_getattr inode %p SNAPDIR\n", inode);
+ return 0;
+ }
+
+ dout("do_getattr inode %p mask %s\n", inode, ceph_cap_string(mask));
+ if (ceph_caps_issued_mask(ceph_inode(inode), mask, 1))
+ return 0;
+
+ req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_GETATTR, USE_ANY_MDS);
+ if (IS_ERR(req))
+ return PTR_ERR(req);
+ req->r_inode = igrab(inode);
+ req->r_num_caps = 1;
+ req->r_args.getattr.mask = cpu_to_le32(mask);
+ err = ceph_mdsc_do_request(mdsc, NULL, req);
+ ceph_mdsc_put_request(req);
+ dout("do_getattr result=%d\n", err);
+ return err;
+}
+
+
+/*
+ * Check inode permissions. We verify we have a valid value for
+ * the AUTH cap, then call the generic handler.
+ */
+int ceph_permission(struct inode *inode, int mask)
+{
+ int err = ceph_do_getattr(inode, CEPH_CAP_AUTH_SHARED);
+
+ if (!err)
+ err = generic_permission(inode, mask, NULL);
+ return err;
+}
+
+/*
+ * Get all attributes. Hopefully somedata we'll have a statlite()
+ * and can limit the fields we require to be accurate.
+ */
+int ceph_getattr(struct vfsmount *mnt, struct dentry *dentry,
+ struct kstat *stat)
+{
+ struct inode *inode = dentry->d_inode;
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ int err;
+
+ err = ceph_do_getattr(inode, CEPH_STAT_CAP_INODE_ALL);
+ if (!err) {
+ generic_fillattr(inode, stat);
+ stat->ino = inode->i_ino;
+ if (ceph_snap(inode) != CEPH_NOSNAP)
+ stat->dev = ceph_snap(inode);
+ else
+ stat->dev = 0;
+ if (S_ISDIR(inode->i_mode)) {
+ stat->size = ci->i_rbytes;
+ stat->blocks = 0;
+ stat->blksize = 65536;
+ }
+ }
+ return err;
+}
--- /dev/null
+#include <linux/in.h>
+
+#include "ioctl.h"
+#include "super.h"
+#include "ceph_debug.h"
+
+
+/*
+ * ioctls
+ */
+
+/*
+ * get and set the file layout
+ */
+static long ceph_ioctl_get_layout(struct file *file, void __user *arg)
+{
+ struct ceph_inode_info *ci = ceph_inode(file->f_dentry->d_inode);
+ struct ceph_ioctl_layout l;
+ int err;
+
+ err = ceph_do_getattr(file->f_dentry->d_inode, CEPH_STAT_CAP_LAYOUT);
+ if (!err) {
+ l.stripe_unit = ceph_file_layout_su(ci->i_layout);
+ l.stripe_count = ceph_file_layout_stripe_count(ci->i_layout);
+ l.object_size = ceph_file_layout_object_size(ci->i_layout);
+ l.data_pool = le32_to_cpu(ci->i_layout.fl_pg_pool);
+ l.preferred_osd =
+ (s32)le32_to_cpu(ci->i_layout.fl_pg_preferred);
+ if (copy_to_user(arg, &l, sizeof(l)))
+ return -EFAULT;
+ }
+
+ return err;
+}
+
+static long ceph_ioctl_set_layout(struct file *file, void __user *arg)
+{
+ struct inode *inode = file->f_dentry->d_inode;
+ struct inode *parent_inode = file->f_dentry->d_parent->d_inode;
+ struct ceph_mds_client *mdsc = &ceph_sb_to_client(inode->i_sb)->mdsc;
+ struct ceph_mds_request *req;
+ struct ceph_ioctl_layout l;
+ int err, i;
+
+ /* copy and validate */
+ if (copy_from_user(&l, arg, sizeof(l)))
+ return -EFAULT;
+
+ if ((l.object_size & ~PAGE_MASK) ||
+ (l.stripe_unit & ~PAGE_MASK) ||
+ !l.stripe_unit ||
+ (l.object_size &&
+ (unsigned)l.object_size % (unsigned)l.stripe_unit))
+ return -EINVAL;
+
+ /* make sure it's a valid data pool */
+ if (l.data_pool > 0) {
+ mutex_lock(&mdsc->mutex);
+ err = -EINVAL;
+ for (i = 0; i < mdsc->mdsmap->m_num_data_pg_pools; i++)
+ if (mdsc->mdsmap->m_data_pg_pools[i] == l.data_pool) {
+ err = 0;
+ break;
+ }
+ mutex_unlock(&mdsc->mutex);
+ if (err)
+ return err;
+ }
+
+ req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_SETLAYOUT,
+ USE_AUTH_MDS);
+ if (IS_ERR(req))
+ return PTR_ERR(req);
+ req->r_inode = igrab(inode);
+ req->r_inode_drop = CEPH_CAP_FILE_SHARED | CEPH_CAP_FILE_EXCL;
+
+ req->r_args.setlayout.layout.fl_stripe_unit =
+ cpu_to_le32(l.stripe_unit);
+ req->r_args.setlayout.layout.fl_stripe_count =
+ cpu_to_le32(l.stripe_count);
+ req->r_args.setlayout.layout.fl_object_size =
+ cpu_to_le32(l.object_size);
+ req->r_args.setlayout.layout.fl_pg_pool = cpu_to_le32(l.data_pool);
+ req->r_args.setlayout.layout.fl_pg_preferred =
+ cpu_to_le32(l.preferred_osd);
+
+ err = ceph_mdsc_do_request(mdsc, parent_inode, req);
+ ceph_mdsc_put_request(req);
+ return err;
+}
+
+/*
+ * Return object name, size/offset information, and location (OSD
+ * number, network address) for a given file offset.
+ */
+static long ceph_ioctl_get_dataloc(struct file *file, void __user *arg)
+{
+ struct ceph_ioctl_dataloc dl;
+ struct inode *inode = file->f_dentry->d_inode;
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ struct ceph_osd_client *osdc = &ceph_client(inode->i_sb)->osdc;
+ u64 len = 1, olen;
+ u64 tmp;
+ struct ceph_object_layout ol;
+ struct ceph_pg pgid;
+
+ /* copy and validate */
+ if (copy_from_user(&dl, arg, sizeof(dl)))
+ return -EFAULT;
+
+ down_read(&osdc->map_sem);
+ ceph_calc_file_object_mapping(&ci->i_layout, dl.file_offset, &len,
+ &dl.object_no, &dl.object_offset, &olen);
+ dl.file_offset -= dl.object_offset;
+ dl.object_size = ceph_file_layout_object_size(ci->i_layout);
+ dl.block_size = ceph_file_layout_su(ci->i_layout);
+
+ /* block_offset = object_offset % block_size */
+ tmp = dl.object_offset;
+ dl.block_offset = do_div(tmp, dl.block_size);
+
+ snprintf(dl.object_name, sizeof(dl.object_name), "%llx.%08llx",
+ ceph_ino(inode), dl.object_no);
+ ceph_calc_object_layout(&ol, dl.object_name, &ci->i_layout,
+ osdc->osdmap);
+
+ pgid = ol.ol_pgid;
+ dl.osd = ceph_calc_pg_primary(osdc->osdmap, pgid);
+ if (dl.osd >= 0) {
+ struct ceph_entity_addr *a =
+ ceph_osd_addr(osdc->osdmap, dl.osd);
+ if (a)
+ memcpy(&dl.osd_addr, &a->in_addr, sizeof(dl.osd_addr));
+ } else {
+ memset(&dl.osd_addr, 0, sizeof(dl.osd_addr));
+ }
+ up_read(&osdc->map_sem);
+
+ /* send result back to user */
+ if (copy_to_user(arg, &dl, sizeof(dl)))
+ return -EFAULT;
+
+ return 0;
+}
+
+long ceph_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
+{
+ dout("ioctl file %p cmd %u arg %lu\n", file, cmd, arg);
+ switch (cmd) {
+ case CEPH_IOC_GET_LAYOUT:
+ return ceph_ioctl_get_layout(file, (void __user *)arg);
+
+ case CEPH_IOC_SET_LAYOUT:
+ return ceph_ioctl_set_layout(file, (void __user *)arg);
+
+ case CEPH_IOC_GET_DATALOC:
+ return ceph_ioctl_get_dataloc(file, (void __user *)arg);
+ }
+ return -ENOTTY;
+}
--- /dev/null
+#ifndef FS_CEPH_IOCTL_H
+#define FS_CEPH_IOCTL_H
+
+#include <linux/ioctl.h>
+#include <linux/types.h>
+
+#define CEPH_IOCTL_MAGIC 0x97
+
+/* just use u64 to align sanely on all archs */
+struct ceph_ioctl_layout {
+ __u64 stripe_unit, stripe_count, object_size;
+ __u64 data_pool;
+ __s64 preferred_osd;
+};
+
+#define CEPH_IOC_GET_LAYOUT _IOR(CEPH_IOCTL_MAGIC, 1, \
+ struct ceph_ioctl_layout)
+#define CEPH_IOC_SET_LAYOUT _IOW(CEPH_IOCTL_MAGIC, 2, \
+ struct ceph_ioctl_layout)
+
+/*
+ * Extract identity, address of the OSD and object storing a given
+ * file offset.
+ */
+struct ceph_ioctl_dataloc {
+ __u64 file_offset; /* in+out: file offset */
+ __u64 object_offset; /* out: offset in object */
+ __u64 object_no; /* out: object # */
+ __u64 object_size; /* out: object size */
+ char object_name[64]; /* out: object name */
+ __u64 block_offset; /* out: offset in block */
+ __u64 block_size; /* out: block length */
+ __s64 osd; /* out: osd # */
+ struct sockaddr_storage osd_addr; /* out: osd address */
+};
+
+#define CEPH_IOC_GET_DATALOC _IOWR(CEPH_IOCTL_MAGIC, 3, \
+ struct ceph_ioctl_dataloc)
+
+#endif
--- /dev/null
+#include "ceph_debug.h"
+
+#include <linux/wait.h>
+#include <linux/slab.h>
+#include <linux/sched.h>
+
+#include "mds_client.h"
+#include "mon_client.h"
+#include "super.h"
+#include "messenger.h"
+#include "decode.h"
+#include "auth.h"
+#include "pagelist.h"
+
+/*
+ * A cluster of MDS (metadata server) daemons is responsible for
+ * managing the file system namespace (the directory hierarchy and
+ * inodes) and for coordinating shared access to storage. Metadata is
+ * partitioning hierarchically across a number of servers, and that
+ * partition varies over time as the cluster adjusts the distribution
+ * in order to balance load.
+ *
+ * The MDS client is primarily responsible to managing synchronous
+ * metadata requests for operations like open, unlink, and so forth.
+ * If there is a MDS failure, we find out about it when we (possibly
+ * request and) receive a new MDS map, and can resubmit affected
+ * requests.
+ *
+ * For the most part, though, we take advantage of a lossless
+ * communications channel to the MDS, and do not need to worry about
+ * timing out or resubmitting requests.
+ *
+ * We maintain a stateful "session" with each MDS we interact with.
+ * Within each session, we sent periodic heartbeat messages to ensure
+ * any capabilities or leases we have been issues remain valid. If
+ * the session times out and goes stale, our leases and capabilities
+ * are no longer valid.
+ */
+
+static void __wake_requests(struct ceph_mds_client *mdsc,
+ struct list_head *head);
+
+const static struct ceph_connection_operations mds_con_ops;
+
+
+/*
+ * mds reply parsing
+ */
+
+/*
+ * parse individual inode info
+ */
+static int parse_reply_info_in(void **p, void *end,
+ struct ceph_mds_reply_info_in *info)
+{
+ int err = -EIO;
+
+ info->in = *p;
+ *p += sizeof(struct ceph_mds_reply_inode) +
+ sizeof(*info->in->fragtree.splits) *
+ le32_to_cpu(info->in->fragtree.nsplits);
+
+ ceph_decode_32_safe(p, end, info->symlink_len, bad);
+ ceph_decode_need(p, end, info->symlink_len, bad);
+ info->symlink = *p;
+ *p += info->symlink_len;
+
+ ceph_decode_32_safe(p, end, info->xattr_len, bad);
+ ceph_decode_need(p, end, info->xattr_len, bad);
+ info->xattr_data = *p;
+ *p += info->xattr_len;
+ return 0;
+bad:
+ return err;
+}
+
+/*
+ * parse a normal reply, which may contain a (dir+)dentry and/or a
+ * target inode.
+ */
+static int parse_reply_info_trace(void **p, void *end,
+ struct ceph_mds_reply_info_parsed *info)
+{
+ int err;
+
+ if (info->head->is_dentry) {
+ err = parse_reply_info_in(p, end, &info->diri);
+ if (err < 0)
+ goto out_bad;
+
+ if (unlikely(*p + sizeof(*info->dirfrag) > end))
+ goto bad;
+ info->dirfrag = *p;
+ *p += sizeof(*info->dirfrag) +
+ sizeof(u32)*le32_to_cpu(info->dirfrag->ndist);
+ if (unlikely(*p > end))
+ goto bad;
+
+ ceph_decode_32_safe(p, end, info->dname_len, bad);
+ ceph_decode_need(p, end, info->dname_len, bad);
+ info->dname = *p;
+ *p += info->dname_len;
+ info->dlease = *p;
+ *p += sizeof(*info->dlease);
+ }
+
+ if (info->head->is_target) {
+ err = parse_reply_info_in(p, end, &info->targeti);
+ if (err < 0)
+ goto out_bad;
+ }
+
+ if (unlikely(*p != end))
+ goto bad;
+ return 0;
+
+bad:
+ err = -EIO;
+out_bad:
+ pr_err("problem parsing mds trace %d\n", err);
+ return err;
+}
+
+/*
+ * parse readdir results
+ */
+static int parse_reply_info_dir(void **p, void *end,
+ struct ceph_mds_reply_info_parsed *info)
+{
+ u32 num, i = 0;
+ int err;
+
+ info->dir_dir = *p;
+ if (*p + sizeof(*info->dir_dir) > end)
+ goto bad;
+ *p += sizeof(*info->dir_dir) +
+ sizeof(u32)*le32_to_cpu(info->dir_dir->ndist);
+ if (*p > end)
+ goto bad;
+
+ ceph_decode_need(p, end, sizeof(num) + 2, bad);
+ num = ceph_decode_32(p);
+ info->dir_end = ceph_decode_8(p);
+ info->dir_complete = ceph_decode_8(p);
+ if (num == 0)
+ goto done;
+
+ /* alloc large array */
+ info->dir_nr = num;
+ info->dir_in = kcalloc(num, sizeof(*info->dir_in) +
+ sizeof(*info->dir_dname) +
+ sizeof(*info->dir_dname_len) +
+ sizeof(*info->dir_dlease),
+ GFP_NOFS);
+ if (info->dir_in == NULL) {
+ err = -ENOMEM;
+ goto out_bad;
+ }
+ info->dir_dname = (void *)(info->dir_in + num);
+ info->dir_dname_len = (void *)(info->dir_dname + num);
+ info->dir_dlease = (void *)(info->dir_dname_len + num);
+
+ while (num) {
+ /* dentry */
+ ceph_decode_need(p, end, sizeof(u32)*2, bad);
+ info->dir_dname_len[i] = ceph_decode_32(p);
+ ceph_decode_need(p, end, info->dir_dname_len[i], bad);
+ info->dir_dname[i] = *p;
+ *p += info->dir_dname_len[i];
+ dout("parsed dir dname '%.*s'\n", info->dir_dname_len[i],
+ info->dir_dname[i]);
+ info->dir_dlease[i] = *p;
+ *p += sizeof(struct ceph_mds_reply_lease);
+
+ /* inode */
+ err = parse_reply_info_in(p, end, &info->dir_in[i]);
+ if (err < 0)
+ goto out_bad;
+ i++;
+ num--;
+ }
+
+done:
+ if (*p != end)
+ goto bad;
+ return 0;
+
+bad:
+ err = -EIO;
+out_bad:
+ pr_err("problem parsing dir contents %d\n", err);
+ return err;
+}
+
+/*
+ * parse entire mds reply
+ */
+static int parse_reply_info(struct ceph_msg *msg,
+ struct ceph_mds_reply_info_parsed *info)
+{
+ void *p, *end;
+ u32 len;
+ int err;
+
+ info->head = msg->front.iov_base;
+ p = msg->front.iov_base + sizeof(struct ceph_mds_reply_head);
+ end = p + msg->front.iov_len - sizeof(struct ceph_mds_reply_head);
+
+ /* trace */
+ ceph_decode_32_safe(&p, end, len, bad);
+ if (len > 0) {
+ err = parse_reply_info_trace(&p, p+len, info);
+ if (err < 0)
+ goto out_bad;
+ }
+
+ /* dir content */
+ ceph_decode_32_safe(&p, end, len, bad);
+ if (len > 0) {
+ err = parse_reply_info_dir(&p, p+len, info);
+ if (err < 0)
+ goto out_bad;
+ }
+
+ /* snap blob */
+ ceph_decode_32_safe(&p, end, len, bad);
+ info->snapblob_len = len;
+ info->snapblob = p;
+ p += len;
+
+ if (p != end)
+ goto bad;
+ return 0;
+
+bad:
+ err = -EIO;
+out_bad:
+ pr_err("mds parse_reply err %d\n", err);
+ return err;
+}
+
+static void destroy_reply_info(struct ceph_mds_reply_info_parsed *info)
+{
+ kfree(info->dir_in);
+}
+
+
+/*
+ * sessions
+ */
+static const char *session_state_name(int s)
+{
+ switch (s) {
+ case CEPH_MDS_SESSION_NEW: return "new";
+ case CEPH_MDS_SESSION_OPENING: return "opening";
+ case CEPH_MDS_SESSION_OPEN: return "open";
+ case CEPH_MDS_SESSION_HUNG: return "hung";
+ case CEPH_MDS_SESSION_CLOSING: return "closing";
+ case CEPH_MDS_SESSION_RESTARTING: return "restarting";
+ case CEPH_MDS_SESSION_RECONNECTING: return "reconnecting";
+ default: return "???";
+ }
+}
+
+static struct ceph_mds_session *get_session(struct ceph_mds_session *s)
+{
+ if (atomic_inc_not_zero(&s->s_ref)) {
+ dout("mdsc get_session %p %d -> %d\n", s,
+ atomic_read(&s->s_ref)-1, atomic_read(&s->s_ref));
+ return s;
+ } else {
+ dout("mdsc get_session %p 0 -- FAIL", s);
+ return NULL;
+ }
+}
+
+void ceph_put_mds_session(struct ceph_mds_session *s)
+{
+ dout("mdsc put_session %p %d -> %d\n", s,
+ atomic_read(&s->s_ref), atomic_read(&s->s_ref)-1);
+ if (atomic_dec_and_test(&s->s_ref)) {
+ if (s->s_authorizer)
+ s->s_mdsc->client->monc.auth->ops->destroy_authorizer(
+ s->s_mdsc->client->monc.auth, s->s_authorizer);
+ kfree(s);
+ }
+}
+
+/*
+ * called under mdsc->mutex
+ */
+struct ceph_mds_session *__ceph_lookup_mds_session(struct ceph_mds_client *mdsc,
+ int mds)
+{
+ struct ceph_mds_session *session;
+
+ if (mds >= mdsc->max_sessions || mdsc->sessions[mds] == NULL)
+ return NULL;
+ session = mdsc->sessions[mds];
+ dout("lookup_mds_session %p %d\n", session,
+ atomic_read(&session->s_ref));
+ get_session(session);
+ return session;
+}
+
+static bool __have_session(struct ceph_mds_client *mdsc, int mds)
+{
+ if (mds >= mdsc->max_sessions)
+ return false;
+ return mdsc->sessions[mds];
+}
+
+static int __verify_registered_session(struct ceph_mds_client *mdsc,
+ struct ceph_mds_session *s)
+{
+ if (s->s_mds >= mdsc->max_sessions ||
+ mdsc->sessions[s->s_mds] != s)
+ return -ENOENT;
+ return 0;
+}
+
+/*
+ * create+register a new session for given mds.
+ * called under mdsc->mutex.
+ */
+static struct ceph_mds_session *register_session(struct ceph_mds_client *mdsc,
+ int mds)
+{
+ struct ceph_mds_session *s;
+
+ s = kzalloc(sizeof(*s), GFP_NOFS);
+ if (!s)
+ return ERR_PTR(-ENOMEM);
+ s->s_mdsc = mdsc;
+ s->s_mds = mds;
+ s->s_state = CEPH_MDS_SESSION_NEW;
+ s->s_ttl = 0;
+ s->s_seq = 0;
+ mutex_init(&s->s_mutex);
+
+ ceph_con_init(mdsc->client->msgr, &s->s_con);
+ s->s_con.private = s;
+ s->s_con.ops = &mds_con_ops;
+ s->s_con.peer_name.type = CEPH_ENTITY_TYPE_MDS;
+ s->s_con.peer_name.num = cpu_to_le64(mds);
+
+ spin_lock_init(&s->s_cap_lock);
+ s->s_cap_gen = 0;
+ s->s_cap_ttl = 0;
+ s->s_renew_requested = 0;
+ s->s_renew_seq = 0;
+ INIT_LIST_HEAD(&s->s_caps);
+ s->s_nr_caps = 0;
+ s->s_trim_caps = 0;
+ atomic_set(&s->s_ref, 1);
+ INIT_LIST_HEAD(&s->s_waiting);
+ INIT_LIST_HEAD(&s->s_unsafe);
+ s->s_num_cap_releases = 0;
+ s->s_cap_iterator = NULL;
+ INIT_LIST_HEAD(&s->s_cap_releases);
+ INIT_LIST_HEAD(&s->s_cap_releases_done);
+ INIT_LIST_HEAD(&s->s_cap_flushing);
+ INIT_LIST_HEAD(&s->s_cap_snaps_flushing);
+
+ dout("register_session mds%d\n", mds);
+ if (mds >= mdsc->max_sessions) {
+ int newmax = 1 << get_count_order(mds+1);
+ struct ceph_mds_session **sa;
+
+ dout("register_session realloc to %d\n", newmax);
+ sa = kcalloc(newmax, sizeof(void *), GFP_NOFS);
+ if (sa == NULL)
+ goto fail_realloc;
+ if (mdsc->sessions) {
+ memcpy(sa, mdsc->sessions,
+ mdsc->max_sessions * sizeof(void *));
+ kfree(mdsc->sessions);
+ }
+ mdsc->sessions = sa;
+ mdsc->max_sessions = newmax;
+ }
+ mdsc->sessions[mds] = s;
+ atomic_inc(&s->s_ref); /* one ref to sessions[], one to caller */
+
+ ceph_con_open(&s->s_con, ceph_mdsmap_get_addr(mdsc->mdsmap, mds));
+
+ return s;
+
+fail_realloc:
+ kfree(s);
+ return ERR_PTR(-ENOMEM);
+}
+
+/*
+ * called under mdsc->mutex
+ */
+static void __unregister_session(struct ceph_mds_client *mdsc,
+ struct ceph_mds_session *s)
+{
+ dout("__unregister_session mds%d %p\n", s->s_mds, s);
+ BUG_ON(mdsc->sessions[s->s_mds] != s);
+ mdsc->sessions[s->s_mds] = NULL;
+ ceph_con_close(&s->s_con);
+ ceph_put_mds_session(s);
+}
+
+/*
+ * drop session refs in request.
+ *
+ * should be last request ref, or hold mdsc->mutex
+ */
+static void put_request_session(struct ceph_mds_request *req)
+{
+ if (req->r_session) {
+ ceph_put_mds_session(req->r_session);
+ req->r_session = NULL;
+ }
+}
+
+void ceph_mdsc_release_request(struct kref *kref)
+{
+ struct ceph_mds_request *req = container_of(kref,
+ struct ceph_mds_request,
+ r_kref);
+ if (req->r_request)
+ ceph_msg_put(req->r_request);
+ if (req->r_reply) {
+ ceph_msg_put(req->r_reply);
+ destroy_reply_info(&req->r_reply_info);
+ }
+ if (req->r_inode) {
+ ceph_put_cap_refs(ceph_inode(req->r_inode),
+ CEPH_CAP_PIN);
+ iput(req->r_inode);
+ }
+ if (req->r_locked_dir)
+ ceph_put_cap_refs(ceph_inode(req->r_locked_dir),
+ CEPH_CAP_PIN);
+ if (req->r_target_inode)
+ iput(req->r_target_inode);
+ if (req->r_dentry)
+ dput(req->r_dentry);
+ if (req->r_old_dentry) {
+ ceph_put_cap_refs(
+ ceph_inode(req->r_old_dentry->d_parent->d_inode),
+ CEPH_CAP_PIN);
+ dput(req->r_old_dentry);
+ }
+ kfree(req->r_path1);
+ kfree(req->r_path2);
+ put_request_session(req);
+ ceph_unreserve_caps(&req->r_caps_reservation);
+ kfree(req);
+}
+
+/*
+ * lookup session, bump ref if found.
+ *
+ * called under mdsc->mutex.
+ */
+static struct ceph_mds_request *__lookup_request(struct ceph_mds_client *mdsc,
+ u64 tid)
+{
+ struct ceph_mds_request *req;
+ struct rb_node *n = mdsc->request_tree.rb_node;
+
+ while (n) {
+ req = rb_entry(n, struct ceph_mds_request, r_node);
+ if (tid < req->r_tid)
+ n = n->rb_left;
+ else if (tid > req->r_tid)
+ n = n->rb_right;
+ else {
+ ceph_mdsc_get_request(req);
+ return req;
+ }
+ }
+ return NULL;
+}
+
+static void __insert_request(struct ceph_mds_client *mdsc,
+ struct ceph_mds_request *new)
+{
+ struct rb_node **p = &mdsc->request_tree.rb_node;
+ struct rb_node *parent = NULL;
+ struct ceph_mds_request *req = NULL;
+
+ while (*p) {
+ parent = *p;
+ req = rb_entry(parent, struct ceph_mds_request, r_node);
+ if (new->r_tid < req->r_tid)
+ p = &(*p)->rb_left;
+ else if (new->r_tid > req->r_tid)
+ p = &(*p)->rb_right;
+ else
+ BUG();
+ }
+
+ rb_link_node(&new->r_node, parent, p);
+ rb_insert_color(&new->r_node, &mdsc->request_tree);
+}
+
+/*
+ * Register an in-flight request, and assign a tid. Link to directory
+ * are modifying (if any).
+ *
+ * Called under mdsc->mutex.
+ */
+static void __register_request(struct ceph_mds_client *mdsc,
+ struct ceph_mds_request *req,
+ struct inode *dir)
+{
+ req->r_tid = ++mdsc->last_tid;
+ if (req->r_num_caps)
+ ceph_reserve_caps(&req->r_caps_reservation, req->r_num_caps);
+ dout("__register_request %p tid %lld\n", req, req->r_tid);
+ ceph_mdsc_get_request(req);
+ __insert_request(mdsc, req);
+
+ if (dir) {
+ struct ceph_inode_info *ci = ceph_inode(dir);
+
+ spin_lock(&ci->i_unsafe_lock);
+ req->r_unsafe_dir = dir;
+ list_add_tail(&req->r_unsafe_dir_item, &ci->i_unsafe_dirops);
+ spin_unlock(&ci->i_unsafe_lock);
+ }
+}
+
+static void __unregister_request(struct ceph_mds_client *mdsc,
+ struct ceph_mds_request *req)
+{
+ dout("__unregister_request %p tid %lld\n", req, req->r_tid);
+ rb_erase(&req->r_node, &mdsc->request_tree);
+ RB_CLEAR_NODE(&req->r_node);
+
+ if (req->r_unsafe_dir) {
+ struct ceph_inode_info *ci = ceph_inode(req->r_unsafe_dir);
+
+ spin_lock(&ci->i_unsafe_lock);
+ list_del_init(&req->r_unsafe_dir_item);
+ spin_unlock(&ci->i_unsafe_lock);
+ }
+
+ ceph_mdsc_put_request(req);
+}
+
+/*
+ * Choose mds to send request to next. If there is a hint set in the
+ * request (e.g., due to a prior forward hint from the mds), use that.
+ * Otherwise, consult frag tree and/or caps to identify the
+ * appropriate mds. If all else fails, choose randomly.
+ *
+ * Called under mdsc->mutex.
+ */
+static int __choose_mds(struct ceph_mds_client *mdsc,
+ struct ceph_mds_request *req)
+{
+ struct inode *inode;
+ struct ceph_inode_info *ci;
+ struct ceph_cap *cap;
+ int mode = req->r_direct_mode;
+ int mds = -1;
+ u32 hash = req->r_direct_hash;
+ bool is_hash = req->r_direct_is_hash;
+
+ /*
+ * is there a specific mds we should try? ignore hint if we have
+ * no session and the mds is not up (active or recovering).
+ */
+ if (req->r_resend_mds >= 0 &&
+ (__have_session(mdsc, req->r_resend_mds) ||
+ ceph_mdsmap_get_state(mdsc->mdsmap, req->r_resend_mds) > 0)) {
+ dout("choose_mds using resend_mds mds%d\n",
+ req->r_resend_mds);
+ return req->r_resend_mds;
+ }
+
+ if (mode == USE_RANDOM_MDS)
+ goto random;
+
+ inode = NULL;
+ if (req->r_inode) {
+ inode = req->r_inode;
+ } else if (req->r_dentry) {
+ if (req->r_dentry->d_inode) {
+ inode = req->r_dentry->d_inode;
+ } else {
+ inode = req->r_dentry->d_parent->d_inode;
+ hash = req->r_dentry->d_name.hash;
+ is_hash = true;
+ }
+ }
+ dout("__choose_mds %p is_hash=%d (%d) mode %d\n", inode, (int)is_hash,
+ (int)hash, mode);
+ if (!inode)
+ goto random;
+ ci = ceph_inode(inode);
+
+ if (is_hash && S_ISDIR(inode->i_mode)) {
+ struct ceph_inode_frag frag;
+ int found;
+
+ ceph_choose_frag(ci, hash, &frag, &found);
+ if (found) {
+ if (mode == USE_ANY_MDS && frag.ndist > 0) {
+ u8 r;
+
+ /* choose a random replica */
+ get_random_bytes(&r, 1);
+ r %= frag.ndist;
+ mds = frag.dist[r];
+ dout("choose_mds %p %llx.%llx "
+ "frag %u mds%d (%d/%d)\n",
+ inode, ceph_vinop(inode),
+ frag.frag, frag.mds,
+ (int)r, frag.ndist);
+ return mds;
+ }
+
+ /* since this file/dir wasn't known to be
+ * replicated, then we want to look for the
+ * authoritative mds. */
+ mode = USE_AUTH_MDS;
+ if (frag.mds >= 0) {
+ /* choose auth mds */
+ mds = frag.mds;
+ dout("choose_mds %p %llx.%llx "
+ "frag %u mds%d (auth)\n",
+ inode, ceph_vinop(inode), frag.frag, mds);
+ return mds;
+ }
+ }
+ }
+
+ spin_lock(&inode->i_lock);
+ cap = NULL;
+ if (mode == USE_AUTH_MDS)
+ cap = ci->i_auth_cap;
+ if (!cap && !RB_EMPTY_ROOT(&ci->i_caps))
+ cap = rb_entry(rb_first(&ci->i_caps), struct ceph_cap, ci_node);
+ if (!cap) {
+ spin_unlock(&inode->i_lock);
+ goto random;
+ }
+ mds = cap->session->s_mds;
+ dout("choose_mds %p %llx.%llx mds%d (%scap %p)\n",
+ inode, ceph_vinop(inode), mds,
+ cap == ci->i_auth_cap ? "auth " : "", cap);
+ spin_unlock(&inode->i_lock);
+ return mds;
+
+random:
+ mds = ceph_mdsmap_get_random_mds(mdsc->mdsmap);
+ dout("choose_mds chose random mds%d\n", mds);
+ return mds;
+}
+
+
+/*
+ * session messages
+ */
+static struct ceph_msg *create_session_msg(u32 op, u64 seq)
+{
+ struct ceph_msg *msg;
+ struct ceph_mds_session_head *h;
+
+ msg = ceph_msg_new(CEPH_MSG_CLIENT_SESSION, sizeof(*h), 0, 0, NULL);
+ if (IS_ERR(msg)) {
+ pr_err("create_session_msg ENOMEM creating msg\n");
+ return ERR_PTR(PTR_ERR(msg));
+ }
+ h = msg->front.iov_base;
+ h->op = cpu_to_le32(op);
+ h->seq = cpu_to_le64(seq);
+ return msg;
+}
+
+/*
+ * send session open request.
+ *
+ * called under mdsc->mutex
+ */
+static int __open_session(struct ceph_mds_client *mdsc,
+ struct ceph_mds_session *session)
+{
+ struct ceph_msg *msg;
+ int mstate;
+ int mds = session->s_mds;
+ int err = 0;
+
+ /* wait for mds to go active? */
+ mstate = ceph_mdsmap_get_state(mdsc->mdsmap, mds);
+ dout("open_session to mds%d (%s)\n", mds,
+ ceph_mds_state_name(mstate));
+ session->s_state = CEPH_MDS_SESSION_OPENING;
+ session->s_renew_requested = jiffies;
+
+ /* send connect message */
+ msg = create_session_msg(CEPH_SESSION_REQUEST_OPEN, session->s_seq);
+ if (IS_ERR(msg)) {
+ err = PTR_ERR(msg);
+ goto out;
+ }
+ ceph_con_send(&session->s_con, msg);
+
+out:
+ return 0;
+}
+
+/*
+ * session caps
+ */
+
+/*
+ * Free preallocated cap messages assigned to this session
+ */
+static void cleanup_cap_releases(struct ceph_mds_session *session)
+{
+ struct ceph_msg *msg;
+
+ spin_lock(&session->s_cap_lock);
+ while (!list_empty(&session->s_cap_releases)) {
+ msg = list_first_entry(&session->s_cap_releases,
+ struct ceph_msg, list_head);
+ list_del_init(&msg->list_head);
+ ceph_msg_put(msg);
+ }
+ while (!list_empty(&session->s_cap_releases_done)) {
+ msg = list_first_entry(&session->s_cap_releases_done,
+ struct ceph_msg, list_head);
+ list_del_init(&msg->list_head);
+ ceph_msg_put(msg);
+ }
+ spin_unlock(&session->s_cap_lock);
+}
+
+/*
+ * Helper to safely iterate over all caps associated with a session.
+ *
+ * caller must hold session s_mutex
+ */
+static int iterate_session_caps(struct ceph_mds_session *session,
+ int (*cb)(struct inode *, struct ceph_cap *,
+ void *), void *arg)
+{
+ struct list_head *p;
+ struct ceph_cap *cap;
+ struct inode *inode, *last_inode = NULL;
+ struct ceph_cap *old_cap = NULL;
+ int ret;
+
+ dout("iterate_session_caps %p mds%d\n", session, session->s_mds);
+ spin_lock(&session->s_cap_lock);
+ p = session->s_caps.next;
+ while (p != &session->s_caps) {
+ cap = list_entry(p, struct ceph_cap, session_caps);
+ inode = igrab(&cap->ci->vfs_inode);
+ if (!inode) {
+ p = p->next;
+ continue;
+ }
+ session->s_cap_iterator = cap;
+ spin_unlock(&session->s_cap_lock);
+
+ if (last_inode) {
+ iput(last_inode);
+ last_inode = NULL;
+ }
+ if (old_cap) {
+ ceph_put_cap(old_cap);
+ old_cap = NULL;
+ }
+
+ ret = cb(inode, cap, arg);
+ last_inode = inode;
+
+ spin_lock(&session->s_cap_lock);
+ p = p->next;
+ if (cap->ci == NULL) {
+ dout("iterate_session_caps finishing cap %p removal\n",
+ cap);
+ BUG_ON(cap->session != session);
+ list_del_init(&cap->session_caps);
+ session->s_nr_caps--;
+ cap->session = NULL;
+ old_cap = cap; /* put_cap it w/o locks held */
+ }
+ if (ret < 0)
+ goto out;
+ }
+ ret = 0;
+out:
+ session->s_cap_iterator = NULL;
+ spin_unlock(&session->s_cap_lock);
+
+ if (last_inode)
+ iput(last_inode);
+ if (old_cap)
+ ceph_put_cap(old_cap);
+
+ return ret;
+}
+
+static int remove_session_caps_cb(struct inode *inode, struct ceph_cap *cap,
+ void *arg)
+{
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ dout("removing cap %p, ci is %p, inode is %p\n",
+ cap, ci, &ci->vfs_inode);
+ ceph_remove_cap(cap);
+ return 0;
+}
+
+/*
+ * caller must hold session s_mutex
+ */
+static void remove_session_caps(struct ceph_mds_session *session)
+{
+ dout("remove_session_caps on %p\n", session);
+ iterate_session_caps(session, remove_session_caps_cb, NULL);
+ BUG_ON(session->s_nr_caps > 0);
+ cleanup_cap_releases(session);
+}
+
+/*
+ * wake up any threads waiting on this session's caps. if the cap is
+ * old (didn't get renewed on the client reconnect), remove it now.
+ *
+ * caller must hold s_mutex.
+ */
+static int wake_up_session_cb(struct inode *inode, struct ceph_cap *cap,
+ void *arg)
+{
+ struct ceph_inode_info *ci = ceph_inode(inode);
+
+ wake_up(&ci->i_cap_wq);
+ if (arg) {
+ spin_lock(&inode->i_lock);
+ ci->i_wanted_max_size = 0;
+ ci->i_requested_max_size = 0;
+ spin_unlock(&inode->i_lock);
+ }
+ return 0;
+}
+
+static void wake_up_session_caps(struct ceph_mds_session *session,
+ int reconnect)
+{
+ dout("wake_up_session_caps %p mds%d\n", session, session->s_mds);
+ iterate_session_caps(session, wake_up_session_cb,
+ (void *)(unsigned long)reconnect);
+}
+
+/*
+ * Send periodic message to MDS renewing all currently held caps. The
+ * ack will reset the expiration for all caps from this session.
+ *
+ * caller holds s_mutex
+ */
+static int send_renew_caps(struct ceph_mds_client *mdsc,
+ struct ceph_mds_session *session)
+{
+ struct ceph_msg *msg;
+ int state;
+
+ if (time_after_eq(jiffies, session->s_cap_ttl) &&
+ time_after_eq(session->s_cap_ttl, session->s_renew_requested))
+ pr_info("mds%d caps stale\n", session->s_mds);
+ session->s_renew_requested = jiffies;
+
+ /* do not try to renew caps until a recovering mds has reconnected
+ * with its clients. */
+ state = ceph_mdsmap_get_state(mdsc->mdsmap, session->s_mds);
+ if (state < CEPH_MDS_STATE_RECONNECT) {
+ dout("send_renew_caps ignoring mds%d (%s)\n",
+ session->s_mds, ceph_mds_state_name(state));
+ return 0;
+ }
+
+ dout("send_renew_caps to mds%d (%s)\n", session->s_mds,
+ ceph_mds_state_name(state));
+ msg = create_session_msg(CEPH_SESSION_REQUEST_RENEWCAPS,
+ ++session->s_renew_seq);
+ if (IS_ERR(msg))
+ return PTR_ERR(msg);
+ ceph_con_send(&session->s_con, msg);
+ return 0;
+}
+
+/*
+ * Note new cap ttl, and any transition from stale -> not stale (fresh?).
+ *
+ * Called under session->s_mutex
+ */
+static void renewed_caps(struct ceph_mds_client *mdsc,
+ struct ceph_mds_session *session, int is_renew)
+{
+ int was_stale;
+ int wake = 0;
+
+ spin_lock(&session->s_cap_lock);
+ was_stale = is_renew && (session->s_cap_ttl == 0 ||
+ time_after_eq(jiffies, session->s_cap_ttl));
+
+ session->s_cap_ttl = session->s_renew_requested +
+ mdsc->mdsmap->m_session_timeout*HZ;
+
+ if (was_stale) {
+ if (time_before(jiffies, session->s_cap_ttl)) {
+ pr_info("mds%d caps renewed\n", session->s_mds);
+ wake = 1;
+ } else {
+ pr_info("mds%d caps still stale\n", session->s_mds);
+ }
+ }
+ dout("renewed_caps mds%d ttl now %lu, was %s, now %s\n",
+ session->s_mds, session->s_cap_ttl, was_stale ? "stale" : "fresh",
+ time_before(jiffies, session->s_cap_ttl) ? "stale" : "fresh");
+ spin_unlock(&session->s_cap_lock);
+
+ if (wake)
+ wake_up_session_caps(session, 0);
+}
+
+/*
+ * send a session close request
+ */
+static int request_close_session(struct ceph_mds_client *mdsc,
+ struct ceph_mds_session *session)
+{
+ struct ceph_msg *msg;
+ int err = 0;
+
+ dout("request_close_session mds%d state %s seq %lld\n",
+ session->s_mds, session_state_name(session->s_state),
+ session->s_seq);
+ msg = create_session_msg(CEPH_SESSION_REQUEST_CLOSE, session->s_seq);
+ if (IS_ERR(msg))
+ err = PTR_ERR(msg);
+ else
+ ceph_con_send(&session->s_con, msg);
+ return err;
+}
+
+/*
+ * Called with s_mutex held.
+ */
+static int __close_session(struct ceph_mds_client *mdsc,
+ struct ceph_mds_session *session)
+{
+ if (session->s_state >= CEPH_MDS_SESSION_CLOSING)
+ return 0;
+ session->s_state = CEPH_MDS_SESSION_CLOSING;
+ return request_close_session(mdsc, session);
+}
+
+/*
+ * Trim old(er) caps.
+ *
+ * Because we can't cache an inode without one or more caps, we do
+ * this indirectly: if a cap is unused, we prune its aliases, at which
+ * point the inode will hopefully get dropped to.
+ *
+ * Yes, this is a bit sloppy. Our only real goal here is to respond to
+ * memory pressure from the MDS, though, so it needn't be perfect.
+ */
+static int trim_caps_cb(struct inode *inode, struct ceph_cap *cap, void *arg)
+{
+ struct ceph_mds_session *session = arg;
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ int used, oissued, mine;
+
+ if (session->s_trim_caps <= 0)
+ return -1;
+
+ spin_lock(&inode->i_lock);
+ mine = cap->issued | cap->implemented;
+ used = __ceph_caps_used(ci);
+ oissued = __ceph_caps_issued_other(ci, cap);
+
+ dout("trim_caps_cb %p cap %p mine %s oissued %s used %s\n",
+ inode, cap, ceph_cap_string(mine), ceph_cap_string(oissued),
+ ceph_cap_string(used));
+ if (ci->i_dirty_caps)
+ goto out; /* dirty caps */
+ if ((used & ~oissued) & mine)
+ goto out; /* we need these caps */
+
+ session->s_trim_caps--;
+ if (oissued) {
+ /* we aren't the only cap.. just remove us */
+ __ceph_remove_cap(cap);
+ } else {
+ /* try to drop referring dentries */
+ spin_unlock(&inode->i_lock);
+ d_prune_aliases(inode);
+ dout("trim_caps_cb %p cap %p pruned, count now %d\n",
+ inode, cap, atomic_read(&inode->i_count));
+ return 0;
+ }
+
+out:
+ spin_unlock(&inode->i_lock);
+ return 0;
+}
+
+/*
+ * Trim session cap count down to some max number.
+ */
+static int trim_caps(struct ceph_mds_client *mdsc,
+ struct ceph_mds_session *session,
+ int max_caps)
+{
+ int trim_caps = session->s_nr_caps - max_caps;
+
+ dout("trim_caps mds%d start: %d / %d, trim %d\n",
+ session->s_mds, session->s_nr_caps, max_caps, trim_caps);
+ if (trim_caps > 0) {
+ session->s_trim_caps = trim_caps;
+ iterate_session_caps(session, trim_caps_cb, session);
+ dout("trim_caps mds%d done: %d / %d, trimmed %d\n",
+ session->s_mds, session->s_nr_caps, max_caps,
+ trim_caps - session->s_trim_caps);
+ session->s_trim_caps = 0;
+ }
+ return 0;
+}
+
+/*
+ * Allocate cap_release messages. If there is a partially full message
+ * in the queue, try to allocate enough to cover it's remainder, so that
+ * we can send it immediately.
+ *
+ * Called under s_mutex.
+ */
+static int add_cap_releases(struct ceph_mds_client *mdsc,
+ struct ceph_mds_session *session,
+ int extra)
+{
+ struct ceph_msg *msg;
+ struct ceph_mds_cap_release *head;
+ int err = -ENOMEM;
+
+ if (extra < 0)
+ extra = mdsc->client->mount_args->cap_release_safety;
+
+ spin_lock(&session->s_cap_lock);
+
+ if (!list_empty(&session->s_cap_releases)) {
+ msg = list_first_entry(&session->s_cap_releases,
+ struct ceph_msg,
+ list_head);
+ head = msg->front.iov_base;
+ extra += CEPH_CAPS_PER_RELEASE - le32_to_cpu(head->num);
+ }
+
+ while (session->s_num_cap_releases < session->s_nr_caps + extra) {
+ spin_unlock(&session->s_cap_lock);
+ msg = ceph_msg_new(CEPH_MSG_CLIENT_CAPRELEASE, PAGE_CACHE_SIZE,
+ 0, 0, NULL);
+ if (!msg)
+ goto out_unlocked;
+ dout("add_cap_releases %p msg %p now %d\n", session, msg,
+ (int)msg->front.iov_len);
+ head = msg->front.iov_base;
+ head->num = cpu_to_le32(0);
+ msg->front.iov_len = sizeof(*head);
+ spin_lock(&session->s_cap_lock);
+ list_add(&msg->list_head, &session->s_cap_releases);
+ session->s_num_cap_releases += CEPH_CAPS_PER_RELEASE;
+ }
+
+ if (!list_empty(&session->s_cap_releases)) {
+ msg = list_first_entry(&session->s_cap_releases,
+ struct ceph_msg,
+ list_head);
+ head = msg->front.iov_base;
+ if (head->num) {
+ dout(" queueing non-full %p (%d)\n", msg,
+ le32_to_cpu(head->num));
+ list_move_tail(&msg->list_head,
+ &session->s_cap_releases_done);
+ session->s_num_cap_releases -=
+ CEPH_CAPS_PER_RELEASE - le32_to_cpu(head->num);
+ }
+ }
+ err = 0;
+ spin_unlock(&session->s_cap_lock);
+out_unlocked:
+ return err;
+}
+
+/*
+ * flush all dirty inode data to disk.
+ *
+ * returns true if we've flushed through want_flush_seq
+ */
+static int check_cap_flush(struct ceph_mds_client *mdsc, u64 want_flush_seq)
+{
+ int mds, ret = 1;
+
+ dout("check_cap_flush want %lld\n", want_flush_seq);
+ mutex_lock(&mdsc->mutex);
+ for (mds = 0; ret && mds < mdsc->max_sessions; mds++) {
+ struct ceph_mds_session *session = mdsc->sessions[mds];
+
+ if (!session)
+ continue;
+ get_session(session);
+ mutex_unlock(&mdsc->mutex);
+
+ mutex_lock(&session->s_mutex);
+ if (!list_empty(&session->s_cap_flushing)) {
+ struct ceph_inode_info *ci =
+ list_entry(session->s_cap_flushing.next,
+ struct ceph_inode_info,
+ i_flushing_item);
+ struct inode *inode = &ci->vfs_inode;
+
+ spin_lock(&inode->i_lock);
+ if (ci->i_cap_flush_seq <= want_flush_seq) {
+ dout("check_cap_flush still flushing %p "
+ "seq %lld <= %lld to mds%d\n", inode,
+ ci->i_cap_flush_seq, want_flush_seq,
+ session->s_mds);
+ ret = 0;
+ }
+ spin_unlock(&inode->i_lock);
+ }
+ mutex_unlock(&session->s_mutex);
+ ceph_put_mds_session(session);
+
+ if (!ret)
+ return ret;
+ mutex_lock(&mdsc->mutex);
+ }
+
+ mutex_unlock(&mdsc->mutex);
+ dout("check_cap_flush ok, flushed thru %lld\n", want_flush_seq);
+ return ret;
+}
+
+/*
+ * called under s_mutex
+ */
+static void send_cap_releases(struct ceph_mds_client *mdsc,
+ struct ceph_mds_session *session)
+{
+ struct ceph_msg *msg;
+
+ dout("send_cap_releases mds%d\n", session->s_mds);
+ while (1) {
+ spin_lock(&session->s_cap_lock);
+ if (list_empty(&session->s_cap_releases_done))
+ break;
+ msg = list_first_entry(&session->s_cap_releases_done,
+ struct ceph_msg, list_head);
+ list_del_init(&msg->list_head);
+ spin_unlock(&session->s_cap_lock);
+ msg->hdr.front_len = cpu_to_le32(msg->front.iov_len);
+ dout("send_cap_releases mds%d %p\n", session->s_mds, msg);
+ ceph_con_send(&session->s_con, msg);
+ }
+ spin_unlock(&session->s_cap_lock);
+}
+
+/*
+ * requests
+ */
+
+/*
+ * Create an mds request.
+ */
+struct ceph_mds_request *
+ceph_mdsc_create_request(struct ceph_mds_client *mdsc, int op, int mode)
+{
+ struct ceph_mds_request *req = kzalloc(sizeof(*req), GFP_NOFS);
+
+ if (!req)
+ return ERR_PTR(-ENOMEM);
+
+ req->r_started = jiffies;
+ req->r_resend_mds = -1;
+ INIT_LIST_HEAD(&req->r_unsafe_dir_item);
+ req->r_fmode = -1;
+ kref_init(&req->r_kref);
+ INIT_LIST_HEAD(&req->r_wait);
+ init_completion(&req->r_completion);
+ init_completion(&req->r_safe_completion);
+ INIT_LIST_HEAD(&req->r_unsafe_item);
+
+ req->r_op = op;
+ req->r_direct_mode = mode;
+ return req;
+}
+
+/*
+ * return oldest (lowest) request, tid in request tree, 0 if none.
+ *
+ * called under mdsc->mutex.
+ */
+static struct ceph_mds_request *__get_oldest_req(struct ceph_mds_client *mdsc)
+{
+ if (RB_EMPTY_ROOT(&mdsc->request_tree))
+ return NULL;
+ return rb_entry(rb_first(&mdsc->request_tree),
+ struct ceph_mds_request, r_node);
+}
+
+static u64 __get_oldest_tid(struct ceph_mds_client *mdsc)
+{
+ struct ceph_mds_request *req = __get_oldest_req(mdsc);
+
+ if (req)
+ return req->r_tid;
+ return 0;
+}
+
+/*
+ * Build a dentry's path. Allocate on heap; caller must kfree. Based
+ * on build_path_from_dentry in fs/cifs/dir.c.
+ *
+ * If @stop_on_nosnap, generate path relative to the first non-snapped
+ * inode.
+ *
+ * Encode hidden .snap dirs as a double /, i.e.
+ * foo/.snap/bar -> foo//bar
+ */
+char *ceph_mdsc_build_path(struct dentry *dentry, int *plen, u64 *base,
+ int stop_on_nosnap)
+{
+ struct dentry *temp;
+ char *path;
+ int len, pos;
+
+ if (dentry == NULL)
+ return ERR_PTR(-EINVAL);
+
+retry:
+ len = 0;
+ for (temp = dentry; !IS_ROOT(temp);) {
+ struct inode *inode = temp->d_inode;
+ if (inode && ceph_snap(inode) == CEPH_SNAPDIR)
+ len++; /* slash only */
+ else if (stop_on_nosnap && inode &&
+ ceph_snap(inode) == CEPH_NOSNAP)
+ break;
+ else
+ len += 1 + temp->d_name.len;
+ temp = temp->d_parent;
+ if (temp == NULL) {
+ pr_err("build_path_dentry corrupt dentry %p\n", dentry);
+ return ERR_PTR(-EINVAL);
+ }
+ }
+ if (len)
+ len--; /* no leading '/' */
+
+ path = kmalloc(len+1, GFP_NOFS);
+ if (path == NULL)
+ return ERR_PTR(-ENOMEM);
+ pos = len;
+ path[pos] = 0; /* trailing null */
+ for (temp = dentry; !IS_ROOT(temp) && pos != 0; ) {
+ struct inode *inode = temp->d_inode;
+
+ if (inode && ceph_snap(inode) == CEPH_SNAPDIR) {
+ dout("build_path_dentry path+%d: %p SNAPDIR\n",
+ pos, temp);
+ } else if (stop_on_nosnap && inode &&
+ ceph_snap(inode) == CEPH_NOSNAP) {
+ break;
+ } else {
+ pos -= temp->d_name.len;
+ if (pos < 0)
+ break;
+ strncpy(path + pos, temp->d_name.name,
+ temp->d_name.len);
+ dout("build_path_dentry path+%d: %p '%.*s'\n",
+ pos, temp, temp->d_name.len, path + pos);
+ }
+ if (pos)
+ path[--pos] = '/';
+ temp = temp->d_parent;
+ if (temp == NULL) {
+ pr_err("build_path_dentry corrupt dentry\n");
+ kfree(path);
+ return ERR_PTR(-EINVAL);
+ }
+ }
+ if (pos != 0) {
+ pr_err("build_path_dentry did not end path lookup where "
+ "expected, namelen is %d, pos is %d\n", len, pos);
+ /* presumably this is only possible if racing with a
+ rename of one of the parent directories (we can not
+ lock the dentries above us to prevent this, but
+ retrying should be harmless) */
+ kfree(path);
+ goto retry;
+ }
+
+ *base = ceph_ino(temp->d_inode);
+ *plen = len;
+ dout("build_path_dentry on %p %d built %llx '%.*s'\n",
+ dentry, atomic_read(&dentry->d_count), *base, len, path);
+ return path;
+}
+
+static int build_dentry_path(struct dentry *dentry,
+ const char **ppath, int *ppathlen, u64 *pino,
+ int *pfreepath)
+{
+ char *path;
+
+ if (ceph_snap(dentry->d_parent->d_inode) == CEPH_NOSNAP) {
+ *pino = ceph_ino(dentry->d_parent->d_inode);
+ *ppath = dentry->d_name.name;
+ *ppathlen = dentry->d_name.len;
+ return 0;
+ }
+ path = ceph_mdsc_build_path(dentry, ppathlen, pino, 1);
+ if (IS_ERR(path))
+ return PTR_ERR(path);
+ *ppath = path;
+ *pfreepath = 1;
+ return 0;
+}
+
+static int build_inode_path(struct inode *inode,
+ const char **ppath, int *ppathlen, u64 *pino,
+ int *pfreepath)
+{
+ struct dentry *dentry;
+ char *path;
+
+ if (ceph_snap(inode) == CEPH_NOSNAP) {
+ *pino = ceph_ino(inode);
+ *ppathlen = 0;
+ return 0;
+ }
+ dentry = d_find_alias(inode);
+ path = ceph_mdsc_build_path(dentry, ppathlen, pino, 1);
+ dput(dentry);
+ if (IS_ERR(path))
+ return PTR_ERR(path);
+ *ppath = path;
+ *pfreepath = 1;
+ return 0;
+}
+
+/*
+ * request arguments may be specified via an inode *, a dentry *, or
+ * an explicit ino+path.
+ */
+static int set_request_path_attr(struct inode *rinode, struct dentry *rdentry,
+ const char *rpath, u64 rino,
+ const char **ppath, int *pathlen,
+ u64 *ino, int *freepath)
+{
+ int r = 0;
+
+ if (rinode) {
+ r = build_inode_path(rinode, ppath, pathlen, ino, freepath);
+ dout(" inode %p %llx.%llx\n", rinode, ceph_ino(rinode),
+ ceph_snap(rinode));
+ } else if (rdentry) {
+ r = build_dentry_path(rdentry, ppath, pathlen, ino, freepath);
+ dout(" dentry %p %llx/%.*s\n", rdentry, *ino, *pathlen,
+ *ppath);
+ } else if (rpath) {
+ *ino = rino;
+ *ppath = rpath;
+ *pathlen = strlen(rpath);
+ dout(" path %.*s\n", *pathlen, rpath);
+ }
+
+ return r;
+}
+
+/*
+ * called under mdsc->mutex
+ */
+static struct ceph_msg *create_request_message(struct ceph_mds_client *mdsc,
+ struct ceph_mds_request *req,
+ int mds)
+{
+ struct ceph_msg *msg;
+ struct ceph_mds_request_head *head;
+ const char *path1 = NULL;
+ const char *path2 = NULL;
+ u64 ino1 = 0, ino2 = 0;
+ int pathlen1 = 0, pathlen2 = 0;
+ int freepath1 = 0, freepath2 = 0;
+ int len;
+ u16 releases;
+ void *p, *end;
+ int ret;
+
+ ret = set_request_path_attr(req->r_inode, req->r_dentry,
+ req->r_path1, req->r_ino1.ino,
+ &path1, &pathlen1, &ino1, &freepath1);
+ if (ret < 0) {
+ msg = ERR_PTR(ret);
+ goto out;
+ }
+
+ ret = set_request_path_attr(NULL, req->r_old_dentry,
+ req->r_path2, req->r_ino2.ino,
+ &path2, &pathlen2, &ino2, &freepath2);
+ if (ret < 0) {
+ msg = ERR_PTR(ret);
+ goto out_free1;
+ }
+
+ len = sizeof(*head) +
+ pathlen1 + pathlen2 + 2*(1 + sizeof(u32) + sizeof(u64));
+
+ /* calculate (max) length for cap releases */
+ len += sizeof(struct ceph_mds_request_release) *
+ (!!req->r_inode_drop + !!req->r_dentry_drop +
+ !!req->r_old_inode_drop + !!req->r_old_dentry_drop);
+ if (req->r_dentry_drop)
+ len += req->r_dentry->d_name.len;
+ if (req->r_old_dentry_drop)
+ len += req->r_old_dentry->d_name.len;
+
+ msg = ceph_msg_new(CEPH_MSG_CLIENT_REQUEST, len, 0, 0, NULL);
+ if (IS_ERR(msg))
+ goto out_free2;
+
+ msg->hdr.tid = cpu_to_le64(req->r_tid);
+
+ head = msg->front.iov_base;
+ p = msg->front.iov_base + sizeof(*head);
+ end = msg->front.iov_base + msg->front.iov_len;
+
+ head->mdsmap_epoch = cpu_to_le32(mdsc->mdsmap->m_epoch);
+ head->op = cpu_to_le32(req->r_op);
+ head->caller_uid = cpu_to_le32(current_fsuid());
+ head->caller_gid = cpu_to_le32(current_fsgid());
+ head->args = req->r_args;
+
+ ceph_encode_filepath(&p, end, ino1, path1);
+ ceph_encode_filepath(&p, end, ino2, path2);
+
+ /* cap releases */
+ releases = 0;
+ if (req->r_inode_drop)
+ releases += ceph_encode_inode_release(&p,
+ req->r_inode ? req->r_inode : req->r_dentry->d_inode,
+ mds, req->r_inode_drop, req->r_inode_unless, 0);
+ if (req->r_dentry_drop)
+ releases += ceph_encode_dentry_release(&p, req->r_dentry,
+ mds, req->r_dentry_drop, req->r_dentry_unless);
+ if (req->r_old_dentry_drop)
+ releases += ceph_encode_dentry_release(&p, req->r_old_dentry,
+ mds, req->r_old_dentry_drop, req->r_old_dentry_unless);
+ if (req->r_old_inode_drop)
+ releases += ceph_encode_inode_release(&p,
+ req->r_old_dentry->d_inode,
+ mds, req->r_old_inode_drop, req->r_old_inode_unless, 0);
+ head->num_releases = cpu_to_le16(releases);
+
+ BUG_ON(p > end);
+ msg->front.iov_len = p - msg->front.iov_base;
+ msg->hdr.front_len = cpu_to_le32(msg->front.iov_len);
+
+ msg->pages = req->r_pages;
+ msg->nr_pages = req->r_num_pages;
+ msg->hdr.data_len = cpu_to_le32(req->r_data_len);
+ msg->hdr.data_off = cpu_to_le16(0);
+
+out_free2:
+ if (freepath2)
+ kfree((char *)path2);
+out_free1:
+ if (freepath1)
+ kfree((char *)path1);
+out:
+ return msg;
+}
+
+/*
+ * called under mdsc->mutex if error, under no mutex if
+ * success.
+ */
+static void complete_request(struct ceph_mds_client *mdsc,
+ struct ceph_mds_request *req)
+{
+ if (req->r_callback)
+ req->r_callback(mdsc, req);
+ else
+ complete(&req->r_completion);
+}
+
+/*
+ * called under mdsc->mutex
+ */
+static int __prepare_send_request(struct ceph_mds_client *mdsc,
+ struct ceph_mds_request *req,
+ int mds)
+{
+ struct ceph_mds_request_head *rhead;
+ struct ceph_msg *msg;
+ int flags = 0;
+
+ req->r_mds = mds;
+ req->r_attempts++;
+ dout("prepare_send_request %p tid %lld %s (attempt %d)\n", req,
+ req->r_tid, ceph_mds_op_name(req->r_op), req->r_attempts);
+
+ if (req->r_request) {
+ ceph_msg_put(req->r_request);
+ req->r_request = NULL;
+ }
+ msg = create_request_message(mdsc, req, mds);
+ if (IS_ERR(msg)) {
+ req->r_reply = ERR_PTR(PTR_ERR(msg));
+ complete_request(mdsc, req);
+ return -PTR_ERR(msg);
+ }
+ req->r_request = msg;
+
+ rhead = msg->front.iov_base;
+ rhead->oldest_client_tid = cpu_to_le64(__get_oldest_tid(mdsc));
+ if (req->r_got_unsafe)
+ flags |= CEPH_MDS_FLAG_REPLAY;
+ if (req->r_locked_dir)
+ flags |= CEPH_MDS_FLAG_WANT_DENTRY;
+ rhead->flags = cpu_to_le32(flags);
+ rhead->num_fwd = req->r_num_fwd;
+ rhead->num_retry = req->r_attempts - 1;
+
+ dout(" r_locked_dir = %p\n", req->r_locked_dir);
+
+ if (req->r_target_inode && req->r_got_unsafe)
+ rhead->ino = cpu_to_le64(ceph_ino(req->r_target_inode));
+ else
+ rhead->ino = 0;
+ return 0;
+}
+
+/*
+ * send request, or put it on the appropriate wait list.
+ */
+static int __do_request(struct ceph_mds_client *mdsc,
+ struct ceph_mds_request *req)
+{
+ struct ceph_mds_session *session = NULL;
+ int mds = -1;
+ int err = -EAGAIN;
+
+ if (req->r_reply)
+ goto out;
+
+ if (req->r_timeout &&
+ time_after_eq(jiffies, req->r_started + req->r_timeout)) {
+ dout("do_request timed out\n");
+ err = -EIO;
+ goto finish;
+ }
+
+ mds = __choose_mds(mdsc, req);
+ if (mds < 0 ||
+ ceph_mdsmap_get_state(mdsc->mdsmap, mds) < CEPH_MDS_STATE_ACTIVE) {
+ dout("do_request no mds or not active, waiting for map\n");
+ list_add(&req->r_wait, &mdsc->waiting_for_map);
+ goto out;
+ }
+
+ /* get, open session */
+ session = __ceph_lookup_mds_session(mdsc, mds);
+ if (!session) {
+ session = register_session(mdsc, mds);
+ if (IS_ERR(session)) {
+ err = PTR_ERR(session);
+ goto finish;
+ }
+ }
+ dout("do_request mds%d session %p state %s\n", mds, session,
+ session_state_name(session->s_state));
+ if (session->s_state != CEPH_MDS_SESSION_OPEN &&
+ session->s_state != CEPH_MDS_SESSION_HUNG) {
+ if (session->s_state == CEPH_MDS_SESSION_NEW ||
+ session->s_state == CEPH_MDS_SESSION_CLOSING)
+ __open_session(mdsc, session);
+ list_add(&req->r_wait, &session->s_waiting);
+ goto out_session;
+ }
+
+ /* send request */
+ req->r_session = get_session(session);
+ req->r_resend_mds = -1; /* forget any previous mds hint */
+
+ if (req->r_request_started == 0) /* note request start time */
+ req->r_request_started = jiffies;
+
+ err = __prepare_send_request(mdsc, req, mds);
+ if (!err) {
+ ceph_msg_get(req->r_request);
+ ceph_con_send(&session->s_con, req->r_request);
+ }
+
+out_session:
+ ceph_put_mds_session(session);
+out:
+ return err;
+
+finish:
+ req->r_reply = ERR_PTR(err);
+ complete_request(mdsc, req);
+ goto out;
+}
+
+/*
+ * called under mdsc->mutex
+ */
+static void __wake_requests(struct ceph_mds_client *mdsc,
+ struct list_head *head)
+{
+ struct ceph_mds_request *req, *nreq;
+
+ list_for_each_entry_safe(req, nreq, head, r_wait) {
+ list_del_init(&req->r_wait);
+ __do_request(mdsc, req);
+ }
+}
+
+/*
+ * Wake up threads with requests pending for @mds, so that they can
+ * resubmit their requests to a possibly different mds. If @all is set,
+ * wake up if their requests has been forwarded to @mds, too.
+ */
+static void kick_requests(struct ceph_mds_client *mdsc, int mds, int all)
+{
+ struct ceph_mds_request *req;
+ struct rb_node *p;
+
+ dout("kick_requests mds%d\n", mds);
+ for (p = rb_first(&mdsc->request_tree); p; p = rb_next(p)) {
+ req = rb_entry(p, struct ceph_mds_request, r_node);
+ if (req->r_got_unsafe)
+ continue;
+ if (req->r_session &&
+ req->r_session->s_mds == mds) {
+ dout(" kicking tid %llu\n", req->r_tid);
+ put_request_session(req);
+ __do_request(mdsc, req);
+ }
+ }
+}
+
+void ceph_mdsc_submit_request(struct ceph_mds_client *mdsc,
+ struct ceph_mds_request *req)
+{
+ dout("submit_request on %p\n", req);
+ mutex_lock(&mdsc->mutex);
+ __register_request(mdsc, req, NULL);
+ __do_request(mdsc, req);
+ mutex_unlock(&mdsc->mutex);
+}
+
+/*
+ * Synchrously perform an mds request. Take care of all of the
+ * session setup, forwarding, retry details.
+ */
+int ceph_mdsc_do_request(struct ceph_mds_client *mdsc,
+ struct inode *dir,
+ struct ceph_mds_request *req)
+{
+ int err;
+
+ dout("do_request on %p\n", req);
+
+ /* take CAP_PIN refs for r_inode, r_locked_dir, r_old_dentry */
+ if (req->r_inode)
+ ceph_get_cap_refs(ceph_inode(req->r_inode), CEPH_CAP_PIN);
+ if (req->r_locked_dir)
+ ceph_get_cap_refs(ceph_inode(req->r_locked_dir), CEPH_CAP_PIN);
+ if (req->r_old_dentry)
+ ceph_get_cap_refs(
+ ceph_inode(req->r_old_dentry->d_parent->d_inode),
+ CEPH_CAP_PIN);
+
+ /* issue */
+ mutex_lock(&mdsc->mutex);
+ __register_request(mdsc, req, dir);
+ __do_request(mdsc, req);
+
+ /* wait */
+ if (!req->r_reply) {
+ mutex_unlock(&mdsc->mutex);
+ if (req->r_timeout) {
+ err = (long)wait_for_completion_interruptible_timeout(
+ &req->r_completion, req->r_timeout);
+ if (err == 0)
+ req->r_reply = ERR_PTR(-EIO);
+ else if (err < 0)
+ req->r_reply = ERR_PTR(err);
+ } else {
+ err = wait_for_completion_interruptible(
+ &req->r_completion);
+ if (err)
+ req->r_reply = ERR_PTR(err);
+ }
+ mutex_lock(&mdsc->mutex);
+ }
+
+ if (IS_ERR(req->r_reply)) {
+ err = PTR_ERR(req->r_reply);
+ req->r_reply = NULL;
+
+ if (err == -ERESTARTSYS) {
+ /* aborted */
+ req->r_aborted = true;
+
+ if (req->r_locked_dir &&
+ (req->r_op & CEPH_MDS_OP_WRITE)) {
+ struct ceph_inode_info *ci =
+ ceph_inode(req->r_locked_dir);
+
+ dout("aborted, clearing I_COMPLETE on %p\n",
+ req->r_locked_dir);
+ spin_lock(&req->r_locked_dir->i_lock);
+ ci->i_ceph_flags &= ~CEPH_I_COMPLETE;
+ ci->i_release_count++;
+ spin_unlock(&req->r_locked_dir->i_lock);
+ }
+ } else {
+ /* clean up this request */
+ __unregister_request(mdsc, req);
+ if (!list_empty(&req->r_unsafe_item))
+ list_del_init(&req->r_unsafe_item);
+ complete(&req->r_safe_completion);
+ }
+ } else if (req->r_err) {
+ err = req->r_err;
+ } else {
+ err = le32_to_cpu(req->r_reply_info.head->result);
+ }
+ mutex_unlock(&mdsc->mutex);
+
+ dout("do_request %p done, result %d\n", req, err);
+ return err;
+}
+
+/*
+ * Handle mds reply.
+ *
+ * We take the session mutex and parse and process the reply immediately.
+ * This preserves the logical ordering of replies, capabilities, etc., sent
+ * by the MDS as they are applied to our local cache.
+ */
+static void handle_reply(struct ceph_mds_session *session, struct ceph_msg *msg)
+{
+ struct ceph_mds_client *mdsc = session->s_mdsc;
+ struct ceph_mds_request *req;
+ struct ceph_mds_reply_head *head = msg->front.iov_base;
+ struct ceph_mds_reply_info_parsed *rinfo; /* parsed reply info */
+ u64 tid;
+ int err, result;
+ int mds = session->s_mds;
+
+ if (msg->front.iov_len < sizeof(*head)) {
+ pr_err("mdsc_handle_reply got corrupt (short) reply\n");
+ ceph_msg_dump(msg);
+ return;
+ }
+
+ /* get request, session */
+ tid = le64_to_cpu(msg->hdr.tid);
+ mutex_lock(&mdsc->mutex);
+ req = __lookup_request(mdsc, tid);
+ if (!req) {
+ dout("handle_reply on unknown tid %llu\n", tid);
+ mutex_unlock(&mdsc->mutex);
+ return;
+ }
+ dout("handle_reply %p\n", req);
+
+ /* correct session? */
+ if (req->r_session != session) {
+ pr_err("mdsc_handle_reply got %llu on session mds%d"
+ " not mds%d\n", tid, session->s_mds,
+ req->r_session ? req->r_session->s_mds : -1);
+ mutex_unlock(&mdsc->mutex);
+ goto out;
+ }
+
+ /* dup? */
+ if ((req->r_got_unsafe && !head->safe) ||
+ (req->r_got_safe && head->safe)) {
+ pr_warning("got a dup %s reply on %llu from mds%d\n",
+ head->safe ? "safe" : "unsafe", tid, mds);
+ mutex_unlock(&mdsc->mutex);
+ goto out;
+ }
+
+ result = le32_to_cpu(head->result);
+
+ /*
+ * Tolerate 2 consecutive ESTALEs from the same mds.
+ * FIXME: we should be looking at the cap migrate_seq.
+ */
+ if (result == -ESTALE) {
+ req->r_direct_mode = USE_AUTH_MDS;
+ req->r_num_stale++;
+ if (req->r_num_stale <= 2) {
+ __do_request(mdsc, req);
+ mutex_unlock(&mdsc->mutex);
+ goto out;
+ }
+ } else {
+ req->r_num_stale = 0;
+ }
+
+ if (head->safe) {
+ req->r_got_safe = true;
+ __unregister_request(mdsc, req);
+ complete(&req->r_safe_completion);
+
+ if (req->r_got_unsafe) {
+ /*
+ * We already handled the unsafe response, now do the
+ * cleanup. No need to examine the response; the MDS
+ * doesn't include any result info in the safe
+ * response. And even if it did, there is nothing
+ * useful we could do with a revised return value.
+ */
+ dout("got safe reply %llu, mds%d\n", tid, mds);
+ list_del_init(&req->r_unsafe_item);
+
+ /* last unsafe request during umount? */
+ if (mdsc->stopping && !__get_oldest_req(mdsc))
+ complete(&mdsc->safe_umount_waiters);
+ mutex_unlock(&mdsc->mutex);
+ goto out;
+ }
+ }
+
+ BUG_ON(req->r_reply);
+
+ if (!head->safe) {
+ req->r_got_unsafe = true;
+ list_add_tail(&req->r_unsafe_item, &req->r_session->s_unsafe);
+ }
+
+ dout("handle_reply tid %lld result %d\n", tid, result);
+ rinfo = &req->r_reply_info;
+ err = parse_reply_info(msg, rinfo);
+ mutex_unlock(&mdsc->mutex);
+
+ mutex_lock(&session->s_mutex);
+ if (err < 0) {
+ pr_err("mdsc_handle_reply got corrupt reply mds%d\n", mds);
+ ceph_msg_dump(msg);
+ goto out_err;
+ }
+
+ /* snap trace */
+ if (rinfo->snapblob_len) {
+ down_write(&mdsc->snap_rwsem);
+ ceph_update_snap_trace(mdsc, rinfo->snapblob,
+ rinfo->snapblob + rinfo->snapblob_len,
+ le32_to_cpu(head->op) == CEPH_MDS_OP_RMSNAP);
+ downgrade_write(&mdsc->snap_rwsem);
+ } else {
+ down_read(&mdsc->snap_rwsem);
+ }
+
+ /* insert trace into our cache */
+ err = ceph_fill_trace(mdsc->client->sb, req, req->r_session);
+ if (err == 0) {
+ if (result == 0 && rinfo->dir_nr)
+ ceph_readdir_prepopulate(req, req->r_session);
+ ceph_unreserve_caps(&req->r_caps_reservation);
+ }
+
+ up_read(&mdsc->snap_rwsem);
+out_err:
+ if (err) {
+ req->r_err = err;
+ } else {
+ req->r_reply = msg;
+ ceph_msg_get(msg);
+ }
+
+ add_cap_releases(mdsc, req->r_session, -1);
+ mutex_unlock(&session->s_mutex);
+
+ /* kick calling process */
+ complete_request(mdsc, req);
+out:
+ ceph_mdsc_put_request(req);
+ return;
+}
+
+
+
+/*
+ * handle mds notification that our request has been forwarded.
+ */
+static void handle_forward(struct ceph_mds_client *mdsc,
+ struct ceph_mds_session *session,
+ struct ceph_msg *msg)
+{
+ struct ceph_mds_request *req;
+ u64 tid = le64_to_cpu(msg->hdr.tid);
+ u32 next_mds;
+ u32 fwd_seq;
+ int err = -EINVAL;
+ void *p = msg->front.iov_base;
+ void *end = p + msg->front.iov_len;
+
+ ceph_decode_need(&p, end, 2*sizeof(u32), bad);
+ next_mds = ceph_decode_32(&p);
+ fwd_seq = ceph_decode_32(&p);
+
+ mutex_lock(&mdsc->mutex);
+ req = __lookup_request(mdsc, tid);
+ if (!req) {
+ dout("forward %llu to mds%d - req dne\n", tid, next_mds);
+ goto out; /* dup reply? */
+ }
+
+ if (fwd_seq <= req->r_num_fwd) {
+ dout("forward %llu to mds%d - old seq %d <= %d\n",
+ tid, next_mds, req->r_num_fwd, fwd_seq);
+ } else {
+ /* resend. forward race not possible; mds would drop */
+ dout("forward %llu to mds%d (we resend)\n", tid, next_mds);
+ req->r_num_fwd = fwd_seq;
+ req->r_resend_mds = next_mds;
+ put_request_session(req);
+ __do_request(mdsc, req);
+ }
+ ceph_mdsc_put_request(req);
+out:
+ mutex_unlock(&mdsc->mutex);
+ return;
+
+bad:
+ pr_err("mdsc_handle_forward decode error err=%d\n", err);
+}
+
+/*
+ * handle a mds session control message
+ */
+static void handle_session(struct ceph_mds_session *session,
+ struct ceph_msg *msg)
+{
+ struct ceph_mds_client *mdsc = session->s_mdsc;
+ u32 op;
+ u64 seq;
+ int mds = session->s_mds;
+ struct ceph_mds_session_head *h = msg->front.iov_base;
+ int wake = 0;
+
+ /* decode */
+ if (msg->front.iov_len != sizeof(*h))
+ goto bad;
+ op = le32_to_cpu(h->op);
+ seq = le64_to_cpu(h->seq);
+
+ mutex_lock(&mdsc->mutex);
+ if (op == CEPH_SESSION_CLOSE)
+ __unregister_session(mdsc, session);
+ /* FIXME: this ttl calculation is generous */
+ session->s_ttl = jiffies + HZ*mdsc->mdsmap->m_session_autoclose;
+ mutex_unlock(&mdsc->mutex);
+
+ mutex_lock(&session->s_mutex);
+
+ dout("handle_session mds%d %s %p state %s seq %llu\n",
+ mds, ceph_session_op_name(op), session,
+ session_state_name(session->s_state), seq);
+
+ if (session->s_state == CEPH_MDS_SESSION_HUNG) {
+ session->s_state = CEPH_MDS_SESSION_OPEN;
+ pr_info("mds%d came back\n", session->s_mds);
+ }
+
+ switch (op) {
+ case CEPH_SESSION_OPEN:
+ session->s_state = CEPH_MDS_SESSION_OPEN;
+ renewed_caps(mdsc, session, 0);
+ wake = 1;
+ if (mdsc->stopping)
+ __close_session(mdsc, session);
+ break;
+
+ case CEPH_SESSION_RENEWCAPS:
+ if (session->s_renew_seq == seq)
+ renewed_caps(mdsc, session, 1);
+ break;
+
+ case CEPH_SESSION_CLOSE:
+ remove_session_caps(session);
+ wake = 1; /* for good measure */
+ complete(&mdsc->session_close_waiters);
+ kick_requests(mdsc, mds, 0); /* cur only */
+ break;
+
+ case CEPH_SESSION_STALE:
+ pr_info("mds%d caps went stale, renewing\n",
+ session->s_mds);
+ spin_lock(&session->s_cap_lock);
+ session->s_cap_gen++;
+ session->s_cap_ttl = 0;
+ spin_unlock(&session->s_cap_lock);
+ send_renew_caps(mdsc, session);
+ break;
+
+ case CEPH_SESSION_RECALL_STATE:
+ trim_caps(mdsc, session, le32_to_cpu(h->max_caps));
+ break;
+
+ default:
+ pr_err("mdsc_handle_session bad op %d mds%d\n", op, mds);
+ WARN_ON(1);
+ }
+
+ mutex_unlock(&session->s_mutex);
+ if (wake) {
+ mutex_lock(&mdsc->mutex);
+ __wake_requests(mdsc, &session->s_waiting);
+ mutex_unlock(&mdsc->mutex);
+ }
+ return;
+
+bad:
+ pr_err("mdsc_handle_session corrupt message mds%d len %d\n", mds,
+ (int)msg->front.iov_len);
+ ceph_msg_dump(msg);
+ return;
+}
+
+
+/*
+ * called under session->mutex.
+ */
+static void replay_unsafe_requests(struct ceph_mds_client *mdsc,
+ struct ceph_mds_session *session)
+{
+ struct ceph_mds_request *req, *nreq;
+ int err;
+
+ dout("replay_unsafe_requests mds%d\n", session->s_mds);
+
+ mutex_lock(&mdsc->mutex);
+ list_for_each_entry_safe(req, nreq, &session->s_unsafe, r_unsafe_item) {
+ err = __prepare_send_request(mdsc, req, session->s_mds);
+ if (!err) {
+ ceph_msg_get(req->r_request);
+ ceph_con_send(&session->s_con, req->r_request);
+ }
+ }
+ mutex_unlock(&mdsc->mutex);
+}
+
+/*
+ * Encode information about a cap for a reconnect with the MDS.
+ */
+static int encode_caps_cb(struct inode *inode, struct ceph_cap *cap,
+ void *arg)
+{
+ struct ceph_mds_cap_reconnect rec;
+ struct ceph_inode_info *ci;
+ struct ceph_pagelist *pagelist = arg;
+ char *path;
+ int pathlen, err;
+ u64 pathbase;
+ struct dentry *dentry;
+
+ ci = cap->ci;
+
+ dout(" adding %p ino %llx.%llx cap %p %lld %s\n",
+ inode, ceph_vinop(inode), cap, cap->cap_id,
+ ceph_cap_string(cap->issued));
+ err = ceph_pagelist_encode_64(pagelist, ceph_ino(inode));
+ if (err)
+ return err;
+
+ dentry = d_find_alias(inode);
+ if (dentry) {
+ path = ceph_mdsc_build_path(dentry, &pathlen, &pathbase, 0);
+ if (IS_ERR(path)) {
+ err = PTR_ERR(path);
+ BUG_ON(err);
+ }
+ } else {
+ path = NULL;
+ pathlen = 0;
+ }
+ err = ceph_pagelist_encode_string(pagelist, path, pathlen);
+ if (err)
+ goto out;
+
+ spin_lock(&inode->i_lock);
+ cap->seq = 0; /* reset cap seq */
+ cap->issue_seq = 0; /* and issue_seq */
+ rec.cap_id = cpu_to_le64(cap->cap_id);
+ rec.pathbase = cpu_to_le64(pathbase);
+ rec.wanted = cpu_to_le32(__ceph_caps_wanted(ci));
+ rec.issued = cpu_to_le32(cap->issued);
+ rec.size = cpu_to_le64(inode->i_size);
+ ceph_encode_timespec(&rec.mtime, &inode->i_mtime);
+ ceph_encode_timespec(&rec.atime, &inode->i_atime);
+ rec.snaprealm = cpu_to_le64(ci->i_snap_realm->ino);
+ spin_unlock(&inode->i_lock);
+
+ err = ceph_pagelist_append(pagelist, &rec, sizeof(rec));
+
+out:
+ kfree(path);
+ dput(dentry);
+ return err;
+}
+
+
+/*
+ * If an MDS fails and recovers, clients need to reconnect in order to
+ * reestablish shared state. This includes all caps issued through
+ * this session _and_ the snap_realm hierarchy. Because it's not
+ * clear which snap realms the mds cares about, we send everything we
+ * know about.. that ensures we'll then get any new info the
+ * recovering MDS might have.
+ *
+ * This is a relatively heavyweight operation, but it's rare.
+ *
+ * called with mdsc->mutex held.
+ */
+static void send_mds_reconnect(struct ceph_mds_client *mdsc, int mds)
+{
+ struct ceph_mds_session *session = NULL;
+ struct ceph_msg *reply;
+ struct rb_node *p;
+ int err;
+ struct ceph_pagelist *pagelist;
+
+ pr_info("reconnect to recovering mds%d\n", mds);
+
+ pagelist = kmalloc(sizeof(*pagelist), GFP_NOFS);
+ if (!pagelist)
+ goto fail_nopagelist;
+ ceph_pagelist_init(pagelist);
+
+ reply = ceph_msg_new(CEPH_MSG_CLIENT_RECONNECT, 0, 0, 0, NULL);
+ if (IS_ERR(reply)) {
+ err = PTR_ERR(reply);
+ goto fail_nomsg;
+ }
+
+ /* find session */
+ session = __ceph_lookup_mds_session(mdsc, mds);
+ mutex_unlock(&mdsc->mutex); /* drop lock for duration */
+
+ if (session) {
+ mutex_lock(&session->s_mutex);
+
+ session->s_state = CEPH_MDS_SESSION_RECONNECTING;
+ session->s_seq = 0;
+
+ ceph_con_open(&session->s_con,
+ ceph_mdsmap_get_addr(mdsc->mdsmap, mds));
+
+ /* replay unsafe requests */
+ replay_unsafe_requests(mdsc, session);
+ } else {
+ dout("no session for mds%d, will send short reconnect\n",
+ mds);
+ }
+
+ down_read(&mdsc->snap_rwsem);
+
+ if (!session)
+ goto send;
+ dout("session %p state %s\n", session,
+ session_state_name(session->s_state));
+
+ /* traverse this session's caps */
+ err = ceph_pagelist_encode_32(pagelist, session->s_nr_caps);
+ if (err)
+ goto fail;
+ err = iterate_session_caps(session, encode_caps_cb, pagelist);
+ if (err < 0)
+ goto out;
+
+ /*
+ * snaprealms. we provide mds with the ino, seq (version), and
+ * parent for all of our realms. If the mds has any newer info,
+ * it will tell us.
+ */
+ for (p = rb_first(&mdsc->snap_realms); p; p = rb_next(p)) {
+ struct ceph_snap_realm *realm =
+ rb_entry(p, struct ceph_snap_realm, node);
+ struct ceph_mds_snaprealm_reconnect sr_rec;
+
+ dout(" adding snap realm %llx seq %lld parent %llx\n",
+ realm->ino, realm->seq, realm->parent_ino);
+ sr_rec.ino = cpu_to_le64(realm->ino);
+ sr_rec.seq = cpu_to_le64(realm->seq);
+ sr_rec.parent = cpu_to_le64(realm->parent_ino);
+ err = ceph_pagelist_append(pagelist, &sr_rec, sizeof(sr_rec));
+ if (err)
+ goto fail;
+ }
+
+send:
+ reply->pagelist = pagelist;
+ reply->hdr.data_len = cpu_to_le32(pagelist->length);
+ reply->nr_pages = calc_pages_for(0, pagelist->length);
+ ceph_con_send(&session->s_con, reply);
+
+ if (session) {
+ session->s_state = CEPH_MDS_SESSION_OPEN;
+ __wake_requests(mdsc, &session->s_waiting);
+ }
+
+out:
+ up_read(&mdsc->snap_rwsem);
+ if (session) {
+ mutex_unlock(&session->s_mutex);
+ ceph_put_mds_session(session);
+ }
+ mutex_lock(&mdsc->mutex);
+ return;
+
+fail:
+ ceph_msg_put(reply);
+fail_nomsg:
+ ceph_pagelist_release(pagelist);
+ kfree(pagelist);
+fail_nopagelist:
+ pr_err("ENOMEM preparing reconnect for mds%d\n", mds);
+ goto out;
+}
+
+
+/*
+ * compare old and new mdsmaps, kicking requests
+ * and closing out old connections as necessary
+ *
+ * called under mdsc->mutex.
+ */
+static void check_new_map(struct ceph_mds_client *mdsc,
+ struct ceph_mdsmap *newmap,
+ struct ceph_mdsmap *oldmap)
+{
+ int i;
+ int oldstate, newstate;
+ struct ceph_mds_session *s;
+
+ dout("check_new_map new %u old %u\n",
+ newmap->m_epoch, oldmap->m_epoch);
+
+ for (i = 0; i < oldmap->m_max_mds && i < mdsc->max_sessions; i++) {
+ if (mdsc->sessions[i] == NULL)
+ continue;
+ s = mdsc->sessions[i];
+ oldstate = ceph_mdsmap_get_state(oldmap, i);
+ newstate = ceph_mdsmap_get_state(newmap, i);
+
+ dout("check_new_map mds%d state %s -> %s (session %s)\n",
+ i, ceph_mds_state_name(oldstate),
+ ceph_mds_state_name(newstate),
+ session_state_name(s->s_state));
+
+ if (memcmp(ceph_mdsmap_get_addr(oldmap, i),
+ ceph_mdsmap_get_addr(newmap, i),
+ sizeof(struct ceph_entity_addr))) {
+ if (s->s_state == CEPH_MDS_SESSION_OPENING) {
+ /* the session never opened, just close it
+ * out now */
+ __wake_requests(mdsc, &s->s_waiting);
+ __unregister_session(mdsc, s);
+ } else {
+ /* just close it */
+ mutex_unlock(&mdsc->mutex);
+ mutex_lock(&s->s_mutex);
+ mutex_lock(&mdsc->mutex);
+ ceph_con_close(&s->s_con);
+ mutex_unlock(&s->s_mutex);
+ s->s_state = CEPH_MDS_SESSION_RESTARTING;
+ }
+
+ /* kick any requests waiting on the recovering mds */
+ kick_requests(mdsc, i, 1);
+ } else if (oldstate == newstate) {
+ continue; /* nothing new with this mds */
+ }
+
+ /*
+ * send reconnect?
+ */
+ if (s->s_state == CEPH_MDS_SESSION_RESTARTING &&
+ newstate >= CEPH_MDS_STATE_RECONNECT)
+ send_mds_reconnect(mdsc, i);
+
+ /*
+ * kick requests on any mds that has gone active.
+ *
+ * kick requests on cur or forwarder: we may have sent
+ * the request to mds1, mds1 told us it forwarded it
+ * to mds2, but then we learn mds1 failed and can't be
+ * sure it successfully forwarded our request before
+ * it died.
+ */
+ if (oldstate < CEPH_MDS_STATE_ACTIVE &&
+ newstate >= CEPH_MDS_STATE_ACTIVE) {
+ pr_info("mds%d reconnect completed\n", s->s_mds);
+ kick_requests(mdsc, i, 1);
+ ceph_kick_flushing_caps(mdsc, s);
+ wake_up_session_caps(s, 1);
+ }
+ }
+}
+
+
+
+/*
+ * leases
+ */
+
+/*
+ * caller must hold session s_mutex, dentry->d_lock
+ */
+void __ceph_mdsc_drop_dentry_lease(struct dentry *dentry)
+{
+ struct ceph_dentry_info *di = ceph_dentry(dentry);
+
+ ceph_put_mds_session(di->lease_session);
+ di->lease_session = NULL;
+}
+
+static void handle_lease(struct ceph_mds_client *mdsc,
+ struct ceph_mds_session *session,
+ struct ceph_msg *msg)
+{
+ struct super_block *sb = mdsc->client->sb;
+ struct inode *inode;
+ struct ceph_inode_info *ci;
+ struct dentry *parent, *dentry;
+ struct ceph_dentry_info *di;
+ int mds = session->s_mds;
+ struct ceph_mds_lease *h = msg->front.iov_base;
+ struct ceph_vino vino;
+ int mask;
+ struct qstr dname;
+ int release = 0;
+
+ dout("handle_lease from mds%d\n", mds);
+
+ /* decode */
+ if (msg->front.iov_len < sizeof(*h) + sizeof(u32))
+ goto bad;
+ vino.ino = le64_to_cpu(h->ino);
+ vino.snap = CEPH_NOSNAP;
+ mask = le16_to_cpu(h->mask);
+ dname.name = (void *)h + sizeof(*h) + sizeof(u32);
+ dname.len = msg->front.iov_len - sizeof(*h) - sizeof(u32);
+ if (dname.len != get_unaligned_le32(h+1))
+ goto bad;
+
+ mutex_lock(&session->s_mutex);
+ session->s_seq++;
+
+ /* lookup inode */
+ inode = ceph_find_inode(sb, vino);
+ dout("handle_lease '%s', mask %d, ino %llx %p\n",
+ ceph_lease_op_name(h->action), mask, vino.ino, inode);
+ if (inode == NULL) {
+ dout("handle_lease no inode %llx\n", vino.ino);
+ goto release;
+ }
+ ci = ceph_inode(inode);
+
+ /* dentry */
+ parent = d_find_alias(inode);
+ if (!parent) {
+ dout("no parent dentry on inode %p\n", inode);
+ WARN_ON(1);
+ goto release; /* hrm... */
+ }
+ dname.hash = full_name_hash(dname.name, dname.len);
+ dentry = d_lookup(parent, &dname);
+ dput(parent);
+ if (!dentry)
+ goto release;
+
+ spin_lock(&dentry->d_lock);
+ di = ceph_dentry(dentry);
+ switch (h->action) {
+ case CEPH_MDS_LEASE_REVOKE:
+ if (di && di->lease_session == session) {
+ h->seq = cpu_to_le32(di->lease_seq);
+ __ceph_mdsc_drop_dentry_lease(dentry);
+ }
+ release = 1;
+ break;
+
+ case CEPH_MDS_LEASE_RENEW:
+ if (di && di->lease_session == session &&
+ di->lease_gen == session->s_cap_gen &&
+ di->lease_renew_from &&
+ di->lease_renew_after == 0) {
+ unsigned long duration =
+ le32_to_cpu(h->duration_ms) * HZ / 1000;
+
+ di->lease_seq = le32_to_cpu(h->seq);
+ dentry->d_time = di->lease_renew_from + duration;
+ di->lease_renew_after = di->lease_renew_from +
+ (duration >> 1);
+ di->lease_renew_from = 0;
+ }
+ break;
+ }
+ spin_unlock(&dentry->d_lock);
+ dput(dentry);
+
+ if (!release)
+ goto out;
+
+release:
+ /* let's just reuse the same message */
+ h->action = CEPH_MDS_LEASE_REVOKE_ACK;
+ ceph_msg_get(msg);
+ ceph_con_send(&session->s_con, msg);
+
+out:
+ iput(inode);
+ mutex_unlock(&session->s_mutex);
+ return;
+
+bad:
+ pr_err("corrupt lease message\n");
+ ceph_msg_dump(msg);
+}
+
+void ceph_mdsc_lease_send_msg(struct ceph_mds_session *session,
+ struct inode *inode,
+ struct dentry *dentry, char action,
+ u32 seq)
+{
+ struct ceph_msg *msg;
+ struct ceph_mds_lease *lease;
+ int len = sizeof(*lease) + sizeof(u32);
+ int dnamelen = 0;
+
+ dout("lease_send_msg inode %p dentry %p %s to mds%d\n",
+ inode, dentry, ceph_lease_op_name(action), session->s_mds);
+ dnamelen = dentry->d_name.len;
+ len += dnamelen;
+
+ msg = ceph_msg_new(CEPH_MSG_CLIENT_LEASE, len, 0, 0, NULL);
+ if (IS_ERR(msg))
+ return;
+ lease = msg->front.iov_base;
+ lease->action = action;
+ lease->mask = cpu_to_le16(CEPH_LOCK_DN);
+ lease->ino = cpu_to_le64(ceph_vino(inode).ino);
+ lease->first = lease->last = cpu_to_le64(ceph_vino(inode).snap);
+ lease->seq = cpu_to_le32(seq);
+ put_unaligned_le32(dnamelen, lease + 1);
+ memcpy((void *)(lease + 1) + 4, dentry->d_name.name, dnamelen);
+
+ /*
+ * if this is a preemptive lease RELEASE, no need to
+ * flush request stream, since the actual request will
+ * soon follow.
+ */
+ msg->more_to_follow = (action == CEPH_MDS_LEASE_RELEASE);
+
+ ceph_con_send(&session->s_con, msg);
+}
+
+/*
+ * Preemptively release a lease we expect to invalidate anyway.
+ * Pass @inode always, @dentry is optional.
+ */
+void ceph_mdsc_lease_release(struct ceph_mds_client *mdsc, struct inode *inode,
+ struct dentry *dentry, int mask)
+{
+ struct ceph_dentry_info *di;
+ struct ceph_mds_session *session;
+ u32 seq;
+
+ BUG_ON(inode == NULL);
+ BUG_ON(dentry == NULL);
+ BUG_ON(mask != CEPH_LOCK_DN);
+
+ /* is dentry lease valid? */
+ spin_lock(&dentry->d_lock);
+ di = ceph_dentry(dentry);
+ if (!di || !di->lease_session ||
+ di->lease_session->s_mds < 0 ||
+ di->lease_gen != di->lease_session->s_cap_gen ||
+ !time_before(jiffies, dentry->d_time)) {
+ dout("lease_release inode %p dentry %p -- "
+ "no lease on %d\n",
+ inode, dentry, mask);
+ spin_unlock(&dentry->d_lock);
+ return;
+ }
+
+ /* we do have a lease on this dentry; note mds and seq */
+ session = ceph_get_mds_session(di->lease_session);
+ seq = di->lease_seq;
+ __ceph_mdsc_drop_dentry_lease(dentry);
+ spin_unlock(&dentry->d_lock);
+
+ dout("lease_release inode %p dentry %p mask %d to mds%d\n",
+ inode, dentry, mask, session->s_mds);
+ ceph_mdsc_lease_send_msg(session, inode, dentry,
+ CEPH_MDS_LEASE_RELEASE, seq);
+ ceph_put_mds_session(session);
+}
+
+/*
+ * drop all leases (and dentry refs) in preparation for umount
+ */
+static void drop_leases(struct ceph_mds_client *mdsc)
+{
+ int i;
+
+ dout("drop_leases\n");
+ mutex_lock(&mdsc->mutex);
+ for (i = 0; i < mdsc->max_sessions; i++) {
+ struct ceph_mds_session *s = __ceph_lookup_mds_session(mdsc, i);
+ if (!s)
+ continue;
+ mutex_unlock(&mdsc->mutex);
+ mutex_lock(&s->s_mutex);
+ mutex_unlock(&s->s_mutex);
+ ceph_put_mds_session(s);
+ mutex_lock(&mdsc->mutex);
+ }
+ mutex_unlock(&mdsc->mutex);
+}
+
+
+
+/*
+ * delayed work -- periodically trim expired leases, renew caps with mds
+ */
+static void schedule_delayed(struct ceph_mds_client *mdsc)
+{
+ int delay = 5;
+ unsigned hz = round_jiffies_relative(HZ * delay);
+ schedule_delayed_work(&mdsc->delayed_work, hz);
+}
+
+static void delayed_work(struct work_struct *work)
+{
+ int i;
+ struct ceph_mds_client *mdsc =
+ container_of(work, struct ceph_mds_client, delayed_work.work);
+ int renew_interval;
+ int renew_caps;
+
+ dout("mdsc delayed_work\n");
+ ceph_check_delayed_caps(mdsc);
+
+ mutex_lock(&mdsc->mutex);
+ renew_interval = mdsc->mdsmap->m_session_timeout >> 2;
+ renew_caps = time_after_eq(jiffies, HZ*renew_interval +
+ mdsc->last_renew_caps);
+ if (renew_caps)
+ mdsc->last_renew_caps = jiffies;
+
+ for (i = 0; i < mdsc->max_sessions; i++) {
+ struct ceph_mds_session *s = __ceph_lookup_mds_session(mdsc, i);
+ if (s == NULL)
+ continue;
+ if (s->s_state == CEPH_MDS_SESSION_CLOSING) {
+ dout("resending session close request for mds%d\n",
+ s->s_mds);
+ request_close_session(mdsc, s);
+ ceph_put_mds_session(s);
+ continue;
+ }
+ if (s->s_ttl && time_after(jiffies, s->s_ttl)) {
+ if (s->s_state == CEPH_MDS_SESSION_OPEN) {
+ s->s_state = CEPH_MDS_SESSION_HUNG;
+ pr_info("mds%d hung\n", s->s_mds);
+ }
+ }
+ if (s->s_state < CEPH_MDS_SESSION_OPEN) {
+ /* this mds is failed or recovering, just wait */
+ ceph_put_mds_session(s);
+ continue;
+ }
+ mutex_unlock(&mdsc->mutex);
+
+ mutex_lock(&s->s_mutex);
+ if (renew_caps)
+ send_renew_caps(mdsc, s);
+ else
+ ceph_con_keepalive(&s->s_con);
+ add_cap_releases(mdsc, s, -1);
+ send_cap_releases(mdsc, s);
+ mutex_unlock(&s->s_mutex);
+ ceph_put_mds_session(s);
+
+ mutex_lock(&mdsc->mutex);
+ }
+ mutex_unlock(&mdsc->mutex);
+
+ schedule_delayed(mdsc);
+}
+
+
+int ceph_mdsc_init(struct ceph_mds_client *mdsc, struct ceph_client *client)
+{
+ mdsc->client = client;
+ mutex_init(&mdsc->mutex);
+ mdsc->mdsmap = kzalloc(sizeof(*mdsc->mdsmap), GFP_NOFS);
+ init_completion(&mdsc->safe_umount_waiters);
+ init_completion(&mdsc->session_close_waiters);
+ INIT_LIST_HEAD(&mdsc->waiting_for_map);
+ mdsc->sessions = NULL;
+ mdsc->max_sessions = 0;
+ mdsc->stopping = 0;
+ init_rwsem(&mdsc->snap_rwsem);
+ mdsc->snap_realms = RB_ROOT;
+ INIT_LIST_HEAD(&mdsc->snap_empty);
+ spin_lock_init(&mdsc->snap_empty_lock);
+ mdsc->last_tid = 0;
+ mdsc->request_tree = RB_ROOT;
+ INIT_DELAYED_WORK(&mdsc->delayed_work, delayed_work);
+ mdsc->last_renew_caps = jiffies;
+ INIT_LIST_HEAD(&mdsc->cap_delay_list);
+ spin_lock_init(&mdsc->cap_delay_lock);
+ INIT_LIST_HEAD(&mdsc->snap_flush_list);
+ spin_lock_init(&mdsc->snap_flush_lock);
+ mdsc->cap_flush_seq = 0;
+ INIT_LIST_HEAD(&mdsc->cap_dirty);
+ mdsc->num_cap_flushing = 0;
+ spin_lock_init(&mdsc->cap_dirty_lock);
+ init_waitqueue_head(&mdsc->cap_flushing_wq);
+ spin_lock_init(&mdsc->dentry_lru_lock);
+ INIT_LIST_HEAD(&mdsc->dentry_lru);
+ return 0;
+}
+
+/*
+ * Wait for safe replies on open mds requests. If we time out, drop
+ * all requests from the tree to avoid dangling dentry refs.
+ */
+static void wait_requests(struct ceph_mds_client *mdsc)
+{
+ struct ceph_mds_request *req;
+ struct ceph_client *client = mdsc->client;
+
+ mutex_lock(&mdsc->mutex);
+ if (__get_oldest_req(mdsc)) {
+ mutex_unlock(&mdsc->mutex);
+
+ dout("wait_requests waiting for requests\n");
+ wait_for_completion_timeout(&mdsc->safe_umount_waiters,
+ client->mount_args->mount_timeout * HZ);
+
+ /* tear down remaining requests */
+ mutex_lock(&mdsc->mutex);
+ while ((req = __get_oldest_req(mdsc))) {
+ dout("wait_requests timed out on tid %llu\n",
+ req->r_tid);
+ __unregister_request(mdsc, req);
+ }
+ }
+ mutex_unlock(&mdsc->mutex);
+ dout("wait_requests done\n");
+}
+
+/*
+ * called before mount is ro, and before dentries are torn down.
+ * (hmm, does this still race with new lookups?)
+ */
+void ceph_mdsc_pre_umount(struct ceph_mds_client *mdsc)
+{
+ dout("pre_umount\n");
+ mdsc->stopping = 1;
+
+ drop_leases(mdsc);
+ ceph_flush_dirty_caps(mdsc);
+ wait_requests(mdsc);
+}
+
+/*
+ * wait for all write mds requests to flush.
+ */
+static void wait_unsafe_requests(struct ceph_mds_client *mdsc, u64 want_tid)
+{
+ struct ceph_mds_request *req = NULL, *nextreq;
+ struct rb_node *n;
+
+ mutex_lock(&mdsc->mutex);
+ dout("wait_unsafe_requests want %lld\n", want_tid);
+restart:
+ req = __get_oldest_req(mdsc);
+ while (req && req->r_tid <= want_tid) {
+ /* find next request */
+ n = rb_next(&req->r_node);
+ if (n)
+ nextreq = rb_entry(n, struct ceph_mds_request, r_node);
+ else
+ nextreq = NULL;
+ if ((req->r_op & CEPH_MDS_OP_WRITE)) {
+ /* write op */
+ ceph_mdsc_get_request(req);
+ if (nextreq)
+ ceph_mdsc_get_request(nextreq);
+ mutex_unlock(&mdsc->mutex);
+ dout("wait_unsafe_requests wait on %llu (want %llu)\n",
+ req->r_tid, want_tid);
+ wait_for_completion(&req->r_safe_completion);
+ mutex_lock(&mdsc->mutex);
+ ceph_mdsc_put_request(req);
+ if (!nextreq)
+ break; /* next dne before, so we're done! */
+ if (RB_EMPTY_NODE(&nextreq->r_node)) {
+ /* next request was removed from tree */
+ ceph_mdsc_put_request(nextreq);
+ goto restart;
+ }
+ ceph_mdsc_put_request(nextreq); /* won't go away */
+ }
+ req = nextreq;
+ }
+ mutex_unlock(&mdsc->mutex);
+ dout("wait_unsafe_requests done\n");
+}
+
+void ceph_mdsc_sync(struct ceph_mds_client *mdsc)
+{
+ u64 want_tid, want_flush;
+
+ dout("sync\n");
+ mutex_lock(&mdsc->mutex);
+ want_tid = mdsc->last_tid;
+ want_flush = mdsc->cap_flush_seq;
+ mutex_unlock(&mdsc->mutex);
+ dout("sync want tid %lld flush_seq %lld\n", want_tid, want_flush);
+
+ ceph_flush_dirty_caps(mdsc);
+
+ wait_unsafe_requests(mdsc, want_tid);
+ wait_event(mdsc->cap_flushing_wq, check_cap_flush(mdsc, want_flush));
+}
+
+
+/*
+ * called after sb is ro.
+ */
+void ceph_mdsc_close_sessions(struct ceph_mds_client *mdsc)
+{
+ struct ceph_mds_session *session;
+ int i;
+ int n;
+ struct ceph_client *client = mdsc->client;
+ unsigned long started, timeout = client->mount_args->mount_timeout * HZ;
+
+ dout("close_sessions\n");
+
+ mutex_lock(&mdsc->mutex);
+
+ /* close sessions */
+ started = jiffies;
+ while (time_before(jiffies, started + timeout)) {
+ dout("closing sessions\n");
+ n = 0;
+ for (i = 0; i < mdsc->max_sessions; i++) {
+ session = __ceph_lookup_mds_session(mdsc, i);
+ if (!session)
+ continue;
+ mutex_unlock(&mdsc->mutex);
+ mutex_lock(&session->s_mutex);
+ __close_session(mdsc, session);
+ mutex_unlock(&session->s_mutex);
+ ceph_put_mds_session(session);
+ mutex_lock(&mdsc->mutex);
+ n++;
+ }
+ if (n == 0)
+ break;
+
+ if (client->mount_state == CEPH_MOUNT_SHUTDOWN)
+ break;
+
+ dout("waiting for sessions to close\n");
+ mutex_unlock(&mdsc->mutex);
+ wait_for_completion_timeout(&mdsc->session_close_waiters,
+ timeout);
+ mutex_lock(&mdsc->mutex);
+ }
+
+ /* tear down remaining sessions */
+ for (i = 0; i < mdsc->max_sessions; i++) {
+ if (mdsc->sessions[i]) {
+ session = get_session(mdsc->sessions[i]);
+ __unregister_session(mdsc, session);
+ mutex_unlock(&mdsc->mutex);
+ mutex_lock(&session->s_mutex);
+ remove_session_caps(session);
+ mutex_unlock(&session->s_mutex);
+ ceph_put_mds_session(session);
+ mutex_lock(&mdsc->mutex);
+ }
+ }
+
+ WARN_ON(!list_empty(&mdsc->cap_delay_list));
+
+ mutex_unlock(&mdsc->mutex);
+
+ ceph_cleanup_empty_realms(mdsc);
+
+ cancel_delayed_work_sync(&mdsc->delayed_work); /* cancel timer */
+
+ dout("stopped\n");
+}
+
+void ceph_mdsc_stop(struct ceph_mds_client *mdsc)
+{
+ dout("stop\n");
+ cancel_delayed_work_sync(&mdsc->delayed_work); /* cancel timer */
+ if (mdsc->mdsmap)
+ ceph_mdsmap_destroy(mdsc->mdsmap);
+ kfree(mdsc->sessions);
+}
+
+
+/*
+ * handle mds map update.
+ */
+void ceph_mdsc_handle_map(struct ceph_mds_client *mdsc, struct ceph_msg *msg)
+{
+ u32 epoch;
+ u32 maplen;
+ void *p = msg->front.iov_base;
+ void *end = p + msg->front.iov_len;
+ struct ceph_mdsmap *newmap, *oldmap;
+ struct ceph_fsid fsid;
+ int err = -EINVAL;
+
+ ceph_decode_need(&p, end, sizeof(fsid)+2*sizeof(u32), bad);
+ ceph_decode_copy(&p, &fsid, sizeof(fsid));
+ if (ceph_check_fsid(mdsc->client, &fsid) < 0)
+ return;
+ epoch = ceph_decode_32(&p);
+ maplen = ceph_decode_32(&p);
+ dout("handle_map epoch %u len %d\n", epoch, (int)maplen);
+
+ /* do we need it? */
+ ceph_monc_got_mdsmap(&mdsc->client->monc, epoch);
+ mutex_lock(&mdsc->mutex);
+ if (mdsc->mdsmap && epoch <= mdsc->mdsmap->m_epoch) {
+ dout("handle_map epoch %u <= our %u\n",
+ epoch, mdsc->mdsmap->m_epoch);
+ mutex_unlock(&mdsc->mutex);
+ return;
+ }
+
+ newmap = ceph_mdsmap_decode(&p, end);
+ if (IS_ERR(newmap)) {
+ err = PTR_ERR(newmap);
+ goto bad_unlock;
+ }
+
+ /* swap into place */
+ if (mdsc->mdsmap) {
+ oldmap = mdsc->mdsmap;
+ mdsc->mdsmap = newmap;
+ check_new_map(mdsc, newmap, oldmap);
+ ceph_mdsmap_destroy(oldmap);
+ } else {
+ mdsc->mdsmap = newmap; /* first mds map */
+ }
+ mdsc->client->sb->s_maxbytes = mdsc->mdsmap->m_max_file_size;
+
+ __wake_requests(mdsc, &mdsc->waiting_for_map);
+
+ mutex_unlock(&mdsc->mutex);
+ schedule_delayed(mdsc);
+ return;
+
+bad_unlock:
+ mutex_unlock(&mdsc->mutex);
+bad:
+ pr_err("error decoding mdsmap %d\n", err);
+ return;
+}
+
+static struct ceph_connection *con_get(struct ceph_connection *con)
+{
+ struct ceph_mds_session *s = con->private;
+
+ if (get_session(s)) {
+ dout("mdsc con_get %p ok (%d)\n", s, atomic_read(&s->s_ref));
+ return con;
+ }
+ dout("mdsc con_get %p FAIL\n", s);
+ return NULL;
+}
+
+static void con_put(struct ceph_connection *con)
+{
+ struct ceph_mds_session *s = con->private;
+
+ ceph_put_mds_session(s);
+ dout("mdsc con_put %p (%d)\n", s, atomic_read(&s->s_ref));
+}
+
+/*
+ * if the client is unresponsive for long enough, the mds will kill
+ * the session entirely.
+ */
+static void peer_reset(struct ceph_connection *con)
+{
+ struct ceph_mds_session *s = con->private;
+
+ pr_err("mds%d gave us the boot. IMPLEMENT RECONNECT.\n",
+ s->s_mds);
+}
+
+static void dispatch(struct ceph_connection *con, struct ceph_msg *msg)
+{
+ struct ceph_mds_session *s = con->private;
+ struct ceph_mds_client *mdsc = s->s_mdsc;
+ int type = le16_to_cpu(msg->hdr.type);
+
+ mutex_lock(&mdsc->mutex);
+ if (__verify_registered_session(mdsc, s) < 0) {
+ mutex_unlock(&mdsc->mutex);
+ goto out;
+ }
+ mutex_unlock(&mdsc->mutex);
+
+ switch (type) {
+ case CEPH_MSG_MDS_MAP:
+ ceph_mdsc_handle_map(mdsc, msg);
+ break;
+ case CEPH_MSG_CLIENT_SESSION:
+ handle_session(s, msg);
+ break;
+ case CEPH_MSG_CLIENT_REPLY:
+ handle_reply(s, msg);
+ break;
+ case CEPH_MSG_CLIENT_REQUEST_FORWARD:
+ handle_forward(mdsc, s, msg);
+ break;
+ case CEPH_MSG_CLIENT_CAPS:
+ ceph_handle_caps(s, msg);
+ break;
+ case CEPH_MSG_CLIENT_SNAP:
+ ceph_handle_snap(mdsc, s, msg);
+ break;
+ case CEPH_MSG_CLIENT_LEASE:
+ handle_lease(mdsc, s, msg);
+ break;
+
+ default:
+ pr_err("received unknown message type %d %s\n", type,
+ ceph_msg_type_name(type));
+ }
+out:
+ ceph_msg_put(msg);
+}
+
+/*
+ * authentication
+ */
+static int get_authorizer(struct ceph_connection *con,
+ void **buf, int *len, int *proto,
+ void **reply_buf, int *reply_len, int force_new)
+{
+ struct ceph_mds_session *s = con->private;
+ struct ceph_mds_client *mdsc = s->s_mdsc;
+ struct ceph_auth_client *ac = mdsc->client->monc.auth;
+ int ret = 0;
+
+ if (force_new && s->s_authorizer) {
+ ac->ops->destroy_authorizer(ac, s->s_authorizer);
+ s->s_authorizer = NULL;
+ }
+ if (s->s_authorizer == NULL) {
+ if (ac->ops->create_authorizer) {
+ ret = ac->ops->create_authorizer(
+ ac, CEPH_ENTITY_TYPE_MDS,
+ &s->s_authorizer,
+ &s->s_authorizer_buf,
+ &s->s_authorizer_buf_len,
+ &s->s_authorizer_reply_buf,
+ &s->s_authorizer_reply_buf_len);
+ if (ret)
+ return ret;
+ }
+ }
+
+ *proto = ac->protocol;
+ *buf = s->s_authorizer_buf;
+ *len = s->s_authorizer_buf_len;
+ *reply_buf = s->s_authorizer_reply_buf;
+ *reply_len = s->s_authorizer_reply_buf_len;
+ return 0;
+}
+
+
+static int verify_authorizer_reply(struct ceph_connection *con, int len)
+{
+ struct ceph_mds_session *s = con->private;
+ struct ceph_mds_client *mdsc = s->s_mdsc;
+ struct ceph_auth_client *ac = mdsc->client->monc.auth;
+
+ return ac->ops->verify_authorizer_reply(ac, s->s_authorizer, len);
+}
+
+static int invalidate_authorizer(struct ceph_connection *con)
+{
+ struct ceph_mds_session *s = con->private;
+ struct ceph_mds_client *mdsc = s->s_mdsc;
+ struct ceph_auth_client *ac = mdsc->client->monc.auth;
+
+ if (ac->ops->invalidate_authorizer)
+ ac->ops->invalidate_authorizer(ac, CEPH_ENTITY_TYPE_MDS);
+
+ return ceph_monc_validate_auth(&mdsc->client->monc);
+}
+
+const static struct ceph_connection_operations mds_con_ops = {
+ .get = con_get,
+ .put = con_put,
+ .dispatch = dispatch,
+ .get_authorizer = get_authorizer,
+ .verify_authorizer_reply = verify_authorizer_reply,
+ .invalidate_authorizer = invalidate_authorizer,
+ .peer_reset = peer_reset,
+};
+
+
+
+
+/* eof */
--- /dev/null
+#ifndef _FS_CEPH_MDS_CLIENT_H
+#define _FS_CEPH_MDS_CLIENT_H
+
+#include <linux/completion.h>
+#include <linux/kref.h>
+#include <linux/list.h>
+#include <linux/mutex.h>
+#include <linux/rbtree.h>
+#include <linux/spinlock.h>
+
+#include "types.h"
+#include "messenger.h"
+#include "mdsmap.h"
+
+/*
+ * Some lock dependencies:
+ *
+ * session->s_mutex
+ * mdsc->mutex
+ *
+ * mdsc->snap_rwsem
+ *
+ * inode->i_lock
+ * mdsc->snap_flush_lock
+ * mdsc->cap_delay_lock
+ *
+ */
+
+struct ceph_client;
+struct ceph_cap;
+
+/*
+ * parsed info about a single inode. pointers are into the encoded
+ * on-wire structures within the mds reply message payload.
+ */
+struct ceph_mds_reply_info_in {
+ struct ceph_mds_reply_inode *in;
+ u32 symlink_len;
+ char *symlink;
+ u32 xattr_len;
+ char *xattr_data;
+};
+
+/*
+ * parsed info about an mds reply, including information about the
+ * target inode and/or its parent directory and dentry, and directory
+ * contents (for readdir results).
+ */
+struct ceph_mds_reply_info_parsed {
+ struct ceph_mds_reply_head *head;
+
+ struct ceph_mds_reply_info_in diri, targeti;
+ struct ceph_mds_reply_dirfrag *dirfrag;
+ char *dname;
+ u32 dname_len;
+ struct ceph_mds_reply_lease *dlease;
+
+ struct ceph_mds_reply_dirfrag *dir_dir;
+ int dir_nr;
+ char **dir_dname;
+ u32 *dir_dname_len;
+ struct ceph_mds_reply_lease **dir_dlease;
+ struct ceph_mds_reply_info_in *dir_in;
+ u8 dir_complete, dir_end;
+
+ /* encoded blob describing snapshot contexts for certain
+ operations (e.g., open) */
+ void *snapblob;
+ int snapblob_len;
+};
+
+
+/*
+ * cap releases are batched and sent to the MDS en masse.
+ */
+#define CEPH_CAPS_PER_RELEASE ((PAGE_CACHE_SIZE - \
+ sizeof(struct ceph_mds_cap_release)) / \
+ sizeof(struct ceph_mds_cap_item))
+
+
+/*
+ * state associated with each MDS<->client session
+ */
+enum {
+ CEPH_MDS_SESSION_NEW = 1,
+ CEPH_MDS_SESSION_OPENING = 2,
+ CEPH_MDS_SESSION_OPEN = 3,
+ CEPH_MDS_SESSION_HUNG = 4,
+ CEPH_MDS_SESSION_CLOSING = 5,
+ CEPH_MDS_SESSION_RESTARTING = 6,
+ CEPH_MDS_SESSION_RECONNECTING = 7,
+};
+
+struct ceph_mds_session {
+ struct ceph_mds_client *s_mdsc;
+ int s_mds;
+ int s_state;
+ unsigned long s_ttl; /* time until mds kills us */
+ u64 s_seq; /* incoming msg seq # */
+ struct mutex s_mutex; /* serialize session messages */
+
+ struct ceph_connection s_con;
+
+ struct ceph_authorizer *s_authorizer;
+ void *s_authorizer_buf, *s_authorizer_reply_buf;
+ size_t s_authorizer_buf_len, s_authorizer_reply_buf_len;
+
+ /* protected by s_cap_lock */
+ spinlock_t s_cap_lock;
+ u32 s_cap_gen; /* inc each time we get mds stale msg */
+ unsigned long s_cap_ttl; /* when session caps expire */
+ struct list_head s_caps; /* all caps issued by this session */
+ int s_nr_caps, s_trim_caps;
+ int s_num_cap_releases;
+ struct list_head s_cap_releases; /* waiting cap_release messages */
+ struct list_head s_cap_releases_done; /* ready to send */
+ struct ceph_cap *s_cap_iterator;
+
+ /* protected by mutex */
+ struct list_head s_cap_flushing; /* inodes w/ flushing caps */
+ struct list_head s_cap_snaps_flushing;
+ unsigned long s_renew_requested; /* last time we sent a renew req */
+ u64 s_renew_seq;
+
+ atomic_t s_ref;
+ struct list_head s_waiting; /* waiting requests */
+ struct list_head s_unsafe; /* unsafe requests */
+};
+
+/*
+ * modes of choosing which MDS to send a request to
+ */
+enum {
+ USE_ANY_MDS,
+ USE_RANDOM_MDS,
+ USE_AUTH_MDS, /* prefer authoritative mds for this metadata item */
+};
+
+struct ceph_mds_request;
+struct ceph_mds_client;
+
+/*
+ * request completion callback
+ */
+typedef void (*ceph_mds_request_callback_t) (struct ceph_mds_client *mdsc,
+ struct ceph_mds_request *req);
+
+/*
+ * an in-flight mds request
+ */
+struct ceph_mds_request {
+ u64 r_tid; /* transaction id */
+ struct rb_node r_node;
+
+ int r_op; /* mds op code */
+ int r_mds;
+
+ /* operation on what? */
+ struct inode *r_inode; /* arg1 */
+ struct dentry *r_dentry; /* arg1 */
+ struct dentry *r_old_dentry; /* arg2: rename from or link from */
+ char *r_path1, *r_path2;
+ struct ceph_vino r_ino1, r_ino2;
+
+ struct inode *r_locked_dir; /* dir (if any) i_mutex locked by vfs */
+ struct inode *r_target_inode; /* resulting inode */
+
+ union ceph_mds_request_args r_args;
+ int r_fmode; /* file mode, if expecting cap */
+
+ /* for choosing which mds to send this request to */
+ int r_direct_mode;
+ u32 r_direct_hash; /* choose dir frag based on this dentry hash */
+ bool r_direct_is_hash; /* true if r_direct_hash is valid */
+
+ /* data payload is used for xattr ops */
+ struct page **r_pages;
+ int r_num_pages;
+ int r_data_len;
+
+ /* what caps shall we drop? */
+ int r_inode_drop, r_inode_unless;
+ int r_dentry_drop, r_dentry_unless;
+ int r_old_dentry_drop, r_old_dentry_unless;
+ struct inode *r_old_inode;
+ int r_old_inode_drop, r_old_inode_unless;
+
+ struct ceph_msg *r_request; /* original request */
+ struct ceph_msg *r_reply;
+ struct ceph_mds_reply_info_parsed r_reply_info;
+ int r_err;
+ bool r_aborted;
+
+ unsigned long r_timeout; /* optional. jiffies */
+ unsigned long r_started; /* start time to measure timeout against */
+ unsigned long r_request_started; /* start time for mds request only,
+ used to measure lease durations */
+
+ /* link unsafe requests to parent directory, for fsync */
+ struct inode *r_unsafe_dir;
+ struct list_head r_unsafe_dir_item;
+
+ struct ceph_mds_session *r_session;
+
+ int r_attempts; /* resend attempts */
+ int r_num_fwd; /* number of forward attempts */
+ int r_num_stale;
+ int r_resend_mds; /* mds to resend to next, if any*/
+
+ struct kref r_kref;
+ struct list_head r_wait;
+ struct completion r_completion;
+ struct completion r_safe_completion;
+ ceph_mds_request_callback_t r_callback;
+ struct list_head r_unsafe_item; /* per-session unsafe list item */
+ bool r_got_unsafe, r_got_safe;
+
+ bool r_did_prepopulate;
+ u32 r_readdir_offset;
+
+ struct ceph_cap_reservation r_caps_reservation;
+ int r_num_caps;
+};
+
+/*
+ * mds client state
+ */
+struct ceph_mds_client {
+ struct ceph_client *client;
+ struct mutex mutex; /* all nested structures */
+
+ struct ceph_mdsmap *mdsmap;
+ struct completion safe_umount_waiters, session_close_waiters;
+ struct list_head waiting_for_map;
+
+ struct ceph_mds_session **sessions; /* NULL for mds if no session */
+ int max_sessions; /* len of s_mds_sessions */
+ int stopping; /* true if shutting down */
+
+ /*
+ * snap_rwsem will cover cap linkage into snaprealms, and
+ * realm snap contexts. (later, we can do per-realm snap
+ * contexts locks..) the empty list contains realms with no
+ * references (implying they contain no inodes with caps) that
+ * should be destroyed.
+ */
+ struct rw_semaphore snap_rwsem;
+ struct rb_root snap_realms;
+ struct list_head snap_empty;
+ spinlock_t snap_empty_lock; /* protect snap_empty */
+
+ u64 last_tid; /* most recent mds request */
+ struct rb_root request_tree; /* pending mds requests */
+ struct delayed_work delayed_work; /* delayed work */
+ unsigned long last_renew_caps; /* last time we renewed our caps */
+ struct list_head cap_delay_list; /* caps with delayed release */
+ spinlock_t cap_delay_lock; /* protects cap_delay_list */
+ struct list_head snap_flush_list; /* cap_snaps ready to flush */
+ spinlock_t snap_flush_lock;
+
+ u64 cap_flush_seq;
+ struct list_head cap_dirty; /* inodes with dirty caps */
+ int num_cap_flushing; /* # caps we are flushing */
+ spinlock_t cap_dirty_lock; /* protects above items */
+ wait_queue_head_t cap_flushing_wq;
+
+#ifdef CONFIG_DEBUG_FS
+ struct dentry *debugfs_file;
+#endif
+
+ spinlock_t dentry_lru_lock;
+ struct list_head dentry_lru;
+ int num_dentry;
+};
+
+extern const char *ceph_mds_op_name(int op);
+
+extern struct ceph_mds_session *
+__ceph_lookup_mds_session(struct ceph_mds_client *, int mds);
+
+static inline struct ceph_mds_session *
+ceph_get_mds_session(struct ceph_mds_session *s)
+{
+ atomic_inc(&s->s_ref);
+ return s;
+}
+
+extern void ceph_put_mds_session(struct ceph_mds_session *s);
+
+extern int ceph_send_msg_mds(struct ceph_mds_client *mdsc,
+ struct ceph_msg *msg, int mds);
+
+extern int ceph_mdsc_init(struct ceph_mds_client *mdsc,
+ struct ceph_client *client);
+extern void ceph_mdsc_close_sessions(struct ceph_mds_client *mdsc);
+extern void ceph_mdsc_stop(struct ceph_mds_client *mdsc);
+
+extern void ceph_mdsc_sync(struct ceph_mds_client *mdsc);
+
+extern void ceph_mdsc_lease_release(struct ceph_mds_client *mdsc,
+ struct inode *inode,
+ struct dentry *dn, int mask);
+
+extern struct ceph_mds_request *
+ceph_mdsc_create_request(struct ceph_mds_client *mdsc, int op, int mode);
+extern void ceph_mdsc_submit_request(struct ceph_mds_client *mdsc,
+ struct ceph_mds_request *req);
+extern int ceph_mdsc_do_request(struct ceph_mds_client *mdsc,
+ struct inode *dir,
+ struct ceph_mds_request *req);
+static inline void ceph_mdsc_get_request(struct ceph_mds_request *req)
+{
+ kref_get(&req->r_kref);
+}
+extern void ceph_mdsc_release_request(struct kref *kref);
+static inline void ceph_mdsc_put_request(struct ceph_mds_request *req)
+{
+ kref_put(&req->r_kref, ceph_mdsc_release_request);
+}
+
+extern void ceph_mdsc_pre_umount(struct ceph_mds_client *mdsc);
+
+extern char *ceph_mdsc_build_path(struct dentry *dentry, int *plen, u64 *base,
+ int stop_on_nosnap);
+
+extern void __ceph_mdsc_drop_dentry_lease(struct dentry *dentry);
+extern void ceph_mdsc_lease_send_msg(struct ceph_mds_session *session,
+ struct inode *inode,
+ struct dentry *dentry, char action,
+ u32 seq);
+
+extern void ceph_mdsc_handle_map(struct ceph_mds_client *mdsc,
+ struct ceph_msg *msg);
+
+#endif
--- /dev/null
+#include "ceph_debug.h"
+
+#include <linux/bug.h>
+#include <linux/err.h>
+#include <linux/random.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+
+#include "mdsmap.h"
+#include "messenger.h"
+#include "decode.h"
+
+#include "super.h"
+
+
+/*
+ * choose a random mds that is "up" (i.e. has a state > 0), or -1.
+ */
+int ceph_mdsmap_get_random_mds(struct ceph_mdsmap *m)
+{
+ int n = 0;
+ int i;
+ char r;
+
+ /* count */
+ for (i = 0; i < m->m_max_mds; i++)
+ if (m->m_info[i].state > 0)
+ n++;
+ if (n == 0)
+ return -1;
+
+ /* pick */
+ get_random_bytes(&r, 1);
+ n = r % n;
+ i = 0;
+ for (i = 0; n > 0; i++, n--)
+ while (m->m_info[i].state <= 0)
+ i++;
+
+ return i;
+}
+
+/*
+ * Decode an MDS map
+ *
+ * Ignore any fields we don't care about (there are quite a few of
+ * them).
+ */
+struct ceph_mdsmap *ceph_mdsmap_decode(void **p, void *end)
+{
+ struct ceph_mdsmap *m;
+ const void *start = *p;
+ int i, j, n;
+ int err = -EINVAL;
+ u16 version;
+
+ m = kzalloc(sizeof(*m), GFP_NOFS);
+ if (m == NULL)
+ return ERR_PTR(-ENOMEM);
+
+ ceph_decode_16_safe(p, end, version, bad);
+
+ ceph_decode_need(p, end, 8*sizeof(u32) + sizeof(u64), bad);
+ m->m_epoch = ceph_decode_32(p);
+ m->m_client_epoch = ceph_decode_32(p);
+ m->m_last_failure = ceph_decode_32(p);
+ m->m_root = ceph_decode_32(p);
+ m->m_session_timeout = ceph_decode_32(p);
+ m->m_session_autoclose = ceph_decode_32(p);
+ m->m_max_file_size = ceph_decode_64(p);
+ m->m_max_mds = ceph_decode_32(p);
+
+ m->m_info = kcalloc(m->m_max_mds, sizeof(*m->m_info), GFP_NOFS);
+ if (m->m_info == NULL)
+ goto badmem;
+
+ /* pick out active nodes from mds_info (state > 0) */
+ n = ceph_decode_32(p);
+ for (i = 0; i < n; i++) {
+ u64 global_id;
+ u32 namelen;
+ s32 mds, inc, state;
+ u64 state_seq;
+ u8 infoversion;
+ struct ceph_entity_addr addr;
+ u32 num_export_targets;
+ void *pexport_targets = NULL;
+
+ ceph_decode_need(p, end, sizeof(u64)*2 + 1 + sizeof(u32), bad);
+ global_id = ceph_decode_64(p);
+ infoversion = ceph_decode_8(p);
+ *p += sizeof(u64);
+ namelen = ceph_decode_32(p); /* skip mds name */
+ *p += namelen;
+
+ ceph_decode_need(p, end,
+ 4*sizeof(u32) + sizeof(u64) +
+ sizeof(addr) + sizeof(struct ceph_timespec),
+ bad);
+ mds = ceph_decode_32(p);
+ inc = ceph_decode_32(p);
+ state = ceph_decode_32(p);
+ state_seq = ceph_decode_64(p);
+ ceph_decode_copy(p, &addr, sizeof(addr));
+ ceph_decode_addr(&addr);
+ *p += sizeof(struct ceph_timespec);
+ *p += sizeof(u32);
+ ceph_decode_32_safe(p, end, namelen, bad);
+ *p += namelen;
+ if (infoversion >= 2) {
+ ceph_decode_32_safe(p, end, num_export_targets, bad);
+ pexport_targets = *p;
+ *p += num_export_targets * sizeof(u32);
+ } else {
+ num_export_targets = 0;
+ }
+
+ dout("mdsmap_decode %d/%d %lld mds%d.%d %s %s\n",
+ i+1, n, global_id, mds, inc, pr_addr(&addr.in_addr),
+ ceph_mds_state_name(state));
+ if (mds >= 0 && mds < m->m_max_mds && state > 0) {
+ m->m_info[mds].global_id = global_id;
+ m->m_info[mds].state = state;
+ m->m_info[mds].addr = addr;
+ m->m_info[mds].num_export_targets = num_export_targets;
+ if (num_export_targets) {
+ m->m_info[mds].export_targets =
+ kcalloc(num_export_targets, sizeof(u32),
+ GFP_NOFS);
+ for (j = 0; j < num_export_targets; j++)
+ m->m_info[mds].export_targets[j] =
+ ceph_decode_32(&pexport_targets);
+ } else {
+ m->m_info[mds].export_targets = NULL;
+ }
+ }
+ }
+
+ /* pg_pools */
+ ceph_decode_32_safe(p, end, n, bad);
+ m->m_num_data_pg_pools = n;
+ m->m_data_pg_pools = kcalloc(n, sizeof(u32), GFP_NOFS);
+ if (!m->m_data_pg_pools)
+ goto badmem;
+ ceph_decode_need(p, end, sizeof(u32)*(n+1), bad);
+ for (i = 0; i < n; i++)
+ m->m_data_pg_pools[i] = ceph_decode_32(p);
+ m->m_cas_pg_pool = ceph_decode_32(p);
+
+ /* ok, we don't care about the rest. */
+ dout("mdsmap_decode success epoch %u\n", m->m_epoch);
+ return m;
+
+badmem:
+ err = -ENOMEM;
+bad:
+ pr_err("corrupt mdsmap\n");
+ print_hex_dump(KERN_DEBUG, "mdsmap: ",
+ DUMP_PREFIX_OFFSET, 16, 1,
+ start, end - start, true);
+ ceph_mdsmap_destroy(m);
+ return ERR_PTR(-EINVAL);
+}
+
+void ceph_mdsmap_destroy(struct ceph_mdsmap *m)
+{
+ int i;
+
+ for (i = 0; i < m->m_max_mds; i++)
+ kfree(m->m_info[i].export_targets);
+ kfree(m->m_info);
+ kfree(m->m_data_pg_pools);
+ kfree(m);
+}
--- /dev/null
+#ifndef _FS_CEPH_MDSMAP_H
+#define _FS_CEPH_MDSMAP_H
+
+#include "types.h"
+
+/*
+ * mds map - describe servers in the mds cluster.
+ *
+ * we limit fields to those the client actually xcares about
+ */
+struct ceph_mds_info {
+ u64 global_id;
+ struct ceph_entity_addr addr;
+ s32 state;
+ int num_export_targets;
+ u32 *export_targets;
+};
+
+struct ceph_mdsmap {
+ u32 m_epoch, m_client_epoch, m_last_failure;
+ u32 m_root;
+ u32 m_session_timeout; /* seconds */
+ u32 m_session_autoclose; /* seconds */
+ u64 m_max_file_size;
+ u32 m_max_mds; /* size of m_addr, m_state arrays */
+ struct ceph_mds_info *m_info;
+
+ /* which object pools file data can be stored in */
+ int m_num_data_pg_pools;
+ u32 *m_data_pg_pools;
+ u32 m_cas_pg_pool;
+};
+
+static inline struct ceph_entity_addr *
+ceph_mdsmap_get_addr(struct ceph_mdsmap *m, int w)
+{
+ if (w >= m->m_max_mds)
+ return NULL;
+ return &m->m_info[w].addr;
+}
+
+static inline int ceph_mdsmap_get_state(struct ceph_mdsmap *m, int w)
+{
+ BUG_ON(w < 0);
+ if (w >= m->m_max_mds)
+ return CEPH_MDS_STATE_DNE;
+ return m->m_info[w].state;
+}
+
+extern int ceph_mdsmap_get_random_mds(struct ceph_mdsmap *m);
+extern struct ceph_mdsmap *ceph_mdsmap_decode(void **p, void *end);
+extern void ceph_mdsmap_destroy(struct ceph_mdsmap *m);
+
+#endif
--- /dev/null
+#include "ceph_debug.h"
+
+#include <linux/crc32c.h>
+#include <linux/ctype.h>
+#include <linux/highmem.h>
+#include <linux/inet.h>
+#include <linux/kthread.h>
+#include <linux/net.h>
+#include <linux/slab.h>
+#include <linux/socket.h>
+#include <linux/string.h>
+#include <net/tcp.h>
+
+#include "super.h"
+#include "messenger.h"
+#include "decode.h"
+#include "pagelist.h"
+
+/*
+ * Ceph uses the messenger to exchange ceph_msg messages with other
+ * hosts in the system. The messenger provides ordered and reliable
+ * delivery. We tolerate TCP disconnects by reconnecting (with
+ * exponential backoff) in the case of a fault (disconnection, bad
+ * crc, protocol error). Acks allow sent messages to be discarded by
+ * the sender.
+ */
+
+/* static tag bytes (protocol control messages) */
+static char tag_msg = CEPH_MSGR_TAG_MSG;
+static char tag_ack = CEPH_MSGR_TAG_ACK;
+static char tag_keepalive = CEPH_MSGR_TAG_KEEPALIVE;
+
+
+static void queue_con(struct ceph_connection *con);
+static void con_work(struct work_struct *);
+static void ceph_fault(struct ceph_connection *con);
+
+const char *ceph_name_type_str(int t)
+{
+ switch (t) {
+ case CEPH_ENTITY_TYPE_MON: return "mon";
+ case CEPH_ENTITY_TYPE_MDS: return "mds";
+ case CEPH_ENTITY_TYPE_OSD: return "osd";
+ case CEPH_ENTITY_TYPE_CLIENT: return "client";
+ case CEPH_ENTITY_TYPE_ADMIN: return "admin";
+ default: return "???";
+ }
+}
+
+/*
+ * nicely render a sockaddr as a string.
+ */
+#define MAX_ADDR_STR 20
+static char addr_str[MAX_ADDR_STR][40];
+static DEFINE_SPINLOCK(addr_str_lock);
+static int last_addr_str;
+
+const char *pr_addr(const struct sockaddr_storage *ss)
+{
+ int i;
+ char *s;
+ struct sockaddr_in *in4 = (void *)ss;
+ unsigned char *quad = (void *)&in4->sin_addr.s_addr;
+ struct sockaddr_in6 *in6 = (void *)ss;
+
+ spin_lock(&addr_str_lock);
+ i = last_addr_str++;
+ if (last_addr_str == MAX_ADDR_STR)
+ last_addr_str = 0;
+ spin_unlock(&addr_str_lock);
+ s = addr_str[i];
+
+ switch (ss->ss_family) {
+ case AF_INET:
+ sprintf(s, "%u.%u.%u.%u:%u",
+ (unsigned int)quad[0],
+ (unsigned int)quad[1],
+ (unsigned int)quad[2],
+ (unsigned int)quad[3],
+ (unsigned int)ntohs(in4->sin_port));
+ break;
+
+ case AF_INET6:
+ sprintf(s, "%04x:%04x:%04x:%04x:%04x:%04x:%04x:%04x:%u",
+ in6->sin6_addr.s6_addr16[0],
+ in6->sin6_addr.s6_addr16[1],
+ in6->sin6_addr.s6_addr16[2],
+ in6->sin6_addr.s6_addr16[3],
+ in6->sin6_addr.s6_addr16[4],
+ in6->sin6_addr.s6_addr16[5],
+ in6->sin6_addr.s6_addr16[6],
+ in6->sin6_addr.s6_addr16[7],
+ (unsigned int)ntohs(in6->sin6_port));
+ break;
+
+ default:
+ sprintf(s, "(unknown sockaddr family %d)", (int)ss->ss_family);
+ }
+
+ return s;
+}
+
+static void encode_my_addr(struct ceph_messenger *msgr)
+{
+ memcpy(&msgr->my_enc_addr, &msgr->inst.addr, sizeof(msgr->my_enc_addr));
+ ceph_encode_addr(&msgr->my_enc_addr);
+}
+
+/*
+ * work queue for all reading and writing to/from the socket.
+ */
+struct workqueue_struct *ceph_msgr_wq;
+
+int __init ceph_msgr_init(void)
+{
+ ceph_msgr_wq = create_workqueue("ceph-msgr");
+ if (IS_ERR(ceph_msgr_wq)) {
+ int ret = PTR_ERR(ceph_msgr_wq);
+ pr_err("msgr_init failed to create workqueue: %d\n", ret);
+ ceph_msgr_wq = NULL;
+ return ret;
+ }
+ return 0;
+}
+
+void ceph_msgr_exit(void)
+{
+ destroy_workqueue(ceph_msgr_wq);
+}
+
+/*
+ * socket callback functions
+ */
+
+/* data available on socket, or listen socket received a connect */
+static void ceph_data_ready(struct sock *sk, int count_unused)
+{
+ struct ceph_connection *con =
+ (struct ceph_connection *)sk->sk_user_data;
+ if (sk->sk_state != TCP_CLOSE_WAIT) {
+ dout("ceph_data_ready on %p state = %lu, queueing work\n",
+ con, con->state);
+ queue_con(con);
+ }
+}
+
+/* socket has buffer space for writing */
+static void ceph_write_space(struct sock *sk)
+{
+ struct ceph_connection *con =
+ (struct ceph_connection *)sk->sk_user_data;
+
+ /* only queue to workqueue if there is data we want to write. */
+ if (test_bit(WRITE_PENDING, &con->state)) {
+ dout("ceph_write_space %p queueing write work\n", con);
+ queue_con(con);
+ } else {
+ dout("ceph_write_space %p nothing to write\n", con);
+ }
+
+ /* since we have our own write_space, clear the SOCK_NOSPACE flag */
+ clear_bit(SOCK_NOSPACE, &sk->sk_socket->flags);
+}
+
+/* socket's state has changed */
+static void ceph_state_change(struct sock *sk)
+{
+ struct ceph_connection *con =
+ (struct ceph_connection *)sk->sk_user_data;
+
+ dout("ceph_state_change %p state = %lu sk_state = %u\n",
+ con, con->state, sk->sk_state);
+
+ if (test_bit(CLOSED, &con->state))
+ return;
+
+ switch (sk->sk_state) {
+ case TCP_CLOSE:
+ dout("ceph_state_change TCP_CLOSE\n");
+ case TCP_CLOSE_WAIT:
+ dout("ceph_state_change TCP_CLOSE_WAIT\n");
+ if (test_and_set_bit(SOCK_CLOSED, &con->state) == 0) {
+ if (test_bit(CONNECTING, &con->state))
+ con->error_msg = "connection failed";
+ else
+ con->error_msg = "socket closed";
+ queue_con(con);
+ }
+ break;
+ case TCP_ESTABLISHED:
+ dout("ceph_state_change TCP_ESTABLISHED\n");
+ queue_con(con);
+ break;
+ }
+}
+
+/*
+ * set up socket callbacks
+ */
+static void set_sock_callbacks(struct socket *sock,
+ struct ceph_connection *con)
+{
+ struct sock *sk = sock->sk;
+ sk->sk_user_data = (void *)con;
+ sk->sk_data_ready = ceph_data_ready;
+ sk->sk_write_space = ceph_write_space;
+ sk->sk_state_change = ceph_state_change;
+}
+
+
+/*
+ * socket helpers
+ */
+
+/*
+ * initiate connection to a remote socket.
+ */
+static struct socket *ceph_tcp_connect(struct ceph_connection *con)
+{
+ struct sockaddr *paddr = (struct sockaddr *)&con->peer_addr.in_addr;
+ struct socket *sock;
+ int ret;
+
+ BUG_ON(con->sock);
+ ret = sock_create_kern(AF_INET, SOCK_STREAM, IPPROTO_TCP, &sock);
+ if (ret)
+ return ERR_PTR(ret);
+ con->sock = sock;
+ sock->sk->sk_allocation = GFP_NOFS;
+
+ set_sock_callbacks(sock, con);
+
+ dout("connect %s\n", pr_addr(&con->peer_addr.in_addr));
+
+ ret = sock->ops->connect(sock, paddr, sizeof(*paddr), O_NONBLOCK);
+ if (ret == -EINPROGRESS) {
+ dout("connect %s EINPROGRESS sk_state = %u\n",
+ pr_addr(&con->peer_addr.in_addr),
+ sock->sk->sk_state);
+ ret = 0;
+ }
+ if (ret < 0) {
+ pr_err("connect %s error %d\n",
+ pr_addr(&con->peer_addr.in_addr), ret);
+ sock_release(sock);
+ con->sock = NULL;
+ con->error_msg = "connect error";
+ }
+
+ if (ret < 0)
+ return ERR_PTR(ret);
+ return sock;
+}
+
+static int ceph_tcp_recvmsg(struct socket *sock, void *buf, size_t len)
+{
+ struct kvec iov = {buf, len};
+ struct msghdr msg = { .msg_flags = MSG_DONTWAIT | MSG_NOSIGNAL };
+
+ return kernel_recvmsg(sock, &msg, &iov, 1, len, msg.msg_flags);
+}
+
+/*
+ * write something. @more is true if caller will be sending more data
+ * shortly.
+ */
+static int ceph_tcp_sendmsg(struct socket *sock, struct kvec *iov,
+ size_t kvlen, size_t len, int more)
+{
+ struct msghdr msg = { .msg_flags = MSG_DONTWAIT | MSG_NOSIGNAL };
+
+ if (more)
+ msg.msg_flags |= MSG_MORE;
+ else
+ msg.msg_flags |= MSG_EOR; /* superfluous, but what the hell */
+
+ return kernel_sendmsg(sock, &msg, iov, kvlen, len);
+}
+
+
+/*
+ * Shutdown/close the socket for the given connection.
+ */
+static int con_close_socket(struct ceph_connection *con)
+{
+ int rc;
+
+ dout("con_close_socket on %p sock %p\n", con, con->sock);
+ if (!con->sock)
+ return 0;
+ set_bit(SOCK_CLOSED, &con->state);
+ rc = con->sock->ops->shutdown(con->sock, SHUT_RDWR);
+ sock_release(con->sock);
+ con->sock = NULL;
+ clear_bit(SOCK_CLOSED, &con->state);
+ return rc;
+}
+
+/*
+ * Reset a connection. Discard all incoming and outgoing messages
+ * and clear *_seq state.
+ */
+static void ceph_msg_remove(struct ceph_msg *msg)
+{
+ list_del_init(&msg->list_head);
+ ceph_msg_put(msg);
+}
+static void ceph_msg_remove_list(struct list_head *head)
+{
+ while (!list_empty(head)) {
+ struct ceph_msg *msg = list_first_entry(head, struct ceph_msg,
+ list_head);
+ ceph_msg_remove(msg);
+ }
+}
+
+static void reset_connection(struct ceph_connection *con)
+{
+ /* reset connection, out_queue, msg_ and connect_seq */
+ /* discard existing out_queue and msg_seq */
+ ceph_msg_remove_list(&con->out_queue);
+ ceph_msg_remove_list(&con->out_sent);
+
+ if (con->in_msg) {
+ ceph_msg_put(con->in_msg);
+ con->in_msg = NULL;
+ }
+
+ con->connect_seq = 0;
+ con->out_seq = 0;
+ if (con->out_msg) {
+ ceph_msg_put(con->out_msg);
+ con->out_msg = NULL;
+ }
+ con->in_seq = 0;
+}
+
+/*
+ * mark a peer down. drop any open connections.
+ */
+void ceph_con_close(struct ceph_connection *con)
+{
+ dout("con_close %p peer %s\n", con, pr_addr(&con->peer_addr.in_addr));
+ set_bit(CLOSED, &con->state); /* in case there's queued work */
+ clear_bit(STANDBY, &con->state); /* avoid connect_seq bump */
+ clear_bit(LOSSYTX, &con->state); /* so we retry next connect */
+ clear_bit(KEEPALIVE_PENDING, &con->state);
+ clear_bit(WRITE_PENDING, &con->state);
+ mutex_lock(&con->mutex);
+ reset_connection(con);
+ cancel_delayed_work(&con->work);
+ mutex_unlock(&con->mutex);
+ queue_con(con);
+}
+
+/*
+ * Reopen a closed connection, with a new peer address.
+ */
+void ceph_con_open(struct ceph_connection *con, struct ceph_entity_addr *addr)
+{
+ dout("con_open %p %s\n", con, pr_addr(&addr->in_addr));
+ set_bit(OPENING, &con->state);
+ clear_bit(CLOSED, &con->state);
+ memcpy(&con->peer_addr, addr, sizeof(*addr));
+ con->delay = 0; /* reset backoff memory */
+ queue_con(con);
+}
+
+/*
+ * return true if this connection ever successfully opened
+ */
+bool ceph_con_opened(struct ceph_connection *con)
+{
+ return con->connect_seq > 0;
+}
+
+/*
+ * generic get/put
+ */
+struct ceph_connection *ceph_con_get(struct ceph_connection *con)
+{
+ dout("con_get %p nref = %d -> %d\n", con,
+ atomic_read(&con->nref), atomic_read(&con->nref) + 1);
+ if (atomic_inc_not_zero(&con->nref))
+ return con;
+ return NULL;
+}
+
+void ceph_con_put(struct ceph_connection *con)
+{
+ dout("con_put %p nref = %d -> %d\n", con,
+ atomic_read(&con->nref), atomic_read(&con->nref) - 1);
+ BUG_ON(atomic_read(&con->nref) == 0);
+ if (atomic_dec_and_test(&con->nref)) {
+ BUG_ON(con->sock);
+ kfree(con);
+ }
+}
+
+/*
+ * initialize a new connection.
+ */
+void ceph_con_init(struct ceph_messenger *msgr, struct ceph_connection *con)
+{
+ dout("con_init %p\n", con);
+ memset(con, 0, sizeof(*con));
+ atomic_set(&con->nref, 1);
+ con->msgr = msgr;
+ mutex_init(&con->mutex);
+ INIT_LIST_HEAD(&con->out_queue);
+ INIT_LIST_HEAD(&con->out_sent);
+ INIT_DELAYED_WORK(&con->work, con_work);
+}
+
+
+/*
+ * We maintain a global counter to order connection attempts. Get
+ * a unique seq greater than @gt.
+ */
+static u32 get_global_seq(struct ceph_messenger *msgr, u32 gt)
+{
+ u32 ret;
+
+ spin_lock(&msgr->global_seq_lock);
+ if (msgr->global_seq < gt)
+ msgr->global_seq = gt;
+ ret = ++msgr->global_seq;
+ spin_unlock(&msgr->global_seq_lock);
+ return ret;
+}
+
+
+/*
+ * Prepare footer for currently outgoing message, and finish things
+ * off. Assumes out_kvec* are already valid.. we just add on to the end.
+ */
+static void prepare_write_message_footer(struct ceph_connection *con, int v)
+{
+ struct ceph_msg *m = con->out_msg;
+
+ dout("prepare_write_message_footer %p\n", con);
+ con->out_kvec_is_msg = true;
+ con->out_kvec[v].iov_base = &m->footer;
+ con->out_kvec[v].iov_len = sizeof(m->footer);
+ con->out_kvec_bytes += sizeof(m->footer);
+ con->out_kvec_left++;
+ con->out_more = m->more_to_follow;
+ con->out_msg_done = true;
+}
+
+/*
+ * Prepare headers for the next outgoing message.
+ */
+static void prepare_write_message(struct ceph_connection *con)
+{
+ struct ceph_msg *m;
+ int v = 0;
+
+ con->out_kvec_bytes = 0;
+ con->out_kvec_is_msg = true;
+ con->out_msg_done = false;
+
+ /* Sneak an ack in there first? If we can get it into the same
+ * TCP packet that's a good thing. */
+ if (con->in_seq > con->in_seq_acked) {
+ con->in_seq_acked = con->in_seq;
+ con->out_kvec[v].iov_base = &tag_ack;
+ con->out_kvec[v++].iov_len = 1;
+ con->out_temp_ack = cpu_to_le64(con->in_seq_acked);
+ con->out_kvec[v].iov_base = &con->out_temp_ack;
+ con->out_kvec[v++].iov_len = sizeof(con->out_temp_ack);
+ con->out_kvec_bytes = 1 + sizeof(con->out_temp_ack);
+ }
+
+ m = list_first_entry(&con->out_queue,
+ struct ceph_msg, list_head);
+ con->out_msg = m;
+ if (test_bit(LOSSYTX, &con->state)) {
+ list_del_init(&m->list_head);
+ } else {
+ /* put message on sent list */
+ ceph_msg_get(m);
+ list_move_tail(&m->list_head, &con->out_sent);
+ }
+
+ m->hdr.seq = cpu_to_le64(++con->out_seq);
+
+ dout("prepare_write_message %p seq %lld type %d len %d+%d+%d %d pgs\n",
+ m, con->out_seq, le16_to_cpu(m->hdr.type),
+ le32_to_cpu(m->hdr.front_len), le32_to_cpu(m->hdr.middle_len),
+ le32_to_cpu(m->hdr.data_len),
+ m->nr_pages);
+ BUG_ON(le32_to_cpu(m->hdr.front_len) != m->front.iov_len);
+
+ /* tag + hdr + front + middle */
+ con->out_kvec[v].iov_base = &tag_msg;
+ con->out_kvec[v++].iov_len = 1;
+ con->out_kvec[v].iov_base = &m->hdr;
+ con->out_kvec[v++].iov_len = sizeof(m->hdr);
+ con->out_kvec[v++] = m->front;
+ if (m->middle)
+ con->out_kvec[v++] = m->middle->vec;
+ con->out_kvec_left = v;
+ con->out_kvec_bytes += 1 + sizeof(m->hdr) + m->front.iov_len +
+ (m->middle ? m->middle->vec.iov_len : 0);
+ con->out_kvec_cur = con->out_kvec;
+
+ /* fill in crc (except data pages), footer */
+ con->out_msg->hdr.crc =
+ cpu_to_le32(crc32c(0, (void *)&m->hdr,
+ sizeof(m->hdr) - sizeof(m->hdr.crc)));
+ con->out_msg->footer.flags = CEPH_MSG_FOOTER_COMPLETE;
+ con->out_msg->footer.front_crc =
+ cpu_to_le32(crc32c(0, m->front.iov_base, m->front.iov_len));
+ if (m->middle)
+ con->out_msg->footer.middle_crc =
+ cpu_to_le32(crc32c(0, m->middle->vec.iov_base,
+ m->middle->vec.iov_len));
+ else
+ con->out_msg->footer.middle_crc = 0;
+ con->out_msg->footer.data_crc = 0;
+ dout("prepare_write_message front_crc %u data_crc %u\n",
+ le32_to_cpu(con->out_msg->footer.front_crc),
+ le32_to_cpu(con->out_msg->footer.middle_crc));
+
+ /* is there a data payload? */
+ if (le32_to_cpu(m->hdr.data_len) > 0) {
+ /* initialize page iterator */
+ con->out_msg_pos.page = 0;
+ con->out_msg_pos.page_pos =
+ le16_to_cpu(m->hdr.data_off) & ~PAGE_MASK;
+ con->out_msg_pos.data_pos = 0;
+ con->out_msg_pos.did_page_crc = 0;
+ con->out_more = 1; /* data + footer will follow */
+ } else {
+ /* no, queue up footer too and be done */
+ prepare_write_message_footer(con, v);
+ }
+
+ set_bit(WRITE_PENDING, &con->state);
+}
+
+/*
+ * Prepare an ack.
+ */
+static void prepare_write_ack(struct ceph_connection *con)
+{
+ dout("prepare_write_ack %p %llu -> %llu\n", con,
+ con->in_seq_acked, con->in_seq);
+ con->in_seq_acked = con->in_seq;
+
+ con->out_kvec[0].iov_base = &tag_ack;
+ con->out_kvec[0].iov_len = 1;
+ con->out_temp_ack = cpu_to_le64(con->in_seq_acked);
+ con->out_kvec[1].iov_base = &con->out_temp_ack;
+ con->out_kvec[1].iov_len = sizeof(con->out_temp_ack);
+ con->out_kvec_left = 2;
+ con->out_kvec_bytes = 1 + sizeof(con->out_temp_ack);
+ con->out_kvec_cur = con->out_kvec;
+ con->out_more = 1; /* more will follow.. eventually.. */
+ set_bit(WRITE_PENDING, &con->state);
+}
+
+/*
+ * Prepare to write keepalive byte.
+ */
+static void prepare_write_keepalive(struct ceph_connection *con)
+{
+ dout("prepare_write_keepalive %p\n", con);
+ con->out_kvec[0].iov_base = &tag_keepalive;
+ con->out_kvec[0].iov_len = 1;
+ con->out_kvec_left = 1;
+ con->out_kvec_bytes = 1;
+ con->out_kvec_cur = con->out_kvec;
+ set_bit(WRITE_PENDING, &con->state);
+}
+
+/*
+ * Connection negotiation.
+ */
+
+static void prepare_connect_authorizer(struct ceph_connection *con)
+{
+ void *auth_buf;
+ int auth_len = 0;
+ int auth_protocol = 0;
+
+ mutex_unlock(&con->mutex);
+ if (con->ops->get_authorizer)
+ con->ops->get_authorizer(con, &auth_buf, &auth_len,
+ &auth_protocol, &con->auth_reply_buf,
+ &con->auth_reply_buf_len,
+ con->auth_retry);
+ mutex_lock(&con->mutex);
+
+ con->out_connect.authorizer_protocol = cpu_to_le32(auth_protocol);
+ con->out_connect.authorizer_len = cpu_to_le32(auth_len);
+
+ con->out_kvec[con->out_kvec_left].iov_base = auth_buf;
+ con->out_kvec[con->out_kvec_left].iov_len = auth_len;
+ con->out_kvec_left++;
+ con->out_kvec_bytes += auth_len;
+}
+
+/*
+ * We connected to a peer and are saying hello.
+ */
+static void prepare_write_banner(struct ceph_messenger *msgr,
+ struct ceph_connection *con)
+{
+ int len = strlen(CEPH_BANNER);
+
+ con->out_kvec[0].iov_base = CEPH_BANNER;
+ con->out_kvec[0].iov_len = len;
+ con->out_kvec[1].iov_base = &msgr->my_enc_addr;
+ con->out_kvec[1].iov_len = sizeof(msgr->my_enc_addr);
+ con->out_kvec_left = 2;
+ con->out_kvec_bytes = len + sizeof(msgr->my_enc_addr);
+ con->out_kvec_cur = con->out_kvec;
+ con->out_more = 0;
+ set_bit(WRITE_PENDING, &con->state);
+}
+
+static void prepare_write_connect(struct ceph_messenger *msgr,
+ struct ceph_connection *con,
+ int after_banner)
+{
+ unsigned global_seq = get_global_seq(con->msgr, 0);
+ int proto;
+
+ switch (con->peer_name.type) {
+ case CEPH_ENTITY_TYPE_MON:
+ proto = CEPH_MONC_PROTOCOL;
+ break;
+ case CEPH_ENTITY_TYPE_OSD:
+ proto = CEPH_OSDC_PROTOCOL;
+ break;
+ case CEPH_ENTITY_TYPE_MDS:
+ proto = CEPH_MDSC_PROTOCOL;
+ break;
+ default:
+ BUG();
+ }
+
+ dout("prepare_write_connect %p cseq=%d gseq=%d proto=%d\n", con,
+ con->connect_seq, global_seq, proto);
+
+ con->out_connect.features = CEPH_FEATURE_SUPPORTED;
+ con->out_connect.host_type = cpu_to_le32(CEPH_ENTITY_TYPE_CLIENT);
+ con->out_connect.connect_seq = cpu_to_le32(con->connect_seq);
+ con->out_connect.global_seq = cpu_to_le32(global_seq);
+ con->out_connect.protocol_version = cpu_to_le32(proto);
+ con->out_connect.flags = 0;
+
+ if (!after_banner) {
+ con->out_kvec_left = 0;
+ con->out_kvec_bytes = 0;
+ }
+ con->out_kvec[con->out_kvec_left].iov_base = &con->out_connect;
+ con->out_kvec[con->out_kvec_left].iov_len = sizeof(con->out_connect);
+ con->out_kvec_left++;
+ con->out_kvec_bytes += sizeof(con->out_connect);
+ con->out_kvec_cur = con->out_kvec;
+ con->out_more = 0;
+ set_bit(WRITE_PENDING, &con->state);
+
+ prepare_connect_authorizer(con);
+}
+
+
+/*
+ * write as much of pending kvecs to the socket as we can.
+ * 1 -> done
+ * 0 -> socket full, but more to do
+ * <0 -> error
+ */
+static int write_partial_kvec(struct ceph_connection *con)
+{
+ int ret;
+
+ dout("write_partial_kvec %p %d left\n", con, con->out_kvec_bytes);
+ while (con->out_kvec_bytes > 0) {
+ ret = ceph_tcp_sendmsg(con->sock, con->out_kvec_cur,
+ con->out_kvec_left, con->out_kvec_bytes,
+ con->out_more);
+ if (ret <= 0)
+ goto out;
+ con->out_kvec_bytes -= ret;
+ if (con->out_kvec_bytes == 0)
+ break; /* done */
+ while (ret > 0) {
+ if (ret >= con->out_kvec_cur->iov_len) {
+ ret -= con->out_kvec_cur->iov_len;
+ con->out_kvec_cur++;
+ con->out_kvec_left--;
+ } else {
+ con->out_kvec_cur->iov_len -= ret;
+ con->out_kvec_cur->iov_base += ret;
+ ret = 0;
+ break;
+ }
+ }
+ }
+ con->out_kvec_left = 0;
+ con->out_kvec_is_msg = false;
+ ret = 1;
+out:
+ dout("write_partial_kvec %p %d left in %d kvecs ret = %d\n", con,
+ con->out_kvec_bytes, con->out_kvec_left, ret);
+ return ret; /* done! */
+}
+
+/*
+ * Write as much message data payload as we can. If we finish, queue
+ * up the footer.
+ * 1 -> done, footer is now queued in out_kvec[].
+ * 0 -> socket full, but more to do
+ * <0 -> error
+ */
+static int write_partial_msg_pages(struct ceph_connection *con)
+{
+ struct ceph_msg *msg = con->out_msg;
+ unsigned data_len = le32_to_cpu(msg->hdr.data_len);
+ size_t len;
+ int crc = con->msgr->nocrc;
+ int ret;
+
+ dout("write_partial_msg_pages %p msg %p page %d/%d offset %d\n",
+ con, con->out_msg, con->out_msg_pos.page, con->out_msg->nr_pages,
+ con->out_msg_pos.page_pos);
+
+ while (con->out_msg_pos.page < con->out_msg->nr_pages) {
+ struct page *page = NULL;
+ void *kaddr = NULL;
+
+ /*
+ * if we are calculating the data crc (the default), we need
+ * to map the page. if our pages[] has been revoked, use the
+ * zero page.
+ */
+ if (msg->pages) {
+ page = msg->pages[con->out_msg_pos.page];
+ if (crc)
+ kaddr = kmap(page);
+ } else if (msg->pagelist) {
+ page = list_first_entry(&msg->pagelist->head,
+ struct page, lru);
+ if (crc)
+ kaddr = kmap(page);
+ } else {
+ page = con->msgr->zero_page;
+ if (crc)
+ kaddr = page_address(con->msgr->zero_page);
+ }
+ len = min((int)(PAGE_SIZE - con->out_msg_pos.page_pos),
+ (int)(data_len - con->out_msg_pos.data_pos));
+ if (crc && !con->out_msg_pos.did_page_crc) {
+ void *base = kaddr + con->out_msg_pos.page_pos;
+ u32 tmpcrc = le32_to_cpu(con->out_msg->footer.data_crc);
+
+ BUG_ON(kaddr == NULL);
+ con->out_msg->footer.data_crc =
+ cpu_to_le32(crc32c(tmpcrc, base, len));
+ con->out_msg_pos.did_page_crc = 1;
+ }
+
+ ret = kernel_sendpage(con->sock, page,
+ con->out_msg_pos.page_pos, len,
+ MSG_DONTWAIT | MSG_NOSIGNAL |
+ MSG_MORE);
+
+ if (crc && (msg->pages || msg->pagelist))
+ kunmap(page);
+
+ if (ret <= 0)
+ goto out;
+
+ con->out_msg_pos.data_pos += ret;
+ con->out_msg_pos.page_pos += ret;
+ if (ret == len) {
+ con->out_msg_pos.page_pos = 0;
+ con->out_msg_pos.page++;
+ con->out_msg_pos.did_page_crc = 0;
+ if (msg->pagelist)
+ list_move_tail(&page->lru,
+ &msg->pagelist->head);
+ }
+ }
+
+ dout("write_partial_msg_pages %p msg %p done\n", con, msg);
+
+ /* prepare and queue up footer, too */
+ if (!crc)
+ con->out_msg->footer.flags |= CEPH_MSG_FOOTER_NOCRC;
+ con->out_kvec_bytes = 0;
+ con->out_kvec_left = 0;
+ con->out_kvec_cur = con->out_kvec;
+ prepare_write_message_footer(con, 0);
+ ret = 1;
+out:
+ return ret;
+}
+
+/*
+ * write some zeros
+ */
+static int write_partial_skip(struct ceph_connection *con)
+{
+ int ret;
+
+ while (con->out_skip > 0) {
+ struct kvec iov = {
+ .iov_base = page_address(con->msgr->zero_page),
+ .iov_len = min(con->out_skip, (int)PAGE_CACHE_SIZE)
+ };
+
+ ret = ceph_tcp_sendmsg(con->sock, &iov, 1, iov.iov_len, 1);
+ if (ret <= 0)
+ goto out;
+ con->out_skip -= ret;
+ }
+ ret = 1;
+out:
+ return ret;
+}
+
+/*
+ * Prepare to read connection handshake, or an ack.
+ */
+static void prepare_read_banner(struct ceph_connection *con)
+{
+ dout("prepare_read_banner %p\n", con);
+ con->in_base_pos = 0;
+}
+
+static void prepare_read_connect(struct ceph_connection *con)
+{
+ dout("prepare_read_connect %p\n", con);
+ con->in_base_pos = 0;
+}
+
+static void prepare_read_ack(struct ceph_connection *con)
+{
+ dout("prepare_read_ack %p\n", con);
+ con->in_base_pos = 0;
+}
+
+static void prepare_read_tag(struct ceph_connection *con)
+{
+ dout("prepare_read_tag %p\n", con);
+ con->in_base_pos = 0;
+ con->in_tag = CEPH_MSGR_TAG_READY;
+}
+
+/*
+ * Prepare to read a message.
+ */
+static int prepare_read_message(struct ceph_connection *con)
+{
+ dout("prepare_read_message %p\n", con);
+ BUG_ON(con->in_msg != NULL);
+ con->in_base_pos = 0;
+ con->in_front_crc = con->in_middle_crc = con->in_data_crc = 0;
+ return 0;
+}
+
+
+static int read_partial(struct ceph_connection *con,
+ int *to, int size, void *object)
+{
+ *to += size;
+ while (con->in_base_pos < *to) {
+ int left = *to - con->in_base_pos;
+ int have = size - left;
+ int ret = ceph_tcp_recvmsg(con->sock, object + have, left);
+ if (ret <= 0)
+ return ret;
+ con->in_base_pos += ret;
+ }
+ return 1;
+}
+
+
+/*
+ * Read all or part of the connect-side handshake on a new connection
+ */
+static int read_partial_banner(struct ceph_connection *con)
+{
+ int ret, to = 0;
+
+ dout("read_partial_banner %p at %d\n", con, con->in_base_pos);
+
+ /* peer's banner */
+ ret = read_partial(con, &to, strlen(CEPH_BANNER), con->in_banner);
+ if (ret <= 0)
+ goto out;
+ ret = read_partial(con, &to, sizeof(con->actual_peer_addr),
+ &con->actual_peer_addr);
+ if (ret <= 0)
+ goto out;
+ ret = read_partial(con, &to, sizeof(con->peer_addr_for_me),
+ &con->peer_addr_for_me);
+ if (ret <= 0)
+ goto out;
+out:
+ return ret;
+}
+
+static int read_partial_connect(struct ceph_connection *con)
+{
+ int ret, to = 0;
+
+ dout("read_partial_connect %p at %d\n", con, con->in_base_pos);
+
+ ret = read_partial(con, &to, sizeof(con->in_reply), &con->in_reply);
+ if (ret <= 0)
+ goto out;
+ ret = read_partial(con, &to, le32_to_cpu(con->in_reply.authorizer_len),
+ con->auth_reply_buf);
+ if (ret <= 0)
+ goto out;
+
+ dout("read_partial_connect %p tag %d, con_seq = %u, g_seq = %u\n",
+ con, (int)con->in_reply.tag,
+ le32_to_cpu(con->in_reply.connect_seq),
+ le32_to_cpu(con->in_reply.global_seq));
+out:
+ return ret;
+
+}
+
+/*
+ * Verify the hello banner looks okay.
+ */
+static int verify_hello(struct ceph_connection *con)
+{
+ if (memcmp(con->in_banner, CEPH_BANNER, strlen(CEPH_BANNER))) {
+ pr_err("connect to %s got bad banner\n",
+ pr_addr(&con->peer_addr.in_addr));
+ con->error_msg = "protocol error, bad banner";
+ return -1;
+ }
+ return 0;
+}
+
+static bool addr_is_blank(struct sockaddr_storage *ss)
+{
+ switch (ss->ss_family) {
+ case AF_INET:
+ return ((struct sockaddr_in *)ss)->sin_addr.s_addr == 0;
+ case AF_INET6:
+ return
+ ((struct sockaddr_in6 *)ss)->sin6_addr.s6_addr32[0] == 0 &&
+ ((struct sockaddr_in6 *)ss)->sin6_addr.s6_addr32[1] == 0 &&
+ ((struct sockaddr_in6 *)ss)->sin6_addr.s6_addr32[2] == 0 &&
+ ((struct sockaddr_in6 *)ss)->sin6_addr.s6_addr32[3] == 0;
+ }
+ return false;
+}
+
+static int addr_port(struct sockaddr_storage *ss)
+{
+ switch (ss->ss_family) {
+ case AF_INET:
+ return ntohs(((struct sockaddr_in *)ss)->sin_port);
+ case AF_INET6:
+ return ntohs(((struct sockaddr_in6 *)ss)->sin6_port);
+ }
+ return 0;
+}
+
+static void addr_set_port(struct sockaddr_storage *ss, int p)
+{
+ switch (ss->ss_family) {
+ case AF_INET:
+ ((struct sockaddr_in *)ss)->sin_port = htons(p);
+ case AF_INET6:
+ ((struct sockaddr_in6 *)ss)->sin6_port = htons(p);
+ }
+}
+
+/*
+ * Parse an ip[:port] list into an addr array. Use the default
+ * monitor port if a port isn't specified.
+ */
+int ceph_parse_ips(const char *c, const char *end,
+ struct ceph_entity_addr *addr,
+ int max_count, int *count)
+{
+ int i;
+ const char *p = c;
+
+ dout("parse_ips on '%.*s'\n", (int)(end-c), c);
+ for (i = 0; i < max_count; i++) {
+ const char *ipend;
+ struct sockaddr_storage *ss = &addr[i].in_addr;
+ struct sockaddr_in *in4 = (void *)ss;
+ struct sockaddr_in6 *in6 = (void *)ss;
+ int port;
+
+ memset(ss, 0, sizeof(*ss));
+ if (in4_pton(p, end - p, (u8 *)&in4->sin_addr.s_addr,
+ ',', &ipend)) {
+ ss->ss_family = AF_INET;
+ } else if (in6_pton(p, end - p, (u8 *)&in6->sin6_addr.s6_addr,
+ ',', &ipend)) {
+ ss->ss_family = AF_INET6;
+ } else {
+ goto bad;
+ }
+ p = ipend;
+
+ /* port? */
+ if (p < end && *p == ':') {
+ port = 0;
+ p++;
+ while (p < end && *p >= '0' && *p <= '9') {
+ port = (port * 10) + (*p - '0');
+ p++;
+ }
+ if (port > 65535 || port == 0)
+ goto bad;
+ } else {
+ port = CEPH_MON_PORT;
+ }
+
+ addr_set_port(ss, port);
+
+ dout("parse_ips got %s\n", pr_addr(ss));
+
+ if (p == end)
+ break;
+ if (*p != ',')
+ goto bad;
+ p++;
+ }
+
+ if (p != end)
+ goto bad;
+
+ if (count)
+ *count = i + 1;
+ return 0;
+
+bad:
+ pr_err("parse_ips bad ip '%s'\n", c);
+ return -EINVAL;
+}
+
+static int process_banner(struct ceph_connection *con)
+{
+ dout("process_banner on %p\n", con);
+
+ if (verify_hello(con) < 0)
+ return -1;
+
+ ceph_decode_addr(&con->actual_peer_addr);
+ ceph_decode_addr(&con->peer_addr_for_me);
+
+ /*
+ * Make sure the other end is who we wanted. note that the other
+ * end may not yet know their ip address, so if it's 0.0.0.0, give
+ * them the benefit of the doubt.
+ */
+ if (memcmp(&con->peer_addr, &con->actual_peer_addr,
+ sizeof(con->peer_addr)) != 0 &&
+ !(addr_is_blank(&con->actual_peer_addr.in_addr) &&
+ con->actual_peer_addr.nonce == con->peer_addr.nonce)) {
+ pr_warning("wrong peer, want %s/%lld, got %s/%lld\n",
+ pr_addr(&con->peer_addr.in_addr),
+ le64_to_cpu(con->peer_addr.nonce),
+ pr_addr(&con->actual_peer_addr.in_addr),
+ le64_to_cpu(con->actual_peer_addr.nonce));
+ con->error_msg = "wrong peer at address";
+ return -1;
+ }
+
+ /*
+ * did we learn our address?
+ */
+ if (addr_is_blank(&con->msgr->inst.addr.in_addr)) {
+ int port = addr_port(&con->msgr->inst.addr.in_addr);
+
+ memcpy(&con->msgr->inst.addr.in_addr,
+ &con->peer_addr_for_me.in_addr,
+ sizeof(con->peer_addr_for_me.in_addr));
+ addr_set_port(&con->msgr->inst.addr.in_addr, port);
+ encode_my_addr(con->msgr);
+ dout("process_banner learned my addr is %s\n",
+ pr_addr(&con->msgr->inst.addr.in_addr));
+ }
+
+ set_bit(NEGOTIATING, &con->state);
+ prepare_read_connect(con);
+ return 0;
+}
+
+static void fail_protocol(struct ceph_connection *con)
+{
+ reset_connection(con);
+ set_bit(CLOSED, &con->state); /* in case there's queued work */
+
+ mutex_unlock(&con->mutex);
+ if (con->ops->bad_proto)
+ con->ops->bad_proto(con);
+ mutex_lock(&con->mutex);
+}
+
+static int process_connect(struct ceph_connection *con)
+{
+ u64 sup_feat = CEPH_FEATURE_SUPPORTED;
+ u64 req_feat = CEPH_FEATURE_REQUIRED;
+ u64 server_feat = le64_to_cpu(con->in_reply.features);
+
+ dout("process_connect on %p tag %d\n", con, (int)con->in_tag);
+
+ switch (con->in_reply.tag) {
+ case CEPH_MSGR_TAG_FEATURES:
+ pr_err("%s%lld %s feature set mismatch,"
+ " my %llx < server's %llx, missing %llx\n",
+ ENTITY_NAME(con->peer_name),
+ pr_addr(&con->peer_addr.in_addr),
+ sup_feat, server_feat, server_feat & ~sup_feat);
+ con->error_msg = "missing required protocol features";
+ fail_protocol(con);
+ return -1;
+
+ case CEPH_MSGR_TAG_BADPROTOVER:
+ pr_err("%s%lld %s protocol version mismatch,"
+ " my %d != server's %d\n",
+ ENTITY_NAME(con->peer_name),
+ pr_addr(&con->peer_addr.in_addr),
+ le32_to_cpu(con->out_connect.protocol_version),
+ le32_to_cpu(con->in_reply.protocol_version));
+ con->error_msg = "protocol version mismatch";
+ fail_protocol(con);
+ return -1;
+
+ case CEPH_MSGR_TAG_BADAUTHORIZER:
+ con->auth_retry++;
+ dout("process_connect %p got BADAUTHORIZER attempt %d\n", con,
+ con->auth_retry);
+ if (con->auth_retry == 2) {
+ con->error_msg = "connect authorization failure";
+ reset_connection(con);
+ set_bit(CLOSED, &con->state);
+ return -1;
+ }
+ con->auth_retry = 1;
+ prepare_write_connect(con->msgr, con, 0);
+ prepare_read_connect(con);
+ break;
+
+ case CEPH_MSGR_TAG_RESETSESSION:
+ /*
+ * If we connected with a large connect_seq but the peer
+ * has no record of a session with us (no connection, or
+ * connect_seq == 0), they will send RESETSESION to indicate
+ * that they must have reset their session, and may have
+ * dropped messages.
+ */
+ dout("process_connect got RESET peer seq %u\n",
+ le32_to_cpu(con->in_connect.connect_seq));
+ pr_err("%s%lld %s connection reset\n",
+ ENTITY_NAME(con->peer_name),
+ pr_addr(&con->peer_addr.in_addr));
+ reset_connection(con);
+ prepare_write_connect(con->msgr, con, 0);
+ prepare_read_connect(con);
+
+ /* Tell ceph about it. */
+ mutex_unlock(&con->mutex);
+ pr_info("reset on %s%lld\n", ENTITY_NAME(con->peer_name));
+ if (con->ops->peer_reset)
+ con->ops->peer_reset(con);
+ mutex_lock(&con->mutex);
+ break;
+
+ case CEPH_MSGR_TAG_RETRY_SESSION:
+ /*
+ * If we sent a smaller connect_seq than the peer has, try
+ * again with a larger value.
+ */
+ dout("process_connect got RETRY my seq = %u, peer_seq = %u\n",
+ le32_to_cpu(con->out_connect.connect_seq),
+ le32_to_cpu(con->in_connect.connect_seq));
+ con->connect_seq = le32_to_cpu(con->in_connect.connect_seq);
+ prepare_write_connect(con->msgr, con, 0);
+ prepare_read_connect(con);
+ break;
+
+ case CEPH_MSGR_TAG_RETRY_GLOBAL:
+ /*
+ * If we sent a smaller global_seq than the peer has, try
+ * again with a larger value.
+ */
+ dout("process_connect got RETRY_GLOBAL my %u peer_gseq %u\n",
+ con->peer_global_seq,
+ le32_to_cpu(con->in_connect.global_seq));
+ get_global_seq(con->msgr,
+ le32_to_cpu(con->in_connect.global_seq));
+ prepare_write_connect(con->msgr, con, 0);
+ prepare_read_connect(con);
+ break;
+
+ case CEPH_MSGR_TAG_READY:
+ if (req_feat & ~server_feat) {
+ pr_err("%s%lld %s protocol feature mismatch,"
+ " my required %llx > server's %llx, need %llx\n",
+ ENTITY_NAME(con->peer_name),
+ pr_addr(&con->peer_addr.in_addr),
+ req_feat, server_feat, req_feat & ~server_feat);
+ con->error_msg = "missing required protocol features";
+ fail_protocol(con);
+ return -1;
+ }
+ clear_bit(CONNECTING, &con->state);
+ con->peer_global_seq = le32_to_cpu(con->in_reply.global_seq);
+ con->connect_seq++;
+ dout("process_connect got READY gseq %d cseq %d (%d)\n",
+ con->peer_global_seq,
+ le32_to_cpu(con->in_reply.connect_seq),
+ con->connect_seq);
+ WARN_ON(con->connect_seq !=
+ le32_to_cpu(con->in_reply.connect_seq));
+
+ if (con->in_reply.flags & CEPH_MSG_CONNECT_LOSSY)
+ set_bit(LOSSYTX, &con->state);
+
+ prepare_read_tag(con);
+ break;
+
+ case CEPH_MSGR_TAG_WAIT:
+ /*
+ * If there is a connection race (we are opening
+ * connections to each other), one of us may just have
+ * to WAIT. This shouldn't happen if we are the
+ * client.
+ */
+ pr_err("process_connect peer connecting WAIT\n");
+
+ default:
+ pr_err("connect protocol error, will retry\n");
+ con->error_msg = "protocol error, garbage tag during connect";
+ return -1;
+ }
+ return 0;
+}
+
+
+/*
+ * read (part of) an ack
+ */
+static int read_partial_ack(struct ceph_connection *con)
+{
+ int to = 0;
+
+ return read_partial(con, &to, sizeof(con->in_temp_ack),
+ &con->in_temp_ack);
+}
+
+
+/*
+ * We can finally discard anything that's been acked.
+ */
+static void process_ack(struct ceph_connection *con)
+{
+ struct ceph_msg *m;
+ u64 ack = le64_to_cpu(con->in_temp_ack);
+ u64 seq;
+
+ while (!list_empty(&con->out_sent)) {
+ m = list_first_entry(&con->out_sent, struct ceph_msg,
+ list_head);
+ seq = le64_to_cpu(m->hdr.seq);
+ if (seq > ack)
+ break;
+ dout("got ack for seq %llu type %d at %p\n", seq,
+ le16_to_cpu(m->hdr.type), m);
+ ceph_msg_remove(m);
+ }
+ prepare_read_tag(con);
+}
+
+
+
+
+static int read_partial_message_section(struct ceph_connection *con,
+ struct kvec *section, unsigned int sec_len,
+ u32 *crc)
+{
+ int left;
+ int ret;
+
+ BUG_ON(!section);
+
+ while (section->iov_len < sec_len) {
+ BUG_ON(section->iov_base == NULL);
+ left = sec_len - section->iov_len;
+ ret = ceph_tcp_recvmsg(con->sock, (char *)section->iov_base +
+ section->iov_len, left);
+ if (ret <= 0)
+ return ret;
+ section->iov_len += ret;
+ if (section->iov_len == sec_len)
+ *crc = crc32c(0, section->iov_base,
+ section->iov_len);
+ }
+
+ return 1;
+}
+
+static struct ceph_msg *ceph_alloc_msg(struct ceph_connection *con,
+ struct ceph_msg_header *hdr,
+ int *skip);
+/*
+ * read (part of) a message.
+ */
+static int read_partial_message(struct ceph_connection *con)
+{
+ struct ceph_msg *m = con->in_msg;
+ void *p;
+ int ret;
+ int to, left;
+ unsigned front_len, middle_len, data_len, data_off;
+ int datacrc = con->msgr->nocrc;
+ int skip;
+
+ dout("read_partial_message con %p msg %p\n", con, m);
+
+ /* header */
+ while (con->in_base_pos < sizeof(con->in_hdr)) {
+ left = sizeof(con->in_hdr) - con->in_base_pos;
+ ret = ceph_tcp_recvmsg(con->sock,
+ (char *)&con->in_hdr + con->in_base_pos,
+ left);
+ if (ret <= 0)
+ return ret;
+ con->in_base_pos += ret;
+ if (con->in_base_pos == sizeof(con->in_hdr)) {
+ u32 crc = crc32c(0, (void *)&con->in_hdr,
+ sizeof(con->in_hdr) - sizeof(con->in_hdr.crc));
+ if (crc != le32_to_cpu(con->in_hdr.crc)) {
+ pr_err("read_partial_message bad hdr "
+ " crc %u != expected %u\n",
+ crc, con->in_hdr.crc);
+ return -EBADMSG;
+ }
+ }
+ }
+ front_len = le32_to_cpu(con->in_hdr.front_len);
+ if (front_len > CEPH_MSG_MAX_FRONT_LEN)
+ return -EIO;
+ middle_len = le32_to_cpu(con->in_hdr.middle_len);
+ if (middle_len > CEPH_MSG_MAX_DATA_LEN)
+ return -EIO;
+ data_len = le32_to_cpu(con->in_hdr.data_len);
+ if (data_len > CEPH_MSG_MAX_DATA_LEN)
+ return -EIO;
+ data_off = le16_to_cpu(con->in_hdr.data_off);
+
+ /* allocate message? */
+ if (!con->in_msg) {
+ dout("got hdr type %d front %d data %d\n", con->in_hdr.type,
+ con->in_hdr.front_len, con->in_hdr.data_len);
+ con->in_msg = ceph_alloc_msg(con, &con->in_hdr, &skip);
+ if (skip) {
+ /* skip this message */
+ dout("alloc_msg returned NULL, skipping message\n");
+ con->in_base_pos = -front_len - middle_len - data_len -
+ sizeof(m->footer);
+ con->in_tag = CEPH_MSGR_TAG_READY;
+ return 0;
+ }
+ if (IS_ERR(con->in_msg)) {
+ ret = PTR_ERR(con->in_msg);
+ con->in_msg = NULL;
+ con->error_msg =
+ "error allocating memory for incoming message";
+ return ret;
+ }
+ m = con->in_msg;
+ m->front.iov_len = 0; /* haven't read it yet */
+ if (m->middle)
+ m->middle->vec.iov_len = 0;
+
+ con->in_msg_pos.page = 0;
+ con->in_msg_pos.page_pos = data_off & ~PAGE_MASK;
+ con->in_msg_pos.data_pos = 0;
+ }
+
+ /* front */
+ ret = read_partial_message_section(con, &m->front, front_len,
+ &con->in_front_crc);
+ if (ret <= 0)
+ return ret;
+
+ /* middle */
+ if (m->middle) {
+ ret = read_partial_message_section(con, &m->middle->vec, middle_len,
+ &con->in_middle_crc);
+ if (ret <= 0)
+ return ret;
+ }
+
+ /* (page) data */
+ while (con->in_msg_pos.data_pos < data_len) {
+ left = min((int)(data_len - con->in_msg_pos.data_pos),
+ (int)(PAGE_SIZE - con->in_msg_pos.page_pos));
+ BUG_ON(m->pages == NULL);
+ p = kmap(m->pages[con->in_msg_pos.page]);
+ ret = ceph_tcp_recvmsg(con->sock, p + con->in_msg_pos.page_pos,
+ left);
+ if (ret > 0 && datacrc)
+ con->in_data_crc =
+ crc32c(con->in_data_crc,
+ p + con->in_msg_pos.page_pos, ret);
+ kunmap(m->pages[con->in_msg_pos.page]);
+ if (ret <= 0)
+ return ret;
+ con->in_msg_pos.data_pos += ret;
+ con->in_msg_pos.page_pos += ret;
+ if (con->in_msg_pos.page_pos == PAGE_SIZE) {
+ con->in_msg_pos.page_pos = 0;
+ con->in_msg_pos.page++;
+ }
+ }
+
+ /* footer */
+ to = sizeof(m->hdr) + sizeof(m->footer);
+ while (con->in_base_pos < to) {
+ left = to - con->in_base_pos;
+ ret = ceph_tcp_recvmsg(con->sock, (char *)&m->footer +
+ (con->in_base_pos - sizeof(m->hdr)),
+ left);
+ if (ret <= 0)
+ return ret;
+ con->in_base_pos += ret;
+ }
+ dout("read_partial_message got msg %p %d (%u) + %d (%u) + %d (%u)\n",
+ m, front_len, m->footer.front_crc, middle_len,
+ m->footer.middle_crc, data_len, m->footer.data_crc);
+
+ /* crc ok? */
+ if (con->in_front_crc != le32_to_cpu(m->footer.front_crc)) {
+ pr_err("read_partial_message %p front crc %u != exp. %u\n",
+ m, con->in_front_crc, m->footer.front_crc);
+ return -EBADMSG;
+ }
+ if (con->in_middle_crc != le32_to_cpu(m->footer.middle_crc)) {
+ pr_err("read_partial_message %p middle crc %u != exp %u\n",
+ m, con->in_middle_crc, m->footer.middle_crc);
+ return -EBADMSG;
+ }
+ if (datacrc &&
+ (m->footer.flags & CEPH_MSG_FOOTER_NOCRC) == 0 &&
+ con->in_data_crc != le32_to_cpu(m->footer.data_crc)) {
+ pr_err("read_partial_message %p data crc %u != exp. %u\n", m,
+ con->in_data_crc, le32_to_cpu(m->footer.data_crc));
+ return -EBADMSG;
+ }
+
+ return 1; /* done! */
+}
+
+/*
+ * Process message. This happens in the worker thread. The callback should
+ * be careful not to do anything that waits on other incoming messages or it
+ * may deadlock.
+ */
+static void process_message(struct ceph_connection *con)
+{
+ struct ceph_msg *msg;
+
+ msg = con->in_msg;
+ con->in_msg = NULL;
+
+ /* if first message, set peer_name */
+ if (con->peer_name.type == 0)
+ con->peer_name = msg->hdr.src.name;
+
+ con->in_seq++;
+ mutex_unlock(&con->mutex);
+
+ dout("===== %p %llu from %s%lld %d=%s len %d+%d (%u %u %u) =====\n",
+ msg, le64_to_cpu(msg->hdr.seq),
+ ENTITY_NAME(msg->hdr.src.name),
+ le16_to_cpu(msg->hdr.type),
+ ceph_msg_type_name(le16_to_cpu(msg->hdr.type)),
+ le32_to_cpu(msg->hdr.front_len),
+ le32_to_cpu(msg->hdr.data_len),
+ con->in_front_crc, con->in_middle_crc, con->in_data_crc);
+ con->ops->dispatch(con, msg);
+
+ mutex_lock(&con->mutex);
+ prepare_read_tag(con);
+}
+
+
+/*
+ * Write something to the socket. Called in a worker thread when the
+ * socket appears to be writeable and we have something ready to send.
+ */
+static int try_write(struct ceph_connection *con)
+{
+ struct ceph_messenger *msgr = con->msgr;
+ int ret = 1;
+
+ dout("try_write start %p state %lu nref %d\n", con, con->state,
+ atomic_read(&con->nref));
+
+ mutex_lock(&con->mutex);
+more:
+ dout("try_write out_kvec_bytes %d\n", con->out_kvec_bytes);
+
+ /* open the socket first? */
+ if (con->sock == NULL) {
+ /*
+ * if we were STANDBY and are reconnecting _this_
+ * connection, bump connect_seq now. Always bump
+ * global_seq.
+ */
+ if (test_and_clear_bit(STANDBY, &con->state))
+ con->connect_seq++;
+
+ prepare_write_banner(msgr, con);
+ prepare_write_connect(msgr, con, 1);
+ prepare_read_banner(con);
+ set_bit(CONNECTING, &con->state);
+ clear_bit(NEGOTIATING, &con->state);
+
+ BUG_ON(con->in_msg);
+ con->in_tag = CEPH_MSGR_TAG_READY;
+ dout("try_write initiating connect on %p new state %lu\n",
+ con, con->state);
+ con->sock = ceph_tcp_connect(con);
+ if (IS_ERR(con->sock)) {
+ con->sock = NULL;
+ con->error_msg = "connect error";
+ ret = -1;
+ goto out;
+ }
+ }
+
+more_kvec:
+ /* kvec data queued? */
+ if (con->out_skip) {
+ ret = write_partial_skip(con);
+ if (ret <= 0)
+ goto done;
+ if (ret < 0) {
+ dout("try_write write_partial_skip err %d\n", ret);
+ goto done;
+ }
+ }
+ if (con->out_kvec_left) {
+ ret = write_partial_kvec(con);
+ if (ret <= 0)
+ goto done;
+ }
+
+ /* msg pages? */
+ if (con->out_msg) {
+ if (con->out_msg_done) {
+ ceph_msg_put(con->out_msg);
+ con->out_msg = NULL; /* we're done with this one */
+ goto do_next;
+ }
+
+ ret = write_partial_msg_pages(con);
+ if (ret == 1)
+ goto more_kvec; /* we need to send the footer, too! */
+ if (ret == 0)
+ goto done;
+ if (ret < 0) {
+ dout("try_write write_partial_msg_pages err %d\n",
+ ret);
+ goto done;
+ }
+ }
+
+do_next:
+ if (!test_bit(CONNECTING, &con->state)) {
+ /* is anything else pending? */
+ if (!list_empty(&con->out_queue)) {
+ prepare_write_message(con);
+ goto more;
+ }
+ if (con->in_seq > con->in_seq_acked) {
+ prepare_write_ack(con);
+ goto more;
+ }
+ if (test_and_clear_bit(KEEPALIVE_PENDING, &con->state)) {
+ prepare_write_keepalive(con);
+ goto more;
+ }
+ }
+
+ /* Nothing to do! */
+ clear_bit(WRITE_PENDING, &con->state);
+ dout("try_write nothing else to write.\n");
+done:
+ ret = 0;
+out:
+ mutex_unlock(&con->mutex);
+ dout("try_write done on %p\n", con);
+ return ret;
+}
+
+
+
+/*
+ * Read what we can from the socket.
+ */
+static int try_read(struct ceph_connection *con)
+{
+ struct ceph_messenger *msgr;
+ int ret = -1;
+
+ if (!con->sock)
+ return 0;
+
+ if (test_bit(STANDBY, &con->state))
+ return 0;
+
+ dout("try_read start on %p\n", con);
+ msgr = con->msgr;
+
+ mutex_lock(&con->mutex);
+
+more:
+ dout("try_read tag %d in_base_pos %d\n", (int)con->in_tag,
+ con->in_base_pos);
+ if (test_bit(CONNECTING, &con->state)) {
+ if (!test_bit(NEGOTIATING, &con->state)) {
+ dout("try_read connecting\n");
+ ret = read_partial_banner(con);
+ if (ret <= 0)
+ goto done;
+ if (process_banner(con) < 0) {
+ ret = -1;
+ goto out;
+ }
+ }
+ ret = read_partial_connect(con);
+ if (ret <= 0)
+ goto done;
+ if (process_connect(con) < 0) {
+ ret = -1;
+ goto out;
+ }
+ goto more;
+ }
+
+ if (con->in_base_pos < 0) {
+ /*
+ * skipping + discarding content.
+ *
+ * FIXME: there must be a better way to do this!
+ */
+ static char buf[1024];
+ int skip = min(1024, -con->in_base_pos);
+ dout("skipping %d / %d bytes\n", skip, -con->in_base_pos);
+ ret = ceph_tcp_recvmsg(con->sock, buf, skip);
+ if (ret <= 0)
+ goto done;
+ con->in_base_pos += ret;
+ if (con->in_base_pos)
+ goto more;
+ }
+ if (con->in_tag == CEPH_MSGR_TAG_READY) {
+ /*
+ * what's next?
+ */
+ ret = ceph_tcp_recvmsg(con->sock, &con->in_tag, 1);
+ if (ret <= 0)
+ goto done;
+ dout("try_read got tag %d\n", (int)con->in_tag);
+ switch (con->in_tag) {
+ case CEPH_MSGR_TAG_MSG:
+ prepare_read_message(con);
+ break;
+ case CEPH_MSGR_TAG_ACK:
+ prepare_read_ack(con);
+ break;
+ case CEPH_MSGR_TAG_CLOSE:
+ set_bit(CLOSED, &con->state); /* fixme */
+ goto done;
+ default:
+ goto bad_tag;
+ }
+ }
+ if (con->in_tag == CEPH_MSGR_TAG_MSG) {
+ ret = read_partial_message(con);
+ if (ret <= 0) {
+ switch (ret) {
+ case -EBADMSG:
+ con->error_msg = "bad crc";
+ ret = -EIO;
+ goto out;
+ case -EIO:
+ con->error_msg = "io error";
+ goto out;
+ default:
+ goto done;
+ }
+ }
+ if (con->in_tag == CEPH_MSGR_TAG_READY)
+ goto more;
+ process_message(con);
+ goto more;
+ }
+ if (con->in_tag == CEPH_MSGR_TAG_ACK) {
+ ret = read_partial_ack(con);
+ if (ret <= 0)
+ goto done;
+ process_ack(con);
+ goto more;
+ }
+
+done:
+ ret = 0;
+out:
+ mutex_unlock(&con->mutex);
+ dout("try_read done on %p\n", con);
+ return ret;
+
+bad_tag:
+ pr_err("try_read bad con->in_tag = %d\n", (int)con->in_tag);
+ con->error_msg = "protocol error, garbage tag";
+ ret = -1;
+ goto out;
+}
+
+
+/*
+ * Atomically queue work on a connection. Bump @con reference to
+ * avoid races with connection teardown.
+ *
+ * There is some trickery going on with QUEUED and BUSY because we
+ * only want a _single_ thread operating on each connection at any
+ * point in time, but we want to use all available CPUs.
+ *
+ * The worker thread only proceeds if it can atomically set BUSY. It
+ * clears QUEUED and does it's thing. When it thinks it's done, it
+ * clears BUSY, then rechecks QUEUED.. if it's set again, it loops
+ * (tries again to set BUSY).
+ *
+ * To queue work, we first set QUEUED, _then_ if BUSY isn't set, we
+ * try to queue work. If that fails (work is already queued, or BUSY)
+ * we give up (work also already being done or is queued) but leave QUEUED
+ * set so that the worker thread will loop if necessary.
+ */
+static void queue_con(struct ceph_connection *con)
+{
+ if (test_bit(DEAD, &con->state)) {
+ dout("queue_con %p ignoring: DEAD\n",
+ con);
+ return;
+ }
+
+ if (!con->ops->get(con)) {
+ dout("queue_con %p ref count 0\n", con);
+ return;
+ }
+
+ set_bit(QUEUED, &con->state);
+ if (test_bit(BUSY, &con->state)) {
+ dout("queue_con %p - already BUSY\n", con);
+ con->ops->put(con);
+ } else if (!queue_work(ceph_msgr_wq, &con->work.work)) {
+ dout("queue_con %p - already queued\n", con);
+ con->ops->put(con);
+ } else {
+ dout("queue_con %p\n", con);
+ }
+}
+
+/*
+ * Do some work on a connection. Drop a connection ref when we're done.
+ */
+static void con_work(struct work_struct *work)
+{
+ struct ceph_connection *con = container_of(work, struct ceph_connection,
+ work.work);
+ int backoff = 0;
+
+more:
+ if (test_and_set_bit(BUSY, &con->state) != 0) {
+ dout("con_work %p BUSY already set\n", con);
+ goto out;
+ }
+ dout("con_work %p start, clearing QUEUED\n", con);
+ clear_bit(QUEUED, &con->state);
+
+ if (test_bit(CLOSED, &con->state)) { /* e.g. if we are replaced */
+ dout("con_work CLOSED\n");
+ con_close_socket(con);
+ goto done;
+ }
+ if (test_and_clear_bit(OPENING, &con->state)) {
+ /* reopen w/ new peer */
+ dout("con_work OPENING\n");
+ con_close_socket(con);
+ }
+
+ if (test_and_clear_bit(SOCK_CLOSED, &con->state) ||
+ try_read(con) < 0 ||
+ try_write(con) < 0) {
+ backoff = 1;
+ ceph_fault(con); /* error/fault path */
+ }
+
+done:
+ clear_bit(BUSY, &con->state);
+ dout("con->state=%lu\n", con->state);
+ if (test_bit(QUEUED, &con->state)) {
+ if (!backoff || test_bit(OPENING, &con->state)) {
+ dout("con_work %p QUEUED reset, looping\n", con);
+ goto more;
+ }
+ dout("con_work %p QUEUED reset, but just faulted\n", con);
+ clear_bit(QUEUED, &con->state);
+ }
+ dout("con_work %p done\n", con);
+
+out:
+ con->ops->put(con);
+}
+
+
+/*
+ * Generic error/fault handler. A retry mechanism is used with
+ * exponential backoff
+ */
+static void ceph_fault(struct ceph_connection *con)
+{
+ pr_err("%s%lld %s %s\n", ENTITY_NAME(con->peer_name),
+ pr_addr(&con->peer_addr.in_addr), con->error_msg);
+ dout("fault %p state %lu to peer %s\n",
+ con, con->state, pr_addr(&con->peer_addr.in_addr));
+
+ if (test_bit(LOSSYTX, &con->state)) {
+ dout("fault on LOSSYTX channel\n");
+ goto out;
+ }
+
+ mutex_lock(&con->mutex);
+ if (test_bit(CLOSED, &con->state))
+ goto out_unlock;
+
+ con_close_socket(con);
+
+ if (con->in_msg) {
+ ceph_msg_put(con->in_msg);
+ con->in_msg = NULL;
+ }
+
+ /* Requeue anything that hasn't been acked */
+ list_splice_init(&con->out_sent, &con->out_queue);
+
+ /* If there are no messages in the queue, place the connection
+ * in a STANDBY state (i.e., don't try to reconnect just yet). */
+ if (list_empty(&con->out_queue) && !con->out_keepalive_pending) {
+ dout("fault setting STANDBY\n");
+ set_bit(STANDBY, &con->state);
+ } else {
+ /* retry after a delay. */
+ if (con->delay == 0)
+ con->delay = BASE_DELAY_INTERVAL;
+ else if (con->delay < MAX_DELAY_INTERVAL)
+ con->delay *= 2;
+ dout("fault queueing %p delay %lu\n", con, con->delay);
+ con->ops->get(con);
+ if (queue_delayed_work(ceph_msgr_wq, &con->work,
+ round_jiffies_relative(con->delay)) == 0)
+ con->ops->put(con);
+ }
+
+out_unlock:
+ mutex_unlock(&con->mutex);
+out:
+ /*
+ * in case we faulted due to authentication, invalidate our
+ * current tickets so that we can get new ones.
+ */
+ if (con->auth_retry && con->ops->invalidate_authorizer) {
+ dout("calling invalidate_authorizer()\n");
+ con->ops->invalidate_authorizer(con);
+ }
+
+ if (con->ops->fault)
+ con->ops->fault(con);
+}
+
+
+
+/*
+ * create a new messenger instance
+ */
+struct ceph_messenger *ceph_messenger_create(struct ceph_entity_addr *myaddr)
+{
+ struct ceph_messenger *msgr;
+
+ msgr = kzalloc(sizeof(*msgr), GFP_KERNEL);
+ if (msgr == NULL)
+ return ERR_PTR(-ENOMEM);
+
+ spin_lock_init(&msgr->global_seq_lock);
+
+ /* the zero page is needed if a request is "canceled" while the message
+ * is being written over the socket */
+ msgr->zero_page = alloc_page(GFP_KERNEL | __GFP_ZERO);
+ if (!msgr->zero_page) {
+ kfree(msgr);
+ return ERR_PTR(-ENOMEM);
+ }
+ kmap(msgr->zero_page);
+
+ if (myaddr)
+ msgr->inst.addr = *myaddr;
+
+ /* select a random nonce */
+ msgr->inst.addr.type = 0;
+ get_random_bytes(&msgr->inst.addr.nonce, sizeof(msgr->inst.addr.nonce));
+ encode_my_addr(msgr);
+
+ dout("messenger_create %p\n", msgr);
+ return msgr;
+}
+
+void ceph_messenger_destroy(struct ceph_messenger *msgr)
+{
+ dout("destroy %p\n", msgr);
+ kunmap(msgr->zero_page);
+ __free_page(msgr->zero_page);
+ kfree(msgr);
+ dout("destroyed messenger %p\n", msgr);
+}
+
+/*
+ * Queue up an outgoing message on the given connection.
+ */
+void ceph_con_send(struct ceph_connection *con, struct ceph_msg *msg)
+{
+ if (test_bit(CLOSED, &con->state)) {
+ dout("con_send %p closed, dropping %p\n", con, msg);
+ ceph_msg_put(msg);
+ return;
+ }
+
+ /* set src+dst */
+ msg->hdr.src.name = con->msgr->inst.name;
+ msg->hdr.src.addr = con->msgr->my_enc_addr;
+ msg->hdr.orig_src = msg->hdr.src;
+
+ BUG_ON(msg->front.iov_len != le32_to_cpu(msg->hdr.front_len));
+
+ /* queue */
+ mutex_lock(&con->mutex);
+ BUG_ON(!list_empty(&msg->list_head));
+ list_add_tail(&msg->list_head, &con->out_queue);
+ dout("----- %p to %s%lld %d=%s len %d+%d+%d -----\n", msg,
+ ENTITY_NAME(con->peer_name), le16_to_cpu(msg->hdr.type),
+ ceph_msg_type_name(le16_to_cpu(msg->hdr.type)),
+ le32_to_cpu(msg->hdr.front_len),
+ le32_to_cpu(msg->hdr.middle_len),
+ le32_to_cpu(msg->hdr.data_len));
+ mutex_unlock(&con->mutex);
+
+ /* if there wasn't anything waiting to send before, queue
+ * new work */
+ if (test_and_set_bit(WRITE_PENDING, &con->state) == 0)
+ queue_con(con);
+}
+
+/*
+ * Revoke a message that was previously queued for send
+ */
+void ceph_con_revoke(struct ceph_connection *con, struct ceph_msg *msg)
+{
+ mutex_lock(&con->mutex);
+ if (!list_empty(&msg->list_head)) {
+ dout("con_revoke %p msg %p\n", con, msg);
+ list_del_init(&msg->list_head);
+ ceph_msg_put(msg);
+ msg->hdr.seq = 0;
+ if (con->out_msg == msg) {
+ ceph_msg_put(con->out_msg);
+ con->out_msg = NULL;
+ }
+ if (con->out_kvec_is_msg) {
+ con->out_skip = con->out_kvec_bytes;
+ con->out_kvec_is_msg = false;
+ }
+ } else {
+ dout("con_revoke %p msg %p - not queued (sent?)\n", con, msg);
+ }
+ mutex_unlock(&con->mutex);
+}
+
+/*
+ * Revoke a message that we may be reading data into
+ */
+void ceph_con_revoke_message(struct ceph_connection *con, struct ceph_msg *msg)
+{
+ mutex_lock(&con->mutex);
+ if (con->in_msg && con->in_msg == msg) {
+ unsigned front_len = le32_to_cpu(con->in_hdr.front_len);
+ unsigned middle_len = le32_to_cpu(con->in_hdr.middle_len);
+ unsigned data_len = le32_to_cpu(con->in_hdr.data_len);
+
+ /* skip rest of message */
+ dout("con_revoke_pages %p msg %p revoked\n", con, msg);
+ con->in_base_pos = con->in_base_pos -
+ sizeof(struct ceph_msg_header) -
+ front_len -
+ middle_len -
+ data_len -
+ sizeof(struct ceph_msg_footer);
+ ceph_msg_put(con->in_msg);
+ con->in_msg = NULL;
+ con->in_tag = CEPH_MSGR_TAG_READY;
+ } else {
+ dout("con_revoke_pages %p msg %p pages %p no-op\n",
+ con, con->in_msg, msg);
+ }
+ mutex_unlock(&con->mutex);
+}
+
+/*
+ * Queue a keepalive byte to ensure the tcp connection is alive.
+ */
+void ceph_con_keepalive(struct ceph_connection *con)
+{
+ if (test_and_set_bit(KEEPALIVE_PENDING, &con->state) == 0 &&
+ test_and_set_bit(WRITE_PENDING, &con->state) == 0)
+ queue_con(con);
+}
+
+
+/*
+ * construct a new message with given type, size
+ * the new msg has a ref count of 1.
+ */
+struct ceph_msg *ceph_msg_new(int type, int front_len,
+ int page_len, int page_off, struct page **pages)
+{
+ struct ceph_msg *m;
+
+ m = kmalloc(sizeof(*m), GFP_NOFS);
+ if (m == NULL)
+ goto out;
+ kref_init(&m->kref);
+ INIT_LIST_HEAD(&m->list_head);
+
+ m->hdr.type = cpu_to_le16(type);
+ m->hdr.front_len = cpu_to_le32(front_len);
+ m->hdr.middle_len = 0;
+ m->hdr.data_len = cpu_to_le32(page_len);
+ m->hdr.data_off = cpu_to_le16(page_off);
+ m->hdr.priority = cpu_to_le16(CEPH_MSG_PRIO_DEFAULT);
+ m->footer.front_crc = 0;
+ m->footer.middle_crc = 0;
+ m->footer.data_crc = 0;
+ m->front_max = front_len;
+ m->front_is_vmalloc = false;
+ m->more_to_follow = false;
+ m->pool = NULL;
+
+ /* front */
+ if (front_len) {
+ if (front_len > PAGE_CACHE_SIZE) {
+ m->front.iov_base = __vmalloc(front_len, GFP_NOFS,
+ PAGE_KERNEL);
+ m->front_is_vmalloc = true;
+ } else {
+ m->front.iov_base = kmalloc(front_len, GFP_NOFS);
+ }
+ if (m->front.iov_base == NULL) {
+ pr_err("msg_new can't allocate %d bytes\n",
+ front_len);
+ goto out2;
+ }
+ } else {
+ m->front.iov_base = NULL;
+ }
+ m->front.iov_len = front_len;
+
+ /* middle */
+ m->middle = NULL;
+
+ /* data */
+ m->nr_pages = calc_pages_for(page_off, page_len);
+ m->pages = pages;
+ m->pagelist = NULL;
+
+ dout("ceph_msg_new %p page %d~%d -> %d\n", m, page_off, page_len,
+ m->nr_pages);
+ return m;
+
+out2:
+ ceph_msg_put(m);
+out:
+ pr_err("msg_new can't create type %d len %d\n", type, front_len);
+ return ERR_PTR(-ENOMEM);
+}
+
+/*
+ * Allocate "middle" portion of a message, if it is needed and wasn't
+ * allocated by alloc_msg. This allows us to read a small fixed-size
+ * per-type header in the front and then gracefully fail (i.e.,
+ * propagate the error to the caller based on info in the front) when
+ * the middle is too large.
+ */
+static int ceph_alloc_middle(struct ceph_connection *con, struct ceph_msg *msg)
+{
+ int type = le16_to_cpu(msg->hdr.type);
+ int middle_len = le32_to_cpu(msg->hdr.middle_len);
+
+ dout("alloc_middle %p type %d %s middle_len %d\n", msg, type,
+ ceph_msg_type_name(type), middle_len);
+ BUG_ON(!middle_len);
+ BUG_ON(msg->middle);
+
+ msg->middle = ceph_buffer_new(middle_len, GFP_NOFS);
+ if (!msg->middle)
+ return -ENOMEM;
+ return 0;
+}
+
+/*
+ * Generic message allocator, for incoming messages.
+ */
+static struct ceph_msg *ceph_alloc_msg(struct ceph_connection *con,
+ struct ceph_msg_header *hdr,
+ int *skip)
+{
+ int type = le16_to_cpu(hdr->type);
+ int front_len = le32_to_cpu(hdr->front_len);
+ int middle_len = le32_to_cpu(hdr->middle_len);
+ struct ceph_msg *msg = NULL;
+ int ret;
+
+ if (con->ops->alloc_msg) {
+ mutex_unlock(&con->mutex);
+ msg = con->ops->alloc_msg(con, hdr, skip);
+ mutex_lock(&con->mutex);
+ if (IS_ERR(msg))
+ return msg;
+
+ if (*skip)
+ return NULL;
+ }
+ if (!msg) {
+ *skip = 0;
+ msg = ceph_msg_new(type, front_len, 0, 0, NULL);
+ if (!msg) {
+ pr_err("unable to allocate msg type %d len %d\n",
+ type, front_len);
+ return ERR_PTR(-ENOMEM);
+ }
+ }
+ memcpy(&msg->hdr, &con->in_hdr, sizeof(con->in_hdr));
+
+ if (middle_len) {
+ ret = ceph_alloc_middle(con, msg);
+
+ if (ret < 0) {
+ ceph_msg_put(msg);
+ return msg;
+ }
+ }
+
+ return msg;
+}
+
+
+/*
+ * Free a generically kmalloc'd message.
+ */
+void ceph_msg_kfree(struct ceph_msg *m)
+{
+ dout("msg_kfree %p\n", m);
+ if (m->front_is_vmalloc)
+ vfree(m->front.iov_base);
+ else
+ kfree(m->front.iov_base);
+ kfree(m);
+}
+
+/*
+ * Drop a msg ref. Destroy as needed.
+ */
+void ceph_msg_last_put(struct kref *kref)
+{
+ struct ceph_msg *m = container_of(kref, struct ceph_msg, kref);
+
+ dout("ceph_msg_put last one on %p\n", m);
+ WARN_ON(!list_empty(&m->list_head));
+
+ /* drop middle, data, if any */
+ if (m->middle) {
+ ceph_buffer_put(m->middle);
+ m->middle = NULL;
+ }
+ m->nr_pages = 0;
+ m->pages = NULL;
+
+ if (m->pagelist) {
+ ceph_pagelist_release(m->pagelist);
+ kfree(m->pagelist);
+ m->pagelist = NULL;
+ }
+
+ if (m->pool)
+ ceph_msgpool_put(m->pool, m);
+ else
+ ceph_msg_kfree(m);
+}
+
+void ceph_msg_dump(struct ceph_msg *msg)
+{
+ pr_debug("msg_dump %p (front_max %d nr_pages %d)\n", msg,
+ msg->front_max, msg->nr_pages);
+ print_hex_dump(KERN_DEBUG, "header: ",
+ DUMP_PREFIX_OFFSET, 16, 1,
+ &msg->hdr, sizeof(msg->hdr), true);
+ print_hex_dump(KERN_DEBUG, " front: ",
+ DUMP_PREFIX_OFFSET, 16, 1,
+ msg->front.iov_base, msg->front.iov_len, true);
+ if (msg->middle)
+ print_hex_dump(KERN_DEBUG, "middle: ",
+ DUMP_PREFIX_OFFSET, 16, 1,
+ msg->middle->vec.iov_base,
+ msg->middle->vec.iov_len, true);
+ print_hex_dump(KERN_DEBUG, "footer: ",
+ DUMP_PREFIX_OFFSET, 16, 1,
+ &msg->footer, sizeof(msg->footer), true);
+}
--- /dev/null
+#ifndef __FS_CEPH_MESSENGER_H
+#define __FS_CEPH_MESSENGER_H
+
+#include <linux/kref.h>
+#include <linux/mutex.h>
+#include <linux/net.h>
+#include <linux/radix-tree.h>
+#include <linux/uio.h>
+#include <linux/version.h>
+#include <linux/workqueue.h>
+
+#include "types.h"
+#include "buffer.h"
+
+struct ceph_msg;
+struct ceph_connection;
+
+extern struct workqueue_struct *ceph_msgr_wq; /* receive work queue */
+
+/*
+ * Ceph defines these callbacks for handling connection events.
+ */
+struct ceph_connection_operations {
+ struct ceph_connection *(*get)(struct ceph_connection *);
+ void (*put)(struct ceph_connection *);
+
+ /* handle an incoming message. */
+ void (*dispatch) (struct ceph_connection *con, struct ceph_msg *m);
+
+ /* authorize an outgoing connection */
+ int (*get_authorizer) (struct ceph_connection *con,
+ void **buf, int *len, int *proto,
+ void **reply_buf, int *reply_len, int force_new);
+ int (*verify_authorizer_reply) (struct ceph_connection *con, int len);
+ int (*invalidate_authorizer)(struct ceph_connection *con);
+
+ /* protocol version mismatch */
+ void (*bad_proto) (struct ceph_connection *con);
+
+ /* there was some error on the socket (disconnect, whatever) */
+ void (*fault) (struct ceph_connection *con);
+
+ /* a remote host as terminated a message exchange session, and messages
+ * we sent (or they tried to send us) may be lost. */
+ void (*peer_reset) (struct ceph_connection *con);
+
+ struct ceph_msg * (*alloc_msg) (struct ceph_connection *con,
+ struct ceph_msg_header *hdr,
+ int *skip);
+};
+
+extern const char *ceph_name_type_str(int t);
+
+/* use format string %s%d */
+#define ENTITY_NAME(n) ceph_name_type_str((n).type), le64_to_cpu((n).num)
+
+struct ceph_messenger {
+ struct ceph_entity_inst inst; /* my name+address */
+ struct ceph_entity_addr my_enc_addr;
+ struct page *zero_page; /* used in certain error cases */
+
+ bool nocrc;
+
+ /*
+ * the global_seq counts connections i (attempt to) initiate
+ * in order to disambiguate certain connect race conditions.
+ */
+ u32 global_seq;
+ spinlock_t global_seq_lock;
+};
+
+/*
+ * a single message. it contains a header (src, dest, message type, etc.),
+ * footer (crc values, mainly), a "front" message body, and possibly a
+ * data payload (stored in some number of pages).
+ */
+struct ceph_msg {
+ struct ceph_msg_header hdr; /* header */
+ struct ceph_msg_footer footer; /* footer */
+ struct kvec front; /* unaligned blobs of message */
+ struct ceph_buffer *middle;
+ struct page **pages; /* data payload. NOT OWNER. */
+ unsigned nr_pages; /* size of page array */
+ struct ceph_pagelist *pagelist; /* instead of pages */
+ struct list_head list_head;
+ struct kref kref;
+ bool front_is_vmalloc;
+ bool more_to_follow;
+ int front_max;
+
+ struct ceph_msgpool *pool;
+};
+
+struct ceph_msg_pos {
+ int page, page_pos; /* which page; offset in page */
+ int data_pos; /* offset in data payload */
+ int did_page_crc; /* true if we've calculated crc for current page */
+};
+
+/* ceph connection fault delay defaults, for exponential backoff */
+#define BASE_DELAY_INTERVAL (HZ/2)
+#define MAX_DELAY_INTERVAL (5 * 60 * HZ)
+
+/*
+ * ceph_connection state bit flags
+ *
+ * QUEUED and BUSY are used together to ensure that only a single
+ * thread is currently opening, reading or writing data to the socket.
+ */
+#define LOSSYTX 0 /* we can close channel or drop messages on errors */
+#define CONNECTING 1
+#define NEGOTIATING 2
+#define KEEPALIVE_PENDING 3
+#define WRITE_PENDING 4 /* we have data ready to send */
+#define QUEUED 5 /* there is work queued on this connection */
+#define BUSY 6 /* work is being done */
+#define STANDBY 8 /* no outgoing messages, socket closed. we keep
+ * the ceph_connection around to maintain shared
+ * state with the peer. */
+#define CLOSED 10 /* we've closed the connection */
+#define SOCK_CLOSED 11 /* socket state changed to closed */
+#define OPENING 13 /* open connection w/ (possibly new) peer */
+#define DEAD 14 /* dead, about to kfree */
+
+/*
+ * A single connection with another host.
+ *
+ * We maintain a queue of outgoing messages, and some session state to
+ * ensure that we can preserve the lossless, ordered delivery of
+ * messages in the case of a TCP disconnect.
+ */
+struct ceph_connection {
+ void *private;
+ atomic_t nref;
+
+ const struct ceph_connection_operations *ops;
+
+ struct ceph_messenger *msgr;
+ struct socket *sock;
+ unsigned long state; /* connection state (see flags above) */
+ const char *error_msg; /* error message, if any */
+
+ struct ceph_entity_addr peer_addr; /* peer address */
+ struct ceph_entity_name peer_name; /* peer name */
+ struct ceph_entity_addr peer_addr_for_me;
+ u32 connect_seq; /* identify the most recent connection
+ attempt for this connection, client */
+ u32 peer_global_seq; /* peer's global seq for this connection */
+
+ int auth_retry; /* true if we need a newer authorizer */
+ void *auth_reply_buf; /* where to put the authorizer reply */
+ int auth_reply_buf_len;
+
+ struct mutex mutex;
+
+ /* out queue */
+ struct list_head out_queue;
+ struct list_head out_sent; /* sending or sent but unacked */
+ u64 out_seq; /* last message queued for send */
+ u64 out_seq_sent; /* last message sent */
+ bool out_keepalive_pending;
+
+ u64 in_seq, in_seq_acked; /* last message received, acked */
+
+ /* connection negotiation temps */
+ char in_banner[CEPH_BANNER_MAX_LEN];
+ union {
+ struct { /* outgoing connection */
+ struct ceph_msg_connect out_connect;
+ struct ceph_msg_connect_reply in_reply;
+ };
+ struct { /* incoming */
+ struct ceph_msg_connect in_connect;
+ struct ceph_msg_connect_reply out_reply;
+ };
+ };
+ struct ceph_entity_addr actual_peer_addr;
+
+ /* message out temps */
+ struct ceph_msg *out_msg; /* sending message (== tail of
+ out_sent) */
+ bool out_msg_done;
+ struct ceph_msg_pos out_msg_pos;
+
+ struct kvec out_kvec[8], /* sending header/footer data */
+ *out_kvec_cur;
+ int out_kvec_left; /* kvec's left in out_kvec */
+ int out_skip; /* skip this many bytes */
+ int out_kvec_bytes; /* total bytes left */
+ bool out_kvec_is_msg; /* kvec refers to out_msg */
+ int out_more; /* there is more data after the kvecs */
+ __le64 out_temp_ack; /* for writing an ack */
+
+ /* message in temps */
+ struct ceph_msg_header in_hdr;
+ struct ceph_msg *in_msg;
+ struct ceph_msg_pos in_msg_pos;
+ u32 in_front_crc, in_middle_crc, in_data_crc; /* calculated crc */
+
+ char in_tag; /* protocol control byte */
+ int in_base_pos; /* bytes read */
+ __le64 in_temp_ack; /* for reading an ack */
+
+ struct delayed_work work; /* send|recv work */
+ unsigned long delay; /* current delay interval */
+};
+
+
+extern const char *pr_addr(const struct sockaddr_storage *ss);
+extern int ceph_parse_ips(const char *c, const char *end,
+ struct ceph_entity_addr *addr,
+ int max_count, int *count);
+
+
+extern int ceph_msgr_init(void);
+extern void ceph_msgr_exit(void);
+
+extern struct ceph_messenger *ceph_messenger_create(
+ struct ceph_entity_addr *myaddr);
+extern void ceph_messenger_destroy(struct ceph_messenger *);
+
+extern void ceph_con_init(struct ceph_messenger *msgr,
+ struct ceph_connection *con);
+extern void ceph_con_open(struct ceph_connection *con,
+ struct ceph_entity_addr *addr);
+extern bool ceph_con_opened(struct ceph_connection *con);
+extern void ceph_con_close(struct ceph_connection *con);
+extern void ceph_con_send(struct ceph_connection *con, struct ceph_msg *msg);
+extern void ceph_con_revoke(struct ceph_connection *con, struct ceph_msg *msg);
+extern void ceph_con_revoke_message(struct ceph_connection *con,
+ struct ceph_msg *msg);
+extern void ceph_con_keepalive(struct ceph_connection *con);
+extern struct ceph_connection *ceph_con_get(struct ceph_connection *con);
+extern void ceph_con_put(struct ceph_connection *con);
+
+extern struct ceph_msg *ceph_msg_new(int type, int front_len,
+ int page_len, int page_off,
+ struct page **pages);
+extern void ceph_msg_kfree(struct ceph_msg *m);
+
+
+static inline struct ceph_msg *ceph_msg_get(struct ceph_msg *msg)
+{
+ kref_get(&msg->kref);
+ return msg;
+}
+extern void ceph_msg_last_put(struct kref *kref);
+static inline void ceph_msg_put(struct ceph_msg *msg)
+{
+ kref_put(&msg->kref, ceph_msg_last_put);
+}
+
+extern void ceph_msg_dump(struct ceph_msg *msg);
+
+#endif
--- /dev/null
+#include "ceph_debug.h"
+
+#include <linux/types.h>
+#include <linux/slab.h>
+#include <linux/random.h>
+#include <linux/sched.h>
+
+#include "mon_client.h"
+#include "super.h"
+#include "auth.h"
+#include "decode.h"
+
+/*
+ * Interact with Ceph monitor cluster. Handle requests for new map
+ * versions, and periodically resend as needed. Also implement
+ * statfs() and umount().
+ *
+ * A small cluster of Ceph "monitors" are responsible for managing critical
+ * cluster configuration and state information. An odd number (e.g., 3, 5)
+ * of cmon daemons use a modified version of the Paxos part-time parliament
+ * algorithm to manage the MDS map (mds cluster membership), OSD map, and
+ * list of clients who have mounted the file system.
+ *
+ * We maintain an open, active session with a monitor at all times in order to
+ * receive timely MDSMap updates. We periodically send a keepalive byte on the
+ * TCP socket to ensure we detect a failure. If the connection does break, we
+ * randomly hunt for a new monitor. Once the connection is reestablished, we
+ * resend any outstanding requests.
+ */
+
+const static struct ceph_connection_operations mon_con_ops;
+
+static int __validate_auth(struct ceph_mon_client *monc);
+
+/*
+ * Decode a monmap blob (e.g., during mount).
+ */
+struct ceph_monmap *ceph_monmap_decode(void *p, void *end)
+{
+ struct ceph_monmap *m = NULL;
+ int i, err = -EINVAL;
+ struct ceph_fsid fsid;
+ u32 epoch, num_mon;
+ u16 version;
+ u32 len;
+
+ ceph_decode_32_safe(&p, end, len, bad);
+ ceph_decode_need(&p, end, len, bad);
+
+ dout("monmap_decode %p %p len %d\n", p, end, (int)(end-p));
+
+ ceph_decode_16_safe(&p, end, version, bad);
+
+ ceph_decode_need(&p, end, sizeof(fsid) + 2*sizeof(u32), bad);
+ ceph_decode_copy(&p, &fsid, sizeof(fsid));
+ epoch = ceph_decode_32(&p);
+
+ num_mon = ceph_decode_32(&p);
+ ceph_decode_need(&p, end, num_mon*sizeof(m->mon_inst[0]), bad);
+
+ if (num_mon >= CEPH_MAX_MON)
+ goto bad;
+ m = kmalloc(sizeof(*m) + sizeof(m->mon_inst[0])*num_mon, GFP_NOFS);
+ if (m == NULL)
+ return ERR_PTR(-ENOMEM);
+ m->fsid = fsid;
+ m->epoch = epoch;
+ m->num_mon = num_mon;
+ ceph_decode_copy(&p, m->mon_inst, num_mon*sizeof(m->mon_inst[0]));
+ for (i = 0; i < num_mon; i++)
+ ceph_decode_addr(&m->mon_inst[i].addr);
+
+ dout("monmap_decode epoch %d, num_mon %d\n", m->epoch,
+ m->num_mon);
+ for (i = 0; i < m->num_mon; i++)
+ dout("monmap_decode mon%d is %s\n", i,
+ pr_addr(&m->mon_inst[i].addr.in_addr));
+ return m;
+
+bad:
+ dout("monmap_decode failed with %d\n", err);
+ kfree(m);
+ return ERR_PTR(err);
+}
+
+/*
+ * return true if *addr is included in the monmap.
+ */
+int ceph_monmap_contains(struct ceph_monmap *m, struct ceph_entity_addr *addr)
+{
+ int i;
+
+ for (i = 0; i < m->num_mon; i++)
+ if (memcmp(addr, &m->mon_inst[i].addr, sizeof(*addr)) == 0)
+ return 1;
+ return 0;
+}
+
+/*
+ * Send an auth request.
+ */
+static void __send_prepared_auth_request(struct ceph_mon_client *monc, int len)
+{
+ monc->pending_auth = 1;
+ monc->m_auth->front.iov_len = len;
+ monc->m_auth->hdr.front_len = cpu_to_le32(len);
+ ceph_msg_get(monc->m_auth); /* keep our ref */
+ ceph_con_send(monc->con, monc->m_auth);
+}
+
+/*
+ * Close monitor session, if any.
+ */
+static void __close_session(struct ceph_mon_client *monc)
+{
+ if (monc->con) {
+ dout("__close_session closing mon%d\n", monc->cur_mon);
+ ceph_con_revoke(monc->con, monc->m_auth);
+ ceph_con_close(monc->con);
+ monc->cur_mon = -1;
+ monc->pending_auth = 0;
+ ceph_auth_reset(monc->auth);
+ }
+}
+
+/*
+ * Open a session with a (new) monitor.
+ */
+static int __open_session(struct ceph_mon_client *monc)
+{
+ char r;
+ int ret;
+
+ if (monc->cur_mon < 0) {
+ get_random_bytes(&r, 1);
+ monc->cur_mon = r % monc->monmap->num_mon;
+ dout("open_session num=%d r=%d -> mon%d\n",
+ monc->monmap->num_mon, r, monc->cur_mon);
+ monc->sub_sent = 0;
+ monc->sub_renew_after = jiffies; /* i.e., expired */
+ monc->want_next_osdmap = !!monc->want_next_osdmap;
+
+ dout("open_session mon%d opening\n", monc->cur_mon);
+ monc->con->peer_name.type = CEPH_ENTITY_TYPE_MON;
+ monc->con->peer_name.num = cpu_to_le64(monc->cur_mon);
+ ceph_con_open(monc->con,
+ &monc->monmap->mon_inst[monc->cur_mon].addr);
+
+ /* initiatiate authentication handshake */
+ ret = ceph_auth_build_hello(monc->auth,
+ monc->m_auth->front.iov_base,
+ monc->m_auth->front_max);
+ __send_prepared_auth_request(monc, ret);
+ } else {
+ dout("open_session mon%d already open\n", monc->cur_mon);
+ }
+ return 0;
+}
+
+static bool __sub_expired(struct ceph_mon_client *monc)
+{
+ return time_after_eq(jiffies, monc->sub_renew_after);
+}
+
+/*
+ * Reschedule delayed work timer.
+ */
+static void __schedule_delayed(struct ceph_mon_client *monc)
+{
+ unsigned delay;
+
+ if (monc->cur_mon < 0 || __sub_expired(monc))
+ delay = 10 * HZ;
+ else
+ delay = 20 * HZ;
+ dout("__schedule_delayed after %u\n", delay);
+ schedule_delayed_work(&monc->delayed_work, delay);
+}
+
+/*
+ * Send subscribe request for mdsmap and/or osdmap.
+ */
+static void __send_subscribe(struct ceph_mon_client *monc)
+{
+ dout("__send_subscribe sub_sent=%u exp=%u want_osd=%d\n",
+ (unsigned)monc->sub_sent, __sub_expired(monc),
+ monc->want_next_osdmap);
+ if ((__sub_expired(monc) && !monc->sub_sent) ||
+ monc->want_next_osdmap == 1) {
+ struct ceph_msg *msg;
+ struct ceph_mon_subscribe_item *i;
+ void *p, *end;
+
+ msg = ceph_msg_new(CEPH_MSG_MON_SUBSCRIBE, 96, 0, 0, NULL);
+ if (!msg)
+ return;
+
+ p = msg->front.iov_base;
+ end = p + msg->front.iov_len;
+
+ dout("__send_subscribe to 'mdsmap' %u+\n",
+ (unsigned)monc->have_mdsmap);
+ if (monc->want_next_osdmap) {
+ dout("__send_subscribe to 'osdmap' %u\n",
+ (unsigned)monc->have_osdmap);
+ ceph_encode_32(&p, 3);
+ ceph_encode_string(&p, end, "osdmap", 6);
+ i = p;
+ i->have = cpu_to_le64(monc->have_osdmap);
+ i->onetime = 1;
+ p += sizeof(*i);
+ monc->want_next_osdmap = 2; /* requested */
+ } else {
+ ceph_encode_32(&p, 2);
+ }
+ ceph_encode_string(&p, end, "mdsmap", 6);
+ i = p;
+ i->have = cpu_to_le64(monc->have_mdsmap);
+ i->onetime = 0;
+ p += sizeof(*i);
+ ceph_encode_string(&p, end, "monmap", 6);
+ i = p;
+ i->have = 0;
+ i->onetime = 0;
+ p += sizeof(*i);
+
+ msg->front.iov_len = p - msg->front.iov_base;
+ msg->hdr.front_len = cpu_to_le32(msg->front.iov_len);
+ ceph_con_send(monc->con, msg);
+
+ monc->sub_sent = jiffies | 1; /* never 0 */
+ }
+}
+
+static void handle_subscribe_ack(struct ceph_mon_client *monc,
+ struct ceph_msg *msg)
+{
+ unsigned seconds;
+ struct ceph_mon_subscribe_ack *h = msg->front.iov_base;
+
+ if (msg->front.iov_len < sizeof(*h))
+ goto bad;
+ seconds = le32_to_cpu(h->duration);
+
+ mutex_lock(&monc->mutex);
+ if (monc->hunting) {
+ pr_info("mon%d %s session established\n",
+ monc->cur_mon, pr_addr(&monc->con->peer_addr.in_addr));
+ monc->hunting = false;
+ }
+ dout("handle_subscribe_ack after %d seconds\n", seconds);
+ monc->sub_renew_after = monc->sub_sent + (seconds >> 1)*HZ - 1;
+ monc->sub_sent = 0;
+ mutex_unlock(&monc->mutex);
+ return;
+bad:
+ pr_err("got corrupt subscribe-ack msg\n");
+ ceph_msg_dump(msg);
+}
+
+/*
+ * Keep track of which maps we have
+ */
+int ceph_monc_got_mdsmap(struct ceph_mon_client *monc, u32 got)
+{
+ mutex_lock(&monc->mutex);
+ monc->have_mdsmap = got;
+ mutex_unlock(&monc->mutex);
+ return 0;
+}
+
+int ceph_monc_got_osdmap(struct ceph_mon_client *monc, u32 got)
+{
+ mutex_lock(&monc->mutex);
+ monc->have_osdmap = got;
+ monc->want_next_osdmap = 0;
+ mutex_unlock(&monc->mutex);
+ return 0;
+}
+
+/*
+ * Register interest in the next osdmap
+ */
+void ceph_monc_request_next_osdmap(struct ceph_mon_client *monc)
+{
+ dout("request_next_osdmap have %u\n", monc->have_osdmap);
+ mutex_lock(&monc->mutex);
+ if (!monc->want_next_osdmap)
+ monc->want_next_osdmap = 1;
+ if (monc->want_next_osdmap < 2)
+ __send_subscribe(monc);
+ mutex_unlock(&monc->mutex);
+}
+
+/*
+ *
+ */
+int ceph_monc_open_session(struct ceph_mon_client *monc)
+{
+ if (!monc->con) {
+ monc->con = kmalloc(sizeof(*monc->con), GFP_KERNEL);
+ if (!monc->con)
+ return -ENOMEM;
+ ceph_con_init(monc->client->msgr, monc->con);
+ monc->con->private = monc;
+ monc->con->ops = &mon_con_ops;
+ }
+
+ mutex_lock(&monc->mutex);
+ __open_session(monc);
+ __schedule_delayed(monc);
+ mutex_unlock(&monc->mutex);
+ return 0;
+}
+
+/*
+ * The monitor responds with mount ack indicate mount success. The
+ * included client ticket allows the client to talk to MDSs and OSDs.
+ */
+static void ceph_monc_handle_map(struct ceph_mon_client *monc,
+ struct ceph_msg *msg)
+{
+ struct ceph_client *client = monc->client;
+ struct ceph_monmap *monmap = NULL, *old = monc->monmap;
+ void *p, *end;
+
+ mutex_lock(&monc->mutex);
+
+ dout("handle_monmap\n");
+ p = msg->front.iov_base;
+ end = p + msg->front.iov_len;
+
+ monmap = ceph_monmap_decode(p, end);
+ if (IS_ERR(monmap)) {
+ pr_err("problem decoding monmap, %d\n",
+ (int)PTR_ERR(monmap));
+ goto out;
+ }
+
+ if (ceph_check_fsid(monc->client, &monmap->fsid) < 0) {
+ kfree(monmap);
+ goto out;
+ }
+
+ client->monc.monmap = monmap;
+ kfree(old);
+
+out:
+ mutex_unlock(&monc->mutex);
+ wake_up(&client->auth_wq);
+}
+
+/*
+ * statfs
+ */
+static struct ceph_mon_statfs_request *__lookup_statfs(
+ struct ceph_mon_client *monc, u64 tid)
+{
+ struct ceph_mon_statfs_request *req;
+ struct rb_node *n = monc->statfs_request_tree.rb_node;
+
+ while (n) {
+ req = rb_entry(n, struct ceph_mon_statfs_request, node);
+ if (tid < req->tid)
+ n = n->rb_left;
+ else if (tid > req->tid)
+ n = n->rb_right;
+ else
+ return req;
+ }
+ return NULL;
+}
+
+static void __insert_statfs(struct ceph_mon_client *monc,
+ struct ceph_mon_statfs_request *new)
+{
+ struct rb_node **p = &monc->statfs_request_tree.rb_node;
+ struct rb_node *parent = NULL;
+ struct ceph_mon_statfs_request *req = NULL;
+
+ while (*p) {
+ parent = *p;
+ req = rb_entry(parent, struct ceph_mon_statfs_request, node);
+ if (new->tid < req->tid)
+ p = &(*p)->rb_left;
+ else if (new->tid > req->tid)
+ p = &(*p)->rb_right;
+ else
+ BUG();
+ }
+
+ rb_link_node(&new->node, parent, p);
+ rb_insert_color(&new->node, &monc->statfs_request_tree);
+}
+
+static void handle_statfs_reply(struct ceph_mon_client *monc,
+ struct ceph_msg *msg)
+{
+ struct ceph_mon_statfs_request *req;
+ struct ceph_mon_statfs_reply *reply = msg->front.iov_base;
+ u64 tid;
+
+ if (msg->front.iov_len != sizeof(*reply))
+ goto bad;
+ tid = le64_to_cpu(msg->hdr.tid);
+ dout("handle_statfs_reply %p tid %llu\n", msg, tid);
+
+ mutex_lock(&monc->mutex);
+ req = __lookup_statfs(monc, tid);
+ if (req) {
+ *req->buf = reply->st;
+ req->result = 0;
+ }
+ mutex_unlock(&monc->mutex);
+ if (req)
+ complete(&req->completion);
+ return;
+
+bad:
+ pr_err("corrupt statfs reply, no tid\n");
+ ceph_msg_dump(msg);
+}
+
+/*
+ * (re)send a statfs request
+ */
+static int send_statfs(struct ceph_mon_client *monc,
+ struct ceph_mon_statfs_request *req)
+{
+ struct ceph_msg *msg;
+ struct ceph_mon_statfs *h;
+
+ dout("send_statfs tid %llu\n", req->tid);
+ msg = ceph_msg_new(CEPH_MSG_STATFS, sizeof(*h), 0, 0, NULL);
+ if (IS_ERR(msg))
+ return PTR_ERR(msg);
+ req->request = msg;
+ msg->hdr.tid = cpu_to_le64(req->tid);
+ h = msg->front.iov_base;
+ h->monhdr.have_version = 0;
+ h->monhdr.session_mon = cpu_to_le16(-1);
+ h->monhdr.session_mon_tid = 0;
+ h->fsid = monc->monmap->fsid;
+ ceph_con_send(monc->con, msg);
+ return 0;
+}
+
+/*
+ * Do a synchronous statfs().
+ */
+int ceph_monc_do_statfs(struct ceph_mon_client *monc, struct ceph_statfs *buf)
+{
+ struct ceph_mon_statfs_request req;
+ int err;
+
+ req.buf = buf;
+ init_completion(&req.completion);
+
+ /* allocate memory for reply */
+ err = ceph_msgpool_resv(&monc->msgpool_statfs_reply, 1);
+ if (err)
+ return err;
+
+ /* register request */
+ mutex_lock(&monc->mutex);
+ req.tid = ++monc->last_tid;
+ req.last_attempt = jiffies;
+ req.delay = BASE_DELAY_INTERVAL;
+ __insert_statfs(monc, &req);
+ monc->num_statfs_requests++;
+ mutex_unlock(&monc->mutex);
+
+ /* send request and wait */
+ err = send_statfs(monc, &req);
+ if (!err)
+ err = wait_for_completion_interruptible(&req.completion);
+
+ mutex_lock(&monc->mutex);
+ rb_erase(&req.node, &monc->statfs_request_tree);
+ monc->num_statfs_requests--;
+ ceph_msgpool_resv(&monc->msgpool_statfs_reply, -1);
+ mutex_unlock(&monc->mutex);
+
+ if (!err)
+ err = req.result;
+ return err;
+}
+
+/*
+ * Resend pending statfs requests.
+ */
+static void __resend_statfs(struct ceph_mon_client *monc)
+{
+ struct ceph_mon_statfs_request *req;
+ struct rb_node *p;
+
+ for (p = rb_first(&monc->statfs_request_tree); p; p = rb_next(p)) {
+ req = rb_entry(p, struct ceph_mon_statfs_request, node);
+ send_statfs(monc, req);
+ }
+}
+
+/*
+ * Delayed work. If we haven't mounted yet, retry. Otherwise,
+ * renew/retry subscription as needed (in case it is timing out, or we
+ * got an ENOMEM). And keep the monitor connection alive.
+ */
+static void delayed_work(struct work_struct *work)
+{
+ struct ceph_mon_client *monc =
+ container_of(work, struct ceph_mon_client, delayed_work.work);
+
+ dout("monc delayed_work\n");
+ mutex_lock(&monc->mutex);
+ if (monc->hunting) {
+ __close_session(monc);
+ __open_session(monc); /* continue hunting */
+ } else {
+ ceph_con_keepalive(monc->con);
+
+ __validate_auth(monc);
+
+ if (monc->auth->ops->is_authenticated(monc->auth))
+ __send_subscribe(monc);
+ }
+ __schedule_delayed(monc);
+ mutex_unlock(&monc->mutex);
+}
+
+/*
+ * On startup, we build a temporary monmap populated with the IPs
+ * provided by mount(2).
+ */
+static int build_initial_monmap(struct ceph_mon_client *monc)
+{
+ struct ceph_mount_args *args = monc->client->mount_args;
+ struct ceph_entity_addr *mon_addr = args->mon_addr;
+ int num_mon = args->num_mon;
+ int i;
+
+ /* build initial monmap */
+ monc->monmap = kzalloc(sizeof(*monc->monmap) +
+ num_mon*sizeof(monc->monmap->mon_inst[0]),
+ GFP_KERNEL);
+ if (!monc->monmap)
+ return -ENOMEM;
+ for (i = 0; i < num_mon; i++) {
+ monc->monmap->mon_inst[i].addr = mon_addr[i];
+ monc->monmap->mon_inst[i].addr.nonce = 0;
+ monc->monmap->mon_inst[i].name.type =
+ CEPH_ENTITY_TYPE_MON;
+ monc->monmap->mon_inst[i].name.num = cpu_to_le64(i);
+ }
+ monc->monmap->num_mon = num_mon;
+ monc->have_fsid = false;
+
+ /* release addr memory */
+ kfree(args->mon_addr);
+ args->mon_addr = NULL;
+ args->num_mon = 0;
+ return 0;
+}
+
+int ceph_monc_init(struct ceph_mon_client *monc, struct ceph_client *cl)
+{
+ int err = 0;
+
+ dout("init\n");
+ memset(monc, 0, sizeof(*monc));
+ monc->client = cl;
+ monc->monmap = NULL;
+ mutex_init(&monc->mutex);
+
+ err = build_initial_monmap(monc);
+ if (err)
+ goto out;
+
+ monc->con = NULL;
+
+ /* authentication */
+ monc->auth = ceph_auth_init(cl->mount_args->name,
+ cl->mount_args->secret);
+ if (IS_ERR(monc->auth))
+ return PTR_ERR(monc->auth);
+ monc->auth->want_keys =
+ CEPH_ENTITY_TYPE_AUTH | CEPH_ENTITY_TYPE_MON |
+ CEPH_ENTITY_TYPE_OSD | CEPH_ENTITY_TYPE_MDS;
+
+ /* msg pools */
+ err = ceph_msgpool_init(&monc->msgpool_subscribe_ack,
+ sizeof(struct ceph_mon_subscribe_ack), 1, false);
+ if (err < 0)
+ goto out_monmap;
+ err = ceph_msgpool_init(&monc->msgpool_statfs_reply,
+ sizeof(struct ceph_mon_statfs_reply), 0, false);
+ if (err < 0)
+ goto out_pool1;
+ err = ceph_msgpool_init(&monc->msgpool_auth_reply, 4096, 1, false);
+ if (err < 0)
+ goto out_pool2;
+
+ monc->m_auth = ceph_msg_new(CEPH_MSG_AUTH, 4096, 0, 0, NULL);
+ monc->pending_auth = 0;
+ if (IS_ERR(monc->m_auth)) {
+ err = PTR_ERR(monc->m_auth);
+ monc->m_auth = NULL;
+ goto out_pool3;
+ }
+
+ monc->cur_mon = -1;
+ monc->hunting = true;
+ monc->sub_renew_after = jiffies;
+ monc->sub_sent = 0;
+
+ INIT_DELAYED_WORK(&monc->delayed_work, delayed_work);
+ monc->statfs_request_tree = RB_ROOT;
+ monc->num_statfs_requests = 0;
+ monc->last_tid = 0;
+
+ monc->have_mdsmap = 0;
+ monc->have_osdmap = 0;
+ monc->want_next_osdmap = 1;
+ return 0;
+
+out_pool3:
+ ceph_msgpool_destroy(&monc->msgpool_auth_reply);
+out_pool2:
+ ceph_msgpool_destroy(&monc->msgpool_subscribe_ack);
+out_pool1:
+ ceph_msgpool_destroy(&monc->msgpool_statfs_reply);
+out_monmap:
+ kfree(monc->monmap);
+out:
+ return err;
+}
+
+void ceph_monc_stop(struct ceph_mon_client *monc)
+{
+ dout("stop\n");
+ cancel_delayed_work_sync(&monc->delayed_work);
+
+ mutex_lock(&monc->mutex);
+ __close_session(monc);
+ if (monc->con) {
+ monc->con->private = NULL;
+ monc->con->ops->put(monc->con);
+ monc->con = NULL;
+ }
+ mutex_unlock(&monc->mutex);
+
+ ceph_auth_destroy(monc->auth);
+
+ ceph_msg_put(monc->m_auth);
+ ceph_msgpool_destroy(&monc->msgpool_subscribe_ack);
+ ceph_msgpool_destroy(&monc->msgpool_statfs_reply);
+ ceph_msgpool_destroy(&monc->msgpool_auth_reply);
+
+ kfree(monc->monmap);
+}
+
+static void handle_auth_reply(struct ceph_mon_client *monc,
+ struct ceph_msg *msg)
+{
+ int ret;
+
+ mutex_lock(&monc->mutex);
+ monc->pending_auth = 0;
+ ret = ceph_handle_auth_reply(monc->auth, msg->front.iov_base,
+ msg->front.iov_len,
+ monc->m_auth->front.iov_base,
+ monc->m_auth->front_max);
+ if (ret < 0) {
+ monc->client->auth_err = ret;
+ wake_up(&monc->client->auth_wq);
+ } else if (ret > 0) {
+ __send_prepared_auth_request(monc, ret);
+ } else if (monc->auth->ops->is_authenticated(monc->auth)) {
+ dout("authenticated, starting session\n");
+
+ monc->client->msgr->inst.name.type = CEPH_ENTITY_TYPE_CLIENT;
+ monc->client->msgr->inst.name.num = monc->auth->global_id;
+
+ __send_subscribe(monc);
+ __resend_statfs(monc);
+ }
+ mutex_unlock(&monc->mutex);
+}
+
+static int __validate_auth(struct ceph_mon_client *monc)
+{
+ int ret;
+
+ if (monc->pending_auth)
+ return 0;
+
+ ret = ceph_build_auth(monc->auth, monc->m_auth->front.iov_base,
+ monc->m_auth->front_max);
+ if (ret <= 0)
+ return ret; /* either an error, or no need to authenticate */
+ __send_prepared_auth_request(monc, ret);
+ return 0;
+}
+
+int ceph_monc_validate_auth(struct ceph_mon_client *monc)
+{
+ int ret;
+
+ mutex_lock(&monc->mutex);
+ ret = __validate_auth(monc);
+ mutex_unlock(&monc->mutex);
+ return ret;
+}
+
+/*
+ * handle incoming message
+ */
+static void dispatch(struct ceph_connection *con, struct ceph_msg *msg)
+{
+ struct ceph_mon_client *monc = con->private;
+ int type = le16_to_cpu(msg->hdr.type);
+
+ if (!monc)
+ return;
+
+ switch (type) {
+ case CEPH_MSG_AUTH_REPLY:
+ handle_auth_reply(monc, msg);
+ break;
+
+ case CEPH_MSG_MON_SUBSCRIBE_ACK:
+ handle_subscribe_ack(monc, msg);
+ break;
+
+ case CEPH_MSG_STATFS_REPLY:
+ handle_statfs_reply(monc, msg);
+ break;
+
+ case CEPH_MSG_MON_MAP:
+ ceph_monc_handle_map(monc, msg);
+ break;
+
+ case CEPH_MSG_MDS_MAP:
+ ceph_mdsc_handle_map(&monc->client->mdsc, msg);
+ break;
+
+ case CEPH_MSG_OSD_MAP:
+ ceph_osdc_handle_map(&monc->client->osdc, msg);
+ break;
+
+ default:
+ pr_err("received unknown message type %d %s\n", type,
+ ceph_msg_type_name(type));
+ }
+ ceph_msg_put(msg);
+}
+
+/*
+ * Allocate memory for incoming message
+ */
+static struct ceph_msg *mon_alloc_msg(struct ceph_connection *con,
+ struct ceph_msg_header *hdr,
+ int *skip)
+{
+ struct ceph_mon_client *monc = con->private;
+ int type = le16_to_cpu(hdr->type);
+ int front_len = le32_to_cpu(hdr->front_len);
+ struct ceph_msg *m = NULL;
+
+ *skip = 0;
+
+ switch (type) {
+ case CEPH_MSG_MON_SUBSCRIBE_ACK:
+ m = ceph_msgpool_get(&monc->msgpool_subscribe_ack, front_len);
+ break;
+ case CEPH_MSG_STATFS_REPLY:
+ m = ceph_msgpool_get(&monc->msgpool_statfs_reply, front_len);
+ break;
+ case CEPH_MSG_AUTH_REPLY:
+ m = ceph_msgpool_get(&monc->msgpool_auth_reply, front_len);
+ break;
+ case CEPH_MSG_MON_MAP:
+ case CEPH_MSG_MDS_MAP:
+ case CEPH_MSG_OSD_MAP:
+ m = ceph_msg_new(type, front_len, 0, 0, NULL);
+ break;
+ }
+
+ if (!m) {
+ pr_info("alloc_msg unknown type %d\n", type);
+ *skip = 1;
+ }
+ return m;
+}
+
+/*
+ * If the monitor connection resets, pick a new monitor and resubmit
+ * any pending requests.
+ */
+static void mon_fault(struct ceph_connection *con)
+{
+ struct ceph_mon_client *monc = con->private;
+
+ if (!monc)
+ return;
+
+ dout("mon_fault\n");
+ mutex_lock(&monc->mutex);
+ if (!con->private)
+ goto out;
+
+ if (monc->con && !monc->hunting)
+ pr_info("mon%d %s session lost, "
+ "hunting for new mon\n", monc->cur_mon,
+ pr_addr(&monc->con->peer_addr.in_addr));
+
+ __close_session(monc);
+ if (!monc->hunting) {
+ /* start hunting */
+ monc->hunting = true;
+ __open_session(monc);
+ } else {
+ /* already hunting, let's wait a bit */
+ __schedule_delayed(monc);
+ }
+out:
+ mutex_unlock(&monc->mutex);
+}
+
+const static struct ceph_connection_operations mon_con_ops = {
+ .get = ceph_con_get,
+ .put = ceph_con_put,
+ .dispatch = dispatch,
+ .fault = mon_fault,
+ .alloc_msg = mon_alloc_msg,
+};
--- /dev/null
+#ifndef _FS_CEPH_MON_CLIENT_H
+#define _FS_CEPH_MON_CLIENT_H
+
+#include <linux/completion.h>
+#include <linux/rbtree.h>
+
+#include "messenger.h"
+#include "msgpool.h"
+
+struct ceph_client;
+struct ceph_mount_args;
+struct ceph_auth_client;
+
+/*
+ * The monitor map enumerates the set of all monitors.
+ */
+struct ceph_monmap {
+ struct ceph_fsid fsid;
+ u32 epoch;
+ u32 num_mon;
+ struct ceph_entity_inst mon_inst[0];
+};
+
+struct ceph_mon_client;
+struct ceph_mon_statfs_request;
+
+
+/*
+ * Generic mechanism for resending monitor requests.
+ */
+typedef void (*ceph_monc_request_func_t)(struct ceph_mon_client *monc,
+ int newmon);
+
+/* a pending monitor request */
+struct ceph_mon_request {
+ struct ceph_mon_client *monc;
+ struct delayed_work delayed_work;
+ unsigned long delay;
+ ceph_monc_request_func_t do_request;
+};
+
+/*
+ * statfs() is done a bit differently because we need to get data back
+ * to the caller
+ */
+struct ceph_mon_statfs_request {
+ u64 tid;
+ struct rb_node node;
+ int result;
+ struct ceph_statfs *buf;
+ struct completion completion;
+ unsigned long last_attempt, delay; /* jiffies */
+ struct ceph_msg *request; /* original request */
+};
+
+struct ceph_mon_client {
+ struct ceph_client *client;
+ struct ceph_monmap *monmap;
+
+ struct mutex mutex;
+ struct delayed_work delayed_work;
+
+ struct ceph_auth_client *auth;
+ struct ceph_msg *m_auth;
+ int pending_auth;
+
+ bool hunting;
+ int cur_mon; /* last monitor i contacted */
+ unsigned long sub_sent, sub_renew_after;
+ struct ceph_connection *con;
+ bool have_fsid;
+
+ /* msg pools */
+ struct ceph_msgpool msgpool_subscribe_ack;
+ struct ceph_msgpool msgpool_statfs_reply;
+ struct ceph_msgpool msgpool_auth_reply;
+
+ /* pending statfs requests */
+ struct rb_root statfs_request_tree;
+ int num_statfs_requests;
+ u64 last_tid;
+
+ /* mds/osd map */
+ int want_next_osdmap; /* 1 = want, 2 = want+asked */
+ u32 have_osdmap, have_mdsmap;
+
+#ifdef CONFIG_DEBUG_FS
+ struct dentry *debugfs_file;
+#endif
+};
+
+extern struct ceph_monmap *ceph_monmap_decode(void *p, void *end);
+extern int ceph_monmap_contains(struct ceph_monmap *m,
+ struct ceph_entity_addr *addr);
+
+extern int ceph_monc_init(struct ceph_mon_client *monc, struct ceph_client *cl);
+extern void ceph_monc_stop(struct ceph_mon_client *monc);
+
+/*
+ * The model here is to indicate that we need a new map of at least
+ * epoch @want, and also call in when we receive a map. We will
+ * periodically rerequest the map from the monitor cluster until we
+ * get what we want.
+ */
+extern int ceph_monc_got_mdsmap(struct ceph_mon_client *monc, u32 have);
+extern int ceph_monc_got_osdmap(struct ceph_mon_client *monc, u32 have);
+
+extern void ceph_monc_request_next_osdmap(struct ceph_mon_client *monc);
+
+extern int ceph_monc_do_statfs(struct ceph_mon_client *monc,
+ struct ceph_statfs *buf);
+
+extern int ceph_monc_open_session(struct ceph_mon_client *monc);
+
+extern int ceph_monc_validate_auth(struct ceph_mon_client *monc);
+
+
+
+#endif
--- /dev/null
+#include "ceph_debug.h"
+
+#include <linux/err.h>
+#include <linux/sched.h>
+#include <linux/types.h>
+#include <linux/vmalloc.h>
+
+#include "msgpool.h"
+
+/*
+ * We use msg pools to preallocate memory for messages we expect to
+ * receive over the wire, to avoid getting ourselves into OOM
+ * conditions at unexpected times. We take use a few different
+ * strategies:
+ *
+ * - for request/response type interactions, we preallocate the
+ * memory needed for the response when we generate the request.
+ *
+ * - for messages we can receive at any time from the MDS, we preallocate
+ * a pool of messages we can re-use.
+ *
+ * - for writeback, we preallocate some number of messages to use for
+ * requests and their replies, so that we always make forward
+ * progress.
+ *
+ * The msgpool behaves like a mempool_t, but keeps preallocated
+ * ceph_msgs strung together on a list_head instead of using a pointer
+ * vector. This avoids vector reallocation when we adjust the number
+ * of preallocated items (which happens frequently).
+ */
+
+
+/*
+ * Allocate or release as necessary to meet our target pool size.
+ */
+static int __fill_msgpool(struct ceph_msgpool *pool)
+{
+ struct ceph_msg *msg;
+
+ while (pool->num < pool->min) {
+ dout("fill_msgpool %p %d/%d allocating\n", pool, pool->num,
+ pool->min);
+ spin_unlock(&pool->lock);
+ msg = ceph_msg_new(0, pool->front_len, 0, 0, NULL);
+ spin_lock(&pool->lock);
+ if (IS_ERR(msg))
+ return PTR_ERR(msg);
+ msg->pool = pool;
+ list_add(&msg->list_head, &pool->msgs);
+ pool->num++;
+ }
+ while (pool->num > pool->min) {
+ msg = list_first_entry(&pool->msgs, struct ceph_msg, list_head);
+ dout("fill_msgpool %p %d/%d releasing %p\n", pool, pool->num,
+ pool->min, msg);
+ list_del_init(&msg->list_head);
+ pool->num--;
+ ceph_msg_kfree(msg);
+ }
+ return 0;
+}
+
+int ceph_msgpool_init(struct ceph_msgpool *pool,
+ int front_len, int min, bool blocking)
+{
+ int ret;
+
+ dout("msgpool_init %p front_len %d min %d\n", pool, front_len, min);
+ spin_lock_init(&pool->lock);
+ pool->front_len = front_len;
+ INIT_LIST_HEAD(&pool->msgs);
+ pool->num = 0;
+ pool->min = min;
+ pool->blocking = blocking;
+ init_waitqueue_head(&pool->wait);
+
+ spin_lock(&pool->lock);
+ ret = __fill_msgpool(pool);
+ spin_unlock(&pool->lock);
+ return ret;
+}
+
+void ceph_msgpool_destroy(struct ceph_msgpool *pool)
+{
+ dout("msgpool_destroy %p\n", pool);
+ spin_lock(&pool->lock);
+ pool->min = 0;
+ __fill_msgpool(pool);
+ spin_unlock(&pool->lock);
+}
+
+int ceph_msgpool_resv(struct ceph_msgpool *pool, int delta)
+{
+ int ret;
+
+ spin_lock(&pool->lock);
+ dout("msgpool_resv %p delta %d\n", pool, delta);
+ pool->min += delta;
+ ret = __fill_msgpool(pool);
+ spin_unlock(&pool->lock);
+ return ret;
+}
+
+struct ceph_msg *ceph_msgpool_get(struct ceph_msgpool *pool, int front_len)
+{
+ wait_queue_t wait;
+ struct ceph_msg *msg;
+
+ if (front_len && front_len > pool->front_len) {
+ pr_err("msgpool_get pool %p need front %d, pool size is %d\n",
+ pool, front_len, pool->front_len);
+ WARN_ON(1);
+
+ /* try to alloc a fresh message */
+ msg = ceph_msg_new(0, front_len, 0, 0, NULL);
+ if (!IS_ERR(msg))
+ return msg;
+ }
+
+ if (!front_len)
+ front_len = pool->front_len;
+
+ if (pool->blocking) {
+ /* mempool_t behavior; first try to alloc */
+ msg = ceph_msg_new(0, front_len, 0, 0, NULL);
+ if (!IS_ERR(msg))
+ return msg;
+ }
+
+ while (1) {
+ spin_lock(&pool->lock);
+ if (likely(pool->num)) {
+ msg = list_entry(pool->msgs.next, struct ceph_msg,
+ list_head);
+ list_del_init(&msg->list_head);
+ pool->num--;
+ dout("msgpool_get %p got %p, now %d/%d\n", pool, msg,
+ pool->num, pool->min);
+ spin_unlock(&pool->lock);
+ return msg;
+ }
+ pr_err("msgpool_get %p now %d/%d, %s\n", pool, pool->num,
+ pool->min, pool->blocking ? "waiting" : "may fail");
+ spin_unlock(&pool->lock);
+
+ if (!pool->blocking) {
+ WARN_ON(1);
+
+ /* maybe we can allocate it now? */
+ msg = ceph_msg_new(0, front_len, 0, 0, NULL);
+ if (!IS_ERR(msg))
+ return msg;
+
+ pr_err("msgpool_get %p empty + alloc failed\n", pool);
+ return ERR_PTR(-ENOMEM);
+ }
+
+ init_wait(&wait);
+ prepare_to_wait(&pool->wait, &wait, TASK_UNINTERRUPTIBLE);
+ schedule();
+ finish_wait(&pool->wait, &wait);
+ }
+}
+
+void ceph_msgpool_put(struct ceph_msgpool *pool, struct ceph_msg *msg)
+{
+ spin_lock(&pool->lock);
+ if (pool->num < pool->min) {
+ /* reset msg front_len; user may have changed it */
+ msg->front.iov_len = pool->front_len;
+ msg->hdr.front_len = cpu_to_le32(pool->front_len);
+
+ kref_set(&msg->kref, 1); /* retake a single ref */
+ list_add(&msg->list_head, &pool->msgs);
+ pool->num++;
+ dout("msgpool_put %p reclaim %p, now %d/%d\n", pool, msg,
+ pool->num, pool->min);
+ spin_unlock(&pool->lock);
+ wake_up(&pool->wait);
+ } else {
+ dout("msgpool_put %p drop %p, at %d/%d\n", pool, msg,
+ pool->num, pool->min);
+ spin_unlock(&pool->lock);
+ ceph_msg_kfree(msg);
+ }
+}
--- /dev/null
+#ifndef _FS_CEPH_MSGPOOL
+#define _FS_CEPH_MSGPOOL
+
+#include "messenger.h"
+
+/*
+ * we use memory pools for preallocating messages we may receive, to
+ * avoid unexpected OOM conditions.
+ */
+struct ceph_msgpool {
+ spinlock_t lock;
+ int front_len; /* preallocated payload size */
+ struct list_head msgs; /* msgs in the pool; each has 1 ref */
+ int num, min; /* cur, min # msgs in the pool */
+ bool blocking;
+ wait_queue_head_t wait;
+};
+
+extern int ceph_msgpool_init(struct ceph_msgpool *pool,
+ int front_len, int size, bool blocking);
+extern void ceph_msgpool_destroy(struct ceph_msgpool *pool);
+extern int ceph_msgpool_resv(struct ceph_msgpool *, int delta);
+extern struct ceph_msg *ceph_msgpool_get(struct ceph_msgpool *,
+ int front_len);
+extern void ceph_msgpool_put(struct ceph_msgpool *, struct ceph_msg *);
+
+#endif
--- /dev/null
+#ifndef __MSGR_H
+#define __MSGR_H
+
+/*
+ * Data types for message passing layer used by Ceph.
+ */
+
+#define CEPH_MON_PORT 6789 /* default monitor port */
+
+/*
+ * client-side processes will try to bind to ports in this
+ * range, simply for the benefit of tools like nmap or wireshark
+ * that would like to identify the protocol.
+ */
+#define CEPH_PORT_FIRST 6789
+#define CEPH_PORT_START 6800 /* non-monitors start here */
+#define CEPH_PORT_LAST 6900
+
+/*
+ * tcp connection banner. include a protocol version. and adjust
+ * whenever the wire protocol changes. try to keep this string length
+ * constant.
+ */
+#define CEPH_BANNER "ceph v027"
+#define CEPH_BANNER_MAX_LEN 30
+
+
+/*
+ * Rollover-safe type and comparator for 32-bit sequence numbers.
+ * Comparator returns -1, 0, or 1.
+ */
+typedef __u32 ceph_seq_t;
+
+static inline __s32 ceph_seq_cmp(__u32 a, __u32 b)
+{
+ return (__s32)a - (__s32)b;
+}
+
+
+/*
+ * entity_name -- logical name for a process participating in the
+ * network, e.g. 'mds0' or 'osd3'.
+ */
+struct ceph_entity_name {
+ __u8 type; /* CEPH_ENTITY_TYPE_* */
+ __le64 num;
+} __attribute__ ((packed));
+
+#define CEPH_ENTITY_TYPE_MON 0x01
+#define CEPH_ENTITY_TYPE_MDS 0x02
+#define CEPH_ENTITY_TYPE_OSD 0x04
+#define CEPH_ENTITY_TYPE_CLIENT 0x08
+#define CEPH_ENTITY_TYPE_ADMIN 0x10
+#define CEPH_ENTITY_TYPE_AUTH 0x20
+
+#define CEPH_ENTITY_TYPE_ANY 0xFF
+
+extern const char *ceph_entity_type_name(int type);
+
+/*
+ * entity_addr -- network address
+ */
+struct ceph_entity_addr {
+ __le32 type;
+ __le32 nonce; /* unique id for process (e.g. pid) */
+ struct sockaddr_storage in_addr;
+} __attribute__ ((packed));
+
+struct ceph_entity_inst {
+ struct ceph_entity_name name;
+ struct ceph_entity_addr addr;
+} __attribute__ ((packed));
+
+
+/* used by message exchange protocol */
+#define CEPH_MSGR_TAG_READY 1 /* server->client: ready for messages */
+#define CEPH_MSGR_TAG_RESETSESSION 2 /* server->client: reset, try again */
+#define CEPH_MSGR_TAG_WAIT 3 /* server->client: wait for racing
+ incoming connection */
+#define CEPH_MSGR_TAG_RETRY_SESSION 4 /* server->client + cseq: try again
+ with higher cseq */
+#define CEPH_MSGR_TAG_RETRY_GLOBAL 5 /* server->client + gseq: try again
+ with higher gseq */
+#define CEPH_MSGR_TAG_CLOSE 6 /* closing pipe */
+#define CEPH_MSGR_TAG_MSG 7 /* message */
+#define CEPH_MSGR_TAG_ACK 8 /* message ack */
+#define CEPH_MSGR_TAG_KEEPALIVE 9 /* just a keepalive byte! */
+#define CEPH_MSGR_TAG_BADPROTOVER 10 /* bad protocol version */
+#define CEPH_MSGR_TAG_BADAUTHORIZER 11 /* bad authorizer */
+#define CEPH_MSGR_TAG_FEATURES 12 /* insufficient features */
+
+
+/*
+ * connection negotiation
+ */
+struct ceph_msg_connect {
+ __le64 features; /* supported feature bits */
+ __le32 host_type; /* CEPH_ENTITY_TYPE_* */
+ __le32 global_seq; /* count connections initiated by this host */
+ __le32 connect_seq; /* count connections initiated in this session */
+ __le32 protocol_version;
+ __le32 authorizer_protocol;
+ __le32 authorizer_len;
+ __u8 flags; /* CEPH_MSG_CONNECT_* */
+} __attribute__ ((packed));
+
+struct ceph_msg_connect_reply {
+ __u8 tag;
+ __le64 features; /* feature bits for this session */
+ __le32 global_seq;
+ __le32 connect_seq;
+ __le32 protocol_version;
+ __le32 authorizer_len;
+ __u8 flags;
+} __attribute__ ((packed));
+
+#define CEPH_MSG_CONNECT_LOSSY 1 /* messages i send may be safely dropped */
+
+
+/*
+ * message header
+ */
+struct ceph_msg_header {
+ __le64 seq; /* message seq# for this session */
+ __le64 tid; /* transaction id */
+ __le16 type; /* message type */
+ __le16 priority; /* priority. higher value == higher priority */
+ __le16 version; /* version of message encoding */
+
+ __le32 front_len; /* bytes in main payload */
+ __le32 middle_len;/* bytes in middle payload */
+ __le32 data_len; /* bytes of data payload */
+ __le16 data_off; /* sender: include full offset;
+ receiver: mask against ~PAGE_MASK */
+
+ struct ceph_entity_inst src, orig_src;
+ __le32 reserved;
+ __le32 crc; /* header crc32c */
+} __attribute__ ((packed));
+
+#define CEPH_MSG_PRIO_LOW 64
+#define CEPH_MSG_PRIO_DEFAULT 127
+#define CEPH_MSG_PRIO_HIGH 196
+#define CEPH_MSG_PRIO_HIGHEST 255
+
+/*
+ * follows data payload
+ */
+struct ceph_msg_footer {
+ __le32 front_crc, middle_crc, data_crc;
+ __u8 flags;
+} __attribute__ ((packed));
+
+#define CEPH_MSG_FOOTER_COMPLETE (1<<0) /* msg wasn't aborted */
+#define CEPH_MSG_FOOTER_NOCRC (1<<1) /* no data crc */
+
+
+#endif
--- /dev/null
+#include "ceph_debug.h"
+
+#include <linux/err.h>
+#include <linux/highmem.h>
+#include <linux/mm.h>
+#include <linux/pagemap.h>
+#include <linux/slab.h>
+#include <linux/uaccess.h>
+
+#include "super.h"
+#include "osd_client.h"
+#include "messenger.h"
+#include "decode.h"
+#include "auth.h"
+
+#define OSD_OP_FRONT_LEN 4096
+#define OSD_OPREPLY_FRONT_LEN 512
+
+const static struct ceph_connection_operations osd_con_ops;
+static int __kick_requests(struct ceph_osd_client *osdc,
+ struct ceph_osd *kickosd);
+
+static void kick_requests(struct ceph_osd_client *osdc, struct ceph_osd *osd);
+
+/*
+ * Implement client access to distributed object storage cluster.
+ *
+ * All data objects are stored within a cluster/cloud of OSDs, or
+ * "object storage devices." (Note that Ceph OSDs have _nothing_ to
+ * do with the T10 OSD extensions to SCSI.) Ceph OSDs are simply
+ * remote daemons serving up and coordinating consistent and safe
+ * access to storage.
+ *
+ * Cluster membership and the mapping of data objects onto storage devices
+ * are described by the osd map.
+ *
+ * We keep track of pending OSD requests (read, write), resubmit
+ * requests to different OSDs when the cluster topology/data layout
+ * change, or retry the affected requests when the communications
+ * channel with an OSD is reset.
+ */
+
+/*
+ * calculate the mapping of a file extent onto an object, and fill out the
+ * request accordingly. shorten extent as necessary if it crosses an
+ * object boundary.
+ *
+ * fill osd op in request message.
+ */
+static void calc_layout(struct ceph_osd_client *osdc,
+ struct ceph_vino vino, struct ceph_file_layout *layout,
+ u64 off, u64 *plen,
+ struct ceph_osd_request *req)
+{
+ struct ceph_osd_request_head *reqhead = req->r_request->front.iov_base;
+ struct ceph_osd_op *op = (void *)(reqhead + 1);
+ u64 orig_len = *plen;
+ u64 objoff, objlen; /* extent in object */
+ u64 bno;
+
+ reqhead->snapid = cpu_to_le64(vino.snap);
+
+ /* object extent? */
+ ceph_calc_file_object_mapping(layout, off, plen, &bno,
+ &objoff, &objlen);
+ if (*plen < orig_len)
+ dout(" skipping last %llu, final file extent %llu~%llu\n",
+ orig_len - *plen, off, *plen);
+
+ sprintf(req->r_oid, "%llx.%08llx", vino.ino, bno);
+ req->r_oid_len = strlen(req->r_oid);
+
+ op->extent.offset = cpu_to_le64(objoff);
+ op->extent.length = cpu_to_le64(objlen);
+ req->r_num_pages = calc_pages_for(off, *plen);
+
+ dout("calc_layout %s (%d) %llu~%llu (%d pages)\n",
+ req->r_oid, req->r_oid_len, objoff, objlen, req->r_num_pages);
+}
+
+/*
+ * requests
+ */
+void ceph_osdc_release_request(struct kref *kref)
+{
+ struct ceph_osd_request *req = container_of(kref,
+ struct ceph_osd_request,
+ r_kref);
+
+ if (req->r_request)
+ ceph_msg_put(req->r_request);
+ if (req->r_reply)
+ ceph_msg_put(req->r_reply);
+ if (req->r_con_filling_msg) {
+ dout("release_request revoking pages %p from con %p\n",
+ req->r_pages, req->r_con_filling_msg);
+ ceph_con_revoke_message(req->r_con_filling_msg,
+ req->r_reply);
+ ceph_con_put(req->r_con_filling_msg);
+ }
+ if (req->r_own_pages)
+ ceph_release_page_vector(req->r_pages,
+ req->r_num_pages);
+ ceph_put_snap_context(req->r_snapc);
+ if (req->r_mempool)
+ mempool_free(req, req->r_osdc->req_mempool);
+ else
+ kfree(req);
+}
+
+/*
+ * build new request AND message, calculate layout, and adjust file
+ * extent as needed.
+ *
+ * if the file was recently truncated, we include information about its
+ * old and new size so that the object can be updated appropriately. (we
+ * avoid synchronously deleting truncated objects because it's slow.)
+ *
+ * if @do_sync, include a 'startsync' command so that the osd will flush
+ * data quickly.
+ */
+struct ceph_osd_request *ceph_osdc_new_request(struct ceph_osd_client *osdc,
+ struct ceph_file_layout *layout,
+ struct ceph_vino vino,
+ u64 off, u64 *plen,
+ int opcode, int flags,
+ struct ceph_snap_context *snapc,
+ int do_sync,
+ u32 truncate_seq,
+ u64 truncate_size,
+ struct timespec *mtime,
+ bool use_mempool, int num_reply)
+{
+ struct ceph_osd_request *req;
+ struct ceph_msg *msg;
+ struct ceph_osd_request_head *head;
+ struct ceph_osd_op *op;
+ void *p;
+ int num_op = 1 + do_sync;
+ size_t msg_size = sizeof(*head) + num_op*sizeof(*op);
+ int i;
+
+ if (use_mempool) {
+ req = mempool_alloc(osdc->req_mempool, GFP_NOFS);
+ memset(req, 0, sizeof(*req));
+ } else {
+ req = kzalloc(sizeof(*req), GFP_NOFS);
+ }
+ if (req == NULL)
+ return ERR_PTR(-ENOMEM);
+
+ req->r_osdc = osdc;
+ req->r_mempool = use_mempool;
+ kref_init(&req->r_kref);
+ init_completion(&req->r_completion);
+ init_completion(&req->r_safe_completion);
+ INIT_LIST_HEAD(&req->r_unsafe_item);
+ req->r_flags = flags;
+
+ WARN_ON((flags & (CEPH_OSD_FLAG_READ|CEPH_OSD_FLAG_WRITE)) == 0);
+
+ /* create reply message */
+ if (use_mempool)
+ msg = ceph_msgpool_get(&osdc->msgpool_op_reply, 0);
+ else
+ msg = ceph_msg_new(CEPH_MSG_OSD_OPREPLY,
+ OSD_OPREPLY_FRONT_LEN, 0, 0, NULL);
+ if (IS_ERR(msg)) {
+ ceph_osdc_put_request(req);
+ return ERR_PTR(PTR_ERR(msg));
+ }
+ req->r_reply = msg;
+
+ /* create request message; allow space for oid */
+ msg_size += 40;
+ if (snapc)
+ msg_size += sizeof(u64) * snapc->num_snaps;
+ if (use_mempool)
+ msg = ceph_msgpool_get(&osdc->msgpool_op, 0);
+ else
+ msg = ceph_msg_new(CEPH_MSG_OSD_OP, msg_size, 0, 0, NULL);
+ if (IS_ERR(msg)) {
+ ceph_osdc_put_request(req);
+ return ERR_PTR(PTR_ERR(msg));
+ }
+ msg->hdr.type = cpu_to_le16(CEPH_MSG_OSD_OP);
+ memset(msg->front.iov_base, 0, msg->front.iov_len);
+ head = msg->front.iov_base;
+ op = (void *)(head + 1);
+ p = (void *)(op + num_op);
+
+ req->r_request = msg;
+ req->r_snapc = ceph_get_snap_context(snapc);
+
+ head->client_inc = cpu_to_le32(1); /* always, for now. */
+ head->flags = cpu_to_le32(flags);
+ if (flags & CEPH_OSD_FLAG_WRITE)
+ ceph_encode_timespec(&head->mtime, mtime);
+ head->num_ops = cpu_to_le16(num_op);
+ op->op = cpu_to_le16(opcode);
+
+ /* calculate max write size */
+ calc_layout(osdc, vino, layout, off, plen, req);
+ req->r_file_layout = *layout; /* keep a copy */
+
+ if (flags & CEPH_OSD_FLAG_WRITE) {
+ req->r_request->hdr.data_off = cpu_to_le16(off);
+ req->r_request->hdr.data_len = cpu_to_le32(*plen);
+ op->payload_len = cpu_to_le32(*plen);
+ }
+ op->extent.truncate_size = cpu_to_le64(truncate_size);
+ op->extent.truncate_seq = cpu_to_le32(truncate_seq);
+
+ /* fill in oid */
+ head->object_len = cpu_to_le32(req->r_oid_len);
+ memcpy(p, req->r_oid, req->r_oid_len);
+ p += req->r_oid_len;
+
+ if (do_sync) {
+ op++;
+ op->op = cpu_to_le16(CEPH_OSD_OP_STARTSYNC);
+ }
+ if (snapc) {
+ head->snap_seq = cpu_to_le64(snapc->seq);
+ head->num_snaps = cpu_to_le32(snapc->num_snaps);
+ for (i = 0; i < snapc->num_snaps; i++) {
+ put_unaligned_le64(snapc->snaps[i], p);
+ p += sizeof(u64);
+ }
+ }
+
+ BUG_ON(p > msg->front.iov_base + msg->front.iov_len);
+ msg_size = p - msg->front.iov_base;
+ msg->front.iov_len = msg_size;
+ msg->hdr.front_len = cpu_to_le32(msg_size);
+ return req;
+}
+
+/*
+ * We keep osd requests in an rbtree, sorted by ->r_tid.
+ */
+static void __insert_request(struct ceph_osd_client *osdc,
+ struct ceph_osd_request *new)
+{
+ struct rb_node **p = &osdc->requests.rb_node;
+ struct rb_node *parent = NULL;
+ struct ceph_osd_request *req = NULL;
+
+ while (*p) {
+ parent = *p;
+ req = rb_entry(parent, struct ceph_osd_request, r_node);
+ if (new->r_tid < req->r_tid)
+ p = &(*p)->rb_left;
+ else if (new->r_tid > req->r_tid)
+ p = &(*p)->rb_right;
+ else
+ BUG();
+ }
+
+ rb_link_node(&new->r_node, parent, p);
+ rb_insert_color(&new->r_node, &osdc->requests);
+}
+
+static struct ceph_osd_request *__lookup_request(struct ceph_osd_client *osdc,
+ u64 tid)
+{
+ struct ceph_osd_request *req;
+ struct rb_node *n = osdc->requests.rb_node;
+
+ while (n) {
+ req = rb_entry(n, struct ceph_osd_request, r_node);
+ if (tid < req->r_tid)
+ n = n->rb_left;
+ else if (tid > req->r_tid)
+ n = n->rb_right;
+ else
+ return req;
+ }
+ return NULL;
+}
+
+static struct ceph_osd_request *
+__lookup_request_ge(struct ceph_osd_client *osdc,
+ u64 tid)
+{
+ struct ceph_osd_request *req;
+ struct rb_node *n = osdc->requests.rb_node;
+
+ while (n) {
+ req = rb_entry(n, struct ceph_osd_request, r_node);
+ if (tid < req->r_tid) {
+ if (!n->rb_left)
+ return req;
+ n = n->rb_left;
+ } else if (tid > req->r_tid) {
+ n = n->rb_right;
+ } else {
+ return req;
+ }
+ }
+ return NULL;
+}
+
+
+/*
+ * If the osd connection drops, we need to resubmit all requests.
+ */
+static void osd_reset(struct ceph_connection *con)
+{
+ struct ceph_osd *osd = con->private;
+ struct ceph_osd_client *osdc;
+
+ if (!osd)
+ return;
+ dout("osd_reset osd%d\n", osd->o_osd);
+ osdc = osd->o_osdc;
+ down_read(&osdc->map_sem);
+ kick_requests(osdc, osd);
+ up_read(&osdc->map_sem);
+}
+
+/*
+ * Track open sessions with osds.
+ */
+static struct ceph_osd *create_osd(struct ceph_osd_client *osdc)
+{
+ struct ceph_osd *osd;
+
+ osd = kzalloc(sizeof(*osd), GFP_NOFS);
+ if (!osd)
+ return NULL;
+
+ atomic_set(&osd->o_ref, 1);
+ osd->o_osdc = osdc;
+ INIT_LIST_HEAD(&osd->o_requests);
+ INIT_LIST_HEAD(&osd->o_osd_lru);
+ osd->o_incarnation = 1;
+
+ ceph_con_init(osdc->client->msgr, &osd->o_con);
+ osd->o_con.private = osd;
+ osd->o_con.ops = &osd_con_ops;
+ osd->o_con.peer_name.type = CEPH_ENTITY_TYPE_OSD;
+
+ INIT_LIST_HEAD(&osd->o_keepalive_item);
+ return osd;
+}
+
+static struct ceph_osd *get_osd(struct ceph_osd *osd)
+{
+ if (atomic_inc_not_zero(&osd->o_ref)) {
+ dout("get_osd %p %d -> %d\n", osd, atomic_read(&osd->o_ref)-1,
+ atomic_read(&osd->o_ref));
+ return osd;
+ } else {
+ dout("get_osd %p FAIL\n", osd);
+ return NULL;
+ }
+}
+
+static void put_osd(struct ceph_osd *osd)
+{
+ dout("put_osd %p %d -> %d\n", osd, atomic_read(&osd->o_ref),
+ atomic_read(&osd->o_ref) - 1);
+ if (atomic_dec_and_test(&osd->o_ref))
+ kfree(osd);
+}
+
+/*
+ * remove an osd from our map
+ */
+static void __remove_osd(struct ceph_osd_client *osdc, struct ceph_osd *osd)
+{
+ dout("__remove_osd %p\n", osd);
+ BUG_ON(!list_empty(&osd->o_requests));
+ rb_erase(&osd->o_node, &osdc->osds);
+ list_del_init(&osd->o_osd_lru);
+ ceph_con_close(&osd->o_con);
+ put_osd(osd);
+}
+
+static void __move_osd_to_lru(struct ceph_osd_client *osdc,
+ struct ceph_osd *osd)
+{
+ dout("__move_osd_to_lru %p\n", osd);
+ BUG_ON(!list_empty(&osd->o_osd_lru));
+ list_add_tail(&osd->o_osd_lru, &osdc->osd_lru);
+ osd->lru_ttl = jiffies + osdc->client->mount_args->osd_idle_ttl * HZ;
+}
+
+static void __remove_osd_from_lru(struct ceph_osd *osd)
+{
+ dout("__remove_osd_from_lru %p\n", osd);
+ if (!list_empty(&osd->o_osd_lru))
+ list_del_init(&osd->o_osd_lru);
+}
+
+static void remove_old_osds(struct ceph_osd_client *osdc, int remove_all)
+{
+ struct ceph_osd *osd, *nosd;
+
+ dout("__remove_old_osds %p\n", osdc);
+ mutex_lock(&osdc->request_mutex);
+ list_for_each_entry_safe(osd, nosd, &osdc->osd_lru, o_osd_lru) {
+ if (!remove_all && time_before(jiffies, osd->lru_ttl))
+ break;
+ __remove_osd(osdc, osd);
+ }
+ mutex_unlock(&osdc->request_mutex);
+}
+
+/*
+ * reset osd connect
+ */
+static int __reset_osd(struct ceph_osd_client *osdc, struct ceph_osd *osd)
+{
+ struct ceph_osd_request *req;
+ int ret = 0;
+
+ dout("__reset_osd %p osd%d\n", osd, osd->o_osd);
+ if (list_empty(&osd->o_requests)) {
+ __remove_osd(osdc, osd);
+ } else if (memcmp(&osdc->osdmap->osd_addr[osd->o_osd],
+ &osd->o_con.peer_addr,
+ sizeof(osd->o_con.peer_addr)) == 0 &&
+ !ceph_con_opened(&osd->o_con)) {
+ dout(" osd addr hasn't changed and connection never opened,"
+ " letting msgr retry");
+ /* touch each r_stamp for handle_timeout()'s benfit */
+ list_for_each_entry(req, &osd->o_requests, r_osd_item)
+ req->r_stamp = jiffies;
+ ret = -EAGAIN;
+ } else {
+ ceph_con_close(&osd->o_con);
+ ceph_con_open(&osd->o_con, &osdc->osdmap->osd_addr[osd->o_osd]);
+ osd->o_incarnation++;
+ }
+ return ret;
+}
+
+static void __insert_osd(struct ceph_osd_client *osdc, struct ceph_osd *new)
+{
+ struct rb_node **p = &osdc->osds.rb_node;
+ struct rb_node *parent = NULL;
+ struct ceph_osd *osd = NULL;
+
+ while (*p) {
+ parent = *p;
+ osd = rb_entry(parent, struct ceph_osd, o_node);
+ if (new->o_osd < osd->o_osd)
+ p = &(*p)->rb_left;
+ else if (new->o_osd > osd->o_osd)
+ p = &(*p)->rb_right;
+ else
+ BUG();
+ }
+
+ rb_link_node(&new->o_node, parent, p);
+ rb_insert_color(&new->o_node, &osdc->osds);
+}
+
+static struct ceph_osd *__lookup_osd(struct ceph_osd_client *osdc, int o)
+{
+ struct ceph_osd *osd;
+ struct rb_node *n = osdc->osds.rb_node;
+
+ while (n) {
+ osd = rb_entry(n, struct ceph_osd, o_node);
+ if (o < osd->o_osd)
+ n = n->rb_left;
+ else if (o > osd->o_osd)
+ n = n->rb_right;
+ else
+ return osd;
+ }
+ return NULL;
+}
+
+static void __schedule_osd_timeout(struct ceph_osd_client *osdc)
+{
+ schedule_delayed_work(&osdc->timeout_work,
+ osdc->client->mount_args->osd_keepalive_timeout * HZ);
+}
+
+static void __cancel_osd_timeout(struct ceph_osd_client *osdc)
+{
+ cancel_delayed_work(&osdc->timeout_work);
+}
+
+/*
+ * Register request, assign tid. If this is the first request, set up
+ * the timeout event.
+ */
+static void register_request(struct ceph_osd_client *osdc,
+ struct ceph_osd_request *req)
+{
+ mutex_lock(&osdc->request_mutex);
+ req->r_tid = ++osdc->last_tid;
+ req->r_request->hdr.tid = cpu_to_le64(req->r_tid);
+ INIT_LIST_HEAD(&req->r_req_lru_item);
+
+ dout("register_request %p tid %lld\n", req, req->r_tid);
+ __insert_request(osdc, req);
+ ceph_osdc_get_request(req);
+ osdc->num_requests++;
+
+ if (osdc->num_requests == 1) {
+ dout(" first request, scheduling timeout\n");
+ __schedule_osd_timeout(osdc);
+ }
+ mutex_unlock(&osdc->request_mutex);
+}
+
+/*
+ * called under osdc->request_mutex
+ */
+static void __unregister_request(struct ceph_osd_client *osdc,
+ struct ceph_osd_request *req)
+{
+ dout("__unregister_request %p tid %lld\n", req, req->r_tid);
+ rb_erase(&req->r_node, &osdc->requests);
+ osdc->num_requests--;
+
+ if (req->r_osd) {
+ /* make sure the original request isn't in flight. */
+ ceph_con_revoke(&req->r_osd->o_con, req->r_request);
+
+ list_del_init(&req->r_osd_item);
+ if (list_empty(&req->r_osd->o_requests))
+ __move_osd_to_lru(osdc, req->r_osd);
+ req->r_osd = NULL;
+ }
+
+ ceph_osdc_put_request(req);
+
+ list_del_init(&req->r_req_lru_item);
+ if (osdc->num_requests == 0) {
+ dout(" no requests, canceling timeout\n");
+ __cancel_osd_timeout(osdc);
+ }
+}
+
+/*
+ * Cancel a previously queued request message
+ */
+static void __cancel_request(struct ceph_osd_request *req)
+{
+ if (req->r_sent) {
+ ceph_con_revoke(&req->r_osd->o_con, req->r_request);
+ req->r_sent = 0;
+ }
+ list_del_init(&req->r_req_lru_item);
+}
+
+/*
+ * Pick an osd (the first 'up' osd in the pg), allocate the osd struct
+ * (as needed), and set the request r_osd appropriately. If there is
+ * no up osd, set r_osd to NULL.
+ *
+ * Return 0 if unchanged, 1 if changed, or negative on error.
+ *
+ * Caller should hold map_sem for read and request_mutex.
+ */
+static int __map_osds(struct ceph_osd_client *osdc,
+ struct ceph_osd_request *req)
+{
+ struct ceph_osd_request_head *reqhead = req->r_request->front.iov_base;
+ struct ceph_pg pgid;
+ int o = -1;
+ int err;
+
+ dout("map_osds %p tid %lld\n", req, req->r_tid);
+ err = ceph_calc_object_layout(&reqhead->layout, req->r_oid,
+ &req->r_file_layout, osdc->osdmap);
+ if (err)
+ return err;
+ pgid = reqhead->layout.ol_pgid;
+ req->r_pgid = pgid;
+
+ o = ceph_calc_pg_primary(osdc->osdmap, pgid);
+
+ if ((req->r_osd && req->r_osd->o_osd == o &&
+ req->r_sent >= req->r_osd->o_incarnation) ||
+ (req->r_osd == NULL && o == -1))
+ return 0; /* no change */
+
+ dout("map_osds tid %llu pgid %d.%x osd%d (was osd%d)\n",
+ req->r_tid, le32_to_cpu(pgid.pool), le16_to_cpu(pgid.ps), o,
+ req->r_osd ? req->r_osd->o_osd : -1);
+
+ if (req->r_osd) {
+ __cancel_request(req);
+ list_del_init(&req->r_osd_item);
+ req->r_osd = NULL;
+ }
+
+ req->r_osd = __lookup_osd(osdc, o);
+ if (!req->r_osd && o >= 0) {
+ err = -ENOMEM;
+ req->r_osd = create_osd(osdc);
+ if (!req->r_osd)
+ goto out;
+
+ dout("map_osds osd %p is osd%d\n", req->r_osd, o);
+ req->r_osd->o_osd = o;
+ req->r_osd->o_con.peer_name.num = cpu_to_le64(o);
+ __insert_osd(osdc, req->r_osd);
+
+ ceph_con_open(&req->r_osd->o_con, &osdc->osdmap->osd_addr[o]);
+ }
+
+ if (req->r_osd) {
+ __remove_osd_from_lru(req->r_osd);
+ list_add(&req->r_osd_item, &req->r_osd->o_requests);
+ }
+ err = 1; /* osd changed */
+
+out:
+ return err;
+}
+
+/*
+ * caller should hold map_sem (for read) and request_mutex
+ */
+static int __send_request(struct ceph_osd_client *osdc,
+ struct ceph_osd_request *req)
+{
+ struct ceph_osd_request_head *reqhead;
+ int err;
+
+ err = __map_osds(osdc, req);
+ if (err < 0)
+ return err;
+ if (req->r_osd == NULL) {
+ dout("send_request %p no up osds in pg\n", req);
+ ceph_monc_request_next_osdmap(&osdc->client->monc);
+ return 0;
+ }
+
+ dout("send_request %p tid %llu to osd%d flags %d\n",
+ req, req->r_tid, req->r_osd->o_osd, req->r_flags);
+
+ reqhead = req->r_request->front.iov_base;
+ reqhead->osdmap_epoch = cpu_to_le32(osdc->osdmap->epoch);
+ reqhead->flags |= cpu_to_le32(req->r_flags); /* e.g., RETRY */
+ reqhead->reassert_version = req->r_reassert_version;
+
+ req->r_stamp = jiffies;
+ list_move_tail(&osdc->req_lru, &req->r_req_lru_item);
+
+ ceph_msg_get(req->r_request); /* send consumes a ref */
+ ceph_con_send(&req->r_osd->o_con, req->r_request);
+ req->r_sent = req->r_osd->o_incarnation;
+ return 0;
+}
+
+/*
+ * Timeout callback, called every N seconds when 1 or more osd
+ * requests has been active for more than N seconds. When this
+ * happens, we ping all OSDs with requests who have timed out to
+ * ensure any communications channel reset is detected. Reset the
+ * request timeouts another N seconds in the future as we go.
+ * Reschedule the timeout event another N seconds in future (unless
+ * there are no open requests).
+ */
+static void handle_timeout(struct work_struct *work)
+{
+ struct ceph_osd_client *osdc =
+ container_of(work, struct ceph_osd_client, timeout_work.work);
+ struct ceph_osd_request *req, *last_req = NULL;
+ struct ceph_osd *osd;
+ unsigned long timeout = osdc->client->mount_args->osd_timeout * HZ;
+ unsigned long keepalive =
+ osdc->client->mount_args->osd_keepalive_timeout * HZ;
+ unsigned long last_stamp = 0;
+ struct rb_node *p;
+ struct list_head slow_osds;
+
+ dout("timeout\n");
+ down_read(&osdc->map_sem);
+
+ ceph_monc_request_next_osdmap(&osdc->client->monc);
+
+ mutex_lock(&osdc->request_mutex);
+ for (p = rb_first(&osdc->requests); p; p = rb_next(p)) {
+ req = rb_entry(p, struct ceph_osd_request, r_node);
+
+ if (req->r_resend) {
+ int err;
+
+ dout("osdc resending prev failed %lld\n", req->r_tid);
+ err = __send_request(osdc, req);
+ if (err)
+ dout("osdc failed again on %lld\n", req->r_tid);
+ else
+ req->r_resend = false;
+ continue;
+ }
+ }
+
+ /*
+ * reset osds that appear to be _really_ unresponsive. this
+ * is a failsafe measure.. we really shouldn't be getting to
+ * this point if the system is working properly. the monitors
+ * should mark the osd as failed and we should find out about
+ * it from an updated osd map.
+ */
+ while (!list_empty(&osdc->req_lru)) {
+ req = list_entry(osdc->req_lru.next, struct ceph_osd_request,
+ r_req_lru_item);
+
+ if (time_before(jiffies, req->r_stamp + timeout))
+ break;
+
+ BUG_ON(req == last_req && req->r_stamp == last_stamp);
+ last_req = req;
+ last_stamp = req->r_stamp;
+
+ osd = req->r_osd;
+ BUG_ON(!osd);
+ pr_warning(" tid %llu timed out on osd%d, will reset osd\n",
+ req->r_tid, osd->o_osd);
+ __kick_requests(osdc, osd);
+ }
+
+ /*
+ * ping osds that are a bit slow. this ensures that if there
+ * is a break in the TCP connection we will notice, and reopen
+ * a connection with that osd (from the fault callback).
+ */
+ INIT_LIST_HEAD(&slow_osds);
+ list_for_each_entry(req, &osdc->req_lru, r_req_lru_item) {
+ if (time_before(jiffies, req->r_stamp + keepalive))
+ break;
+
+ osd = req->r_osd;
+ BUG_ON(!osd);
+ dout(" tid %llu is slow, will send keepalive on osd%d\n",
+ req->r_tid, osd->o_osd);
+ list_move_tail(&osd->o_keepalive_item, &slow_osds);
+ }
+ while (!list_empty(&slow_osds)) {
+ osd = list_entry(slow_osds.next, struct ceph_osd,
+ o_keepalive_item);
+ list_del_init(&osd->o_keepalive_item);
+ ceph_con_keepalive(&osd->o_con);
+ }
+
+ __schedule_osd_timeout(osdc);
+ mutex_unlock(&osdc->request_mutex);
+
+ up_read(&osdc->map_sem);
+}
+
+static void handle_osds_timeout(struct work_struct *work)
+{
+ struct ceph_osd_client *osdc =
+ container_of(work, struct ceph_osd_client,
+ osds_timeout_work.work);
+ unsigned long delay =
+ osdc->client->mount_args->osd_idle_ttl * HZ >> 2;
+
+ dout("osds timeout\n");
+ down_read(&osdc->map_sem);
+ remove_old_osds(osdc, 0);
+ up_read(&osdc->map_sem);
+
+ schedule_delayed_work(&osdc->osds_timeout_work,
+ round_jiffies_relative(delay));
+}
+
+/*
+ * handle osd op reply. either call the callback if it is specified,
+ * or do the completion to wake up the waiting thread.
+ */
+static void handle_reply(struct ceph_osd_client *osdc, struct ceph_msg *msg,
+ struct ceph_connection *con)
+{
+ struct ceph_osd_reply_head *rhead = msg->front.iov_base;
+ struct ceph_osd_request *req;
+ u64 tid;
+ int numops, object_len, flags;
+
+ tid = le64_to_cpu(msg->hdr.tid);
+ if (msg->front.iov_len < sizeof(*rhead))
+ goto bad;
+ numops = le32_to_cpu(rhead->num_ops);
+ object_len = le32_to_cpu(rhead->object_len);
+ if (msg->front.iov_len != sizeof(*rhead) + object_len +
+ numops * sizeof(struct ceph_osd_op))
+ goto bad;
+ dout("handle_reply %p tid %llu\n", msg, tid);
+
+ /* lookup */
+ mutex_lock(&osdc->request_mutex);
+ req = __lookup_request(osdc, tid);
+ if (req == NULL) {
+ dout("handle_reply tid %llu dne\n", tid);
+ mutex_unlock(&osdc->request_mutex);
+ return;
+ }
+ ceph_osdc_get_request(req);
+ flags = le32_to_cpu(rhead->flags);
+
+ /*
+ * if this connection filled our message, drop our reference now, to
+ * avoid a (safe but slower) revoke later.
+ */
+ if (req->r_con_filling_msg == con && req->r_reply == msg) {
+ dout(" dropping con_filling_msg ref %p\n", con);
+ req->r_con_filling_msg = NULL;
+ ceph_con_put(con);
+ }
+
+ if (!req->r_got_reply) {
+ unsigned bytes;
+
+ req->r_result = le32_to_cpu(rhead->result);
+ bytes = le32_to_cpu(msg->hdr.data_len);
+ dout("handle_reply result %d bytes %d\n", req->r_result,
+ bytes);
+ if (req->r_result == 0)
+ req->r_result = bytes;
+
+ /* in case this is a write and we need to replay, */
+ req->r_reassert_version = rhead->reassert_version;
+
+ req->r_got_reply = 1;
+ } else if ((flags & CEPH_OSD_FLAG_ONDISK) == 0) {
+ dout("handle_reply tid %llu dup ack\n", tid);
+ mutex_unlock(&osdc->request_mutex);
+ goto done;
+ }
+
+ dout("handle_reply tid %llu flags %d\n", tid, flags);
+
+ /* either this is a read, or we got the safe response */
+ if ((flags & CEPH_OSD_FLAG_ONDISK) ||
+ ((flags & CEPH_OSD_FLAG_WRITE) == 0))
+ __unregister_request(osdc, req);
+
+ mutex_unlock(&osdc->request_mutex);
+
+ if (req->r_callback)
+ req->r_callback(req, msg);
+ else
+ complete(&req->r_completion);
+
+ if (flags & CEPH_OSD_FLAG_ONDISK) {
+ if (req->r_safe_callback)
+ req->r_safe_callback(req, msg);
+ complete(&req->r_safe_completion); /* fsync waiter */
+ }
+
+done:
+ ceph_osdc_put_request(req);
+ return;
+
+bad:
+ pr_err("corrupt osd_op_reply got %d %d expected %d\n",
+ (int)msg->front.iov_len, le32_to_cpu(msg->hdr.front_len),
+ (int)sizeof(*rhead));
+ ceph_msg_dump(msg);
+}
+
+
+static int __kick_requests(struct ceph_osd_client *osdc,
+ struct ceph_osd *kickosd)
+{
+ struct ceph_osd_request *req;
+ struct rb_node *p, *n;
+ int needmap = 0;
+ int err;
+
+ dout("kick_requests osd%d\n", kickosd ? kickosd->o_osd : -1);
+ if (kickosd) {
+ err = __reset_osd(osdc, kickosd);
+ if (err == -EAGAIN)
+ return 1;
+ } else {
+ for (p = rb_first(&osdc->osds); p; p = n) {
+ struct ceph_osd *osd =
+ rb_entry(p, struct ceph_osd, o_node);
+
+ n = rb_next(p);
+ if (!ceph_osd_is_up(osdc->osdmap, osd->o_osd) ||
+ memcmp(&osd->o_con.peer_addr,
+ ceph_osd_addr(osdc->osdmap,
+ osd->o_osd),
+ sizeof(struct ceph_entity_addr)) != 0)
+ __reset_osd(osdc, osd);
+ }
+ }
+
+ for (p = rb_first(&osdc->requests); p; p = rb_next(p)) {
+ req = rb_entry(p, struct ceph_osd_request, r_node);
+
+ if (req->r_resend) {
+ dout(" r_resend set on tid %llu\n", req->r_tid);
+ __cancel_request(req);
+ goto kick;
+ }
+ if (req->r_osd && kickosd == req->r_osd) {
+ __cancel_request(req);
+ goto kick;
+ }
+
+ err = __map_osds(osdc, req);
+ if (err == 0)
+ continue; /* no change */
+ if (err < 0) {
+ /*
+ * FIXME: really, we should set the request
+ * error and fail if this isn't a 'nofail'
+ * request, but that's a fair bit more
+ * complicated to do. So retry!
+ */
+ dout(" setting r_resend on %llu\n", req->r_tid);
+ req->r_resend = true;
+ continue;
+ }
+ if (req->r_osd == NULL) {
+ dout("tid %llu maps to no valid osd\n", req->r_tid);
+ needmap++; /* request a newer map */
+ continue;
+ }
+
+kick:
+ dout("kicking %p tid %llu osd%d\n", req, req->r_tid,
+ req->r_osd ? req->r_osd->o_osd : -1);
+ req->r_flags |= CEPH_OSD_FLAG_RETRY;
+ err = __send_request(osdc, req);
+ if (err) {
+ dout(" setting r_resend on %llu\n", req->r_tid);
+ req->r_resend = true;
+ }
+ }
+
+ return needmap;
+}
+
+/*
+ * Resubmit osd requests whose osd or osd address has changed. Request
+ * a new osd map if osds are down, or we are otherwise unable to determine
+ * how to direct a request.
+ *
+ * Close connections to down osds.
+ *
+ * If @who is specified, resubmit requests for that specific osd.
+ *
+ * Caller should hold map_sem for read and request_mutex.
+ */
+static void kick_requests(struct ceph_osd_client *osdc,
+ struct ceph_osd *kickosd)
+{
+ int needmap;
+
+ mutex_lock(&osdc->request_mutex);
+ needmap = __kick_requests(osdc, kickosd);
+ mutex_unlock(&osdc->request_mutex);
+
+ if (needmap) {
+ dout("%d requests for down osds, need new map\n", needmap);
+ ceph_monc_request_next_osdmap(&osdc->client->monc);
+ }
+
+}
+/*
+ * Process updated osd map.
+ *
+ * The message contains any number of incremental and full maps, normally
+ * indicating some sort of topology change in the cluster. Kick requests
+ * off to different OSDs as needed.
+ */
+void ceph_osdc_handle_map(struct ceph_osd_client *osdc, struct ceph_msg *msg)
+{
+ void *p, *end, *next;
+ u32 nr_maps, maplen;
+ u32 epoch;
+ struct ceph_osdmap *newmap = NULL, *oldmap;
+ int err;
+ struct ceph_fsid fsid;
+
+ dout("handle_map have %u\n", osdc->osdmap ? osdc->osdmap->epoch : 0);
+ p = msg->front.iov_base;
+ end = p + msg->front.iov_len;
+
+ /* verify fsid */
+ ceph_decode_need(&p, end, sizeof(fsid), bad);
+ ceph_decode_copy(&p, &fsid, sizeof(fsid));
+ if (ceph_check_fsid(osdc->client, &fsid) < 0)
+ return;
+
+ down_write(&osdc->map_sem);
+
+ /* incremental maps */
+ ceph_decode_32_safe(&p, end, nr_maps, bad);
+ dout(" %d inc maps\n", nr_maps);
+ while (nr_maps > 0) {
+ ceph_decode_need(&p, end, 2*sizeof(u32), bad);
+ epoch = ceph_decode_32(&p);
+ maplen = ceph_decode_32(&p);
+ ceph_decode_need(&p, end, maplen, bad);
+ next = p + maplen;
+ if (osdc->osdmap && osdc->osdmap->epoch+1 == epoch) {
+ dout("applying incremental map %u len %d\n",
+ epoch, maplen);
+ newmap = osdmap_apply_incremental(&p, next,
+ osdc->osdmap,
+ osdc->client->msgr);
+ if (IS_ERR(newmap)) {
+ err = PTR_ERR(newmap);
+ goto bad;
+ }
+ BUG_ON(!newmap);
+ if (newmap != osdc->osdmap) {
+ ceph_osdmap_destroy(osdc->osdmap);
+ osdc->osdmap = newmap;
+ }
+ } else {
+ dout("ignoring incremental map %u len %d\n",
+ epoch, maplen);
+ }
+ p = next;
+ nr_maps--;
+ }
+ if (newmap)
+ goto done;
+
+ /* full maps */
+ ceph_decode_32_safe(&p, end, nr_maps, bad);
+ dout(" %d full maps\n", nr_maps);
+ while (nr_maps) {
+ ceph_decode_need(&p, end, 2*sizeof(u32), bad);
+ epoch = ceph_decode_32(&p);
+ maplen = ceph_decode_32(&p);
+ ceph_decode_need(&p, end, maplen, bad);
+ if (nr_maps > 1) {
+ dout("skipping non-latest full map %u len %d\n",
+ epoch, maplen);
+ } else if (osdc->osdmap && osdc->osdmap->epoch >= epoch) {
+ dout("skipping full map %u len %d, "
+ "older than our %u\n", epoch, maplen,
+ osdc->osdmap->epoch);
+ } else {
+ dout("taking full map %u len %d\n", epoch, maplen);
+ newmap = osdmap_decode(&p, p+maplen);
+ if (IS_ERR(newmap)) {
+ err = PTR_ERR(newmap);
+ goto bad;
+ }
+ BUG_ON(!newmap);
+ oldmap = osdc->osdmap;
+ osdc->osdmap = newmap;
+ if (oldmap)
+ ceph_osdmap_destroy(oldmap);
+ }
+ p += maplen;
+ nr_maps--;
+ }
+
+done:
+ downgrade_write(&osdc->map_sem);
+ ceph_monc_got_osdmap(&osdc->client->monc, osdc->osdmap->epoch);
+ if (newmap)
+ kick_requests(osdc, NULL);
+ up_read(&osdc->map_sem);
+ return;
+
+bad:
+ pr_err("osdc handle_map corrupt msg\n");
+ ceph_msg_dump(msg);
+ up_write(&osdc->map_sem);
+ return;
+}
+
+
+/*
+ * A read request prepares specific pages that data is to be read into.
+ * When a message is being read off the wire, we call prepare_pages to
+ * find those pages.
+ * 0 = success, -1 failure.
+ */
+static int __prepare_pages(struct ceph_connection *con,
+ struct ceph_msg_header *hdr,
+ struct ceph_osd_request *req,
+ u64 tid,
+ struct ceph_msg *m)
+{
+ struct ceph_osd *osd = con->private;
+ struct ceph_osd_client *osdc;
+ int ret = -1;
+ int data_len = le32_to_cpu(hdr->data_len);
+ unsigned data_off = le16_to_cpu(hdr->data_off);
+
+ int want = calc_pages_for(data_off & ~PAGE_MASK, data_len);
+
+ if (!osd)
+ return -1;
+
+ osdc = osd->o_osdc;
+
+ dout("__prepare_pages on msg %p tid %llu, has %d pages, want %d\n", m,
+ tid, req->r_num_pages, want);
+ if (unlikely(req->r_num_pages < want))
+ goto out;
+ m->pages = req->r_pages;
+ m->nr_pages = req->r_num_pages;
+ ret = 0; /* success */
+out:
+ BUG_ON(ret < 0 || m->nr_pages < want);
+
+ return ret;
+}
+
+/*
+ * Register request, send initial attempt.
+ */
+int ceph_osdc_start_request(struct ceph_osd_client *osdc,
+ struct ceph_osd_request *req,
+ bool nofail)
+{
+ int rc = 0;
+
+ req->r_request->pages = req->r_pages;
+ req->r_request->nr_pages = req->r_num_pages;
+
+ register_request(osdc, req);
+
+ down_read(&osdc->map_sem);
+ mutex_lock(&osdc->request_mutex);
+ /*
+ * a racing kick_requests() may have sent the message for us
+ * while we dropped request_mutex above, so only send now if
+ * the request still han't been touched yet.
+ */
+ if (req->r_sent == 0) {
+ rc = __send_request(osdc, req);
+ if (rc) {
+ if (nofail) {
+ dout("osdc_start_request failed send, "
+ " marking %lld\n", req->r_tid);
+ req->r_resend = true;
+ rc = 0;
+ } else {
+ __unregister_request(osdc, req);
+ }
+ }
+ }
+ mutex_unlock(&osdc->request_mutex);
+ up_read(&osdc->map_sem);
+ return rc;
+}
+
+/*
+ * wait for a request to complete
+ */
+int ceph_osdc_wait_request(struct ceph_osd_client *osdc,
+ struct ceph_osd_request *req)
+{
+ int rc;
+
+ rc = wait_for_completion_interruptible(&req->r_completion);
+ if (rc < 0) {
+ mutex_lock(&osdc->request_mutex);
+ __cancel_request(req);
+ __unregister_request(osdc, req);
+ mutex_unlock(&osdc->request_mutex);
+ dout("wait_request tid %llu canceled/timed out\n", req->r_tid);
+ return rc;
+ }
+
+ dout("wait_request tid %llu result %d\n", req->r_tid, req->r_result);
+ return req->r_result;
+}
+
+/*
+ * sync - wait for all in-flight requests to flush. avoid starvation.
+ */
+void ceph_osdc_sync(struct ceph_osd_client *osdc)
+{
+ struct ceph_osd_request *req;
+ u64 last_tid, next_tid = 0;
+
+ mutex_lock(&osdc->request_mutex);
+ last_tid = osdc->last_tid;
+ while (1) {
+ req = __lookup_request_ge(osdc, next_tid);
+ if (!req)
+ break;
+ if (req->r_tid > last_tid)
+ break;
+
+ next_tid = req->r_tid + 1;
+ if ((req->r_flags & CEPH_OSD_FLAG_WRITE) == 0)
+ continue;
+
+ ceph_osdc_get_request(req);
+ mutex_unlock(&osdc->request_mutex);
+ dout("sync waiting on tid %llu (last is %llu)\n",
+ req->r_tid, last_tid);
+ wait_for_completion(&req->r_safe_completion);
+ mutex_lock(&osdc->request_mutex);
+ ceph_osdc_put_request(req);
+ }
+ mutex_unlock(&osdc->request_mutex);
+ dout("sync done (thru tid %llu)\n", last_tid);
+}
+
+/*
+ * init, shutdown
+ */
+int ceph_osdc_init(struct ceph_osd_client *osdc, struct ceph_client *client)
+{
+ int err;
+
+ dout("init\n");
+ osdc->client = client;
+ osdc->osdmap = NULL;
+ init_rwsem(&osdc->map_sem);
+ init_completion(&osdc->map_waiters);
+ osdc->last_requested_map = 0;
+ mutex_init(&osdc->request_mutex);
+ osdc->last_tid = 0;
+ osdc->osds = RB_ROOT;
+ INIT_LIST_HEAD(&osdc->osd_lru);
+ osdc->requests = RB_ROOT;
+ INIT_LIST_HEAD(&osdc->req_lru);
+ osdc->num_requests = 0;
+ INIT_DELAYED_WORK(&osdc->timeout_work, handle_timeout);
+ INIT_DELAYED_WORK(&osdc->osds_timeout_work, handle_osds_timeout);
+
+ schedule_delayed_work(&osdc->osds_timeout_work,
+ round_jiffies_relative(osdc->client->mount_args->osd_idle_ttl * HZ));
+
+ err = -ENOMEM;
+ osdc->req_mempool = mempool_create_kmalloc_pool(10,
+ sizeof(struct ceph_osd_request));
+ if (!osdc->req_mempool)
+ goto out;
+
+ err = ceph_msgpool_init(&osdc->msgpool_op, OSD_OP_FRONT_LEN, 10, true);
+ if (err < 0)
+ goto out_mempool;
+ err = ceph_msgpool_init(&osdc->msgpool_op_reply,
+ OSD_OPREPLY_FRONT_LEN, 10, true);
+ if (err < 0)
+ goto out_msgpool;
+ return 0;
+
+out_msgpool:
+ ceph_msgpool_destroy(&osdc->msgpool_op);
+out_mempool:
+ mempool_destroy(osdc->req_mempool);
+out:
+ return err;
+}
+
+void ceph_osdc_stop(struct ceph_osd_client *osdc)
+{
+ cancel_delayed_work_sync(&osdc->timeout_work);
+ cancel_delayed_work_sync(&osdc->osds_timeout_work);
+ if (osdc->osdmap) {
+ ceph_osdmap_destroy(osdc->osdmap);
+ osdc->osdmap = NULL;
+ }
+ remove_old_osds(osdc, 1);
+ mempool_destroy(osdc->req_mempool);
+ ceph_msgpool_destroy(&osdc->msgpool_op);
+ ceph_msgpool_destroy(&osdc->msgpool_op_reply);
+}
+
+/*
+ * Read some contiguous pages. If we cross a stripe boundary, shorten
+ * *plen. Return number of bytes read, or error.
+ */
+int ceph_osdc_readpages(struct ceph_osd_client *osdc,
+ struct ceph_vino vino, struct ceph_file_layout *layout,
+ u64 off, u64 *plen,
+ u32 truncate_seq, u64 truncate_size,
+ struct page **pages, int num_pages)
+{
+ struct ceph_osd_request *req;
+ int rc = 0;
+
+ dout("readpages on ino %llx.%llx on %llu~%llu\n", vino.ino,
+ vino.snap, off, *plen);
+ req = ceph_osdc_new_request(osdc, layout, vino, off, plen,
+ CEPH_OSD_OP_READ, CEPH_OSD_FLAG_READ,
+ NULL, 0, truncate_seq, truncate_size, NULL,
+ false, 1);
+ if (IS_ERR(req))
+ return PTR_ERR(req);
+
+ /* it may be a short read due to an object boundary */
+ req->r_pages = pages;
+ num_pages = calc_pages_for(off, *plen);
+ req->r_num_pages = num_pages;
+
+ dout("readpages final extent is %llu~%llu (%d pages)\n",
+ off, *plen, req->r_num_pages);
+
+ rc = ceph_osdc_start_request(osdc, req, false);
+ if (!rc)
+ rc = ceph_osdc_wait_request(osdc, req);
+
+ ceph_osdc_put_request(req);
+ dout("readpages result %d\n", rc);
+ return rc;
+}
+
+/*
+ * do a synchronous write on N pages
+ */
+int ceph_osdc_writepages(struct ceph_osd_client *osdc, struct ceph_vino vino,
+ struct ceph_file_layout *layout,
+ struct ceph_snap_context *snapc,
+ u64 off, u64 len,
+ u32 truncate_seq, u64 truncate_size,
+ struct timespec *mtime,
+ struct page **pages, int num_pages,
+ int flags, int do_sync, bool nofail)
+{
+ struct ceph_osd_request *req;
+ int rc = 0;
+
+ BUG_ON(vino.snap != CEPH_NOSNAP);
+ req = ceph_osdc_new_request(osdc, layout, vino, off, &len,
+ CEPH_OSD_OP_WRITE,
+ flags | CEPH_OSD_FLAG_ONDISK |
+ CEPH_OSD_FLAG_WRITE,
+ snapc, do_sync,
+ truncate_seq, truncate_size, mtime,
+ nofail, 1);
+ if (IS_ERR(req))
+ return PTR_ERR(req);
+
+ /* it may be a short write due to an object boundary */
+ req->r_pages = pages;
+ req->r_num_pages = calc_pages_for(off, len);
+ dout("writepages %llu~%llu (%d pages)\n", off, len,
+ req->r_num_pages);
+
+ rc = ceph_osdc_start_request(osdc, req, nofail);
+ if (!rc)
+ rc = ceph_osdc_wait_request(osdc, req);
+
+ ceph_osdc_put_request(req);
+ if (rc == 0)
+ rc = len;
+ dout("writepages result %d\n", rc);
+ return rc;
+}
+
+/*
+ * handle incoming message
+ */
+static void dispatch(struct ceph_connection *con, struct ceph_msg *msg)
+{
+ struct ceph_osd *osd = con->private;
+ struct ceph_osd_client *osdc;
+ int type = le16_to_cpu(msg->hdr.type);
+
+ if (!osd)
+ return;
+ osdc = osd->o_osdc;
+
+ switch (type) {
+ case CEPH_MSG_OSD_MAP:
+ ceph_osdc_handle_map(osdc, msg);
+ break;
+ case CEPH_MSG_OSD_OPREPLY:
+ handle_reply(osdc, msg, con);
+ break;
+
+ default:
+ pr_err("received unknown message type %d %s\n", type,
+ ceph_msg_type_name(type));
+ }
+ ceph_msg_put(msg);
+}
+
+/*
+ * lookup and return message for incoming reply
+ */
+static struct ceph_msg *get_reply(struct ceph_connection *con,
+ struct ceph_msg_header *hdr,
+ int *skip)
+{
+ struct ceph_osd *osd = con->private;
+ struct ceph_osd_client *osdc = osd->o_osdc;
+ struct ceph_msg *m;
+ struct ceph_osd_request *req;
+ int front = le32_to_cpu(hdr->front_len);
+ int data_len = le32_to_cpu(hdr->data_len);
+ u64 tid;
+ int err;
+
+ tid = le64_to_cpu(hdr->tid);
+ mutex_lock(&osdc->request_mutex);
+ req = __lookup_request(osdc, tid);
+ if (!req) {
+ *skip = 1;
+ m = NULL;
+ pr_info("get_reply unknown tid %llu from osd%d\n", tid,
+ osd->o_osd);
+ goto out;
+ }
+
+ if (req->r_con_filling_msg) {
+ dout("get_reply revoking msg %p from old con %p\n",
+ req->r_reply, req->r_con_filling_msg);
+ ceph_con_revoke_message(req->r_con_filling_msg, req->r_reply);
+ ceph_con_put(req->r_con_filling_msg);
+ }
+
+ if (front > req->r_reply->front.iov_len) {
+ pr_warning("get_reply front %d > preallocated %d\n",
+ front, (int)req->r_reply->front.iov_len);
+ m = ceph_msg_new(CEPH_MSG_OSD_OPREPLY, front, 0, 0, NULL);
+ if (IS_ERR(m))
+ goto out;
+ ceph_msg_put(req->r_reply);
+ req->r_reply = m;
+ }
+ m = ceph_msg_get(req->r_reply);
+
+ if (data_len > 0) {
+ err = __prepare_pages(con, hdr, req, tid, m);
+ if (err < 0) {
+ *skip = 1;
+ ceph_msg_put(m);
+ m = ERR_PTR(err);
+ }
+ }
+ *skip = 0;
+ req->r_con_filling_msg = ceph_con_get(con);
+ dout("get_reply tid %lld %p\n", tid, m);
+
+out:
+ mutex_unlock(&osdc->request_mutex);
+ return m;
+
+}
+
+static struct ceph_msg *alloc_msg(struct ceph_connection *con,
+ struct ceph_msg_header *hdr,
+ int *skip)
+{
+ struct ceph_osd *osd = con->private;
+ int type = le16_to_cpu(hdr->type);
+ int front = le32_to_cpu(hdr->front_len);
+
+ switch (type) {
+ case CEPH_MSG_OSD_MAP:
+ return ceph_msg_new(type, front, 0, 0, NULL);
+ case CEPH_MSG_OSD_OPREPLY:
+ return get_reply(con, hdr, skip);
+ default:
+ pr_info("alloc_msg unexpected msg type %d from osd%d\n", type,
+ osd->o_osd);
+ *skip = 1;
+ return NULL;
+ }
+}
+
+/*
+ * Wrappers to refcount containing ceph_osd struct
+ */
+static struct ceph_connection *get_osd_con(struct ceph_connection *con)
+{
+ struct ceph_osd *osd = con->private;
+ if (get_osd(osd))
+ return con;
+ return NULL;
+}
+
+static void put_osd_con(struct ceph_connection *con)
+{
+ struct ceph_osd *osd = con->private;
+ put_osd(osd);
+}
+
+/*
+ * authentication
+ */
+static int get_authorizer(struct ceph_connection *con,
+ void **buf, int *len, int *proto,
+ void **reply_buf, int *reply_len, int force_new)
+{
+ struct ceph_osd *o = con->private;
+ struct ceph_osd_client *osdc = o->o_osdc;
+ struct ceph_auth_client *ac = osdc->client->monc.auth;
+ int ret = 0;
+
+ if (force_new && o->o_authorizer) {
+ ac->ops->destroy_authorizer(ac, o->o_authorizer);
+ o->o_authorizer = NULL;
+ }
+ if (o->o_authorizer == NULL) {
+ ret = ac->ops->create_authorizer(
+ ac, CEPH_ENTITY_TYPE_OSD,
+ &o->o_authorizer,
+ &o->o_authorizer_buf,
+ &o->o_authorizer_buf_len,
+ &o->o_authorizer_reply_buf,
+ &o->o_authorizer_reply_buf_len);
+ if (ret)
+ return ret;
+ }
+
+ *proto = ac->protocol;
+ *buf = o->o_authorizer_buf;
+ *len = o->o_authorizer_buf_len;
+ *reply_buf = o->o_authorizer_reply_buf;
+ *reply_len = o->o_authorizer_reply_buf_len;
+ return 0;
+}
+
+
+static int verify_authorizer_reply(struct ceph_connection *con, int len)
+{
+ struct ceph_osd *o = con->private;
+ struct ceph_osd_client *osdc = o->o_osdc;
+ struct ceph_auth_client *ac = osdc->client->monc.auth;
+
+ return ac->ops->verify_authorizer_reply(ac, o->o_authorizer, len);
+}
+
+static int invalidate_authorizer(struct ceph_connection *con)
+{
+ struct ceph_osd *o = con->private;
+ struct ceph_osd_client *osdc = o->o_osdc;
+ struct ceph_auth_client *ac = osdc->client->monc.auth;
+
+ if (ac->ops->invalidate_authorizer)
+ ac->ops->invalidate_authorizer(ac, CEPH_ENTITY_TYPE_OSD);
+
+ return ceph_monc_validate_auth(&osdc->client->monc);
+}
+
+const static struct ceph_connection_operations osd_con_ops = {
+ .get = get_osd_con,
+ .put = put_osd_con,
+ .dispatch = dispatch,
+ .get_authorizer = get_authorizer,
+ .verify_authorizer_reply = verify_authorizer_reply,
+ .invalidate_authorizer = invalidate_authorizer,
+ .alloc_msg = alloc_msg,
+ .fault = osd_reset,
+};
--- /dev/null
+#ifndef _FS_CEPH_OSD_CLIENT_H
+#define _FS_CEPH_OSD_CLIENT_H
+
+#include <linux/completion.h>
+#include <linux/kref.h>
+#include <linux/mempool.h>
+#include <linux/rbtree.h>
+
+#include "types.h"
+#include "osdmap.h"
+#include "messenger.h"
+
+struct ceph_msg;
+struct ceph_snap_context;
+struct ceph_osd_request;
+struct ceph_osd_client;
+struct ceph_authorizer;
+
+/*
+ * completion callback for async writepages
+ */
+typedef void (*ceph_osdc_callback_t)(struct ceph_osd_request *,
+ struct ceph_msg *);
+
+/* a given osd we're communicating with */
+struct ceph_osd {
+ atomic_t o_ref;
+ struct ceph_osd_client *o_osdc;
+ int o_osd;
+ int o_incarnation;
+ struct rb_node o_node;
+ struct ceph_connection o_con;
+ struct list_head o_requests;
+ struct list_head o_osd_lru;
+ struct ceph_authorizer *o_authorizer;
+ void *o_authorizer_buf, *o_authorizer_reply_buf;
+ size_t o_authorizer_buf_len, o_authorizer_reply_buf_len;
+ unsigned long lru_ttl;
+ int o_marked_for_keepalive;
+ struct list_head o_keepalive_item;
+};
+
+/* an in-flight request */
+struct ceph_osd_request {
+ u64 r_tid; /* unique for this client */
+ struct rb_node r_node;
+ struct list_head r_req_lru_item;
+ struct list_head r_osd_item;
+ struct ceph_osd *r_osd;
+ struct ceph_pg r_pgid;
+
+ struct ceph_connection *r_con_filling_msg;
+
+ struct ceph_msg *r_request, *r_reply;
+ int r_result;
+ int r_flags; /* any additional flags for the osd */
+ u32 r_sent; /* >0 if r_request is sending/sent */
+ int r_got_reply;
+
+ struct ceph_osd_client *r_osdc;
+ struct kref r_kref;
+ bool r_mempool;
+ struct completion r_completion, r_safe_completion;
+ ceph_osdc_callback_t r_callback, r_safe_callback;
+ struct ceph_eversion r_reassert_version;
+ struct list_head r_unsafe_item;
+
+ struct inode *r_inode; /* for use by callbacks */
+ struct writeback_control *r_wbc; /* ditto */
+
+ char r_oid[40]; /* object name */
+ int r_oid_len;
+ unsigned long r_stamp; /* send OR check time */
+ bool r_resend; /* msg send failed, needs retry */
+
+ struct ceph_file_layout r_file_layout;
+ struct ceph_snap_context *r_snapc; /* snap context for writes */
+ unsigned r_num_pages; /* size of page array (follows) */
+ struct page **r_pages; /* pages for data payload */
+ int r_pages_from_pool;
+ int r_own_pages; /* if true, i own page list */
+};
+
+struct ceph_osd_client {
+ struct ceph_client *client;
+
+ struct ceph_osdmap *osdmap; /* current map */
+ struct rw_semaphore map_sem;
+ struct completion map_waiters;
+ u64 last_requested_map;
+
+ struct mutex request_mutex;
+ struct rb_root osds; /* osds */
+ struct list_head osd_lru; /* idle osds */
+ u64 timeout_tid; /* tid of timeout triggering rq */
+ u64 last_tid; /* tid of last request */
+ struct rb_root requests; /* pending requests */
+ struct list_head req_lru; /* pending requests lru */
+ int num_requests;
+ struct delayed_work timeout_work;
+ struct delayed_work osds_timeout_work;
+#ifdef CONFIG_DEBUG_FS
+ struct dentry *debugfs_file;
+#endif
+
+ mempool_t *req_mempool;
+
+ struct ceph_msgpool msgpool_op;
+ struct ceph_msgpool msgpool_op_reply;
+};
+
+extern int ceph_osdc_init(struct ceph_osd_client *osdc,
+ struct ceph_client *client);
+extern void ceph_osdc_stop(struct ceph_osd_client *osdc);
+
+extern void ceph_osdc_handle_reply(struct ceph_osd_client *osdc,
+ struct ceph_msg *msg);
+extern void ceph_osdc_handle_map(struct ceph_osd_client *osdc,
+ struct ceph_msg *msg);
+
+extern struct ceph_osd_request *ceph_osdc_new_request(struct ceph_osd_client *,
+ struct ceph_file_layout *layout,
+ struct ceph_vino vino,
+ u64 offset, u64 *len, int op, int flags,
+ struct ceph_snap_context *snapc,
+ int do_sync, u32 truncate_seq,
+ u64 truncate_size,
+ struct timespec *mtime,
+ bool use_mempool, int num_reply);
+
+static inline void ceph_osdc_get_request(struct ceph_osd_request *req)
+{
+ kref_get(&req->r_kref);
+}
+extern void ceph_osdc_release_request(struct kref *kref);
+static inline void ceph_osdc_put_request(struct ceph_osd_request *req)
+{
+ kref_put(&req->r_kref, ceph_osdc_release_request);
+}
+
+extern int ceph_osdc_start_request(struct ceph_osd_client *osdc,
+ struct ceph_osd_request *req,
+ bool nofail);
+extern int ceph_osdc_wait_request(struct ceph_osd_client *osdc,
+ struct ceph_osd_request *req);
+extern void ceph_osdc_sync(struct ceph_osd_client *osdc);
+
+extern int ceph_osdc_readpages(struct ceph_osd_client *osdc,
+ struct ceph_vino vino,
+ struct ceph_file_layout *layout,
+ u64 off, u64 *plen,
+ u32 truncate_seq, u64 truncate_size,
+ struct page **pages, int nr_pages);
+
+extern int ceph_osdc_writepages(struct ceph_osd_client *osdc,
+ struct ceph_vino vino,
+ struct ceph_file_layout *layout,
+ struct ceph_snap_context *sc,
+ u64 off, u64 len,
+ u32 truncate_seq, u64 truncate_size,
+ struct timespec *mtime,
+ struct page **pages, int nr_pages,
+ int flags, int do_sync, bool nofail);
+
+#endif
+
--- /dev/null
+
+#include "ceph_debug.h"
+
+#include <linux/slab.h>
+#include <asm/div64.h>
+
+#include "super.h"
+#include "osdmap.h"
+#include "crush/hash.h"
+#include "crush/mapper.h"
+#include "decode.h"
+
+char *ceph_osdmap_state_str(char *str, int len, int state)
+{
+ int flag = 0;
+
+ if (!len)
+ goto done;
+
+ *str = '\0';
+ if (state) {
+ if (state & CEPH_OSD_EXISTS) {
+ snprintf(str, len, "exists");
+ flag = 1;
+ }
+ if (state & CEPH_OSD_UP) {
+ snprintf(str, len, "%s%s%s", str, (flag ? ", " : ""),
+ "up");
+ flag = 1;
+ }
+ } else {
+ snprintf(str, len, "doesn't exist");
+ }
+done:
+ return str;
+}
+
+/* maps */
+
+static int calc_bits_of(unsigned t)
+{
+ int b = 0;
+ while (t) {
+ t = t >> 1;
+ b++;
+ }
+ return b;
+}
+
+/*
+ * the foo_mask is the smallest value 2^n-1 that is >= foo.
+ */
+static void calc_pg_masks(struct ceph_pg_pool_info *pi)
+{
+ pi->pg_num_mask = (1 << calc_bits_of(le32_to_cpu(pi->v.pg_num)-1)) - 1;
+ pi->pgp_num_mask =
+ (1 << calc_bits_of(le32_to_cpu(pi->v.pgp_num)-1)) - 1;
+ pi->lpg_num_mask =
+ (1 << calc_bits_of(le32_to_cpu(pi->v.lpg_num)-1)) - 1;
+ pi->lpgp_num_mask =
+ (1 << calc_bits_of(le32_to_cpu(pi->v.lpgp_num)-1)) - 1;
+}
+
+/*
+ * decode crush map
+ */
+static int crush_decode_uniform_bucket(void **p, void *end,
+ struct crush_bucket_uniform *b)
+{
+ dout("crush_decode_uniform_bucket %p to %p\n", *p, end);
+ ceph_decode_need(p, end, (1+b->h.size) * sizeof(u32), bad);
+ b->item_weight = ceph_decode_32(p);
+ return 0;
+bad:
+ return -EINVAL;
+}
+
+static int crush_decode_list_bucket(void **p, void *end,
+ struct crush_bucket_list *b)
+{
+ int j;
+ dout("crush_decode_list_bucket %p to %p\n", *p, end);
+ b->item_weights = kcalloc(b->h.size, sizeof(u32), GFP_NOFS);
+ if (b->item_weights == NULL)
+ return -ENOMEM;
+ b->sum_weights = kcalloc(b->h.size, sizeof(u32), GFP_NOFS);
+ if (b->sum_weights == NULL)
+ return -ENOMEM;
+ ceph_decode_need(p, end, 2 * b->h.size * sizeof(u32), bad);
+ for (j = 0; j < b->h.size; j++) {
+ b->item_weights[j] = ceph_decode_32(p);
+ b->sum_weights[j] = ceph_decode_32(p);
+ }
+ return 0;
+bad:
+ return -EINVAL;
+}
+
+static int crush_decode_tree_bucket(void **p, void *end,
+ struct crush_bucket_tree *b)
+{
+ int j;
+ dout("crush_decode_tree_bucket %p to %p\n", *p, end);
+ ceph_decode_32_safe(p, end, b->num_nodes, bad);
+ b->node_weights = kcalloc(b->num_nodes, sizeof(u32), GFP_NOFS);
+ if (b->node_weights == NULL)
+ return -ENOMEM;
+ ceph_decode_need(p, end, b->num_nodes * sizeof(u32), bad);
+ for (j = 0; j < b->num_nodes; j++)
+ b->node_weights[j] = ceph_decode_32(p);
+ return 0;
+bad:
+ return -EINVAL;
+}
+
+static int crush_decode_straw_bucket(void **p, void *end,
+ struct crush_bucket_straw *b)
+{
+ int j;
+ dout("crush_decode_straw_bucket %p to %p\n", *p, end);
+ b->item_weights = kcalloc(b->h.size, sizeof(u32), GFP_NOFS);
+ if (b->item_weights == NULL)
+ return -ENOMEM;
+ b->straws = kcalloc(b->h.size, sizeof(u32), GFP_NOFS);
+ if (b->straws == NULL)
+ return -ENOMEM;
+ ceph_decode_need(p, end, 2 * b->h.size * sizeof(u32), bad);
+ for (j = 0; j < b->h.size; j++) {
+ b->item_weights[j] = ceph_decode_32(p);
+ b->straws[j] = ceph_decode_32(p);
+ }
+ return 0;
+bad:
+ return -EINVAL;
+}
+
+static struct crush_map *crush_decode(void *pbyval, void *end)
+{
+ struct crush_map *c;
+ int err = -EINVAL;
+ int i, j;
+ void **p = &pbyval;
+ void *start = pbyval;
+ u32 magic;
+
+ dout("crush_decode %p to %p len %d\n", *p, end, (int)(end - *p));
+
+ c = kzalloc(sizeof(*c), GFP_NOFS);
+ if (c == NULL)
+ return ERR_PTR(-ENOMEM);
+
+ ceph_decode_need(p, end, 4*sizeof(u32), bad);
+ magic = ceph_decode_32(p);
+ if (magic != CRUSH_MAGIC) {
+ pr_err("crush_decode magic %x != current %x\n",
+ (unsigned)magic, (unsigned)CRUSH_MAGIC);
+ goto bad;
+ }
+ c->max_buckets = ceph_decode_32(p);
+ c->max_rules = ceph_decode_32(p);
+ c->max_devices = ceph_decode_32(p);
+
+ c->device_parents = kcalloc(c->max_devices, sizeof(u32), GFP_NOFS);
+ if (c->device_parents == NULL)
+ goto badmem;
+ c->bucket_parents = kcalloc(c->max_buckets, sizeof(u32), GFP_NOFS);
+ if (c->bucket_parents == NULL)
+ goto badmem;
+
+ c->buckets = kcalloc(c->max_buckets, sizeof(*c->buckets), GFP_NOFS);
+ if (c->buckets == NULL)
+ goto badmem;
+ c->rules = kcalloc(c->max_rules, sizeof(*c->rules), GFP_NOFS);
+ if (c->rules == NULL)
+ goto badmem;
+
+ /* buckets */
+ for (i = 0; i < c->max_buckets; i++) {
+ int size = 0;
+ u32 alg;
+ struct crush_bucket *b;
+
+ ceph_decode_32_safe(p, end, alg, bad);
+ if (alg == 0) {
+ c->buckets[i] = NULL;
+ continue;
+ }
+ dout("crush_decode bucket %d off %x %p to %p\n",
+ i, (int)(*p-start), *p, end);
+
+ switch (alg) {
+ case CRUSH_BUCKET_UNIFORM:
+ size = sizeof(struct crush_bucket_uniform);
+ break;
+ case CRUSH_BUCKET_LIST:
+ size = sizeof(struct crush_bucket_list);
+ break;
+ case CRUSH_BUCKET_TREE:
+ size = sizeof(struct crush_bucket_tree);
+ break;
+ case CRUSH_BUCKET_STRAW:
+ size = sizeof(struct crush_bucket_straw);
+ break;
+ default:
+ err = -EINVAL;
+ goto bad;
+ }
+ BUG_ON(size == 0);
+ b = c->buckets[i] = kzalloc(size, GFP_NOFS);
+ if (b == NULL)
+ goto badmem;
+
+ ceph_decode_need(p, end, 4*sizeof(u32), bad);
+ b->id = ceph_decode_32(p);
+ b->type = ceph_decode_16(p);
+ b->alg = ceph_decode_8(p);
+ b->hash = ceph_decode_8(p);
+ b->weight = ceph_decode_32(p);
+ b->size = ceph_decode_32(p);
+
+ dout("crush_decode bucket size %d off %x %p to %p\n",
+ b->size, (int)(*p-start), *p, end);
+
+ b->items = kcalloc(b->size, sizeof(__s32), GFP_NOFS);
+ if (b->items == NULL)
+ goto badmem;
+ b->perm = kcalloc(b->size, sizeof(u32), GFP_NOFS);
+ if (b->perm == NULL)
+ goto badmem;
+ b->perm_n = 0;
+
+ ceph_decode_need(p, end, b->size*sizeof(u32), bad);
+ for (j = 0; j < b->size; j++)
+ b->items[j] = ceph_decode_32(p);
+
+ switch (b->alg) {
+ case CRUSH_BUCKET_UNIFORM:
+ err = crush_decode_uniform_bucket(p, end,
+ (struct crush_bucket_uniform *)b);
+ if (err < 0)
+ goto bad;
+ break;
+ case CRUSH_BUCKET_LIST:
+ err = crush_decode_list_bucket(p, end,
+ (struct crush_bucket_list *)b);
+ if (err < 0)
+ goto bad;
+ break;
+ case CRUSH_BUCKET_TREE:
+ err = crush_decode_tree_bucket(p, end,
+ (struct crush_bucket_tree *)b);
+ if (err < 0)
+ goto bad;
+ break;
+ case CRUSH_BUCKET_STRAW:
+ err = crush_decode_straw_bucket(p, end,
+ (struct crush_bucket_straw *)b);
+ if (err < 0)
+ goto bad;
+ break;
+ }
+ }
+
+ /* rules */
+ dout("rule vec is %p\n", c->rules);
+ for (i = 0; i < c->max_rules; i++) {
+ u32 yes;
+ struct crush_rule *r;
+
+ ceph_decode_32_safe(p, end, yes, bad);
+ if (!yes) {
+ dout("crush_decode NO rule %d off %x %p to %p\n",
+ i, (int)(*p-start), *p, end);
+ c->rules[i] = NULL;
+ continue;
+ }
+
+ dout("crush_decode rule %d off %x %p to %p\n",
+ i, (int)(*p-start), *p, end);
+
+ /* len */
+ ceph_decode_32_safe(p, end, yes, bad);
+#if BITS_PER_LONG == 32
+ err = -EINVAL;
+ if (yes > ULONG_MAX / sizeof(struct crush_rule_step))
+ goto bad;
+#endif
+ r = c->rules[i] = kmalloc(sizeof(*r) +
+ yes*sizeof(struct crush_rule_step),
+ GFP_NOFS);
+ if (r == NULL)
+ goto badmem;
+ dout(" rule %d is at %p\n", i, r);
+ r->len = yes;
+ ceph_decode_copy_safe(p, end, &r->mask, 4, bad); /* 4 u8's */
+ ceph_decode_need(p, end, r->len*3*sizeof(u32), bad);
+ for (j = 0; j < r->len; j++) {
+ r->steps[j].op = ceph_decode_32(p);
+ r->steps[j].arg1 = ceph_decode_32(p);
+ r->steps[j].arg2 = ceph_decode_32(p);
+ }
+ }
+
+ /* ignore trailing name maps. */
+
+ dout("crush_decode success\n");
+ return c;
+
+badmem:
+ err = -ENOMEM;
+bad:
+ dout("crush_decode fail %d\n", err);
+ crush_destroy(c);
+ return ERR_PTR(err);
+}
+
+
+/*
+ * osd map
+ */
+void ceph_osdmap_destroy(struct ceph_osdmap *map)
+{
+ dout("osdmap_destroy %p\n", map);
+ if (map->crush)
+ crush_destroy(map->crush);
+ while (!RB_EMPTY_ROOT(&map->pg_temp)) {
+ struct ceph_pg_mapping *pg =
+ rb_entry(rb_first(&map->pg_temp),
+ struct ceph_pg_mapping, node);
+ rb_erase(&pg->node, &map->pg_temp);
+ kfree(pg);
+ }
+ while (!RB_EMPTY_ROOT(&map->pg_pools)) {
+ struct ceph_pg_pool_info *pi =
+ rb_entry(rb_first(&map->pg_pools),
+ struct ceph_pg_pool_info, node);
+ rb_erase(&pi->node, &map->pg_pools);
+ kfree(pi);
+ }
+ kfree(map->osd_state);
+ kfree(map->osd_weight);
+ kfree(map->osd_addr);
+ kfree(map);
+}
+
+/*
+ * adjust max osd value. reallocate arrays.
+ */
+static int osdmap_set_max_osd(struct ceph_osdmap *map, int max)
+{
+ u8 *state;
+ struct ceph_entity_addr *addr;
+ u32 *weight;
+
+ state = kcalloc(max, sizeof(*state), GFP_NOFS);
+ addr = kcalloc(max, sizeof(*addr), GFP_NOFS);
+ weight = kcalloc(max, sizeof(*weight), GFP_NOFS);
+ if (state == NULL || addr == NULL || weight == NULL) {
+ kfree(state);
+ kfree(addr);
+ kfree(weight);
+ return -ENOMEM;
+ }
+
+ /* copy old? */
+ if (map->osd_state) {
+ memcpy(state, map->osd_state, map->max_osd*sizeof(*state));
+ memcpy(addr, map->osd_addr, map->max_osd*sizeof(*addr));
+ memcpy(weight, map->osd_weight, map->max_osd*sizeof(*weight));
+ kfree(map->osd_state);
+ kfree(map->osd_addr);
+ kfree(map->osd_weight);
+ }
+
+ map->osd_state = state;
+ map->osd_weight = weight;
+ map->osd_addr = addr;
+ map->max_osd = max;
+ return 0;
+}
+
+/*
+ * rbtree of pg_mapping for handling pg_temp (explicit mapping of pgid
+ * to a set of osds)
+ */
+static int pgid_cmp(struct ceph_pg l, struct ceph_pg r)
+{
+ u64 a = *(u64 *)&l;
+ u64 b = *(u64 *)&r;
+
+ if (a < b)
+ return -1;
+ if (a > b)
+ return 1;
+ return 0;
+}
+
+static int __insert_pg_mapping(struct ceph_pg_mapping *new,
+ struct rb_root *root)
+{
+ struct rb_node **p = &root->rb_node;
+ struct rb_node *parent = NULL;
+ struct ceph_pg_mapping *pg = NULL;
+ int c;
+
+ while (*p) {
+ parent = *p;
+ pg = rb_entry(parent, struct ceph_pg_mapping, node);
+ c = pgid_cmp(new->pgid, pg->pgid);
+ if (c < 0)
+ p = &(*p)->rb_left;
+ else if (c > 0)
+ p = &(*p)->rb_right;
+ else
+ return -EEXIST;
+ }
+
+ rb_link_node(&new->node, parent, p);
+ rb_insert_color(&new->node, root);
+ return 0;
+}
+
+static struct ceph_pg_mapping *__lookup_pg_mapping(struct rb_root *root,
+ struct ceph_pg pgid)
+{
+ struct rb_node *n = root->rb_node;
+ struct ceph_pg_mapping *pg;
+ int c;
+
+ while (n) {
+ pg = rb_entry(n, struct ceph_pg_mapping, node);
+ c = pgid_cmp(pgid, pg->pgid);
+ if (c < 0)
+ n = n->rb_left;
+ else if (c > 0)
+ n = n->rb_right;
+ else
+ return pg;
+ }
+ return NULL;
+}
+
+/*
+ * rbtree of pg pool info
+ */
+static int __insert_pg_pool(struct rb_root *root, struct ceph_pg_pool_info *new)
+{
+ struct rb_node **p = &root->rb_node;
+ struct rb_node *parent = NULL;
+ struct ceph_pg_pool_info *pi = NULL;
+
+ while (*p) {
+ parent = *p;
+ pi = rb_entry(parent, struct ceph_pg_pool_info, node);
+ if (new->id < pi->id)
+ p = &(*p)->rb_left;
+ else if (new->id > pi->id)
+ p = &(*p)->rb_right;
+ else
+ return -EEXIST;
+ }
+
+ rb_link_node(&new->node, parent, p);
+ rb_insert_color(&new->node, root);
+ return 0;
+}
+
+static struct ceph_pg_pool_info *__lookup_pg_pool(struct rb_root *root, int id)
+{
+ struct ceph_pg_pool_info *pi;
+ struct rb_node *n = root->rb_node;
+
+ while (n) {
+ pi = rb_entry(n, struct ceph_pg_pool_info, node);
+ if (id < pi->id)
+ n = n->rb_left;
+ else if (id > pi->id)
+ n = n->rb_right;
+ else
+ return pi;
+ }
+ return NULL;
+}
+
+void __decode_pool(void **p, struct ceph_pg_pool_info *pi)
+{
+ ceph_decode_copy(p, &pi->v, sizeof(pi->v));
+ calc_pg_masks(pi);
+ *p += le32_to_cpu(pi->v.num_snaps) * sizeof(u64);
+ *p += le32_to_cpu(pi->v.num_removed_snap_intervals) * sizeof(u64) * 2;
+}
+
+/*
+ * decode a full map.
+ */
+struct ceph_osdmap *osdmap_decode(void **p, void *end)
+{
+ struct ceph_osdmap *map;
+ u16 version;
+ u32 len, max, i;
+ u8 ev;
+ int err = -EINVAL;
+ void *start = *p;
+ struct ceph_pg_pool_info *pi;
+
+ dout("osdmap_decode %p to %p len %d\n", *p, end, (int)(end - *p));
+
+ map = kzalloc(sizeof(*map), GFP_NOFS);
+ if (map == NULL)
+ return ERR_PTR(-ENOMEM);
+ map->pg_temp = RB_ROOT;
+
+ ceph_decode_16_safe(p, end, version, bad);
+ if (version > CEPH_OSDMAP_VERSION) {
+ pr_warning("got unknown v %d > %d of osdmap\n", version,
+ CEPH_OSDMAP_VERSION);
+ goto bad;
+ }
+
+ ceph_decode_need(p, end, 2*sizeof(u64)+6*sizeof(u32), bad);
+ ceph_decode_copy(p, &map->fsid, sizeof(map->fsid));
+ map->epoch = ceph_decode_32(p);
+ ceph_decode_copy(p, &map->created, sizeof(map->created));
+ ceph_decode_copy(p, &map->modified, sizeof(map->modified));
+
+ ceph_decode_32_safe(p, end, max, bad);
+ while (max--) {
+ ceph_decode_need(p, end, 4 + 1 + sizeof(pi->v), bad);
+ pi = kmalloc(sizeof(*pi), GFP_NOFS);
+ if (!pi)
+ goto bad;
+ pi->id = ceph_decode_32(p);
+ ev = ceph_decode_8(p); /* encoding version */
+ if (ev > CEPH_PG_POOL_VERSION) {
+ pr_warning("got unknown v %d > %d of ceph_pg_pool\n",
+ ev, CEPH_PG_POOL_VERSION);
+ goto bad;
+ }
+ __decode_pool(p, pi);
+ __insert_pg_pool(&map->pg_pools, pi);
+ }
+ ceph_decode_32_safe(p, end, map->pool_max, bad);
+
+ ceph_decode_32_safe(p, end, map->flags, bad);
+
+ max = ceph_decode_32(p);
+
+ /* (re)alloc osd arrays */
+ err = osdmap_set_max_osd(map, max);
+ if (err < 0)
+ goto bad;
+ dout("osdmap_decode max_osd = %d\n", map->max_osd);
+
+ /* osds */
+ err = -EINVAL;
+ ceph_decode_need(p, end, 3*sizeof(u32) +
+ map->max_osd*(1 + sizeof(*map->osd_weight) +
+ sizeof(*map->osd_addr)), bad);
+ *p += 4; /* skip length field (should match max) */
+ ceph_decode_copy(p, map->osd_state, map->max_osd);
+
+ *p += 4; /* skip length field (should match max) */
+ for (i = 0; i < map->max_osd; i++)
+ map->osd_weight[i] = ceph_decode_32(p);
+
+ *p += 4; /* skip length field (should match max) */
+ ceph_decode_copy(p, map->osd_addr, map->max_osd*sizeof(*map->osd_addr));
+ for (i = 0; i < map->max_osd; i++)
+ ceph_decode_addr(&map->osd_addr[i]);
+
+ /* pg_temp */
+ ceph_decode_32_safe(p, end, len, bad);
+ for (i = 0; i < len; i++) {
+ int n, j;
+ struct ceph_pg pgid;
+ struct ceph_pg_mapping *pg;
+
+ ceph_decode_need(p, end, sizeof(u32) + sizeof(u64), bad);
+ ceph_decode_copy(p, &pgid, sizeof(pgid));
+ n = ceph_decode_32(p);
+ ceph_decode_need(p, end, n * sizeof(u32), bad);
+ err = -ENOMEM;
+ pg = kmalloc(sizeof(*pg) + n*sizeof(u32), GFP_NOFS);
+ if (!pg)
+ goto bad;
+ pg->pgid = pgid;
+ pg->len = n;
+ for (j = 0; j < n; j++)
+ pg->osds[j] = ceph_decode_32(p);
+
+ err = __insert_pg_mapping(pg, &map->pg_temp);
+ if (err)
+ goto bad;
+ dout(" added pg_temp %llx len %d\n", *(u64 *)&pgid, len);
+ }
+
+ /* crush */
+ ceph_decode_32_safe(p, end, len, bad);
+ dout("osdmap_decode crush len %d from off 0x%x\n", len,
+ (int)(*p - start));
+ ceph_decode_need(p, end, len, bad);
+ map->crush = crush_decode(*p, end);
+ *p += len;
+ if (IS_ERR(map->crush)) {
+ err = PTR_ERR(map->crush);
+ map->crush = NULL;
+ goto bad;
+ }
+
+ /* ignore the rest of the map */
+ *p = end;
+
+ dout("osdmap_decode done %p %p\n", *p, end);
+ return map;
+
+bad:
+ dout("osdmap_decode fail\n");
+ ceph_osdmap_destroy(map);
+ return ERR_PTR(err);
+}
+
+/*
+ * decode and apply an incremental map update.
+ */
+struct ceph_osdmap *osdmap_apply_incremental(void **p, void *end,
+ struct ceph_osdmap *map,
+ struct ceph_messenger *msgr)
+{
+ struct crush_map *newcrush = NULL;
+ struct ceph_fsid fsid;
+ u32 epoch = 0;
+ struct ceph_timespec modified;
+ u32 len, pool;
+ __s32 new_pool_max, new_flags, max;
+ void *start = *p;
+ int err = -EINVAL;
+ u16 version;
+ struct rb_node *rbp;
+
+ ceph_decode_16_safe(p, end, version, bad);
+ if (version > CEPH_OSDMAP_INC_VERSION) {
+ pr_warning("got unknown v %d > %d of inc osdmap\n", version,
+ CEPH_OSDMAP_INC_VERSION);
+ goto bad;
+ }
+
+ ceph_decode_need(p, end, sizeof(fsid)+sizeof(modified)+2*sizeof(u32),
+ bad);
+ ceph_decode_copy(p, &fsid, sizeof(fsid));
+ epoch = ceph_decode_32(p);
+ BUG_ON(epoch != map->epoch+1);
+ ceph_decode_copy(p, &modified, sizeof(modified));
+ new_pool_max = ceph_decode_32(p);
+ new_flags = ceph_decode_32(p);
+
+ /* full map? */
+ ceph_decode_32_safe(p, end, len, bad);
+ if (len > 0) {
+ dout("apply_incremental full map len %d, %p to %p\n",
+ len, *p, end);
+ return osdmap_decode(p, min(*p+len, end));
+ }
+
+ /* new crush? */
+ ceph_decode_32_safe(p, end, len, bad);
+ if (len > 0) {
+ dout("apply_incremental new crush map len %d, %p to %p\n",
+ len, *p, end);
+ newcrush = crush_decode(*p, min(*p+len, end));
+ if (IS_ERR(newcrush))
+ return ERR_PTR(PTR_ERR(newcrush));
+ }
+
+ /* new flags? */
+ if (new_flags >= 0)
+ map->flags = new_flags;
+ if (new_pool_max >= 0)
+ map->pool_max = new_pool_max;
+
+ ceph_decode_need(p, end, 5*sizeof(u32), bad);
+
+ /* new max? */
+ max = ceph_decode_32(p);
+ if (max >= 0) {
+ err = osdmap_set_max_osd(map, max);
+ if (err < 0)
+ goto bad;
+ }
+
+ map->epoch++;
+ map->modified = map->modified;
+ if (newcrush) {
+ if (map->crush)
+ crush_destroy(map->crush);
+ map->crush = newcrush;
+ newcrush = NULL;
+ }
+
+ /* new_pool */
+ ceph_decode_32_safe(p, end, len, bad);
+ while (len--) {
+ __u8 ev;
+ struct ceph_pg_pool_info *pi;
+
+ ceph_decode_32_safe(p, end, pool, bad);
+ ceph_decode_need(p, end, 1 + sizeof(pi->v), bad);
+ ev = ceph_decode_8(p); /* encoding version */
+ if (ev > CEPH_PG_POOL_VERSION) {
+ pr_warning("got unknown v %d > %d of ceph_pg_pool\n",
+ ev, CEPH_PG_POOL_VERSION);
+ goto bad;
+ }
+ pi = __lookup_pg_pool(&map->pg_pools, pool);
+ if (!pi) {
+ pi = kmalloc(sizeof(*pi), GFP_NOFS);
+ if (!pi) {
+ err = -ENOMEM;
+ goto bad;
+ }
+ pi->id = pool;
+ __insert_pg_pool(&map->pg_pools, pi);
+ }
+ __decode_pool(p, pi);
+ }
+
+ /* old_pool */
+ ceph_decode_32_safe(p, end, len, bad);
+ while (len--) {
+ struct ceph_pg_pool_info *pi;
+
+ ceph_decode_32_safe(p, end, pool, bad);
+ pi = __lookup_pg_pool(&map->pg_pools, pool);
+ if (pi) {
+ rb_erase(&pi->node, &map->pg_pools);
+ kfree(pi);
+ }
+ }
+
+ /* new_up */
+ err = -EINVAL;
+ ceph_decode_32_safe(p, end, len, bad);
+ while (len--) {
+ u32 osd;
+ struct ceph_entity_addr addr;
+ ceph_decode_32_safe(p, end, osd, bad);
+ ceph_decode_copy_safe(p, end, &addr, sizeof(addr), bad);
+ ceph_decode_addr(&addr);
+ pr_info("osd%d up\n", osd);
+ BUG_ON(osd >= map->max_osd);
+ map->osd_state[osd] |= CEPH_OSD_UP;
+ map->osd_addr[osd] = addr;
+ }
+
+ /* new_down */
+ ceph_decode_32_safe(p, end, len, bad);
+ while (len--) {
+ u32 osd;
+ ceph_decode_32_safe(p, end, osd, bad);
+ (*p)++; /* clean flag */
+ pr_info("osd%d down\n", osd);
+ if (osd < map->max_osd)
+ map->osd_state[osd] &= ~CEPH_OSD_UP;
+ }
+
+ /* new_weight */
+ ceph_decode_32_safe(p, end, len, bad);
+ while (len--) {
+ u32 osd, off;
+ ceph_decode_need(p, end, sizeof(u32)*2, bad);
+ osd = ceph_decode_32(p);
+ off = ceph_decode_32(p);
+ pr_info("osd%d weight 0x%x %s\n", osd, off,
+ off == CEPH_OSD_IN ? "(in)" :
+ (off == CEPH_OSD_OUT ? "(out)" : ""));
+ if (osd < map->max_osd)
+ map->osd_weight[osd] = off;
+ }
+
+ /* new_pg_temp */
+ rbp = rb_first(&map->pg_temp);
+ ceph_decode_32_safe(p, end, len, bad);
+ while (len--) {
+ struct ceph_pg_mapping *pg;
+ int j;
+ struct ceph_pg pgid;
+ u32 pglen;
+ ceph_decode_need(p, end, sizeof(u64) + sizeof(u32), bad);
+ ceph_decode_copy(p, &pgid, sizeof(pgid));
+ pglen = ceph_decode_32(p);
+
+ /* remove any? */
+ while (rbp && pgid_cmp(rb_entry(rbp, struct ceph_pg_mapping,
+ node)->pgid, pgid) <= 0) {
+ struct rb_node *cur = rbp;
+ rbp = rb_next(rbp);
+ dout(" removed pg_temp %llx\n",
+ *(u64 *)&rb_entry(cur, struct ceph_pg_mapping,
+ node)->pgid);
+ rb_erase(cur, &map->pg_temp);
+ }
+
+ if (pglen) {
+ /* insert */
+ ceph_decode_need(p, end, pglen*sizeof(u32), bad);
+ pg = kmalloc(sizeof(*pg) + sizeof(u32)*pglen, GFP_NOFS);
+ if (!pg) {
+ err = -ENOMEM;
+ goto bad;
+ }
+ pg->pgid = pgid;
+ pg->len = pglen;
+ for (j = 0; j < pglen; j++)
+ pg->osds[j] = ceph_decode_32(p);
+ err = __insert_pg_mapping(pg, &map->pg_temp);
+ if (err)
+ goto bad;
+ dout(" added pg_temp %llx len %d\n", *(u64 *)&pgid,
+ pglen);
+ }
+ }
+ while (rbp) {
+ struct rb_node *cur = rbp;
+ rbp = rb_next(rbp);
+ dout(" removed pg_temp %llx\n",
+ *(u64 *)&rb_entry(cur, struct ceph_pg_mapping,
+ node)->pgid);
+ rb_erase(cur, &map->pg_temp);
+ }
+
+ /* ignore the rest */
+ *p = end;
+ return map;
+
+bad:
+ pr_err("corrupt inc osdmap epoch %d off %d (%p of %p-%p)\n",
+ epoch, (int)(*p - start), *p, start, end);
+ print_hex_dump(KERN_DEBUG, "osdmap: ",
+ DUMP_PREFIX_OFFSET, 16, 1,
+ start, end - start, true);
+ if (newcrush)
+ crush_destroy(newcrush);
+ return ERR_PTR(err);
+}
+
+
+
+
+/*
+ * calculate file layout from given offset, length.
+ * fill in correct oid, logical length, and object extent
+ * offset, length.
+ *
+ * for now, we write only a single su, until we can
+ * pass a stride back to the caller.
+ */
+void ceph_calc_file_object_mapping(struct ceph_file_layout *layout,
+ u64 off, u64 *plen,
+ u64 *ono,
+ u64 *oxoff, u64 *oxlen)
+{
+ u32 osize = le32_to_cpu(layout->fl_object_size);
+ u32 su = le32_to_cpu(layout->fl_stripe_unit);
+ u32 sc = le32_to_cpu(layout->fl_stripe_count);
+ u32 bl, stripeno, stripepos, objsetno;
+ u32 su_per_object;
+ u64 t, su_offset;
+
+ dout("mapping %llu~%llu osize %u fl_su %u\n", off, *plen,
+ osize, su);
+ su_per_object = osize / su;
+ dout("osize %u / su %u = su_per_object %u\n", osize, su,
+ su_per_object);
+
+ BUG_ON((su & ~PAGE_MASK) != 0);
+ /* bl = *off / su; */
+ t = off;
+ do_div(t, su);
+ bl = t;
+ dout("off %llu / su %u = bl %u\n", off, su, bl);
+
+ stripeno = bl / sc;
+ stripepos = bl % sc;
+ objsetno = stripeno / su_per_object;
+
+ *ono = objsetno * sc + stripepos;
+ dout("objset %u * sc %u = ono %u\n", objsetno, sc, (unsigned)*ono);
+
+ /* *oxoff = *off % layout->fl_stripe_unit; # offset in su */
+ t = off;
+ su_offset = do_div(t, su);
+ *oxoff = su_offset + (stripeno % su_per_object) * su;
+
+ /*
+ * Calculate the length of the extent being written to the selected
+ * object. This is the minimum of the full length requested (plen) or
+ * the remainder of the current stripe being written to.
+ */
+ *oxlen = min_t(u64, *plen, su - su_offset);
+ *plen = *oxlen;
+
+ dout(" obj extent %llu~%llu\n", *oxoff, *oxlen);
+}
+
+/*
+ * calculate an object layout (i.e. pgid) from an oid,
+ * file_layout, and osdmap
+ */
+int ceph_calc_object_layout(struct ceph_object_layout *ol,
+ const char *oid,
+ struct ceph_file_layout *fl,
+ struct ceph_osdmap *osdmap)
+{
+ unsigned num, num_mask;
+ struct ceph_pg pgid;
+ s32 preferred = (s32)le32_to_cpu(fl->fl_pg_preferred);
+ int poolid = le32_to_cpu(fl->fl_pg_pool);
+ struct ceph_pg_pool_info *pool;
+ unsigned ps;
+
+ BUG_ON(!osdmap);
+
+ pool = __lookup_pg_pool(&osdmap->pg_pools, poolid);
+ if (!pool)
+ return -EIO;
+ ps = ceph_str_hash(pool->v.object_hash, oid, strlen(oid));
+ if (preferred >= 0) {
+ ps += preferred;
+ num = le32_to_cpu(pool->v.lpg_num);
+ num_mask = pool->lpg_num_mask;
+ } else {
+ num = le32_to_cpu(pool->v.pg_num);
+ num_mask = pool->pg_num_mask;
+ }
+
+ pgid.ps = cpu_to_le16(ps);
+ pgid.preferred = cpu_to_le16(preferred);
+ pgid.pool = fl->fl_pg_pool;
+ if (preferred >= 0)
+ dout("calc_object_layout '%s' pgid %d.%xp%d\n", oid, poolid, ps,
+ (int)preferred);
+ else
+ dout("calc_object_layout '%s' pgid %d.%x\n", oid, poolid, ps);
+
+ ol->ol_pgid = pgid;
+ ol->ol_stripe_unit = fl->fl_object_stripe_unit;
+ return 0;
+}
+
+/*
+ * Calculate raw osd vector for the given pgid. Return pointer to osd
+ * array, or NULL on failure.
+ */
+static int *calc_pg_raw(struct ceph_osdmap *osdmap, struct ceph_pg pgid,
+ int *osds, int *num)
+{
+ struct ceph_pg_mapping *pg;
+ struct ceph_pg_pool_info *pool;
+ int ruleno;
+ unsigned poolid, ps, pps;
+ int preferred;
+
+ /* pg_temp? */
+ pg = __lookup_pg_mapping(&osdmap->pg_temp, pgid);
+ if (pg) {
+ *num = pg->len;
+ return pg->osds;
+ }
+
+ /* crush */
+ poolid = le32_to_cpu(pgid.pool);
+ ps = le16_to_cpu(pgid.ps);
+ preferred = (s16)le16_to_cpu(pgid.preferred);
+
+ /* don't forcefeed bad device ids to crush */
+ if (preferred >= osdmap->max_osd ||
+ preferred >= osdmap->crush->max_devices)
+ preferred = -1;
+
+ pool = __lookup_pg_pool(&osdmap->pg_pools, poolid);
+ if (!pool)
+ return NULL;
+ ruleno = crush_find_rule(osdmap->crush, pool->v.crush_ruleset,
+ pool->v.type, pool->v.size);
+ if (ruleno < 0) {
+ pr_err("no crush rule pool %d type %d size %d\n",
+ poolid, pool->v.type, pool->v.size);
+ return NULL;
+ }
+
+ if (preferred >= 0)
+ pps = ceph_stable_mod(ps,
+ le32_to_cpu(pool->v.lpgp_num),
+ pool->lpgp_num_mask);
+ else
+ pps = ceph_stable_mod(ps,
+ le32_to_cpu(pool->v.pgp_num),
+ pool->pgp_num_mask);
+ pps += poolid;
+ *num = crush_do_rule(osdmap->crush, ruleno, pps, osds,
+ min_t(int, pool->v.size, *num),
+ preferred, osdmap->osd_weight);
+ return osds;
+}
+
+/*
+ * Return primary osd for given pgid, or -1 if none.
+ */
+int ceph_calc_pg_primary(struct ceph_osdmap *osdmap, struct ceph_pg pgid)
+{
+ int rawosds[10], *osds;
+ int i, num = ARRAY_SIZE(rawosds);
+
+ osds = calc_pg_raw(osdmap, pgid, rawosds, &num);
+ if (!osds)
+ return -1;
+
+ /* primary is first up osd */
+ for (i = 0; i < num; i++)
+ if (ceph_osd_is_up(osdmap, osds[i])) {
+ return osds[i];
+ break;
+ }
+ return -1;
+}
--- /dev/null
+#ifndef _FS_CEPH_OSDMAP_H
+#define _FS_CEPH_OSDMAP_H
+
+#include <linux/rbtree.h>
+#include "types.h"
+#include "ceph_fs.h"
+#include "crush/crush.h"
+
+/*
+ * The osd map describes the current membership of the osd cluster and
+ * specifies the mapping of objects to placement groups and placement
+ * groups to (sets of) osds. That is, it completely specifies the
+ * (desired) distribution of all data objects in the system at some
+ * point in time.
+ *
+ * Each map version is identified by an epoch, which increases monotonically.
+ *
+ * The map can be updated either via an incremental map (diff) describing
+ * the change between two successive epochs, or as a fully encoded map.
+ */
+struct ceph_pg_pool_info {
+ struct rb_node node;
+ int id;
+ struct ceph_pg_pool v;
+ int pg_num_mask, pgp_num_mask, lpg_num_mask, lpgp_num_mask;
+};
+
+struct ceph_pg_mapping {
+ struct rb_node node;
+ struct ceph_pg pgid;
+ int len;
+ int osds[];
+};
+
+struct ceph_osdmap {
+ struct ceph_fsid fsid;
+ u32 epoch;
+ u32 mkfs_epoch;
+ struct ceph_timespec created, modified;
+
+ u32 flags; /* CEPH_OSDMAP_* */
+
+ u32 max_osd; /* size of osd_state, _offload, _addr arrays */
+ u8 *osd_state; /* CEPH_OSD_* */
+ u32 *osd_weight; /* 0 = failed, 0x10000 = 100% normal */
+ struct ceph_entity_addr *osd_addr;
+
+ struct rb_root pg_temp;
+ struct rb_root pg_pools;
+ u32 pool_max;
+
+ /* the CRUSH map specifies the mapping of placement groups to
+ * the list of osds that store+replicate them. */
+ struct crush_map *crush;
+};
+
+/*
+ * file layout helpers
+ */
+#define ceph_file_layout_su(l) ((__s32)le32_to_cpu((l).fl_stripe_unit))
+#define ceph_file_layout_stripe_count(l) \
+ ((__s32)le32_to_cpu((l).fl_stripe_count))
+#define ceph_file_layout_object_size(l) ((__s32)le32_to_cpu((l).fl_object_size))
+#define ceph_file_layout_cas_hash(l) ((__s32)le32_to_cpu((l).fl_cas_hash))
+#define ceph_file_layout_object_su(l) \
+ ((__s32)le32_to_cpu((l).fl_object_stripe_unit))
+#define ceph_file_layout_pg_preferred(l) \
+ ((__s32)le32_to_cpu((l).fl_pg_preferred))
+#define ceph_file_layout_pg_pool(l) \
+ ((__s32)le32_to_cpu((l).fl_pg_pool))
+
+static inline unsigned ceph_file_layout_stripe_width(struct ceph_file_layout *l)
+{
+ return le32_to_cpu(l->fl_stripe_unit) *
+ le32_to_cpu(l->fl_stripe_count);
+}
+
+/* "period" == bytes before i start on a new set of objects */
+static inline unsigned ceph_file_layout_period(struct ceph_file_layout *l)
+{
+ return le32_to_cpu(l->fl_object_size) *
+ le32_to_cpu(l->fl_stripe_count);
+}
+
+
+static inline int ceph_osd_is_up(struct ceph_osdmap *map, int osd)
+{
+ return (osd < map->max_osd) && (map->osd_state[osd] & CEPH_OSD_UP);
+}
+
+static inline bool ceph_osdmap_flag(struct ceph_osdmap *map, int flag)
+{
+ return map && (map->flags & flag);
+}
+
+extern char *ceph_osdmap_state_str(char *str, int len, int state);
+
+static inline struct ceph_entity_addr *ceph_osd_addr(struct ceph_osdmap *map,
+ int osd)
+{
+ if (osd >= map->max_osd)
+ return NULL;
+ return &map->osd_addr[osd];
+}
+
+extern struct ceph_osdmap *osdmap_decode(void **p, void *end);
+extern struct ceph_osdmap *osdmap_apply_incremental(void **p, void *end,
+ struct ceph_osdmap *map,
+ struct ceph_messenger *msgr);
+extern void ceph_osdmap_destroy(struct ceph_osdmap *map);
+
+/* calculate mapping of a file extent to an object */
+extern void ceph_calc_file_object_mapping(struct ceph_file_layout *layout,
+ u64 off, u64 *plen,
+ u64 *bno, u64 *oxoff, u64 *oxlen);
+
+/* calculate mapping of object to a placement group */
+extern int ceph_calc_object_layout(struct ceph_object_layout *ol,
+ const char *oid,
+ struct ceph_file_layout *fl,
+ struct ceph_osdmap *osdmap);
+extern int ceph_calc_pg_primary(struct ceph_osdmap *osdmap,
+ struct ceph_pg pgid);
+
+#endif
--- /dev/null
+
+#include <linux/gfp.h>
+#include <linux/pagemap.h>
+#include <linux/highmem.h>
+
+#include "pagelist.h"
+
+int ceph_pagelist_release(struct ceph_pagelist *pl)
+{
+ if (pl->mapped_tail)
+ kunmap(pl->mapped_tail);
+ while (!list_empty(&pl->head)) {
+ struct page *page = list_first_entry(&pl->head, struct page,
+ lru);
+ list_del(&page->lru);
+ __free_page(page);
+ }
+ return 0;
+}
+
+static int ceph_pagelist_addpage(struct ceph_pagelist *pl)
+{
+ struct page *page = alloc_page(GFP_NOFS);
+ if (!page)
+ return -ENOMEM;
+ pl->room += PAGE_SIZE;
+ list_add_tail(&page->lru, &pl->head);
+ if (pl->mapped_tail)
+ kunmap(pl->mapped_tail);
+ pl->mapped_tail = kmap(page);
+ return 0;
+}
+
+int ceph_pagelist_append(struct ceph_pagelist *pl, void *buf, size_t len)
+{
+ while (pl->room < len) {
+ size_t bit = pl->room;
+ int ret;
+
+ memcpy(pl->mapped_tail + (pl->length & ~PAGE_CACHE_MASK),
+ buf, bit);
+ pl->length += bit;
+ pl->room -= bit;
+ buf += bit;
+ len -= bit;
+ ret = ceph_pagelist_addpage(pl);
+ if (ret)
+ return ret;
+ }
+
+ memcpy(pl->mapped_tail + (pl->length & ~PAGE_CACHE_MASK), buf, len);
+ pl->length += len;
+ pl->room -= len;
+ return 0;
+}
--- /dev/null
+#ifndef __FS_CEPH_PAGELIST_H
+#define __FS_CEPH_PAGELIST_H
+
+#include <linux/list.h>
+
+struct ceph_pagelist {
+ struct list_head head;
+ void *mapped_tail;
+ size_t length;
+ size_t room;
+};
+
+static inline void ceph_pagelist_init(struct ceph_pagelist *pl)
+{
+ INIT_LIST_HEAD(&pl->head);
+ pl->mapped_tail = NULL;
+ pl->length = 0;
+ pl->room = 0;
+}
+extern int ceph_pagelist_release(struct ceph_pagelist *pl);
+
+extern int ceph_pagelist_append(struct ceph_pagelist *pl, void *d, size_t l);
+
+static inline int ceph_pagelist_encode_64(struct ceph_pagelist *pl, u64 v)
+{
+ __le64 ev = cpu_to_le64(v);
+ return ceph_pagelist_append(pl, &ev, sizeof(ev));
+}
+static inline int ceph_pagelist_encode_32(struct ceph_pagelist *pl, u32 v)
+{
+ __le32 ev = cpu_to_le32(v);
+ return ceph_pagelist_append(pl, &ev, sizeof(ev));
+}
+static inline int ceph_pagelist_encode_16(struct ceph_pagelist *pl, u16 v)
+{
+ __le16 ev = cpu_to_le16(v);
+ return ceph_pagelist_append(pl, &ev, sizeof(ev));
+}
+static inline int ceph_pagelist_encode_8(struct ceph_pagelist *pl, u8 v)
+{
+ return ceph_pagelist_append(pl, &v, 1);
+}
+static inline int ceph_pagelist_encode_string(struct ceph_pagelist *pl,
+ char *s, size_t len)
+{
+ int ret = ceph_pagelist_encode_32(pl, len);
+ if (ret)
+ return ret;
+ if (len)
+ return ceph_pagelist_append(pl, s, len);
+ return 0;
+}
+
+#endif
--- /dev/null
+#ifndef __RADOS_H
+#define __RADOS_H
+
+/*
+ * Data types for the Ceph distributed object storage layer RADOS
+ * (Reliable Autonomic Distributed Object Store).
+ */
+
+#include "msgr.h"
+
+/*
+ * osdmap encoding versions
+ */
+#define CEPH_OSDMAP_INC_VERSION 4
+#define CEPH_OSDMAP_VERSION 4
+
+/*
+ * fs id
+ */
+struct ceph_fsid {
+ unsigned char fsid[16];
+};
+
+static inline int ceph_fsid_compare(const struct ceph_fsid *a,
+ const struct ceph_fsid *b)
+{
+ return memcmp(a, b, sizeof(*a));
+}
+
+/*
+ * ino, object, etc.
+ */
+typedef __le64 ceph_snapid_t;
+#define CEPH_SNAPDIR ((__u64)(-1)) /* reserved for hidden .snap dir */
+#define CEPH_NOSNAP ((__u64)(-2)) /* "head", "live" revision */
+#define CEPH_MAXSNAP ((__u64)(-3)) /* largest valid snapid */
+
+struct ceph_timespec {
+ __le32 tv_sec;
+ __le32 tv_nsec;
+} __attribute__ ((packed));
+
+
+/*
+ * object layout - how objects are mapped into PGs
+ */
+#define CEPH_OBJECT_LAYOUT_HASH 1
+#define CEPH_OBJECT_LAYOUT_LINEAR 2
+#define CEPH_OBJECT_LAYOUT_HASHINO 3
+
+/*
+ * pg layout -- how PGs are mapped onto (sets of) OSDs
+ */
+#define CEPH_PG_LAYOUT_CRUSH 0
+#define CEPH_PG_LAYOUT_HASH 1
+#define CEPH_PG_LAYOUT_LINEAR 2
+#define CEPH_PG_LAYOUT_HYBRID 3
+
+
+/*
+ * placement group.
+ * we encode this into one __le64.
+ */
+struct ceph_pg {
+ __le16 preferred; /* preferred primary osd */
+ __le16 ps; /* placement seed */
+ __le32 pool; /* object pool */
+} __attribute__ ((packed));
+
+/*
+ * pg_pool is a set of pgs storing a pool of objects
+ *
+ * pg_num -- base number of pseudorandomly placed pgs
+ *
+ * pgp_num -- effective number when calculating pg placement. this
+ * is used for pg_num increases. new pgs result in data being "split"
+ * into new pgs. for this to proceed smoothly, new pgs are intiially
+ * colocated with their parents; that is, pgp_num doesn't increase
+ * until the new pgs have successfully split. only _then_ are the new
+ * pgs placed independently.
+ *
+ * lpg_num -- localized pg count (per device). replicas are randomly
+ * selected.
+ *
+ * lpgp_num -- as above.
+ */
+#define CEPH_PG_TYPE_REP 1
+#define CEPH_PG_TYPE_RAID4 2
+#define CEPH_PG_POOL_VERSION 2
+struct ceph_pg_pool {
+ __u8 type; /* CEPH_PG_TYPE_* */
+ __u8 size; /* number of osds in each pg */
+ __u8 crush_ruleset; /* crush placement rule */
+ __u8 object_hash; /* hash mapping object name to ps */
+ __le32 pg_num, pgp_num; /* number of pg's */
+ __le32 lpg_num, lpgp_num; /* number of localized pg's */
+ __le32 last_change; /* most recent epoch changed */
+ __le64 snap_seq; /* seq for per-pool snapshot */
+ __le32 snap_epoch; /* epoch of last snap */
+ __le32 num_snaps;
+ __le32 num_removed_snap_intervals;
+ __le64 uid;
+} __attribute__ ((packed));
+
+/*
+ * stable_mod func is used to control number of placement groups.
+ * similar to straight-up modulo, but produces a stable mapping as b
+ * increases over time. b is the number of bins, and bmask is the
+ * containing power of 2 minus 1.
+ *
+ * b <= bmask and bmask=(2**n)-1
+ * e.g., b=12 -> bmask=15, b=123 -> bmask=127
+ */
+static inline int ceph_stable_mod(int x, int b, int bmask)
+{
+ if ((x & bmask) < b)
+ return x & bmask;
+ else
+ return x & (bmask >> 1);
+}
+
+/*
+ * object layout - how a given object should be stored.
+ */
+struct ceph_object_layout {
+ struct ceph_pg ol_pgid; /* raw pg, with _full_ ps precision. */
+ __le32 ol_stripe_unit; /* for per-object parity, if any */
+} __attribute__ ((packed));
+
+/*
+ * compound epoch+version, used by storage layer to serialize mutations
+ */
+struct ceph_eversion {
+ __le32 epoch;
+ __le64 version;
+} __attribute__ ((packed));
+
+/*
+ * osd map bits
+ */
+
+/* status bits */
+#define CEPH_OSD_EXISTS 1
+#define CEPH_OSD_UP 2
+
+/* osd weights. fixed point value: 0x10000 == 1.0 ("in"), 0 == "out" */
+#define CEPH_OSD_IN 0x10000
+#define CEPH_OSD_OUT 0
+
+
+/*
+ * osd map flag bits
+ */
+#define CEPH_OSDMAP_NEARFULL (1<<0) /* sync writes (near ENOSPC) */
+#define CEPH_OSDMAP_FULL (1<<1) /* no data writes (ENOSPC) */
+#define CEPH_OSDMAP_PAUSERD (1<<2) /* pause all reads */
+#define CEPH_OSDMAP_PAUSEWR (1<<3) /* pause all writes */
+#define CEPH_OSDMAP_PAUSEREC (1<<4) /* pause recovery */
+
+/*
+ * osd ops
+ */
+#define CEPH_OSD_OP_MODE 0xf000
+#define CEPH_OSD_OP_MODE_RD 0x1000
+#define CEPH_OSD_OP_MODE_WR 0x2000
+#define CEPH_OSD_OP_MODE_RMW 0x3000
+#define CEPH_OSD_OP_MODE_SUB 0x4000
+
+#define CEPH_OSD_OP_TYPE 0x0f00
+#define CEPH_OSD_OP_TYPE_LOCK 0x0100
+#define CEPH_OSD_OP_TYPE_DATA 0x0200
+#define CEPH_OSD_OP_TYPE_ATTR 0x0300
+#define CEPH_OSD_OP_TYPE_EXEC 0x0400
+#define CEPH_OSD_OP_TYPE_PG 0x0500
+
+enum {
+ /** data **/
+ /* read */
+ CEPH_OSD_OP_READ = CEPH_OSD_OP_MODE_RD | CEPH_OSD_OP_TYPE_DATA | 1,
+ CEPH_OSD_OP_STAT = CEPH_OSD_OP_MODE_RD | CEPH_OSD_OP_TYPE_DATA | 2,
+
+ /* fancy read */
+ CEPH_OSD_OP_MASKTRUNC = CEPH_OSD_OP_MODE_RD | CEPH_OSD_OP_TYPE_DATA | 4,
+
+ /* write */
+ CEPH_OSD_OP_WRITE = CEPH_OSD_OP_MODE_WR | CEPH_OSD_OP_TYPE_DATA | 1,
+ CEPH_OSD_OP_WRITEFULL = CEPH_OSD_OP_MODE_WR | CEPH_OSD_OP_TYPE_DATA | 2,
+ CEPH_OSD_OP_TRUNCATE = CEPH_OSD_OP_MODE_WR | CEPH_OSD_OP_TYPE_DATA | 3,
+ CEPH_OSD_OP_ZERO = CEPH_OSD_OP_MODE_WR | CEPH_OSD_OP_TYPE_DATA | 4,
+ CEPH_OSD_OP_DELETE = CEPH_OSD_OP_MODE_WR | CEPH_OSD_OP_TYPE_DATA | 5,
+
+ /* fancy write */
+ CEPH_OSD_OP_APPEND = CEPH_OSD_OP_MODE_WR | CEPH_OSD_OP_TYPE_DATA | 6,
+ CEPH_OSD_OP_STARTSYNC = CEPH_OSD_OP_MODE_WR | CEPH_OSD_OP_TYPE_DATA | 7,
+ CEPH_OSD_OP_SETTRUNC = CEPH_OSD_OP_MODE_WR | CEPH_OSD_OP_TYPE_DATA | 8,
+ CEPH_OSD_OP_TRIMTRUNC = CEPH_OSD_OP_MODE_WR | CEPH_OSD_OP_TYPE_DATA | 9,
+
+ CEPH_OSD_OP_TMAPUP = CEPH_OSD_OP_MODE_RMW | CEPH_OSD_OP_TYPE_DATA | 10,
+ CEPH_OSD_OP_TMAPPUT = CEPH_OSD_OP_MODE_WR | CEPH_OSD_OP_TYPE_DATA | 11,
+ CEPH_OSD_OP_TMAPGET = CEPH_OSD_OP_MODE_RD | CEPH_OSD_OP_TYPE_DATA | 12,
+
+ CEPH_OSD_OP_CREATE = CEPH_OSD_OP_MODE_WR | CEPH_OSD_OP_TYPE_DATA | 13,
+
+ /** attrs **/
+ /* read */
+ CEPH_OSD_OP_GETXATTR = CEPH_OSD_OP_MODE_RD | CEPH_OSD_OP_TYPE_ATTR | 1,
+ CEPH_OSD_OP_GETXATTRS = CEPH_OSD_OP_MODE_RD | CEPH_OSD_OP_TYPE_ATTR | 2,
+
+ /* write */
+ CEPH_OSD_OP_SETXATTR = CEPH_OSD_OP_MODE_WR | CEPH_OSD_OP_TYPE_ATTR | 1,
+ CEPH_OSD_OP_SETXATTRS = CEPH_OSD_OP_MODE_WR | CEPH_OSD_OP_TYPE_ATTR | 2,
+ CEPH_OSD_OP_RESETXATTRS = CEPH_OSD_OP_MODE_WR|CEPH_OSD_OP_TYPE_ATTR | 3,
+ CEPH_OSD_OP_RMXATTR = CEPH_OSD_OP_MODE_WR | CEPH_OSD_OP_TYPE_ATTR | 4,
+
+ /** subop **/
+ CEPH_OSD_OP_PULL = CEPH_OSD_OP_MODE_SUB | 1,
+ CEPH_OSD_OP_PUSH = CEPH_OSD_OP_MODE_SUB | 2,
+ CEPH_OSD_OP_BALANCEREADS = CEPH_OSD_OP_MODE_SUB | 3,
+ CEPH_OSD_OP_UNBALANCEREADS = CEPH_OSD_OP_MODE_SUB | 4,
+ CEPH_OSD_OP_SCRUB = CEPH_OSD_OP_MODE_SUB | 5,
+
+ /** lock **/
+ CEPH_OSD_OP_WRLOCK = CEPH_OSD_OP_MODE_WR | CEPH_OSD_OP_TYPE_LOCK | 1,
+ CEPH_OSD_OP_WRUNLOCK = CEPH_OSD_OP_MODE_WR | CEPH_OSD_OP_TYPE_LOCK | 2,
+ CEPH_OSD_OP_RDLOCK = CEPH_OSD_OP_MODE_WR | CEPH_OSD_OP_TYPE_LOCK | 3,
+ CEPH_OSD_OP_RDUNLOCK = CEPH_OSD_OP_MODE_WR | CEPH_OSD_OP_TYPE_LOCK | 4,
+ CEPH_OSD_OP_UPLOCK = CEPH_OSD_OP_MODE_WR | CEPH_OSD_OP_TYPE_LOCK | 5,
+ CEPH_OSD_OP_DNLOCK = CEPH_OSD_OP_MODE_WR | CEPH_OSD_OP_TYPE_LOCK | 6,
+
+ /** exec **/
+ CEPH_OSD_OP_CALL = CEPH_OSD_OP_MODE_RD | CEPH_OSD_OP_TYPE_EXEC | 1,
+
+ /** pg **/
+ CEPH_OSD_OP_PGLS = CEPH_OSD_OP_MODE_RD | CEPH_OSD_OP_TYPE_PG | 1,
+};
+
+static inline int ceph_osd_op_type_lock(int op)
+{
+ return (op & CEPH_OSD_OP_TYPE) == CEPH_OSD_OP_TYPE_LOCK;
+}
+static inline int ceph_osd_op_type_data(int op)
+{
+ return (op & CEPH_OSD_OP_TYPE) == CEPH_OSD_OP_TYPE_DATA;
+}
+static inline int ceph_osd_op_type_attr(int op)
+{
+ return (op & CEPH_OSD_OP_TYPE) == CEPH_OSD_OP_TYPE_ATTR;
+}
+static inline int ceph_osd_op_type_exec(int op)
+{
+ return (op & CEPH_OSD_OP_TYPE) == CEPH_OSD_OP_TYPE_EXEC;
+}
+static inline int ceph_osd_op_type_pg(int op)
+{
+ return (op & CEPH_OSD_OP_TYPE) == CEPH_OSD_OP_TYPE_PG;
+}
+
+static inline int ceph_osd_op_mode_subop(int op)
+{
+ return (op & CEPH_OSD_OP_MODE) == CEPH_OSD_OP_MODE_SUB;
+}
+static inline int ceph_osd_op_mode_read(int op)
+{
+ return (op & CEPH_OSD_OP_MODE) == CEPH_OSD_OP_MODE_RD;
+}
+static inline int ceph_osd_op_mode_modify(int op)
+{
+ return (op & CEPH_OSD_OP_MODE) == CEPH_OSD_OP_MODE_WR;
+}
+
+#define CEPH_OSD_TMAP_HDR 'h'
+#define CEPH_OSD_TMAP_SET 's'
+#define CEPH_OSD_TMAP_RM 'r'
+
+extern const char *ceph_osd_op_name(int op);
+
+
+/*
+ * osd op flags
+ *
+ * An op may be READ, WRITE, or READ|WRITE.
+ */
+enum {
+ CEPH_OSD_FLAG_ACK = 1, /* want (or is) "ack" ack */
+ CEPH_OSD_FLAG_ONNVRAM = 2, /* want (or is) "onnvram" ack */
+ CEPH_OSD_FLAG_ONDISK = 4, /* want (or is) "ondisk" ack */
+ CEPH_OSD_FLAG_RETRY = 8, /* resend attempt */
+ CEPH_OSD_FLAG_READ = 16, /* op may read */
+ CEPH_OSD_FLAG_WRITE = 32, /* op may write */
+ CEPH_OSD_FLAG_ORDERSNAP = 64, /* EOLDSNAP if snapc is out of order */
+ CEPH_OSD_FLAG_PEERSTAT = 128, /* msg includes osd_peer_stat */
+ CEPH_OSD_FLAG_BALANCE_READS = 256,
+ CEPH_OSD_FLAG_PARALLELEXEC = 512, /* execute op in parallel */
+ CEPH_OSD_FLAG_PGOP = 1024, /* pg op, no object */
+ CEPH_OSD_FLAG_EXEC = 2048, /* op may exec */
+};
+
+enum {
+ CEPH_OSD_OP_FLAG_EXCL = 1, /* EXCL object create */
+};
+
+#define EOLDSNAPC ERESTART /* ORDERSNAP flag set; writer has old snapc*/
+#define EBLACKLISTED ESHUTDOWN /* blacklisted */
+
+/*
+ * an individual object operation. each may be accompanied by some data
+ * payload
+ */
+struct ceph_osd_op {
+ __le16 op; /* CEPH_OSD_OP_* */
+ __le32 flags; /* CEPH_OSD_FLAG_* */
+ union {
+ struct {
+ __le64 offset, length;
+ __le64 truncate_size;
+ __le32 truncate_seq;
+ } __attribute__ ((packed)) extent;
+ struct {
+ __le32 name_len;
+ __le32 value_len;
+ } __attribute__ ((packed)) xattr;
+ struct {
+ __u8 class_len;
+ __u8 method_len;
+ __u8 argc;
+ __le32 indata_len;
+ } __attribute__ ((packed)) cls;
+ struct {
+ __le64 cookie, count;
+ } __attribute__ ((packed)) pgls;
+ };
+ __le32 payload_len;
+} __attribute__ ((packed));
+
+/*
+ * osd request message header. each request may include multiple
+ * ceph_osd_op object operations.
+ */
+struct ceph_osd_request_head {
+ __le32 client_inc; /* client incarnation */
+ struct ceph_object_layout layout; /* pgid */
+ __le32 osdmap_epoch; /* client's osdmap epoch */
+
+ __le32 flags;
+
+ struct ceph_timespec mtime; /* for mutations only */
+ struct ceph_eversion reassert_version; /* if we are replaying op */
+
+ __le32 object_len; /* length of object name */
+
+ __le64 snapid; /* snapid to read */
+ __le64 snap_seq; /* writer's snap context */
+ __le32 num_snaps;
+
+ __le16 num_ops;
+ struct ceph_osd_op ops[]; /* followed by ops[], obj, ticket, snaps */
+} __attribute__ ((packed));
+
+struct ceph_osd_reply_head {
+ __le32 client_inc; /* client incarnation */
+ __le32 flags;
+ struct ceph_object_layout layout;
+ __le32 osdmap_epoch;
+ struct ceph_eversion reassert_version; /* for replaying uncommitted */
+
+ __le32 result; /* result code */
+
+ __le32 object_len; /* length of object name */
+ __le32 num_ops;
+ struct ceph_osd_op ops[0]; /* ops[], object */
+} __attribute__ ((packed));
+
+
+#endif
--- /dev/null
+#include "ceph_debug.h"
+
+#include <linux/sort.h>
+#include <linux/slab.h>
+
+#include "super.h"
+#include "decode.h"
+
+/*
+ * Snapshots in ceph are driven in large part by cooperation from the
+ * client. In contrast to local file systems or file servers that
+ * implement snapshots at a single point in the system, ceph's
+ * distributed access to storage requires clients to help decide
+ * whether a write logically occurs before or after a recently created
+ * snapshot.
+ *
+ * This provides a perfect instantanous client-wide snapshot. Between
+ * clients, however, snapshots may appear to be applied at slightly
+ * different points in time, depending on delays in delivering the
+ * snapshot notification.
+ *
+ * Snapshots are _not_ file system-wide. Instead, each snapshot
+ * applies to the subdirectory nested beneath some directory. This
+ * effectively divides the hierarchy into multiple "realms," where all
+ * of the files contained by each realm share the same set of
+ * snapshots. An individual realm's snap set contains snapshots
+ * explicitly created on that realm, as well as any snaps in its
+ * parent's snap set _after_ the point at which the parent became it's
+ * parent (due to, say, a rename). Similarly, snaps from prior parents
+ * during the time intervals during which they were the parent are included.
+ *
+ * The client is spared most of this detail, fortunately... it must only
+ * maintains a hierarchy of realms reflecting the current parent/child
+ * realm relationship, and for each realm has an explicit list of snaps
+ * inherited from prior parents.
+ *
+ * A snap_realm struct is maintained for realms containing every inode
+ * with an open cap in the system. (The needed snap realm information is
+ * provided by the MDS whenever a cap is issued, i.e., on open.) A 'seq'
+ * version number is used to ensure that as realm parameters change (new
+ * snapshot, new parent, etc.) the client's realm hierarchy is updated.
+ *
+ * The realm hierarchy drives the generation of a 'snap context' for each
+ * realm, which simply lists the resulting set of snaps for the realm. This
+ * is attached to any writes sent to OSDs.
+ */
+/*
+ * Unfortunately error handling is a bit mixed here. If we get a snap
+ * update, but don't have enough memory to update our realm hierarchy,
+ * it's not clear what we can do about it (besides complaining to the
+ * console).
+ */
+
+
+/*
+ * increase ref count for the realm
+ *
+ * caller must hold snap_rwsem for write.
+ */
+void ceph_get_snap_realm(struct ceph_mds_client *mdsc,
+ struct ceph_snap_realm *realm)
+{
+ dout("get_realm %p %d -> %d\n", realm,
+ atomic_read(&realm->nref), atomic_read(&realm->nref)+1);
+ /*
+ * since we _only_ increment realm refs or empty the empty
+ * list with snap_rwsem held, adjusting the empty list here is
+ * safe. we do need to protect against concurrent empty list
+ * additions, however.
+ */
+ if (atomic_read(&realm->nref) == 0) {
+ spin_lock(&mdsc->snap_empty_lock);
+ list_del_init(&realm->empty_item);
+ spin_unlock(&mdsc->snap_empty_lock);
+ }
+
+ atomic_inc(&realm->nref);
+}
+
+static void __insert_snap_realm(struct rb_root *root,
+ struct ceph_snap_realm *new)
+{
+ struct rb_node **p = &root->rb_node;
+ struct rb_node *parent = NULL;
+ struct ceph_snap_realm *r = NULL;
+
+ while (*p) {
+ parent = *p;
+ r = rb_entry(parent, struct ceph_snap_realm, node);
+ if (new->ino < r->ino)
+ p = &(*p)->rb_left;
+ else if (new->ino > r->ino)
+ p = &(*p)->rb_right;
+ else
+ BUG();
+ }
+
+ rb_link_node(&new->node, parent, p);
+ rb_insert_color(&new->node, root);
+}
+
+/*
+ * create and get the realm rooted at @ino and bump its ref count.
+ *
+ * caller must hold snap_rwsem for write.
+ */
+static struct ceph_snap_realm *ceph_create_snap_realm(
+ struct ceph_mds_client *mdsc,
+ u64 ino)
+{
+ struct ceph_snap_realm *realm;
+
+ realm = kzalloc(sizeof(*realm), GFP_NOFS);
+ if (!realm)
+ return ERR_PTR(-ENOMEM);
+
+ atomic_set(&realm->nref, 0); /* tree does not take a ref */
+ realm->ino = ino;
+ INIT_LIST_HEAD(&realm->children);
+ INIT_LIST_HEAD(&realm->child_item);
+ INIT_LIST_HEAD(&realm->empty_item);
+ INIT_LIST_HEAD(&realm->inodes_with_caps);
+ spin_lock_init(&realm->inodes_with_caps_lock);
+ __insert_snap_realm(&mdsc->snap_realms, realm);
+ dout("create_snap_realm %llx %p\n", realm->ino, realm);
+ return realm;
+}
+
+/*
+ * lookup the realm rooted at @ino.
+ *
+ * caller must hold snap_rwsem for write.
+ */
+struct ceph_snap_realm *ceph_lookup_snap_realm(struct ceph_mds_client *mdsc,
+ u64 ino)
+{
+ struct rb_node *n = mdsc->snap_realms.rb_node;
+ struct ceph_snap_realm *r;
+
+ while (n) {
+ r = rb_entry(n, struct ceph_snap_realm, node);
+ if (ino < r->ino)
+ n = n->rb_left;
+ else if (ino > r->ino)
+ n = n->rb_right;
+ else {
+ dout("lookup_snap_realm %llx %p\n", r->ino, r);
+ return r;
+ }
+ }
+ return NULL;
+}
+
+static void __put_snap_realm(struct ceph_mds_client *mdsc,
+ struct ceph_snap_realm *realm);
+
+/*
+ * called with snap_rwsem (write)
+ */
+static void __destroy_snap_realm(struct ceph_mds_client *mdsc,
+ struct ceph_snap_realm *realm)
+{
+ dout("__destroy_snap_realm %p %llx\n", realm, realm->ino);
+
+ rb_erase(&realm->node, &mdsc->snap_realms);
+
+ if (realm->parent) {
+ list_del_init(&realm->child_item);
+ __put_snap_realm(mdsc, realm->parent);
+ }
+
+ kfree(realm->prior_parent_snaps);
+ kfree(realm->snaps);
+ ceph_put_snap_context(realm->cached_context);
+ kfree(realm);
+}
+
+/*
+ * caller holds snap_rwsem (write)
+ */
+static void __put_snap_realm(struct ceph_mds_client *mdsc,
+ struct ceph_snap_realm *realm)
+{
+ dout("__put_snap_realm %llx %p %d -> %d\n", realm->ino, realm,
+ atomic_read(&realm->nref), atomic_read(&realm->nref)-1);
+ if (atomic_dec_and_test(&realm->nref))
+ __destroy_snap_realm(mdsc, realm);
+}
+
+/*
+ * caller needn't hold any locks
+ */
+void ceph_put_snap_realm(struct ceph_mds_client *mdsc,
+ struct ceph_snap_realm *realm)
+{
+ dout("put_snap_realm %llx %p %d -> %d\n", realm->ino, realm,
+ atomic_read(&realm->nref), atomic_read(&realm->nref)-1);
+ if (!atomic_dec_and_test(&realm->nref))
+ return;
+
+ if (down_write_trylock(&mdsc->snap_rwsem)) {
+ __destroy_snap_realm(mdsc, realm);
+ up_write(&mdsc->snap_rwsem);
+ } else {
+ spin_lock(&mdsc->snap_empty_lock);
+ list_add(&mdsc->snap_empty, &realm->empty_item);
+ spin_unlock(&mdsc->snap_empty_lock);
+ }
+}
+
+/*
+ * Clean up any realms whose ref counts have dropped to zero. Note
+ * that this does not include realms who were created but not yet
+ * used.
+ *
+ * Called under snap_rwsem (write)
+ */
+static void __cleanup_empty_realms(struct ceph_mds_client *mdsc)
+{
+ struct ceph_snap_realm *realm;
+
+ spin_lock(&mdsc->snap_empty_lock);
+ while (!list_empty(&mdsc->snap_empty)) {
+ realm = list_first_entry(&mdsc->snap_empty,
+ struct ceph_snap_realm, empty_item);
+ list_del(&realm->empty_item);
+ spin_unlock(&mdsc->snap_empty_lock);
+ __destroy_snap_realm(mdsc, realm);
+ spin_lock(&mdsc->snap_empty_lock);
+ }
+ spin_unlock(&mdsc->snap_empty_lock);
+}
+
+void ceph_cleanup_empty_realms(struct ceph_mds_client *mdsc)
+{
+ down_write(&mdsc->snap_rwsem);
+ __cleanup_empty_realms(mdsc);
+ up_write(&mdsc->snap_rwsem);
+}
+
+/*
+ * adjust the parent realm of a given @realm. adjust child list, and parent
+ * pointers, and ref counts appropriately.
+ *
+ * return true if parent was changed, 0 if unchanged, <0 on error.
+ *
+ * caller must hold snap_rwsem for write.
+ */
+static int adjust_snap_realm_parent(struct ceph_mds_client *mdsc,
+ struct ceph_snap_realm *realm,
+ u64 parentino)
+{
+ struct ceph_snap_realm *parent;
+
+ if (realm->parent_ino == parentino)
+ return 0;
+
+ parent = ceph_lookup_snap_realm(mdsc, parentino);
+ if (!parent) {
+ parent = ceph_create_snap_realm(mdsc, parentino);
+ if (IS_ERR(parent))
+ return PTR_ERR(parent);
+ }
+ dout("adjust_snap_realm_parent %llx %p: %llx %p -> %llx %p\n",
+ realm->ino, realm, realm->parent_ino, realm->parent,
+ parentino, parent);
+ if (realm->parent) {
+ list_del_init(&realm->child_item);
+ ceph_put_snap_realm(mdsc, realm->parent);
+ }
+ realm->parent_ino = parentino;
+ realm->parent = parent;
+ ceph_get_snap_realm(mdsc, parent);
+ list_add(&realm->child_item, &parent->children);
+ return 1;
+}
+
+
+static int cmpu64_rev(const void *a, const void *b)
+{
+ if (*(u64 *)a < *(u64 *)b)
+ return 1;
+ if (*(u64 *)a > *(u64 *)b)
+ return -1;
+ return 0;
+}
+
+/*
+ * build the snap context for a given realm.
+ */
+static int build_snap_context(struct ceph_snap_realm *realm)
+{
+ struct ceph_snap_realm *parent = realm->parent;
+ struct ceph_snap_context *snapc;
+ int err = 0;
+ int i;
+ int num = realm->num_prior_parent_snaps + realm->num_snaps;
+
+ /*
+ * build parent context, if it hasn't been built.
+ * conservatively estimate that all parent snaps might be
+ * included by us.
+ */
+ if (parent) {
+ if (!parent->cached_context) {
+ err = build_snap_context(parent);
+ if (err)
+ goto fail;
+ }
+ num += parent->cached_context->num_snaps;
+ }
+
+ /* do i actually need to update? not if my context seq
+ matches realm seq, and my parents' does to. (this works
+ because we rebuild_snap_realms() works _downward_ in
+ hierarchy after each update.) */
+ if (realm->cached_context &&
+ realm->cached_context->seq == realm->seq &&
+ (!parent ||
+ realm->cached_context->seq >= parent->cached_context->seq)) {
+ dout("build_snap_context %llx %p: %p seq %lld (%d snaps)"
+ " (unchanged)\n",
+ realm->ino, realm, realm->cached_context,
+ realm->cached_context->seq,
+ realm->cached_context->num_snaps);
+ return 0;
+ }
+
+ /* alloc new snap context */
+ err = -ENOMEM;
+ if (num > ULONG_MAX / sizeof(u64) - sizeof(*snapc))
+ goto fail;
+ snapc = kzalloc(sizeof(*snapc) + num*sizeof(u64), GFP_NOFS);
+ if (!snapc)
+ goto fail;
+ atomic_set(&snapc->nref, 1);
+
+ /* build (reverse sorted) snap vector */
+ num = 0;
+ snapc->seq = realm->seq;
+ if (parent) {
+ /* include any of parent's snaps occuring _after_ my
+ parent became my parent */
+ for (i = 0; i < parent->cached_context->num_snaps; i++)
+ if (parent->cached_context->snaps[i] >=
+ realm->parent_since)
+ snapc->snaps[num++] =
+ parent->cached_context->snaps[i];
+ if (parent->cached_context->seq > snapc->seq)
+ snapc->seq = parent->cached_context->seq;
+ }
+ memcpy(snapc->snaps + num, realm->snaps,
+ sizeof(u64)*realm->num_snaps);
+ num += realm->num_snaps;
+ memcpy(snapc->snaps + num, realm->prior_parent_snaps,
+ sizeof(u64)*realm->num_prior_parent_snaps);
+ num += realm->num_prior_parent_snaps;
+
+ sort(snapc->snaps, num, sizeof(u64), cmpu64_rev, NULL);
+ snapc->num_snaps = num;
+ dout("build_snap_context %llx %p: %p seq %lld (%d snaps)\n",
+ realm->ino, realm, snapc, snapc->seq, snapc->num_snaps);
+
+ if (realm->cached_context)
+ ceph_put_snap_context(realm->cached_context);
+ realm->cached_context = snapc;
+ return 0;
+
+fail:
+ /*
+ * if we fail, clear old (incorrect) cached_context... hopefully
+ * we'll have better luck building it later
+ */
+ if (realm->cached_context) {
+ ceph_put_snap_context(realm->cached_context);
+ realm->cached_context = NULL;
+ }
+ pr_err("build_snap_context %llx %p fail %d\n", realm->ino,
+ realm, err);
+ return err;
+}
+
+/*
+ * rebuild snap context for the given realm and all of its children.
+ */
+static void rebuild_snap_realms(struct ceph_snap_realm *realm)
+{
+ struct ceph_snap_realm *child;
+
+ dout("rebuild_snap_realms %llx %p\n", realm->ino, realm);
+ build_snap_context(realm);
+
+ list_for_each_entry(child, &realm->children, child_item)
+ rebuild_snap_realms(child);
+}
+
+
+/*
+ * helper to allocate and decode an array of snapids. free prior
+ * instance, if any.
+ */
+static int dup_array(u64 **dst, __le64 *src, int num)
+{
+ int i;
+
+ kfree(*dst);
+ if (num) {
+ *dst = kcalloc(num, sizeof(u64), GFP_NOFS);
+ if (!*dst)
+ return -ENOMEM;
+ for (i = 0; i < num; i++)
+ (*dst)[i] = get_unaligned_le64(src + i);
+ } else {
+ *dst = NULL;
+ }
+ return 0;
+}
+
+
+/*
+ * When a snapshot is applied, the size/mtime inode metadata is queued
+ * in a ceph_cap_snap (one for each snapshot) until writeback
+ * completes and the metadata can be flushed back to the MDS.
+ *
+ * However, if a (sync) write is currently in-progress when we apply
+ * the snapshot, we have to wait until the write succeeds or fails
+ * (and a final size/mtime is known). In this case the
+ * cap_snap->writing = 1, and is said to be "pending." When the write
+ * finishes, we __ceph_finish_cap_snap().
+ *
+ * Caller must hold snap_rwsem for read (i.e., the realm topology won't
+ * change).
+ */
+void ceph_queue_cap_snap(struct ceph_inode_info *ci,
+ struct ceph_snap_context *snapc)
+{
+ struct inode *inode = &ci->vfs_inode;
+ struct ceph_cap_snap *capsnap;
+ int used;
+
+ capsnap = kzalloc(sizeof(*capsnap), GFP_NOFS);
+ if (!capsnap) {
+ pr_err("ENOMEM allocating ceph_cap_snap on %p\n", inode);
+ return;
+ }
+
+ spin_lock(&inode->i_lock);
+ used = __ceph_caps_used(ci);
+ if (__ceph_have_pending_cap_snap(ci)) {
+ /* there is no point in queuing multiple "pending" cap_snaps,
+ as no new writes are allowed to start when pending, so any
+ writes in progress now were started before the previous
+ cap_snap. lucky us. */
+ dout("queue_cap_snap %p snapc %p seq %llu used %d"
+ " already pending\n", inode, snapc, snapc->seq, used);
+ kfree(capsnap);
+ } else if (ci->i_wrbuffer_ref_head || (used & CEPH_CAP_FILE_WR)) {
+ igrab(inode);
+
+ atomic_set(&capsnap->nref, 1);
+ capsnap->ci = ci;
+ INIT_LIST_HEAD(&capsnap->ci_item);
+ INIT_LIST_HEAD(&capsnap->flushing_item);
+
+ capsnap->follows = snapc->seq - 1;
+ capsnap->context = ceph_get_snap_context(snapc);
+ capsnap->issued = __ceph_caps_issued(ci, NULL);
+ capsnap->dirty = __ceph_caps_dirty(ci);
+
+ capsnap->mode = inode->i_mode;
+ capsnap->uid = inode->i_uid;
+ capsnap->gid = inode->i_gid;
+
+ /* fixme? */
+ capsnap->xattr_blob = NULL;
+ capsnap->xattr_len = 0;
+
+ /* dirty page count moved from _head to this cap_snap;
+ all subsequent writes page dirties occur _after_ this
+ snapshot. */
+ capsnap->dirty_pages = ci->i_wrbuffer_ref_head;
+ ci->i_wrbuffer_ref_head = 0;
+ ceph_put_snap_context(ci->i_head_snapc);
+ ci->i_head_snapc = NULL;
+ list_add_tail(&capsnap->ci_item, &ci->i_cap_snaps);
+
+ if (used & CEPH_CAP_FILE_WR) {
+ dout("queue_cap_snap %p cap_snap %p snapc %p"
+ " seq %llu used WR, now pending\n", inode,
+ capsnap, snapc, snapc->seq);
+ capsnap->writing = 1;
+ } else {
+ /* note mtime, size NOW. */
+ __ceph_finish_cap_snap(ci, capsnap);
+ }
+ } else {
+ dout("queue_cap_snap %p nothing dirty|writing\n", inode);
+ kfree(capsnap);
+ }
+
+ spin_unlock(&inode->i_lock);
+}
+
+/*
+ * Finalize the size, mtime for a cap_snap.. that is, settle on final values
+ * to be used for the snapshot, to be flushed back to the mds.
+ *
+ * If capsnap can now be flushed, add to snap_flush list, and return 1.
+ *
+ * Caller must hold i_lock.
+ */
+int __ceph_finish_cap_snap(struct ceph_inode_info *ci,
+ struct ceph_cap_snap *capsnap)
+{
+ struct inode *inode = &ci->vfs_inode;
+ struct ceph_mds_client *mdsc = &ceph_client(inode->i_sb)->mdsc;
+
+ BUG_ON(capsnap->writing);
+ capsnap->size = inode->i_size;
+ capsnap->mtime = inode->i_mtime;
+ capsnap->atime = inode->i_atime;
+ capsnap->ctime = inode->i_ctime;
+ capsnap->time_warp_seq = ci->i_time_warp_seq;
+ if (capsnap->dirty_pages) {
+ dout("finish_cap_snap %p cap_snap %p snapc %p %llu s=%llu "
+ "still has %d dirty pages\n", inode, capsnap,
+ capsnap->context, capsnap->context->seq,
+ capsnap->size, capsnap->dirty_pages);
+ return 0;
+ }
+ dout("finish_cap_snap %p cap_snap %p snapc %p %llu s=%llu clean\n",
+ inode, capsnap, capsnap->context,
+ capsnap->context->seq, capsnap->size);
+
+ spin_lock(&mdsc->snap_flush_lock);
+ list_add_tail(&ci->i_snap_flush_item, &mdsc->snap_flush_list);
+ spin_unlock(&mdsc->snap_flush_lock);
+ return 1; /* caller may want to ceph_flush_snaps */
+}
+
+
+/*
+ * Parse and apply a snapblob "snap trace" from the MDS. This specifies
+ * the snap realm parameters from a given realm and all of its ancestors,
+ * up to the root.
+ *
+ * Caller must hold snap_rwsem for write.
+ */
+int ceph_update_snap_trace(struct ceph_mds_client *mdsc,
+ void *p, void *e, bool deletion)
+{
+ struct ceph_mds_snap_realm *ri; /* encoded */
+ __le64 *snaps; /* encoded */
+ __le64 *prior_parent_snaps; /* encoded */
+ struct ceph_snap_realm *realm;
+ int invalidate = 0;
+ int err = -ENOMEM;
+
+ dout("update_snap_trace deletion=%d\n", deletion);
+more:
+ ceph_decode_need(&p, e, sizeof(*ri), bad);
+ ri = p;
+ p += sizeof(*ri);
+ ceph_decode_need(&p, e, sizeof(u64)*(le32_to_cpu(ri->num_snaps) +
+ le32_to_cpu(ri->num_prior_parent_snaps)), bad);
+ snaps = p;
+ p += sizeof(u64) * le32_to_cpu(ri->num_snaps);
+ prior_parent_snaps = p;
+ p += sizeof(u64) * le32_to_cpu(ri->num_prior_parent_snaps);
+
+ realm = ceph_lookup_snap_realm(mdsc, le64_to_cpu(ri->ino));
+ if (!realm) {
+ realm = ceph_create_snap_realm(mdsc, le64_to_cpu(ri->ino));
+ if (IS_ERR(realm)) {
+ err = PTR_ERR(realm);
+ goto fail;
+ }
+ }
+
+ if (le64_to_cpu(ri->seq) > realm->seq) {
+ dout("update_snap_trace updating %llx %p %lld -> %lld\n",
+ realm->ino, realm, realm->seq, le64_to_cpu(ri->seq));
+ /*
+ * if the realm seq has changed, queue a cap_snap for every
+ * inode with open caps. we do this _before_ we update
+ * the realm info so that we prepare for writeback under the
+ * _previous_ snap context.
+ *
+ * ...unless it's a snap deletion!
+ */
+ if (!deletion) {
+ struct ceph_inode_info *ci;
+ struct inode *lastinode = NULL;
+
+ spin_lock(&realm->inodes_with_caps_lock);
+ list_for_each_entry(ci, &realm->inodes_with_caps,
+ i_snap_realm_item) {
+ struct inode *inode = igrab(&ci->vfs_inode);
+ if (!inode)
+ continue;
+ spin_unlock(&realm->inodes_with_caps_lock);
+ if (lastinode)
+ iput(lastinode);
+ lastinode = inode;
+ ceph_queue_cap_snap(ci, realm->cached_context);
+ spin_lock(&realm->inodes_with_caps_lock);
+ }
+ spin_unlock(&realm->inodes_with_caps_lock);
+ if (lastinode)
+ iput(lastinode);
+ dout("update_snap_trace cap_snaps queued\n");
+ }
+
+ } else {
+ dout("update_snap_trace %llx %p seq %lld unchanged\n",
+ realm->ino, realm, realm->seq);
+ }
+
+ /* ensure the parent is correct */
+ err = adjust_snap_realm_parent(mdsc, realm, le64_to_cpu(ri->parent));
+ if (err < 0)
+ goto fail;
+ invalidate += err;
+
+ if (le64_to_cpu(ri->seq) > realm->seq) {
+ /* update realm parameters, snap lists */
+ realm->seq = le64_to_cpu(ri->seq);
+ realm->created = le64_to_cpu(ri->created);
+ realm->parent_since = le64_to_cpu(ri->parent_since);
+
+ realm->num_snaps = le32_to_cpu(ri->num_snaps);
+ err = dup_array(&realm->snaps, snaps, realm->num_snaps);
+ if (err < 0)
+ goto fail;
+
+ realm->num_prior_parent_snaps =
+ le32_to_cpu(ri->num_prior_parent_snaps);
+ err = dup_array(&realm->prior_parent_snaps, prior_parent_snaps,
+ realm->num_prior_parent_snaps);
+ if (err < 0)
+ goto fail;
+
+ invalidate = 1;
+ } else if (!realm->cached_context) {
+ invalidate = 1;
+ }
+
+ dout("done with %llx %p, invalidated=%d, %p %p\n", realm->ino,
+ realm, invalidate, p, e);
+
+ if (p < e)
+ goto more;
+
+ /* invalidate when we reach the _end_ (root) of the trace */
+ if (invalidate)
+ rebuild_snap_realms(realm);
+
+ __cleanup_empty_realms(mdsc);
+ return 0;
+
+bad:
+ err = -EINVAL;
+fail:
+ pr_err("update_snap_trace error %d\n", err);
+ return err;
+}
+
+
+/*
+ * Send any cap_snaps that are queued for flush. Try to carry
+ * s_mutex across multiple snap flushes to avoid locking overhead.
+ *
+ * Caller holds no locks.
+ */
+static void flush_snaps(struct ceph_mds_client *mdsc)
+{
+ struct ceph_inode_info *ci;
+ struct inode *inode;
+ struct ceph_mds_session *session = NULL;
+
+ dout("flush_snaps\n");
+ spin_lock(&mdsc->snap_flush_lock);
+ while (!list_empty(&mdsc->snap_flush_list)) {
+ ci = list_first_entry(&mdsc->snap_flush_list,
+ struct ceph_inode_info, i_snap_flush_item);
+ inode = &ci->vfs_inode;
+ igrab(inode);
+ spin_unlock(&mdsc->snap_flush_lock);
+ spin_lock(&inode->i_lock);
+ __ceph_flush_snaps(ci, &session);
+ spin_unlock(&inode->i_lock);
+ iput(inode);
+ spin_lock(&mdsc->snap_flush_lock);
+ }
+ spin_unlock(&mdsc->snap_flush_lock);
+
+ if (session) {
+ mutex_unlock(&session->s_mutex);
+ ceph_put_mds_session(session);
+ }
+ dout("flush_snaps done\n");
+}
+
+
+/*
+ * Handle a snap notification from the MDS.
+ *
+ * This can take two basic forms: the simplest is just a snap creation
+ * or deletion notification on an existing realm. This should update the
+ * realm and its children.
+ *
+ * The more difficult case is realm creation, due to snap creation at a
+ * new point in the file hierarchy, or due to a rename that moves a file or
+ * directory into another realm.
+ */
+void ceph_handle_snap(struct ceph_mds_client *mdsc,
+ struct ceph_mds_session *session,
+ struct ceph_msg *msg)
+{
+ struct super_block *sb = mdsc->client->sb;
+ int mds = session->s_mds;
+ u64 split;
+ int op;
+ int trace_len;
+ struct ceph_snap_realm *realm = NULL;
+ void *p = msg->front.iov_base;
+ void *e = p + msg->front.iov_len;
+ struct ceph_mds_snap_head *h;
+ int num_split_inos, num_split_realms;
+ __le64 *split_inos = NULL, *split_realms = NULL;
+ int i;
+ int locked_rwsem = 0;
+
+ /* decode */
+ if (msg->front.iov_len < sizeof(*h))
+ goto bad;
+ h = p;
+ op = le32_to_cpu(h->op);
+ split = le64_to_cpu(h->split); /* non-zero if we are splitting an
+ * existing realm */
+ num_split_inos = le32_to_cpu(h->num_split_inos);
+ num_split_realms = le32_to_cpu(h->num_split_realms);
+ trace_len = le32_to_cpu(h->trace_len);
+ p += sizeof(*h);
+
+ dout("handle_snap from mds%d op %s split %llx tracelen %d\n", mds,
+ ceph_snap_op_name(op), split, trace_len);
+
+ mutex_lock(&session->s_mutex);
+ session->s_seq++;
+ mutex_unlock(&session->s_mutex);
+
+ down_write(&mdsc->snap_rwsem);
+ locked_rwsem = 1;
+
+ if (op == CEPH_SNAP_OP_SPLIT) {
+ struct ceph_mds_snap_realm *ri;
+
+ /*
+ * A "split" breaks part of an existing realm off into
+ * a new realm. The MDS provides a list of inodes
+ * (with caps) and child realms that belong to the new
+ * child.
+ */
+ split_inos = p;
+ p += sizeof(u64) * num_split_inos;
+ split_realms = p;
+ p += sizeof(u64) * num_split_realms;
+ ceph_decode_need(&p, e, sizeof(*ri), bad);
+ /* we will peek at realm info here, but will _not_
+ * advance p, as the realm update will occur below in
+ * ceph_update_snap_trace. */
+ ri = p;
+
+ realm = ceph_lookup_snap_realm(mdsc, split);
+ if (!realm) {
+ realm = ceph_create_snap_realm(mdsc, split);
+ if (IS_ERR(realm))
+ goto out;
+ }
+ ceph_get_snap_realm(mdsc, realm);
+
+ dout("splitting snap_realm %llx %p\n", realm->ino, realm);
+ for (i = 0; i < num_split_inos; i++) {
+ struct ceph_vino vino = {
+ .ino = le64_to_cpu(split_inos[i]),
+ .snap = CEPH_NOSNAP,
+ };
+ struct inode *inode = ceph_find_inode(sb, vino);
+ struct ceph_inode_info *ci;
+
+ if (!inode)
+ continue;
+ ci = ceph_inode(inode);
+
+ spin_lock(&inode->i_lock);
+ if (!ci->i_snap_realm)
+ goto skip_inode;
+ /*
+ * If this inode belongs to a realm that was
+ * created after our new realm, we experienced
+ * a race (due to another split notifications
+ * arriving from a different MDS). So skip
+ * this inode.
+ */
+ if (ci->i_snap_realm->created >
+ le64_to_cpu(ri->created)) {
+ dout(" leaving %p in newer realm %llx %p\n",
+ inode, ci->i_snap_realm->ino,
+ ci->i_snap_realm);
+ goto skip_inode;
+ }
+ dout(" will move %p to split realm %llx %p\n",
+ inode, realm->ino, realm);
+ /*
+ * Remove the inode from the realm's inode
+ * list, but don't add it to the new realm
+ * yet. We don't want the cap_snap to be
+ * queued (again) by ceph_update_snap_trace()
+ * below. Queue it _now_, under the old context.
+ */
+ spin_lock(&realm->inodes_with_caps_lock);
+ list_del_init(&ci->i_snap_realm_item);
+ spin_unlock(&realm->inodes_with_caps_lock);
+ spin_unlock(&inode->i_lock);
+
+ ceph_queue_cap_snap(ci,
+ ci->i_snap_realm->cached_context);
+
+ iput(inode);
+ continue;
+
+skip_inode:
+ spin_unlock(&inode->i_lock);
+ iput(inode);
+ }
+
+ /* we may have taken some of the old realm's children. */
+ for (i = 0; i < num_split_realms; i++) {
+ struct ceph_snap_realm *child =
+ ceph_lookup_snap_realm(mdsc,
+ le64_to_cpu(split_realms[i]));
+ if (!child)
+ continue;
+ adjust_snap_realm_parent(mdsc, child, realm->ino);
+ }
+ }
+
+ /*
+ * update using the provided snap trace. if we are deleting a
+ * snap, we can avoid queueing cap_snaps.
+ */
+ ceph_update_snap_trace(mdsc, p, e,
+ op == CEPH_SNAP_OP_DESTROY);
+
+ if (op == CEPH_SNAP_OP_SPLIT) {
+ /*
+ * ok, _now_ add the inodes into the new realm.
+ */
+ for (i = 0; i < num_split_inos; i++) {
+ struct ceph_vino vino = {
+ .ino = le64_to_cpu(split_inos[i]),
+ .snap = CEPH_NOSNAP,
+ };
+ struct inode *inode = ceph_find_inode(sb, vino);
+ struct ceph_inode_info *ci;
+
+ if (!inode)
+ continue;
+ ci = ceph_inode(inode);
+ spin_lock(&inode->i_lock);
+ if (!ci->i_snap_realm)
+ goto split_skip_inode;
+ ceph_put_snap_realm(mdsc, ci->i_snap_realm);
+ spin_lock(&realm->inodes_with_caps_lock);
+ list_add(&ci->i_snap_realm_item,
+ &realm->inodes_with_caps);
+ ci->i_snap_realm = realm;
+ spin_unlock(&realm->inodes_with_caps_lock);
+ ceph_get_snap_realm(mdsc, realm);
+split_skip_inode:
+ spin_unlock(&inode->i_lock);
+ iput(inode);
+ }
+
+ /* we took a reference when we created the realm, above */
+ ceph_put_snap_realm(mdsc, realm);
+ }
+
+ __cleanup_empty_realms(mdsc);
+
+ up_write(&mdsc->snap_rwsem);
+
+ flush_snaps(mdsc);
+ return;
+
+bad:
+ pr_err("corrupt snap message from mds%d\n", mds);
+ ceph_msg_dump(msg);
+out:
+ if (locked_rwsem)
+ up_write(&mdsc->snap_rwsem);
+ return;
+}
+
+
+
--- /dev/null
+
+#include "ceph_debug.h"
+
+#include <linux/backing-dev.h>
+#include <linux/fs.h>
+#include <linux/inet.h>
+#include <linux/in6.h>
+#include <linux/module.h>
+#include <linux/mount.h>
+#include <linux/parser.h>
+#include <linux/rwsem.h>
+#include <linux/sched.h>
+#include <linux/seq_file.h>
+#include <linux/slab.h>
+#include <linux/statfs.h>
+#include <linux/string.h>
+#include <linux/version.h>
+#include <linux/vmalloc.h>
+
+#include "decode.h"
+#include "super.h"
+#include "mon_client.h"
+#include "auth.h"
+
+/*
+ * Ceph superblock operations
+ *
+ * Handle the basics of mounting, unmounting.
+ */
+
+
+/*
+ * find filename portion of a path (/foo/bar/baz -> baz)
+ */
+const char *ceph_file_part(const char *s, int len)
+{
+ const char *e = s + len;
+
+ while (e != s && *(e-1) != '/')
+ e--;
+ return e;
+}
+
+
+/*
+ * super ops
+ */
+static void ceph_put_super(struct super_block *s)
+{
+ struct ceph_client *cl = ceph_client(s);
+
+ dout("put_super\n");
+ ceph_mdsc_close_sessions(&cl->mdsc);
+ return;
+}
+
+static int ceph_statfs(struct dentry *dentry, struct kstatfs *buf)
+{
+ struct ceph_client *client = ceph_inode_to_client(dentry->d_inode);
+ struct ceph_monmap *monmap = client->monc.monmap;
+ struct ceph_statfs st;
+ u64 fsid;
+ int err;
+
+ dout("statfs\n");
+ err = ceph_monc_do_statfs(&client->monc, &st);
+ if (err < 0)
+ return err;
+
+ /* fill in kstatfs */
+ buf->f_type = CEPH_SUPER_MAGIC; /* ?? */
+
+ /*
+ * express utilization in terms of large blocks to avoid
+ * overflow on 32-bit machines.
+ */
+ buf->f_bsize = 1 << CEPH_BLOCK_SHIFT;
+ buf->f_blocks = le64_to_cpu(st.kb) >> (CEPH_BLOCK_SHIFT-10);
+ buf->f_bfree = (le64_to_cpu(st.kb) - le64_to_cpu(st.kb_used)) >>
+ (CEPH_BLOCK_SHIFT-10);
+ buf->f_bavail = le64_to_cpu(st.kb_avail) >> (CEPH_BLOCK_SHIFT-10);
+
+ buf->f_files = le64_to_cpu(st.num_objects);
+ buf->f_ffree = -1;
+ buf->f_namelen = PATH_MAX;
+ buf->f_frsize = PAGE_CACHE_SIZE;
+
+ /* leave fsid little-endian, regardless of host endianness */
+ fsid = *(u64 *)(&monmap->fsid) ^ *((u64 *)&monmap->fsid + 1);
+ buf->f_fsid.val[0] = fsid & 0xffffffff;
+ buf->f_fsid.val[1] = fsid >> 32;
+
+ return 0;
+}
+
+
+static int ceph_syncfs(struct super_block *sb, int wait)
+{
+ dout("sync_fs %d\n", wait);
+ ceph_osdc_sync(&ceph_client(sb)->osdc);
+ ceph_mdsc_sync(&ceph_client(sb)->mdsc);
+ dout("sync_fs %d done\n", wait);
+ return 0;
+}
+
+
+/**
+ * ceph_show_options - Show mount options in /proc/mounts
+ * @m: seq_file to write to
+ * @mnt: mount descriptor
+ */
+static int ceph_show_options(struct seq_file *m, struct vfsmount *mnt)
+{
+ struct ceph_client *client = ceph_sb_to_client(mnt->mnt_sb);
+ struct ceph_mount_args *args = client->mount_args;
+
+ if (args->flags & CEPH_OPT_FSID)
+ seq_printf(m, ",fsidmajor=%llu,fsidminor%llu",
+ le64_to_cpu(*(__le64 *)&args->fsid.fsid[0]),
+ le64_to_cpu(*(__le64 *)&args->fsid.fsid[8]));
+ if (args->flags & CEPH_OPT_NOSHARE)
+ seq_puts(m, ",noshare");
+ if (args->flags & CEPH_OPT_DIRSTAT)
+ seq_puts(m, ",dirstat");
+ if ((args->flags & CEPH_OPT_RBYTES) == 0)
+ seq_puts(m, ",norbytes");
+ if (args->flags & CEPH_OPT_NOCRC)
+ seq_puts(m, ",nocrc");
+ if (args->flags & CEPH_OPT_NOASYNCREADDIR)
+ seq_puts(m, ",noasyncreaddir");
+ if (strcmp(args->snapdir_name, CEPH_SNAPDIRNAME_DEFAULT))
+ seq_printf(m, ",snapdirname=%s", args->snapdir_name);
+ if (args->name)
+ seq_printf(m, ",name=%s", args->name);
+ if (args->secret)
+ seq_puts(m, ",secret=<hidden>");
+ return 0;
+}
+
+/*
+ * caches
+ */
+struct kmem_cache *ceph_inode_cachep;
+struct kmem_cache *ceph_cap_cachep;
+struct kmem_cache *ceph_dentry_cachep;
+struct kmem_cache *ceph_file_cachep;
+
+static void ceph_inode_init_once(void *foo)
+{
+ struct ceph_inode_info *ci = foo;
+ inode_init_once(&ci->vfs_inode);
+}
+
+static int default_congestion_kb(void)
+{
+ int congestion_kb;
+
+ /*
+ * Copied from NFS
+ *
+ * congestion size, scale with available memory.
+ *
+ * 64MB: 8192k
+ * 128MB: 11585k
+ * 256MB: 16384k
+ * 512MB: 23170k
+ * 1GB: 32768k
+ * 2GB: 46340k
+ * 4GB: 65536k
+ * 8GB: 92681k
+ * 16GB: 131072k
+ *
+ * This allows larger machines to have larger/more transfers.
+ * Limit the default to 256M
+ */
+ congestion_kb = (16*int_sqrt(totalram_pages)) << (PAGE_SHIFT-10);
+ if (congestion_kb > 256*1024)
+ congestion_kb = 256*1024;
+
+ return congestion_kb;
+}
+
+static int __init init_caches(void)
+{
+ ceph_inode_cachep = kmem_cache_create("ceph_inode_info",
+ sizeof(struct ceph_inode_info),
+ __alignof__(struct ceph_inode_info),
+ (SLAB_RECLAIM_ACCOUNT|SLAB_MEM_SPREAD),
+ ceph_inode_init_once);
+ if (ceph_inode_cachep == NULL)
+ return -ENOMEM;
+
+ ceph_cap_cachep = KMEM_CACHE(ceph_cap,
+ SLAB_RECLAIM_ACCOUNT|SLAB_MEM_SPREAD);
+ if (ceph_cap_cachep == NULL)
+ goto bad_cap;
+
+ ceph_dentry_cachep = KMEM_CACHE(ceph_dentry_info,
+ SLAB_RECLAIM_ACCOUNT|SLAB_MEM_SPREAD);
+ if (ceph_dentry_cachep == NULL)
+ goto bad_dentry;
+
+ ceph_file_cachep = KMEM_CACHE(ceph_file_info,
+ SLAB_RECLAIM_ACCOUNT|SLAB_MEM_SPREAD);
+ if (ceph_file_cachep == NULL)
+ goto bad_file;
+
+ return 0;
+
+bad_file:
+ kmem_cache_destroy(ceph_dentry_cachep);
+bad_dentry:
+ kmem_cache_destroy(ceph_cap_cachep);
+bad_cap:
+ kmem_cache_destroy(ceph_inode_cachep);
+ return -ENOMEM;
+}
+
+static void destroy_caches(void)
+{
+ kmem_cache_destroy(ceph_inode_cachep);
+ kmem_cache_destroy(ceph_cap_cachep);
+ kmem_cache_destroy(ceph_dentry_cachep);
+ kmem_cache_destroy(ceph_file_cachep);
+}
+
+
+/*
+ * ceph_umount_begin - initiate forced umount. Tear down down the
+ * mount, skipping steps that may hang while waiting for server(s).
+ */
+static void ceph_umount_begin(struct super_block *sb)
+{
+ struct ceph_client *client = ceph_sb_to_client(sb);
+
+ dout("ceph_umount_begin - starting forced umount\n");
+ if (!client)
+ return;
+ client->mount_state = CEPH_MOUNT_SHUTDOWN;
+ return;
+}
+
+static const struct super_operations ceph_super_ops = {
+ .alloc_inode = ceph_alloc_inode,
+ .destroy_inode = ceph_destroy_inode,
+ .write_inode = ceph_write_inode,
+ .sync_fs = ceph_syncfs,
+ .put_super = ceph_put_super,
+ .show_options = ceph_show_options,
+ .statfs = ceph_statfs,
+ .umount_begin = ceph_umount_begin,
+};
+
+
+const char *ceph_msg_type_name(int type)
+{
+ switch (type) {
+ case CEPH_MSG_SHUTDOWN: return "shutdown";
+ case CEPH_MSG_PING: return "ping";
+ case CEPH_MSG_AUTH: return "auth";
+ case CEPH_MSG_AUTH_REPLY: return "auth_reply";
+ case CEPH_MSG_MON_MAP: return "mon_map";
+ case CEPH_MSG_MON_GET_MAP: return "mon_get_map";
+ case CEPH_MSG_MON_SUBSCRIBE: return "mon_subscribe";
+ case CEPH_MSG_MON_SUBSCRIBE_ACK: return "mon_subscribe_ack";
+ case CEPH_MSG_STATFS: return "statfs";
+ case CEPH_MSG_STATFS_REPLY: return "statfs_reply";
+ case CEPH_MSG_MDS_MAP: return "mds_map";
+ case CEPH_MSG_CLIENT_SESSION: return "client_session";
+ case CEPH_MSG_CLIENT_RECONNECT: return "client_reconnect";
+ case CEPH_MSG_CLIENT_REQUEST: return "client_request";
+ case CEPH_MSG_CLIENT_REQUEST_FORWARD: return "client_request_forward";
+ case CEPH_MSG_CLIENT_REPLY: return "client_reply";
+ case CEPH_MSG_CLIENT_CAPS: return "client_caps";
+ case CEPH_MSG_CLIENT_CAPRELEASE: return "client_cap_release";
+ case CEPH_MSG_CLIENT_SNAP: return "client_snap";
+ case CEPH_MSG_CLIENT_LEASE: return "client_lease";
+ case CEPH_MSG_OSD_MAP: return "osd_map";
+ case CEPH_MSG_OSD_OP: return "osd_op";
+ case CEPH_MSG_OSD_OPREPLY: return "osd_opreply";
+ default: return "unknown";
+ }
+}
+
+
+/*
+ * mount options
+ */
+enum {
+ Opt_fsidmajor,
+ Opt_fsidminor,
+ Opt_monport,
+ Opt_wsize,
+ Opt_rsize,
+ Opt_osdtimeout,
+ Opt_osdkeepalivetimeout,
+ Opt_mount_timeout,
+ Opt_osd_idle_ttl,
+ Opt_caps_wanted_delay_min,
+ Opt_caps_wanted_delay_max,
+ Opt_readdir_max_entries,
+ Opt_congestion_kb,
+ Opt_last_int,
+ /* int args above */
+ Opt_snapdirname,
+ Opt_name,
+ Opt_secret,
+ Opt_last_string,
+ /* string args above */
+ Opt_ip,
+ Opt_noshare,
+ Opt_dirstat,
+ Opt_nodirstat,
+ Opt_rbytes,
+ Opt_norbytes,
+ Opt_nocrc,
+ Opt_noasyncreaddir,
+};
+
+static match_table_t arg_tokens = {
+ {Opt_fsidmajor, "fsidmajor=%ld"},
+ {Opt_fsidminor, "fsidminor=%ld"},
+ {Opt_monport, "monport=%d"},
+ {Opt_wsize, "wsize=%d"},
+ {Opt_rsize, "rsize=%d"},
+ {Opt_osdtimeout, "osdtimeout=%d"},
+ {Opt_osdkeepalivetimeout, "osdkeepalive=%d"},
+ {Opt_mount_timeout, "mount_timeout=%d"},
+ {Opt_osd_idle_ttl, "osd_idle_ttl=%d"},
+ {Opt_caps_wanted_delay_min, "caps_wanted_delay_min=%d"},
+ {Opt_caps_wanted_delay_max, "caps_wanted_delay_max=%d"},
+ {Opt_readdir_max_entries, "readdir_max_entries=%d"},
+ {Opt_congestion_kb, "write_congestion_kb=%d"},
+ /* int args above */
+ {Opt_snapdirname, "snapdirname=%s"},
+ {Opt_name, "name=%s"},
+ {Opt_secret, "secret=%s"},
+ /* string args above */
+ {Opt_ip, "ip=%s"},
+ {Opt_noshare, "noshare"},
+ {Opt_dirstat, "dirstat"},
+ {Opt_nodirstat, "nodirstat"},
+ {Opt_rbytes, "rbytes"},
+ {Opt_norbytes, "norbytes"},
+ {Opt_nocrc, "nocrc"},
+ {Opt_noasyncreaddir, "noasyncreaddir"},
+ {-1, NULL}
+};
+
+
+static struct ceph_mount_args *parse_mount_args(int flags, char *options,
+ const char *dev_name,
+ const char **path)
+{
+ struct ceph_mount_args *args;
+ const char *c;
+ int err = -ENOMEM;
+ substring_t argstr[MAX_OPT_ARGS];
+
+ args = kzalloc(sizeof(*args), GFP_KERNEL);
+ if (!args)
+ return ERR_PTR(-ENOMEM);
+ args->mon_addr = kcalloc(CEPH_MAX_MON, sizeof(*args->mon_addr),
+ GFP_KERNEL);
+ if (!args->mon_addr)
+ goto out;
+
+ dout("parse_mount_args %p, dev_name '%s'\n", args, dev_name);
+
+ /* start with defaults */
+ args->sb_flags = flags;
+ args->flags = CEPH_OPT_DEFAULT;
+ args->osd_timeout = CEPH_OSD_TIMEOUT_DEFAULT;
+ args->osd_keepalive_timeout = CEPH_OSD_KEEPALIVE_DEFAULT;
+ args->mount_timeout = CEPH_MOUNT_TIMEOUT_DEFAULT; /* seconds */
+ args->osd_idle_ttl = CEPH_OSD_IDLE_TTL_DEFAULT; /* seconds */
+ args->caps_wanted_delay_min = CEPH_CAPS_WANTED_DELAY_MIN_DEFAULT;
+ args->caps_wanted_delay_max = CEPH_CAPS_WANTED_DELAY_MAX_DEFAULT;
+ args->rsize = CEPH_MOUNT_RSIZE_DEFAULT;
+ args->snapdir_name = kstrdup(CEPH_SNAPDIRNAME_DEFAULT, GFP_KERNEL);
+ args->cap_release_safety = CEPH_CAPS_PER_RELEASE * 4;
+ args->max_readdir = 1024;
+ args->congestion_kb = default_congestion_kb();
+
+ /* ip1[:port1][,ip2[:port2]...]:/subdir/in/fs */
+ err = -EINVAL;
+ if (!dev_name)
+ goto out;
+ *path = strstr(dev_name, ":/");
+ if (*path == NULL) {
+ pr_err("device name is missing path (no :/ in %s)\n",
+ dev_name);
+ goto out;
+ }
+
+ /* get mon ip(s) */
+ err = ceph_parse_ips(dev_name, *path, args->mon_addr,
+ CEPH_MAX_MON, &args->num_mon);
+ if (err < 0)
+ goto out;
+
+ /* path on server */
+ *path += 2;
+ dout("server path '%s'\n", *path);
+
+ /* parse mount options */
+ while ((c = strsep(&options, ",")) != NULL) {
+ int token, intval, ret;
+ if (!*c)
+ continue;
+ err = -EINVAL;
+ token = match_token((char *)c, arg_tokens, argstr);
+ if (token < 0) {
+ pr_err("bad mount option at '%s'\n", c);
+ goto out;
+ }
+ if (token < Opt_last_int) {
+ ret = match_int(&argstr[0], &intval);
+ if (ret < 0) {
+ pr_err("bad mount option arg (not int) "
+ "at '%s'\n", c);
+ continue;
+ }
+ dout("got int token %d val %d\n", token, intval);
+ } else if (token > Opt_last_int && token < Opt_last_string) {
+ dout("got string token %d val %s\n", token,
+ argstr[0].from);
+ } else {
+ dout("got token %d\n", token);
+ }
+ switch (token) {
+ case Opt_fsidmajor:
+ *(__le64 *)&args->fsid.fsid[0] = cpu_to_le64(intval);
+ break;
+ case Opt_fsidminor:
+ *(__le64 *)&args->fsid.fsid[8] = cpu_to_le64(intval);
+ break;
+ case Opt_ip:
+ err = ceph_parse_ips(argstr[0].from,
+ argstr[0].to,
+ &args->my_addr,
+ 1, NULL);
+ if (err < 0)
+ goto out;
+ args->flags |= CEPH_OPT_MYIP;
+ break;
+
+ case Opt_snapdirname:
+ kfree(args->snapdir_name);
+ args->snapdir_name = kstrndup(argstr[0].from,
+ argstr[0].to-argstr[0].from,
+ GFP_KERNEL);
+ break;
+ case Opt_name:
+ args->name = kstrndup(argstr[0].from,
+ argstr[0].to-argstr[0].from,
+ GFP_KERNEL);
+ break;
+ case Opt_secret:
+ args->secret = kstrndup(argstr[0].from,
+ argstr[0].to-argstr[0].from,
+ GFP_KERNEL);
+ break;
+
+ /* misc */
+ case Opt_wsize:
+ args->wsize = intval;
+ break;
+ case Opt_rsize:
+ args->rsize = intval;
+ break;
+ case Opt_osdtimeout:
+ args->osd_timeout = intval;
+ break;
+ case Opt_osdkeepalivetimeout:
+ args->osd_keepalive_timeout = intval;
+ break;
+ case Opt_mount_timeout:
+ args->mount_timeout = intval;
+ break;
+ case Opt_caps_wanted_delay_min:
+ args->caps_wanted_delay_min = intval;
+ break;
+ case Opt_caps_wanted_delay_max:
+ args->caps_wanted_delay_max = intval;
+ break;
+ case Opt_readdir_max_entries:
+ args->max_readdir = intval;
+ break;
+ case Opt_congestion_kb:
+ args->congestion_kb = intval;
+ break;
+
+ case Opt_noshare:
+ args->flags |= CEPH_OPT_NOSHARE;
+ break;
+
+ case Opt_dirstat:
+ args->flags |= CEPH_OPT_DIRSTAT;
+ break;
+ case Opt_nodirstat:
+ args->flags &= ~CEPH_OPT_DIRSTAT;
+ break;
+ case Opt_rbytes:
+ args->flags |= CEPH_OPT_RBYTES;
+ break;
+ case Opt_norbytes:
+ args->flags &= ~CEPH_OPT_RBYTES;
+ break;
+ case Opt_nocrc:
+ args->flags |= CEPH_OPT_NOCRC;
+ break;
+ case Opt_noasyncreaddir:
+ args->flags |= CEPH_OPT_NOASYNCREADDIR;
+ break;
+
+ default:
+ BUG_ON(token);
+ }
+ }
+ return args;
+
+out:
+ kfree(args->mon_addr);
+ kfree(args);
+ return ERR_PTR(err);
+}
+
+static void destroy_mount_args(struct ceph_mount_args *args)
+{
+ dout("destroy_mount_args %p\n", args);
+ kfree(args->snapdir_name);
+ args->snapdir_name = NULL;
+ kfree(args->name);
+ args->name = NULL;
+ kfree(args->secret);
+ args->secret = NULL;
+ kfree(args);
+}
+
+/*
+ * create a fresh client instance
+ */
+static struct ceph_client *ceph_create_client(struct ceph_mount_args *args)
+{
+ struct ceph_client *client;
+ int err = -ENOMEM;
+
+ client = kzalloc(sizeof(*client), GFP_KERNEL);
+ if (client == NULL)
+ return ERR_PTR(-ENOMEM);
+
+ mutex_init(&client->mount_mutex);
+
+ init_waitqueue_head(&client->auth_wq);
+
+ client->sb = NULL;
+ client->mount_state = CEPH_MOUNT_MOUNTING;
+ client->mount_args = args;
+
+ client->msgr = NULL;
+
+ client->auth_err = 0;
+ atomic_long_set(&client->writeback_count, 0);
+
+ err = bdi_init(&client->backing_dev_info);
+ if (err < 0)
+ goto fail;
+
+ err = -ENOMEM;
+ client->wb_wq = create_workqueue("ceph-writeback");
+ if (client->wb_wq == NULL)
+ goto fail_bdi;
+ client->pg_inv_wq = create_singlethread_workqueue("ceph-pg-invalid");
+ if (client->pg_inv_wq == NULL)
+ goto fail_wb_wq;
+ client->trunc_wq = create_singlethread_workqueue("ceph-trunc");
+ if (client->trunc_wq == NULL)
+ goto fail_pg_inv_wq;
+
+ /* set up mempools */
+ err = -ENOMEM;
+ client->wb_pagevec_pool = mempool_create_kmalloc_pool(10,
+ client->mount_args->wsize >> PAGE_CACHE_SHIFT);
+ if (!client->wb_pagevec_pool)
+ goto fail_trunc_wq;
+
+ /* caps */
+ client->min_caps = args->max_readdir;
+ ceph_adjust_min_caps(client->min_caps);
+
+ /* subsystems */
+ err = ceph_monc_init(&client->monc, client);
+ if (err < 0)
+ goto fail_mempool;
+ err = ceph_osdc_init(&client->osdc, client);
+ if (err < 0)
+ goto fail_monc;
+ err = ceph_mdsc_init(&client->mdsc, client);
+ if (err < 0)
+ goto fail_osdc;
+ return client;
+
+fail_osdc:
+ ceph_osdc_stop(&client->osdc);
+fail_monc:
+ ceph_monc_stop(&client->monc);
+fail_mempool:
+ mempool_destroy(client->wb_pagevec_pool);
+fail_trunc_wq:
+ destroy_workqueue(client->trunc_wq);
+fail_pg_inv_wq:
+ destroy_workqueue(client->pg_inv_wq);
+fail_wb_wq:
+ destroy_workqueue(client->wb_wq);
+fail_bdi:
+ bdi_destroy(&client->backing_dev_info);
+fail:
+ kfree(client);
+ return ERR_PTR(err);
+}
+
+static void ceph_destroy_client(struct ceph_client *client)
+{
+ dout("destroy_client %p\n", client);
+
+ /* unmount */
+ ceph_mdsc_stop(&client->mdsc);
+ ceph_monc_stop(&client->monc);
+ ceph_osdc_stop(&client->osdc);
+
+ ceph_adjust_min_caps(-client->min_caps);
+
+ ceph_debugfs_client_cleanup(client);
+ destroy_workqueue(client->wb_wq);
+ destroy_workqueue(client->pg_inv_wq);
+ destroy_workqueue(client->trunc_wq);
+
+ if (client->msgr)
+ ceph_messenger_destroy(client->msgr);
+ mempool_destroy(client->wb_pagevec_pool);
+
+ destroy_mount_args(client->mount_args);
+
+ kfree(client);
+ dout("destroy_client %p done\n", client);
+}
+
+/*
+ * Initially learn our fsid, or verify an fsid matches.
+ */
+int ceph_check_fsid(struct ceph_client *client, struct ceph_fsid *fsid)
+{
+ if (client->have_fsid) {
+ if (ceph_fsid_compare(&client->fsid, fsid)) {
+ pr_err("bad fsid, had " FSID_FORMAT " got " FSID_FORMAT,
+ PR_FSID(&client->fsid), PR_FSID(fsid));
+ return -1;
+ }
+ } else {
+ pr_info("client%lld fsid " FSID_FORMAT "\n",
+ client->monc.auth->global_id, PR_FSID(fsid));
+ memcpy(&client->fsid, fsid, sizeof(*fsid));
+ ceph_debugfs_client_init(client);
+ client->have_fsid = true;
+ }
+ return 0;
+}
+
+/*
+ * true if we have the mon map (and have thus joined the cluster)
+ */
+static int have_mon_map(struct ceph_client *client)
+{
+ return client->monc.monmap && client->monc.monmap->epoch;
+}
+
+/*
+ * Bootstrap mount by opening the root directory. Note the mount
+ * @started time from caller, and time out if this takes too long.
+ */
+static struct dentry *open_root_dentry(struct ceph_client *client,
+ const char *path,
+ unsigned long started)
+{
+ struct ceph_mds_client *mdsc = &client->mdsc;
+ struct ceph_mds_request *req = NULL;
+ int err;
+ struct dentry *root;
+
+ /* open dir */
+ dout("open_root_inode opening '%s'\n", path);
+ req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_GETATTR, USE_ANY_MDS);
+ if (IS_ERR(req))
+ return ERR_PTR(PTR_ERR(req));
+ req->r_path1 = kstrdup(path, GFP_NOFS);
+ req->r_ino1.ino = CEPH_INO_ROOT;
+ req->r_ino1.snap = CEPH_NOSNAP;
+ req->r_started = started;
+ req->r_timeout = client->mount_args->mount_timeout * HZ;
+ req->r_args.getattr.mask = cpu_to_le32(CEPH_STAT_CAP_INODE);
+ req->r_num_caps = 2;
+ err = ceph_mdsc_do_request(mdsc, NULL, req);
+ if (err == 0) {
+ dout("open_root_inode success\n");
+ if (ceph_ino(req->r_target_inode) == CEPH_INO_ROOT &&
+ client->sb->s_root == NULL)
+ root = d_alloc_root(req->r_target_inode);
+ else
+ root = d_obtain_alias(req->r_target_inode);
+ req->r_target_inode = NULL;
+ dout("open_root_inode success, root dentry is %p\n", root);
+ } else {
+ root = ERR_PTR(err);
+ }
+ ceph_mdsc_put_request(req);
+ return root;
+}
+
+/*
+ * mount: join the ceph cluster, and open root directory.
+ */
+static int ceph_mount(struct ceph_client *client, struct vfsmount *mnt,
+ const char *path)
+{
+ struct ceph_entity_addr *myaddr = NULL;
+ int err;
+ unsigned long timeout = client->mount_args->mount_timeout * HZ;
+ unsigned long started = jiffies; /* note the start time */
+ struct dentry *root;
+
+ dout("mount start\n");
+ mutex_lock(&client->mount_mutex);
+
+ /* initialize the messenger */
+ if (client->msgr == NULL) {
+ if (ceph_test_opt(client, MYIP))
+ myaddr = &client->mount_args->my_addr;
+ client->msgr = ceph_messenger_create(myaddr);
+ if (IS_ERR(client->msgr)) {
+ err = PTR_ERR(client->msgr);
+ client->msgr = NULL;
+ goto out;
+ }
+ client->msgr->nocrc = ceph_test_opt(client, NOCRC);
+ }
+
+ /* open session, and wait for mon, mds, and osd maps */
+ err = ceph_monc_open_session(&client->monc);
+ if (err < 0)
+ goto out;
+
+ while (!have_mon_map(client)) {
+ err = -EIO;
+ if (timeout && time_after_eq(jiffies, started + timeout))
+ goto out;
+
+ /* wait */
+ dout("mount waiting for mon_map\n");
+ err = wait_event_interruptible_timeout(client->auth_wq,
+ have_mon_map(client) || (client->auth_err < 0),
+ timeout);
+ if (err == -EINTR || err == -ERESTARTSYS)
+ goto out;
+ if (client->auth_err < 0) {
+ err = client->auth_err;
+ goto out;
+ }
+ }
+
+ dout("mount opening root\n");
+ root = open_root_dentry(client, "", started);
+ if (IS_ERR(root)) {
+ err = PTR_ERR(root);
+ goto out;
+ }
+ if (client->sb->s_root)
+ dput(root);
+ else
+ client->sb->s_root = root;
+
+ if (path[0] == 0) {
+ dget(root);
+ } else {
+ dout("mount opening base mountpoint\n");
+ root = open_root_dentry(client, path, started);
+ if (IS_ERR(root)) {
+ err = PTR_ERR(root);
+ dput(client->sb->s_root);
+ client->sb->s_root = NULL;
+ goto out;
+ }
+ }
+
+ mnt->mnt_root = root;
+ mnt->mnt_sb = client->sb;
+
+ client->mount_state = CEPH_MOUNT_MOUNTED;
+ dout("mount success\n");
+ err = 0;
+
+out:
+ mutex_unlock(&client->mount_mutex);
+ return err;
+}
+
+static int ceph_set_super(struct super_block *s, void *data)
+{
+ struct ceph_client *client = data;
+ int ret;
+
+ dout("set_super %p data %p\n", s, data);
+
+ s->s_flags = client->mount_args->sb_flags;
+ s->s_maxbytes = 1ULL << 40; /* temp value until we get mdsmap */
+
+ s->s_fs_info = client;
+ client->sb = s;
+
+ s->s_op = &ceph_super_ops;
+ s->s_export_op = &ceph_export_ops;
+
+ s->s_time_gran = 1000; /* 1000 ns == 1 us */
+
+ ret = set_anon_super(s, NULL); /* what is that second arg for? */
+ if (ret != 0)
+ goto fail;
+
+ return ret;
+
+fail:
+ s->s_fs_info = NULL;
+ client->sb = NULL;
+ return ret;
+}
+
+/*
+ * share superblock if same fs AND options
+ */
+static int ceph_compare_super(struct super_block *sb, void *data)
+{
+ struct ceph_client *new = data;
+ struct ceph_mount_args *args = new->mount_args;
+ struct ceph_client *other = ceph_sb_to_client(sb);
+ int i;
+
+ dout("ceph_compare_super %p\n", sb);
+ if (args->flags & CEPH_OPT_FSID) {
+ if (ceph_fsid_compare(&args->fsid, &other->fsid)) {
+ dout("fsid doesn't match\n");
+ return 0;
+ }
+ } else {
+ /* do we share (a) monitor? */
+ for (i = 0; i < new->monc.monmap->num_mon; i++)
+ if (ceph_monmap_contains(other->monc.monmap,
+ &new->monc.monmap->mon_inst[i].addr))
+ break;
+ if (i == new->monc.monmap->num_mon) {
+ dout("mon ip not part of monmap\n");
+ return 0;
+ }
+ dout("mon ip matches existing sb %p\n", sb);
+ }
+ if (args->sb_flags != other->mount_args->sb_flags) {
+ dout("flags differ\n");
+ return 0;
+ }
+ return 1;
+}
+
+/*
+ * construct our own bdi so we can control readahead, etc.
+ */
+static int ceph_register_bdi(struct super_block *sb, struct ceph_client *client)
+{
+ int err;
+
+ sb->s_bdi = &client->backing_dev_info;
+
+ /* set ra_pages based on rsize mount option? */
+ if (client->mount_args->rsize >= PAGE_CACHE_SIZE)
+ client->backing_dev_info.ra_pages =
+ (client->mount_args->rsize + PAGE_CACHE_SIZE - 1)
+ >> PAGE_SHIFT;
+ err = bdi_register_dev(&client->backing_dev_info, sb->s_dev);
+ return err;
+}
+
+static int ceph_get_sb(struct file_system_type *fs_type,
+ int flags, const char *dev_name, void *data,
+ struct vfsmount *mnt)
+{
+ struct super_block *sb;
+ struct ceph_client *client;
+ int err;
+ int (*compare_super)(struct super_block *, void *) = ceph_compare_super;
+ const char *path = NULL;
+ struct ceph_mount_args *args;
+
+ dout("ceph_get_sb\n");
+ args = parse_mount_args(flags, data, dev_name, &path);
+ if (IS_ERR(args)) {
+ err = PTR_ERR(args);
+ goto out_final;
+ }
+
+ /* create client (which we may/may not use) */
+ client = ceph_create_client(args);
+ if (IS_ERR(client)) {
+ err = PTR_ERR(client);
+ goto out_final;
+ }
+
+ if (client->mount_args->flags & CEPH_OPT_NOSHARE)
+ compare_super = NULL;
+ sb = sget(fs_type, compare_super, ceph_set_super, client);
+ if (IS_ERR(sb)) {
+ err = PTR_ERR(sb);
+ goto out;
+ }
+
+ if (ceph_client(sb) != client) {
+ ceph_destroy_client(client);
+ client = ceph_client(sb);
+ dout("get_sb got existing client %p\n", client);
+ } else {
+ dout("get_sb using new client %p\n", client);
+ err = ceph_register_bdi(sb, client);
+ if (err < 0)
+ goto out_splat;
+ }
+
+ err = ceph_mount(client, mnt, path);
+ if (err < 0)
+ goto out_splat;
+ dout("root %p inode %p ino %llx.%llx\n", mnt->mnt_root,
+ mnt->mnt_root->d_inode, ceph_vinop(mnt->mnt_root->d_inode));
+ return 0;
+
+out_splat:
+ ceph_mdsc_close_sessions(&client->mdsc);
+ up_write(&sb->s_umount);
+ deactivate_super(sb);
+ goto out_final;
+
+out:
+ ceph_destroy_client(client);
+out_final:
+ dout("ceph_get_sb fail %d\n", err);
+ return err;
+}
+
+static void ceph_kill_sb(struct super_block *s)
+{
+ struct ceph_client *client = ceph_sb_to_client(s);
+ dout("kill_sb %p\n", s);
+ ceph_mdsc_pre_umount(&client->mdsc);
+ kill_anon_super(s); /* will call put_super after sb is r/o */
+ if (s->s_bdi == &client->backing_dev_info)
+ bdi_unregister(&client->backing_dev_info);
+ bdi_destroy(&client->backing_dev_info);
+ ceph_destroy_client(client);
+}
+
+static struct file_system_type ceph_fs_type = {
+ .owner = THIS_MODULE,
+ .name = "ceph",
+ .get_sb = ceph_get_sb,
+ .kill_sb = ceph_kill_sb,
+ .fs_flags = FS_RENAME_DOES_D_MOVE,
+};
+
+#define _STRINGIFY(x) #x
+#define STRINGIFY(x) _STRINGIFY(x)
+
+static int __init init_ceph(void)
+{
+ int ret = 0;
+
+ ret = ceph_debugfs_init();
+ if (ret < 0)
+ goto out;
+
+ ret = ceph_msgr_init();
+ if (ret < 0)
+ goto out_debugfs;
+
+ ret = init_caches();
+ if (ret)
+ goto out_msgr;
+
+ ceph_caps_init();
+
+ ret = register_filesystem(&ceph_fs_type);
+ if (ret)
+ goto out_icache;
+
+ pr_info("loaded %d.%d.%d (mon/mds/osd proto %d/%d/%d)\n",
+ CEPH_VERSION_MAJOR, CEPH_VERSION_MINOR, CEPH_VERSION_PATCH,
+ CEPH_MONC_PROTOCOL, CEPH_MDSC_PROTOCOL, CEPH_OSDC_PROTOCOL);
+ return 0;
+
+out_icache:
+ destroy_caches();
+out_msgr:
+ ceph_msgr_exit();
+out_debugfs:
+ ceph_debugfs_cleanup();
+out:
+ return ret;
+}
+
+static void __exit exit_ceph(void)
+{
+ dout("exit_ceph\n");
+ unregister_filesystem(&ceph_fs_type);
+ ceph_caps_finalize();
+ destroy_caches();
+ ceph_msgr_exit();
+ ceph_debugfs_cleanup();
+}
+
+module_init(init_ceph);
+module_exit(exit_ceph);
+
+MODULE_AUTHOR("Sage Weil <sage@newdream.net>");
+MODULE_AUTHOR("Yehuda Sadeh <yehuda@hq.newdream.net>");
+MODULE_AUTHOR("Patience Warnick <patience@newdream.net>");
+MODULE_DESCRIPTION("Ceph filesystem for Linux");
+MODULE_LICENSE("GPL");
--- /dev/null
+#ifndef _FS_CEPH_SUPER_H
+#define _FS_CEPH_SUPER_H
+
+#include "ceph_debug.h"
+
+#include <asm/unaligned.h>
+#include <linux/backing-dev.h>
+#include <linux/completion.h>
+#include <linux/exportfs.h>
+#include <linux/fs.h>
+#include <linux/mempool.h>
+#include <linux/pagemap.h>
+#include <linux/wait.h>
+#include <linux/writeback.h>
+#include <linux/slab.h>
+
+#include "types.h"
+#include "messenger.h"
+#include "msgpool.h"
+#include "mon_client.h"
+#include "mds_client.h"
+#include "osd_client.h"
+#include "ceph_fs.h"
+
+/* f_type in struct statfs */
+#define CEPH_SUPER_MAGIC 0x00c36400
+
+/* large granularity for statfs utilization stats to facilitate
+ * large volume sizes on 32-bit machines. */
+#define CEPH_BLOCK_SHIFT 20 /* 1 MB */
+#define CEPH_BLOCK (1 << CEPH_BLOCK_SHIFT)
+
+/*
+ * mount options
+ */
+#define CEPH_OPT_FSID (1<<0)
+#define CEPH_OPT_NOSHARE (1<<1) /* don't share client with other sbs */
+#define CEPH_OPT_MYIP (1<<2) /* specified my ip */
+#define CEPH_OPT_DIRSTAT (1<<4) /* funky `cat dirname` for stats */
+#define CEPH_OPT_RBYTES (1<<5) /* dir st_bytes = rbytes */
+#define CEPH_OPT_NOCRC (1<<6) /* no data crc on writes */
+#define CEPH_OPT_NOASYNCREADDIR (1<<7) /* no dcache readdir */
+
+#define CEPH_OPT_DEFAULT (CEPH_OPT_RBYTES)
+
+#define ceph_set_opt(client, opt) \
+ (client)->mount_args->flags |= CEPH_OPT_##opt;
+#define ceph_test_opt(client, opt) \
+ (!!((client)->mount_args->flags & CEPH_OPT_##opt))
+
+
+struct ceph_mount_args {
+ int sb_flags;
+ int num_mon;
+ struct ceph_entity_addr *mon_addr;
+ int flags;
+ int mount_timeout;
+ int osd_idle_ttl;
+ int caps_wanted_delay_min, caps_wanted_delay_max;
+ struct ceph_fsid fsid;
+ struct ceph_entity_addr my_addr;
+ int wsize;
+ int rsize; /* max readahead */
+ int max_readdir; /* max readdir size */
+ int congestion_kb; /* max readdir size */
+ int osd_timeout;
+ int osd_keepalive_timeout;
+ char *snapdir_name; /* default ".snap" */
+ char *name;
+ char *secret;
+ int cap_release_safety;
+};
+
+/*
+ * defaults
+ */
+#define CEPH_MOUNT_TIMEOUT_DEFAULT 60
+#define CEPH_OSD_TIMEOUT_DEFAULT 60 /* seconds */
+#define CEPH_OSD_KEEPALIVE_DEFAULT 5
+#define CEPH_OSD_IDLE_TTL_DEFAULT 60
+#define CEPH_MOUNT_RSIZE_DEFAULT (512*1024) /* readahead */
+
+#define CEPH_MSG_MAX_FRONT_LEN (16*1024*1024)
+#define CEPH_MSG_MAX_DATA_LEN (16*1024*1024)
+
+#define CEPH_SNAPDIRNAME_DEFAULT ".snap"
+#define CEPH_AUTH_NAME_DEFAULT "guest"
+
+/*
+ * Delay telling the MDS we no longer want caps, in case we reopen
+ * the file. Delay a minimum amount of time, even if we send a cap
+ * message for some other reason. Otherwise, take the oppotunity to
+ * update the mds to avoid sending another message later.
+ */
+#define CEPH_CAPS_WANTED_DELAY_MIN_DEFAULT 5 /* cap release delay */
+#define CEPH_CAPS_WANTED_DELAY_MAX_DEFAULT 60 /* cap release delay */
+
+
+/* mount state */
+enum {
+ CEPH_MOUNT_MOUNTING,
+ CEPH_MOUNT_MOUNTED,
+ CEPH_MOUNT_UNMOUNTING,
+ CEPH_MOUNT_UNMOUNTED,
+ CEPH_MOUNT_SHUTDOWN,
+};
+
+/*
+ * subtract jiffies
+ */
+static inline unsigned long time_sub(unsigned long a, unsigned long b)
+{
+ BUG_ON(time_after(b, a));
+ return (long)a - (long)b;
+}
+
+/*
+ * per-filesystem client state
+ *
+ * possibly shared by multiple mount points, if they are
+ * mounting the same ceph filesystem/cluster.
+ */
+struct ceph_client {
+ struct ceph_fsid fsid;
+ bool have_fsid;
+
+ struct mutex mount_mutex; /* serialize mount attempts */
+ struct ceph_mount_args *mount_args;
+
+ struct super_block *sb;
+
+ unsigned long mount_state;
+ wait_queue_head_t auth_wq;
+
+ int auth_err;
+
+ int min_caps; /* min caps i added */
+
+ struct ceph_messenger *msgr; /* messenger instance */
+ struct ceph_mon_client monc;
+ struct ceph_mds_client mdsc;
+ struct ceph_osd_client osdc;
+
+ /* writeback */
+ mempool_t *wb_pagevec_pool;
+ struct workqueue_struct *wb_wq;
+ struct workqueue_struct *pg_inv_wq;
+ struct workqueue_struct *trunc_wq;
+ atomic_long_t writeback_count;
+
+ struct backing_dev_info backing_dev_info;
+
+#ifdef CONFIG_DEBUG_FS
+ struct dentry *debugfs_monmap;
+ struct dentry *debugfs_mdsmap, *debugfs_osdmap;
+ struct dentry *debugfs_dir, *debugfs_dentry_lru, *debugfs_caps;
+ struct dentry *debugfs_congestion_kb;
+ struct dentry *debugfs_bdi;
+#endif
+};
+
+static inline struct ceph_client *ceph_client(struct super_block *sb)
+{
+ return sb->s_fs_info;
+}
+
+
+/*
+ * File i/o capability. This tracks shared state with the metadata
+ * server that allows us to cache or writeback attributes or to read
+ * and write data. For any given inode, we should have one or more
+ * capabilities, one issued by each metadata server, and our
+ * cumulative access is the OR of all issued capabilities.
+ *
+ * Each cap is referenced by the inode's i_caps rbtree and by per-mds
+ * session capability lists.
+ */
+struct ceph_cap {
+ struct ceph_inode_info *ci;
+ struct rb_node ci_node; /* per-ci cap tree */
+ struct ceph_mds_session *session;
+ struct list_head session_caps; /* per-session caplist */
+ int mds;
+ u64 cap_id; /* unique cap id (mds provided) */
+ int issued; /* latest, from the mds */
+ int implemented; /* implemented superset of issued (for revocation) */
+ int mds_wanted;
+ u32 seq, issue_seq, mseq;
+ u32 cap_gen; /* active/stale cycle */
+ unsigned long last_used;
+ struct list_head caps_item;
+};
+
+#define CHECK_CAPS_NODELAY 1 /* do not delay any further */
+#define CHECK_CAPS_AUTHONLY 2 /* only check auth cap */
+#define CHECK_CAPS_FLUSH 4 /* flush any dirty caps */
+
+/*
+ * Snapped cap state that is pending flush to mds. When a snapshot occurs,
+ * we first complete any in-process sync writes and writeback any dirty
+ * data before flushing the snapped state (tracked here) back to the MDS.
+ */
+struct ceph_cap_snap {
+ atomic_t nref;
+ struct ceph_inode_info *ci;
+ struct list_head ci_item, flushing_item;
+
+ u64 follows, flush_tid;
+ int issued, dirty;
+ struct ceph_snap_context *context;
+
+ mode_t mode;
+ uid_t uid;
+ gid_t gid;
+
+ void *xattr_blob;
+ int xattr_len;
+ u64 xattr_version;
+
+ u64 size;
+ struct timespec mtime, atime, ctime;
+ u64 time_warp_seq;
+ int writing; /* a sync write is still in progress */
+ int dirty_pages; /* dirty pages awaiting writeback */
+};
+
+static inline void ceph_put_cap_snap(struct ceph_cap_snap *capsnap)
+{
+ if (atomic_dec_and_test(&capsnap->nref))
+ kfree(capsnap);
+}
+
+/*
+ * The frag tree describes how a directory is fragmented, potentially across
+ * multiple metadata servers. It is also used to indicate points where
+ * metadata authority is delegated, and whether/where metadata is replicated.
+ *
+ * A _leaf_ frag will be present in the i_fragtree IFF there is
+ * delegation info. That is, if mds >= 0 || ndist > 0.
+ */
+#define CEPH_MAX_DIRFRAG_REP 4
+
+struct ceph_inode_frag {
+ struct rb_node node;
+
+ /* fragtree state */
+ u32 frag;
+ int split_by; /* i.e. 2^(split_by) children */
+
+ /* delegation and replication info */
+ int mds; /* -1 if same authority as parent */
+ int ndist; /* >0 if replicated */
+ int dist[CEPH_MAX_DIRFRAG_REP];
+};
+
+/*
+ * We cache inode xattrs as an encoded blob until they are first used,
+ * at which point we parse them into an rbtree.
+ */
+struct ceph_inode_xattr {
+ struct rb_node node;
+
+ const char *name;
+ int name_len;
+ const char *val;
+ int val_len;
+ int dirty;
+
+ int should_free_name;
+ int should_free_val;
+};
+
+struct ceph_inode_xattrs_info {
+ /*
+ * (still encoded) xattr blob. we avoid the overhead of parsing
+ * this until someone actually calls getxattr, etc.
+ *
+ * blob->vec.iov_len == 4 implies there are no xattrs; blob ==
+ * NULL means we don't know.
+ */
+ struct ceph_buffer *blob, *prealloc_blob;
+
+ struct rb_root index;
+ bool dirty;
+ int count;
+ int names_size;
+ int vals_size;
+ u64 version, index_version;
+};
+
+/*
+ * Ceph inode.
+ */
+#define CEPH_I_COMPLETE 1 /* we have complete directory cached */
+#define CEPH_I_NODELAY 4 /* do not delay cap release */
+#define CEPH_I_FLUSH 8 /* do not delay flush of dirty metadata */
+#define CEPH_I_NOFLUSH 16 /* do not flush dirty caps */
+
+struct ceph_inode_info {
+ struct ceph_vino i_vino; /* ceph ino + snap */
+
+ u64 i_version;
+ u32 i_time_warp_seq;
+
+ unsigned i_ceph_flags;
+ unsigned long i_release_count;
+
+ struct ceph_file_layout i_layout;
+ char *i_symlink;
+
+ /* for dirs */
+ struct timespec i_rctime;
+ u64 i_rbytes, i_rfiles, i_rsubdirs;
+ u64 i_files, i_subdirs;
+ u64 i_max_offset; /* largest readdir offset, set with I_COMPLETE */
+
+ struct rb_root i_fragtree;
+ struct mutex i_fragtree_mutex;
+
+ struct ceph_inode_xattrs_info i_xattrs;
+
+ /* capabilities. protected _both_ by i_lock and cap->session's
+ * s_mutex. */
+ struct rb_root i_caps; /* cap list */
+ struct ceph_cap *i_auth_cap; /* authoritative cap, if any */
+ unsigned i_dirty_caps, i_flushing_caps; /* mask of dirtied fields */
+ struct list_head i_dirty_item, i_flushing_item;
+ u64 i_cap_flush_seq;
+ /* we need to track cap writeback on a per-cap-bit basis, to allow
+ * overlapping, pipelined cap flushes to the mds. we can probably
+ * reduce the tid to 8 bits if we're concerned about inode size. */
+ u16 i_cap_flush_last_tid, i_cap_flush_tid[CEPH_CAP_BITS];
+ wait_queue_head_t i_cap_wq; /* threads waiting on a capability */
+ unsigned long i_hold_caps_min; /* jiffies */
+ unsigned long i_hold_caps_max; /* jiffies */
+ struct list_head i_cap_delay_list; /* for delayed cap release to mds */
+ int i_cap_exporting_mds; /* to handle cap migration between */
+ unsigned i_cap_exporting_mseq; /* mds's. */
+ unsigned i_cap_exporting_issued;
+ struct ceph_cap_reservation i_cap_migration_resv;
+ struct list_head i_cap_snaps; /* snapped state pending flush to mds */
+ struct ceph_snap_context *i_head_snapc; /* set if wr_buffer_head > 0 */
+ unsigned i_snap_caps; /* cap bits for snapped files */
+
+ int i_nr_by_mode[CEPH_FILE_MODE_NUM]; /* open file counts */
+
+ u32 i_truncate_seq; /* last truncate to smaller size */
+ u64 i_truncate_size; /* and the size we last truncated down to */
+ int i_truncate_pending; /* still need to call vmtruncate */
+
+ u64 i_max_size; /* max file size authorized by mds */
+ u64 i_reported_size; /* (max_)size reported to or requested of mds */
+ u64 i_wanted_max_size; /* offset we'd like to write too */
+ u64 i_requested_max_size; /* max_size we've requested */
+
+ /* held references to caps */
+ int i_pin_ref;
+ int i_rd_ref, i_rdcache_ref, i_wr_ref;
+ int i_wrbuffer_ref, i_wrbuffer_ref_head;
+ u32 i_shared_gen; /* increment each time we get FILE_SHARED */
+ u32 i_rdcache_gen; /* we increment this each time we get
+ FILE_CACHE. If it's non-zero, we
+ _may_ have cached pages. */
+ u32 i_rdcache_revoking; /* RDCACHE gen to async invalidate, if any */
+
+ struct list_head i_unsafe_writes; /* uncommitted sync writes */
+ struct list_head i_unsafe_dirops; /* uncommitted mds dir ops */
+ spinlock_t i_unsafe_lock;
+
+ struct ceph_snap_realm *i_snap_realm; /* snap realm (if caps) */
+ int i_snap_realm_counter; /* snap realm (if caps) */
+ struct list_head i_snap_realm_item;
+ struct list_head i_snap_flush_item;
+
+ struct work_struct i_wb_work; /* writeback work */
+ struct work_struct i_pg_inv_work; /* page invalidation work */
+
+ struct work_struct i_vmtruncate_work;
+
+ struct inode vfs_inode; /* at end */
+};
+
+static inline struct ceph_inode_info *ceph_inode(struct inode *inode)
+{
+ return container_of(inode, struct ceph_inode_info, vfs_inode);
+}
+
+static inline void ceph_i_clear(struct inode *inode, unsigned mask)
+{
+ struct ceph_inode_info *ci = ceph_inode(inode);
+
+ spin_lock(&inode->i_lock);
+ ci->i_ceph_flags &= ~mask;
+ spin_unlock(&inode->i_lock);
+}
+
+static inline void ceph_i_set(struct inode *inode, unsigned mask)
+{
+ struct ceph_inode_info *ci = ceph_inode(inode);
+
+ spin_lock(&inode->i_lock);
+ ci->i_ceph_flags |= mask;
+ spin_unlock(&inode->i_lock);
+}
+
+static inline bool ceph_i_test(struct inode *inode, unsigned mask)
+{
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ bool r;
+
+ smp_mb();
+ r = (ci->i_ceph_flags & mask) == mask;
+ return r;
+}
+
+
+/* find a specific frag @f */
+extern struct ceph_inode_frag *__ceph_find_frag(struct ceph_inode_info *ci,
+ u32 f);
+
+/*
+ * choose fragment for value @v. copy frag content to pfrag, if leaf
+ * exists
+ */
+extern u32 ceph_choose_frag(struct ceph_inode_info *ci, u32 v,
+ struct ceph_inode_frag *pfrag,
+ int *found);
+
+/*
+ * Ceph dentry state
+ */
+struct ceph_dentry_info {
+ struct ceph_mds_session *lease_session;
+ u32 lease_gen, lease_shared_gen;
+ u32 lease_seq;
+ unsigned long lease_renew_after, lease_renew_from;
+ struct list_head lru;
+ struct dentry *dentry;
+ u64 time;
+ u64 offset;
+};
+
+static inline struct ceph_dentry_info *ceph_dentry(struct dentry *dentry)
+{
+ return (struct ceph_dentry_info *)dentry->d_fsdata;
+}
+
+static inline loff_t ceph_make_fpos(unsigned frag, unsigned off)
+{
+ return ((loff_t)frag << 32) | (loff_t)off;
+}
+
+/*
+ * ino_t is <64 bits on many architectures, blech.
+ *
+ * don't include snap in ino hash, at least for now.
+ */
+static inline ino_t ceph_vino_to_ino(struct ceph_vino vino)
+{
+ ino_t ino = (ino_t)vino.ino; /* ^ (vino.snap << 20); */
+#if BITS_PER_LONG == 32
+ ino ^= vino.ino >> (sizeof(u64)-sizeof(ino_t)) * 8;
+ if (!ino)
+ ino = 1;
+#endif
+ return ino;
+}
+
+static inline int ceph_set_ino_cb(struct inode *inode, void *data)
+{
+ ceph_inode(inode)->i_vino = *(struct ceph_vino *)data;
+ inode->i_ino = ceph_vino_to_ino(*(struct ceph_vino *)data);
+ return 0;
+}
+
+static inline struct ceph_vino ceph_vino(struct inode *inode)
+{
+ return ceph_inode(inode)->i_vino;
+}
+
+/* for printf-style formatting */
+#define ceph_vinop(i) ceph_inode(i)->i_vino.ino, ceph_inode(i)->i_vino.snap
+
+static inline u64 ceph_ino(struct inode *inode)
+{
+ return ceph_inode(inode)->i_vino.ino;
+}
+static inline u64 ceph_snap(struct inode *inode)
+{
+ return ceph_inode(inode)->i_vino.snap;
+}
+
+static inline int ceph_ino_compare(struct inode *inode, void *data)
+{
+ struct ceph_vino *pvino = (struct ceph_vino *)data;
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ return ci->i_vino.ino == pvino->ino &&
+ ci->i_vino.snap == pvino->snap;
+}
+
+static inline struct inode *ceph_find_inode(struct super_block *sb,
+ struct ceph_vino vino)
+{
+ ino_t t = ceph_vino_to_ino(vino);
+ return ilookup5(sb, t, ceph_ino_compare, &vino);
+}
+
+
+/*
+ * caps helpers
+ */
+static inline bool __ceph_is_any_real_caps(struct ceph_inode_info *ci)
+{
+ return !RB_EMPTY_ROOT(&ci->i_caps);
+}
+
+extern int __ceph_caps_issued(struct ceph_inode_info *ci, int *implemented);
+extern int __ceph_caps_issued_mask(struct ceph_inode_info *ci, int mask, int t);
+extern int __ceph_caps_issued_other(struct ceph_inode_info *ci,
+ struct ceph_cap *cap);
+
+static inline int ceph_caps_issued(struct ceph_inode_info *ci)
+{
+ int issued;
+ spin_lock(&ci->vfs_inode.i_lock);
+ issued = __ceph_caps_issued(ci, NULL);
+ spin_unlock(&ci->vfs_inode.i_lock);
+ return issued;
+}
+
+static inline int ceph_caps_issued_mask(struct ceph_inode_info *ci, int mask,
+ int touch)
+{
+ int r;
+ spin_lock(&ci->vfs_inode.i_lock);
+ r = __ceph_caps_issued_mask(ci, mask, touch);
+ spin_unlock(&ci->vfs_inode.i_lock);
+ return r;
+}
+
+static inline int __ceph_caps_dirty(struct ceph_inode_info *ci)
+{
+ return ci->i_dirty_caps | ci->i_flushing_caps;
+}
+extern void __ceph_mark_dirty_caps(struct ceph_inode_info *ci, int mask);
+
+extern int ceph_caps_revoking(struct ceph_inode_info *ci, int mask);
+extern int __ceph_caps_used(struct ceph_inode_info *ci);
+
+extern int __ceph_caps_file_wanted(struct ceph_inode_info *ci);
+
+/*
+ * wanted, by virtue of open file modes AND cap refs (buffered/cached data)
+ */
+static inline int __ceph_caps_wanted(struct ceph_inode_info *ci)
+{
+ int w = __ceph_caps_file_wanted(ci) | __ceph_caps_used(ci);
+ if (w & CEPH_CAP_FILE_BUFFER)
+ w |= CEPH_CAP_FILE_EXCL; /* we want EXCL if dirty data */
+ return w;
+}
+
+/* what the mds thinks we want */
+extern int __ceph_caps_mds_wanted(struct ceph_inode_info *ci);
+
+extern void ceph_caps_init(void);
+extern void ceph_caps_finalize(void);
+extern void ceph_adjust_min_caps(int delta);
+extern int ceph_reserve_caps(struct ceph_cap_reservation *ctx, int need);
+extern int ceph_unreserve_caps(struct ceph_cap_reservation *ctx);
+extern void ceph_reservation_status(struct ceph_client *client,
+ int *total, int *avail, int *used,
+ int *reserved, int *min);
+
+static inline struct ceph_client *ceph_inode_to_client(struct inode *inode)
+{
+ return (struct ceph_client *)inode->i_sb->s_fs_info;
+}
+
+static inline struct ceph_client *ceph_sb_to_client(struct super_block *sb)
+{
+ return (struct ceph_client *)sb->s_fs_info;
+}
+
+
+/*
+ * we keep buffered readdir results attached to file->private_data
+ */
+struct ceph_file_info {
+ int fmode; /* initialized on open */
+
+ /* readdir: position within the dir */
+ u32 frag;
+ struct ceph_mds_request *last_readdir;
+ int at_end;
+
+ /* readdir: position within a frag */
+ unsigned offset; /* offset of last chunk, adjusted for . and .. */
+ u64 next_offset; /* offset of next chunk (last_name's + 1) */
+ char *last_name; /* last entry in previous chunk */
+ struct dentry *dentry; /* next dentry (for dcache readdir) */
+ unsigned long dir_release_count;
+
+ /* used for -o dirstat read() on directory thing */
+ char *dir_info;
+ int dir_info_len;
+};
+
+
+
+/*
+ * snapshots
+ */
+
+/*
+ * A "snap context" is the set of existing snapshots when we
+ * write data. It is used by the OSD to guide its COW behavior.
+ *
+ * The ceph_snap_context is refcounted, and attached to each dirty
+ * page, indicating which context the dirty data belonged when it was
+ * dirtied.
+ */
+struct ceph_snap_context {
+ atomic_t nref;
+ u64 seq;
+ int num_snaps;
+ u64 snaps[];
+};
+
+static inline struct ceph_snap_context *
+ceph_get_snap_context(struct ceph_snap_context *sc)
+{
+ /*
+ printk("get_snap_context %p %d -> %d\n", sc, atomic_read(&sc->nref),
+ atomic_read(&sc->nref)+1);
+ */
+ if (sc)
+ atomic_inc(&sc->nref);
+ return sc;
+}
+
+static inline void ceph_put_snap_context(struct ceph_snap_context *sc)
+{
+ if (!sc)
+ return;
+ /*
+ printk("put_snap_context %p %d -> %d\n", sc, atomic_read(&sc->nref),
+ atomic_read(&sc->nref)-1);
+ */
+ if (atomic_dec_and_test(&sc->nref)) {
+ /*printk(" deleting snap_context %p\n", sc);*/
+ kfree(sc);
+ }
+}
+
+/*
+ * A "snap realm" describes a subset of the file hierarchy sharing
+ * the same set of snapshots that apply to it. The realms themselves
+ * are organized into a hierarchy, such that children inherit (some of)
+ * the snapshots of their parents.
+ *
+ * All inodes within the realm that have capabilities are linked into a
+ * per-realm list.
+ */
+struct ceph_snap_realm {
+ u64 ino;
+ atomic_t nref;
+ struct rb_node node;
+
+ u64 created, seq;
+ u64 parent_ino;
+ u64 parent_since; /* snapid when our current parent became so */
+
+ u64 *prior_parent_snaps; /* snaps inherited from any parents we */
+ int num_prior_parent_snaps; /* had prior to parent_since */
+ u64 *snaps; /* snaps specific to this realm */
+ int num_snaps;
+
+ struct ceph_snap_realm *parent;
+ struct list_head children; /* list of child realms */
+ struct list_head child_item;
+
+ struct list_head empty_item; /* if i have ref==0 */
+
+ /* the current set of snaps for this realm */
+ struct ceph_snap_context *cached_context;
+
+ struct list_head inodes_with_caps;
+ spinlock_t inodes_with_caps_lock;
+};
+
+
+
+/*
+ * calculate the number of pages a given length and offset map onto,
+ * if we align the data.
+ */
+static inline int calc_pages_for(u64 off, u64 len)
+{
+ return ((off+len+PAGE_CACHE_SIZE-1) >> PAGE_CACHE_SHIFT) -
+ (off >> PAGE_CACHE_SHIFT);
+}
+
+
+
+/* snap.c */
+struct ceph_snap_realm *ceph_lookup_snap_realm(struct ceph_mds_client *mdsc,
+ u64 ino);
+extern void ceph_get_snap_realm(struct ceph_mds_client *mdsc,
+ struct ceph_snap_realm *realm);
+extern void ceph_put_snap_realm(struct ceph_mds_client *mdsc,
+ struct ceph_snap_realm *realm);
+extern int ceph_update_snap_trace(struct ceph_mds_client *m,
+ void *p, void *e, bool deletion);
+extern void ceph_handle_snap(struct ceph_mds_client *mdsc,
+ struct ceph_mds_session *session,
+ struct ceph_msg *msg);
+extern void ceph_queue_cap_snap(struct ceph_inode_info *ci,
+ struct ceph_snap_context *snapc);
+extern int __ceph_finish_cap_snap(struct ceph_inode_info *ci,
+ struct ceph_cap_snap *capsnap);
+extern void ceph_cleanup_empty_realms(struct ceph_mds_client *mdsc);
+
+/*
+ * a cap_snap is "pending" if it is still awaiting an in-progress
+ * sync write (that may/may not still update size, mtime, etc.).
+ */
+static inline bool __ceph_have_pending_cap_snap(struct ceph_inode_info *ci)
+{
+ return !list_empty(&ci->i_cap_snaps) &&
+ list_entry(ci->i_cap_snaps.prev, struct ceph_cap_snap,
+ ci_item)->writing;
+}
+
+
+/* super.c */
+extern struct kmem_cache *ceph_inode_cachep;
+extern struct kmem_cache *ceph_cap_cachep;
+extern struct kmem_cache *ceph_dentry_cachep;
+extern struct kmem_cache *ceph_file_cachep;
+
+extern const char *ceph_msg_type_name(int type);
+extern int ceph_check_fsid(struct ceph_client *client, struct ceph_fsid *fsid);
+
+#define FSID_FORMAT "%02x%02x%02x%02x-%02x%02x-%02x%02x-%02x%02x-" \
+ "%02x%02x%02x%02x%02x%02x"
+#define PR_FSID(f) (f)->fsid[0], (f)->fsid[1], (f)->fsid[2], (f)->fsid[3], \
+ (f)->fsid[4], (f)->fsid[5], (f)->fsid[6], (f)->fsid[7], \
+ (f)->fsid[8], (f)->fsid[9], (f)->fsid[10], (f)->fsid[11], \
+ (f)->fsid[12], (f)->fsid[13], (f)->fsid[14], (f)->fsid[15]
+
+/* inode.c */
+extern const struct inode_operations ceph_file_iops;
+
+extern struct inode *ceph_alloc_inode(struct super_block *sb);
+extern void ceph_destroy_inode(struct inode *inode);
+
+extern struct inode *ceph_get_inode(struct super_block *sb,
+ struct ceph_vino vino);
+extern struct inode *ceph_get_snapdir(struct inode *parent);
+extern int ceph_fill_file_size(struct inode *inode, int issued,
+ u32 truncate_seq, u64 truncate_size, u64 size);
+extern void ceph_fill_file_time(struct inode *inode, int issued,
+ u64 time_warp_seq, struct timespec *ctime,
+ struct timespec *mtime, struct timespec *atime);
+extern int ceph_fill_trace(struct super_block *sb,
+ struct ceph_mds_request *req,
+ struct ceph_mds_session *session);
+extern int ceph_readdir_prepopulate(struct ceph_mds_request *req,
+ struct ceph_mds_session *session);
+
+extern int ceph_inode_holds_cap(struct inode *inode, int mask);
+
+extern int ceph_inode_set_size(struct inode *inode, loff_t size);
+extern void __ceph_do_pending_vmtruncate(struct inode *inode);
+extern void ceph_queue_vmtruncate(struct inode *inode);
+
+extern void ceph_queue_invalidate(struct inode *inode);
+extern void ceph_queue_writeback(struct inode *inode);
+
+extern int ceph_do_getattr(struct inode *inode, int mask);
+extern int ceph_permission(struct inode *inode, int mask);
+extern int ceph_setattr(struct dentry *dentry, struct iattr *attr);
+extern int ceph_getattr(struct vfsmount *mnt, struct dentry *dentry,
+ struct kstat *stat);
+
+/* xattr.c */
+extern int ceph_setxattr(struct dentry *, const char *, const void *,
+ size_t, int);
+extern ssize_t ceph_getxattr(struct dentry *, const char *, void *, size_t);
+extern ssize_t ceph_listxattr(struct dentry *, char *, size_t);
+extern int ceph_removexattr(struct dentry *, const char *);
+extern void __ceph_build_xattrs_blob(struct ceph_inode_info *ci);
+extern void __ceph_destroy_xattrs(struct ceph_inode_info *ci);
+
+/* caps.c */
+extern const char *ceph_cap_string(int c);
+extern void ceph_handle_caps(struct ceph_mds_session *session,
+ struct ceph_msg *msg);
+extern int ceph_add_cap(struct inode *inode,
+ struct ceph_mds_session *session, u64 cap_id,
+ int fmode, unsigned issued, unsigned wanted,
+ unsigned cap, unsigned seq, u64 realmino, int flags,
+ struct ceph_cap_reservation *caps_reservation);
+extern void __ceph_remove_cap(struct ceph_cap *cap);
+static inline void ceph_remove_cap(struct ceph_cap *cap)
+{
+ struct inode *inode = &cap->ci->vfs_inode;
+ spin_lock(&inode->i_lock);
+ __ceph_remove_cap(cap);
+ spin_unlock(&inode->i_lock);
+}
+extern void ceph_put_cap(struct ceph_cap *cap);
+
+extern void ceph_queue_caps_release(struct inode *inode);
+extern int ceph_write_inode(struct inode *inode, struct writeback_control *wbc);
+extern int ceph_fsync(struct file *file, struct dentry *dentry, int datasync);
+extern void ceph_kick_flushing_caps(struct ceph_mds_client *mdsc,
+ struct ceph_mds_session *session);
+extern int ceph_get_cap_mds(struct inode *inode);
+extern void ceph_get_cap_refs(struct ceph_inode_info *ci, int caps);
+extern void ceph_put_cap_refs(struct ceph_inode_info *ci, int had);
+extern void ceph_put_wrbuffer_cap_refs(struct ceph_inode_info *ci, int nr,
+ struct ceph_snap_context *snapc);
+extern void __ceph_flush_snaps(struct ceph_inode_info *ci,
+ struct ceph_mds_session **psession);
+extern void ceph_check_caps(struct ceph_inode_info *ci, int flags,
+ struct ceph_mds_session *session);
+extern void ceph_check_delayed_caps(struct ceph_mds_client *mdsc);
+extern void ceph_flush_dirty_caps(struct ceph_mds_client *mdsc);
+
+extern int ceph_encode_inode_release(void **p, struct inode *inode,
+ int mds, int drop, int unless, int force);
+extern int ceph_encode_dentry_release(void **p, struct dentry *dn,
+ int mds, int drop, int unless);
+
+extern int ceph_get_caps(struct ceph_inode_info *ci, int need, int want,
+ int *got, loff_t endoff);
+
+/* for counting open files by mode */
+static inline void __ceph_get_fmode(struct ceph_inode_info *ci, int mode)
+{
+ ci->i_nr_by_mode[mode]++;
+}
+extern void ceph_put_fmode(struct ceph_inode_info *ci, int mode);
+
+/* addr.c */
+extern const struct address_space_operations ceph_aops;
+extern int ceph_mmap(struct file *file, struct vm_area_struct *vma);
+
+/* file.c */
+extern const struct file_operations ceph_file_fops;
+extern const struct address_space_operations ceph_aops;
+extern int ceph_open(struct inode *inode, struct file *file);
+extern struct dentry *ceph_lookup_open(struct inode *dir, struct dentry *dentry,
+ struct nameidata *nd, int mode,
+ int locked_dir);
+extern int ceph_release(struct inode *inode, struct file *filp);
+extern void ceph_release_page_vector(struct page **pages, int num_pages);
+
+/* dir.c */
+extern const struct file_operations ceph_dir_fops;
+extern const struct inode_operations ceph_dir_iops;
+extern struct dentry_operations ceph_dentry_ops, ceph_snap_dentry_ops,
+ ceph_snapdir_dentry_ops;
+
+extern int ceph_handle_notrace_create(struct inode *dir, struct dentry *dentry);
+extern struct dentry *ceph_finish_lookup(struct ceph_mds_request *req,
+ struct dentry *dentry, int err);
+
+extern void ceph_dentry_lru_add(struct dentry *dn);
+extern void ceph_dentry_lru_touch(struct dentry *dn);
+extern void ceph_dentry_lru_del(struct dentry *dn);
+
+/*
+ * our d_ops vary depending on whether the inode is live,
+ * snapshotted (read-only), or a virtual ".snap" directory.
+ */
+int ceph_init_dentry(struct dentry *dentry);
+
+
+/* ioctl.c */
+extern long ceph_ioctl(struct file *file, unsigned int cmd, unsigned long arg);
+
+/* export.c */
+extern const struct export_operations ceph_export_ops;
+
+/* debugfs.c */
+extern int ceph_debugfs_init(void);
+extern void ceph_debugfs_cleanup(void);
+extern int ceph_debugfs_client_init(struct ceph_client *client);
+extern void ceph_debugfs_client_cleanup(struct ceph_client *client);
+
+static inline struct inode *get_dentry_parent_inode(struct dentry *dentry)
+{
+ if (dentry && dentry->d_parent)
+ return dentry->d_parent->d_inode;
+
+ return NULL;
+}
+
+#endif /* _FS_CEPH_SUPER_H */
--- /dev/null
+#ifndef _FS_CEPH_TYPES_H
+#define _FS_CEPH_TYPES_H
+
+/* needed before including ceph_fs.h */
+#include <linux/in.h>
+#include <linux/types.h>
+#include <linux/fcntl.h>
+#include <linux/string.h>
+
+#include "ceph_fs.h"
+#include "ceph_frag.h"
+#include "ceph_hash.h"
+
+/*
+ * Identify inodes by both their ino AND snapshot id (a u64).
+ */
+struct ceph_vino {
+ u64 ino;
+ u64 snap;
+};
+
+
+/* context for the caps reservation mechanism */
+struct ceph_cap_reservation {
+ int count;
+};
+
+
+#endif
--- /dev/null
+#include "ceph_debug.h"
+#include "super.h"
+#include "decode.h"
+
+#include <linux/xattr.h>
+#include <linux/slab.h>
+
+static bool ceph_is_valid_xattr(const char *name)
+{
+ return !strncmp(name, XATTR_SECURITY_PREFIX,
+ XATTR_SECURITY_PREFIX_LEN) ||
+ !strncmp(name, XATTR_TRUSTED_PREFIX, XATTR_TRUSTED_PREFIX_LEN) ||
+ !strncmp(name, XATTR_USER_PREFIX, XATTR_USER_PREFIX_LEN);
+}
+
+/*
+ * These define virtual xattrs exposing the recursive directory
+ * statistics and layout metadata.
+ */
+struct ceph_vxattr_cb {
+ bool readonly;
+ char *name;
+ size_t (*getxattr_cb)(struct ceph_inode_info *ci, char *val,
+ size_t size);
+};
+
+/* directories */
+
+static size_t ceph_vxattrcb_entries(struct ceph_inode_info *ci, char *val,
+ size_t size)
+{
+ return snprintf(val, size, "%lld", ci->i_files + ci->i_subdirs);
+}
+
+static size_t ceph_vxattrcb_files(struct ceph_inode_info *ci, char *val,
+ size_t size)
+{
+ return snprintf(val, size, "%lld", ci->i_files);
+}
+
+static size_t ceph_vxattrcb_subdirs(struct ceph_inode_info *ci, char *val,
+ size_t size)
+{
+ return snprintf(val, size, "%lld", ci->i_subdirs);
+}
+
+static size_t ceph_vxattrcb_rentries(struct ceph_inode_info *ci, char *val,
+ size_t size)
+{
+ return snprintf(val, size, "%lld", ci->i_rfiles + ci->i_rsubdirs);
+}
+
+static size_t ceph_vxattrcb_rfiles(struct ceph_inode_info *ci, char *val,
+ size_t size)
+{
+ return snprintf(val, size, "%lld", ci->i_rfiles);
+}
+
+static size_t ceph_vxattrcb_rsubdirs(struct ceph_inode_info *ci, char *val,
+ size_t size)
+{
+ return snprintf(val, size, "%lld", ci->i_rsubdirs);
+}
+
+static size_t ceph_vxattrcb_rbytes(struct ceph_inode_info *ci, char *val,
+ size_t size)
+{
+ return snprintf(val, size, "%lld", ci->i_rbytes);
+}
+
+static size_t ceph_vxattrcb_rctime(struct ceph_inode_info *ci, char *val,
+ size_t size)
+{
+ return snprintf(val, size, "%ld.%ld", (long)ci->i_rctime.tv_sec,
+ (long)ci->i_rctime.tv_nsec);
+}
+
+static struct ceph_vxattr_cb ceph_dir_vxattrs[] = {
+ { true, "user.ceph.dir.entries", ceph_vxattrcb_entries},
+ { true, "user.ceph.dir.files", ceph_vxattrcb_files},
+ { true, "user.ceph.dir.subdirs", ceph_vxattrcb_subdirs},
+ { true, "user.ceph.dir.rentries", ceph_vxattrcb_rentries},
+ { true, "user.ceph.dir.rfiles", ceph_vxattrcb_rfiles},
+ { true, "user.ceph.dir.rsubdirs", ceph_vxattrcb_rsubdirs},
+ { true, "user.ceph.dir.rbytes", ceph_vxattrcb_rbytes},
+ { true, "user.ceph.dir.rctime", ceph_vxattrcb_rctime},
+ { true, NULL, NULL }
+};
+
+/* files */
+
+static size_t ceph_vxattrcb_layout(struct ceph_inode_info *ci, char *val,
+ size_t size)
+{
+ int ret;
+
+ ret = snprintf(val, size,
+ "chunk_bytes=%lld\nstripe_count=%lld\nobject_size=%lld\n",
+ (unsigned long long)ceph_file_layout_su(ci->i_layout),
+ (unsigned long long)ceph_file_layout_stripe_count(ci->i_layout),
+ (unsigned long long)ceph_file_layout_object_size(ci->i_layout));
+ if (ceph_file_layout_pg_preferred(ci->i_layout))
+ ret += snprintf(val + ret, size, "preferred_osd=%lld\n",
+ (unsigned long long)ceph_file_layout_pg_preferred(
+ ci->i_layout));
+ return ret;
+}
+
+static struct ceph_vxattr_cb ceph_file_vxattrs[] = {
+ { true, "user.ceph.layout", ceph_vxattrcb_layout},
+ { NULL, NULL }
+};
+
+static struct ceph_vxattr_cb *ceph_inode_vxattrs(struct inode *inode)
+{
+ if (S_ISDIR(inode->i_mode))
+ return ceph_dir_vxattrs;
+ else if (S_ISREG(inode->i_mode))
+ return ceph_file_vxattrs;
+ return NULL;
+}
+
+static struct ceph_vxattr_cb *ceph_match_vxattr(struct ceph_vxattr_cb *vxattr,
+ const char *name)
+{
+ do {
+ if (strcmp(vxattr->name, name) == 0)
+ return vxattr;
+ vxattr++;
+ } while (vxattr->name);
+ return NULL;
+}
+
+static int __set_xattr(struct ceph_inode_info *ci,
+ const char *name, int name_len,
+ const char *val, int val_len,
+ int dirty,
+ int should_free_name, int should_free_val,
+ struct ceph_inode_xattr **newxattr)
+{
+ struct rb_node **p;
+ struct rb_node *parent = NULL;
+ struct ceph_inode_xattr *xattr = NULL;
+ int c;
+ int new = 0;
+
+ p = &ci->i_xattrs.index.rb_node;
+ while (*p) {
+ parent = *p;
+ xattr = rb_entry(parent, struct ceph_inode_xattr, node);
+ c = strncmp(name, xattr->name, min(name_len, xattr->name_len));
+ if (c < 0)
+ p = &(*p)->rb_left;
+ else if (c > 0)
+ p = &(*p)->rb_right;
+ else {
+ if (name_len == xattr->name_len)
+ break;
+ else if (name_len < xattr->name_len)
+ p = &(*p)->rb_left;
+ else
+ p = &(*p)->rb_right;
+ }
+ xattr = NULL;
+ }
+
+ if (!xattr) {
+ new = 1;
+ xattr = *newxattr;
+ xattr->name = name;
+ xattr->name_len = name_len;
+ xattr->should_free_name = should_free_name;
+
+ ci->i_xattrs.count++;
+ dout("__set_xattr count=%d\n", ci->i_xattrs.count);
+ } else {
+ kfree(*newxattr);
+ *newxattr = NULL;
+ if (xattr->should_free_val)
+ kfree((void *)xattr->val);
+
+ if (should_free_name) {
+ kfree((void *)name);
+ name = xattr->name;
+ }
+ ci->i_xattrs.names_size -= xattr->name_len;
+ ci->i_xattrs.vals_size -= xattr->val_len;
+ }
+ if (!xattr) {
+ pr_err("__set_xattr ENOMEM on %p %llx.%llx xattr %s=%s\n",
+ &ci->vfs_inode, ceph_vinop(&ci->vfs_inode), name,
+ xattr->val);
+ return -ENOMEM;
+ }
+ ci->i_xattrs.names_size += name_len;
+ ci->i_xattrs.vals_size += val_len;
+ if (val)
+ xattr->val = val;
+ else
+ xattr->val = "";
+
+ xattr->val_len = val_len;
+ xattr->dirty = dirty;
+ xattr->should_free_val = (val && should_free_val);
+
+ if (new) {
+ rb_link_node(&xattr->node, parent, p);
+ rb_insert_color(&xattr->node, &ci->i_xattrs.index);
+ dout("__set_xattr_val p=%p\n", p);
+ }
+
+ dout("__set_xattr_val added %llx.%llx xattr %p %s=%.*s\n",
+ ceph_vinop(&ci->vfs_inode), xattr, name, val_len, val);
+
+ return 0;
+}
+
+static struct ceph_inode_xattr *__get_xattr(struct ceph_inode_info *ci,
+ const char *name)
+{
+ struct rb_node **p;
+ struct rb_node *parent = NULL;
+ struct ceph_inode_xattr *xattr = NULL;
+ int c;
+
+ p = &ci->i_xattrs.index.rb_node;
+ while (*p) {
+ parent = *p;
+ xattr = rb_entry(parent, struct ceph_inode_xattr, node);
+ c = strncmp(name, xattr->name, xattr->name_len);
+ if (c < 0)
+ p = &(*p)->rb_left;
+ else if (c > 0)
+ p = &(*p)->rb_right;
+ else {
+ dout("__get_xattr %s: found %.*s\n", name,
+ xattr->val_len, xattr->val);
+ return xattr;
+ }
+ }
+
+ dout("__get_xattr %s: not found\n", name);
+
+ return NULL;
+}
+
+static void __free_xattr(struct ceph_inode_xattr *xattr)
+{
+ BUG_ON(!xattr);
+
+ if (xattr->should_free_name)
+ kfree((void *)xattr->name);
+ if (xattr->should_free_val)
+ kfree((void *)xattr->val);
+
+ kfree(xattr);
+}
+
+static int __remove_xattr(struct ceph_inode_info *ci,
+ struct ceph_inode_xattr *xattr)
+{
+ if (!xattr)
+ return -EOPNOTSUPP;
+
+ rb_erase(&xattr->node, &ci->i_xattrs.index);
+
+ if (xattr->should_free_name)
+ kfree((void *)xattr->name);
+ if (xattr->should_free_val)
+ kfree((void *)xattr->val);
+
+ ci->i_xattrs.names_size -= xattr->name_len;
+ ci->i_xattrs.vals_size -= xattr->val_len;
+ ci->i_xattrs.count--;
+ kfree(xattr);
+
+ return 0;
+}
+
+static int __remove_xattr_by_name(struct ceph_inode_info *ci,
+ const char *name)
+{
+ struct rb_node **p;
+ struct ceph_inode_xattr *xattr;
+ int err;
+
+ p = &ci->i_xattrs.index.rb_node;
+ xattr = __get_xattr(ci, name);
+ err = __remove_xattr(ci, xattr);
+ return err;
+}
+
+static char *__copy_xattr_names(struct ceph_inode_info *ci,
+ char *dest)
+{
+ struct rb_node *p;
+ struct ceph_inode_xattr *xattr = NULL;
+
+ p = rb_first(&ci->i_xattrs.index);
+ dout("__copy_xattr_names count=%d\n", ci->i_xattrs.count);
+
+ while (p) {
+ xattr = rb_entry(p, struct ceph_inode_xattr, node);
+ memcpy(dest, xattr->name, xattr->name_len);
+ dest[xattr->name_len] = '\0';
+
+ dout("dest=%s %p (%s) (%d/%d)\n", dest, xattr, xattr->name,
+ xattr->name_len, ci->i_xattrs.names_size);
+
+ dest += xattr->name_len + 1;
+ p = rb_next(p);
+ }
+
+ return dest;
+}
+
+void __ceph_destroy_xattrs(struct ceph_inode_info *ci)
+{
+ struct rb_node *p, *tmp;
+ struct ceph_inode_xattr *xattr = NULL;
+
+ p = rb_first(&ci->i_xattrs.index);
+
+ dout("__ceph_destroy_xattrs p=%p\n", p);
+
+ while (p) {
+ xattr = rb_entry(p, struct ceph_inode_xattr, node);
+ tmp = p;
+ p = rb_next(tmp);
+ dout("__ceph_destroy_xattrs next p=%p (%.*s)\n", p,
+ xattr->name_len, xattr->name);
+ rb_erase(tmp, &ci->i_xattrs.index);
+
+ __free_xattr(xattr);
+ }
+
+ ci->i_xattrs.names_size = 0;
+ ci->i_xattrs.vals_size = 0;
+ ci->i_xattrs.index_version = 0;
+ ci->i_xattrs.count = 0;
+ ci->i_xattrs.index = RB_ROOT;
+}
+
+static int __build_xattrs(struct inode *inode)
+{
+ u32 namelen;
+ u32 numattr = 0;
+ void *p, *end;
+ u32 len;
+ const char *name, *val;
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ int xattr_version;
+ struct ceph_inode_xattr **xattrs = NULL;
+ int err = 0;
+ int i;
+
+ dout("__build_xattrs() len=%d\n",
+ ci->i_xattrs.blob ? (int)ci->i_xattrs.blob->vec.iov_len : 0);
+
+ if (ci->i_xattrs.index_version >= ci->i_xattrs.version)
+ return 0; /* already built */
+
+ __ceph_destroy_xattrs(ci);
+
+start:
+ /* updated internal xattr rb tree */
+ if (ci->i_xattrs.blob && ci->i_xattrs.blob->vec.iov_len > 4) {
+ p = ci->i_xattrs.blob->vec.iov_base;
+ end = p + ci->i_xattrs.blob->vec.iov_len;
+ ceph_decode_32_safe(&p, end, numattr, bad);
+ xattr_version = ci->i_xattrs.version;
+ spin_unlock(&inode->i_lock);
+
+ xattrs = kcalloc(numattr, sizeof(struct ceph_xattr *),
+ GFP_NOFS);
+ err = -ENOMEM;
+ if (!xattrs)
+ goto bad_lock;
+ memset(xattrs, 0, numattr*sizeof(struct ceph_xattr *));
+ for (i = 0; i < numattr; i++) {
+ xattrs[i] = kmalloc(sizeof(struct ceph_inode_xattr),
+ GFP_NOFS);
+ if (!xattrs[i])
+ goto bad_lock;
+ }
+
+ spin_lock(&inode->i_lock);
+ if (ci->i_xattrs.version != xattr_version) {
+ /* lost a race, retry */
+ for (i = 0; i < numattr; i++)
+ kfree(xattrs[i]);
+ kfree(xattrs);
+ goto start;
+ }
+ err = -EIO;
+ while (numattr--) {
+ ceph_decode_32_safe(&p, end, len, bad);
+ namelen = len;
+ name = p;
+ p += len;
+ ceph_decode_32_safe(&p, end, len, bad);
+ val = p;
+ p += len;
+
+ err = __set_xattr(ci, name, namelen, val, len,
+ 0, 0, 0, &xattrs[numattr]);
+
+ if (err < 0)
+ goto bad;
+ }
+ kfree(xattrs);
+ }
+ ci->i_xattrs.index_version = ci->i_xattrs.version;
+ ci->i_xattrs.dirty = false;
+
+ return err;
+bad_lock:
+ spin_lock(&inode->i_lock);
+bad:
+ if (xattrs) {
+ for (i = 0; i < numattr; i++)
+ kfree(xattrs[i]);
+ kfree(xattrs);
+ }
+ ci->i_xattrs.names_size = 0;
+ return err;
+}
+
+static int __get_required_blob_size(struct ceph_inode_info *ci, int name_size,
+ int val_size)
+{
+ /*
+ * 4 bytes for the length, and additional 4 bytes per each xattr name,
+ * 4 bytes per each value
+ */
+ int size = 4 + ci->i_xattrs.count*(4 + 4) +
+ ci->i_xattrs.names_size +
+ ci->i_xattrs.vals_size;
+ dout("__get_required_blob_size c=%d names.size=%d vals.size=%d\n",
+ ci->i_xattrs.count, ci->i_xattrs.names_size,
+ ci->i_xattrs.vals_size);
+
+ if (name_size)
+ size += 4 + 4 + name_size + val_size;
+
+ return size;
+}
+
+/*
+ * If there are dirty xattrs, reencode xattrs into the prealloc_blob
+ * and swap into place.
+ */
+void __ceph_build_xattrs_blob(struct ceph_inode_info *ci)
+{
+ struct rb_node *p;
+ struct ceph_inode_xattr *xattr = NULL;
+ void *dest;
+
+ dout("__build_xattrs_blob %p\n", &ci->vfs_inode);
+ if (ci->i_xattrs.dirty) {
+ int need = __get_required_blob_size(ci, 0, 0);
+
+ BUG_ON(need > ci->i_xattrs.prealloc_blob->alloc_len);
+
+ p = rb_first(&ci->i_xattrs.index);
+ dest = ci->i_xattrs.prealloc_blob->vec.iov_base;
+
+ ceph_encode_32(&dest, ci->i_xattrs.count);
+ while (p) {
+ xattr = rb_entry(p, struct ceph_inode_xattr, node);
+
+ ceph_encode_32(&dest, xattr->name_len);
+ memcpy(dest, xattr->name, xattr->name_len);
+ dest += xattr->name_len;
+ ceph_encode_32(&dest, xattr->val_len);
+ memcpy(dest, xattr->val, xattr->val_len);
+ dest += xattr->val_len;
+
+ p = rb_next(p);
+ }
+
+ /* adjust buffer len; it may be larger than we need */
+ ci->i_xattrs.prealloc_blob->vec.iov_len =
+ dest - ci->i_xattrs.prealloc_blob->vec.iov_base;
+
+ if (ci->i_xattrs.blob)
+ ceph_buffer_put(ci->i_xattrs.blob);
+ ci->i_xattrs.blob = ci->i_xattrs.prealloc_blob;
+ ci->i_xattrs.prealloc_blob = NULL;
+ ci->i_xattrs.dirty = false;
+ }
+}
+
+ssize_t ceph_getxattr(struct dentry *dentry, const char *name, void *value,
+ size_t size)
+{
+ struct inode *inode = dentry->d_inode;
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ struct ceph_vxattr_cb *vxattrs = ceph_inode_vxattrs(inode);
+ int err;
+ struct ceph_inode_xattr *xattr;
+ struct ceph_vxattr_cb *vxattr = NULL;
+
+ if (!ceph_is_valid_xattr(name))
+ return -ENODATA;
+
+ /* let's see if a virtual xattr was requested */
+ if (vxattrs)
+ vxattr = ceph_match_vxattr(vxattrs, name);
+
+ spin_lock(&inode->i_lock);
+ dout("getxattr %p ver=%lld index_ver=%lld\n", inode,
+ ci->i_xattrs.version, ci->i_xattrs.index_version);
+
+ if (__ceph_caps_issued_mask(ci, CEPH_CAP_XATTR_SHARED, 1) &&
+ (ci->i_xattrs.index_version >= ci->i_xattrs.version)) {
+ goto get_xattr;
+ } else {
+ spin_unlock(&inode->i_lock);
+ /* get xattrs from mds (if we don't already have them) */
+ err = ceph_do_getattr(inode, CEPH_STAT_CAP_XATTR);
+ if (err)
+ return err;
+ }
+
+ spin_lock(&inode->i_lock);
+
+ if (vxattr && vxattr->readonly) {
+ err = vxattr->getxattr_cb(ci, value, size);
+ goto out;
+ }
+
+ err = __build_xattrs(inode);
+ if (err < 0)
+ goto out;
+
+get_xattr:
+ err = -ENODATA; /* == ENOATTR */
+ xattr = __get_xattr(ci, name);
+ if (!xattr) {
+ if (vxattr)
+ err = vxattr->getxattr_cb(ci, value, size);
+ goto out;
+ }
+
+ err = -ERANGE;
+ if (size && size < xattr->val_len)
+ goto out;
+
+ err = xattr->val_len;
+ if (size == 0)
+ goto out;
+
+ memcpy(value, xattr->val, xattr->val_len);
+
+out:
+ spin_unlock(&inode->i_lock);
+ return err;
+}
+
+ssize_t ceph_listxattr(struct dentry *dentry, char *names, size_t size)
+{
+ struct inode *inode = dentry->d_inode;
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ struct ceph_vxattr_cb *vxattrs = ceph_inode_vxattrs(inode);
+ u32 vir_namelen = 0;
+ u32 namelen;
+ int err;
+ u32 len;
+ int i;
+
+ spin_lock(&inode->i_lock);
+ dout("listxattr %p ver=%lld index_ver=%lld\n", inode,
+ ci->i_xattrs.version, ci->i_xattrs.index_version);
+
+ if (__ceph_caps_issued_mask(ci, CEPH_CAP_XATTR_SHARED, 1) &&
+ (ci->i_xattrs.index_version > ci->i_xattrs.version)) {
+ goto list_xattr;
+ } else {
+ spin_unlock(&inode->i_lock);
+ err = ceph_do_getattr(inode, CEPH_STAT_CAP_XATTR);
+ if (err)
+ return err;
+ }
+
+ spin_lock(&inode->i_lock);
+
+ err = __build_xattrs(inode);
+ if (err < 0)
+ goto out;
+
+list_xattr:
+ vir_namelen = 0;
+ /* include virtual dir xattrs */
+ if (vxattrs)
+ for (i = 0; vxattrs[i].name; i++)
+ vir_namelen += strlen(vxattrs[i].name) + 1;
+ /* adding 1 byte per each variable due to the null termination */
+ namelen = vir_namelen + ci->i_xattrs.names_size + ci->i_xattrs.count;
+ err = -ERANGE;
+ if (size && namelen > size)
+ goto out;
+
+ err = namelen;
+ if (size == 0)
+ goto out;
+
+ names = __copy_xattr_names(ci, names);
+
+ /* virtual xattr names, too */
+ if (vxattrs)
+ for (i = 0; vxattrs[i].name; i++) {
+ len = sprintf(names, "%s", vxattrs[i].name);
+ names += len + 1;
+ }
+
+out:
+ spin_unlock(&inode->i_lock);
+ return err;
+}
+
+static int ceph_sync_setxattr(struct dentry *dentry, const char *name,
+ const char *value, size_t size, int flags)
+{
+ struct ceph_client *client = ceph_client(dentry->d_sb);
+ struct inode *inode = dentry->d_inode;
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ struct inode *parent_inode = dentry->d_parent->d_inode;
+ struct ceph_mds_request *req;
+ struct ceph_mds_client *mdsc = &client->mdsc;
+ int err;
+ int i, nr_pages;
+ struct page **pages = NULL;
+ void *kaddr;
+
+ /* copy value into some pages */
+ nr_pages = calc_pages_for(0, size);
+ if (nr_pages) {
+ pages = kmalloc(sizeof(pages[0])*nr_pages, GFP_NOFS);
+ if (!pages)
+ return -ENOMEM;
+ err = -ENOMEM;
+ for (i = 0; i < nr_pages; i++) {
+ pages[i] = alloc_page(GFP_NOFS);
+ if (!pages[i]) {
+ nr_pages = i;
+ goto out;
+ }
+ kaddr = kmap(pages[i]);
+ memcpy(kaddr, value + i*PAGE_CACHE_SIZE,
+ min(PAGE_CACHE_SIZE, size-i*PAGE_CACHE_SIZE));
+ }
+ }
+
+ dout("setxattr value=%.*s\n", (int)size, value);
+
+ /* do request */
+ req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_SETXATTR,
+ USE_AUTH_MDS);
+ if (IS_ERR(req)) {
+ err = PTR_ERR(req);
+ goto out;
+ }
+ req->r_inode = igrab(inode);
+ req->r_inode_drop = CEPH_CAP_XATTR_SHARED;
+ req->r_num_caps = 1;
+ req->r_args.setxattr.flags = cpu_to_le32(flags);
+ req->r_path2 = kstrdup(name, GFP_NOFS);
+
+ req->r_pages = pages;
+ req->r_num_pages = nr_pages;
+ req->r_data_len = size;
+
+ dout("xattr.ver (before): %lld\n", ci->i_xattrs.version);
+ err = ceph_mdsc_do_request(mdsc, parent_inode, req);
+ ceph_mdsc_put_request(req);
+ dout("xattr.ver (after): %lld\n", ci->i_xattrs.version);
+
+out:
+ if (pages) {
+ for (i = 0; i < nr_pages; i++)
+ __free_page(pages[i]);
+ kfree(pages);
+ }
+ return err;
+}
+
+int ceph_setxattr(struct dentry *dentry, const char *name,
+ const void *value, size_t size, int flags)
+{
+ struct inode *inode = dentry->d_inode;
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ struct ceph_vxattr_cb *vxattrs = ceph_inode_vxattrs(inode);
+ int err;
+ int name_len = strlen(name);
+ int val_len = size;
+ char *newname = NULL;
+ char *newval = NULL;
+ struct ceph_inode_xattr *xattr = NULL;
+ int issued;
+ int required_blob_size;
+
+ if (ceph_snap(inode) != CEPH_NOSNAP)
+ return -EROFS;
+
+ if (!ceph_is_valid_xattr(name))
+ return -EOPNOTSUPP;
+
+ if (vxattrs) {
+ struct ceph_vxattr_cb *vxattr =
+ ceph_match_vxattr(vxattrs, name);
+ if (vxattr && vxattr->readonly)
+ return -EOPNOTSUPP;
+ }
+
+ /* preallocate memory for xattr name, value, index node */
+ err = -ENOMEM;
+ newname = kmalloc(name_len + 1, GFP_NOFS);
+ if (!newname)
+ goto out;
+ memcpy(newname, name, name_len + 1);
+
+ if (val_len) {
+ newval = kmalloc(val_len + 1, GFP_NOFS);
+ if (!newval)
+ goto out;
+ memcpy(newval, value, val_len);
+ newval[val_len] = '\0';
+ }
+
+ xattr = kmalloc(sizeof(struct ceph_inode_xattr), GFP_NOFS);
+ if (!xattr)
+ goto out;
+
+ spin_lock(&inode->i_lock);
+retry:
+ issued = __ceph_caps_issued(ci, NULL);
+ if (!(issued & CEPH_CAP_XATTR_EXCL))
+ goto do_sync;
+ __build_xattrs(inode);
+
+ required_blob_size = __get_required_blob_size(ci, name_len, val_len);
+
+ if (!ci->i_xattrs.prealloc_blob ||
+ required_blob_size > ci->i_xattrs.prealloc_blob->alloc_len) {
+ struct ceph_buffer *blob = NULL;
+
+ spin_unlock(&inode->i_lock);
+ dout(" preaallocating new blob size=%d\n", required_blob_size);
+ blob = ceph_buffer_new(required_blob_size, GFP_NOFS);
+ if (!blob)
+ goto out;
+ spin_lock(&inode->i_lock);
+ if (ci->i_xattrs.prealloc_blob)
+ ceph_buffer_put(ci->i_xattrs.prealloc_blob);
+ ci->i_xattrs.prealloc_blob = blob;
+ goto retry;
+ }
+
+ dout("setxattr %p issued %s\n", inode, ceph_cap_string(issued));
+ err = __set_xattr(ci, newname, name_len, newval,
+ val_len, 1, 1, 1, &xattr);
+ __ceph_mark_dirty_caps(ci, CEPH_CAP_XATTR_EXCL);
+ ci->i_xattrs.dirty = true;
+ inode->i_ctime = CURRENT_TIME;
+ spin_unlock(&inode->i_lock);
+
+ return err;
+
+do_sync:
+ spin_unlock(&inode->i_lock);
+ err = ceph_sync_setxattr(dentry, name, value, size, flags);
+out:
+ kfree(newname);
+ kfree(newval);
+ kfree(xattr);
+ return err;
+}
+
+static int ceph_send_removexattr(struct dentry *dentry, const char *name)
+{
+ struct ceph_client *client = ceph_client(dentry->d_sb);
+ struct ceph_mds_client *mdsc = &client->mdsc;
+ struct inode *inode = dentry->d_inode;
+ struct inode *parent_inode = dentry->d_parent->d_inode;
+ struct ceph_mds_request *req;
+ int err;
+
+ req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_RMXATTR,
+ USE_AUTH_MDS);
+ if (IS_ERR(req))
+ return PTR_ERR(req);
+ req->r_inode = igrab(inode);
+ req->r_inode_drop = CEPH_CAP_XATTR_SHARED;
+ req->r_num_caps = 1;
+ req->r_path2 = kstrdup(name, GFP_NOFS);
+
+ err = ceph_mdsc_do_request(mdsc, parent_inode, req);
+ ceph_mdsc_put_request(req);
+ return err;
+}
+
+int ceph_removexattr(struct dentry *dentry, const char *name)
+{
+ struct inode *inode = dentry->d_inode;
+ struct ceph_inode_info *ci = ceph_inode(inode);
+ struct ceph_vxattr_cb *vxattrs = ceph_inode_vxattrs(inode);
+ int issued;
+ int err;
+
+ if (ceph_snap(inode) != CEPH_NOSNAP)
+ return -EROFS;
+
+ if (!ceph_is_valid_xattr(name))
+ return -EOPNOTSUPP;
+
+ if (vxattrs) {
+ struct ceph_vxattr_cb *vxattr =
+ ceph_match_vxattr(vxattrs, name);
+ if (vxattr && vxattr->readonly)
+ return -EOPNOTSUPP;
+ }
+
+ spin_lock(&inode->i_lock);
+ __build_xattrs(inode);
+ issued = __ceph_caps_issued(ci, NULL);
+ dout("removexattr %p issued %s\n", inode, ceph_cap_string(issued));
+
+ if (!(issued & CEPH_CAP_XATTR_EXCL))
+ goto do_sync;
+
+ err = __remove_xattr_by_name(ceph_inode(inode), name);
+ __ceph_mark_dirty_caps(ci, CEPH_CAP_XATTR_EXCL);
+ ci->i_xattrs.dirty = true;
+ inode->i_ctime = CURRENT_TIME;
+
+ spin_unlock(&inode->i_lock);
+
+ return err;
+do_sync:
+ spin_unlock(&inode->i_lock);
+ err = ceph_send_removexattr(dentry, name);
+ return err;
+}
+
#include <linux/dcache.h>
#include <linux/mount.h>
#include <linux/namei.h>
+#include <linux/slab.h>
#include <linux/vfs.h>
#include <linux/fs.h>
#include "cifsglob.h"
*/
#include <linux/list.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include <keys/user-type.h>
#include <linux/key-type.h>
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#include <linux/fs.h>
+#include <linux/slab.h>
#include "cifs_unicode.h"
#include "cifs_uniupr.h"
#include "cifspdu.h"
*/
#include <linux/fs.h>
+#include <linux/slab.h>
#include "cifspdu.h"
#include "cifsglob.h"
#include "cifsacl.h"
*/
#include <linux/fs.h>
+#include <linux/slab.h>
#include "cifspdu.h"
#include "cifsglob.h"
#include "cifs_debug.h"
cifs_inode->clientCanCacheRead = false;
cifs_inode->clientCanCacheAll = false;
cifs_inode->delete_pending = false;
+ cifs_inode->invalid_mapping = false;
cifs_inode->vfs_inode.i_blkbits = 14; /* 2**14 = CIFS_MAX_MSGSIZE */
cifs_inode->server_eof = 0;
setting the revalidate time to zero */
CIFS_I(file->f_path.dentry->d_inode)->time = 0;
- retval = cifs_revalidate(file->f_path.dentry);
+ retval = cifs_revalidate_file(file);
if (retval < 0)
return (loff_t)retval;
}
extern int cifs_rmdir(struct inode *, struct dentry *);
extern int cifs_rename(struct inode *, struct dentry *, struct inode *,
struct dentry *);
-extern int cifs_revalidate(struct dentry *);
+extern int cifs_revalidate_file(struct file *filp);
+extern int cifs_revalidate_dentry(struct dentry *);
extern int cifs_getattr(struct vfsmount *, struct dentry *, struct kstat *);
extern int cifs_setattr(struct dentry *, struct iattr *);
*/
#include <linux/in.h>
#include <linux/in6.h>
+#include <linux/slab.h>
#include <linux/slow-work.h>
#include "cifs_fs_sb.h"
#include "cifsacl.h"
bool clientCanCacheRead:1; /* read oplock */
bool clientCanCacheAll:1; /* read and writebehind oplock */
bool delete_pending:1; /* DELETE_ON_CLOSE is set */
+ bool invalid_mapping:1; /* pagecache is invalid */
u64 server_eof; /* current file size on server */
u64 uniqueid; /* server inode number */
struct inode vfs_inode;
extern struct inode *cifs_iget(struct super_block *sb,
struct cifs_fattr *fattr);
+extern int cifs_get_file_info(struct file *filp);
extern int cifs_get_inode_info(struct inode **pinode,
const unsigned char *search_path,
FILE_ALL_INFO *pfile_info,
struct super_block *sb, int xid, const __u16 *pfid);
+extern int cifs_get_file_info_unix(struct file *filp);
extern int cifs_get_inode_info_unix(struct inode **pinode,
const unsigned char *search_path,
struct super_block *sb, int xid);
extern int CIFSFindClose(const int, struct cifsTconInfo *tcon,
const __u16 search_handle);
+extern int CIFSSMBQFileInfo(const int xid, struct cifsTconInfo *tcon,
+ u16 netfid, FILE_ALL_INFO *pFindData);
extern int CIFSSMBQPathInfo(const int xid, struct cifsTconInfo *tcon,
const unsigned char *searchName,
FILE_ALL_INFO *findData,
FILE_ALL_INFO *findData,
const struct nls_table *nls_codepage, int remap);
+extern int CIFSSMBUnixQFileInfo(const int xid, struct cifsTconInfo *tcon,
+ u16 netfid, FILE_UNIX_BASIC_INFO *pFindData);
extern int CIFSSMBUnixQPathInfo(const int xid,
struct cifsTconInfo *tcon,
const unsigned char *searchName,
#include <linux/fs.h>
#include <linux/kernel.h>
#include <linux/vfs.h>
+#include <linux/slab.h>
#include <linux/posix_acl_xattr.h>
#include <asm/uaccess.h>
#include "cifspdu.h"
} else if (pSMBr->hdr.WordCount == 13) {
cERROR(1, ("mount failed, cifs module not built "
"with CIFS_WEAK_PW_HASH support"));
- rc = -EOPNOTSUPP;
+ rc = -EOPNOTSUPP;
#endif /* WEAK_PW_HASH */
goto neg_err_exit;
} else if (pSMBr->hdr.WordCount != 17) {
return rc;
}
+int
+CIFSSMBQFileInfo(const int xid, struct cifsTconInfo *tcon,
+ u16 netfid, FILE_ALL_INFO *pFindData)
+{
+ struct smb_t2_qfi_req *pSMB = NULL;
+ struct smb_t2_qfi_rsp *pSMBr = NULL;
+ int rc = 0;
+ int bytes_returned;
+ __u16 params, byte_count;
+QFileInfoRetry:
+ rc = smb_init(SMB_COM_TRANSACTION2, 15, tcon, (void **) &pSMB,
+ (void **) &pSMBr);
+ if (rc)
+ return rc;
+ params = 2 /* level */ + 2 /* fid */;
+ pSMB->t2.TotalDataCount = 0;
+ pSMB->t2.MaxParameterCount = cpu_to_le16(4);
+ /* BB find exact max data count below from sess structure BB */
+ pSMB->t2.MaxDataCount = cpu_to_le16(CIFSMaxBufSize);
+ pSMB->t2.MaxSetupCount = 0;
+ pSMB->t2.Reserved = 0;
+ pSMB->t2.Flags = 0;
+ pSMB->t2.Timeout = 0;
+ pSMB->t2.Reserved2 = 0;
+ pSMB->t2.ParameterOffset = cpu_to_le16(offsetof(struct smb_t2_qfi_req,
+ Fid) - 4);
+ pSMB->t2.DataCount = 0;
+ pSMB->t2.DataOffset = 0;
+ pSMB->t2.SetupCount = 1;
+ pSMB->t2.Reserved3 = 0;
+ pSMB->t2.SubCommand = cpu_to_le16(TRANS2_QUERY_FILE_INFORMATION);
+ byte_count = params + 1 /* pad */ ;
+ pSMB->t2.TotalParameterCount = cpu_to_le16(params);
+ pSMB->t2.ParameterCount = pSMB->t2.TotalParameterCount;
+ pSMB->InformationLevel = cpu_to_le16(SMB_QUERY_FILE_ALL_INFO);
+ pSMB->Pad = 0;
+ pSMB->Fid = netfid;
+ pSMB->hdr.smb_buf_length += byte_count;
+
+ rc = SendReceive(xid, tcon->ses, (struct smb_hdr *) pSMB,
+ (struct smb_hdr *) pSMBr, &bytes_returned, 0);
+ if (rc) {
+ cFYI(1, ("Send error in QPathInfo = %d", rc));
+ } else { /* decode response */
+ rc = validate_t2((struct smb_t2_rsp *)pSMBr);
+
+ if (rc) /* BB add auto retry on EOPNOTSUPP? */
+ rc = -EIO;
+ else if (pSMBr->ByteCount < 40)
+ rc = -EIO; /* bad smb */
+ else if (pFindData) {
+ __u16 data_offset = le16_to_cpu(pSMBr->t2.DataOffset);
+ memcpy((char *) pFindData,
+ (char *) &pSMBr->hdr.Protocol +
+ data_offset, sizeof(FILE_ALL_INFO));
+ } else
+ rc = -ENOMEM;
+ }
+ cifs_buf_release(pSMB);
+ if (rc == -EAGAIN)
+ goto QFileInfoRetry;
+
+ return rc;
+}
int
CIFSSMBQPathInfo(const int xid, struct cifsTconInfo *tcon,
return rc;
}
+int
+CIFSSMBUnixQFileInfo(const int xid, struct cifsTconInfo *tcon,
+ u16 netfid, FILE_UNIX_BASIC_INFO *pFindData)
+{
+ struct smb_t2_qfi_req *pSMB = NULL;
+ struct smb_t2_qfi_rsp *pSMBr = NULL;
+ int rc = 0;
+ int bytes_returned;
+ __u16 params, byte_count;
+
+UnixQFileInfoRetry:
+ rc = smb_init(SMB_COM_TRANSACTION2, 15, tcon, (void **) &pSMB,
+ (void **) &pSMBr);
+ if (rc)
+ return rc;
+
+ params = 2 /* level */ + 2 /* fid */;
+ pSMB->t2.TotalDataCount = 0;
+ pSMB->t2.MaxParameterCount = cpu_to_le16(4);
+ /* BB find exact max data count below from sess structure BB */
+ pSMB->t2.MaxDataCount = cpu_to_le16(CIFSMaxBufSize);
+ pSMB->t2.MaxSetupCount = 0;
+ pSMB->t2.Reserved = 0;
+ pSMB->t2.Flags = 0;
+ pSMB->t2.Timeout = 0;
+ pSMB->t2.Reserved2 = 0;
+ pSMB->t2.ParameterOffset = cpu_to_le16(offsetof(struct smb_t2_qfi_req,
+ Fid) - 4);
+ pSMB->t2.DataCount = 0;
+ pSMB->t2.DataOffset = 0;
+ pSMB->t2.SetupCount = 1;
+ pSMB->t2.Reserved3 = 0;
+ pSMB->t2.SubCommand = cpu_to_le16(TRANS2_QUERY_FILE_INFORMATION);
+ byte_count = params + 1 /* pad */ ;
+ pSMB->t2.TotalParameterCount = cpu_to_le16(params);
+ pSMB->t2.ParameterCount = pSMB->t2.TotalParameterCount;
+ pSMB->InformationLevel = cpu_to_le16(SMB_QUERY_FILE_UNIX_BASIC);
+ pSMB->Pad = 0;
+ pSMB->Fid = netfid;
+ pSMB->hdr.smb_buf_length += byte_count;
+
+ rc = SendReceive(xid, tcon->ses, (struct smb_hdr *) pSMB,
+ (struct smb_hdr *) pSMBr, &bytes_returned, 0);
+ if (rc) {
+ cFYI(1, ("Send error in QPathInfo = %d", rc));
+ } else { /* decode response */
+ rc = validate_t2((struct smb_t2_rsp *)pSMBr);
+
+ if (rc || (pSMBr->ByteCount < sizeof(FILE_UNIX_BASIC_INFO))) {
+ cERROR(1, ("Malformed FILE_UNIX_BASIC_INFO response.\n"
+ "Unix Extensions can be disabled on mount "
+ "by specifying the nosfu mount option."));
+ rc = -EIO; /* bad smb */
+ } else {
+ __u16 data_offset = le16_to_cpu(pSMBr->t2.DataOffset);
+ memcpy((char *) pFindData,
+ (char *) &pSMBr->hdr.Protocol +
+ data_offset,
+ sizeof(FILE_UNIX_BASIC_INFO));
+ }
+ }
+
+ cifs_buf_release(pSMB);
+ if (rc == -EAGAIN)
+ goto UnixQFileInfoRetry;
+
+ return rc;
+}
+
int
CIFSSMBUnixQPathInfo(const int xid, struct cifsTconInfo *tcon,
const unsigned char *searchName,
#include <linux/string.h>
#include <linux/list.h>
#include <linux/wait.h>
+#include <linux/slab.h>
#include <linux/pagemap.h>
#include <linux/ctype.h>
#include <linux/utsname.h>
int isValid = 1;
if (direntry->d_inode) {
- if (cifs_revalidate(direntry))
+ if (cifs_revalidate_dentry(direntry))
return 0;
} else {
cFYI(1, ("neg dentry 0x%p name = %s",
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
+#include <linux/slab.h>
#include <keys/user-type.h>
#include "dns_resolve.h"
#include "cifsglob.h"
#include <linux/task_io_accounting_ops.h>
#include <linux/delay.h>
#include <linux/mount.h>
+#include <linux/slab.h>
#include <asm/div64.h>
#include "cifsfs.h"
#include "cifspdu.h"
cFYI(1, ("inode unchanged on server"));
} else {
if (file->f_path.dentry->d_inode->i_mapping) {
- /* BB no need to lock inode until after invalidate
- since namei code should already have it locked? */
+ /* BB no need to lock inode until after invalidate
+ since namei code should already have it locked? */
rc = filemap_write_and_wait(file->f_path.dentry->d_inode->i_mapping);
if (rc != 0)
CIFS_I(file->f_path.dentry->d_inode)->write_behind_rc = rc;
int cifs_file_mmap(struct file *file, struct vm_area_struct *vma)
{
- struct dentry *dentry = file->f_path.dentry;
int rc, xid;
xid = GetXid();
- rc = cifs_revalidate(dentry);
+ rc = cifs_revalidate_file(file);
if (rc) {
cFYI(1, ("Validation prior to mmap failed, error=%d", rc));
FreeXid(xid);
*/
#include <linux/fs.h>
#include <linux/stat.h>
+#include <linux/slab.h>
#include <linux/pagemap.h>
#include <asm/div64.h>
#include "cifsfs.h"
}
}
+/* check inode attributes against fattr. If they don't match, tag the
+ * inode for cache invalidation
+ */
+static void
+cifs_revalidate_cache(struct inode *inode, struct cifs_fattr *fattr)
+{
+ struct cifsInodeInfo *cifs_i = CIFS_I(inode);
+
+ cFYI(1, ("%s: revalidating inode %llu", __func__, cifs_i->uniqueid));
+
+ if (inode->i_state & I_NEW) {
+ cFYI(1, ("%s: inode %llu is new", __func__, cifs_i->uniqueid));
+ return;
+ }
+
+ /* don't bother with revalidation if we have an oplock */
+ if (cifs_i->clientCanCacheRead) {
+ cFYI(1, ("%s: inode %llu is oplocked", __func__,
+ cifs_i->uniqueid));
+ return;
+ }
+
+ /* revalidate if mtime or size have changed */
+ if (timespec_equal(&inode->i_mtime, &fattr->cf_mtime) &&
+ cifs_i->server_eof == fattr->cf_eof) {
+ cFYI(1, ("%s: inode %llu is unchanged", __func__,
+ cifs_i->uniqueid));
+ return;
+ }
+
+ cFYI(1, ("%s: invalidating inode %llu mapping", __func__,
+ cifs_i->uniqueid));
+ cifs_i->invalid_mapping = true;
+}
+
/* populate an inode with info from a cifs_fattr struct */
void
cifs_fattr_to_inode(struct inode *inode, struct cifs_fattr *fattr)
struct cifs_sb_info *cifs_sb = CIFS_SB(inode->i_sb);
unsigned long oldtime = cifs_i->time;
+ cifs_revalidate_cache(inode, fattr);
+
inode->i_atime = fattr->cf_atime;
inode->i_mtime = fattr->cf_mtime;
inode->i_ctime = fattr->cf_ctime;
fattr->cf_flags |= CIFS_FATTR_DFS_REFERRAL;
}
+int cifs_get_file_info_unix(struct file *filp)
+{
+ int rc;
+ int xid;
+ FILE_UNIX_BASIC_INFO find_data;
+ struct cifs_fattr fattr;
+ struct inode *inode = filp->f_path.dentry->d_inode;
+ struct cifs_sb_info *cifs_sb = CIFS_SB(inode->i_sb);
+ struct cifsTconInfo *tcon = cifs_sb->tcon;
+ struct cifsFileInfo *cfile = (struct cifsFileInfo *) filp->private_data;
+
+ xid = GetXid();
+ rc = CIFSSMBUnixQFileInfo(xid, tcon, cfile->netfid, &find_data);
+ if (!rc) {
+ cifs_unix_basic_to_fattr(&fattr, &find_data, cifs_sb);
+ } else if (rc == -EREMOTE) {
+ cifs_create_dfs_fattr(&fattr, inode->i_sb);
+ rc = 0;
+ }
+
+ cifs_fattr_to_inode(inode, &fattr);
+ FreeXid(xid);
+ return rc;
+}
+
int cifs_get_inode_info_unix(struct inode **pinode,
const unsigned char *full_path,
struct super_block *sb, int xid)
fattr->cf_gid = cifs_sb->mnt_gid;
}
+int cifs_get_file_info(struct file *filp)
+{
+ int rc;
+ int xid;
+ FILE_ALL_INFO find_data;
+ struct cifs_fattr fattr;
+ struct inode *inode = filp->f_path.dentry->d_inode;
+ struct cifs_sb_info *cifs_sb = CIFS_SB(inode->i_sb);
+ struct cifsTconInfo *tcon = cifs_sb->tcon;
+ struct cifsFileInfo *cfile = (struct cifsFileInfo *) filp->private_data;
+
+ xid = GetXid();
+ rc = CIFSSMBQFileInfo(xid, tcon, cfile->netfid, &find_data);
+ if (rc == -EOPNOTSUPP || rc == -EINVAL) {
+ /*
+ * FIXME: legacy server -- fall back to path-based call?
+ * for now, just skip revalidating and mark inode for
+ * immediate reval.
+ */
+ rc = 0;
+ CIFS_I(inode)->time = 0;
+ goto cgfi_exit;
+ } else if (rc == -EREMOTE) {
+ cifs_create_dfs_fattr(&fattr, inode->i_sb);
+ rc = 0;
+ } else if (rc)
+ goto cgfi_exit;
+
+ /*
+ * don't bother with SFU junk here -- just mark inode as needing
+ * revalidation.
+ */
+ cifs_all_info_to_fattr(&fattr, &find_data, cifs_sb, false);
+ fattr.cf_uniqueid = CIFS_I(inode)->uniqueid;
+ fattr.cf_flags |= CIFS_FATTR_NEED_REVAL;
+ cifs_fattr_to_inode(inode, &fattr);
+cgfi_exit:
+ FreeXid(xid);
+ return rc;
+}
+
int cifs_get_inode_info(struct inode **pinode,
const unsigned char *full_path, FILE_ALL_INFO *pfindData,
struct super_block *sb, int xid, const __u16 *pfid)
return rc;
}
-int cifs_revalidate(struct dentry *direntry)
+static bool
+cifs_inode_needs_reval(struct inode *inode)
{
- int xid;
- int rc = 0, wbrc = 0;
- char *full_path;
- struct cifs_sb_info *cifs_sb;
- struct cifsInodeInfo *cifsInode;
- loff_t local_size;
- struct timespec local_mtime;
- bool invalidate_inode = false;
+ struct cifsInodeInfo *cifs_i = CIFS_I(inode);
- if (direntry->d_inode == NULL)
- return -ENOENT;
+ if (cifs_i->clientCanCacheRead)
+ return false;
- cifsInode = CIFS_I(direntry->d_inode);
+ if (!lookupCacheEnabled)
+ return true;
- if (cifsInode == NULL)
- return -ENOENT;
+ if (cifs_i->time == 0)
+ return true;
- /* no sense revalidating inode info on file that no one can write */
- if (CIFS_I(direntry->d_inode)->clientCanCacheRead)
- return rc;
+ /* FIXME: the actimeo should be tunable */
+ if (time_after_eq(jiffies, cifs_i->time + HZ))
+ return true;
+
+ return false;
+}
+
+/* check invalid_mapping flag and zap the cache if it's set */
+static void
+cifs_invalidate_mapping(struct inode *inode)
+{
+ int rc;
+ struct cifsInodeInfo *cifs_i = CIFS_I(inode);
+
+ cifs_i->invalid_mapping = false;
+
+ /* write back any cached data */
+ if (inode->i_mapping && inode->i_mapping->nrpages != 0) {
+ rc = filemap_write_and_wait(inode->i_mapping);
+ if (rc)
+ cifs_i->write_behind_rc = rc;
+ }
+ invalidate_remote_inode(inode);
+}
+
+int cifs_revalidate_file(struct file *filp)
+{
+ int rc = 0;
+ struct inode *inode = filp->f_path.dentry->d_inode;
+
+ if (!cifs_inode_needs_reval(inode))
+ goto check_inval;
+
+ if (CIFS_SB(inode->i_sb)->tcon->unix_ext)
+ rc = cifs_get_file_info_unix(filp);
+ else
+ rc = cifs_get_file_info(filp);
+
+check_inval:
+ if (CIFS_I(inode)->invalid_mapping)
+ cifs_invalidate_mapping(inode);
+
+ return rc;
+}
+
+/* revalidate a dentry's inode attributes */
+int cifs_revalidate_dentry(struct dentry *dentry)
+{
+ int xid;
+ int rc = 0;
+ char *full_path = NULL;
+ struct inode *inode = dentry->d_inode;
+ struct super_block *sb = dentry->d_sb;
+
+ if (inode == NULL)
+ return -ENOENT;
xid = GetXid();
- cifs_sb = CIFS_SB(direntry->d_sb);
+ if (!cifs_inode_needs_reval(inode))
+ goto check_inval;
/* can not safely grab the rename sem here if rename calls revalidate
since that would deadlock */
- full_path = build_path_from_dentry(direntry);
+ full_path = build_path_from_dentry(dentry);
if (full_path == NULL) {
rc = -ENOMEM;
- FreeXid(xid);
- return rc;
- }
- cFYI(1, ("Revalidate: %s inode 0x%p count %d dentry: 0x%p d_time %ld "
- "jiffies %ld", full_path, direntry->d_inode,
- direntry->d_inode->i_count.counter, direntry,
- direntry->d_time, jiffies));
-
- if (cifsInode->time == 0) {
- /* was set to zero previously to force revalidate */
- } else if (time_before(jiffies, cifsInode->time + HZ) &&
- lookupCacheEnabled) {
- if ((S_ISREG(direntry->d_inode->i_mode) == 0) ||
- (direntry->d_inode->i_nlink == 1)) {
- kfree(full_path);
- FreeXid(xid);
- return rc;
- } else {
- cFYI(1, ("Have to revalidate file due to hardlinks"));
- }
- }
-
- /* save mtime and size */
- local_mtime = direntry->d_inode->i_mtime;
- local_size = direntry->d_inode->i_size;
-
- if (cifs_sb->tcon->unix_ext) {
- rc = cifs_get_inode_info_unix(&direntry->d_inode, full_path,
- direntry->d_sb, xid);
- if (rc) {
- cFYI(1, ("error on getting revalidate info %d", rc));
-/* if (rc != -ENOENT)
- rc = 0; */ /* BB should we cache info on
- certain errors? */
- }
- } else {
- rc = cifs_get_inode_info(&direntry->d_inode, full_path, NULL,
- direntry->d_sb, xid, NULL);
- if (rc) {
- cFYI(1, ("error on getting revalidate info %d", rc));
-/* if (rc != -ENOENT)
- rc = 0; */ /* BB should we cache info on
- certain errors? */
- }
+ goto check_inval;
}
- /* should we remap certain errors, access denied?, to zero */
- /* if not oplocked, we invalidate inode pages if mtime or file size
- had changed on server */
+ cFYI(1, ("Revalidate: %s inode 0x%p count %d dentry: 0x%p d_time %ld "
+ "jiffies %ld", full_path, inode, inode->i_count.counter,
+ dentry, dentry->d_time, jiffies));
- if (timespec_equal(&local_mtime, &direntry->d_inode->i_mtime) &&
- (local_size == direntry->d_inode->i_size)) {
- cFYI(1, ("cifs_revalidate - inode unchanged"));
- } else {
- /* file may have changed on server */
- if (cifsInode->clientCanCacheRead) {
- /* no need to invalidate inode pages since we were the
- only ones who could have modified the file and the
- server copy is staler than ours */
- } else {
- invalidate_inode = true;
- }
- }
+ if (CIFS_SB(sb)->tcon->unix_ext)
+ rc = cifs_get_inode_info_unix(&inode, full_path, sb, xid);
+ else
+ rc = cifs_get_inode_info(&inode, full_path, NULL, sb,
+ xid, NULL);
- /* can not grab this sem since kernel filesys locking documentation
- indicates i_mutex may be taken by the kernel on lookup and rename
- which could deadlock if we grab the i_mutex here as well */
-/* mutex_lock(&direntry->d_inode->i_mutex);*/
- /* need to write out dirty pages here */
- if (direntry->d_inode->i_mapping) {
- /* do we need to lock inode until after invalidate completes
- below? */
- wbrc = filemap_fdatawrite(direntry->d_inode->i_mapping);
- if (wbrc)
- CIFS_I(direntry->d_inode)->write_behind_rc = wbrc;
- }
- if (invalidate_inode) {
- /* shrink_dcache not necessary now that cifs dentry ops
- are exported for negative dentries */
-/* if (S_ISDIR(direntry->d_inode->i_mode))
- shrink_dcache_parent(direntry); */
- if (S_ISREG(direntry->d_inode->i_mode)) {
- if (direntry->d_inode->i_mapping) {
- wbrc = filemap_fdatawait(direntry->d_inode->i_mapping);
- if (wbrc)
- CIFS_I(direntry->d_inode)->write_behind_rc = wbrc;
- }
- /* may eventually have to do this for open files too */
- if (list_empty(&(cifsInode->openFileList))) {
- /* changed on server - flush read ahead pages */
- cFYI(1, ("Invalidating read ahead data on "
- "closed file"));
- invalidate_remote_inode(direntry->d_inode);
- }
- }
- }
-/* mutex_unlock(&direntry->d_inode->i_mutex); */
+check_inval:
+ if (CIFS_I(inode)->invalid_mapping)
+ cifs_invalidate_mapping(inode);
kfree(full_path);
FreeXid(xid);
int cifs_getattr(struct vfsmount *mnt, struct dentry *dentry,
struct kstat *stat)
{
- int err = cifs_revalidate(dentry);
+ int err = cifs_revalidate_dentry(dentry);
if (!err) {
generic_fillattr(dentry->d_inode, stat);
stat->blksize = CIFS_MAX_MSGSIZE;
*/
#include <linux/fs.h>
#include <linux/stat.h>
+#include <linux/slab.h>
#include <linux/namei.h>
#include "cifsfs.h"
#include "cifspdu.h"
*/
#include <linux/fs.h>
#include <linux/pagemap.h>
+#include <linux/slab.h>
#include <linux/stat.h>
#include "cifspdu.h"
#include "cifsglob.h"
#include "ntlmssp.h"
#include "nterr.h"
#include <linux/utsname.h>
+#include <linux/slab.h>
#include "cifs_spnego.h"
extern void SMBNTencrypt(unsigned char *passwd, unsigned char *c8,
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/fs.h>
#include <linux/string.h>
#include <linux/kernel.h>
#include <linux/fs.h>
#include <linux/list.h>
+#include <linux/gfp.h>
#include <linux/wait.h>
#include <linux/net.h>
#include <linux/delay.h>
#include <linux/fs.h>
#include <linux/posix_acl_xattr.h>
+#include <linux/slab.h>
#include "cifsfs.h"
#include "cifspdu.h"
#include "cifsglob.h"
#include <linux/kernel.h>
#include <linux/time.h>
#include <linux/fs.h>
+#include <linux/slab.h>
#include <linux/file.h>
#include <linux/stat.h>
#include <linux/errno.h>
#include <linux/errno.h>
#include <linux/smp_lock.h>
#include <linux/string.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <linux/coda.h>
#include <linux/smp_lock.h>
#include <linux/file.h>
#include <linux/vfs.h>
+#include <linux/slab.h>
#include <asm/system.h>
#include <asm/uaccess.h>
#include <linux/stat.h>
#include <linux/errno.h>
#include <linux/string.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <linux/vmalloc.h>
#include <linux/vfs.h>
#include <linux/mm.h>
#include <linux/eventpoll.h>
#include <linux/fs_struct.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/mmu_context.h>
#include <linux/ioctl.h>
#include <linux/if.h>
#include <linux/if_bridge.h>
-#include <linux/slab.h>
#include <linux/raid/md_u.h>
#include <linux/kd.h>
#include <linux/route.h>
#include <linux/i2c.h>
#include <linux/i2c-dev.h>
#include <linux/atalk.h>
+#include <linux/gfp.h>
#include <net/bluetooth/bluetooth.h>
#include <net/bluetooth/hci.h>
#include <linux/capability.h>
#include <linux/sched.h>
#include <linux/lockdep.h>
+#include <linux/slab.h>
#include <linux/configfs.h>
#include "configfs_internal.h"
#include <linux/mount.h>
#include <linux/pagemap.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/configfs.h>
#include "configfs_internal.h"
#include <linux/fs.h>
#include <linux/module.h>
#include <linux/namei.h>
+#include <linux/slab.h>
#include <linux/configfs.h>
#include "configfs_internal.h"
#include <linux/fsnotify.h>
#include <linux/string.h>
#include <linux/magic.h>
+#include <linux/slab.h>
static struct vfsmount *debugfs_mount;
static int debugfs_mount_count;
#include <linux/fs.h>
#include <linux/sched.h>
#include <linux/namei.h>
+#include <linux/slab.h>
#include <linux/mount.h>
#include <linux/tty.h>
#include <linux/mutex.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/configfs.h>
+#include <linux/slab.h>
#include <linux/in.h>
#include <linux/in6.h>
#include <net/ipv6.h>
#include <linux/module.h>
#include <linux/ctype.h>
#include <linux/debugfs.h>
+#include <linux/slab.h>
#include "dlm_internal.h"
#include "lock.h"
L: receive_xxxx_reply() <- R: send_xxxx_reply()
*/
#include <linux/types.h>
+#include <linux/slab.h>
#include "dlm_internal.h"
#include <linux/dlm_device.h>
#include "memory.h"
#include <linux/file.h>
#include <linux/mutex.h>
#include <linux/sctp.h>
+#include <linux/slab.h>
#include <net/sctp/user.h>
#include <net/ipv6.h>
#include <net/genetlink.h>
#include <linux/dlm.h>
#include <linux/dlm_netlink.h>
+#include <linux/gfp.h>
#include "dlm_internal.h"
#include <linux/poll.h>
#include <linux/dlm.h>
#include <linux/dlm_plock.h>
+#include <linux/slab.h>
#include "dlm_internal.h"
#include "lockspace.h"
#include <linux/spinlock.h>
#include <linux/dlm.h>
#include <linux/dlm_device.h>
+#include <linux/slab.h>
#include "dlm_internal.h"
#include "lockspace.h"
#include <linux/crypto.h>
#include <linux/file.h>
#include <linux/scatterlist.h>
+#include <linux/slab.h>
#include <asm/unaligned.h>
#include "ecryptfs_kernel.h"
#include <linux/namei.h>
#include <linux/mount.h>
#include <linux/fs_stack.h>
+#include <linux/slab.h>
#include "ecryptfs_kernel.h"
/**
#include <linux/file.h>
#include <linux/poll.h>
+#include <linux/slab.h>
#include <linux/mount.h>
#include <linux/pagemap.h>
#include <linux/security.h>
#include <linux/mount.h>
#include <linux/crypto.h>
#include <linux/fs_stack.h>
+#include <linux/slab.h>
#include <asm/unaligned.h>
#include "ecryptfs_kernel.h"
#include <linux/random.h>
#include <linux/crypto.h>
#include <linux/scatterlist.h>
+#include <linux/slab.h>
#include "ecryptfs_kernel.h"
/**
#include <linux/kthread.h>
#include <linux/freezer.h>
+#include <linux/slab.h>
#include <linux/wait.h>
#include <linux/mount.h>
#include "ecryptfs_kernel.h"
#include <linux/key.h>
#include <linux/parser.h>
#include <linux/fs_stack.h>
+#include <linux/slab.h>
#include "ecryptfs_kernel.h"
/**
* 02111-1307, USA.
*/
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/user_namespace.h>
#include <linux/nsproxy.h>
#include "ecryptfs_kernel.h"
#include <linux/random.h>
#include <linux/miscdevice.h>
#include <linux/poll.h>
+#include <linux/slab.h>
#include <linux/wait.h>
#include <linux/module.h>
#include "ecryptfs_kernel.h"
#include <linux/file.h>
#include <linux/crypto.h>
#include <linux/scatterlist.h>
+#include <linux/slab.h>
#include <asm/unaligned.h>
#include "ecryptfs_kernel.h"
#include <linux/fs.h>
#include <linux/mount.h>
#include <linux/key.h>
+#include <linux/slab.h>
#include <linux/seq_file.h>
#include <linux/smp_lock.h>
#include <linux/file.h>
#include <linux/fs.h>
#include <linux/sched.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/list.h>
#include <linux/spinlock.h>
#include <linux/anon_inodes.h>
* Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
*/
+#include <linux/slab.h>
#include <linux/writeback.h>
#include <linux/buffer_head.h>
#include <scsi/scsi_device.h>
* Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
*/
+#include <linux/slab.h>
#include <scsi/scsi_device.h>
#include <asm/div64.h>
#include <linux/vfs.h>
#include <linux/random.h>
#include <linux/exportfs.h>
+#include <linux/slab.h>
#include "exofs.h"
#include "ext2.h"
#include <linux/quotaops.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#include <linux/buffer_head.h>
#include <linux/capability.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include <linux/fs.h>
#include <linux/ext2_fs.h>
#include <linux/time.h>
#include <linux/capability.h>
#include <linux/fs.h>
+#include <linux/slab.h>
#include <linux/jbd.h>
#include <linux/ext3_fs.h>
#include <linux/ext3_jbd.h>
inode->i_generation = sbi->s_next_generation++;
spin_unlock(&sbi->s_next_gen_lock);
- ei->i_state = EXT3_STATE_NEW;
+ ei->i_state_flags = 0;
+ ext3_set_inode_state(inode, EXT3_STATE_NEW);
+
ei->i_extra_isize =
(EXT3_INODE_SIZE(inode->i_sb) > EXT3_GOOD_OLD_INODE_SIZE) ?
sizeof(struct ext3_inode) - EXT3_GOOD_OLD_INODE_SIZE : 0;
inode->i_mtime.tv_sec = (signed)le32_to_cpu(raw_inode->i_mtime);
inode->i_atime.tv_nsec = inode->i_ctime.tv_nsec = inode->i_mtime.tv_nsec = 0;
- ei->i_state = 0;
+ ei->i_state_flags = 0;
ei->i_dir_start_lookup = 0;
ei->i_dtime = le32_to_cpu(raw_inode->i_dtime);
/* We now have enough fields to check if the inode was active or not.
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include <linux/fs.h>
#include <linux/ext3_jbd.h>
#include <linux/pagemap.h>
#include <linux/blkdev.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include "ext4.h"
struct ext4_system_zone {
ext4_group_t f;
f = ext4_flex_group(sbi, block_group);
- atomic_dec(&sbi->s_flex_groups[f].free_inodes);
+ atomic_dec(&sbi->s_flex_groups[f].used_dirs);
}
}
if (sbi->s_log_groups_per_flex) {
ext4_group_t f = ext4_flex_group(sbi, group);
- atomic_inc(&sbi->s_flex_groups[f].free_inodes);
+ atomic_inc(&sbi->s_flex_groups[f].used_dirs);
}
}
gdp->bg_checksum = ext4_group_desc_csum(sbi, group, gdp);
#include <linux/bio.h>
#include <linux/workqueue.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include "ext4_jbd2.h"
#include "xattr.h"
sector_t lblock)
{
struct ext4_inode_info *ei = EXT4_I(inode);
- int dind_mask = EXT4_ADDR_PER_BLOCK(inode->i_sb) - 1;
+ sector_t dind_mask = ~((sector_t)EXT4_ADDR_PER_BLOCK(inode->i_sb) - 1);
int blk_bits;
if (lblock < EXT4_NDIR_BLOCKS)
}
ei->i_da_metadata_calc_last_lblock = lblock & dind_mask;
ei->i_da_metadata_calc_len = 1;
- blk_bits = roundup_pow_of_two(lblock + 1);
+ blk_bits = order_base_2(lblock);
return (blk_bits / EXT4_ADDR_PER_BLOCK_BITS(inode->i_sb)) + 1;
}
#include "mballoc.h"
#include <linux/debugfs.h>
+#include <linux/slab.h>
#include <trace/events/ext4.h>
/*
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include "ext4_jbd2.h"
#include "ext4_extents.h"
#include <linux/fs.h>
#include <linux/quotaops.h>
+#include <linux/slab.h>
#include "ext4_jbd2.h"
#include "ext4_extents.h"
#include "ext4.h"
static int ext4_unfreeze(struct super_block *sb);
static void ext4_write_super(struct super_block *sb);
static int ext4_freeze(struct super_block *sb);
+static int ext4_get_sb(struct file_system_type *fs_type, int flags,
+ const char *dev_name, void *data, struct vfsmount *mnt);
+#if !defined(CONFIG_EXT3_FS) && !defined(CONFIG_EXT3_FS_MODULE) && defined(CONFIG_EXT4_USE_FOR_EXT23)
+static struct file_system_type ext3_fs_type = {
+ .owner = THIS_MODULE,
+ .name = "ext3",
+ .get_sb = ext4_get_sb,
+ .kill_sb = kill_block_super,
+ .fs_flags = FS_REQUIRES_DEV,
+};
+#define IS_EXT3_SB(sb) ((sb)->s_bdev->bd_holder == &ext3_fs_type)
+#else
+#define IS_EXT3_SB(sb) (0)
+#endif
ext4_fsblk_t ext4_block_bitmap(struct super_block *sb,
struct ext4_group_desc *bg)
* enable delayed allocation by default
* Use -o nodelalloc to turn it off
*/
- set_opt(sbi->s_mount_opt, DELALLOC);
+ if (!IS_EXT3_SB(sb))
+ set_opt(sbi->s_mount_opt, DELALLOC);
if (!parse_options((char *) data, sb, &journal_devnum,
&journal_ioprio, NULL, 0))
return get_sb_bdev(fs_type, flags, dev_name, data, ext4_fill_super,mnt);
}
-#if !defined(CONTIG_EXT2_FS) && !defined(CONFIG_EXT2_FS_MODULE) && defined(CONFIG_EXT4_USE_FOR_EXT23)
+#if !defined(CONFIG_EXT2_FS) && !defined(CONFIG_EXT2_FS_MODULE) && defined(CONFIG_EXT4_USE_FOR_EXT23)
static struct file_system_type ext2_fs_type = {
.owner = THIS_MODULE,
.name = "ext2",
static inline void unregister_as_ext2(void) { }
#endif
-#if !defined(CONTIG_EXT3_FS) && !defined(CONFIG_EXT3_FS_MODULE) && defined(CONFIG_EXT4_USE_FOR_EXT23)
-static struct file_system_type ext3_fs_type = {
- .owner = THIS_MODULE,
- .name = "ext3",
- .get_sb = ext4_get_sb,
- .kill_sb = kill_block_super,
- .fs_flags = FS_REQUIRES_DEV,
-};
-
+#if !defined(CONFIG_EXT3_FS) && !defined(CONFIG_EXT3_FS_MODULE) && defined(CONFIG_EXT4_USE_FOR_EXT23)
static inline void register_as_ext3(void)
{
int err = register_filesystem(&ext3_fs_type);
#include <linux/string.h>
#include <linux/fs.h>
#include <linux/security.h>
+#include <linux/slab.h>
#include "ext4_jbd2.h"
#include "ext4.h"
#include "xattr.h"
*/
#include <linux/fs.h>
+#include <linux/slab.h>
#include <linux/buffer_head.h>
#include "fat.h"
{
struct fat_mount_options *opts = &MSDOS_SB(dir->i_sb)->options;
wchar_t *ip, *ext_start, *end, *name_start;
- unsigned char base[9], ext[4], buf[8], *p;
+ unsigned char base[9], ext[4], buf[5], *p;
unsigned char charbuf[NLS_MAX_CHARSET_SIZE];
int chl, chi;
int sz = 0, extlen, baselen, i, numtail_baselen, numtail2_baselen;
return 0;
}
- i = jiffies & 0xffff;
+ i = jiffies;
sz = (jiffies >> 16) & 0x7;
if (baselen > 2) {
baselen = numtail2_baselen;
name_res[baselen + 4] = '~';
name_res[baselen + 5] = '1' + sz;
while (1) {
- sprintf(buf, "%04X", i);
+ snprintf(buf, sizeof(buf), "%04X", i & 0xffff);
memcpy(&name_res[baselen], buf, 4);
if (vfat_find_form(dir, name_res) < 0)
break;
*/
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/fs.h>
#include <linux/sched.h>
#include <linux/pipe_fs_i.h>
#include <linux/fs.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
-#include <linux/slab.h>
#include <linux/kmod.h>
#include <linux/init.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
/*
#include <linux/fs.h>
#include <linux/buffer_head.h>
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/pagemap.h>
#include "vxfs_extern.h"
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/spinlock.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#include <linux/fs.h>
#include <linux/mm.h>
#define FSCACHE_DEBUG_LEVEL COOKIE
#include <linux/module.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include <linux/key.h>
#include <keys/user-type.h>
#include "internal.h"
static void fscache_object_slow_work_put_ref(struct slow_work *);
static int fscache_object_slow_work_get_ref(struct slow_work *);
static void fscache_object_slow_work_execute(struct slow_work *);
-#ifdef CONFIG_SLOW_WORK_PROC
+#ifdef CONFIG_SLOW_WORK_DEBUG
static void fscache_object_slow_work_desc(struct slow_work *, struct seq_file *);
#endif
static void fscache_initialise_object(struct fscache_object *);
.get_ref = fscache_object_slow_work_get_ref,
.put_ref = fscache_object_slow_work_put_ref,
.execute = fscache_object_slow_work_execute,
-#ifdef CONFIG_SLOW_WORK_PROC
+#ifdef CONFIG_SLOW_WORK_DEBUG
.desc = fscache_object_slow_work_desc,
#endif
};
/*
* describe an object for slow-work debugging
*/
-#ifdef CONFIG_SLOW_WORK_PROC
+#ifdef CONFIG_SLOW_WORK_DEBUG
static void fscache_object_slow_work_desc(struct slow_work *work,
struct seq_file *m)
{
#define FSCACHE_DEBUG_LEVEL OPERATION
#include <linux/module.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include "internal.h"
atomic_t fscache_op_debug_id;
/*
* describe an operation for slow-work debugging
*/
-#ifdef CONFIG_SLOW_WORK_PROC
+#ifdef CONFIG_SLOW_WORK_DEBUG
static void fscache_op_desc(struct slow_work *work, struct seq_file *m)
{
struct fscache_operation *op =
.get_ref = fscache_op_get_ref,
.put_ref = fscache_op_put_ref,
.execute = fscache_op_execute,
-#ifdef CONFIG_SLOW_WORK_PROC
+#ifdef CONFIG_SLOW_WORK_DEBUG
.desc = fscache_op_desc,
#endif
};
#include <linux/fscache-cache.h>
#include <linux/buffer_head.h>
#include <linux/pagevec.h>
+#include <linux/slab.h>
#include "internal.h"
/*
goto nobufs;
nobufs_unlock_obj:
+ spin_unlock(&cookie->stores_lock);
spin_unlock(&object->lock);
nobufs:
spin_unlock(&cookie->lock);
#include <linux/magic.h>
#include <linux/miscdevice.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/stat.h>
*/
#include <linux/sched.h>
+#include <linux/gfp.h>
#include <linux/fs.h>
#include <linux/generic_acl.h>
#include <linux/posix_acl.h>
* of the GNU General Public License version 2.
*/
-#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/completion.h>
#include <linux/buffer_head.h>
* of the GNU General Public License version 2.
*/
-#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/completion.h>
#include <linux/buffer_head.h>
* of the GNU General Public License version 2.
*/
-#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/completion.h>
#include <linux/buffer_head.h>
* of the GNU General Public License version 2.
*/
-#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/completion.h>
#include <linux/buffer_head.h>
#include <linux/fs.h>
#include <linux/dlm.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/gfs2_ondisk.h>
#ifndef __RGRP_DOT_H__
#define __RGRP_DOT_H__
+#include <linux/slab.h>
+
struct gfs2_rgrpd;
struct gfs2_sbd;
struct gfs2_holder;
*/
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/completion.h>
#include <linux/buffer_head.h>
* of the GNU General Public License version 2.
*/
-#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/completion.h>
#include <linux/buffer_head.h>
*/
#include <linux/pagemap.h>
+#include <linux/slab.h>
#include <linux/swap.h>
#include "btree.h"
*/
#include <linux/pagemap.h>
+#include <linux/slab.h>
#include <linux/log2.h>
#include "btree.h"
#include <linux/cdrom.h>
#include <linux/genhd.h>
#include <linux/nls.h>
+#include <linux/slab.h>
#include "hfs_fs.h"
#include "btree.h"
#include <linux/nls.h>
#include <linux/parser.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include <linux/smp_lock.h>
#include <linux/vfs.h>
#include <linux/nls.h>
#include <linux/mount.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include "hfsplus_fs.h"
enum {
#include <linux/mm.h>
#include <linux/pagemap.h>
#include <linux/statfs.h>
+#include <linux/slab.h>
#include <linux/seq_file.h>
#include <linux/mount.h>
#include "hostfs.h"
* general buffer i/o
*/
#include <linux/sched.h>
+#include <linux/slab.h>
#include "hpfs_fn.h"
void hpfs_lock_creation(struct super_block *s)
*/
#include <linux/smp_lock.h>
+#include <linux/slab.h>
#include "hpfs_fn.h"
static int hpfs_dir_release(struct inode *inode, struct file *filp)
*/
#include <linux/smp_lock.h>
+#include <linux/slab.h>
#include "hpfs_fn.h"
void hpfs_init_inode(struct inode *i)
#include <linux/sched.h>
#include <linux/smp_lock.h>
#include <linux/bitmap.h>
+#include <linux/slab.h>
/* Mark the filesystem dirty, so that chkdsk checks it when os/2 booted */
* See also Documentation/block/ioprio.txt
*
*/
+#include <linux/gfp.h>
#include <linux/kernel.h>
#include <linux/ioprio.h>
#include <linux/blkdev.h>
* isofs directory handling functions
*/
#include <linux/smp_lock.h>
+#include <linux/gfp.h>
#include "isofs.h"
int isofs_name_translate(struct iso_directory_record *de, char *new, struct inode *inode)
*/
#include <linux/smp_lock.h>
+#include <linux/gfp.h>
#include "isofs.h"
/*
#include <linux/fs.h>
#include <linux/jbd.h>
#include <linux/errno.h>
-#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/pagemap.h>
#include <linux/bio.h>
#include <linux/fs.h>
#include <linux/jbd.h>
#include <linux/errno.h>
-#include <linux/slab.h>
#endif
/*
#include <linux/fs.h>
#include <linux/jbd2.h>
#include <linux/errno.h>
-#include <linux/slab.h>
#include <linux/crc32.h>
#endif
#include <linux/kernel.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/init.h>
#include <linux/lzo.h>
#endif
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/zlib.h>
#include <linux/zutil.h>
#include "nodelist.h"
#include <linux/crc32.h>
#include <linux/jffs2.h>
#include <linux/mtd/mtd.h>
+#include <linux/slab.h>
#include "nodelist.h"
#include "debug.h"
*/
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/fs.h>
#include <linux/time.h>
#include <linux/pagemap.h>
#include <linux/mtd/mtd.h>
#include <linux/rbtree.h>
#include <linux/crc32.h>
-#include <linux/slab.h>
#include <linux/pagemap.h>
#include "nodelist.h"
*/
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/mtd/mtd.h>
#include <linux/compiler.h>
#include <linux/sched.h> /* For cond_resched() */
*/
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/fs.h>
#include <linux/namei.h>
#include "nodelist.h"
#include <linux/kernel.h>
#include <linux/fs.h>
#include <linux/crc32.h>
-#include <linux/slab.h>
#include <linux/pagemap.h>
#include <linux/mtd/mtd.h>
#include "nodelist.h"
*/
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/fs.h>
#include <linux/posix_acl_xattr.h>
#include "jfs_incore.h"
*/
#include <linux/fs.h>
+#include <linux/slab.h>
#include "jfs_incore.h"
#include "jfs_superblock.h"
#include "jfs_dmap.h"
#include <linux/fs.h>
#include <linux/quotaops.h>
+#include <linux/slab.h>
#include "jfs_incore.h"
#include "jfs_superblock.h"
#include "jfs_filsys.h"
#include <linux/buffer_head.h>
#include <linux/pagemap.h>
#include <linux/quotaops.h>
+#include <linux/slab.h>
#include "jfs_incore.h"
#include "jfs_inode.h"
#include <linux/delay.h>
#include <linux/mutex.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include "jfs_incore.h"
#include "jfs_filsys.h"
#include "jfs_metapage.h"
#include <linux/mm.h>
#include <linux/module.h>
#include <linux/bio.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/buffer_head.h>
#include <linux/mempool.h>
#ifndef _H_JFS_UNICODE
#define _H_JFS_UNICODE
+#include <linux/slab.h>
#include <asm/byteorder.h>
#include "jfs_types.h"
#include <linux/buffer_head.h>
#include <linux/exportfs.h>
#include <linux/crc32.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <linux/seq_file.h>
#include <linux/smp_lock.h>
#include <linux/fs.h>
#include <linux/xattr.h>
#include <linux/posix_acl_xattr.h>
+#include <linux/slab.h>
#include <linux/quotaops.h>
#include <linux/security.h>
#include "jfs_incore.h"
#include <linux/module.h>
#include <linux/pagemap.h>
+#include <linux/slab.h>
#include <linux/mount.h>
#include <linux/vfs.h>
#include <linux/mutex.h>
#include <linux/module.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/time.h>
#include <linux/nfs_fs.h>
#include <linux/sunrpc/clnt.h>
#include <linux/module.h>
#include <linux/smp_lock.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/errno.h>
#include <linux/fs.h>
#include <linux/utsname.h>
#include <linux/kernel.h>
#include <linux/ktime.h>
+#include <linux/slab.h>
#include <linux/sunrpc/clnt.h>
#include <linux/sunrpc/xprtsock.h>
#include <linux/errno.h>
#include <linux/in.h>
#include <linux/uio.h>
-#include <linux/slab.h>
#include <linux/smp.h>
#include <linux/smp_lock.h>
#include <linux/mutex.h>
#include <linux/types.h>
#include <linux/time.h>
-#include <linux/slab.h>
#include <linux/smp_lock.h>
#include <linux/lockd/lockd.h>
#include <linux/lockd/share.h>
*/
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/kernel.h>
#include <linux/sched.h>
#include <linux/types.h>
#include <linux/time.h>
-#include <linux/slab.h>
#include <linux/smp_lock.h>
#include <linux/lockd/lockd.h>
#include <linux/lockd/share.h>
#include <linux/string.h>
#include <linux/time.h>
#include <linux/in.h>
+#include <linux/slab.h>
#include <linux/mutex.h>
#include <linux/sunrpc/svc.h>
#include <linux/sunrpc/clnt.h>
#include <linux/bio.h>
#include <linux/blkdev.h>
#include <linux/buffer_head.h>
+#include <linux/gfp.h>
#define PAGE_OFS(ofs) ((ofs) & (PAGE_SIZE-1))
prefetchw(&bvec->bv_page->flags);
end_page_writeback(page);
+ page_cache_release(page);
} while (bvec >= bio->bi_io_vec);
bio_put(bio);
if (atomic_dec_and_test(&super->s_pending_writes))
unsigned int max_pages = queue_max_hw_sectors(q) >> (PAGE_SHIFT - 9);
int i;
+ if (max_pages > BIO_MAX_PAGES)
+ max_pages = BIO_MAX_PAGES;
bio = bio_alloc(GFP_NOFS, max_pages);
- BUG_ON(!bio); /* FIXME: handle this */
+ BUG_ON(!bio);
for (i = 0; i < nr_pages; i++) {
if (i >= max_pages) {
unsigned int max_pages = queue_max_hw_sectors(q) >> (PAGE_SHIFT - 9);
int i;
+ if (max_pages > BIO_MAX_PAGES)
+ max_pages = BIO_MAX_PAGES;
bio = bio_alloc(GFP_NOFS, max_pages);
- BUG_ON(!bio); /* FIXME: handle this */
+ BUG_ON(!bio);
for (i = 0; i < nr_pages; i++) {
if (i >= max_pages) {
* Copyright (c) 2005-2008 Joern Engel <joern@logfs.org>
*/
#include "logfs.h"
-
+#include <linux/slab.h>
/*
* Atomic dir operations
(filler_t *)logfs_readpage, NULL);
if (IS_ERR(page))
return PTR_ERR(page);
- dd = kmap_atomic(page, KM_USER0);
+ dd = kmap(page);
BUG_ON(dd->namelen == 0);
full = filldir(buf, (char *)dd->name, be16_to_cpu(dd->namelen),
pos, be64_to_cpu(dd->ino), dd->type);
- kunmap_atomic(dd, KM_USER0);
+ kunmap(page);
page_cache_release(page);
if (full)
break;
*/
#include "logfs.h"
#include <linux/sched.h>
+#include <linux/slab.h>
/*
* Wear leveling needs to kick in when the difference between low erase
* Copyright (c) 2005-2008 Joern Engel <joern@logfs.org>
*/
#include "logfs.h"
+#include <linux/slab.h>
#include <linux/writeback.h>
#include <linux/backing-dev.h>
* Copyright (c) 2005-2008 Joern Engel <joern@logfs.org>
*/
#include "logfs.h"
+#include <linux/slab.h>
static void logfs_calc_free(struct super_block *sb)
{
{
struct logfs_super *super = logfs_super(sb);
struct logfs_area *area = super->s_journal_area;
+ struct btree_head32 *head = &super->s_reserved_segments;
u32 segno, ec;
int i, err;
/* Drop old segments */
journal_for_each(i)
if (super->s_journal_seg[i]) {
+ btree_remove32(head, super->s_journal_seg[i]);
logfs_set_segment_unreserved(sb,
super->s_journal_seg[i],
super->s_journal_ec[i]);
super->s_journal_seg[i] = segno;
super->s_journal_ec[i] = ec;
logfs_set_segment_reserved(sb, segno);
+ err = btree_insert32(head, segno, (void *)1, GFP_KERNEL);
+ BUG_ON(err); /* mempool should prevent this */
+ err = logfs_erase_segment(sb, segno, 1);
+ BUG_ON(err); /* FIXME: remount-ro would be nicer */
}
/* Manually move journal_area */
+ freeseg(sb, area->a_segno);
area->a_segno = super->s_journal_seg[0];
area->a_is_open = 0;
area->a_used_bytes = 0;
int logfs_init_mapping(struct super_block *sb);
void logfs_sync_area(struct logfs_area *area);
void logfs_sync_segments(struct super_block *sb);
+void freeseg(struct super_block *sb, u32 segno);
/* area handling */
int logfs_init_areas(struct super_block *sb);
*/
#include "logfs.h"
#include <linux/sched.h>
+#include <linux/slab.h>
static u64 adjust_bix(u64 bix, level_t level)
{
return ret;
}
-/* Rewrite cannot mark the inode dirty but has to write it immediatly. */
int logfs_rewrite_block(struct inode *inode, u64 bix, u64 ofs,
gc_level_t gc_level, long flags)
{
if (level != 0)
alloc_indirect_block(inode, page, 0);
err = logfs_write_buf(inode, page, flags);
+ if (!err && shrink_level(gc_level) == 0) {
+ /* Rewrite cannot mark the inode dirty but has to
+ * write it immediatly.
+ * Q: Can't we just create an alias for the inode
+ * instead? And if not, why not?
+ */
+ if (inode->i_ino == LOGFS_INO_MASTER)
+ logfs_write_anchor(inode->i_sb);
+ else {
+ err = __logfs_write_inode(inode, flags);
+ }
+ }
}
logfs_put_write_page(page);
return err;
* three kinds of objects: inodes, dentries and blocks, both data and indirect.
*/
#include "logfs.h"
+#include <linux/slab.h>
static int logfs_mark_segment_bad(struct super_block *sb, u32 segno)
{
} while (len);
}
-/*
- * bdev_writeseg will write full pages. Memset the tail to prevent data leaks.
- */
-static void pad_wbuf(struct logfs_area *area, int final)
+static void pad_partial_page(struct logfs_area *area)
{
struct super_block *sb = area->a_sb;
- struct logfs_super *super = logfs_super(sb);
struct page *page;
u64 ofs = dev_ofs(sb, area->a_segno, area->a_used_bytes);
pgoff_t index = ofs >> PAGE_SHIFT;
long offset = ofs & (PAGE_SIZE-1);
u32 len = PAGE_SIZE - offset;
- if (len == PAGE_SIZE) {
- /* The math in this function can surely use some love */
- len = 0;
- }
- if (len) {
- BUG_ON(area->a_used_bytes >= super->s_segsize);
-
- page = get_mapping_page(area->a_sb, index, 0);
+ if (len % PAGE_SIZE) {
+ page = get_mapping_page(sb, index, 0);
BUG_ON(!page); /* FIXME: reserve a pool */
memset(page_address(page) + offset, 0xff, len);
SetPagePrivate(page);
page_cache_release(page);
}
+}
- if (!final)
- return;
+static void pad_full_pages(struct logfs_area *area)
+{
+ struct super_block *sb = area->a_sb;
+ struct logfs_super *super = logfs_super(sb);
+ u64 ofs = dev_ofs(sb, area->a_segno, area->a_used_bytes);
+ u32 len = super->s_segsize - area->a_used_bytes;
+ pgoff_t index = PAGE_CACHE_ALIGN(ofs) >> PAGE_CACHE_SHIFT;
+ pgoff_t no_indizes = len >> PAGE_CACHE_SHIFT;
+ struct page *page;
- area->a_used_bytes += len;
- for ( ; area->a_used_bytes < super->s_segsize;
- area->a_used_bytes += PAGE_SIZE) {
- /* Memset another page */
- index++;
- page = get_mapping_page(area->a_sb, index, 0);
+ while (no_indizes) {
+ page = get_mapping_page(sb, index, 0);
BUG_ON(!page); /* FIXME: reserve a pool */
- memset(page_address(page), 0xff, PAGE_SIZE);
+ SetPageUptodate(page);
+ memset(page_address(page), 0xff, PAGE_CACHE_SIZE);
SetPagePrivate(page);
page_cache_release(page);
+ index++;
+ no_indizes--;
}
}
+/*
+ * bdev_writeseg will write full pages. Memset the tail to prevent data leaks.
+ * Also make sure we allocate (and memset) all pages for final writeout.
+ */
+static void pad_wbuf(struct logfs_area *area, int final)
+{
+ pad_partial_page(area);
+ if (final)
+ pad_full_pages(area);
+}
+
/*
* We have to be careful with the alias tree. Since lookup is done by bix,
* it needs to be normalized, so 14, 15, 16, etc. all match when dealing with
return 0;
}
-static void freeseg(struct super_block *sb, u32 segno)
+void freeseg(struct super_block *sb, u32 segno)
{
struct logfs_super *super = logfs_super(sb);
struct address_space *mapping = super->s_mapping_inode->i_mapping;
*/
#include "logfs.h"
#include <linux/bio.h>
+#include <linux/slab.h>
#include <linux/mtd/mtd.h>
#include <linux/statfs.h>
#include <linux/buffer_head.h>
}
if (valid0 && valid1 && ds_cmp(ds0, ds1)) {
printk(KERN_INFO"Superblocks don't match - fixing.\n");
- return write_one_sb(sb, super->s_devops->find_last_sb);
+ return logfs_write_sb(sb);
}
/* If neither is valid now, something's wrong. Didn't we properly
* check them before?!? */
{
int err;
+ err = logfs_open_segfile(sb);
+ if (err)
+ return err;
+
/* Repair any broken superblock copies */
err = logfs_recover_sb(sb);
if (err)
if (err)
return err;
- err = logfs_open_segfile(sb);
- if (err)
- return err;
-
/* Do one GC pass before any data gets dirtied */
logfs_gc_pass(sb);
sb->s_root = d_alloc_root(rootdir);
if (!sb->s_root)
- goto fail;
+ goto fail2;
super->s_erase_page = alloc_pages(GFP_KERNEL, 0);
if (!super->s_erase_page)
return 0;
err1:
- up_write(&sb->s_umount);
- deactivate_super(sb);
+ deactivate_locked_super(sb);
return err;
err0:
kfree(super);
#include <linux/buffer_head.h>
+#include <linux/slab.h>
#include "minix.h"
enum {DEPTH = 3, DIRECT = 7}; /* Only double indirect */
#include <linux/module.h>
#include <linux/mm.h>
#include <linux/kdev_t.h>
+#include <linux/gfp.h>
#include <linux/bio.h>
#include <linux/fs.h>
#include <linux/buffer_head.h>
static struct file *do_last(struct nameidata *nd, struct path *path,
int open_flag, int acc_mode,
- int mode, const char *pathname,
- int *want_dir)
+ int mode, const char *pathname)
{
struct dentry *dir = nd->path.dentry;
struct file *filp;
if (nd->last.name[nd->last.len]) {
if (open_flag & O_CREAT)
goto exit;
- *want_dir = 1;
+ nd->flags |= LOOKUP_DIRECTORY;
}
/* just plain open? */
if (path->dentry->d_inode->i_op->follow_link)
return NULL;
error = -ENOTDIR;
- if (*want_dir && !path->dentry->d_inode->i_op->lookup)
- goto exit_dput;
+ if (nd->flags & LOOKUP_DIRECTORY) {
+ if (!path->dentry->d_inode->i_op->lookup)
+ goto exit_dput;
+ }
path_to_nameidata(path, nd);
audit_inode(pathname, nd->path.dentry);
goto ok;
int count = 0;
int flag = open_to_namei_flags(open_flag);
int force_reval = 0;
- int want_dir = open_flag & O_DIRECTORY;
if (!(open_flag & O_CREAT))
mode = 0;
if (open_flag & O_EXCL)
nd.flags |= LOOKUP_EXCL;
}
- filp = do_last(&nd, &path, open_flag, acc_mode, mode, pathname, &want_dir);
+ if (open_flag & O_DIRECTORY)
+ nd.flags |= LOOKUP_DIRECTORY;
+ filp = do_last(&nd, &path, open_flag, acc_mode, mode, pathname);
while (unlikely(!filp)) { /* trailing symlink */
struct path holder;
struct inode *inode = path.dentry->d_inode;
}
holder = path;
nd.flags &= ~LOOKUP_PARENT;
- filp = do_last(&nd, &path, open_flag, acc_mode, mode, pathname, &want_dir);
+ filp = do_last(&nd, &path, open_flag, acc_mode, mode, pathname);
if (inode->i_op->put_link)
inode->i_op->put_link(holder.dentry, &nd, cookie);
path_put(&holder);
#include <linux/errno.h>
#include <linux/stat.h>
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/mm.h>
#include <asm/uaccess.h>
#include <linux/fcntl.h>
#include <linux/stat.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/sched.h>
#include <linux/smp_lock.h>
#include <linux/time.h>
#include <linux/mm.h>
#include <linux/mount.h>
+#include <linux/slab.h>
#include <linux/highuid.h>
#include <linux/smp_lock.h>
#include <linux/vmalloc.h>
#include <linux/stat.h>
#include <linux/time.h>
#include <linux/kernel.h>
+#include <linux/gfp.h>
#include <linux/mm.h>
#include <linux/shm.h>
#include <linux/errno.h>
#include <linux/mman.h>
#include <linux/string.h>
-#include <linux/slab.h>
#include <linux/fcntl.h>
#include <linux/ncp_fs.h>
#include <linux/mm.h>
#include <linux/netdevice.h>
#include <linux/signal.h>
+#include <linux/slab.h>
#include <net/scm.h>
#include <net/sock.h>
#include <linux/ipx.h>
#include <linux/fs.h>
#include <linux/ncp_fs.h>
#include <linux/time.h>
+#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/stat.h>
#include "ncplib_kernel.h"
#include <linux/moduleparam.h>
#include <linux/mount.h>
#include <linux/namei.h>
+#include <linux/slab.h>
#include <linux/sunrpc/cache.h>
#include <linux/sunrpc/rpc_pipe_fs.h>
*/
#include <linux/nfs4.h>
#include <linux/nfs_fs.h>
+#include <linux/slab.h>
#include "nfs4_fs.h"
#include "callback.h"
#include "delegation.h"
#include <linux/sunrpc/svc.h>
#include <linux/nfs4.h>
#include <linux/nfs_fs.h>
+#include <linux/slab.h>
#include "nfs4_fs.h"
#include "callback.h"
#include <linux/vfs.h>
#include <linux/inet.h>
#include <linux/in6.h>
+#include <linux/slab.h>
#include <net/ipv6.h>
#include <linux/nfs_xdr.h>
#include <linux/sunrpc/bc_xprt.h>
#include <linux/kthread.h>
#include <linux/module.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/smp_lock.h>
#include <linux/spinlock.h>
#include <linux/file.h>
#include <linux/pagemap.h>
#include <linux/kref.h>
+#include <linux/slab.h>
#include <linux/nfs_fs.h>
#include <linux/nfs_page.h>
#include <linux/hash.h>
#include <linux/string.h>
#include <linux/kmod.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/socket.h>
#include <linux/seq_file.h>
#include <linux/nfs_fs.h>
#include <linux/nfs_mount.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/pagemap.h>
#include <linux/aio.h>
+#include <linux/gfp.h>
#include <asm/uaccess.h>
#include <asm/system.h>
{
dfprintk(PAGECACHE, "NFS: release_page(%p)\n", page);
- if (gfp & __GFP_WAIT)
+ /* Only do I/O if gfp is a superset of GFP_KERNEL */
+ if ((gfp & GFP_KERNEL) == GFP_KERNEL)
nfs_wb_page(page->mapping->host, page);
/* If PagePrivate() is set, then the page is not freeable */
if (PagePrivate(page))
#include <linux/nfs_fs_sb.h>
#include <linux/in6.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include "internal.h"
#include "iostat.h"
#include <linux/vfs.h>
#include <linux/inet.h>
#include <linux/nfs_xdr.h>
+#include <linux/slab.h>
#include <asm/system.h>
#include <asm/uaccess.h>
*/
#include <linux/dcache.h>
+#include <linux/gfp.h>
#include <linux/mount.h>
#include <linux/namei.h>
#include <linux/nfs_fs.h>
#include <linux/param.h>
#include <linux/time.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/in.h>
#include <linux/fs.h>
+#include <linux/gfp.h>
#include <linux/nfs.h>
#include <linux/nfs3.h>
#include <linux/nfs_fs.h>
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/sunrpc/clnt.h>
+#include <linux/slab.h>
#include <linux/nfs.h>
#include <linux/nfs3.h>
#include <linux/nfs_fs.h>
#include <linux/param.h>
#include <linux/time.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/in.h>
#include <linux/mount.h>
#include <linux/namei.h>
#include <linux/nfs_fs.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include <linux/sunrpc/clnt.h>
#include <linux/vfs.h>
#include <linux/delay.h>
#include <linux/errno.h>
#include <linux/string.h>
+#include <linux/slab.h>
#include <linux/sunrpc/clnt.h>
#include <linux/nfs.h>
#include <linux/nfs4.h>
#include <linux/param.h>
#include <linux/time.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/string.h>
#include <linux/in.h>
if (status != 0)
goto out;
status = decode_delegreturn(&xdr);
+ if (status != 0)
+ goto out;
decode_getfattr(&xdr, res->fattr, res->server,
!RPC_IS_ASYNC(rqstp->rq_task));
out:
#include <linux/types.h>
#include <linux/param.h>
-#include <linux/slab.h>
#include <linux/time.h>
#include <linux/mm.h>
#include <linux/errno.h>
#include <linux/vfs.h>
#include <linux/inet.h>
#include <linux/in6.h>
+#include <linux/slab.h>
#include <net/ipv6.h>
#include <linux/netdevice.h>
#include <linux/nfs_xdr.h>
#include <linux/pagemap.h>
#include <linux/stat.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/namei.h>
#include <linux/module.h>
#include <linux/fs.h>
+#include <linux/gfp.h>
#include <linux/sunrpc/xdr.h>
#include <linux/nfsacl.h>
#include <linux/nfs3.h>
* Copyright (C) 1995, 1996 Olaf Kirch, <okir@monad.swb.de>
*/
+#include <linux/slab.h>
#include <linux/namei.h>
#include <linux/module.h>
#include <linux/exportfs.h>
#include "nfsd.h"
/* FIXME: nfsacl.h is a broken header */
#include <linux/nfsacl.h>
+#include <linux/gfp.h>
#include "cache.h"
#include "xdr3.h"
#include "vfs.h"
#include "nfsd.h"
/* FIXME: nfsacl.h is a broken header */
#include <linux/nfsacl.h>
+#include <linux/gfp.h>
#include "cache.h"
#include "xdr3.h"
#include "vfs.h"
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
+#include <linux/slab.h>
#include <linux/nfs_fs.h>
#include <linux/nfs4_acl.h>
*/
#include <linux/sunrpc/clnt.h>
+#include <linux/slab.h>
#include "nfsd.h"
#include "state.h"
#include <linux/nfsd_idmap.h>
#include <linux/seq_file.h>
#include <linux/sched.h>
+#include <linux/slab.h>
/*
* Cache entry
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <linux/file.h>
+#include <linux/slab.h>
#include "cache.h"
#include "xdr4.h"
*/
#include <linux/file.h>
+#include <linux/slab.h>
#include <linux/namei.h>
#include <linux/crypto.h>
#include <linux/sched.h>
#include <linux/file.h>
#include <linux/smp_lock.h>
+#include <linux/slab.h>
#include <linux/namei.h>
#include <linux/swap.h>
#include <linux/sunrpc/svcauth_gss.h>
* at the end of nfs4svc_decode_compoundargs.
*/
+#include <linux/slab.h>
#include <linux/namei.h>
#include <linux/statfs.h>
#include <linux/utsname.h>
* Copyright (C) 1995, 1996 Olaf Kirch <okir@monad.swb.de>
*/
+#include <linux/slab.h>
+
#include "nfsd.h"
#include "cache.h"
* Copyright (C) 1995, 1996 Olaf Kirch <okir@monad.swb.de>
*/
+#include <linux/slab.h>
#include <linux/namei.h>
#include <linux/ctype.h>
#include <linux/xattr.h>
#include <linux/jhash.h>
#include <linux/ima.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <linux/exportfs.h>
#include <linux/writeback.h>
#include <linux/buffer_head.h>
#include <linux/fs.h>
#include <linux/bitops.h>
+#include <linux/slab.h>
#include "mdt.h"
#include "alloc.h"
#include <linux/buffer_head.h>
#include <linux/mm.h>
#include <linux/backing-dev.h>
+#include <linux/gfp.h>
#include "nilfs.h"
#include "mdt.h"
#include "dat.h"
#include <linux/buffer_head.h>
#include <linux/mpage.h>
#include <linux/hash.h>
+#include <linux/slab.h>
#include <linux/swap.h>
#include "nilfs.h"
#include "page.h"
*/
#include <linux/buffer_head.h>
+#include <linux/gfp.h>
#include <linux/mpage.h>
#include <linux/writeback.h>
#include <linux/uio.h>
#include <linux/fs.h>
#include <linux/wait.h>
#include <linux/smp_lock.h> /* lock_kernel(), unlock_kernel() */
+#include <linux/slab.h>
#include <linux/capability.h> /* capable() */
#include <linux/uaccess.h> /* copy_from_user(), copy_to_user() */
#include <linux/vmalloc.h>
#include <linux/writeback.h>
#include <linux/backing-dev.h>
#include <linux/swap.h>
+#include <linux/slab.h>
#include "nilfs.h"
#include "segment.h"
#include "page.h"
#include <linux/list.h>
#include <linux/highmem.h>
#include <linux/pagevec.h>
+#include <linux/gfp.h>
#include "nilfs.h"
#include "page.h"
#include "mdt.h"
#include <linux/buffer_head.h>
#include <linux/blkdev.h>
#include <linux/swap.h>
+#include <linux/slab.h>
#include <linux/crc32.h>
#include "nilfs.h"
#include "segment.h"
#include <linux/writeback.h>
#include <linux/crc32.h>
#include <linux/backing-dev.h>
+#include <linux/slab.h>
#include "page.h"
#include "segbuf.h"
int nilfs_wait_on_logs(struct list_head *logs)
{
struct nilfs_segment_buffer *segbuf;
- int err;
+ int err, ret = 0;
list_for_each_entry(segbuf, logs, sb_list) {
err = nilfs_segbuf_wait(segbuf);
- if (err)
- return err;
+ if (err && !ret)
+ ret = err;
}
- return 0;
+ return ret;
}
/*
#include <linux/kthread.h>
#include <linux/crc32.h>
#include <linux/pagevec.h>
+#include <linux/slab.h>
#include "nilfs.h"
#include "btnode.h"
#include "page.h"
if (mode != SC_LSEG_SR || sci->sc_stage.scnt < NILFS_ST_CPFILE)
break;
+ nilfs_clear_logs(&sci->sc_segbufs);
+
+ err = nilfs_segctor_extend_segments(sci, nilfs, nadd);
+ if (unlikely(err))
+ return err;
+
if (sci->sc_stage.flags & NILFS_CF_SUFREED) {
err = nilfs_sufile_cancel_freev(nilfs->ns_sufile,
sci->sc_freesegs,
NULL);
WARN_ON(err); /* do not happen */
}
- nilfs_clear_logs(&sci->sc_segbufs);
-
- err = nilfs_segctor_extend_segments(sci, nilfs, nadd);
- if (unlikely(err))
- return err;
-
nadd = min_t(int, nadd << 1, SC_MAX_SEGDELTA);
sci->sc_stage = prev_stage;
}
list_splice_tail_init(&sci->sc_write_logs, &logs);
ret = nilfs_wait_on_logs(&logs);
- if (ret)
- nilfs_abort_logs(&logs, NULL, sci->sc_super_root, ret);
+ nilfs_abort_logs(&logs, NULL, sci->sc_super_root, ret ? : err);
list_splice_tail_init(&sci->sc_segbufs, &logs);
nilfs_cancel_segusage(&logs, nilfs->ns_sufile);
#include <linux/fs.h>
#include <linux/blkdev.h>
#include <linux/backing-dev.h>
+#include <linux/slab.h>
#include "sb.h"
/* the_nilfs struct */
#include <linux/dcache.h>
#include <linux/fs.h>
+#include <linux/gfp.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/srcu.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/mutex.h>
-#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/writeback.h> /* for inode_lock */
#include <linux/errno.h>
#include <linux/fs.h>
+#include <linux/gfp.h>
#include <linux/mm.h>
#include <linux/pagemap.h>
#include <linux/swap.h>
#include <linux/buffer_head.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/swap.h>
#include <linux/writeback.h>
#include <linux/buffer_head.h>
#include <linux/blkdev.h>
#include <linux/vmalloc.h>
+#include <linux/slab.h>
#include "attrib.h"
#include "inode.h"
*/
#include <linux/buffer_head.h>
+#include <linux/slab.h>
#include "dir.h"
#include "aops.h"
*/
#include <linux/buffer_head.h>
+#include <linux/gfp.h>
#include <linux/pagemap.h>
#include <linux/pagevec.h>
#include <linux/sched.h>
* Foundation,Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
+#include <linux/slab.h>
+
#include "aops.h"
#include "collate.h"
#include "debug.h"
*/
#include <linux/buffer_head.h>
+#include <linux/slab.h>
#include <linux/swap.h>
#include "attrib.h"
#include <linux/dcache.h>
#include <linux/exportfs.h>
#include <linux/security.h>
+#include <linux/slab.h>
#include "attrib.h"
#include "debug.h"
#include <linux/init.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/string.h>
#define MLOG_MASK_PREFIX ML_INODE
#include "alloc.h"
#include "dlmglue.h"
#include "file.h"
+#include "inode.h"
+#include "journal.h"
#include "ocfs2_fs.h"
#include "xattr.h"
return acl;
}
+/*
+ * Helper function to set i_mode in memory and disk. Some call paths
+ * will not have di_bh or a journal handle to pass, in which case it
+ * will create it's own.
+ */
+static int ocfs2_acl_set_mode(struct inode *inode, struct buffer_head *di_bh,
+ handle_t *handle, umode_t new_mode)
+{
+ int ret, commit_handle = 0;
+ struct ocfs2_dinode *di;
+
+ if (di_bh == NULL) {
+ ret = ocfs2_read_inode_block(inode, &di_bh);
+ if (ret) {
+ mlog_errno(ret);
+ goto out;
+ }
+ } else
+ get_bh(di_bh);
+
+ if (handle == NULL) {
+ handle = ocfs2_start_trans(OCFS2_SB(inode->i_sb),
+ OCFS2_INODE_UPDATE_CREDITS);
+ if (IS_ERR(handle)) {
+ ret = PTR_ERR(handle);
+ mlog_errno(ret);
+ goto out_brelse;
+ }
+
+ commit_handle = 1;
+ }
+
+ di = (struct ocfs2_dinode *)di_bh->b_data;
+ ret = ocfs2_journal_access_di(handle, INODE_CACHE(inode), di_bh,
+ OCFS2_JOURNAL_ACCESS_WRITE);
+ if (ret) {
+ mlog_errno(ret);
+ goto out_commit;
+ }
+
+ inode->i_mode = new_mode;
+ di->i_mode = cpu_to_le16(inode->i_mode);
+
+ ocfs2_journal_dirty(handle, di_bh);
+
+out_commit:
+ if (commit_handle)
+ ocfs2_commit_trans(OCFS2_SB(inode->i_sb), handle);
+out_brelse:
+ brelse(di_bh);
+out:
+ return ret;
+}
+
/*
* Set the access or default ACL of an inode.
*/
if (ret < 0)
return ret;
else {
- inode->i_mode = mode;
if (ret == 0)
acl = NULL;
+
+ ret = ocfs2_acl_set_mode(inode, di_bh,
+ handle, mode);
+ if (ret)
+ return ret;
+
}
}
break;
struct ocfs2_super *osb = OCFS2_SB(inode->i_sb);
struct posix_acl *acl = NULL;
int ret = 0;
+ mode_t mode;
if (!S_ISLNK(inode->i_mode)) {
if (osb->s_mount_opt & OCFS2_MOUNT_POSIX_ACL) {
if (IS_ERR(acl))
return PTR_ERR(acl);
}
- if (!acl)
- inode->i_mode &= ~current_umask();
+ if (!acl) {
+ mode = inode->i_mode & ~current_umask();
+ ret = ocfs2_acl_set_mode(inode, di_bh, handle, mode);
+ if (ret) {
+ mlog_errno(ret);
+ goto cleanup;
+ }
+ }
}
if ((osb->s_mount_opt & OCFS2_MOUNT_POSIX_ACL) && acl) {
struct posix_acl *clone;
- mode_t mode;
if (S_ISDIR(inode->i_mode)) {
ret = ocfs2_set_acl(handle, inode, di_bh,
mode = inode->i_mode;
ret = posix_acl_create_masq(clone, &mode);
if (ret >= 0) {
- inode->i_mode = mode;
+ ret = ocfs2_acl_set_mode(inode, di_bh, handle, mode);
if (ret > 0) {
ret = ocfs2_set_acl(handle, inode,
di_bh, ACL_TYPE_ACCESS,
#include <linux/fs.h>
#include <linux/types.h>
-#include <linux/slab.h>
#include <linux/highmem.h>
#include <cluster/masklog.h>
#include <linux/crc32.h>
#include <linux/time.h>
#include <linux/debugfs.h>
+#include <linux/slab.h>
#include "heartbeat.h"
#include "tcp.h"
* Boston, MA 021110-1307, USA.
*/
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/configfs.h>
* and if they're the last, they fire off the decision.
*/
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/workqueue.h>
#include <linux/reboot.h>
#include <linux/module.h>
#include <linux/fs.h>
#include <linux/types.h>
-#include <linux/slab.h>
#include <linux/highmem.h>
#include <linux/init.h>
#include <linux/sysctl.h>
#include <linux/module.h>
#include <linux/fs.h>
#include <linux/types.h>
-#include <linux/slab.h>
#include <linux/highmem.h>
#include <linux/init.h>
#include <linux/sysctl.h>
ok:
spin_unlock(&res->spinlock);
}
- spin_unlock(&dlm->spinlock);
// mlog(0, "woo! got an assert_master from node %u!\n",
// assert->node_idx);
/* master is known, detach if not already detached.
* ensures that only one assert_master call will happen
* on this mle. */
- spin_lock(&dlm->spinlock);
spin_lock(&dlm->master_lock);
rr = atomic_read(&mle->mle_refs.refcount);
__dlm_put_mle(mle);
}
spin_unlock(&dlm->master_lock);
- spin_unlock(&dlm->spinlock);
} else if (res) {
if (res->owner != assert->node_idx) {
mlog(0, "assert_master from %u, but current "
res->owner, namelen, name);
}
}
+ spin_unlock(&dlm->spinlock);
done:
ret = 0;
#include <linux/module.h>
#include <linux/fs.h>
#include <linux/types.h>
-#include <linux/slab.h>
#include <linux/highmem.h>
#include <linux/init.h>
#include <linux/sysctl.h>
#include <linux/module.h>
#include <linux/fs.h>
#include <linux/types.h>
-#include <linux/slab.h>
#include <linux/highmem.h>
#include <linux/init.h>
#include <linux/sysctl.h>
#include <linux/fs.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/fiemap.h>
#include <linux/fs.h>
#include <linux/types.h>
-#include <linux/slab.h>
#include <linux/highmem.h>
#define MLOG_MASK_PREFIX ML_SUPER
#include <linux/fs.h>
#include <linux/types.h>
-#include <linux/slab.h>
#include <linux/highmem.h>
#include <linux/pagemap.h>
#include <linux/quotaops.h>
/* Do some basic inode verification... */
di = (struct ocfs2_dinode *) di_bh->b_data;
if (!(di->i_flags & cpu_to_le32(OCFS2_ORPHANED_FL))) {
+ /*
+ * Inodes in the orphan dir must have ORPHANED_FL. The only
+ * inodes that come back out of the orphan dir are reflink
+ * targets. A reflink target may be moved out of the orphan
+ * dir between the time we scan the directory and the time we
+ * process it. This would lead to HAS_REFCOUNT_FL being set but
+ * ORPHANED_FL not.
+ */
+ if (di->i_dyn_features & cpu_to_le16(OCFS2_HAS_REFCOUNT_FL)) {
+ mlog(0, "Reflinked inode %llu is no longer orphaned. "
+ "it shouldn't be deleted\n",
+ (unsigned long long)oi->ip_blkno);
+ goto bail;
+ }
+
/* for lack of a better error? */
status = -EEXIST;
mlog(ML_ERROR,
(unsigned long long)la_start_blk,
(unsigned long long)blkno);
- status = ocfs2_free_clusters(handle, main_bm_inode,
- main_bm_bh, blkno, count);
+ status = ocfs2_release_clusters(handle,
+ main_bm_inode,
+ main_bm_bh, blkno,
+ count);
if (status < 0) {
mlog_errno(status);
goto bail;
}
retry_enospc:
- (*ac)->ac_bits_wanted = osb->local_alloc_bits;
-
+ (*ac)->ac_bits_wanted = osb->local_alloc_default_bits;
status = ocfs2_reserve_cluster_bitmap_bits(osb, *ac);
if (status == -ENOSPC) {
if (ocfs2_recalc_la_window(osb, OCFS2_LA_EVENT_ENOSPC) ==
OCFS2_LA_DISABLED)
goto bail;
+ ac->ac_bits_wanted = osb->local_alloc_default_bits;
status = ocfs2_claim_clusters(osb, handle, ac,
osb->local_alloc_bits,
&cluster_off,
if (!(fl->fl_flags & FL_POSIX))
return -ENOLCK;
- if (__mandatory_lock(inode))
+ if (__mandatory_lock(inode) && fl->fl_type != F_UNLCK)
return -ENOLCK;
return ocfs2_plock(osb->cconn, OCFS2_I(inode)->ip_blkno, file, cmd, fl);
#include <linux/fs.h>
#include <linux/types.h>
-#include <linux/slab.h>
#include <linux/highmem.h>
#include <linux/pagemap.h>
#include <linux/uio.h>
static int ocfs2_orphan_add(struct ocfs2_super *osb,
handle_t *handle,
struct inode *inode,
- struct ocfs2_dinode *fe,
+ struct buffer_head *fe_bh,
char *name,
struct ocfs2_dir_lookup_result *lookup,
struct inode *orphan_dir_inode);
fe = (struct ocfs2_dinode *) fe_bh->b_data;
if (inode_is_unlinkable(inode)) {
- status = ocfs2_orphan_add(osb, handle, inode, fe, orphan_name,
+ status = ocfs2_orphan_add(osb, handle, inode, fe_bh, orphan_name,
&orphan_insert, orphan_dir);
if (status < 0) {
mlog_errno(status);
if (S_ISDIR(new_inode->i_mode) ||
(ocfs2_read_links_count(newfe) == 1)) {
status = ocfs2_orphan_add(osb, handle, new_inode,
- newfe, orphan_name,
+ newfe_bh, orphan_name,
&orphan_insert, orphan_dir);
if (status < 0) {
mlog_errno(status);
static int ocfs2_orphan_add(struct ocfs2_super *osb,
handle_t *handle,
struct inode *inode,
- struct ocfs2_dinode *fe,
+ struct buffer_head *fe_bh,
char *name,
struct ocfs2_dir_lookup_result *lookup,
struct inode *orphan_dir_inode)
struct buffer_head *orphan_dir_bh = NULL;
int status = 0;
struct ocfs2_dinode *orphan_fe;
+ struct ocfs2_dinode *fe = (struct ocfs2_dinode *) fe_bh->b_data;
mlog_entry("(inode->i_ino = %lu)\n", inode->i_ino);
goto leave;
}
+ /*
+ * We're going to journal the change of i_flags and i_orphaned_slot.
+ * It's safe anyway, though some callers may duplicate the journaling.
+ * Journaling within the func just make the logic look more
+ * straightforward.
+ */
+ status = ocfs2_journal_access_di(handle,
+ INODE_CACHE(inode),
+ fe_bh,
+ OCFS2_JOURNAL_ACCESS_WRITE);
+ if (status < 0) {
+ mlog_errno(status);
+ goto leave;
+ }
+
le32_add_cpu(&fe->i_flags, OCFS2_ORPHANED_FL);
/* Record which orphan dir our inode now resides
* dir to lock. */
fe->i_orphaned_slot = cpu_to_le16(osb->slot_num);
+ ocfs2_journal_dirty(handle, fe_bh);
+
mlog(0, "Inode %llu orphaned in slot %d\n",
(unsigned long long)OCFS2_I(inode)->ip_blkno, osb->slot_num);
}
di = (struct ocfs2_dinode *)new_di_bh->b_data;
- status = ocfs2_orphan_add(osb, handle, inode, di, orphan_name,
+ status = ocfs2_orphan_add(osb, handle, inode, new_di_bh, orphan_name,
&orphan_insert, orphan_dir);
if (status < 0) {
mlog_errno(status);
return megs << (20 - OCFS2_SB(sb)->s_clustersize_bits);
}
-#define ocfs2_set_bit ext2_set_bit
-#define ocfs2_clear_bit ext2_clear_bit
+static inline void _ocfs2_set_bit(unsigned int bit, unsigned long *bitmap)
+{
+ ext2_set_bit(bit, bitmap);
+}
+#define ocfs2_set_bit(bit, addr) _ocfs2_set_bit((bit), (unsigned long *)(addr))
+
+static inline void _ocfs2_clear_bit(unsigned int bit, unsigned long *bitmap)
+{
+ ext2_clear_bit(bit, bitmap);
+}
+#define ocfs2_clear_bit(bit, addr) _ocfs2_clear_bit((bit), (unsigned long *)(addr))
+
#define ocfs2_test_bit ext2_test_bit
#define ocfs2_find_next_zero_bit ext2_find_next_zero_bit
#define ocfs2_find_next_bit ext2_find_next_bit
*/
#include <linux/spinlock.h>
#include <linux/fs.h>
+#include <linux/slab.h>
#include <linux/quota.h>
#include <linux/quotaops.h>
#include <linux/dqblk_qtree.h>
*/
#include <linux/fs.h>
+#include <linux/slab.h>
#include <linux/quota.h>
#include <linux/quotaops.h>
#include <linux/module.h>
#include <linux/bio.h>
#include <linux/blkdev.h>
-#include <linux/gfp.h>
#include <linux/slab.h>
#include <linux/writeback.h>
#include <linux/pagevec.h>
OCFS2_I(t_inode)->ip_dyn_features = OCFS2_I(s_inode)->ip_dyn_features;
spin_unlock(&OCFS2_I(t_inode)->ip_lock);
i_size_write(t_inode, size);
+ t_inode->i_blocks = s_inode->i_blocks;
di->i_xattr_inline_size = s_di->i_xattr_inline_size;
di->i_clusters = s_di->i_clusters;
#include <linux/kernel.h>
#include <linux/crc32.h>
+#include <linux/slab.h>
#include <linux/module.h>
/* Needed for AOP_TRUNCATED_PAGE in mlog_errno() */
#include <linux/fs.h>
#include <linux/miscdevice.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <linux/smp_lock.h>
#include <linux/reboot.h>
#include <asm/uaccess.h>
struct buffer_head *group_bh,
unsigned int bit_off,
unsigned int num_bits);
-static inline int ocfs2_block_group_clear_bits(handle_t *handle,
- struct inode *alloc_inode,
- struct ocfs2_group_desc *bg,
- struct buffer_head *group_bh,
- unsigned int bit_off,
- unsigned int num_bits);
-
static int ocfs2_relink_block_group(handle_t *handle,
struct inode *alloc_inode,
struct buffer_head *fe_bh,
#define do_error(fmt, ...) \
do{ \
- if (clean_error) \
+ if (resize) \
mlog(ML_ERROR, fmt "\n", ##__VA_ARGS__); \
else \
ocfs2_error(sb, fmt, ##__VA_ARGS__); \
static int ocfs2_validate_gd_self(struct super_block *sb,
struct buffer_head *bh,
- int clean_error)
+ int resize)
{
struct ocfs2_group_desc *gd = (struct ocfs2_group_desc *)bh->b_data;
static int ocfs2_validate_gd_parent(struct super_block *sb,
struct ocfs2_dinode *di,
struct buffer_head *bh,
- int clean_error)
+ int resize)
{
unsigned int max_bits;
struct ocfs2_group_desc *gd = (struct ocfs2_group_desc *)bh->b_data;
return -EINVAL;
}
- if (le16_to_cpu(gd->bg_chain) >=
- le16_to_cpu(di->id2.i_chain.cl_next_free_rec)) {
+ /* In resize, we may meet the case bg_chain == cl_next_free_rec. */
+ if ((le16_to_cpu(gd->bg_chain) >
+ le16_to_cpu(di->id2.i_chain.cl_next_free_rec)) ||
+ ((le16_to_cpu(gd->bg_chain) ==
+ le16_to_cpu(di->id2.i_chain.cl_next_free_rec)) && !resize)) {
do_error("Group descriptor #%llu has bad chain %u",
(unsigned long long)bh->b_blocknr,
le16_to_cpu(gd->bg_chain));
bits_wanted, cluster_start, num_clusters);
}
-static inline int ocfs2_block_group_clear_bits(handle_t *handle,
- struct inode *alloc_inode,
- struct ocfs2_group_desc *bg,
- struct buffer_head *group_bh,
- unsigned int bit_off,
- unsigned int num_bits)
+static int ocfs2_block_group_clear_bits(handle_t *handle,
+ struct inode *alloc_inode,
+ struct ocfs2_group_desc *bg,
+ struct buffer_head *group_bh,
+ unsigned int bit_off,
+ unsigned int num_bits,
+ void (*undo_fn)(unsigned int bit,
+ unsigned long *bmap))
{
int status;
unsigned int tmp;
- int journal_type = OCFS2_JOURNAL_ACCESS_WRITE;
struct ocfs2_group_desc *undo_bg = NULL;
- int cluster_bitmap = 0;
mlog_entry_void();
mlog(0, "off = %u, num = %u\n", bit_off, num_bits);
- if (ocfs2_is_cluster_bitmap(alloc_inode))
- journal_type = OCFS2_JOURNAL_ACCESS_UNDO;
-
+ BUG_ON(undo_fn && !ocfs2_is_cluster_bitmap(alloc_inode));
status = ocfs2_journal_access_gd(handle, INODE_CACHE(alloc_inode),
- group_bh, journal_type);
+ group_bh,
+ undo_fn ?
+ OCFS2_JOURNAL_ACCESS_UNDO :
+ OCFS2_JOURNAL_ACCESS_WRITE);
if (status < 0) {
mlog_errno(status);
goto bail;
}
- if (ocfs2_is_cluster_bitmap(alloc_inode))
- cluster_bitmap = 1;
-
- if (cluster_bitmap) {
+ if (undo_fn) {
jbd_lock_bh_state(group_bh);
undo_bg = (struct ocfs2_group_desc *)
bh2jh(group_bh)->b_committed_data;
while(tmp--) {
ocfs2_clear_bit((bit_off + tmp),
(unsigned long *) bg->bg_bitmap);
- if (cluster_bitmap)
- ocfs2_set_bit(bit_off + tmp,
- (unsigned long *) undo_bg->bg_bitmap);
+ if (undo_fn)
+ undo_fn(bit_off + tmp,
+ (unsigned long *) undo_bg->bg_bitmap);
}
le16_add_cpu(&bg->bg_free_bits_count, num_bits);
- if (cluster_bitmap)
+ if (undo_fn)
jbd_unlock_bh_state(group_bh);
status = ocfs2_journal_dirty(handle, group_bh);
/*
* expects the suballoc inode to already be locked.
*/
-int ocfs2_free_suballoc_bits(handle_t *handle,
- struct inode *alloc_inode,
- struct buffer_head *alloc_bh,
- unsigned int start_bit,
- u64 bg_blkno,
- unsigned int count)
+static int _ocfs2_free_suballoc_bits(handle_t *handle,
+ struct inode *alloc_inode,
+ struct buffer_head *alloc_bh,
+ unsigned int start_bit,
+ u64 bg_blkno,
+ unsigned int count,
+ void (*undo_fn)(unsigned int bit,
+ unsigned long *bitmap))
{
int status = 0;
u32 tmp_used;
status = ocfs2_block_group_clear_bits(handle, alloc_inode,
group, group_bh,
- start_bit, count);
+ start_bit, count, undo_fn);
if (status < 0) {
mlog_errno(status);
goto bail;
return status;
}
+int ocfs2_free_suballoc_bits(handle_t *handle,
+ struct inode *alloc_inode,
+ struct buffer_head *alloc_bh,
+ unsigned int start_bit,
+ u64 bg_blkno,
+ unsigned int count)
+{
+ return _ocfs2_free_suballoc_bits(handle, alloc_inode, alloc_bh,
+ start_bit, bg_blkno, count, NULL);
+}
+
int ocfs2_free_dinode(handle_t *handle,
struct inode *inode_alloc_inode,
struct buffer_head *inode_alloc_bh,
inode_alloc_bh, bit, bg_blkno, 1);
}
-int ocfs2_free_clusters(handle_t *handle,
- struct inode *bitmap_inode,
- struct buffer_head *bitmap_bh,
- u64 start_blk,
- unsigned int num_clusters)
+static int _ocfs2_free_clusters(handle_t *handle,
+ struct inode *bitmap_inode,
+ struct buffer_head *bitmap_bh,
+ u64 start_blk,
+ unsigned int num_clusters,
+ void (*undo_fn)(unsigned int bit,
+ unsigned long *bitmap))
{
int status;
u16 bg_start_bit;
mlog(0, "bg_blkno = %llu, bg_start_bit = %u\n",
(unsigned long long)bg_blkno, bg_start_bit);
- status = ocfs2_free_suballoc_bits(handle, bitmap_inode, bitmap_bh,
- bg_start_bit, bg_blkno,
- num_clusters);
+ status = _ocfs2_free_suballoc_bits(handle, bitmap_inode, bitmap_bh,
+ bg_start_bit, bg_blkno,
+ num_clusters, undo_fn);
if (status < 0) {
mlog_errno(status);
goto out;
return status;
}
+int ocfs2_free_clusters(handle_t *handle,
+ struct inode *bitmap_inode,
+ struct buffer_head *bitmap_bh,
+ u64 start_blk,
+ unsigned int num_clusters)
+{
+ return _ocfs2_free_clusters(handle, bitmap_inode, bitmap_bh,
+ start_blk, num_clusters,
+ _ocfs2_set_bit);
+}
+
+/*
+ * Give never-used clusters back to the global bitmap. We don't need
+ * to protect these bits in the undo buffer.
+ */
+int ocfs2_release_clusters(handle_t *handle,
+ struct inode *bitmap_inode,
+ struct buffer_head *bitmap_bh,
+ u64 start_blk,
+ unsigned int num_clusters)
+{
+ return _ocfs2_free_clusters(handle, bitmap_inode, bitmap_bh,
+ start_blk, num_clusters,
+ _ocfs2_clear_bit);
+}
+
static inline void ocfs2_debug_bg(struct ocfs2_group_desc *bg)
{
printk("Block Group:\n");
struct buffer_head *bitmap_bh,
u64 start_blk,
unsigned int num_clusters);
+int ocfs2_release_clusters(handle_t *handle,
+ struct inode *bitmap_inode,
+ struct buffer_head *bitmap_bh,
+ u64 start_blk,
+ unsigned int num_clusters);
static inline u64 ocfs2_which_suballoc_group(u64 block, unsigned int bit)
{
#include <linux/fs.h>
#include <linux/types.h>
-#include <linux/slab.h>
#include <linux/highmem.h>
#define MLOG_MASK_PREFIX ML_INODE
/* Now tell xh->xh_entries about it */
for (i = 0; i < count; i++) {
offset = le16_to_cpu(xh->xh_entries[i].xe_name_offset);
- if (offset < namevalue_offset)
+ if (offset <= namevalue_offset)
le16_add_cpu(&xh->xh_entries[i].xe_name_offset,
namevalue_size);
}
int indexed)
{
int ret;
- struct ocfs2_alloc_context *meta_ac;
struct ocfs2_super *osb = OCFS2_SB(inode->i_sb);
- struct ocfs2_xattr_set_ctxt ctxt = {
- .meta_ac = meta_ac,
- };
+ struct ocfs2_xattr_set_ctxt ctxt;
- ret = ocfs2_reserve_new_metadata_blocks(osb, 1, &meta_ac);
+ memset(&ctxt, 0, sizeof(ctxt));
+ ret = ocfs2_reserve_new_metadata_blocks(osb, 1, &ctxt.meta_ac);
if (ret < 0) {
mlog_errno(ret);
return ret;
ocfs2_commit_trans(osb, ctxt.handle);
out:
- ocfs2_free_alloc_context(meta_ac);
+ ocfs2_free_alloc_context(ctxt.meta_ac);
return ret;
}
#include <linux/version.h>
#include <linux/module.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/fs.h>
#include <linux/vfs.h>
#include <linux/parser.h>
#include <linux/fdtable.h>
#include <linux/fsnotify.h>
#include <linux/module.h>
-#include <linux/slab.h>
#include <linux/tty.h>
#include <linux/namei.h>
#include <linux/backing-dev.h>
#include <linux/mount.h>
#include <linux/vfs.h>
#include <linux/fcntl.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <linux/fs.h>
#include <linux/personality.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/fs.h>
+#include <linux/slab.h>
#include <linux/kmod.h>
#include <linux/ctype.h>
#include <linux/genhd.h>
************************************************************/
#include <linux/crc32.h>
#include <linux/math64.h>
+#include <linux/slab.h>
#include "check.h"
#include "efi.h"
*/
#include <asm/unaligned.h>
-#define SYS_IND(p) (get_unaligned(&p->sys_ind))
-#define NR_SECTS(p) ({ __le32 __a = get_unaligned(&p->nr_sects); \
- le32_to_cpu(__a); \
- })
+#define SYS_IND(p) get_unaligned(&p->sys_ind)
-#define START_SECT(p) ({ __le32 __a = get_unaligned(&p->start_sect); \
- le32_to_cpu(__a); \
- })
+static inline sector_t nr_sects(struct partition *p)
+{
+ return (sector_t)get_unaligned_le32(&p->nr_sects);
+}
+
+static inline sector_t start_sect(struct partition *p)
+{
+ return (sector_t)get_unaligned_le32(&p->start_sect);
+}
static inline int is_extended_partition(struct partition *p)
{
static void
parse_extended(struct parsed_partitions *state, struct block_device *bdev,
- u32 first_sector, u32 first_size)
+ sector_t first_sector, sector_t first_size)
{
struct partition *p;
Sector sect;
unsigned char *data;
- u32 this_sector, this_size;
- int sector_size = bdev_logical_block_size(bdev) / 512;
+ sector_t this_sector, this_size;
+ sector_t sector_size = bdev_logical_block_size(bdev) / 512;
int loopct = 0; /* number of links followed
without finding a data partition */
int i;
* First process the data partition(s)
*/
for (i=0; i<4; i++, p++) {
- u32 offs, size, next;
- if (!NR_SECTS(p) || is_extended_partition(p))
+ sector_t offs, size, next;
+ if (!nr_sects(p) || is_extended_partition(p))
continue;
/* Check the 3rd and 4th entries -
these sometimes contain random garbage */
- offs = START_SECT(p)*sector_size;
- size = NR_SECTS(p)*sector_size;
+ offs = start_sect(p)*sector_size;
+ size = nr_sects(p)*sector_size;
next = this_sector + offs;
if (i >= 2) {
if (offs + size > this_size)
*/
p -= 4;
for (i=0; i<4; i++, p++)
- if (NR_SECTS(p) && is_extended_partition(p))
+ if (nr_sects(p) && is_extended_partition(p))
break;
if (i == 4)
goto done; /* nothing left to do */
- this_sector = first_sector + START_SECT(p) * sector_size;
- this_size = NR_SECTS(p) * sector_size;
+ this_sector = first_sector + start_sect(p) * sector_size;
+ this_size = nr_sects(p) * sector_size;
put_dev_sector(sect);
}
done:
static void
parse_solaris_x86(struct parsed_partitions *state, struct block_device *bdev,
- u32 offset, u32 size, int origin)
+ sector_t offset, sector_t size, int origin)
{
#ifdef CONFIG_SOLARIS_X86_PARTITION
Sector sect;
*/
static void
parse_bsd(struct parsed_partitions *state, struct block_device *bdev,
- u32 offset, u32 size, int origin, char *flavour,
+ sector_t offset, sector_t size, int origin, char *flavour,
int max_partitions)
{
Sector sect;
if (le16_to_cpu(l->d_npartitions) < max_partitions)
max_partitions = le16_to_cpu(l->d_npartitions);
for (p = l->d_partitions; p - l->d_partitions < max_partitions; p++) {
- u32 bsd_start, bsd_size;
+ sector_t bsd_start, bsd_size;
if (state->next == state->limit)
break;
static void
parse_freebsd(struct parsed_partitions *state, struct block_device *bdev,
- u32 offset, u32 size, int origin)
+ sector_t offset, sector_t size, int origin)
{
#ifdef CONFIG_BSD_DISKLABEL
parse_bsd(state, bdev, offset, size, origin,
static void
parse_netbsd(struct parsed_partitions *state, struct block_device *bdev,
- u32 offset, u32 size, int origin)
+ sector_t offset, sector_t size, int origin)
{
#ifdef CONFIG_BSD_DISKLABEL
parse_bsd(state, bdev, offset, size, origin,
static void
parse_openbsd(struct parsed_partitions *state, struct block_device *bdev,
- u32 offset, u32 size, int origin)
+ sector_t offset, sector_t size, int origin)
{
#ifdef CONFIG_BSD_DISKLABEL
parse_bsd(state, bdev, offset, size, origin,
*/
static void
parse_unixware(struct parsed_partitions *state, struct block_device *bdev,
- u32 offset, u32 size, int origin)
+ sector_t offset, sector_t size, int origin)
{
#ifdef CONFIG_UNIXWARE_DISKLABEL
Sector sect;
if (p->s_label != UNIXWARE_FS_UNUSED)
put_partition(state, state->next++,
- START_SECT(p), NR_SECTS(p));
+ le32_to_cpu(p->start_sect),
+ le32_to_cpu(p->nr_sects));
p++;
}
put_dev_sector(sect);
*/
static void
parse_minix(struct parsed_partitions *state, struct block_device *bdev,
- u32 offset, u32 size, int origin)
+ sector_t offset, sector_t size, int origin)
{
#ifdef CONFIG_MINIX_SUBPARTITION
Sector sect;
/* add each partition in use */
if (SYS_IND(p) == MINIX_PARTITION)
put_partition(state, state->next++,
- START_SECT(p), NR_SECTS(p));
+ start_sect(p), nr_sects(p));
}
printk(" >\n");
}
static struct {
unsigned char id;
void (*parse)(struct parsed_partitions *, struct block_device *,
- u32, u32, int);
+ sector_t, sector_t, int);
} subtypes[] = {
{FREEBSD_PARTITION, parse_freebsd},
{NETBSD_PARTITION, parse_netbsd},
int msdos_partition(struct parsed_partitions *state, struct block_device *bdev)
{
- int sector_size = bdev_logical_block_size(bdev) / 512;
+ sector_t sector_size = bdev_logical_block_size(bdev) / 512;
Sector sect;
unsigned char *data;
struct partition *p;
state->next = 5;
for (slot = 1 ; slot <= 4 ; slot++, p++) {
- u32 start = START_SECT(p)*sector_size;
- u32 size = NR_SECTS(p)*sector_size;
+ sector_t start = start_sect(p)*sector_size;
+ sector_t size = nr_sects(p)*sector_size;
if (!size)
continue;
if (is_extended_partition(p)) {
- /* prevent someone doing mkfs or mkswap on an
- extended partition, but leave room for LILO */
- put_partition(state, slot, start, size == 1 ? 1 : 2);
+ /*
+ * prevent someone doing mkfs or mkswap on an
+ * extended partition, but leave room for LILO
+ * FIXME: this uses one logical sector for > 512b
+ * sector, although it may not be enough/proper.
+ */
+ sector_t n = 2;
+ n = min(size, max(sector_size, n));
+ put_partition(state, slot, start, n);
+
printk(" <");
parse_extended(state, bdev, start, size);
printk(" >");
unsigned char id = SYS_IND(p);
int n;
- if (!NR_SECTS(p))
+ if (!nr_sects(p))
continue;
for (n = 0; subtypes[n].parse && id != subtypes[n].id; n++)
if (!subtypes[n].parse)
continue;
- subtypes[n].parse(state, bdev, START_SECT(p)*sector_size,
- NR_SECTS(p)*sector_size, slot);
+ subtypes[n].parse(state, bdev, start_sect(p)*sector_size,
+ nr_sects(p)*sector_size, slot);
}
put_dev_sector(sect);
return 1;
#include <linux/hugetlb.h>
#include <linux/pagemap.h>
#include <linux/swap.h>
-#include <linux/slab.h>
#include <linux/smp.h>
#include <linux/signal.h>
#include <linux/highmem.h>
#include <linux/elf.h>
#include <linux/pid_namespace.h>
#include <linux/fs_struct.h>
+#include <linux/slab.h>
#include "internal.h"
/* NOTE:
unsigned long badness(struct task_struct *p, unsigned long uptime);
static int proc_oom_score(struct task_struct *task, char *buffer)
{
- unsigned long points;
+ unsigned long points = 0;
struct timespec uptime;
do_posix_clock_monotonic_gettime(&uptime);
read_lock(&tasklist_lock);
- points = badness(task->group_leader, uptime.tv_sec);
+ if (pid_alive(task))
+ points = badness(task, uptime.tv_sec);
read_unlock(&tasklist_lock);
return sprintf(buffer, "%lu\n", points);
}
#include <linux/proc_fs.h>
#include <linux/stat.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/mount.h>
#include <linux/init.h>
#include <linux/idr.h>
#include <linux/module.h>
#include <linux/smp_lock.h>
#include <linux/sysctl.h>
+#include <linux/slab.h>
#include <asm/system.h>
#include <asm/uaccess.h>
#include <linux/highmem.h>
#include <linux/bootmem.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/io.h>
#include <linux/list.h>
}
read_unlock(&kclist_lock);
- if (m == NULL) {
+ if (&m->list == &kclist_head) {
if (clear_user(buffer, tsz))
return -EFAULT;
} else if (is_vmalloc_or_module_addr((void *)start)) {
#include <linux/mmzone.h>
#include <linux/pagemap.h>
#include <linux/swap.h>
-#include <linux/slab.h>
#include <linux/smp.h>
#include <linux/seq_file.h>
#include <linux/hugetlb.h>
#include <linux/string.h>
#include <linux/of.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <asm/prom.h>
#include <asm/uaccess.h>
#include "internal.h"
#include <linux/time.h>
#include <linux/proc_fs.h>
#include <linux/stat.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/sched.h>
#include <linux/module.h>
#include <linux/cpumask.h>
#include <linux/fs.h>
-#include <linux/gfp.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/kernel_stat.h>
#include <linux/seq_file.h>
#include <linux/highmem.h>
#include <linux/ptrace.h>
+#include <linux/slab.h>
#include <linux/pagemap.h>
#include <linux/mempolicy.h>
#include <linux/swap.h>
memset(&mss, 0, sizeof mss);
mss.vma = vma;
+ /* mmap_sem is held in m_start */
if (vma->vm_mm && !is_vm_hugetlb_page(vma))
walk_page_range(vma->vm_start, vma->vm_end, &smaps_walk);
};
struct pagemapread {
- u64 __user *out, *end;
+ int pos, len;
+ u64 *buffer;
};
#define PM_ENTRY_BYTES sizeof(u64)
static int add_to_pagemap(unsigned long addr, u64 pfn,
struct pagemapread *pm)
{
- if (put_user(pfn, pm->out))
- return -EFAULT;
- pm->out++;
- if (pm->out >= pm->end)
+ pm->buffer[pm->pos++] = pfn;
+ if (pm->pos >= pm->len)
return PM_END_OF_BUFFER;
return 0;
}
* determine which areas of memory are actually mapped and llseek to
* skip over unmapped regions.
*/
+#define PAGEMAP_WALK_SIZE (PMD_SIZE)
static ssize_t pagemap_read(struct file *file, char __user *buf,
size_t count, loff_t *ppos)
{
struct task_struct *task = get_proc_task(file->f_path.dentry->d_inode);
- struct page **pages, *page;
- unsigned long uaddr, uend;
struct mm_struct *mm;
struct pagemapread pm;
- int pagecount;
int ret = -ESRCH;
struct mm_walk pagemap_walk = {};
unsigned long src;
unsigned long svpfn;
unsigned long start_vaddr;
unsigned long end_vaddr;
+ int copied = 0;
if (!task)
goto out;
if (!mm)
goto out_task;
-
- uaddr = (unsigned long)buf & PAGE_MASK;
- uend = (unsigned long)(buf + count);
- pagecount = (PAGE_ALIGN(uend) - uaddr) / PAGE_SIZE;
- ret = 0;
- if (pagecount == 0)
- goto out_mm;
- pages = kcalloc(pagecount, sizeof(struct page *), GFP_KERNEL);
+ pm.len = PM_ENTRY_BYTES * (PAGEMAP_WALK_SIZE >> PAGE_SHIFT);
+ pm.buffer = kmalloc(pm.len, GFP_TEMPORARY);
ret = -ENOMEM;
- if (!pages)
+ if (!pm.buffer)
goto out_mm;
- down_read(¤t->mm->mmap_sem);
- ret = get_user_pages(current, current->mm, uaddr, pagecount,
- 1, 0, pages, NULL);
- up_read(¤t->mm->mmap_sem);
-
- if (ret < 0)
- goto out_free;
-
- if (ret != pagecount) {
- pagecount = ret;
- ret = -EFAULT;
- goto out_pages;
- }
-
- pm.out = (u64 __user *)buf;
- pm.end = (u64 __user *)(buf + count);
-
pagemap_walk.pmd_entry = pagemap_pte_range;
pagemap_walk.pte_hole = pagemap_pte_hole;
pagemap_walk.hugetlb_entry = pagemap_hugetlb_range;
* user buffer is tracked in "pm", and the walk
* will stop when we hit the end of the buffer.
*/
- ret = walk_page_range(start_vaddr, end_vaddr, &pagemap_walk);
- if (ret == PM_END_OF_BUFFER)
- ret = 0;
- /* don't need mmap_sem for these, but this looks cleaner */
- *ppos += (char __user *)pm.out - buf;
- if (!ret)
- ret = (char __user *)pm.out - buf;
-
-out_pages:
- for (; pagecount; pagecount--) {
- page = pages[pagecount-1];
- if (!PageReserved(page))
- SetPageDirty(page);
- page_cache_release(page);
+ ret = 0;
+ while (count && (start_vaddr < end_vaddr)) {
+ int len;
+ unsigned long end;
+
+ pm.pos = 0;
+ end = start_vaddr + PAGEMAP_WALK_SIZE;
+ /* overflow ? */
+ if (end < start_vaddr || end > end_vaddr)
+ end = end_vaddr;
+ down_read(&mm->mmap_sem);
+ ret = walk_page_range(start_vaddr, end, &pagemap_walk);
+ up_read(&mm->mmap_sem);
+ start_vaddr = end;
+
+ len = min(count, PM_ENTRY_BYTES * pm.pos);
+ if (copy_to_user(buf, pm.buffer, len)) {
+ ret = -EFAULT;
+ goto out_free;
+ }
+ copied += len;
+ buf += len;
+ count -= len;
}
+ *ppos += copied;
+ if (!ret || ret == PM_END_OF_BUFFER)
+ ret = copied;
+
out_free:
- kfree(pages);
+ kfree(pm.buffer);
out_mm:
mmput(mm);
out_task:
#include <linux/fs_struct.h>
#include <linux/mount.h>
#include <linux/ptrace.h>
+#include <linux/slab.h>
#include <linux/seq_file.h>
#include "internal.h"
#include <linux/user.h>
#include <linux/elf.h>
#include <linux/elfcore.h>
+#include <linux/slab.h>
#include <linux/highmem.h>
#include <linux/bootmem.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/quotaops.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <net/netlink.h>
#include <net/genetlink.h>
#include <linux/pagevec.h>
#include <linux/mman.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include "internal.h"
#include <linux/sched.h>
#include <linux/parser.h>
#include <linux/magic.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include "internal.h"
init_sync_kiocb(&kiocb, filp);
kiocb.ki_pos = *ppos;
kiocb.ki_left = len;
+ kiocb.ki_nbytes = len;
for (;;) {
ret = filp->f_op->aio_read(&kiocb, &iov, 1, kiocb.ki_pos);
init_sync_kiocb(&kiocb, filp);
kiocb.ki_pos = *ppos;
kiocb.ki_left = len;
+ kiocb.ki_nbytes = len;
for (;;) {
ret = filp->f_op->aio_write(&kiocb, &iov, 1, kiocb.ki_pos);
#include <linux/reiserfs_fs.h>
#include <linux/stat.h>
#include <linux/buffer_head.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
extern const struct reiserfs_key MIN_KEY;
**/
#include <linux/time.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include <linux/reiserfs_fs.h>
#include <linux/buffer_head.h>
#include <linux/smp_lock.h>
#include <linux/pagemap.h>
#include <linux/highmem.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/unaligned.h>
#include <linux/buffer_head.h>
#include <linux/blkdev.h>
#include <linux/backing-dev.h>
#include <linux/uaccess.h>
+#include <linux/slab.h>
#include <asm/system.h>
brelse(d_bh);
return 1;
}
+
+ if (bdev_read_only(sb->s_bdev)) {
+ reiserfs_warning(sb, "clm-2076",
+ "device is readonly, unable to replay log");
+ brelse(c_bh);
+ brelse(d_bh);
+ return -EROFS;
+ }
+
trans_id = get_desc_trans_id(desc);
/* now we know we've got a good transaction, and it was inside the valid time ranges */
log_blocks = kmalloc(get_desc_trans_len(desc) *
goto start_log_replay;
}
- if (continue_replay && bdev_read_only(sb->s_bdev)) {
- reiserfs_warning(sb, "clm-2076",
- "device is readonly, unable to replay log");
- return -1;
- }
-
/* ok, there are transactions that need to be replayed. start with the first log block, find
** all the valid transactions, and pick out the oldest.
*/
#include <linux/time.h>
#include <linux/bitops.h>
+#include <linux/slab.h>
#include <linux/reiserfs_fs.h>
#include <linux/reiserfs_acl.h>
#include <linux/reiserfs_xattr.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/time.h>
#include <asm/uaccess.h>
save_mount_options(s, data);
sbi = kzalloc(sizeof(struct reiserfs_sb_info), GFP_KERNEL);
- if (!sbi) {
- errval = -ENOMEM;
- goto error_alloc;
- }
+ if (!sbi)
+ return -ENOMEM;
s->s_fs_info = sbi;
/* Set default values for options: non-aggressive tails, RO on errors */
REISERFS_SB(s)->s_mount_opt |= (1 << REISERFS_SMALLTAIL);
return (0);
error:
- reiserfs_write_unlock(s);
-error_alloc:
if (jinit_done) { /* kill the commit thread, free journal ram */
journal_release_error(NULL, s);
}
+ reiserfs_write_unlock(s);
+
reiserfs_free_bitmap_cache(s);
if (SB_BUFFER_WITH_SB(s))
brelse(SB_BUFFER_WITH_SB(s));
#include <linux/dcache.h>
#include <linux/namei.h>
#include <linux/errno.h>
+#include <linux/gfp.h>
#include <linux/fs.h>
#include <linux/file.h>
#include <linux/pagemap.h>
#include <linux/errno.h>
#include <linux/pagemap.h>
#include <linux/xattr.h>
+#include <linux/slab.h>
#include <linux/posix_acl_xattr.h>
#include <linux/reiserfs_xattr.h>
#include <linux/reiserfs_acl.h>
#include <linux/fs.h>
#include <linux/pagemap.h>
#include <linux/xattr.h>
+#include <linux/slab.h>
#include <linux/reiserfs_xattr.h>
#include <linux/security.h>
#include <asm/uaccess.h>
return error;
}
- if (sec->length) {
+ if (sec->length && reiserfs_xattrs_initialized(inode->i_sb)) {
blocks = reiserfs_xattr_jcreate_nblocks(inode) +
reiserfs_xattr_nblocks(inode, sec->length);
/* We don't want to count the directories twice if we have
#include <linux/init.h>
#include <linux/fs.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/signal.h>
#include <linux/list.h>
#include <linux/fcntl.h>
#include <linux/stat.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/pagemap.h>
#include <linux/smp_lock.h>
#include <linux/net.h>
#include <linux/string.h>
#include <linux/stat.h>
#include <linux/errno.h>
-#include <linux/slab.h>
#include <linux/init.h>
#include <linux/file.h>
#include <linux/dcache.h>
#include <linux/pagemap.h>
#include <linux/net.h>
#include <linux/namei.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/system.h>
#include <linux/syscalls.h>
#include <linux/uio.h>
#include <linux/security.h>
+#include <linux/gfp.h>
/*
* Attempt to steal a page from a pipe buffer. This should perhaps go into
#include <linux/fs.h>
#include <linux/vfs.h>
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/pagemap.h>
#include <linux/mutex.h>
#include <linux/buffer_head.h>
+#include <linux/slab.h>
#include <linux/zlib.h>
#include "squashfs_fs.h"
#include <linux/kernel.h>
#include <linux/file.h>
#include <linux/fs.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/sched.h>
#include <linux/writeback.h>
#include <linux/capability.h>
#include <linux/errno.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/xattr.h>
#include <linux/security.h>
#include "sysfs.h"
#include <linux/init.h>
#include <linux/module.h>
#include <linux/magic.h>
+#include <linux/slab.h>
#include "sysfs.h"
*/
#include <linux/fs.h>
+#include <linux/gfp.h>
#include <linux/mount.h>
#include <linux/module.h>
#include <linux/kobject.h>
#include <linux/fs.h>
#include <linux/sched.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/list.h>
#include <linux/spinlock.h>
#include <linux/time.h>
#include <linux/freezer.h>
#include <linux/kthread.h>
+#include <linux/slab.h>
#include "ubifs.h"
/**
#include <linux/moduleparam.h>
#include <linux/debugfs.h>
#include <linux/math64.h>
+#include <linux/slab.h>
#ifdef CONFIG_UBIFS_FS_DEBUG
#include "ubifs.h"
#include <linux/mount.h>
#include <linux/namei.h>
+#include <linux/slab.h>
static int read_block(struct inode *inode, void *addr, unsigned int block,
struct ubifs_data_node *dn)
* good, and GC takes extra care when moving them.
*/
+#include <linux/slab.h>
#include <linux/pagemap.h>
#include <linux/list_sort.h>
#include "ubifs.h"
*/
#include <linux/crc32.h>
+#include <linux/slab.h>
#include "ubifs.h"
/**
#include "ubifs.h"
#include <linux/crc16.h>
#include <linux/math64.h>
+#include <linux/slab.h>
/**
* do_calc_lpt_geom - calculate sizes for the LPT area.
*/
#include <linux/crc16.h>
+#include <linux/slab.h>
#include "ubifs.h"
/**
*/
#include <linux/crc32.h>
+#include <linux/slab.h>
#include "ubifs.h"
/**
*/
#include "ubifs.h"
+#include <linux/slab.h>
#include <linux/random.h>
#include <linux/math64.h>
*/
#include <linux/crc32.h>
+#include <linux/slab.h>
#include "ubifs.h"
/*
#include <linux/fs.h>
#include <linux/err.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/spinlock.h>
#include <linux/mutex.h>
*/
#include "ubifs.h"
+#include <linux/slab.h>
#include <linux/xattr.h>
#include <linux/posix_acl_xattr.h>
#include <linux/fs.h>
#include <linux/string.h>
-#include <linux/slab.h>
#include <linux/buffer_head.h>
uint32_t udf_get_pblock(struct super_block *sb, uint32_t block,
#include <linux/time.h>
#include <linux/mm.h>
#include <linux/stat.h>
-#include <linux/slab.h>
#include <linux/pagemap.h>
#include <linux/smp_lock.h>
#include <linux/buffer_head.h>
#include <linux/string.h> /* for memset */
#include <linux/nls.h>
#include <linux/crc-itu-t.h>
+#include <linux/slab.h>
#include "udf_sb.h"
*/
#include <linux/module.h>
-#include <linux/slab.h>
#include <linux/fs.h>
#include <linux/posix_acl_xattr.h>
+#include <linux/gfp.h>
/*
*/
#include <linux/mm.h>
#include <linux/highmem.h>
+#include <linux/slab.h>
#include <linux/swap.h>
#include <linux/blkdev.h>
#include <linux/backing-dev.h>
#include "xfs_inode.h"
#include "xfs_vnodeops.h"
#include "xfs_trace.h"
+#include <linux/slab.h>
#include <linux/xattr.h>
#include <linux/posix_acl_xattr.h>
#include "xfs_vnodeops.h"
#include "xfs_trace.h"
#include "xfs_bmap.h"
+#include <linux/gfp.h>
#include <linux/mpage.h>
#include <linux/pagevec.h>
#include <linux/writeback.h>
#include "xfs.h"
#include <linux/stddef.h>
#include <linux/errno.h>
-#include <linux/slab.h>
+#include <linux/gfp.h>
#include <linux/pagemap.h>
#include <linux/init.h>
#include <linux/vmalloc.h>
#include <linux/mount.h>
#include <linux/namei.h>
#include <linux/pagemap.h>
+#include <linux/slab.h>
#include <linux/exportfs.h>
/*
#include <linux/compat.h>
#include <linux/ioctl.h>
#include <linux/mount.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include "xfs.h"
#include "xfs_fs.h"
#include <linux/security.h>
#include <linux/falloc.h>
#include <linux/fiemap.h>
+#include <linux/slab.h>
/*
* Bring the timestamps in the XFS inode uptodate.
#include <linux/namei.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/mount.h>
#include <linux/mempool.h>
#include <linux/writeback.h>
#define ACPI_VIDEO_HID "LNXVIDEO"
#define ACPI_BAY_HID "LNXIOBAY"
#define ACPI_DOCK_HID "LNXDOCK"
+/* Quirk for broken IBM BIOSes */
+#define ACPI_SMBUS_IBM_HID "SMBUSIBM"
/*
* For fixed hardware buttons, we fabricate acpi_devices with HID
#include <linux/mm.h>
#include <linux/cdev.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#if defined(__alpha__) || defined(__powerpc__)
#include <asm/pgtable.h> /* For pte_wrprotect */
#endif
{
}
-
-static __inline__ void *drm_calloc_large(size_t nmemb, size_t size)
-{
- if (size != 0 && nmemb > ULONG_MAX / size)
- return NULL;
-
- if (size * nmemb <= PAGE_SIZE)
- return kcalloc(nmemb, size, GFP_KERNEL);
-
- return __vmalloc(size * nmemb,
- GFP_KERNEL | __GFP_HIGHMEM | __GFP_ZERO, PAGE_KERNEL);
-}
-
-/* Modeled after cairo's malloc_ab, it's like calloc but without the zeroing. */
-static __inline__ void *drm_malloc_ab(size_t nmemb, size_t size)
-{
- if (size != 0 && nmemb > ULONG_MAX / size)
- return NULL;
-
- if (size * nmemb <= PAGE_SIZE)
- return kmalloc(nmemb * size, GFP_KERNEL);
-
- return __vmalloc(size * nmemb,
- GFP_KERNEL | __GFP_HIGHMEM, PAGE_KERNEL);
-}
-
-static __inline void drm_free_large(void *ptr)
-{
- if (!is_vmalloc_addr(ptr))
- return kfree(ptr);
-
- vfree(ptr);
-}
+#include "drm_mem_util.h"
/*@}*/
#endif /* __KERNEL__ */
--- /dev/null
+/*
+ * Copyright © 2008 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ * Authors:
+ * Jesse Barnes <jbarnes@virtuousgeek.org>
+ *
+ */
+#ifndef _DRM_MEM_UTIL_H_
+#define _DRM_MEM_UTIL_H_
+
+#include <linux/vmalloc.h>
+
+static __inline__ void *drm_calloc_large(size_t nmemb, size_t size)
+{
+ if (size != 0 && nmemb > ULONG_MAX / size)
+ return NULL;
+
+ if (size * nmemb <= PAGE_SIZE)
+ return kcalloc(nmemb, size, GFP_KERNEL);
+
+ return __vmalloc(size * nmemb,
+ GFP_KERNEL | __GFP_HIGHMEM | __GFP_ZERO, PAGE_KERNEL);
+}
+
+/* Modeled after cairo's malloc_ab, it's like calloc but without the zeroing. */
+static __inline__ void *drm_malloc_ab(size_t nmemb, size_t size)
+{
+ if (size != 0 && nmemb > ULONG_MAX / size)
+ return NULL;
+
+ if (size * nmemb <= PAGE_SIZE)
+ return kmalloc(nmemb * size, GFP_KERNEL);
+
+ return __vmalloc(size * nmemb,
+ GFP_KERNEL | __GFP_HIGHMEM, PAGE_KERNEL);
+}
+
+static __inline void drm_free_large(void *ptr)
+{
+ if (!is_vmalloc_addr(ptr))
+ return kfree(ptr);
+
+ vfree(ptr);
+}
+
+#endif
{0x1002, 0x9712, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_RS880|RADEON_IS_MOBILITY|RADEON_NEW_MEMMAP|RADEON_IS_IGP}, \
{0x1002, 0x9713, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_RS880|RADEON_IS_MOBILITY|RADEON_NEW_MEMMAP|RADEON_IS_IGP}, \
{0x1002, 0x9714, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_RS880|RADEON_NEW_MEMMAP|RADEON_IS_IGP}, \
+ {0x1002, 0x9715, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_RS880|RADEON_NEW_MEMMAP|RADEON_IS_IGP}, \
{0, 0, 0}
#define r128_PCI_IDS \
struct ttm_backend_func *func;
};
-#define TTM_PAGE_FLAG_VMALLOC (1 << 0)
#define TTM_PAGE_FLAG_USER (1 << 1)
#define TTM_PAGE_FLAG_USER_DIRTY (1 << 2)
#define TTM_PAGE_FLAG_WRITE (1 << 3)
#ifndef ASMARM_AMBA_H
#define ASMARM_AMBA_H
+#include <linux/device.h>
+#include <linux/resource.h>
+
#define AMBA_NR_IRQS 2
struct amba_device {
+#include <linux/types.h>
+
/* platform data for the PL061 GPIO driver */
struct pl061_platform_data {
+/*
+ * See Documentation/circular-buffers.txt for more information.
+ */
+
#ifndef _LINUX_CIRC_BUF_H
#define _LINUX_CIRC_BUF_H 1
* @list: list head for the management code
* @mode: operating mode assigned by the management code
* @next_event: local storage for the next event in oneshot mode
+ * @retries: number of forced programming retries
*/
struct clock_event_device {
const char *name;
struct list_head list;
enum clock_event_mode mode;
ktime_t next_event;
+ unsigned long retries;
};
/*
#define _LINUX_DELAYACCT_H
#include <linux/sched.h>
+#include <linux/slab.h>
/*
* Per-task flags relevant to delay accounting
static inline int ext3_test_inode_state(struct inode *inode, int bit)
{
- return test_bit(bit, &EXT3_I(inode)->i_state);
+ return test_bit(bit, &EXT3_I(inode)->i_state_flags);
}
static inline void ext3_set_inode_state(struct inode *inode, int bit)
{
- set_bit(bit, &EXT3_I(inode)->i_state);
+ set_bit(bit, &EXT3_I(inode)->i_state_flags);
}
static inline void ext3_clear_inode_state(struct inode *inode, int bit)
{
- clear_bit(bit, &EXT3_I(inode)->i_state);
+ clear_bit(bit, &EXT3_I(inode)->i_state_flags);
}
#else
/* Assume that user mode programs are passing in an ext3fs superblock, not
* near to their parent directory's inode.
*/
__u32 i_block_group;
- unsigned long i_state; /* Dynamic state flags for ext3 */
+ unsigned long i_state_flags; /* Dynamic state flags for ext3 */
/* block reservation info */
struct ext3_block_alloc_info *i_block_alloc_info;
extern void cancel_freezing(struct task_struct *p);
#ifdef CONFIG_CGROUP_FREEZER
-extern int cgroup_frozen(struct task_struct *task);
+extern int cgroup_freezing_or_frozen(struct task_struct *task);
#else /* !CONFIG_CGROUP_FREEZER */
-static inline int cgroup_frozen(struct task_struct *task) { return 0; }
+static inline int cgroup_freezing_or_frozen(struct task_struct *task)
+{
+ return 0;
+}
#endif /* !CONFIG_CGROUP_FREEZER */
/*
/* operation releaser */
fscache_operation_release_t release;
-#ifdef CONFIG_SLOW_WORK_PROC
+#ifdef CONFIG_SLOW_WORK_DEBUG
const char *name; /* operation name */
const char *state; /* operation state */
#define fscache_set_op_name(OP, N) do { (OP)->name = (N); } while(0)
#include <linux/inotify.h>
#include <linux/fsnotify_backend.h>
#include <linux/audit.h>
+#include <linux/slab.h>
/*
* fsnotify_d_instantiate - instantiate a dentry for inode
#include <linux/mutex.h>
#include <linux/device.h>
#include <linux/timer.h>
+#include <linux/slab.h>
struct gameport {
* @report_fixup: called before report descriptor parsing (NULL means nop)
* @input_mapping: invoked on input registering before mapping an usage
* @input_mapped: invoked on input registering after mapping an usage
+ * @suspend: invoked on suspend (NULL means nop)
+ * @resume: invoked on resume if device was not reset (NULL means nop)
+ * @reset_resume: invoked on resume if device was reset (NULL means nop)
*
* raw_event and event should return 0 on no action performed, 1 when no
* further processing should be done and negative on error
int (*input_mapped)(struct hid_device *hdev,
struct hid_input *hidinput, struct hid_field *field,
struct hid_usage *usage, unsigned long **bit, int *max);
+#ifdef CONFIG_PM
+ int (*suspend)(struct hid_device *hdev, pm_message_t message);
+ int (*resume)(struct hid_device *hdev);
+ int (*reset_resume)(struct hid_device *hdev);
+#endif
/* private: */
struct device_driver driver;
};
#define _IF_TUNNEL_H_
#include <linux/types.h>
+#include <asm/byteorder.h>
#ifdef __KERNEL__
#include <linux/ip.h>
#define _LINUX_IO_MAPPING_H
#include <linux/types.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/page.h>
#include <asm/iomap.h>
extern struct resource ioport_resource;
extern struct resource iomem_resource;
+extern struct resource *request_resource_conflict(struct resource *root, struct resource *new);
extern int request_resource(struct resource *root, struct resource *new);
extern int release_resource(struct resource *new);
void release_child_resources(struct resource *new);
extern void reserve_region_with_split(struct resource *root,
resource_size_t start, resource_size_t end,
const char *name);
+extern struct resource *insert_resource_conflict(struct resource *parent, struct resource *new);
extern int insert_resource(struct resource *parent, struct resource *new);
extern void insert_resource_expand_to_fit(struct resource *root, struct resource *new);
extern int allocate_resource(struct resource *root, struct resource *new,
#include <linux/mutex.h>
#include <linux/timer.h>
#include <linux/lockdep.h>
+#include <linux/slab.h>
#define journal_oom_retry 1
#include <linux/bit_spinlock.h>
#include <linux/mutex.h>
#include <linux/timer.h>
+#include <linux/slab.h>
#endif
#define journal_oom_retry 1
unsigned char name##kfifo_buffer[size]; \
struct kfifo name = __kfifo_initializer(size, name##kfifo_buffer)
-#undef __kfifo_initializer
-
extern void kfifo_init(struct kfifo *fifo, void *buffer,
unsigned int size);
extern __must_check int kfifo_alloc(struct kfifo *fifo, unsigned int size,
ATA_DFLAG_SLEEPING = (1 << 15), /* device is sleeping */
ATA_DFLAG_DUBIOUS_XFER = (1 << 16), /* data transfer not verified */
ATA_DFLAG_NO_UNLOAD = (1 << 17), /* device doesn't support unload */
+ ATA_DFLAG_UNLOCK_HPA = (1 << 18), /* unlock HPA */
ATA_DFLAG_INIT_MASK = (1 << 24) - 1,
ATA_DFLAG_DETACH = (1 << 24),
#define EXT_CSD_CARD_TYPE_26 (1<<0) /* Card can run at 26MHz */
#define EXT_CSD_CARD_TYPE_52 (1<<1) /* Card can run at 52MHz */
+#define EXT_CSD_CARD_TYPE_MASK 0x3 /* Mask out reserved and DDR bits */
#define EXT_CSD_BUS_WIDTH_1 0 /* Card is in 1 bit mode */
#define EXT_CSD_BUS_WIDTH_4 1 /* Card is in 4 bit mode */
struct module_notes_attrs *notes_attrs;
#endif
+#ifdef CONFIG_SMP
/* Per-cpu data. */
- void *percpu;
+ void __percpu *percpu;
+ unsigned int percpu_size;
+#endif
/* The command line arguments (may be mangled). People like
keeping pointers to this stuff */
void (*exit)(void);
struct module_ref {
- int count;
+ unsigned int incs;
+ unsigned int decs;
} __percpu *refptr;
#endif
struct module *__module_text_address(unsigned long addr);
struct module *__module_address(unsigned long addr);
bool is_module_address(unsigned long addr);
+bool is_module_percpu_address(unsigned long addr);
bool is_module_text_address(unsigned long addr);
static inline int within_module_core(unsigned long addr, struct module *mod)
{
if (module) {
preempt_disable();
- __this_cpu_inc(module->refptr->count);
+ __this_cpu_inc(module->refptr->incs);
trace_module_get(module, _THIS_IP_,
- __this_cpu_read(module->refptr->count));
+ __this_cpu_read(module->refptr->incs));
preempt_enable();
}
}
preempt_disable();
if (likely(module_is_live(module))) {
- __this_cpu_inc(module->refptr->count);
+ __this_cpu_inc(module->refptr->incs);
trace_module_get(module, _THIS_IP_,
- __this_cpu_read(module->refptr->count));
- }
- else
+ __this_cpu_read(module->refptr->incs));
+ } else
ret = 0;
preempt_enable();
return false;
}
+static inline bool is_module_percpu_address(unsigned long addr)
+{
+ return false;
+}
+
static inline bool is_module_text_address(unsigned long addr)
{
return false;
* duplicates except for 802.3ad ETH_P_SLOW, alb non-mcast/bcast, and
* ARP on active-backup slaves with arp_validate enabled.
*/
-static inline int skb_bond_should_drop(struct sk_buff *skb)
+static inline int skb_bond_should_drop(struct sk_buff *skb,
+ struct net_device *master)
{
- struct net_device *dev = skb->dev;
- struct net_device *master = dev->master;
-
if (master) {
+ struct net_device *dev = skb->dev;
+
if (master->priv_flags & IFF_MASTER_ARPMON)
dev->last_rx = jiffies;
extern int nfnetlink_has_listeners(struct net *net, unsigned int group);
extern int nfnetlink_send(struct sk_buff *skb, struct net *net, u32 pid, unsigned group,
int echo, gfp_t flags);
-extern void nfnetlink_set_err(struct net *net, u32 pid, u32 group, int error);
+extern int nfnetlink_set_err(struct net *net, u32 pid, u32 group, int error);
extern int nfnetlink_unicast(struct sk_buff *skb, struct net *net, u_int32_t pid, int flags);
extern void nfnl_lock(void);
enum nf_ip6_hook_priorities {
NF_IP6_PRI_FIRST = INT_MIN,
NF_IP6_PRI_CONNTRACK_DEFRAG = -400,
+ NF_IP6_PRI_RAW = -300,
NF_IP6_PRI_SELINUX_FIRST = -225,
NF_IP6_PRI_CONNTRACK = -200,
NF_IP6_PRI_MANGLE = -150,
extern int netlink_unicast(struct sock *ssk, struct sk_buff *skb, __u32 pid, int nonblock);
extern int netlink_broadcast(struct sock *ssk, struct sk_buff *skb, __u32 pid,
__u32 group, gfp_t allocation);
-extern void netlink_set_err(struct sock *ssk, __u32 pid, __u32 group, int code);
+extern int netlink_set_err(struct sock *ssk, __u32 pid, __u32 group, int code);
extern int netlink_register_notifier(struct notifier_block *nb);
extern int netlink_unregister_notifier(struct notifier_block *nb);
#define __LINUX_PERCPU_H
#include <linux/preempt.h>
-#include <linux/slab.h> /* For kmalloc() */
#include <linux/smp.h>
#include <linux/cpumask.h>
#include <linux/pfn.h>
+#include <linux/init.h>
#include <asm/percpu.h>
#define per_cpu_ptr(ptr, cpu) SHIFT_PERCPU_PTR((ptr), per_cpu_offset((cpu)))
extern void __percpu *__alloc_reserved_percpu(size_t size, size_t align);
-extern void __percpu *__alloc_percpu(size_t size, size_t align);
-extern void free_percpu(void __percpu *__pdata);
-extern phys_addr_t per_cpu_ptr_to_phys(void *addr);
+extern bool is_kernel_percpu_address(unsigned long addr);
#ifndef CONFIG_HAVE_SETUP_PER_CPU_AREA
extern void __init setup_per_cpu_areas(void);
#define per_cpu_ptr(ptr, cpu) ({ (void)(cpu); (ptr); })
-static inline void __percpu *__alloc_percpu(size_t size, size_t align)
+/* can't distinguish from other static vars, always false */
+static inline bool is_kernel_percpu_address(unsigned long addr)
{
- /*
- * Can't easily make larger alignment work with kmalloc. WARN
- * on it. Larger alignment should only be used for module
- * percpu sections on SMP for which this path isn't used.
- */
- WARN_ON_ONCE(align > SMP_CACHE_BYTES);
- return kzalloc(size, GFP_KERNEL);
-}
-
-static inline void free_percpu(void __percpu *p)
-{
- kfree(p);
-}
-
-static inline phys_addr_t per_cpu_ptr_to_phys(void *addr)
-{
- return __pa(addr);
+ return false;
}
static inline void __init setup_per_cpu_areas(void) { }
#endif /* CONFIG_SMP */
+extern void __percpu *__alloc_percpu(size_t size, size_t align);
+extern void free_percpu(void __percpu *__pdata);
+extern phys_addr_t per_cpu_ptr_to_phys(void *addr);
+
#define alloc_percpu(type) \
(typeof(type) __percpu *)__alloc_percpu(sizeof(type), __alignof__(type))
extern void __perf_sw_event(u32, u64, int, struct pt_regs *, u64);
-static inline void
-perf_sw_event(u32 event_id, u64 nr, int nmi, struct pt_regs *regs, u64 addr)
-{
- if (atomic_read(&perf_swevent_enabled[event_id]))
- __perf_sw_event(event_id, nr, nmi, regs, addr);
-}
-
extern void
perf_arch_fetch_caller_regs(struct pt_regs *regs, unsigned long ip, int skip);
return perf_arch_fetch_caller_regs(regs, ip, skip);
}
+static inline void
+perf_sw_event(u32 event_id, u64 nr, int nmi, struct pt_regs *regs, u64 addr)
+{
+ if (atomic_read(&perf_swevent_enabled[event_id])) {
+ struct pt_regs hot_regs;
+
+ if (!regs) {
+ perf_fetch_caller_regs(&hot_regs, 1);
+ regs = &hot_regs;
+ }
+ __perf_sw_event(event_id, nr, nmi, regs, addr);
+ }
+}
+
extern void __perf_event_mmap(struct vm_area_struct *vma);
static inline void perf_event_mmap(struct vm_area_struct *vma)
return lock_is_held(&rcu_lock_map);
}
-/**
- * rcu_read_lock_bh_held - might we be in RCU-bh read-side critical section?
- *
- * If CONFIG_PROVE_LOCKING is selected and enabled, returns nonzero iff in
- * an RCU-bh read-side critical section. In absence of CONFIG_PROVE_LOCKING,
- * this assumes we are in an RCU-bh read-side critical section unless it can
- * prove otherwise.
- *
- * Check rcu_scheduler_active to prevent false positives during boot.
+/*
+ * rcu_read_lock_bh_held() is defined out of line to avoid #include-file
+ * hell.
*/
-static inline int rcu_read_lock_bh_held(void)
-{
- if (!debug_lockdep_rcu_enabled())
- return 1;
- return lock_is_held(&rcu_bh_lock_map);
-}
+extern int rcu_read_lock_bh_held(void);
/**
* rcu_read_lock_sched_held - might we be in RCU-sched read-side critical section?
return 1;
if (debug_locks)
lockdep_opinion = lock_is_held(&rcu_sched_lock_map);
- return lockdep_opinion || preempt_count() != 0;
+ return lockdep_opinion || preempt_count() != 0 || irqs_disabled();
}
#else /* #ifdef CONFIG_PREEMPT */
static inline int rcu_read_lock_sched_held(void)
#ifdef CONFIG_PREEMPT
static inline int rcu_read_lock_sched_held(void)
{
- return !rcu_scheduler_active || preempt_count() != 0;
+ return !rcu_scheduler_active || preempt_count() != 0 || irqs_disabled();
}
#else /* #ifdef CONFIG_PREEMPT */
static inline int rcu_read_lock_sched_held(void)
void reiserfs_security_free(struct reiserfs_security_handle *sec);
#endif
+static inline int reiserfs_xattrs_initialized(struct super_block *sb)
+{
+ return REISERFS_SB(sb)->priv_root != NULL;
+}
+
#define xattr_size(size) ((size) + sizeof(struct reiserfs_xattr_header))
static inline loff_t reiserfs_xattr_nblocks(struct inode *inode, loff_t size)
{
#include <linux/sched.h>
#include <linux/key.h>
#include <linux/xfrm.h>
-#include <linux/gfp.h>
+#include <linux/slab.h>
#include <net/flow.h>
/* Maximum number of letters for an LSM name string */
#define __LINUX_SERIAL_SCI_H
#include <linux/serial_core.h>
+#ifdef CONFIG_SERIAL_SH_SCI_DMA
#include <asm/dmaengine.h>
+#endif
/*
* Generic header for SuperH SCI(F) (used by sh/sh64/h8300 and related parts)
upf_t flags; /* UPF_* flags */
char *clk; /* clock string */
struct device *dma_dev;
+#ifdef CONFIG_SERIAL_SH_SCI_DMA
enum sh_dmae_slave_chan_id dma_slave_tx;
enum sh_dmae_slave_chan_id dma_slave_rx;
+#endif
};
#endif /* __LINUX_SERIAL_SCI_H */
atomic_t dataref;
unsigned short nr_frags;
unsigned short gso_size;
-#ifdef CONFIG_HAS_DMA
- dma_addr_t dma_head;
-#endif
/* Warning: this field is not always filled in (UFO)! */
unsigned short gso_segs;
unsigned short gso_type;
struct sk_buff *frag_list;
struct skb_shared_hwtstamps hwtstamps;
skb_frag_t frags[MAX_SKB_FRAGS];
-#ifdef CONFIG_HAS_DMA
- dma_addr_t dma_maps[MAX_SKB_FRAGS];
-#endif
/* Intermediate layers must ensure that destructor_arg
* remains valid until skb destructor */
void * destructor_arg;
#define MSG_ERRQUEUE 0x2000 /* Fetch message from error queue */
#define MSG_NOSIGNAL 0x4000 /* Do not generate SIGPIPE */
#define MSG_MORE 0x8000 /* Sender will send more */
+#define MSG_WAITFORONE 0x10000 /* recvmmsg(): block until 1+ packets avail */
#define MSG_EOF MSG_FIN
#include <linux/device.h>
#include <linux/mod_devicetable.h>
+#include <linux/slab.h>
/*
* INTERFACES between SPI master-side drivers and SPI infrastructure.
void xprt_free_bc_request(struct rpc_rqst *req);
int xprt_setup_backchannel(struct rpc_xprt *, unsigned int min_reqs);
void xprt_destroy_backchannel(struct rpc_xprt *, int max_reqs);
-void bc_release_request(struct rpc_task *);
int bc_send(struct rpc_rqst *req);
/*
{
return 0;
}
+
+static inline void xprt_free_bc_request(struct rpc_rqst *req)
+{
+}
#endif /* CONFIG_NFS_V4_1 */
#endif /* _LINUX_SUNRPC_BC_XPRT_H */
asmlinkage long sys_shmget(key_t key, size_t size, int flag);
asmlinkage long sys_shmdt(char __user *shmaddr);
asmlinkage long sys_shmctl(int shmid, int cmd, struct shmid_ds __user *buf);
-asmlinkage long sys_ipc(unsigned int call, int first, int second,
+asmlinkage long sys_ipc(unsigned int call, int first, unsigned long second,
unsigned long third, void __user *ptr, long fifth);
asmlinkage long sys_mq_open(const char __user *name, int oflag, mode_t mode, struct mq_attr __user *attr);
#include <linux/taskstats.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#ifdef CONFIG_TASKSTATS
extern struct kmem_cache *taskstats_cache;
void **it_func; \
\
rcu_read_lock_sched_notrace(); \
- it_func = rcu_dereference((tp)->funcs); \
+ it_func = rcu_dereference_sched((tp)->funcs); \
if (it_func) { \
do { \
((void(*)(proto))(*it_func))(args); \
/*
* We default to dicing tty buffer allocations to this many characters
- * in order to avoid multiple page allocations. We assume tty_buffer itself
- * is under 256 bytes. See tty_buffer_find for the allocation logic this
- * must match
+ * in order to avoid multiple page allocations. We know the size of
+ * tty_buffer itself but it must also be taken into account that the
+ * the buffer is 256 byte aligned. See tty_buffer_find for the allocation
+ * logic this must match
*/
-#define TTY_BUFFER_PAGE ((PAGE_SIZE - 256) / 2)
+#define TTY_BUFFER_PAGE (((PAGE_SIZE - sizeof(struct tty_buffer)) / 2) & ~0xFF)
struct tty_bufhead {
wait_queue_head_t close_wait; /* Close waiters */
wait_queue_head_t delta_msr_wait; /* Modem status change */
unsigned long flags; /* TTY flags ASY_*/
+ unsigned char console:1; /* port is a console */
struct mutex mutex; /* Locking */
struct mutex buf_mutex; /* Buffer alloc lock */
unsigned char *xmit_buf; /* Optional buffer */
* @number_of_packets: Lists the number of ISO transfer buffers.
* @interval: Specifies the polling interval for interrupt or isochronous
* transfers. The units are frames (milliseconds) for full and low
- * speed devices, and microframes (1/8 millisecond) for highspeed ones.
+ * speed devices, and microframes (1/8 millisecond) for highspeed
+ * and SuperSpeed devices.
* @error_count: Returns the number of ISO transfers that reported errors.
* @context: For use in completion functions. This normally points to
* request-specific driver context.
*
* Initializes a interrupt urb with the proper information needed to submit
* it to a device.
- * Note that high speed interrupt endpoints use a logarithmic encoding of
- * the endpoint interval, and express polling intervals in microframes
- * (eight per millisecond) rather than in frames (one per millisecond).
+ *
+ * Note that High Speed and SuperSpeed interrupt endpoints use a logarithmic
+ * encoding of the endpoint interval, and express polling intervals in
+ * microframes (eight per millisecond) rather than in frames (one per
+ * millisecond).
+ *
+ * Wireless USB also uses the logarithmic encoding, but specifies it in units of
+ * 128us instead of 125us. For Wireless USB devices, the interval is passed
+ * through to the host controller, rather than being translated into microframe
+ * units.
*/
static inline void usb_fill_int_urb(struct urb *urb,
struct usb_device *dev,
urb->transfer_buffer_length = buffer_length;
urb->complete = complete_fn;
urb->context = context;
- if (dev->speed == USB_SPEED_HIGH)
+ if (dev->speed == USB_SPEED_HIGH || dev->speed == USB_SPEED_SUPER)
urb->interval = 1 << (interval - 1);
else
urb->interval = interval;
#ifndef __LINUX_USB_GADGET_H
#define __LINUX_USB_GADGET_H
+#include <linux/slab.h>
+
struct usb_ep;
/**
#define VT_SETMODE 0x5602 /* set mode of active vt */
#define VT_AUTO 0x00 /* auto vt switching */
#define VT_PROCESS 0x01 /* process controls switching */
-#define VT_PROCESS_AUTO 0x02 /* process is notified of switching */
+#define VT_ACKACQ 0x02 /* acknowledge switch */
struct vt_stat {
unsigned short v_active; /* active vt */
#define VT_SENDSIG 0x5604 /* signal to send to bitmask of vts */
#define VT_RELDISP 0x5605 /* release display */
-#define VT_ACKACQ 0x02 /* acknowledge switch */
#define VT_ACTIVATE 0x5606 /* make vt active */
#define VT_WAITACTIVE 0x5607 /* wait for vt active */
#include <linux/types.h>
#include <linux/device.h>
+#include <linux/slab.h>
/* Backend stuff */
enum p9_trans_status {
Connected,
+ BeginDisconnect,
Disconnected,
Hung,
};
struct p9_client *p9_client_create(const char *dev_name, char *options);
void p9_client_destroy(struct p9_client *clnt);
void p9_client_disconnect(struct p9_client *clnt);
+void p9_client_begin_disconnect(struct p9_client *clnt);
struct p9_fid *p9_client_attach(struct p9_client *clnt, struct p9_fid *afid,
char *uname, u32 n_uname, char *aname);
struct p9_fid *p9_client_auth(struct p9_client *clnt, char *uname,
#include <linux/spinlock.h>
#include <linux/timer.h>
#include <linux/list.h>
+#include <linux/slab.h>
#include <asm/atomic.h>
#define AX25_T1CLAMPLO 1
extern int bt_sysfs_init(void);
extern void bt_sysfs_cleanup(void);
-extern struct class *bt_class;
+extern struct dentry *bt_debugfs;
#endif /* __BLUETOOTH_H */
#define __NET_FIB_RULES_H
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/netdevice.h>
#include <linux/fib_rules.h>
#include <net/flow.h>
#include <net/datalink.h>
#include <linux/ipx.h>
#include <linux/list.h>
+#include <linux/slab.h>
struct ipx_address {
__be32 net;
*/
#include <linux/types.h>
+#include <linux/slab.h>
#include <asm/debug.h>
/*
#ifndef _NF_CONNTRACK_EXTEND_H
#define _NF_CONNTRACK_EXTEND_H
+#include <linux/slab.h>
+
#include <net/netfilter/nf_conntrack.h>
enum nf_ct_ext_id {
#define _NETLABEL_H
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/net.h>
#include <linux/skbuff.h>
#include <linux/in.h>
*/
static inline __be64 nla_get_be64(const struct nlattr *nla)
{
- return *(__be64 *) nla_data(nla);
+ __be64 tmp;
+
+ nla_memcpy(&tmp, nla, sizeof(tmp));
+
+ return tmp;
}
/**
#include <linux/netrom.h>
#include <linux/list.h>
+#include <linux/slab.h>
#include <net/sock.h>
#define NR_NETWORK_LEN 15
#include <linux/skbuff.h> /* struct sk_buff */
#include <linux/mm.h>
#include <linux/security.h>
+#include <linux/slab.h>
#include <linux/filter.h>
#include <linux/rculist_nulls.h>
#ifndef _X25_H
#define _X25_H
#include <linux/x25.h>
+#include <linux/slab.h>
#include <net/sock.h>
#define X25_ADDR_LEN 16
#include <linux/in6.h>
#include <linux/mutex.h>
#include <linux/audit.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/dst.h>
#endif
-/* socket drivers are expected to use these callbacks in their .drv struct */
-extern int pcmcia_socket_dev_suspend(struct device *dev);
-extern void pcmcia_socket_dev_early_resume(struct device *dev);
-extern void pcmcia_socket_dev_late_resume(struct device *dev);
-extern int pcmcia_socket_dev_resume(struct device *dev);
-
/* socket drivers use this callback in their IRQ handler */
extern void pcmcia_parse_events(struct pcmcia_socket *socket,
unsigned int events);
#include <scsi/scsi_cmnd.h>
#include <scsi/scsi_transport_sas.h>
#include <linux/scatterlist.h>
+#include <linux/slab.h>
struct block_device;
#include <linux/mutex.h>
#include <linux/completion.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <xen/interface/xen.h>
#include <xen/interface/grant_table.h>
#include <xen/interface/io/xenbus.h>
#include <linux/initrd.h>
#include <linux/async.h>
#include <linux/fs_struct.h>
+#include <linux/slab.h>
#include <linux/nfs_fs.h>
#include <linux/nfs_fs_sb.h>
#include <linux/cramfs_fs.h>
#include <linux/initrd.h>
#include <linux/string.h>
+#include <linux/slab.h>
#include "do_mounts.h"
#include "../fs/squashfs/squashfs_fs.h"
#include <linux/bootmem.h>
#include <linux/acpi.h>
#include <linux/tty.h>
-#include <linux/gfp.h>
#include <linux/percpu.h>
#include <linux/kmod.h>
#include <linux/vmalloc.h>
#include <linux/kmemtrace.h>
#include <linux/sfi.h>
#include <linux/shmem_fs.h>
+#include <linux/slab.h>
#include <trace/boot.h>
#include <asm/io.h>
/*
* init can allocate pages on any node
*/
- set_mems_allowed(node_possible_map);
+ set_mems_allowed(node_states[N_HIGH_MEMORY]);
/*
* init can run on any cpu.
*/
#include <linux/init.h>
#include <linux/msg.h>
#include <linux/shm.h>
-#include <linux/slab.h>
#include <linux/syscalls.h>
#include <linux/mutex.h>
#include <linux/nsproxy.h>
#include <linux/pid.h>
#include <linux/ipc_namespace.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include "util.h"
*/
#include <linux/capability.h>
-#include <linux/slab.h>
#include <linux/msg.h>
#include <linux/spinlock.h>
#include <linux/init.h>
#include <linux/syscalls.h>
#include <linux/uaccess.h>
-SYSCALL_DEFINE6(ipc, unsigned int, call, int, first, int, second,
+SYSCALL_DEFINE6(ipc, unsigned int, call, int, first, unsigned long, second,
unsigned long, third, void __user *, ptr, long, fifth)
{
int version, ret;
#include <linux/init.h>
#include <linux/kthread.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <asm/atomic.h>
static async_cookie_t next_cookie = 1;
#include <asm/atomic.h>
#include <linux/mm.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/err.h>
#include <linux/kthread.h>
#include <linux/namei.h>
#include <linux/mount.h>
#include <linux/kthread.h>
+#include <linux/slab.h>
struct audit_tree;
struct audit_chunk;
#include <linux/namei.h>
#include <linux/netlink.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/inotify.h>
#include <linux/security.h>
#include "audit.h"
#include <linux/namei.h>
#include <linux/netlink.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/security.h>
#include "audit.h"
#include <linux/namei.h>
#include <linux/mm.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/mount.h>
#include <linux/socket.h>
#include <linux/mqueue.h>
{
if (context->name_count >= AUDIT_NAMES) {
if (inode)
- printk(KERN_DEBUG "name_count maxed, losing inode data: "
+ printk(KERN_DEBUG "audit: name_count maxed, losing inode data: "
"dev=%02x:%02x, inode=%lu\n",
MAJOR(inode->i_sb->s_dev),
MINOR(inode->i_sb->s_dev),
*/
#include <linux/cgroup.h>
-#include <linux/module.h>
#include <linux/ctype.h>
#include <linux/errno.h>
#include <linux/fs.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/cgroup.h>
#include <linux/fs.h>
#include <linux/uaccess.h>
struct freezer, css);
}
-int cgroup_frozen(struct task_struct *task)
+int cgroup_freezing_or_frozen(struct task_struct *task)
{
struct freezer *freezer;
enum freezer_state state;
task_lock(task);
freezer = task_freezer(task);
- state = freezer->state;
+ if (!freezer->css.cgroup->parent)
+ state = CGROUP_THAWED; /* root cgroup can't be frozen */
+ else
+ state = freezer->state;
task_unlock(task);
- return state == CGROUP_FROZEN;
+ return (state == CGROUP_FREEZING) || (state == CGROUP_FROZEN);
}
/*
#include <linux/posix-timers.h>
#include <linux/times.h>
#include <linux/ptrace.h>
+#include <linux/gfp.h>
#include <asm/uaccess.h>
#include <linux/kthread.h>
#include <linux/stop_machine.h>
#include <linux/mutex.h>
+#include <linux/gfp.h>
#ifdef CONFIG_SMP
/* Serializes the updates to cpu_online_mask, cpu_present_mask */
* call to guarantee_online_mems(), as we know no one is changing
* our task's cpuset.
*
- * Hold callback_mutex around the two modifications of our tasks
- * mems_allowed to synchronize with cpuset_mems_allowed().
- *
* While the mm_struct we are migrating is typically from some
* other task, the task_struct mems_allowed that we are hacking
* is for our current task, which must allocate new pages for that
struct cpuset *cs;
int migrate;
const nodemask_t *oldmem = scan->data;
- nodemask_t newmems;
+ NODEMASK_ALLOC(nodemask_t, newmems, GFP_KERNEL);
+
+ if (!newmems)
+ return;
cs = cgroup_cs(scan->cg);
- guarantee_online_mems(cs, &newmems);
+ guarantee_online_mems(cs, newmems);
task_lock(p);
- cpuset_change_task_nodemask(p, &newmems);
+ cpuset_change_task_nodemask(p, newmems);
task_unlock(p);
+ NODEMASK_FREE(newmems);
+
mm = get_task_mm(p);
if (!mm)
return;
static int update_nodemask(struct cpuset *cs, struct cpuset *trialcs,
const char *buf)
{
- nodemask_t oldmem;
+ NODEMASK_ALLOC(nodemask_t, oldmem, GFP_KERNEL);
int retval;
struct ptr_heap heap;
+ if (!oldmem)
+ return -ENOMEM;
+
/*
* top_cpuset.mems_allowed tracks node_stats[N_HIGH_MEMORY];
* it's read-only
*/
- if (cs == &top_cpuset)
- return -EACCES;
+ if (cs == &top_cpuset) {
+ retval = -EACCES;
+ goto done;
+ }
/*
* An empty mems_allowed is ok iff there are no tasks in the cpuset.
goto done;
if (!nodes_subset(trialcs->mems_allowed,
- node_states[N_HIGH_MEMORY]))
- return -EINVAL;
+ node_states[N_HIGH_MEMORY])) {
+ retval = -EINVAL;
+ goto done;
+ }
}
- oldmem = cs->mems_allowed;
- if (nodes_equal(oldmem, trialcs->mems_allowed)) {
+ *oldmem = cs->mems_allowed;
+ if (nodes_equal(*oldmem, trialcs->mems_allowed)) {
retval = 0; /* Too easy - nothing to do */
goto done;
}
cs->mems_allowed = trialcs->mems_allowed;
mutex_unlock(&callback_mutex);
- update_tasks_nodemask(cs, &oldmem, &heap);
+ update_tasks_nodemask(cs, oldmem, &heap);
heap_free(&heap);
done:
+ NODEMASK_FREE(oldmem);
return retval;
}
struct cgroup *oldcont, struct task_struct *tsk,
bool threadgroup)
{
- nodemask_t from, to;
struct mm_struct *mm;
struct cpuset *cs = cgroup_cs(cont);
struct cpuset *oldcs = cgroup_cs(oldcont);
+ NODEMASK_ALLOC(nodemask_t, from, GFP_KERNEL);
+ NODEMASK_ALLOC(nodemask_t, to, GFP_KERNEL);
+
+ if (from == NULL || to == NULL)
+ goto alloc_fail;
if (cs == &top_cpuset) {
cpumask_copy(cpus_attach, cpu_possible_mask);
- to = node_possible_map;
} else {
guarantee_online_cpus(cs, cpus_attach);
- guarantee_online_mems(cs, &to);
}
+ guarantee_online_mems(cs, to);
/* do per-task migration stuff possibly for each in the threadgroup */
- cpuset_attach_task(tsk, &to, cs);
+ cpuset_attach_task(tsk, to, cs);
if (threadgroup) {
struct task_struct *c;
rcu_read_lock();
list_for_each_entry_rcu(c, &tsk->thread_group, thread_group) {
- cpuset_attach_task(c, &to, cs);
+ cpuset_attach_task(c, to, cs);
}
rcu_read_unlock();
}
/* change mm; only needs to be done once even if threadgroup */
- from = oldcs->mems_allowed;
- to = cs->mems_allowed;
+ *from = oldcs->mems_allowed;
+ *to = cs->mems_allowed;
mm = get_task_mm(tsk);
if (mm) {
- mpol_rebind_mm(mm, &to);
+ mpol_rebind_mm(mm, to);
if (is_memory_migrate(cs))
- cpuset_migrate_mm(mm, &from, &to);
+ cpuset_migrate_mm(mm, from, to);
mmput(mm);
}
+
+alloc_fail:
+ NODEMASK_FREE(from);
+ NODEMASK_FREE(to);
}
/* The various types of files and directories in a cpuset file system */
static int cpuset_sprintf_memlist(char *page, struct cpuset *cs)
{
- nodemask_t mask;
+ NODEMASK_ALLOC(nodemask_t, mask, GFP_KERNEL);
+ int retval;
+
+ if (mask == NULL)
+ return -ENOMEM;
mutex_lock(&callback_mutex);
- mask = cs->mems_allowed;
+ *mask = cs->mems_allowed;
mutex_unlock(&callback_mutex);
- return nodelist_scnprintf(page, PAGE_SIZE, mask);
+ retval = nodelist_scnprintf(page, PAGE_SIZE, *mask);
+
+ NODEMASK_FREE(mask);
+
+ return retval;
}
static ssize_t cpuset_common_file_read(struct cgroup *cont,
struct cpuset *cp; /* scans cpusets being updated */
struct cpuset *child; /* scans child cpusets of cp */
struct cgroup *cont;
- nodemask_t oldmems;
+ NODEMASK_ALLOC(nodemask_t, oldmems, GFP_KERNEL);
+
+ if (oldmems == NULL)
+ return;
list_add_tail((struct list_head *)&root->stack_list, &queue);
nodes_subset(cp->mems_allowed, node_states[N_HIGH_MEMORY]))
continue;
- oldmems = cp->mems_allowed;
+ *oldmems = cp->mems_allowed;
/* Remove offline cpus and mems from this cpuset. */
mutex_lock(&callback_mutex);
remove_tasks_in_empty_cpuset(cp);
else {
update_tasks_cpumask(cp, NULL);
- update_tasks_nodemask(cp, &oldmems, NULL);
+ update_tasks_nodemask(cp, oldmems, NULL);
}
}
+ NODEMASK_FREE(oldmems);
}
/*
static int cpuset_track_online_nodes(struct notifier_block *self,
unsigned long action, void *arg)
{
+ NODEMASK_ALLOC(nodemask_t, oldmems, GFP_KERNEL);
+
+ if (oldmems == NULL)
+ return NOTIFY_DONE;
+
cgroup_lock();
switch (action) {
case MEM_ONLINE:
- case MEM_OFFLINE:
+ *oldmems = top_cpuset.mems_allowed;
mutex_lock(&callback_mutex);
top_cpuset.mems_allowed = node_states[N_HIGH_MEMORY];
mutex_unlock(&callback_mutex);
- if (action == MEM_OFFLINE)
- scan_for_empty_cpusets(&top_cpuset);
+ update_tasks_nodemask(&top_cpuset, oldmems, NULL);
+ break;
+ case MEM_OFFLINE:
+ /*
+ * needn't update top_cpuset.mems_allowed explicitly because
+ * scan_for_empty_cpusets() will update it.
+ */
+ scan_for_empty_cpusets(&top_cpuset);
break;
default:
break;
}
cgroup_unlock();
+
+ NODEMASK_FREE(oldmems);
return NOTIFY_OK;
}
#endif
*/
#include <linux/module.h>
#include <linux/cred.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#include <linux/key.h>
#include <linux/keyctl.h>
new = kmem_cache_alloc(cred_jar, GFP_ATOMIC);
if (!new)
- return NULL;
+ goto free_tgcred;
kdebug("prepare_usermodehelper_creds() alloc %p", new);
error:
put_cred(new);
+free_tgcred:
+#ifdef CONFIG_KEYS
+ kfree(tgcred);
+#endif
return NULL;
}
struct early_res *r;
int i;
+ if (start == end)
+ return;
+
+ if (WARN_ONCE(start > end, " wrong range [%#llx, %#llx]\n", start, end))
+ return;
+
try_next:
i = find_overlapped_early(start, end);
if (i >= max_early_res)
if (desc->chip->ack)
desc->chip->ack(irq);
}
+ desc->status |= IRQ_MASKED;
+}
+
+static inline void mask_irq(struct irq_desc *desc, int irq)
+{
+ if (desc->chip->mask) {
+ desc->chip->mask(irq);
+ desc->status |= IRQ_MASKED;
+ }
+}
+
+static inline void unmask_irq(struct irq_desc *desc, int irq)
+{
+ if (desc->chip->unmask) {
+ desc->chip->unmask(irq);
+ desc->status &= ~IRQ_MASKED;
+ }
}
/*
raw_spin_lock(&desc->lock);
desc->status &= ~IRQ_INPROGRESS;
- if (unlikely(desc->status & IRQ_ONESHOT))
- desc->status |= IRQ_MASKED;
- else if (!(desc->status & IRQ_DISABLED) && desc->chip->unmask)
- desc->chip->unmask(irq);
+ if (!(desc->status & (IRQ_DISABLED | IRQ_ONESHOT)))
+ unmask_irq(desc, irq);
out_unlock:
raw_spin_unlock(&desc->lock);
}
action = desc->action;
if (unlikely(!action || (desc->status & IRQ_DISABLED))) {
desc->status |= IRQ_PENDING;
- if (desc->chip->mask)
- desc->chip->mask(irq);
+ mask_irq(desc, irq);
goto out;
}
irqreturn_t action_ret;
if (unlikely(!action)) {
- desc->chip->mask(irq);
+ mask_irq(desc, irq);
goto out_unlock;
}
if (unlikely((desc->status &
(IRQ_PENDING | IRQ_MASKED | IRQ_DISABLED)) ==
(IRQ_PENDING | IRQ_MASKED))) {
- desc->chip->unmask(irq);
- desc->status &= ~IRQ_MASKED;
+ unmask_irq(desc, irq);
}
desc->status &= ~IRQ_PENDING;
__set_irq_handler(irq, handle, 0, name);
}
-void __init set_irq_noprobe(unsigned int irq)
+void set_irq_noprobe(unsigned int irq)
{
struct irq_desc *desc = irq_to_desc(irq);
unsigned long flags;
raw_spin_unlock_irqrestore(&desc->lock, flags);
}
-void __init set_irq_probe(unsigned int irq)
+void set_irq_probe(unsigned int irq)
{
struct irq_desc *desc = irq_to_desc(irq);
unsigned long flags;
{
struct irq_desc *desc = irq_to_desc(irq);
struct irqaction *action;
+ unsigned long flags;
if (!desc)
return 0;
if (desc->status & IRQ_NOREQUEST)
return 0;
+ raw_spin_lock_irqsave(&desc->lock, flags);
action = desc->action;
if (action)
if (irqflags & action->flags & IRQF_SHARED)
action = NULL;
+ raw_spin_unlock_irqrestore(&desc->lock, flags);
+
return !action;
}
*/
static void irq_finalize_oneshot(unsigned int irq, struct irq_desc *desc)
{
+again:
chip_bus_lock(irq, desc);
raw_spin_lock_irq(&desc->lock);
+
+ /*
+ * Implausible though it may be we need to protect us against
+ * the following scenario:
+ *
+ * The thread is faster done than the hard interrupt handler
+ * on the other CPU. If we unmask the irq line then the
+ * interrupt can come in again and masks the line, leaves due
+ * to IRQ_INPROGRESS and the irq line is masked forever.
+ */
+ if (unlikely(desc->status & IRQ_INPROGRESS)) {
+ raw_spin_unlock_irq(&desc->lock);
+ chip_bus_sync_unlock(irq, desc);
+ cpu_relax();
+ goto again;
+ }
+
if (!(desc->status & IRQ_DISABLED) && (desc->status & IRQ_MASKED)) {
desc->status &= ~IRQ_MASKED;
desc->chip->unmask(irq);
*/
#include <linux/irq.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/random.h>
#include <linux/interrupt.h>
*/
#include <linux/irq.h>
+#include <linux/gfp.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
#include <linux/interrupt.h>
#include <linux/sched.h> /* for cond_resched */
#include <linux/mm.h>
#include <linux/ctype.h>
+#include <linux/slab.h>
#include <asm/sections.h>
struct pt_regs *linux_regs;
};
+/* Exception state values */
+#define DCPU_WANT_MASTER 0x1 /* Waiting to become a master kgdb cpu */
+#define DCPU_NEXT_MASTER 0x2 /* Transition from one master cpu to another */
+#define DCPU_IS_SLAVE 0x4 /* Slave cpu enter exception */
+#define DCPU_SSTEP 0x8 /* CPU is single stepping */
+
static struct debuggerinfo_struct {
void *debuggerinfo;
struct task_struct *task;
+ int exception_state;
} kgdb_info[NR_CPUS];
/**
/*
* Copy the binary array pointed to by buf into mem. Fix $, #, and
- * 0x7d escaped with 0x7d. Return a pointer to the character after
- * the last byte written.
+ * 0x7d escaped with 0x7d. Return -EFAULT on failure or 0 on success.
+ * The input buf is overwitten with the result to write to mem.
*/
static int kgdb_ebin2mem(char *buf, char *mem, int count)
{
- int err = 0;
- char c;
+ int size = 0;
+ char *c = buf;
while (count-- > 0) {
- c = *buf++;
- if (c == 0x7d)
- c = *buf++ ^ 0x20;
-
- err = probe_kernel_write(mem, &c, 1);
- if (err)
- break;
-
- mem++;
+ c[size] = *buf++;
+ if (c[size] == 0x7d)
+ c[size] = *buf++ ^ 0x20;
+ size++;
}
- return err;
+ return probe_kernel_write(mem, c, size);
}
/*
return find_task_by_pid_ns(tid, &init_pid_ns);
}
-/*
- * CPU debug state control:
- */
-
-#ifdef CONFIG_SMP
-static void kgdb_wait(struct pt_regs *regs)
-{
- unsigned long flags;
- int cpu;
-
- local_irq_save(flags);
- cpu = raw_smp_processor_id();
- kgdb_info[cpu].debuggerinfo = regs;
- kgdb_info[cpu].task = current;
- /*
- * Make sure the above info reaches the primary CPU before
- * our cpu_in_kgdb[] flag setting does:
- */
- smp_wmb();
- atomic_set(&cpu_in_kgdb[cpu], 1);
-
- /* Disable any cpu specific hw breakpoints */
- kgdb_disable_hw_debug(regs);
-
- /* Wait till primary CPU is done with debugging */
- while (atomic_read(&passive_cpu_wait[cpu]))
- cpu_relax();
-
- kgdb_info[cpu].debuggerinfo = NULL;
- kgdb_info[cpu].task = NULL;
-
- /* fix up hardware debug registers on local cpu */
- if (arch_kgdb_ops.correct_hw_break)
- arch_kgdb_ops.correct_hw_break();
-
- /* Signal the primary CPU that we are done: */
- atomic_set(&cpu_in_kgdb[cpu], 0);
- touch_softlockup_watchdog_sync();
- clocksource_touch_watchdog();
- local_irq_restore(flags);
-}
-#endif
-
/*
* Some architectures need cache flushes when we set/clear a
* breakpoint:
return 1;
}
-/*
- * kgdb_handle_exception() - main entry point from a kernel exception
- *
- * Locking hierarchy:
- * interface locks, if any (begin_session)
- * kgdb lock (kgdb_active)
- */
-int
-kgdb_handle_exception(int evector, int signo, int ecode, struct pt_regs *regs)
+static int kgdb_cpu_enter(struct kgdb_state *ks, struct pt_regs *regs)
{
- struct kgdb_state kgdb_var;
- struct kgdb_state *ks = &kgdb_var;
unsigned long flags;
int sstep_tries = 100;
int error = 0;
int i, cpu;
-
- ks->cpu = raw_smp_processor_id();
- ks->ex_vector = evector;
- ks->signo = signo;
- ks->ex_vector = evector;
- ks->err_code = ecode;
- ks->kgdb_usethreadid = 0;
- ks->linux_regs = regs;
-
- if (kgdb_reenter_check(ks))
- return 0; /* Ouch, double exception ! */
-
+ int trace_on = 0;
acquirelock:
/*
* Interrupts will be restored by the 'trap return' code, except when
*/
local_irq_save(flags);
- cpu = raw_smp_processor_id();
+ cpu = ks->cpu;
+ kgdb_info[cpu].debuggerinfo = regs;
+ kgdb_info[cpu].task = current;
+ /*
+ * Make sure the above info reaches the primary CPU before
+ * our cpu_in_kgdb[] flag setting does:
+ */
+ atomic_inc(&cpu_in_kgdb[cpu]);
/*
- * Acquire the kgdb_active lock:
+ * CPU will loop if it is a slave or request to become a kgdb
+ * master cpu and acquire the kgdb_active lock:
*/
- while (atomic_cmpxchg(&kgdb_active, -1, cpu) != -1)
+ while (1) {
+ if (kgdb_info[cpu].exception_state & DCPU_WANT_MASTER) {
+ if (atomic_cmpxchg(&kgdb_active, -1, cpu) == cpu)
+ break;
+ } else if (kgdb_info[cpu].exception_state & DCPU_IS_SLAVE) {
+ if (!atomic_read(&passive_cpu_wait[cpu]))
+ goto return_normal;
+ } else {
+return_normal:
+ /* Return to normal operation by executing any
+ * hw breakpoint fixup.
+ */
+ if (arch_kgdb_ops.correct_hw_break)
+ arch_kgdb_ops.correct_hw_break();
+ if (trace_on)
+ tracing_on();
+ atomic_dec(&cpu_in_kgdb[cpu]);
+ touch_softlockup_watchdog_sync();
+ clocksource_touch_watchdog();
+ local_irq_restore(flags);
+ return 0;
+ }
cpu_relax();
+ }
/*
* For single stepping, try to only enter on the processor
if (kgdb_io_ops->pre_exception)
kgdb_io_ops->pre_exception();
- kgdb_info[ks->cpu].debuggerinfo = ks->linux_regs;
- kgdb_info[ks->cpu].task = current;
-
kgdb_disable_hw_debug(ks->linux_regs);
/*
*/
if (!kgdb_single_step) {
for (i = 0; i < NR_CPUS; i++)
- atomic_set(&passive_cpu_wait[i], 1);
+ atomic_inc(&passive_cpu_wait[i]);
}
- /*
- * spin_lock code is good enough as a barrier so we don't
- * need one here:
- */
- atomic_set(&cpu_in_kgdb[ks->cpu], 1);
-
#ifdef CONFIG_SMP
/* Signal the other CPUs to enter kgdb_wait() */
if ((!kgdb_single_step) && kgdb_do_roundup)
kgdb_single_step = 0;
kgdb_contthread = current;
exception_level = 0;
+ trace_on = tracing_is_on();
+ if (trace_on)
+ tracing_off();
/* Talk to debugger with gdbserial protocol */
error = gdb_serial_stub(ks);
if (kgdb_io_ops->post_exception)
kgdb_io_ops->post_exception();
- kgdb_info[ks->cpu].debuggerinfo = NULL;
- kgdb_info[ks->cpu].task = NULL;
- atomic_set(&cpu_in_kgdb[ks->cpu], 0);
+ atomic_dec(&cpu_in_kgdb[ks->cpu]);
if (!kgdb_single_step) {
for (i = NR_CPUS-1; i >= 0; i--)
- atomic_set(&passive_cpu_wait[i], 0);
+ atomic_dec(&passive_cpu_wait[i]);
/*
* Wait till all the CPUs have quit
* from the debugger.
else
kgdb_sstep_pid = 0;
}
+ if (trace_on)
+ tracing_on();
/* Free kgdb_active */
atomic_set(&kgdb_active, -1);
touch_softlockup_watchdog_sync();
return error;
}
+/*
+ * kgdb_handle_exception() - main entry point from a kernel exception
+ *
+ * Locking hierarchy:
+ * interface locks, if any (begin_session)
+ * kgdb lock (kgdb_active)
+ */
+int
+kgdb_handle_exception(int evector, int signo, int ecode, struct pt_regs *regs)
+{
+ struct kgdb_state kgdb_var;
+ struct kgdb_state *ks = &kgdb_var;
+ int ret;
+
+ ks->cpu = raw_smp_processor_id();
+ ks->ex_vector = evector;
+ ks->signo = signo;
+ ks->ex_vector = evector;
+ ks->err_code = ecode;
+ ks->kgdb_usethreadid = 0;
+ ks->linux_regs = regs;
+
+ if (kgdb_reenter_check(ks))
+ return 0; /* Ouch, double exception ! */
+ kgdb_info[ks->cpu].exception_state |= DCPU_WANT_MASTER;
+ ret = kgdb_cpu_enter(ks, regs);
+ kgdb_info[ks->cpu].exception_state &= ~DCPU_WANT_MASTER;
+ return ret;
+}
+
int kgdb_nmicallback(int cpu, void *regs)
{
#ifdef CONFIG_SMP
+ struct kgdb_state kgdb_var;
+ struct kgdb_state *ks = &kgdb_var;
+
+ memset(ks, 0, sizeof(struct kgdb_state));
+ ks->cpu = cpu;
+ ks->linux_regs = regs;
+
if (!atomic_read(&cpu_in_kgdb[cpu]) &&
- atomic_read(&kgdb_active) != cpu &&
- atomic_read(&cpu_in_kgdb[atomic_read(&kgdb_active)])) {
- kgdb_wait((struct pt_regs *)regs);
+ atomic_read(&kgdb_active) != -1 &&
+ atomic_read(&kgdb_active) != cpu) {
+ kgdb_info[cpu].exception_state |= DCPU_IS_SLAVE;
+ kgdb_cpu_enter(ks, regs);
+ kgdb_info[cpu].exception_state &= ~DCPU_IS_SLAVE;
return 0;
}
#endif
*/
void kgdb_breakpoint(void)
{
- atomic_set(&kgdb_setting_breakpoint, 1);
+ atomic_inc(&kgdb_setting_breakpoint);
wmb(); /* Sync point before breakpoint */
arch_kgdb_breakpoint();
wmb(); /* Sync point after breakpoint */
- atomic_set(&kgdb_setting_breakpoint, 0);
+ atomic_dec(&kgdb_setting_breakpoint);
}
EXPORT_SYMBOL_GPL(kgdb_breakpoint);
set_task_comm(tsk, "kthreadd");
ignore_signals(tsk);
set_cpus_allowed_ptr(tsk, cpu_all_mask);
- set_mems_allowed(node_possible_map);
+ set_mems_allowed(node_states[N_HIGH_MEMORY]);
current->flags |= PF_NOFREEZE | PF_FREEZER_NOSIG;
#include <linux/module.h>
#include <linux/sched.h>
#include <linux/list.h>
-#include <linux/slab.h>
#include <linux/stacktrace.h>
static DEFINE_SPINLOCK(latency_lock);
#include <linux/ftrace.h>
#include <linux/stringify.h>
#include <linux/bitops.h>
+#include <linux/gfp.h>
#include <asm/sections.h>
unsigned long start = (unsigned long) &_stext,
end = (unsigned long) &_end,
addr = (unsigned long) obj;
-#ifdef CONFIG_SMP
- int i;
-#endif
/*
* static variable?
if (arch_is_kernel_data(addr))
return 1;
-#ifdef CONFIG_SMP
/*
- * percpu var?
+ * in-kernel percpu var?
*/
- for_each_possible_cpu(i) {
- start = (unsigned long) &__per_cpu_start + per_cpu_offset(i);
- end = (unsigned long) &__per_cpu_start + PERCPU_ENOUGH_ROOM
- + per_cpu_offset(i);
-
- if ((addr >= start) && (addr < end))
- return 1;
- }
-#endif
+ if (is_kernel_percpu_address(addr))
+ return 1;
/*
- * module var?
+ * module static or percpu var?
*/
- return is_module_address(addr);
+ return is_module_address(addr) || is_module_percpu_address(addr);
}
/*
#ifdef CONFIG_SMP
-static void *percpu_modalloc(unsigned long size, unsigned long align,
- const char *name)
+static inline void __percpu *mod_percpu(struct module *mod)
{
- void *ptr;
+ return mod->percpu;
+}
+static int percpu_modalloc(struct module *mod,
+ unsigned long size, unsigned long align)
+{
if (align > PAGE_SIZE) {
printk(KERN_WARNING "%s: per-cpu alignment %li > %li\n",
- name, align, PAGE_SIZE);
+ mod->name, align, PAGE_SIZE);
align = PAGE_SIZE;
}
- ptr = __alloc_reserved_percpu(size, align);
- if (!ptr)
+ mod->percpu = __alloc_reserved_percpu(size, align);
+ if (!mod->percpu) {
printk(KERN_WARNING
"Could not allocate %lu bytes percpu data\n", size);
- return ptr;
+ return -ENOMEM;
+ }
+ mod->percpu_size = size;
+ return 0;
}
-static void percpu_modfree(void *freeme)
+static void percpu_modfree(struct module *mod)
{
- free_percpu(freeme);
+ free_percpu(mod->percpu);
}
static unsigned int find_pcpusec(Elf_Ehdr *hdr,
return find_sec(hdr, sechdrs, secstrings, ".data.percpu");
}
-static void percpu_modcopy(void *pcpudest, const void *from, unsigned long size)
+static void percpu_modcopy(struct module *mod,
+ const void *from, unsigned long size)
{
int cpu;
for_each_possible_cpu(cpu)
- memcpy(pcpudest + per_cpu_offset(cpu), from, size);
+ memcpy(per_cpu_ptr(mod->percpu, cpu), from, size);
+}
+
+/**
+ * is_module_percpu_address - test whether address is from module static percpu
+ * @addr: address to test
+ *
+ * Test whether @addr belongs to module static percpu area.
+ *
+ * RETURNS:
+ * %true if @addr is from module static percpu area
+ */
+bool is_module_percpu_address(unsigned long addr)
+{
+ struct module *mod;
+ unsigned int cpu;
+
+ preempt_disable();
+
+ list_for_each_entry_rcu(mod, &modules, list) {
+ if (!mod->percpu_size)
+ continue;
+ for_each_possible_cpu(cpu) {
+ void *start = per_cpu_ptr(mod->percpu, cpu);
+
+ if ((void *)addr >= start &&
+ (void *)addr < start + mod->percpu_size) {
+ preempt_enable();
+ return true;
+ }
+ }
+ }
+
+ preempt_enable();
+ return false;
}
#else /* ... !CONFIG_SMP */
-static inline void *percpu_modalloc(unsigned long size, unsigned long align,
- const char *name)
+static inline void __percpu *mod_percpu(struct module *mod)
{
return NULL;
}
-static inline void percpu_modfree(void *pcpuptr)
+static inline int percpu_modalloc(struct module *mod,
+ unsigned long size, unsigned long align)
+{
+ return -ENOMEM;
+}
+static inline void percpu_modfree(struct module *mod)
{
- BUG();
}
static inline unsigned int find_pcpusec(Elf_Ehdr *hdr,
Elf_Shdr *sechdrs,
{
return 0;
}
-static inline void percpu_modcopy(void *pcpudst, const void *src,
- unsigned long size)
+static inline void percpu_modcopy(struct module *mod,
+ const void *from, unsigned long size)
{
/* pcpusec should be 0, and size of that section should be 0. */
BUG_ON(size != 0);
}
+bool is_module_percpu_address(unsigned long addr)
+{
+ return false;
+}
#endif /* CONFIG_SMP */
int cpu;
INIT_LIST_HEAD(&mod->modules_which_use_me);
- for_each_possible_cpu(cpu)
- per_cpu_ptr(mod->refptr, cpu)->count = 0;
+ for_each_possible_cpu(cpu) {
+ per_cpu_ptr(mod->refptr, cpu)->incs = 0;
+ per_cpu_ptr(mod->refptr, cpu)->decs = 0;
+ }
/* Hold reference count during initialization. */
- __this_cpu_write(mod->refptr->count, 1);
+ __this_cpu_write(mod->refptr->incs, 1);
/* Backwards compatibility macros put refcount during init. */
mod->waiter = current;
}
unsigned int module_refcount(struct module *mod)
{
- unsigned int total = 0;
+ unsigned int incs = 0, decs = 0;
int cpu;
for_each_possible_cpu(cpu)
- total += per_cpu_ptr(mod->refptr, cpu)->count;
- return total;
+ decs += per_cpu_ptr(mod->refptr, cpu)->decs;
+ /*
+ * ensure the incs are added up after the decs.
+ * module_put ensures incs are visible before decs with smp_wmb.
+ *
+ * This 2-count scheme avoids the situation where the refcount
+ * for CPU0 is read, then CPU0 increments the module refcount,
+ * then CPU1 drops that refcount, then the refcount for CPU1 is
+ * read. We would record a decrement but not its corresponding
+ * increment so we would see a low count (disaster).
+ *
+ * Rare situation? But module_refcount can be preempted, and we
+ * might be tallying up 4096+ CPUs. So it is not impossible.
+ */
+ smp_rmb();
+ for_each_possible_cpu(cpu)
+ incs += per_cpu_ptr(mod->refptr, cpu)->incs;
+ return incs - decs;
}
EXPORT_SYMBOL(module_refcount);
{
if (module) {
preempt_disable();
- __this_cpu_dec(module->refptr->count);
+ smp_wmb(); /* see comment in module_refcount */
+ __this_cpu_inc(module->refptr->decs);
trace_module_put(module, _RET_IP_,
- __this_cpu_read(module->refptr->count));
+ __this_cpu_read(module->refptr->decs));
/* Maybe they're waiting for us to drop reference? */
if (unlikely(!module_is_live(module)))
wake_up_process(module->waiter);
/* This may be NULL, but that's OK */
module_free(mod, mod->module_init);
kfree(mod->args);
- if (mod->percpu)
- percpu_modfree(mod->percpu);
+ percpu_modfree(mod);
#if defined(CONFIG_MODULE_UNLOAD)
if (mod->refptr)
free_percpu(mod->refptr);
default:
/* Divert to percpu allocation if a percpu var. */
if (sym[i].st_shndx == pcpuindex)
- secbase = (unsigned long)mod->percpu;
+ secbase = (unsigned long)mod_percpu(mod);
else
secbase = sechdrs[sym[i].st_shndx].sh_addr;
sym[i].st_value += secbase;
unsigned int modindex, versindex, infoindex, pcpuindex;
struct module *mod;
long err = 0;
- void *percpu = NULL, *ptr = NULL; /* Stops spurious gcc warning */
+ void *ptr = NULL; /* Stops spurious gcc warning */
unsigned long symoffs, stroffs, *strmap;
mm_segment_t old_fs;
if (pcpuindex) {
/* We have a special allocation for this section. */
- percpu = percpu_modalloc(sechdrs[pcpuindex].sh_size,
- sechdrs[pcpuindex].sh_addralign,
- mod->name);
- if (!percpu) {
- err = -ENOMEM;
+ err = percpu_modalloc(mod, sechdrs[pcpuindex].sh_size,
+ sechdrs[pcpuindex].sh_addralign);
+ if (err)
goto free_mod;
- }
sechdrs[pcpuindex].sh_flags &= ~(unsigned long)SHF_ALLOC;
- mod->percpu = percpu;
}
/* Determine total sizes, and put offsets in sh_entsize. For now
sort_extable(mod->extable, mod->extable + mod->num_exentries);
/* Finally, copy percpu area over. */
- percpu_modcopy(mod->percpu, (void *)sechdrs[pcpuindex].sh_addr,
+ percpu_modcopy(mod, (void *)sechdrs[pcpuindex].sh_addr,
sechdrs[pcpuindex].sh_size);
add_kallsyms(mod, sechdrs, hdr->e_shnum, symindex, strindex,
module_free(mod, mod->module_core);
/* mod will be freed with core. Don't access it beyond this line! */
free_percpu:
- if (percpu)
- percpu_modfree(percpu);
+ percpu_modfree(mod);
free_mod:
kfree(args);
kfree(strmap);
* Pavel Emelianov <xemul@openvz.org>
*/
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/nsproxy.h>
#include <linux/init_task.h>
#include <linux/padata.h>
#include <linux/mutex.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/rcupdate.h>
#define MAX_SEQ_NR INT_MAX - NR_CPUS
#include <linux/smp.h>
#include <linux/file.h>
#include <linux/poll.h>
+#include <linux/slab.h>
#include <linux/sysfs.h>
#include <linux/dcache.h>
#include <linux/percpu.h>
struct perf_event_context *ctx = task->perf_event_ctxp;
struct perf_event_context *next_ctx;
struct perf_event_context *parent;
- struct pt_regs *regs;
int do_switch = 1;
- regs = task_pt_regs(task);
- perf_sw_event(PERF_COUNT_SW_CONTEXT_SWITCHES, 1, 1, regs, 0);
+ perf_sw_event(PERF_COUNT_SW_CONTEXT_SWITCHES, 1, 1, NULL, 0);
if (likely(!ctx || !cpuctx->task_ctx))
return;
return NULL;
}
-#ifdef CONFIG_EVENT_TRACING
__weak
void perf_arch_fetch_caller_regs(struct pt_regs *regs, unsigned long ip, int skip)
{
}
-#endif
+
/*
* Output
struct perf_task_event *task_event)
{
struct perf_output_handle handle;
- int size;
struct task_struct *task = task_event->task;
- int ret;
+ unsigned long flags;
+ int size, ret;
+
+ /*
+ * If this CPU attempts to acquire an rq lock held by a CPU spinning
+ * in perf_output_lock() from interrupt context, it's game over.
+ */
+ local_irq_save(flags);
size = task_event->event_id.header.size;
ret = perf_output_begin(&handle, event, size, 0, 0);
- if (ret)
+ if (ret) {
+ local_irq_restore(flags);
return;
+ }
task_event->event_id.pid = perf_event_pid(event, task);
task_event->event_id.ppid = perf_event_pid(event, current);
perf_output_put(&handle, task_event->event_id);
perf_output_end(&handle);
+ local_irq_restore(flags);
}
static int perf_event_task_match(struct perf_event *event)
#include <linux/syscalls.h>
#include <linux/err.h>
#include <linux/acct.h>
+#include <linux/slab.h>
#define BITS_PER_PAGE (PAGE_SIZE*8)
}
}
-static void stop_process_timers(struct task_struct *tsk)
+static void stop_process_timers(struct signal_struct *sig)
{
- struct thread_group_cputimer *cputimer = &tsk->signal->cputimer;
+ struct thread_group_cputimer *cputimer = &sig->cputimer;
unsigned long flags;
if (!cputimer->running)
spin_lock_irqsave(&cputimer->lock, flags);
cputimer->running = 0;
spin_unlock_irqrestore(&cputimer->lock, flags);
+
+ sig->cputime_expires.prof_exp = cputime_zero;
+ sig->cputime_expires.virt_exp = cputime_zero;
+ sig->cputime_expires.sched_exp = 0;
}
static u32 onecputick;
list_empty(&timers[CPUCLOCK_VIRT]) &&
cputime_eq(sig->it[CPUCLOCK_VIRT].expires, cputime_zero) &&
list_empty(&timers[CPUCLOCK_SCHED])) {
- stop_process_timers(tsk);
+ stop_process_timers(sig);
return;
}
#include <linux/console.h>
#include <linux/cpu.h>
#include <linux/freezer.h>
+#include <linux/gfp.h>
#include <scsi/scsi_scan.h>
#include <asm/suspend.h>
#include <linux/kernel.h>
#include <linux/list.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include <linux/suspend.h>
/*
printk(KERN_ERR "Freezing of tasks failed after %d.%02d seconds "
"(%d tasks refusing to freeze):\n",
elapsed_csecs / 100, elapsed_csecs % 100, todo);
- show_state();
read_lock(&tasklist_lock);
do_each_thread(g, p) {
task_lock(p);
if (freezing(p) && !freezer_should_skip(p))
- printk(KERN_ERR " %s\n", p->comm);
+ sched_show_task(p);
cancel_freezing(p);
task_unlock(p);
} while_each_thread(g, p);
if (nosig_only && should_send_signal(p))
continue;
- if (cgroup_frozen(p))
+ if (cgroup_freezing_or_frozen(p))
continue;
thaw_process(p);
#include <linux/console.h>
#include <linux/highmem.h>
#include <linux/list.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/mmu_context.h>
#include <linux/console.h>
#include <linux/cpu.h>
#include <linux/syscalls.h>
+#include <linux/gfp.h>
#include "power.h"
#include <linux/swap.h>
#include <linux/swapops.h>
#include <linux/pm.h>
+#include <linux/slab.h>
#include "power.h"
#include <linux/mutex.h>
#include <linux/module.h>
#include <linux/kernel_stat.h>
+#include <linux/hardirq.h>
#ifdef CONFIG_DEBUG_LOCK_ALLOC
static struct lock_class_key rcu_lock_key;
int rcu_scheduler_active __read_mostly;
EXPORT_SYMBOL_GPL(rcu_scheduler_active);
+#ifdef CONFIG_DEBUG_LOCK_ALLOC
+
+/**
+ * rcu_read_lock_bh_held - might we be in RCU-bh read-side critical section?
+ *
+ * Check for bottom half being disabled, which covers both the
+ * CONFIG_PROVE_RCU and not cases. Note that if someone uses
+ * rcu_read_lock_bh(), but then later enables BH, lockdep (if enabled)
+ * will show the situation.
+ *
+ * Check debug_lockdep_rcu_enabled() to prevent false positives during boot.
+ */
+int rcu_read_lock_bh_held(void)
+{
+ if (!debug_lockdep_rcu_enabled())
+ return 1;
+ return in_softirq();
+}
+EXPORT_SYMBOL_GPL(rcu_read_lock_bh_held);
+
+#endif /* #ifdef CONFIG_DEBUG_LOCK_ALLOC */
+
/*
* This function is invoked towards the end of the scheduler's initialization
* process. Before this is called, the idle task might contain
#include <linux/types.h>
#include <linux/parser.h>
#include <linux/fs.h>
-#include <linux/slab.h>
#include <linux/res_counter.h>
#include <linux/uaccess.h>
#include <linux/mm.h>
}
/**
- * request_resource - request and reserve an I/O or memory resource
+ * request_resource_conflict - request and reserve an I/O or memory resource
* @root: root resource descriptor
* @new: resource descriptor desired by caller
*
- * Returns 0 for success, negative error code on error.
+ * Returns 0 for success, conflict resource on error.
*/
-int request_resource(struct resource *root, struct resource *new)
+struct resource *request_resource_conflict(struct resource *root, struct resource *new)
{
struct resource *conflict;
write_lock(&resource_lock);
conflict = __request_resource(root, new);
write_unlock(&resource_lock);
+ return conflict;
+}
+
+/**
+ * request_resource - request and reserve an I/O or memory resource
+ * @root: root resource descriptor
+ * @new: resource descriptor desired by caller
+ *
+ * Returns 0 for success, negative error code on error.
+ */
+int request_resource(struct resource *root, struct resource *new)
+{
+ struct resource *conflict;
+
+ conflict = request_resource_conflict(root, new);
return conflict ? -EBUSY : 0;
}
}
/**
- * insert_resource - Inserts a resource in the resource tree
+ * insert_resource_conflict - Inserts resource in the resource tree
* @parent: parent of the new resource
* @new: new resource to insert
*
- * Returns 0 on success, -EBUSY if the resource can't be inserted.
+ * Returns 0 on success, conflict resource if the resource can't be inserted.
*
- * This function is equivalent to request_resource when no conflict
+ * This function is equivalent to request_resource_conflict when no conflict
* happens. If a conflict happens, and the conflicting resources
* entirely fit within the range of the new resource, then the new
* resource is inserted and the conflicting resources become children of
* the new resource.
*/
-int insert_resource(struct resource *parent, struct resource *new)
+struct resource *insert_resource_conflict(struct resource *parent, struct resource *new)
{
struct resource *conflict;
write_lock(&resource_lock);
conflict = __insert_resource(parent, new);
write_unlock(&resource_lock);
+ return conflict;
+}
+
+/**
+ * insert_resource - Inserts a resource in the resource tree
+ * @parent: parent of the new resource
+ * @new: new resource to insert
+ *
+ * Returns 0 on success, -EBUSY if the resource can't be inserted.
+ */
+int insert_resource(struct resource *parent, struct resource *new)
+{
+ struct resource *conflict;
+
+ conflict = insert_resource_conflict(parent, new);
return conflict ? -EBUSY : 0;
}
#include <linux/debugfs.h>
#include <linux/ctype.h>
#include <linux/ftrace.h>
+#include <linux/slab.h>
#include <asm/tlb.h>
#include <asm/irq_regs.h>
{
unsigned long flags;
struct rq *rq;
- int cpu = get_cpu();
+ int cpu __maybe_unused = get_cpu();
#ifdef CONFIG_SMP
/*
int ret;
cpumask_var_t mask;
- if (len < cpumask_size())
+ if (len < nr_cpu_ids)
+ return -EINVAL;
+ if (len & (sizeof(unsigned long)-1))
return -EINVAL;
if (!alloc_cpumask_var(&mask, GFP_KERNEL))
ret = sched_getaffinity(pid, mask);
if (ret == 0) {
- if (copy_to_user(user_mask_ptr, mask, cpumask_size()))
+ size_t retlen = min_t(size_t, len, cpumask_size());
+
+ if (copy_to_user(user_mask_ptr, mask, retlen))
ret = -EFAULT;
else
- ret = cpumask_size();
+ ret = retlen;
}
free_cpumask_var(mask);
get_task_struct(mt);
task_rq_unlock(rq, &flags);
- wake_up_process(rq->migration_thread);
+ wake_up_process(mt);
put_task_struct(mt);
wait_for_completion(&req.done);
tlb_migrate_finish(p->mm);
* of the License.
*/
+#include <linux/gfp.h>
#include "sched_cpupri.h"
/* Convert between a 140 based task->prio, and our 102 based cpupri */
p->se.nr_wakeups_idle = 0;
p->sched_info.bkl_count = 0;
#endif
- p->se.sum_exec_runtime = 0;
- p->se.prev_sum_exec_runtime = 0;
- p->nvcsw = 0;
- p->nivcsw = 0;
}
goto cancelled;
/* the timer holds a reference whilst it is pending */
- ret = work->ops->get_ref(work);
+ ret = slow_work_get_ref(work);
if (ret < 0)
goto cant_get_ref;
*/
static inline void slow_work_set_thread_pid(int id, pid_t pid)
{
-#ifdef CONFIG_SLOW_WORK_PROC
+#ifdef CONFIG_SLOW_WORK_DEBUG
slow_work_pids[id] = pid;
#endif
}
static inline void slow_work_mark_time(struct slow_work *work)
{
-#ifdef CONFIG_SLOW_WORK_PROC
+#ifdef CONFIG_SLOW_WORK_DEBUG
work->mark = CURRENT_TIME;
#endif
}
static inline void slow_work_begin_exec(int id, struct slow_work *work)
{
-#ifdef CONFIG_SLOW_WORK_PROC
+#ifdef CONFIG_SLOW_WORK_DEBUG
slow_work_execs[id] = work;
#endif
}
static inline void slow_work_end_exec(int id, struct slow_work *work)
{
-#ifdef CONFIG_SLOW_WORK_PROC
+#ifdef CONFIG_SLOW_WORK_DEBUG
write_lock(&slow_work_execs_lock);
slow_work_execs[id] = NULL;
write_unlock(&slow_work_execs_lock);
#include <linux/module.h>
#include <linux/percpu.h>
#include <linux/init.h>
+#include <linux/gfp.h>
#include <linux/smp.h>
#include <linux/cpu.h>
* Wake up the high-prio watchdog task twice per
* threshold timespan.
*/
- if (now > touch_ts + softlockup_thresh/2)
+ if (time_after(now - softlockup_thresh/2, touch_ts))
wake_up_process(per_cpu(softlockup_watchdog, this_cpu));
/* Warn about unreasonable delays: */
- if (now <= (touch_ts + softlockup_thresh))
+ if (time_before_eq(now - softlockup_thresh, touch_ts))
return;
per_cpu(softlockup_print_ts, this_cpu) = touch_ts;
#include <linux/preempt.h>
#include <linux/rcupdate.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/smp.h>
#include <linux/srcu.h>
#include <linux/personality.h>
#include <linux/ptrace.h>
#include <linux/fs_struct.h>
+#include <linux/gfp.h>
#include <linux/compat.h>
#include <linux/syscalls.h>
#include <linux/file.h>
#include <linux/ctype.h>
#include <linux/netdevice.h>
+#include <linux/slab.h>
#ifdef CONFIG_SYSCTL_SYSCALL
#include <linux/delayacct.h>
#include <linux/cpumask.h>
#include <linux/percpu.h>
+#include <linux/slab.h>
#include <linux/cgroupstats.h>
#include <linux/cgroup.h>
#include <linux/fs.h>
#include <linux/syscalls.h>
#include <linux/security.h>
#include <linux/fs.h>
-#include <linux/slab.h>
#include <linux/math64.h>
#include <linux/ptrace.h>
#include "tick-internal.h"
+/* Limit min_delta to a jiffie */
+#define MIN_DELTA_LIMIT (NSEC_PER_SEC / HZ)
+
+static int tick_increase_min_delta(struct clock_event_device *dev)
+{
+ /* Nothing to do if we already reached the limit */
+ if (dev->min_delta_ns >= MIN_DELTA_LIMIT)
+ return -ETIME;
+
+ if (dev->min_delta_ns < 5000)
+ dev->min_delta_ns = 5000;
+ else
+ dev->min_delta_ns += dev->min_delta_ns >> 1;
+
+ if (dev->min_delta_ns > MIN_DELTA_LIMIT)
+ dev->min_delta_ns = MIN_DELTA_LIMIT;
+
+ printk(KERN_WARNING "CE: %s increased min_delta_ns to %llu nsec\n",
+ dev->name ? dev->name : "?",
+ (unsigned long long) dev->min_delta_ns);
+ return 0;
+}
+
/**
* tick_program_event internal worker function
*/
if (!ret || !force)
return ret;
+ dev->retries++;
/*
- * We tried 2 times to program the device with the given
- * min_delta_ns. If that's not working then we double it
+ * We tried 3 times to program the device with the given
+ * min_delta_ns. If that's not working then we increase it
* and emit a warning.
*/
if (++i > 2) {
/* Increase the min. delta and try again */
- if (!dev->min_delta_ns)
- dev->min_delta_ns = 5000;
- else
- dev->min_delta_ns += dev->min_delta_ns >> 1;
-
- printk(KERN_WARNING
- "CE: %s increasing min_delta_ns to %llu nsec\n",
- dev->name ? dev->name : "?",
- (unsigned long long) dev->min_delta_ns << 1);
-
+ if (tick_increase_min_delta(dev)) {
+ /*
+ * Get out of the loop if min_delta_ns
+ * hit the limit already. That's
+ * better than staying here forever.
+ *
+ * We clear next_event so we have a
+ * chance that the box survives.
+ */
+ printk(KERN_WARNING
+ "CE: Reprogramming failure. Giving up\n");
+ dev->next_event.tv64 = KTIME_MAX;
+ return -ETIME;
+ }
i = 0;
}
#include <linux/timecompare.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/math64.h>
/*
shift = min(shift, maxshift);
while (offset >= timekeeper.cycle_interval) {
offset = logarithmic_accumulation(offset, shift);
- shift--;
+ if(offset < timekeeper.cycle_interval<<shift)
+ shift--;
}
/* correct the clock when NTP error is too big */
SEQ_printf(m, " event_handler: ");
print_name_offset(m, dev->event_handler);
SEQ_printf(m, "\n");
+ SEQ_printf(m, " retries: %lu\n", dev->retries);
}
static void timer_list_show_tickdevices(struct seq_file *m)
u64 now = ktime_to_ns(ktime_get());
int cpu;
- SEQ_printf(m, "Timer List Version: v0.5\n");
+ SEQ_printf(m, "Timer List Version: v0.6\n");
SEQ_printf(m, "HRTIMER_MAX_CLOCK_BASES: %d\n", HRTIMER_MAX_CLOCK_BASES);
SEQ_printf(m, "now at %Ld nsecs\n", (unsigned long long)now);
#include <linux/kallsyms.h>
#include <linux/perf_event.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/unistd.h>
if (base->running_timer == timer)
goto out;
+ timer_stats_timer_clear_start_info(timer);
ret = 0;
if (timer_pending(timer)) {
detach_timer(timer, 1);
#include <linux/percpu.h>
#include <linux/init.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <linux/debugfs.h>
#include <linux/smp_lock.h>
#include <linux/time.h>
#include <linux/uaccess.h>
#include <linux/ftrace.h>
#include <linux/sysctl.h>
+#include <linux/slab.h>
#include <linux/ctype.h>
#include <linux/list.h>
#include <linux/hash.h>
#include <linux/workqueue.h>
#include <linux/sched.h>
#include <linux/module.h>
-#include <linux/slab.h>
#define CREATE_TRACE_POINTS
#include <trace/events/power.h>
#include <linux/module.h>
#include <linux/percpu.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/hash.h>
#include <linux/list.h>
#define RB_MAX_SMALL_DATA (RB_ALIGNMENT * RINGBUF_TYPE_DATA_TYPE_LEN_MAX)
#define RB_EVNT_MIN_SIZE 8U /* two 32bit words */
+#if !defined(CONFIG_64BIT) || defined(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS)
+# define RB_FORCE_8BYTE_ALIGNMENT 0
+# define RB_ARCH_ALIGNMENT RB_ALIGNMENT
+#else
+# define RB_FORCE_8BYTE_ALIGNMENT 1
+# define RB_ARCH_ALIGNMENT 8U
+#endif
+
/* define RINGBUF_TYPE_DATA for 'case RINGBUF_TYPE_DATA:' */
#define RINGBUF_TYPE_DATA 0 ... RINGBUF_TYPE_DATA_TYPE_LEN_MAX
for (i = 0; i < nr_pages; i++) {
if (RB_WARN_ON(cpu_buffer, list_empty(cpu_buffer->pages)))
- return;
+ goto out;
p = cpu_buffer->pages->next;
bpage = list_entry(p, struct buffer_page, list);
list_del_init(&bpage->list);
free_buffer_page(bpage);
}
if (RB_WARN_ON(cpu_buffer, list_empty(cpu_buffer->pages)))
- return;
+ goto out;
rb_reset_cpu(cpu_buffer);
rb_check_pages(cpu_buffer);
+out:
spin_unlock_irq(&cpu_buffer->reader_lock);
}
for (i = 0; i < nr_pages; i++) {
if (RB_WARN_ON(cpu_buffer, list_empty(pages)))
- return;
+ goto out;
p = pages->next;
bpage = list_entry(p, struct buffer_page, list);
list_del_init(&bpage->list);
rb_reset_cpu(cpu_buffer);
rb_check_pages(cpu_buffer);
+out:
spin_unlock_irq(&cpu_buffer->reader_lock);
}
case 0:
length -= RB_EVNT_HDR_SIZE;
- if (length > RB_MAX_SMALL_DATA)
+ if (length > RB_MAX_SMALL_DATA || RB_FORCE_8BYTE_ALIGNMENT)
event->array[0] = length;
else
event->type_len = DIV_ROUND_UP(length, RB_ALIGNMENT);
if (!length)
length = 1;
- if (length > RB_MAX_SMALL_DATA)
+ if (length > RB_MAX_SMALL_DATA || RB_FORCE_8BYTE_ALIGNMENT)
length += sizeof(event.array[0]);
length += RB_EVNT_HDR_SIZE;
- length = ALIGN(length, RB_ALIGNMENT);
+ length = ALIGN(length, RB_ARCH_ALIGNMENT);
return length;
}
#include <linux/kdebug.h>
#include <linux/string.h>
#include <linux/rwsem.h>
+#include <linux/slab.h>
#include <linux/ctype.h>
#include <linux/init.h>
#include <linux/poll.h>
-#include <linux/gfp.h>
#include <linux/fs.h>
#include "trace.h"
int this_cpu;
u64 now;
- raw_local_irq_save(flags);
+ local_irq_save(flags);
this_cpu = raw_smp_processor_id();
now = cpu_clock(this_cpu);
arch_spin_unlock(&trace_clock_struct.lock);
out:
- raw_local_irq_restore(flags);
+ local_irq_restore(flags);
return now;
}
static char *perf_trace_buf;
static char *perf_trace_buf_nmi;
-typedef typeof(char [PERF_MAX_TRACE_SIZE]) perf_trace_t ;
+/*
+ * Force it to be aligned to unsigned long to avoid misaligned accesses
+ * suprises
+ */
+typedef typeof(unsigned long [PERF_MAX_TRACE_SIZE / sizeof(unsigned long)])
+ perf_trace_t;
/* Count the events in use (per event id, not per instance) */
static int total_ref_count;
char *trace_buf, *raw_data;
int pc, cpu;
+ BUILD_BUG_ON(PERF_MAX_TRACE_SIZE % sizeof(unsigned long));
+
pc = preempt_count();
/* Protect the per cpu buffer, begin the rcu read side */
raw_data = per_cpu_ptr(trace_buf, cpu);
/* zero the dead bytes from align to not leak stack to user */
- *(u64 *)(&raw_data[size - sizeof(u64)]) = 0ULL;
+ memset(&raw_data[size - sizeof(u64)], 0, sizeof(u64));
entry = (struct trace_entry *)raw_data;
tracing_generic_entry_update(entry, *irq_flags, pc);
#include <linux/uaccess.h>
#include <linux/module.h>
#include <linux/ctype.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <asm/setup.h>
#include <linux/ctype.h>
#include <linux/mutex.h>
#include <linux/perf_event.h>
+#include <linux/slab.h>
#include "trace.h"
#include "trace_output.h"
#include <linux/debugfs.h>
#include <linux/uaccess.h>
#include <linux/ftrace.h>
+#include <linux/slab.h>
#include <linux/fs.h>
#include "trace.h"
#include <linux/debugfs.h>
#include <linux/ftrace.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/fs.h>
#include "trace_output.h"
#include <linux/kernel.h>
#include <linux/mmiotrace.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/time.h>
#include <asm/atomic.h>
#include <linux/stringify.h>
#include <linux/kthread.h>
#include <linux/delay.h>
+#include <linux/slab.h>
static inline int trace_valid_entry(struct trace_entry *entry)
{
#include <linux/list.h>
+#include <linux/slab.h>
#include <linux/rbtree.h>
#include <linux/debugfs.h>
#include "trace_stat.h"
#include <trace/syscall.h>
#include <trace/events/syscalls.h>
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/ftrace.h>
#include <linux/perf_event.h>
#include <trace/events/workqueue.h>
#include <linux/list.h>
#include <linux/percpu.h>
+#include <linux/slab.h>
#include <linux/kref.h>
#include "trace_stat.h"
#include "trace.h"
config DEBUG_SECTION_MISMATCH
bool "Enable full Section mismatch analysis"
- depends on UNDEFINED
+ depends on UNDEFINED || (BLACKFIN)
+ default y
# This option is on purpose disabled for now.
# It will be enabled when we are down to a reasonable number
# of section mismatch warnings (< 10 for an allyesconfig build)
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/bitops.h>
#include <linux/cpumask.h>
#include <linux/module.h>
#include <linux/compiler.h>
#include <linux/types.h>
-#include <linux/slab.h>
#include <linux/init.h>
#include <asm/atomic.h>
#include "crc32defs.h"
#include <linux/sched.h>
#include <linux/seq_file.h>
#include <linux/debugfs.h>
+#include <linux/slab.h>
#include <linux/hash.h>
#define ODEBUG_HASH_BITS 14
#include <linux/pci.h>
#include <linux/io.h>
+#include <linux/gfp.h>
#include <linux/module.h>
void devm_ioremap_release(struct device *dev, void *res)
#include <linux/uaccess.h>
#include <linux/dynamic_debug.h>
#include <linux/debugfs.h>
+#include <linux/slab.h>
extern struct _ddebug __start___verbose[];
extern struct _ddebug __stop___verbose[];
* Version 2. See the file COPYING for more details.
*/
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/bitmap.h>
#include <linux/genalloc.h>
the two sets of lengths.
*/
#include <linux/compiler.h>
+#include <linux/slab.h>
#ifdef RCSID
static char rcsid[] = "#Id: inflate.c,v 0.14 1993/06/10 13:27:04 jloup Exp #";
#include <stdarg.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/string.h>
#include <linux/string.h>
#include <linux/kobject.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/socket.h>
#include <linux/skbuff.h>
#include <linux/kref.h>
#include <linux/module.h>
+#include <linux/slab.h>
/**
* kref_set - initialize object and set refcount to requested number.
#include <linux/slab.h>
#include <linux/notifier.h>
#include <linux/cpu.h>
-#include <linux/gfp.h>
#include <linux/string.h>
#include <linux/bitops.h>
#include <linux/rcupdate.h>
* Version 2. See the file COPYING for more details.
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/scatterlist.h>
#include <linux/highmem.h>
#include <linux/types.h>
#include <linux/ctype.h>
#include <linux/highmem.h>
+#include <linux/gfp.h>
#include <asm/io.h>
#include <asm/dma.h>
#include <linux/rcupdate.h>
#include <linux/err.h>
#include <linux/textsearch.h>
+#include <linux/slab.h>
static LIST_HEAD(ts_ops);
static DEFINE_SPINLOCK(ts_mod_lock);
obj-$(CONFIG_MEMORY_HOTPLUG) += memory_hotplug.o
obj-$(CONFIG_FS_XIP) += filemap_xip.o
obj-$(CONFIG_MIGRATION) += migrate.o
-obj-$(CONFIG_SMP) += percpu.o
+ifdef CONFIG_SMP
+obj-y += percpu.o
+else
+obj-y += percpu_up.o
+endif
obj-$(CONFIG_QUICKLIST) += quicklist.o
obj-$(CONFIG_CGROUP_MEM_RES_CTLR) += memcontrol.o page_cgroup.o
obj-$(CONFIG_MEMORY_FAILURE) += memory-failure.o
*/
#include <linux/init.h>
#include <linux/pfn.h>
+#include <linux/slab.h>
#include <linux/bootmem.h>
#include <linux/module.h>
#include <linux/kmemleak.h>
end_aligned = end & ~(BITS_PER_LONG - 1);
if (end_aligned <= start_aligned) {
-#if 1
- printk(KERN_DEBUG " %lx - %lx\n", start, end);
-#endif
for (i = start; i < end; i++)
__free_pages_bootmem(pfn_to_page(i), 0);
return;
}
-#if 1
- printk(KERN_DEBUG " %lx %lx - %lx %lx\n",
- start, start_aligned, end_aligned, end);
-#endif
for (i = start; i < start_aligned; i++)
__free_pages_bootmem(pfn_to_page(i), 0);
{
#ifdef CONFIG_NO_BOOTMEM
free_early(physaddr, physaddr + size);
-#if 0
- printk(KERN_DEBUG "free %lx %lx\n", physaddr, size);
-#endif
#else
unsigned long start, end;
{
#ifdef CONFIG_NO_BOOTMEM
free_early(addr, addr + size);
-#if 0
- printk(KERN_DEBUG "free %lx %lx\n", addr, size);
-#endif
#else
unsigned long start, end;
#include <linux/mm.h>
#include <linux/module.h>
#include <linux/swap.h>
+#include <linux/gfp.h>
#include <linux/bio.h>
#include <linux/pagemap.h>
#include <linux/mempool.h>
#include <linux/fault-inject.h>
-#include <linux/gfp.h>
#include <linux/slab.h>
static struct {
* the NFS filesystem used to do this differently, for example)
*/
#include <linux/module.h>
-#include <linux/slab.h>
#include <linux/compiler.h>
#include <linux/fs.h>
#include <linux/uaccess.h>
#include <linux/aio.h>
#include <linux/capability.h>
#include <linux/kernel_stat.h>
+#include <linux/gfp.h>
#include <linux/mm.h>
#include <linux/swap.h>
#include <linux/mman.h>
#include <linux/sched.h>
#include <linux/seqlock.h>
#include <linux/mutex.h>
+#include <linux/gfp.h>
#include <asm/tlbflush.h>
#include <asm/io.h>
* Generic hugetlb support.
* (C) William Irwin, April 2004
*/
-#include <linux/gfp.h>
#include <linux/list.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/mutex.h>
#include <linux/bootmem.h>
#include <linux/sysfs.h>
+#include <linux/slab.h>
#include <asm/page.h>
#include <asm/pgtable.h>
#include <linux/module.h>
#include <linux/kthread.h>
#include <linux/prio_tree.h>
-#include <linux/gfp.h>
#include <linux/fs.h>
#include <linux/debugfs.h>
#include <linux/seq_file.h>
* page
*/
if (page_mapcount(page) + 1 + swapped != page_count(page)) {
- set_pte_at_notify(mm, addr, ptep, entry);
+ set_pte_at(mm, addr, ptep, entry);
goto out_unlock;
}
entry = pte_wrprotect(entry);
else
mem = vmalloc(size);
- if (mem)
- memset(mem, 0, size);
+ if (!mem)
+ return NULL;
+
+ memset(mem, 0, size);
mem->stat = alloc_percpu(struct mem_cgroup_stat_cpu);
if (!mem->stat) {
if (size < PAGE_SIZE)
}
return ret;
}
-#else /* !CONFIG_MMU */
-static int mem_cgroup_can_attach(struct cgroup_subsys *ss,
- struct cgroup *cgroup,
- struct task_struct *p,
- bool threadgroup)
-{
- return 0;
-}
-static void mem_cgroup_cancel_attach(struct cgroup_subsys *ss,
- struct cgroup *cgroup,
- struct task_struct *p,
- bool threadgroup)
-{
-}
-static void mem_cgroup_move_task(struct cgroup_subsys *ss,
- struct cgroup *cont,
- struct cgroup *old_cont,
- struct task_struct *p,
- bool threadgroup)
-{
-}
-#endif
/**
* is_target_pte_for_mc - check a pte whether it is valid for move charge
}
mem_cgroup_clear_mc();
}
+#else /* !CONFIG_MMU */
+static int mem_cgroup_can_attach(struct cgroup_subsys *ss,
+ struct cgroup *cgroup,
+ struct task_struct *p,
+ bool threadgroup)
+{
+ return 0;
+}
+static void mem_cgroup_cancel_attach(struct cgroup_subsys *ss,
+ struct cgroup *cgroup,
+ struct task_struct *p,
+ bool threadgroup)
+{
+}
+static void mem_cgroup_move_task(struct cgroup_subsys *ss,
+ struct cgroup *cont,
+ struct cgroup *old_cont,
+ struct task_struct *p,
+ bool threadgroup)
+{
+}
+#endif
struct cgroup_subsys mem_cgroup_subsys = {
.name = "memory",
#include <linux/migrate.h>
#include <linux/page-isolation.h>
#include <linux/suspend.h>
+#include <linux/slab.h>
#include "internal.h"
int sysctl_memory_failure_early_kill __read_mostly = 0;
#include <linux/kallsyms.h>
#include <linux/swapops.h>
#include <linux/elf.h>
+#include <linux/gfp.h>
#include <asm/io.h>
#include <asm/pgalloc.h>
for (i = 0; i < NR_MM_COUNTERS; i++) {
if (task->rss_stat.count[i]) {
+ BUG_ON(!mm);
add_mm_counter(mm, i, task->rss_stat.count[i]);
task->rss_stat.count[i] = 0;
}
#include <linux/sched.h>
#include <linux/nodemask.h>
#include <linux/cpuset.h>
-#include <linux/gfp.h>
#include <linux/slab.h>
#include <linux/string.h>
#include <linux/module.h>
err = 0;
if (nmask) {
- task_lock(current);
- get_policy_nodemask(pol, nmask);
- task_unlock(current);
+ if (mpol_store_user_nodemask(pol)) {
+ *nmask = pol->w.user_nodemask;
+ } else {
+ task_lock(current);
+ get_policy_nodemask(pol, nmask);
+ task_unlock(current);
+ }
}
out:
char *rest = nodelist;
while (isdigit(*rest))
rest++;
- if (!*rest)
- err = 0;
+ if (*rest)
+ goto out;
}
break;
case MPOL_INTERLEAVE:
*/
if (!nodelist)
nodes = node_states[N_HIGH_MEMORY];
- err = 0;
break;
case MPOL_LOCAL:
/*
goto out;
mode = MPOL_PREFERRED;
break;
-
- /*
- * case MPOL_BIND: mpol_new() enforces non-empty nodemask.
- * case MPOL_DEFAULT: mpol_new() enforces empty nodemask, ignores flags.
- */
+ case MPOL_DEFAULT:
+ /*
+ * Insist on a empty nodelist
+ */
+ if (!nodelist)
+ err = 0;
+ goto out;
+ case MPOL_BIND:
+ /*
+ * Insist on a nodelist
+ */
+ if (!nodelist)
+ goto out;
}
mode_flags = 0;
else if (!strcmp(flags, "relative"))
mode_flags |= MPOL_F_RELATIVE_NODES;
else
- err = 1;
+ goto out;
}
new = mpol_new(mode, mode_flags, &nodes);
if (IS_ERR(new))
- err = 1;
- else {
+ goto out;
+
+ {
int ret;
NODEMASK_SCRATCH(scratch);
if (scratch) {
ret = -ENOMEM;
NODEMASK_SCRATCH_FREE(scratch);
if (ret) {
- err = 1;
mpol_put(new);
- } else if (no_context) {
- /* save for contextualization */
- new->w.user_nodemask = nodes;
+ goto out;
}
}
+ err = 0;
+ if (no_context) {
+ /* save for contextualization */
+ new->w.user_nodemask = nodes;
+ }
out:
/* Restore string for error message */
#include <linux/security.h>
#include <linux/memcontrol.h>
#include <linux/syscalls.h>
+#include <linux/gfp.h>
#include "internal.h"
/*
* The mincore() system call.
*/
-#include <linux/slab.h>
#include <linux/pagemap.h>
+#include <linux/gfp.h>
#include <linux/mm.h>
#include <linux/mman.h>
#include <linux/syscalls.h>
struct task_struct *tsk = current;
task_lock(tsk);
+ sync_mm_rss(tsk, mm);
tsk->mm = NULL;
/* active_mm is still 'mm' */
enter_lazy_tlb(mm, tsk);
#include <linux/err.h>
#include <linux/rcupdate.h>
#include <linux/sched.h>
+#include <linux/slab.h>
/*
* This function can't run concurrently against mmu_notifier_register
#include <linux/mm.h>
#include <linux/hugetlb.h>
-#include <linux/slab.h>
#include <linux/shm.h>
#include <linux/mman.h>
#include <linux/fs.h>
#include <linux/mm.h>
#include <linux/hugetlb.h>
-#include <linux/slab.h>
#include <linux/shm.h>
#include <linux/ksm.h>
#include <linux/mman.h>
(VM_MAYREAD | VM_MAYWRITE) : (VM_READ | VM_WRITE);
for (i = 0; i < nr_pages; i++) {
- vma = find_extend_vma(mm, start);
+ vma = find_vma(mm, start);
if (!vma)
goto finish_or_fault;
}
if (vmas)
vmas[i] = vma;
- start += PAGE_SIZE;
+ start = (start + PAGE_SIZE) & PAGE_MASK;
}
return i;
*/
struct vm_area_struct *find_extend_vma(struct mm_struct *mm, unsigned long addr)
{
- return find_vma(mm, addr & PAGE_MASK);
+ return find_vma(mm, addr);
}
/*
if (ret != -ENOSYS)
return ret;
- /* getting an ENOSYS error indicates that direct mmap isn't
- * possible (as opposed to tried but failed) so we'll fall
- * through to making a private copy of the data and mapping
- * that if we can */
+ /* getting -ENOSYS indicates that direct mmap isn't possible (as
+ * opposed to tried but failed) so we can only give a suitable error as
+ * it's not possible to make a private copy if MAP_SHARED was given */
return -ENODEV;
}
#include <linux/oom.h>
#include <linux/mm.h>
#include <linux/err.h>
+#include <linux/gfp.h>
#include <linux/sched.h>
#include <linux/swap.h>
#include <linux/timex.h>
#include <linux/mm.h>
#include <linux/kernel_stat.h>
+#include <linux/gfp.h>
#include <linux/pagemap.h>
#include <linux/swap.h>
#include <linux/bio.h>
}
EXPORT_SYMBOL_GPL(free_percpu);
+/**
+ * is_kernel_percpu_address - test whether address is from static percpu area
+ * @addr: address to test
+ *
+ * Test whether @addr belongs to in-kernel static percpu area. Module
+ * static percpu areas are not considered. For those, use
+ * is_module_percpu_address().
+ *
+ * RETURNS:
+ * %true if @addr is from in-kernel static percpu area, %false otherwise.
+ */
+bool is_kernel_percpu_address(unsigned long addr)
+{
+ const size_t static_size = __per_cpu_end - __per_cpu_start;
+ void __percpu *base = __addr_to_pcpu_ptr(pcpu_base_addr);
+ unsigned int cpu;
+
+ for_each_possible_cpu(cpu) {
+ void *start = per_cpu_ptr(base, cpu);
+
+ if ((void *)addr >= start && (void *)addr < start + static_size)
+ return true;
+ }
+ return false;
+}
+
/**
* per_cpu_ptr_to_phys - convert translated percpu address to physical address
* @addr: the address to be converted to physical address
--- /dev/null
+/*
+ * mm/percpu_up.c - dummy percpu memory allocator implementation for UP
+ */
+
+#include <linux/module.h>
+#include <linux/percpu.h>
+#include <linux/slab.h>
+
+void __percpu *__alloc_percpu(size_t size, size_t align)
+{
+ /*
+ * Can't easily make larger alignment work with kmalloc. WARN
+ * on it. Larger alignment should only be used for module
+ * percpu sections on SMP for which this path isn't used.
+ */
+ WARN_ON_ONCE(align > SMP_CACHE_BYTES);
+ return kzalloc(size, GFP_KERNEL);
+}
+EXPORT_SYMBOL_GPL(__alloc_percpu);
+
+void free_percpu(void __percpu *p)
+{
+ kfree(p);
+}
+EXPORT_SYMBOL_GPL(free_percpu);
+
+phys_addr_t per_cpu_ptr_to_phys(void *addr)
+{
+ return __pa(addr);
+}
*/
#include <linux/kernel.h>
+#include <linux/gfp.h>
#include <linux/mm.h>
#include <linux/mmzone.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/fs.h>
+#include <linux/gfp.h>
#include <linux/mm.h>
#include <linux/module.h>
#include <linux/blkdev.h>
out_error_free_anon_vma:
anon_vma_free(anon_vma);
out_error:
+ unlink_anon_vmas(vma);
return -ENOMEM;
}
#include <linux/bootmem.h>
#include <linux/highmem.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/vmalloc.h>
#include <linux/sched.h>
* sparse memory mappings.
*/
#include <linux/mm.h>
+#include <linux/slab.h>
#include <linux/mmzone.h>
#include <linux/bootmem.h>
#include <linux/highmem.h>
#include <linux/notifier.h>
#include <linux/backing-dev.h>
#include <linux/memcontrol.h>
+#include <linux/gfp.h>
#include "internal.h"
*/
#include <linux/module.h>
#include <linux/mm.h>
+#include <linux/gfp.h>
#include <linux/kernel_stat.h>
#include <linux/swap.h>
#include <linux/swapops.h>
#include <linux/kernel.h>
#include <linux/backing-dev.h>
+#include <linux/gfp.h>
#include <linux/mm.h>
#include <linux/swap.h>
#include <linux/module.h>
#include <linux/mm.h>
#include <linux/module.h>
-#include <linux/slab.h>
+#include <linux/gfp.h>
#include <linux/kernel_stat.h>
#include <linux/swap.h>
#include <linux/pagemap.h>
#include <linux/mm.h>
#include <linux/err.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/cpu.h>
#include <linux/vmstat.h>
#include <linux/sched.h>
#include <linux/etherdevice.h>
#include <linux/rtnetlink.h>
#include <linux/llc.h>
+#include <linux/slab.h>
#include <net/llc.h>
#include <net/llc_pdu.h>
#include <net/garp.h>
#include <linux/module.h>
#include <linux/netdevice.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <net/datalink.h>
#include <linux/mm.h>
#include <linux/in.h>
#include <linux/module.h>
#include <linux/netdevice.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <net/datalink.h>
#include <net/p8022.h>
#include <linux/module.h>
#include <linux/netdevice.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <net/datalink.h>
#include <net/llc.h>
#include <net/psnap.h>
#include <linux/skbuff.h>
#include <linux/etherdevice.h>
#include <linux/llc.h>
+#include <linux/slab.h>
#include <net/llc.h>
#include <net/llc_pdu.h>
#include <net/stp.h>
#include <linux/seq_file.h>
#include <linux/init.h>
#include <linux/sysctl.h>
+#include <linux/slab.h>
#include <net/arp.h>
#include <net/net_namespace.h>
#include <linux/module.h>
#include <linux/netdevice.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/rculist.h>
#include <net/p8022.h>
#if defined(CONFIG_FCOE) || defined(CONFIG_FCOE_MODULE)
vlandev->fcoe_ddp_xid = dev->fcoe_ddp_xid;
#endif
+ vlandev->real_num_tx_queues = dev->real_num_tx_queues;
+ BUG_ON(vlandev->real_num_tx_queues > vlandev->num_tx_queues);
if (old_features != vlandev->features)
netdev_features_change(vlandev);
if (netpoll_rx(skb))
return NET_RX_DROP;
- if (skb_bond_should_drop(skb))
+ if (skb_bond_should_drop(skb, ACCESS_ONCE(skb->dev->master)))
goto drop;
skb->skb_iif = skb->dev->ifindex;
{
struct sk_buff *p;
- if (skb_bond_should_drop(skb))
+ if (skb_bond_should_drop(skb, ACCESS_ONCE(skb->dev->master)))
goto drop;
skb->skb_iif = skb->dev->ifindex;
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/skbuff.h>
#include <linux/netdevice.h>
#include <linux/etherdevice.h>
return ret;
}
+static u16 vlan_dev_select_queue(struct net_device *dev, struct sk_buff *skb)
+{
+ struct net_device *rdev = vlan_dev_info(dev)->real_dev;
+ const struct net_device_ops *ops = rdev->netdev_ops;
+
+ return ops->ndo_select_queue(rdev, skb);
+}
+
static int vlan_dev_change_mtu(struct net_device *dev, int new_mtu)
{
/* TODO: gotta make sure the underlying layer can handle it,
.parse = eth_header_parse,
};
-static const struct net_device_ops vlan_netdev_ops, vlan_netdev_accel_ops;
+static const struct net_device_ops vlan_netdev_ops, vlan_netdev_accel_ops,
+ vlan_netdev_ops_sq, vlan_netdev_accel_ops_sq;
static int vlan_dev_init(struct net_device *dev)
{
if (real_dev->features & NETIF_F_HW_VLAN_TX) {
dev->header_ops = real_dev->header_ops;
dev->hard_header_len = real_dev->hard_header_len;
- dev->netdev_ops = &vlan_netdev_accel_ops;
+ if (real_dev->netdev_ops->ndo_select_queue)
+ dev->netdev_ops = &vlan_netdev_accel_ops_sq;
+ else
+ dev->netdev_ops = &vlan_netdev_accel_ops;
} else {
dev->header_ops = &vlan_header_ops;
dev->hard_header_len = real_dev->hard_header_len + VLAN_HLEN;
- dev->netdev_ops = &vlan_netdev_ops;
+ if (real_dev->netdev_ops->ndo_select_queue)
+ dev->netdev_ops = &vlan_netdev_ops_sq;
+ else
+ dev->netdev_ops = &vlan_netdev_ops;
}
if (is_vlan_dev(real_dev))
#endif
};
+static const struct net_device_ops vlan_netdev_ops_sq = {
+ .ndo_select_queue = vlan_dev_select_queue,
+ .ndo_change_mtu = vlan_dev_change_mtu,
+ .ndo_init = vlan_dev_init,
+ .ndo_uninit = vlan_dev_uninit,
+ .ndo_open = vlan_dev_open,
+ .ndo_stop = vlan_dev_stop,
+ .ndo_start_xmit = vlan_dev_hard_start_xmit,
+ .ndo_validate_addr = eth_validate_addr,
+ .ndo_set_mac_address = vlan_dev_set_mac_address,
+ .ndo_set_rx_mode = vlan_dev_set_rx_mode,
+ .ndo_set_multicast_list = vlan_dev_set_rx_mode,
+ .ndo_change_rx_flags = vlan_dev_change_rx_flags,
+ .ndo_do_ioctl = vlan_dev_ioctl,
+ .ndo_neigh_setup = vlan_dev_neigh_setup,
+ .ndo_get_stats = vlan_dev_get_stats,
+#if defined(CONFIG_FCOE) || defined(CONFIG_FCOE_MODULE)
+ .ndo_fcoe_ddp_setup = vlan_dev_fcoe_ddp_setup,
+ .ndo_fcoe_ddp_done = vlan_dev_fcoe_ddp_done,
+ .ndo_fcoe_enable = vlan_dev_fcoe_enable,
+ .ndo_fcoe_disable = vlan_dev_fcoe_disable,
+ .ndo_fcoe_get_wwn = vlan_dev_fcoe_get_wwn,
+#endif
+};
+
+static const struct net_device_ops vlan_netdev_accel_ops_sq = {
+ .ndo_select_queue = vlan_dev_select_queue,
+ .ndo_change_mtu = vlan_dev_change_mtu,
+ .ndo_init = vlan_dev_init,
+ .ndo_uninit = vlan_dev_uninit,
+ .ndo_open = vlan_dev_open,
+ .ndo_stop = vlan_dev_stop,
+ .ndo_start_xmit = vlan_dev_hwaccel_hard_start_xmit,
+ .ndo_validate_addr = eth_validate_addr,
+ .ndo_set_mac_address = vlan_dev_set_mac_address,
+ .ndo_set_rx_mode = vlan_dev_set_rx_mode,
+ .ndo_set_multicast_list = vlan_dev_set_rx_mode,
+ .ndo_change_rx_flags = vlan_dev_change_rx_flags,
+ .ndo_do_ioctl = vlan_dev_ioctl,
+ .ndo_neigh_setup = vlan_dev_neigh_setup,
+ .ndo_get_stats = vlan_dev_get_stats,
+#if defined(CONFIG_FCOE) || defined(CONFIG_FCOE_MODULE)
+ .ndo_fcoe_ddp_setup = vlan_dev_fcoe_ddp_setup,
+ .ndo_fcoe_ddp_done = vlan_dev_fcoe_ddp_done,
+ .ndo_fcoe_enable = vlan_dev_fcoe_enable,
+ .ndo_fcoe_disable = vlan_dev_fcoe_disable,
+ .ndo_fcoe_get_wwn = vlan_dev_fcoe_get_wwn,
+#endif
+};
+
void vlan_setup(struct net_device *dev)
{
ether_setup(dev);
#include <linux/poll.h>
#include <linux/idr.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#include <linux/uaccess.h>
#include <net/9p/9p.h>
EXPORT_SYMBOL(p9_is_proto_dotu);
/* Interpret mount option for protocol version */
-static unsigned char get_protocol_version(const substring_t *name)
+static int get_protocol_version(const substring_t *name)
{
- unsigned char version = -EINVAL;
+ int version = -EINVAL;
+
if (!strncmp("9p2000", name->from, name->to-name->from)) {
version = p9_proto_legacy;
P9_DPRINTK(P9_DEBUG_9P, "Protocol version: Legacy\n");
P9_DPRINTK(P9_DEBUG_MUX, "client %p op %d\n", c, type);
- if (c->status != Connected)
+ /* we allow for any status other than disconnected */
+ if (c->status == Disconnected)
+ return ERR_PTR(-EIO);
+
+ /* if status is begin_disconnected we allow only clunk request */
+ if ((c->status == BeginDisconnect) && (type != P9_TCLUNK))
return ERR_PTR(-EIO);
if (signal_pending(current)) {
v9fs_put_trans(clnt->trans_mod);
- list_for_each_entry_safe(fid, fidptr, &clnt->fidlist, flist)
+ list_for_each_entry_safe(fid, fidptr, &clnt->fidlist, flist) {
+ printk(KERN_INFO "Found fid %d not clunked\n", fid->fid);
p9_fid_destroy(fid);
+ }
if (clnt->fidpool)
p9_idpool_destroy(clnt->fidpool);
}
EXPORT_SYMBOL(p9_client_disconnect);
+void p9_client_begin_disconnect(struct p9_client *clnt)
+{
+ P9_DPRINTK(P9_DEBUG_9P, "clnt %p\n", clnt);
+ clnt->status = BeginDisconnect;
+}
+EXPORT_SYMBOL(p9_client_begin_disconnect);
+
struct p9_fid *p9_client_attach(struct p9_client *clnt, struct p9_fid *afid,
char *uname, u32 n_uname, char *aname)
{
#include <linux/module.h>
#include <linux/errno.h>
#include <linux/uaccess.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#include <linux/types.h>
#include <net/9p/9p.h>
#include <linux/idr.h>
#include <linux/file.h>
#include <linux/parser.h>
+#include <linux/slab.h>
#include <net/9p/9p.h>
#include <net/9p/client.h>
#include <net/9p/transport.h>
#include <linux/file.h>
#include <linux/parser.h>
#include <linux/semaphore.h>
+#include <linux/slab.h>
#include <net/9p/9p.h>
#include <net/9p/client.h>
#include <net/9p/transport.h>
#include <linux/inet.h>
#include <linux/idr.h>
#include <linux/file.h>
+#include <linux/slab.h>
#include <net/9p/9p.h>
#include <linux/parser.h>
#include <net/9p/client.h>
#include <linux/sched.h>
#include <linux/parser.h>
#include <linux/idr.h>
+#include <linux/slab.h>
#include <net/9p/9p.h>
/**
*/
#include <linux/if_arp.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/datalink.h>
#include <net/psnap.h>
#include <linux/smp_lock.h>
#include <linux/termios.h> /* For TIOCOUTQ/INQ */
#include <linux/compat.h>
+#include <linux/slab.h>
#include <net/datalink.h>
#include <net/psnap.h>
#include <net/sock.h>
#include <linux/atm.h>
#include <linux/atmdev.h>
+#include <linux/slab.h>
#include <linux/uaccess.h>
#include "signaling.h"
/* ATM driver model support. */
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/kobject.h>
#include <linux/atmdev.h>
#include <linux/rtnetlink.h>
#include <linux/ip.h>
#include <linux/uaccess.h>
+#include <linux/slab.h>
#include <net/arp.h>
#include <linux/atm.h>
#include <linux/atmdev.h>
#include <linux/seq_file.h>
#include <linux/rcupdate.h>
#include <linux/jhash.h>
+#include <linux/slab.h>
#include <net/route.h> /* for struct rtable and routing */
#include <net/icmp.h> /* icmp_send */
#include <linux/param.h> /* for HZ */
#include <linux/skbuff.h>
#include <linux/bitops.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <net/sock.h> /* struct sock */
#include <linux/uaccess.h>
#include <linux/poll.h>
#define pr_fmt(fmt) KBUILD_MODNAME ":%s: " fmt, __func__
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/bitops.h>
#include <linux/capability.h>
#include <linux/kernel.h>
#include <linux/string.h>
+#include <linux/slab.h>
#include <linux/timer.h>
#include <linux/init.h>
#include <linux/bitops.h>
#include <linux/types.h>
#include <linux/atmmpc.h>
+#include <linux/slab.h>
#include <linux/time.h>
#include "mpoa_caches.h"
#include <linux/uaccess.h>
#include <linux/atmmpc.h>
#include <linux/atm.h>
+#include <linux/gfp.h>
#include "mpc.h"
#include "mpoa_caches.h"
#include <linux/module.h>
#include <linux/init.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <linux/atm.h>
#include <linux/atmdev.h>
#include <linux/capability.h>
#include <linux/netdevice.h>
#include <linux/atmclip.h>
#include <linux/init.h> /* for __init */
+#include <linux/slab.h>
#include <net/net_namespace.h>
#include <net/atmclip.h>
#include <linux/uaccess.h>
#include <linux/kernel.h>
#include <linux/skbuff.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include "common.h"
#include "protocols.h"
#include <linux/capability.h>
#include <linux/delay.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <net/sock.h> /* for struct sock */
#include <linux/atmsvc.h>
#include <linux/atmdev.h>
#include <linux/bitops.h>
+#include <linux/slab.h>
#include "resources.h"
#include "signaling.h"
#include <linux/string.h>
#include <linux/sockios.h>
#include <linux/net.h>
+#include <linux/slab.h>
#include <net/ax25.h>
#include <linux/inet.h>
#include <linux/netdevice.h>
#include <linux/errno.h>
#include <linux/types.h>
#include <linux/socket.h>
+#include <linux/slab.h>
#include <linux/in.h>
#include <linux/kernel.h>
#include <linux/timer.h>
#include <linux/sockios.h>
#include <linux/spinlock.h>
#include <linux/net.h>
+#include <linux/gfp.h>
#include <net/ax25.h>
#include <linux/inet.h>
#include <linux/netdevice.h>
#include <linux/string.h>
#include <linux/sockios.h>
#include <linux/net.h>
+#include <linux/slab.h>
#include <net/ax25.h>
#include <linux/inet.h>
#include <linux/netdevice.h>
#include <linux/string.h>
#include <linux/sockios.h>
#include <linux/net.h>
+#include <linux/slab.h>
#include <net/ax25.h>
#include <linux/inet.h>
#include <linux/netdevice.h>
#include <linux/string.h>
#include <linux/sockios.h>
#include <linux/net.h>
+#include <linux/slab.h>
#include <net/ax25.h>
#include <linux/inet.h>
#include <linux/netdevice.h>
#include <linux/sockios.h>
#include <linux/spinlock.h>
#include <linux/net.h>
+#include <linux/slab.h>
#include <net/ax25.h>
#include <linux/inet.h>
#include <linux/netdevice.h>
#include <linux/string.h>
#include <linux/sockios.h>
#include <linux/net.h>
+#include <linux/slab.h>
#include <net/ax25.h>
#include <linux/inet.h>
#include <linux/netdevice.h>
#include <linux/string.h>
#include <linux/sockios.h>
#include <linux/net.h>
+#include <linux/slab.h>
#include <net/ax25.h>
#include <linux/inet.h>
#include <linux/netdevice.h>
#include <linux/sockios.h>
#include <linux/net.h>
#include <linux/spinlock.h>
+#include <linux/slab.h>
#include <net/ax25.h>
#include <linux/inet.h>
#include <linux/netdevice.h>
* Copyright (C) 1996 Mike Shaver (shaver@zeroknowledge.com)
*/
#include <linux/mm.h>
+#include <linux/slab.h>
#include <linux/sysctl.h>
#include <linux/spinlock.h>
#include <net/ax25.h>
#include <linux/errno.h>
#include <linux/kernel.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/skbuff.h>
#include <linux/init.h>
#include <linux/poll.h>
#include <linux/freezer.h>
#include <linux/errno.h>
#include <linux/net.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <linux/socket.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/socket.h>
#include <linux/netdevice.h>
#include <linux/capability.h>
#include <linux/errno.h>
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/poll.h>
#include <linux/fcntl.h>
#include <linux/skbuff.h>
#include <linux/file.h>
#include <linux/init.h>
#include <linux/compat.h>
+#include <linux/gfp.h>
#include <net/sock.h>
#include <asm/system.h>
#include <linux/capability.h>
#include <linux/errno.h>
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/poll.h>
#include <linux/fcntl.h>
#include <linux/skbuff.h>
#include <linux/ioctl.h>
#include <linux/file.h>
#include <linux/compat.h>
+#include <linux/gfp.h>
#include <net/sock.h>
#include <linux/isdn/capilli.h>
/* Bluetooth HCI driver model support. */
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/debugfs.h>
#include <linux/seq_file.h>
#include <net/bluetooth/bluetooth.h>
#include <net/bluetooth/hci_core.h>
-struct class *bt_class = NULL;
-EXPORT_SYMBOL_GPL(bt_class);
+static struct class *bt_class;
struct dentry *bt_debugfs = NULL;
EXPORT_SYMBOL_GPL(bt_debugfs);
#include <linux/capability.h>
#include <linux/errno.h>
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/poll.h>
#include <linux/fcntl.h>
#include <linux/skbuff.h>
#include <linux/file.h>
#include <linux/init.h>
#include <linux/compat.h>
+#include <linux/gfp.h>
#include <net/sock.h>
#include "hidp.h"
#include <linux/skbuff.h>
#include <linux/list.h>
#include <linux/device.h>
+#include <linux/debugfs.h>
+#include <linux/seq_file.h>
#include <linux/uaccess.h>
#include <linux/crc16.h>
#include <net/sock.h>
BT_DBG("sk %p", sk);
- if (!addr || addr->sa_family != AF_BLUETOOTH)
+ if (!addr || alen < sizeof(addr->sa_family) ||
+ addr->sa_family != AF_BLUETOOTH)
return -EINVAL;
memset(&la, 0, sizeof(la));
int len = cmd->len - sizeof(*rsp);
char req[64];
+ if (len > sizeof(req) - sizeof(struct l2cap_conf_req)) {
+ l2cap_send_disconn_req(conn, sk);
+ goto done;
+ }
+
/* throw out any old stored conf requests */
result = L2CAP_CONF_SUCCESS;
len = l2cap_parse_conf_rsp(sk, rsp->data,
return 0;
}
-static ssize_t l2cap_sysfs_show(struct class *dev,
- struct class_attribute *attr,
- char *buf)
+static int l2cap_debugfs_show(struct seq_file *f, void *p)
{
struct sock *sk;
struct hlist_node *node;
- char *str = buf;
read_lock_bh(&l2cap_sk_list.lock);
sk_for_each(sk, node, &l2cap_sk_list.head) {
struct l2cap_pinfo *pi = l2cap_pi(sk);
- str += sprintf(str, "%s %s %d %d 0x%4.4x 0x%4.4x %d %d %d\n",
- batostr(&bt_sk(sk)->src), batostr(&bt_sk(sk)->dst),
- sk->sk_state, __le16_to_cpu(pi->psm), pi->scid,
- pi->dcid, pi->imtu, pi->omtu, pi->sec_level);
+ seq_printf(f, "%s %s %d %d 0x%4.4x 0x%4.4x %d %d %d\n",
+ batostr(&bt_sk(sk)->src),
+ batostr(&bt_sk(sk)->dst),
+ sk->sk_state, __le16_to_cpu(pi->psm),
+ pi->scid, pi->dcid,
+ pi->imtu, pi->omtu, pi->sec_level);
}
read_unlock_bh(&l2cap_sk_list.lock);
- return str - buf;
+ return 0;
+}
+
+static int l2cap_debugfs_open(struct inode *inode, struct file *file)
+{
+ return single_open(file, l2cap_debugfs_show, inode->i_private);
}
-static CLASS_ATTR(l2cap, S_IRUGO, l2cap_sysfs_show, NULL);
+static const struct file_operations l2cap_debugfs_fops = {
+ .open = l2cap_debugfs_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = single_release,
+};
+
+static struct dentry *l2cap_debugfs;
static const struct proto_ops l2cap_sock_ops = {
.family = PF_BLUETOOTH,
goto error;
}
- if (class_create_file(bt_class, &class_attr_l2cap) < 0)
- BT_ERR("Failed to create L2CAP info file");
+ if (bt_debugfs) {
+ l2cap_debugfs = debugfs_create_file("l2cap", 0444,
+ bt_debugfs, NULL, &l2cap_debugfs_fops);
+ if (!l2cap_debugfs)
+ BT_ERR("Failed to create L2CAP debug file");
+ }
BT_INFO("L2CAP ver %s", VERSION);
BT_INFO("L2CAP socket layer initialized");
static void __exit l2cap_exit(void)
{
- class_remove_file(bt_class, &class_attr_l2cap);
+ debugfs_remove(l2cap_debugfs);
if (bt_sock_unregister(BTPROTO_L2CAP) < 0)
BT_ERR("L2CAP socket unregistration failed");
#include <linux/init.h>
#include <linux/wait.h>
#include <linux/device.h>
+#include <linux/debugfs.h>
+#include <linux/seq_file.h>
#include <linux/net.h>
#include <linux/mutex.h>
#include <linux/kthread.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <asm/uaccess.h>
.security_cfm = rfcomm_security_cfm
};
-static ssize_t rfcomm_dlc_sysfs_show(struct class *dev,
- struct class_attribute *attr,
- char *buf)
+static int rfcomm_dlc_debugfs_show(struct seq_file *f, void *x)
{
struct rfcomm_session *s;
struct list_head *pp, *p;
- char *str = buf;
rfcomm_lock();
struct sock *sk = s->sock->sk;
struct rfcomm_dlc *d = list_entry(pp, struct rfcomm_dlc, list);
- str += sprintf(str, "%s %s %ld %d %d %d %d\n",
- batostr(&bt_sk(sk)->src), batostr(&bt_sk(sk)->dst),
- d->state, d->dlci, d->mtu, d->rx_credits, d->tx_credits);
+ seq_printf(f, "%s %s %ld %d %d %d %d\n",
+ batostr(&bt_sk(sk)->src),
+ batostr(&bt_sk(sk)->dst),
+ d->state, d->dlci, d->mtu,
+ d->rx_credits, d->tx_credits);
}
}
rfcomm_unlock();
- return (str - buf);
+ return 0;
}
-static CLASS_ATTR(rfcomm_dlc, S_IRUGO, rfcomm_dlc_sysfs_show, NULL);
+static int rfcomm_dlc_debugfs_open(struct inode *inode, struct file *file)
+{
+ return single_open(file, rfcomm_dlc_debugfs_show, inode->i_private);
+}
+
+static const struct file_operations rfcomm_dlc_debugfs_fops = {
+ .open = rfcomm_dlc_debugfs_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = single_release,
+};
+
+static struct dentry *rfcomm_dlc_debugfs;
/* ---- Initialization ---- */
static int __init rfcomm_init(void)
goto unregister;
}
- if (class_create_file(bt_class, &class_attr_rfcomm_dlc) < 0)
- BT_ERR("Failed to create RFCOMM info file");
+ if (bt_debugfs) {
+ rfcomm_dlc_debugfs = debugfs_create_file("rfcomm_dlc", 0444,
+ bt_debugfs, NULL, &rfcomm_dlc_debugfs_fops);
+ if (!rfcomm_dlc_debugfs)
+ BT_ERR("Failed to create RFCOMM debug file");
+ }
err = rfcomm_init_ttys();
if (err < 0)
static void __exit rfcomm_exit(void)
{
- class_remove_file(bt_class, &class_attr_rfcomm_dlc);
+ debugfs_remove(rfcomm_dlc_debugfs);
hci_unregister_cb(&rfcomm_cb);
#include <linux/skbuff.h>
#include <linux/list.h>
#include <linux/device.h>
+#include <linux/debugfs.h>
+#include <linux/seq_file.h>
#include <net/sock.h>
#include <asm/system.h>
BT_DBG("sk %p", sk);
- if (addr->sa_family != AF_BLUETOOTH || alen < sizeof(struct sockaddr_rc))
+ if (alen < sizeof(struct sockaddr_rc) ||
+ addr->sa_family != AF_BLUETOOTH)
return -EINVAL;
lock_sock(sk);
return result;
}
-static ssize_t rfcomm_sock_sysfs_show(struct class *dev,
- struct class_attribute *attr,
- char *buf)
+static int rfcomm_sock_debugfs_show(struct seq_file *f, void *p)
{
struct sock *sk;
struct hlist_node *node;
- char *str = buf;
read_lock_bh(&rfcomm_sk_list.lock);
sk_for_each(sk, node, &rfcomm_sk_list.head) {
- str += sprintf(str, "%s %s %d %d\n",
- batostr(&bt_sk(sk)->src), batostr(&bt_sk(sk)->dst),
+ seq_printf(f, "%s %s %d %d\n",
+ batostr(&bt_sk(sk)->src),
+ batostr(&bt_sk(sk)->dst),
sk->sk_state, rfcomm_pi(sk)->channel);
}
read_unlock_bh(&rfcomm_sk_list.lock);
- return (str - buf);
+ return 0;
}
-static CLASS_ATTR(rfcomm, S_IRUGO, rfcomm_sock_sysfs_show, NULL);
+static int rfcomm_sock_debugfs_open(struct inode *inode, struct file *file)
+{
+ return single_open(file, rfcomm_sock_debugfs_show, inode->i_private);
+}
+
+static const struct file_operations rfcomm_sock_debugfs_fops = {
+ .open = rfcomm_sock_debugfs_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = single_release,
+};
+
+static struct dentry *rfcomm_sock_debugfs;
static const struct proto_ops rfcomm_sock_ops = {
.family = PF_BLUETOOTH,
if (err < 0)
goto error;
- if (class_create_file(bt_class, &class_attr_rfcomm) < 0)
- BT_ERR("Failed to create RFCOMM info file");
+ if (bt_debugfs) {
+ rfcomm_sock_debugfs = debugfs_create_file("rfcomm", 0444,
+ bt_debugfs, NULL, &rfcomm_sock_debugfs_fops);
+ if (!rfcomm_sock_debugfs)
+ BT_ERR("Failed to create RFCOMM debug file");
+ }
BT_INFO("RFCOMM socket layer initialized");
void rfcomm_cleanup_sockets(void)
{
- class_remove_file(bt_class, &class_attr_rfcomm);
+ debugfs_remove(rfcomm_sock_debugfs);
if (bt_sock_unregister(BTPROTO_RFCOMM) < 0)
BT_ERR("RFCOMM socket layer unregistration failed");
#include <linux/socket.h>
#include <linux/skbuff.h>
#include <linux/device.h>
+#include <linux/debugfs.h>
+#include <linux/seq_file.h>
#include <linux/list.h>
#include <net/sock.h>
BT_DBG("sk %p", sk);
- if (addr->sa_family != AF_BLUETOOTH || alen < sizeof(struct sockaddr_sco))
+ if (alen < sizeof(struct sockaddr_sco) ||
+ addr->sa_family != AF_BLUETOOTH)
return -EINVAL;
if (sk->sk_state != BT_OPEN && sk->sk_state != BT_BOUND)
return 0;
}
-static ssize_t sco_sysfs_show(struct class *dev,
- struct class_attribute *attr,
- char *buf)
+static int sco_debugfs_show(struct seq_file *f, void *p)
{
struct sock *sk;
struct hlist_node *node;
- char *str = buf;
read_lock_bh(&sco_sk_list.lock);
sk_for_each(sk, node, &sco_sk_list.head) {
- str += sprintf(str, "%s %s %d\n",
- batostr(&bt_sk(sk)->src), batostr(&bt_sk(sk)->dst),
- sk->sk_state);
+ seq_printf(f, "%s %s %d\n", batostr(&bt_sk(sk)->src),
+ batostr(&bt_sk(sk)->dst), sk->sk_state);
}
read_unlock_bh(&sco_sk_list.lock);
- return (str - buf);
+ return 0;
}
-static CLASS_ATTR(sco, S_IRUGO, sco_sysfs_show, NULL);
+static int sco_debugfs_open(struct inode *inode, struct file *file)
+{
+ return single_open(file, sco_debugfs_show, inode->i_private);
+}
+
+static const struct file_operations sco_debugfs_fops = {
+ .open = sco_debugfs_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = single_release,
+};
+
+static struct dentry *sco_debugfs;
static const struct proto_ops sco_sock_ops = {
.family = PF_BLUETOOTH,
goto error;
}
- if (class_create_file(bt_class, &class_attr_sco) < 0)
- BT_ERR("Failed to create SCO info file");
+ if (bt_debugfs) {
+ sco_debugfs = debugfs_create_file("sco", 0444,
+ bt_debugfs, NULL, &sco_debugfs_fops);
+ if (!sco_debugfs)
+ BT_ERR("Failed to create SCO debug file");
+ }
BT_INFO("SCO (Voice Link) ver %s", VERSION);
BT_INFO("SCO socket layer initialized");
static void __exit sco_exit(void)
{
- class_remove_file(bt_class, &class_attr_sco);
+ debugfs_remove(sco_debugfs);
if (bt_sock_unregister(BTPROTO_SCO) < 0)
BT_ERR("SCO socket unregistration failed");
#include <linux/etherdevice.h>
#include <linux/jhash.h>
#include <linux/random.h>
+#include <linux/slab.h>
#include <asm/atomic.h>
#include <asm/unaligned.h>
#include "br_private.h"
*/
#include <linux/err.h>
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/netdevice.h>
#include <linux/skbuff.h>
#include <linux/init.h>
#include <linux/rtnetlink.h>
#include <linux/if_ether.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include "br_private.h"
* 2 of the License, or (at your option) any later version.
*/
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/netdevice.h>
#include <linux/etherdevice.h>
#include <linux/kernel.h>
#include <linux/if_bridge.h>
#include <linux/netdevice.h>
+#include <linux/slab.h>
#include <linux/times.h>
#include <net/net_namespace.h>
#include <asm/uaccess.h>
#include <linux/module.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/ip.h>
#include <linux/netdevice.h>
#include <linux/skbuff.h>
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <net/rtnetlink.h>
#include <net/net_namespace.h>
#include <net/sock.h>
#include <linux/netfilter_bridge.h>
#include <linux/etherdevice.h>
#include <linux/llc.h>
+#include <linux/slab.h>
#include <net/net_namespace.h>
#include <net/llc.h>
#include <net/llc_pdu.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/socket.h>
#include <linux/skbuff.h>
#include <linux/netfilter_bridge/ebtables.h>
#include <linux/spinlock.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <linux/smp.h>
#include <linux/cpumask.h>
#include <linux/can.h>
#include <linux/can/core.h>
#include <linux/can/bcm.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/net_namespace.h>
struct sock *sk = sock->sk;
struct bcm_sock *bo = bcm_sk(sk);
+ if (len < sizeof(*addr))
+ return -EINVAL;
+
if (bo->bound)
return -EISCONN;
#include <linux/init.h>
#include <linux/uio.h>
#include <linux/net.h>
+#include <linux/slab.h>
#include <linux/netdevice.h>
#include <linux/socket.h>
#include <linux/if_arp.h>
*/
#include <linux/kernel.h>
+#include <linux/gfp.h>
#include <linux/fs.h>
#include <linux/types.h>
#include <linux/file.h>
#include <linux/poll.h>
#include <linux/highmem.h>
#include <linux/spinlock.h>
+#include <linux/slab.h>
#include <net/protocol.h>
#include <linux/skbuff.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/hash.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#include <linux/mutex.h>
#include <linux/string.h>
{
struct packet_type *ptype, *pt_prev;
struct net_device *orig_dev;
+ struct net_device *master;
struct net_device *null_or_orig;
struct net_device *null_or_bond;
int ret = NET_RX_DROP;
null_or_orig = NULL;
orig_dev = skb->dev;
- if (orig_dev->master) {
- if (skb_bond_should_drop(skb))
+ master = ACCESS_ONCE(orig_dev->master);
+ if (master) {
+ if (skb_bond_should_drop(skb, master))
null_or_orig = orig_dev; /* deliver only exact match */
else
- skb->dev = orig_dev->master;
+ skb->dev = master;
}
__get_cpu_var(netdev_rx_stat).total++;
#include <linux/percpu.h>
#include <linux/timer.h>
#include <linux/bitops.h>
+#include <linux/slab.h>
#include <net/genetlink.h>
#include <net/netevent.h>
#include <linux/workqueue.h>
#include <linux/mm.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/netdevice.h>
#include <linux/skbuff.h>
#include <linux/string.h>
#include <linux/ethtool.h>
#include <linux/netdevice.h>
#include <linux/bitops.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
/*
#include <linux/types.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/list.h>
#include <net/net_namespace.h>
#include <net/sock.h>
#include <linux/inet.h>
#include <linux/netdevice.h>
#include <linux/if_packet.h>
+#include <linux/gfp.h>
#include <net/ip.h>
#include <net/protocol.h>
#include <net/netlink.h>
#include <linux/rtnetlink.h>
#include <linux/init.h>
#include <linux/rbtree.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/gen_stats.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/net.h>
#include <linux/in6.h>
#include <asm/uaccess.h>
#include <linux/rtnetlink.h>
#include <linux/jiffies.h>
#include <linux/spinlock.h>
-#include <linux/slab.h>
#include <linux/workqueue.h>
#include <linux/bitops.h>
#include <asm/types.h>
* Harald Welte Add neighbour cache statistics like rtstat
*/
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/netdevice.h>
#include <linux/if_arp.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <linux/rtnetlink.h>
#include <linux/wireless.h>
#include <linux/workqueue.h>
#include <linux/netlink.h>
#include <linux/net_dropmon.h>
+#include <linux/slab.h>
#include <asm/unaligned.h>
#include <asm/bitops.h>
#include <linux/delay.h>
#include <linux/rcupdate.h>
#include <linux/workqueue.h>
+#include <linux/slab.h>
#include <net/tcp.h>
#include <net/udp.h>
#include <asm/unaligned.h>
np->name, np->local_port);
printk(KERN_INFO "%s: local IP %pI4\n",
np->name, &np->local_ip);
- printk(KERN_INFO "%s: interface %s\n",
+ printk(KERN_INFO "%s: interface '%s'\n",
np->name, np->dev_name);
printk(KERN_INFO "%s: remote port %d\n",
np->name, np->remote_port);
if ((delim = strchr(cur, '@')) == NULL)
goto parse_failed;
*delim = 0;
+ if (*cur == ' ' || *cur == '\t')
+ printk(KERN_INFO "%s: warning: whitespace"
+ "is not allowed\n", np->name);
np->remote_port = simple_strtol(cur, NULL, 10);
cur = delim;
}
return 0;
parse_failed:
- printk(KERN_INFO "%s: couldn't parse config at %s!\n",
+ printk(KERN_INFO "%s: couldn't parse config at '%s'!\n",
np->name, cur);
return -1;
}
#include <linux/security.h>
#include <linux/pid.h>
#include <linux/nsproxy.h>
+#include <linux/slab.h>
#include <asm/system.h>
#include <asm/uaccess.h>
#include <linux/netdevice.h>
#include <linux/ratelimit.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <net/ip.h>
#include <net/sock.h>
#include <linux/netdevice.h>
#include <linux/netlink.h>
+#include <linux/slab.h>
#include <net/netlink.h>
#include <net/rtnetlink.h>
#include <linux/dcbnl.h>
* published by the Free Software Foundation.
*/
+#include <linux/slab.h>
+
#include "ccid.h"
#include "ccids/lib/tfrc.h"
/*
* This implementation should follow RFC 4341
*/
+#include <linux/slab.h>
#include "../feat.h"
#include "../ccid.h"
#include "../dccp.h"
* 2 of the License, or (at your option) any later version.
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include "ccid.h"
#include "feat.h"
#include <linux/dccp.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <linux/dccp.h>
#include <linux/icmp.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/skbuff.h>
#include <linux/random.h>
#include <linux/module.h>
#include <linux/random.h>
+#include <linux/slab.h>
#include <linux/xfrm.h>
#include <net/addrconf.h>
*/
#include <linux/dccp.h>
+#include <linux/gfp.h>
#include <linux/kernel.h>
#include <linux/skbuff.h>
#include <linux/timer.h>
#include <linux/dccp.h>
#include <linux/kernel.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <net/inet_sock.h>
#include <net/sock.h>
#include <linux/module.h>
#include <linux/kfifo.h>
#include <linux/vmalloc.h>
+#include <linux/gfp.h>
#include <net/net_namespace.h>
#include "dccp.h"
#include <linux/if_arp.h>
#include <linux/init.h>
#include <linux/random.h>
+#include <linux/slab.h>
#include <net/checksum.h>
#include <net/inet_sock.h>
#include <linux/skbuff.h>
#include <linux/sysctl.h>
#include <linux/notifier.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/system.h>
#include <net/net_namespace.h>
#include <linux/string.h>
#include <linux/net.h>
#include <linux/socket.h>
+#include <linux/slab.h>
#include <linux/sockios.h>
#include <linux/init.h>
#include <linux/skbuff.h>
#include <linux/module.h>
#include <linux/socket.h>
#include <linux/if_arp.h>
+#include <linux/slab.h>
#include <linux/if_ether.h>
#include <linux/init.h>
#include <linux/proc_fs.h>
#include <linux/netdevice.h>
#include <linux/inet.h>
#include <linux/route.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/tcp_states.h>
#include <asm/system.h>
#include <linux/netdevice.h>
#include <linux/inet.h>
#include <linux/route.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <asm/system.h>
#include <linux/fcntl.h>
#include <linux/inet.h>
#include <linux/route.h>
#include <linux/in_route.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <linux/mm.h>
#include <linux/proc_fs.h>
#include <linux/string.h>
#include <linux/net.h>
#include <linux/socket.h>
+#include <linux/slab.h>
#include <linux/sockios.h>
#include <linux/init.h>
#include <linux/skbuff.h>
*/
#include <linux/module.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/netdevice.h>
#include <linux/netfilter.h>
#include <linux/list.h>
#include <linux/netdevice.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <net/dsa.h>
#include "dsa_priv.h"
#include <linux/etherdevice.h>
#include <linux/list.h>
#include <linux/netdevice.h>
+#include <linux/slab.h>
#include "dsa_priv.h"
#define DSA_HLEN 4
#include <linux/etherdevice.h>
#include <linux/list.h>
#include <linux/netdevice.h>
+#include <linux/slab.h>
#include "dsa_priv.h"
#define DSA_HLEN 4
#include <linux/etherdevice.h>
#include <linux/list.h>
#include <linux/netdevice.h>
+#include <linux/slab.h>
#include "dsa_priv.h"
netdev_tx_t trailer_xmit(struct sk_buff *skb, struct net_device *dev)
#include <linux/wireless.h>
#include <linux/skbuff.h>
#include <linux/udp.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/inet_common.h>
#include <linux/stat.h>
#include <linux/module.h>
#include <linux/netdevice.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <net/datalink.h>
#include <linux/if.h>
#include <linux/termios.h> /* For TIOCOUTQ/INQ */
#include <linux/list.h>
+#include <linux/slab.h>
#include <net/datalink.h>
#include <net/psnap.h>
#include <net/sock.h>
{
struct sock *sk = sock->sk;
+ if (addr_len < sizeof(uaddr->sa_family))
+ return -EINVAL;
+
if (uaddr->sa_family == AF_UNSPEC)
return sk->sk_prot->disconnect(sk, flags);
#include <linux/module.h>
#include <linux/if_arp.h>
#include <linux/list.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/af_ieee802154.h>
#include <net/ieee802154.h>
*/
#include <linux/kernel.h>
+#include <linux/gfp.h>
#include <net/genetlink.h>
#include <linux/nl802154.h>
* Maxim Osipov <maxim.osipov@siemens.com>
*/
+#include <linux/gfp.h>
#include <linux/kernel.h>
#include <linux/if_arp.h>
#include <linux/netdevice.h>
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <net/netlink.h>
#include <net/genetlink.h>
#include <net/wpan-phy.h>
#include <linux/module.h>
#include <linux/if_arp.h>
#include <linux/list.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/af_ieee802154.h>
*
*/
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/device.h>
#include <linux/poll.h>
#include <linux/netfilter_ipv4.h>
#include <linux/random.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/system.h>
{
struct sock *sk = sock->sk;
+ if (addr_len < sizeof(uaddr->sa_family))
+ return -EINVAL;
if (uaddr->sa_family == AF_UNSPEC)
return sk->sk_prot->disconnect(sk, flags);
int err;
long timeo;
+ if (addr_len < sizeof(uaddr->sa_family))
+ return -EINVAL;
+
lock_sock(sk);
if (uaddr->sa_family == AF_UNSPEC) {
#include <crypto/hash.h>
#include <linux/err.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <net/ip.h>
#include <net/xfrm.h>
#include <net/ah.h>
#include <linux/net.h>
#include <linux/rcupdate.h>
#include <linux/jhash.h>
+#include <linux/slab.h>
#ifdef CONFIG_SYSCTL
#include <linux/sysctl.h>
#endif
#include <linux/string.h>
#include <linux/jhash.h>
#include <linux/audit.h>
+#include <linux/slab.h>
#include <net/ip.h>
#include <net/icmp.h>
#include <net/tcp.h>
#include <linux/notifier.h>
#include <linux/inetdevice.h>
#include <linux/igmp.h>
+#include <linux/slab.h>
#ifdef CONFIG_SYSCTL
#include <linux/sysctl.h>
#endif
hlist_for_each_entry_rcu(dev, node, head, index_hlist) {
if (idx < s_idx)
goto cont;
- if (idx > s_idx)
+ if (h > s_h || idx > s_idx)
s_ip_idx = 0;
in_dev = __in_dev_get_rcu(dev);
if (!in_dev)
#include <linux/skbuff.h>
#include <linux/init.h>
#include <linux/list.h>
+#include <linux/slab.h>
#include <net/ip.h>
#include <net/protocol.h>
#include <linux/skbuff.h>
#include <linux/netlink.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <net/net_namespace.h>
#include <net/ip.h>
#include <linux/proc_fs.h>
#include <linux/skbuff.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <net/arp.h>
#include <net/ip.h>
#include <linux/netlink.h>
#include <linux/init.h>
#include <linux/list.h>
+#include <linux/slab.h>
#include <net/net_namespace.h>
#include <net/ip.h>
#include <net/protocol.h>
struct node *n;
pos = 0;
- n = rcu_dereference(t->trie);
+ n = rcu_dereference_check(t->trie,
+ rcu_read_lock_held() ||
+ lockdep_rtnl_is_held());
while (n != NULL && NODE_TYPE(n) == T_TNODE) {
tn = (struct tnode *) n;
#include <linux/netdevice.h>
#include <linux/string.h>
#include <linux/netfilter_ipv4.h>
+#include <linux/slab.h>
#include <net/snmp.h>
#include <net/ip.h>
#include <net/route.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/system.h>
#include <linux/types.h>
#include <linux/types.h>
#include <linux/fcntl.h>
#include <linux/random.h>
+#include <linux/slab.h>
#include <linux/cache.h>
#include <linux/init.h>
#include <linux/time.h>
#include <linux/random.h>
#include <linux/skbuff.h>
#include <linux/rtnetlink.h>
+#include <linux/slab.h>
#include <net/inet_frag.h>
#include <linux/kernel.h>
#include <linux/kmemcheck.h>
+#include <linux/slab.h>
#include <net/inet_hashtables.h>
#include <net/inet_timewait_sock.h>
#include <net/ip.h>
#include <linux/ip.h>
#include <linux/icmp.h>
#include <linux/netdevice.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/ip.h>
#include <net/tcp.h>
#include <linux/netdevice.h>
#include <linux/jhash.h>
#include <linux/random.h>
+#include <linux/slab.h>
#include <net/route.h>
#include <net/dst.h>
#include <net/sock.h>
#include <linux/module.h>
#include <linux/types.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <linux/skbuff.h>
#include <linux/netdevice.h>
tunnel->err_count = 0;
}
- max_headroom = LL_RESERVED_SPACE(tdev) + gre_hlen;
+ max_headroom = LL_RESERVED_SPACE(tdev) + gre_hlen + rt->u.dst.header_len;
if (skb_headroom(skb) < max_headroom || skb_shared(skb)||
(skb_cloned(skb) && !skb_clone_writable(skb, 0))) {
struct sk_buff *new_skb = skb_realloc_headroom(skb, max_headroom);
+ if (max_headroom > dev->needed_headroom)
+ dev->needed_headroom = max_headroom;
if (!new_skb) {
ip_rt_put(rt);
txq->tx_dropped++;
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/errno.h>
+#include <linux/slab.h>
#include <linux/net.h>
#include <linux/socket.h>
#include <linux/capability.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <asm/uaccess.h>
#include <linux/skbuff.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/highmem.h>
+#include <linux/slab.h>
#include <linux/socket.h>
#include <linux/sockios.h>
#include <linux/icmp.h>
#include <linux/inetdevice.h>
#include <linux/netdevice.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/ip.h>
#include <net/icmp.h>
#include <linux/root_dev.h>
#include <linux/delay.h>
#include <linux/nfs_fs.h>
+#include <linux/slab.h>
#include <net/net_namespace.h>
#include <net/arp.h>
#include <net/ip.h>
#include <linux/module.h>
#include <linux/types.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <linux/skbuff.h>
#include <linux/netdevice.h>
#include <linux/mroute.h>
#include <linux/init.h>
#include <linux/if_ether.h>
+#include <linux/slab.h>
#include <net/net_namespace.h>
#include <net/ip.h>
#include <net/protocol.h>
int line;
struct mfc_cache *uc, *c, **cp;
+ if (mfc->mfcc_parent >= MAXVIFS)
+ return -ENFILE;
+
line = MFC_HASH(mfc->mfcc_mcastgrp.s_addr, mfc->mfcc_origin.s_addr);
for (cp = &net->ipv4.mfc_cache_array[line];
int ct;
struct rtnexthop *nhp;
struct net *net = mfc_net(c);
- struct net_device *dev = net->ipv4.vif_table[c->mfc_parent].dev;
u8 *b = skb_tail_pointer(skb);
struct rtattr *mp_head;
- if (dev)
- RTA_PUT(skb, RTA_IIF, 4, &dev->ifindex);
+ /* If cache is unresolved, don't try to parse IIF and OIF */
+ if (c->mfc_parent > MAXVIFS)
+ return -ENOENT;
+
+ if (VIF_EXISTS(net, c->mfc_parent))
+ RTA_PUT(skb, RTA_IIF, 4, &net->ipv4.vif_table[c->mfc_parent].dev->ifindex);
mp_head = (struct rtattr *)skb_put(skb, RTA_LENGTH(0));
for (ct = c->mfc_un.res.minvif; ct < c->mfc_un.res.maxvif; ct++) {
- if (c->mfc_un.res.ttls[ct] < 255) {
+ if (VIF_EXISTS(net, ct) && c->mfc_un.res.ttls[ct] < 255) {
if (skb_tailroom(skb) < RTA_ALIGN(RTA_ALIGN(sizeof(*nhp)) + 4))
goto rtattr_failure;
nhp = (struct rtnexthop *)skb_put(skb, RTA_ALIGN(sizeof(*nhp)));
#include <linux/netfilter_ipv4.h>
#include <linux/ip.h>
#include <linux/skbuff.h>
+#include <linux/gfp.h>
#include <net/route.h>
#include <net/xfrm.h>
#include <net/ip.h>
#include <linux/module.h>
#include <linux/netfilter/x_tables.h>
#include <linux/netfilter_arp/arp_tables.h>
+#include <linux/slab.h>
MODULE_LICENSE("GPL");
MODULE_AUTHOR("David S. Miller <davem@redhat.com>");
#include <linux/security.h>
#include <linux/net.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <net/net_namespace.h>
#include <net/sock.h>
#include <net/route.h>
#include <linux/jhash.h>
#include <linux/bitops.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <linux/ip.h>
#include <linux/tcp.h>
#include <linux/udp.h>
#include <linux/module.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <linux/ip.h>
#include <linux/udp.h>
#include <linux/icmp.h>
#include <linux/module.h>
#include <linux/spinlock.h>
#include <linux/socket.h>
+#include <linux/slab.h>
#include <linux/skbuff.h>
#include <linux/kernel.h>
#include <linux/timer.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/netfilter_ipv4/ip_tables.h>
+#include <linux/slab.h>
#include <net/ip.h>
MODULE_LICENSE("GPL");
#include <linux/netfilter_ipv4/ip_tables.h>
#include <linux/netdevice.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/route.h>
#include <linux/ip.h>
*/
#include <linux/module.h>
#include <linux/netfilter_ipv4/ip_tables.h>
+#include <linux/slab.h>
#include <net/ip.h>
#define RAW_VALID_HOOKS ((1 << NF_INET_PRE_ROUTING) | (1 << NF_INET_LOCAL_OUT))
*/
#include <linux/module.h>
#include <linux/netfilter_ipv4/ip_tables.h>
+#include <linux/slab.h>
#include <net/ip.h>
MODULE_LICENSE("GPL");
#include <linux/types.h>
#include <linux/timer.h>
#include <linux/skbuff.h>
+#include <linux/gfp.h>
#include <net/checksum.h>
#include <net/icmp.h>
#include <net/ip.h>
* published by the Free Software Foundation.
*/
#include <linux/module.h>
+#include <linux/gfp.h>
#include <linux/kmod.h>
#include <linux/types.h>
#include <linux/timer.h>
#include <linux/kmod.h>
#include <linux/skbuff.h>
#include <linux/proc_fs.h>
+#include <linux/slab.h>
#include <net/checksum.h>
#include <net/route.h>
#include <linux/bitops.h>
#include <linux/moduleparam.h>
#include <linux/types.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/in.h>
#include <linux/ip.h>
#include <linux/udp.h>
*/
#include <linux/types.h>
#include <linux/icmp.h>
+#include <linux/gfp.h>
#include <linux/ip.h>
#include <linux/netfilter.h>
#include <linux/netfilter_ipv4.h>
#include <net/net_namespace.h>
#include <net/dst.h>
#include <net/sock.h>
-#include <linux/gfp.h>
#include <linux/ip.h>
#include <linux/net.h>
#include <net/ip.h>
#include <linux/jhash.h>
#include <linux/rcupdate.h>
#include <linux/times.h>
+#include <linux/slab.h>
#include <net/dst.h>
#include <net/net_namespace.h>
#include <net/protocol.h>
}
static int rt_intern_hash(unsigned hash, struct rtable *rt,
- struct rtable **rp, struct sk_buff *skb)
+ struct rtable **rp, struct sk_buff *skb, int ifindex)
{
struct rtable *rth, **rthp;
unsigned long now;
slow_chain_length(rt_hash_table[hash].chain) > rt_chain_length_max) {
struct net *net = dev_net(rt->u.dst.dev);
int num = ++net->ipv4.current_rt_cache_rebuild_count;
- if (!rt_caching(dev_net(rt->u.dst.dev))) {
+ if (!rt_caching(net)) {
printk(KERN_WARNING "%s: %d rebuilds is over limit, route caching disabled\n",
rt->u.dst.dev->name, num);
}
- rt_emergency_hash_rebuild(dev_net(rt->u.dst.dev));
+ rt_emergency_hash_rebuild(net);
+ spin_unlock_bh(rt_hash_lock_addr(hash));
+
+ hash = rt_hash(rt->fl.fl4_dst, rt->fl.fl4_src,
+ ifindex, rt_genid(net));
+ goto restart;
}
}
dev_hold(rt->u.dst.dev);
if (rt->idev)
in_dev_hold(rt->idev);
- rt->u.dst.obsolete = 0;
+ rt->u.dst.obsolete = -1;
rt->u.dst.lastuse = jiffies;
rt->u.dst.path = &rt->u.dst;
rt->u.dst.neighbour = NULL;
&netevent);
rt_del(hash, rth);
- if (!rt_intern_hash(hash, rt, &rt, NULL))
+ if (!rt_intern_hash(hash, rt, &rt, NULL, rt->fl.oif))
ip_rt_put(rt);
goto do_next;
}
struct dst_entry *ret = dst;
if (rt) {
- if (dst->obsolete) {
+ if (dst->obsolete > 0) {
ip_rt_put(rt);
ret = NULL;
} else if ((rt->rt_flags & RTCF_REDIRECTED) ||
- rt->u.dst.expires) {
+ (rt->u.dst.expires &&
+ time_after_eq(jiffies, rt->u.dst.expires))) {
unsigned hash = rt_hash(rt->fl.fl4_dst, rt->fl.fl4_src,
rt->fl.oif,
rt_genid(dev_net(dst->dev)));
static struct dst_entry *ipv4_dst_check(struct dst_entry *dst, u32 cookie)
{
- return NULL;
+ if (rt_is_expired((struct rtable *)dst))
+ return NULL;
+ return dst;
}
static void ipv4_dst_destroy(struct dst_entry *dst)
if (!rth)
goto e_nobufs;
- rth->u.dst.output= ip_rt_bug;
+ rth->u.dst.output = ip_rt_bug;
+ rth->u.dst.obsolete = -1;
atomic_set(&rth->u.dst.__refcnt, 1);
rth->u.dst.flags= DST_HOST;
in_dev_put(in_dev);
hash = rt_hash(daddr, saddr, dev->ifindex, rt_genid(dev_net(dev)));
- return rt_intern_hash(hash, rth, NULL, skb);
+ return rt_intern_hash(hash, rth, NULL, skb, dev->ifindex);
e_nobufs:
in_dev_put(in_dev);
rth->fl.oif = 0;
rth->rt_spec_dst= spec_dst;
+ rth->u.dst.obsolete = -1;
rth->u.dst.input = ip_forward;
rth->u.dst.output = ip_output;
rth->rt_genid = rt_genid(dev_net(rth->u.dst.dev));
/* put it into the cache */
hash = rt_hash(daddr, saddr, fl->iif,
rt_genid(dev_net(rth->u.dst.dev)));
- return rt_intern_hash(hash, rth, NULL, skb);
+ return rt_intern_hash(hash, rth, NULL, skb, fl->iif);
}
/*
goto e_nobufs;
rth->u.dst.output= ip_rt_bug;
+ rth->u.dst.obsolete = -1;
rth->rt_genid = rt_genid(net);
atomic_set(&rth->u.dst.__refcnt, 1);
}
rth->rt_type = res.type;
hash = rt_hash(daddr, saddr, fl.iif, rt_genid(net));
- err = rt_intern_hash(hash, rth, NULL, skb);
+ err = rt_intern_hash(hash, rth, NULL, skb, fl.iif);
goto done;
no_route:
rth->rt_spec_dst= fl->fl4_src;
rth->u.dst.output=ip_output;
+ rth->u.dst.obsolete = -1;
rth->rt_genid = rt_genid(dev_net(dev_out));
RT_CACHE_STAT_INC(out_slow_tot);
if (err == 0) {
hash = rt_hash(oldflp->fl4_dst, oldflp->fl4_src, oldflp->oif,
rt_genid(dev_net(dev_out)));
- err = rt_intern_hash(hash, rth, rp, NULL);
+ err = rt_intern_hash(hash, rth, rp, NULL, oldflp->oif);
}
return err;
#include <linux/inetdevice.h>
#include <linux/seqlock.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <net/snmp.h>
#include <net/icmp.h>
#include <net/ip.h>
#include <linux/err.h>
#include <linux/crypto.h>
#include <linux/time.h>
+#include <linux/slab.h>
#include <net/icmp.h>
#include <net/tcp.h>
if (tp->urg_seq == tp->copied_seq &&
!sock_flag(sk, SOCK_URGINLINE) &&
tp->urg_data)
- target--;
+ target++;
/* Potential race condition. If read of tp below will
* escape above sk->sk_state, we can be illegally awaken
tp->ucopy.memory = 0;
}
+#ifdef CONFIG_NET_DMA
+static void tcp_service_net_dma(struct sock *sk, bool wait)
+{
+ dma_cookie_t done, used;
+ dma_cookie_t last_issued;
+ struct tcp_sock *tp = tcp_sk(sk);
+
+ if (!tp->ucopy.dma_chan)
+ return;
+
+ last_issued = tp->ucopy.dma_cookie;
+ dma_async_memcpy_issue_pending(tp->ucopy.dma_chan);
+
+ do {
+ if (dma_async_memcpy_complete(tp->ucopy.dma_chan,
+ last_issued, &done,
+ &used) == DMA_SUCCESS) {
+ /* Safe to free early-copied skbs now */
+ __skb_queue_purge(&sk->sk_async_wait_queue);
+ break;
+ } else {
+ struct sk_buff *skb;
+ while ((skb = skb_peek(&sk->sk_async_wait_queue)) &&
+ (dma_async_is_complete(skb->dma_cookie, done,
+ used) == DMA_SUCCESS)) {
+ __skb_dequeue(&sk->sk_async_wait_queue);
+ kfree_skb(skb);
+ }
+ }
+ } while (wait);
+}
+#endif
+
static inline struct sk_buff *tcp_recv_skb(struct sock *sk, u32 seq, u32 *off)
{
struct sk_buff *skb;
sk_eat_skb(sk, skb, 0);
if (!desc->count)
break;
+ tp->copied_seq = seq;
}
tp->copied_seq = seq;
/* __ Set realtime policy in scheduler __ */
}
+#ifdef CONFIG_NET_DMA
+ if (tp->ucopy.dma_chan)
+ dma_async_memcpy_issue_pending(tp->ucopy.dma_chan);
+#endif
if (copied >= target) {
/* Do not sleep, just process backlog. */
release_sock(sk);
sk_wait_data(sk, &timeo);
#ifdef CONFIG_NET_DMA
+ tcp_service_net_dma(sk, false); /* Don't block */
tp->ucopy.wakeup = 0;
#endif
copied = -EFAULT;
break;
}
+
+ dma_async_memcpy_issue_pending(tp->ucopy.dma_chan);
+
if ((offset + used) == skb->len)
copied_early = 1;
}
#ifdef CONFIG_NET_DMA
- if (tp->ucopy.dma_chan) {
- dma_cookie_t done, used;
-
- dma_async_memcpy_issue_pending(tp->ucopy.dma_chan);
-
- while (dma_async_memcpy_complete(tp->ucopy.dma_chan,
- tp->ucopy.dma_cookie, &done,
- &used) == DMA_IN_PROGRESS) {
- /* do partial cleanup of sk_async_wait_queue */
- while ((skb = skb_peek(&sk->sk_async_wait_queue)) &&
- (dma_async_is_complete(skb->dma_cookie, done,
- used) == DMA_SUCCESS)) {
- __skb_dequeue(&sk->sk_async_wait_queue);
- kfree_skb(skb);
- }
- }
+ tcp_service_net_dma(sk, true); /* Wait for queue to drain */
+ tp->ucopy.dma_chan = NULL;
- /* Safe to free early-copied skbs now */
- __skb_queue_purge(&sk->sk_async_wait_queue);
- tp->ucopy.dma_chan = NULL;
- }
if (tp->ucopy.pinned_list) {
dma_unpin_iovec_pages(tp->ucopy.pinned_list);
tp->ucopy.pinned_list = NULL;
#include <linux/mm.h>
#include <linux/types.h>
#include <linux/list.h>
+#include <linux/gfp.h>
#include <net/tcp.h>
int sysctl_tcp_max_ssthresh = 0;
*/
#include <linux/mm.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/sysctl.h>
#include <linux/kernel.h>
int err;
unsigned int mss;
+ if (packets == 0)
+ return;
+
WARN_ON(packets > tp->packets_out);
if (tp->lost_skb_hint) {
skb = tp->lost_skb_hint;
#include <linux/jhash.h>
#include <linux/init.h>
#include <linux/times.h>
+#include <linux/slab.h>
#include <net/net_namespace.h>
#include <net/icmp.h>
if (sk->sk_state == TCP_CLOSE)
goto out;
+ if (unlikely(iph->ttl < inet_sk(sk)->min_ttl)) {
+ NET_INC_STATS_BH(net, LINUX_MIB_TCPMINTTLDROP);
+ goto out;
+ }
+
icsk = inet_csk(sk);
tp = tcp_sk(sk);
seq = ntohl(th->seq);
#include <linux/mm.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/sysctl.h>
#include <linux/workqueue.h>
#include <net/tcp.h>
#include <net/tcp.h>
#include <linux/compiler.h>
+#include <linux/gfp.h>
#include <linux/module.h>
/* People can turn this off for buggy TCP's found in printers etc. */
#include <linux/kprobes.h>
#include <linux/socket.h>
#include <linux/tcp.h>
+#include <linux/slab.h>
#include <linux/proc_fs.h>
#include <linux/module.h>
#include <linux/ktime.h>
*/
#include <linux/module.h>
+#include <linux/gfp.h>
#include <net/tcp.h>
int sysctl_tcp_syn_retries __read_mostly = TCP_SYN_RETRIES;
#include <linux/mutex.h>
#include <linux/netdevice.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <net/icmp.h>
#include <net/ip.h>
#include <net/protocol.h>
#include <linux/mm.h>
#include <linux/inet.h>
#include <linux/netdevice.h>
+#include <linux/slab.h>
#include <net/tcp_states.h>
#include <linux/skbuff.h>
#include <linux/proc_fs.h>
*
*/
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/string.h>
#include <linux/netfilter.h>
* Copyright (c) 2004-2006 Herbert Xu <herbert@gondor.apana.org.au>
*/
+#include <linux/gfp.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/route.h>
#include <linux/inetdevice.h>
#include <linux/init.h>
+#include <linux/slab.h>
#ifdef CONFIG_SYSCTL
#include <linux/sysctl.h>
#endif
hlist_for_each_entry_rcu(dev, node, head, index_hlist) {
if (idx < s_idx)
goto cont;
- if (idx > s_idx)
+ if (h > s_h || idx > s_idx)
s_ip_idx = 0;
ip_idx = 0;
if ((idev = __in6_dev_get(dev)) == NULL)
#include <linux/list.h>
#include <linux/rcupdate.h>
#include <linux/in6.h>
+#include <linux/slab.h>
#include <net/addrconf.h>
#include <linux/if_addrlabel.h>
#include <linux/netlink.h>
#include <linux/proc_fs.h>
#include <linux/stat.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/inet.h>
#include <linux/netdevice.h>
#include <crypto/hash.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <net/ip.h>
#include <net/ah.h>
#include <linux/crypto.h>
#include <linux/init.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include <net/net_namespace.h>
#include <net/sock.h>
#include <linux/in6.h>
#include <linux/ipv6.h>
#include <linux/route.h>
+#include <linux/slab.h>
#include <net/ipv6.h>
#include <net/ndisc.h>
#include <linux/netdevice.h>
#include <linux/in6.h>
#include <linux/icmpv6.h>
+#include <linux/slab.h>
#include <net/dst.h>
#include <net/sock.h>
#include <linux/skbuff.h>
#include <linux/init.h>
#include <linux/netfilter.h>
+#include <linux/slab.h>
#ifdef CONFIG_SYSCTL
#include <linux/sysctl.h>
#include <linux/in6.h>
#include <linux/ipv6.h>
#include <linux/jhash.h>
+#include <linux/slab.h>
#include <net/addrconf.h>
#include <net/inet_connection_sock.h>
#include <linux/in6.h>
#include <linux/init.h>
#include <linux/list.h>
+#include <linux/slab.h>
#ifdef CONFIG_PROC_FS
#include <linux/proc_fs.h>
#include <linux/route.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include <net/net_namespace.h>
#include <net/sock.h>
#include <linux/in6.h>
#include <linux/icmpv6.h>
#include <linux/mroute6.h>
+#include <linux/slab.h>
#include <linux/netfilter.h>
#include <linux/netfilter_ipv6.h>
#include <linux/tcp.h>
#include <linux/route.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/netfilter.h>
#include <linux/netfilter_ipv6.h>
#include <linux/route.h>
#include <linux/rtnetlink.h>
#include <linux/netfilter_ipv6.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/atomic.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <net/protocol.h>
#include <linux/skbuff.h>
#include <net/sock.h>
unsigned char ttls[MAXMIFS];
int i;
+ if (mfc->mf6cc_parent >= MAXMIFS)
+ return -ENFILE;
+
memset(ttls, 255, MAXMIFS);
for (i = 0; i < MAXMIFS; i++) {
if (IF_ISSET(i, &mfc->mf6cc_ifset))
int ct;
struct rtnexthop *nhp;
struct net *net = mfc6_net(c);
- struct net_device *dev = net->ipv6.vif6_table[c->mf6c_parent].dev;
u8 *b = skb_tail_pointer(skb);
struct rtattr *mp_head;
- if (dev)
- RTA_PUT(skb, RTA_IIF, 4, &dev->ifindex);
+ /* If cache is unresolved, don't try to parse IIF and OIF */
+ if (c->mf6c_parent > MAXMIFS)
+ return -ENOENT;
+
+ if (MIF_EXISTS(net, c->mf6c_parent))
+ RTA_PUT(skb, RTA_IIF, 4, &net->ipv6.vif6_table[c->mf6c_parent].dev->ifindex);
mp_head = (struct rtattr *)skb_put(skb, RTA_LENGTH(0));
for (ct = c->mfc_un.res.minvif; ct < c->mfc_un.res.maxvif; ct++) {
- if (c->mfc_un.res.ttls[ct] < 255) {
+ if (MIF_EXISTS(net, ct) && c->mfc_un.res.ttls[ct] < 255) {
if (skb_tailroom(skb) < RTA_ALIGN(RTA_ALIGN(sizeof(*nhp)) + 4))
goto rtattr_failure;
nhp = (struct rtnexthop *)skb_put(skb, RTA_ALIGN(sizeof(*nhp)));
#include <linux/init.h>
#include <linux/sysctl.h>
#include <linux/netfilter.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/snmp.h>
#include <linux/init.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include <linux/netfilter.h>
#include <linux/netfilter_ipv6.h>
#include <linux/route.h>
#include <linux/init.h>
#include <linux/rcupdate.h>
+#include <linux/slab.h>
#ifdef CONFIG_SYSCTL
#include <linux/sysctl.h>
#endif
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <net/net_namespace.h>
#include <net/sock.h>
#include <net/ipv6.h>
* 2 of the License, or (at your option) any later version.
*/
+#include <linux/gfp.h>
#include <linux/module.h>
#include <linux/skbuff.h>
#include <linux/icmpv6.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/netfilter_ipv6/ip6_tables.h>
+#include <linux/slab.h>
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Netfilter Core Team <coreteam@netfilter.org>");
*/
#include <linux/module.h>
#include <linux/netfilter_ipv6/ip6_tables.h>
+#include <linux/slab.h>
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Netfilter Core Team <coreteam@netfilter.org>");
*/
#include <linux/module.h>
#include <linux/netfilter_ipv6/ip6_tables.h>
+#include <linux/slab.h>
#define RAW_VALID_HOOKS ((1 << NF_INET_PRE_ROUTING) | (1 << NF_INET_LOCAL_OUT))
.valid_hooks = RAW_VALID_HOOKS,
.me = THIS_MODULE,
.af = NFPROTO_IPV6,
- .priority = NF_IP6_PRI_FIRST,
+ .priority = NF_IP6_PRI_RAW,
};
/* The work comes in here from netfilter.c. */
*/
#include <linux/module.h>
#include <linux/netfilter_ipv6/ip6_tables.h>
+#include <linux/slab.h>
MODULE_LICENSE("GPL");
MODULE_AUTHOR("James Morris <jmorris <at> redhat.com>");
#include <linux/ipv6.h>
#include <linux/icmpv6.h>
#include <linux/random.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/snmp.h>
#include <linux/errno.h>
#include <linux/types.h>
#include <linux/socket.h>
+#include <linux/slab.h>
#include <linux/sockios.h>
#include <linux/net.h>
#include <linux/in6.h>
#include <linux/random.h>
#include <linux/jhash.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/snmp.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
#include <linux/nsproxy.h>
+#include <linux/slab.h>
#include <net/net_namespace.h>
#include <net/snmp.h>
#include <net/ipv6.h>
rt = (struct rt6_info *) dst;
- if (rt && rt->rt6i_node && (rt->rt6i_node->fn_sernum == cookie))
+ if (rt->rt6i_node && (rt->rt6i_node->fn_sernum == cookie))
return dst;
return NULL;
struct rt6_info *rt = (struct rt6_info *) dst;
if (rt) {
- if (rt->rt6i_flags & RTF_CACHE)
- ip6_del_rt(rt);
- else
+ if (rt->rt6i_flags & RTF_CACHE) {
+ if (rt6_check_expired(rt)) {
+ ip6_del_rt(rt);
+ dst = NULL;
+ }
+ } else {
dst_release(dst);
+ dst = NULL;
+ }
}
- return NULL;
+ return dst;
}
static void ip6_link_failure(struct sk_buff *skb)
#include <linux/netdevice.h>
#include <linux/if_arp.h>
#include <linux/icmp.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <linux/init.h>
#include <linux/netfilter_ipv4.h>
#include <linux/sysctl.h>
#include <linux/in6.h>
#include <linux/ipv6.h>
+#include <linux/slab.h>
#include <net/ndisc.h>
#include <net/ipv6.h>
#include <net/addrconf.h>
#include <linux/jhash.h>
#include <linux/ipsec.h>
#include <linux/times.h>
+#include <linux/slab.h>
#include <linux/ipv6.h>
#include <linux/icmpv6.h>
#include <linux/mutex.h>
#include <linux/netdevice.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <net/ipv6.h>
#include <net/protocol.h>
#include <net/xfrm.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <net/ndisc.h>
* Copyright (c) 2004-2006 Herbert Xu <herbert@gondor.apana.org.au>
*/
+#include <linux/gfp.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/module.h>
*/
#include <linux/module.h>
#include <linux/xfrm.h>
+#include <linux/slab.h>
#include <linux/rculist.h>
#include <net/ip.h>
#include <net/xfrm.h>
#include <linux/net.h>
#include <linux/netdevice.h>
#include <linux/uio.h>
+#include <linux/slab.h>
#include <linux/skbuff.h>
#include <linux/smp_lock.h>
#include <linux/socket.h>
#include <linux/list.h>
#include <linux/route.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <net/ipx.h>
#include <linux/smp_lock.h>
#include <linux/socket.h>
#include <linux/sockios.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/net.h>
#include <linux/irda.h>
#include <linux/socket.h>
#include <linux/fs.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include <net/irda/irda.h>
#include <net/irda/irlmp.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <net/irda/irda.h>
#include <net/irda/irmod.h>
********************************************************************/
#include <linux/init.h>
+#include <linux/gfp.h>
#include <net/irda/irda.h>
#include <net/irda/irlmp.h>
*
********************************************************************/
+#include <linux/gfp.h>
#include <linux/workqueue.h>
#include <linux/interrupt.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/fs.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#include <linux/seq_file.h>
#include <linux/termios.h>
#include <linux/tty.h>
#include <linux/kmod.h>
#include <linux/spinlock.h>
+#include <linux/slab.h>
#include <asm/ioctls.h>
#include <asm/uaccess.h>
#include <linux/string.h>
#include <linux/init.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include <asm/byteorder.h>
#include <asm/unaligned.h>
*
********************************************************************/
+#include <linux/slab.h>
+
#include <net/irda/irda.h>
#include <net/irda/irlmp.h>
#include <net/irda/iriap.h>
*
********************************************************************/
+#include <linux/slab.h>
#include <linux/string.h>
#include <linux/socket.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/string.h>
+#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/init.h>
#include <linux/netdevice.h>
#include <linux/kernel.h>
#include <linux/string.h>
+#include <linux/gfp.h>
#include <linux/init.h>
#include <linux/errno.h>
#include <linux/proc_fs.h>
#include <linux/init.h>
#include <linux/random.h>
#include <linux/bitops.h>
+#include <linux/slab.h>
#include <asm/system.h>
#include <asm/byteorder.h>
#include <linux/kernel.h>
#include <linux/delay.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <net/irda/irda.h>
#include <net/irda/irlap_event.h>
#include <linux/if_ether.h>
#include <linux/netdevice.h>
#include <linux/irda.h>
+#include <linux/slab.h>
#include <net/pkt_sched.h>
#include <net/sock.h>
#include "irnet_irda.h" /* Private header */
#include <linux/sched.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include <asm/unaligned.h>
/*
*/
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/smp_lock.h>
#include "irnet_ppp.h" /* Private header */
/* Please put other headers in irnet.h - Thanks */
#include <linux/socket.h>
#include <linux/irda.h>
+#include <linux/gfp.h>
#include <net/net_namespace.h>
#include <net/sock.h>
#include <net/irda/irda.h>
* Jean II
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <net/irda/irda.h>
#include <net/irda/irqueue.h>
#include <linux/init.h>
#include <linux/fs.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include <asm/byteorder.h>
#include <asm/unaligned.h>
#include <linux/in6.h>
#include <linux/proc_fs.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <net/net_namespace.h>
#include <net/netns/generic.h>
#include <net/xfrm.h>
int err;
out_skb = pfkey_xfrm_policy2msg_prep(xp);
- if (IS_ERR(out_skb)) {
- err = PTR_ERR(out_skb);
- goto out;
- }
+ if (IS_ERR(out_skb))
+ return PTR_ERR(out_skb);
+
err = pfkey_xfrm_policy2msg(out_skb, xp, dir);
if (err < 0)
return err;
out_hdr->sadb_msg_seq = c->seq;
out_hdr->sadb_msg_pid = c->pid;
pfkey_broadcast(out_skb, GFP_ATOMIC, BROADCAST_ALL, NULL, xp_net(xp));
-out:
return 0;
}
#include <linux/inet.h>
#include <linux/if_arp.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <asm/uaccess.h>
#include <asm/system.h>
#include <linux/inet.h>
#include <linux/netdevice.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <asm/uaccess.h>
#include <asm/system.h>
#include <linux/net.h>
#include <linux/inet.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <asm/uaccess.h>
#include <asm/system.h>
#include <linux/net.h>
#include <linux/inet.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <asm/uaccess.h>
#include <asm/system.h>
#include <linux/module.h>
#include <linux/rtnetlink.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <net/llc.h>
#include <net/llc_sap.h>
#include <net/llc_pdu.h>
* See the GNU General Public License for more details.
*/
#include <linux/netdevice.h>
+#include <linux/slab.h>
#include <net/llc_conn.h>
#include <net/llc_sap.h>
#include <net/sock.h>
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include <net/llc_sap.h>
#include <net/llc_conn.h>
#include <net/sock.h>
*
* See the GNU General Public License for more details.
*/
+#include <linux/gfp.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/netdevice.h>
* See the GNU General Public License for more details.
*/
#include <linux/netdevice.h>
+#include <linux/slab.h>
#include <net/net_namespace.h>
#include <net/llc.h>
#include <net/llc_pdu.h>
#include <net/sock.h>
#include <net/tcp_states.h>
#include <linux/llc.h>
+#include <linux/slab.h>
static int llc_mac_header_len(unsigned short devtype)
{
*/
#include <linux/init.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <net/llc.h>
#include <net/llc_sap.h>
#include <net/llc_conn.h>
*/
#include <linux/ieee80211.h>
+#include <linux/slab.h>
#include <net/mac80211.h>
#include "ieee80211_i.h"
#include "driver-ops.h"
*/
#include <linux/ieee80211.h>
+#include <linux/slab.h>
#include <net/mac80211.h>
#include "ieee80211_i.h"
#include "driver-ops.h"
#include <linux/ieee80211.h>
#include <linux/nl80211.h>
#include <linux/rtnetlink.h>
+#include <linux/slab.h>
#include <net/net_namespace.h>
#include <linux/rcupdate.h>
#include <net/cfg80211.h>
*/
#include <linux/kobject.h>
+#include <linux/slab.h>
#include "ieee80211_i.h"
#include "key.h"
#include "debugfs.h"
#include <linux/interrupt.h>
#include <linux/netdevice.h>
#include <linux/rtnetlink.h>
+#include <linux/slab.h>
#include <linux/notifier.h>
#include <net/mac80211.h>
#include <net/cfg80211.h>
*/
#include <linux/delay.h>
+#include <linux/slab.h>
#include <linux/if_ether.h>
#include <linux/skbuff.h>
#include <linux/if_arp.h>
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
*/
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/if_arp.h>
#include <linux/netdevice.h>
#include <linux/list.h>
#include <linux/rcupdate.h>
#include <linux/rtnetlink.h>
+#include <linux/slab.h>
#include <net/mac80211.h>
#include "ieee80211_i.h"
#include "driver-ops.h"
/* just for IFNAMSIZ */
#include <linux/if.h>
+#include <linux/slab.h>
#include "led.h"
void ieee80211_led_rx(struct ieee80211_local *local)
* published by the Free Software Foundation.
*/
+#include <linux/slab.h>
#include <asm/unaligned.h>
#include "ieee80211_i.h"
#include "mesh.h"
* published by the Free Software Foundation.
*/
+#include <linux/slab.h>
#include "mesh.h"
#ifdef CONFIG_MAC80211_VERBOSE_MHWMP_DEBUG
if (SN_GT(mpath->sn, orig_sn) ||
(mpath->sn == orig_sn &&
action == MPATH_PREQ &&
- new_metric > mpath->metric)) {
+ new_metric >= mpath->metric)) {
process = false;
fresh_info = false;
}
mesh_path_sel_frame_tx(MPATH_PREP, flags, orig_addr,
cpu_to_le32(orig_sn), 0, target_addr,
- cpu_to_le32(target_sn), mpath->next_hop->sta.addr, hopcount,
+ cpu_to_le32(target_sn), next_hop, hopcount,
ttl, cpu_to_le32(lifetime), cpu_to_le32(metric),
0, sdata);
rcu_read_unlock();
#include <linux/etherdevice.h>
#include <linux/list.h>
#include <linux/random.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/string.h>
#include <net/mac80211.h>
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
*/
+#include <linux/gfp.h>
#include <linux/kernel.h>
#include <linux/random.h>
#include "ieee80211_i.h"
#include <linux/rtnetlink.h>
#include <linux/pm_qos_params.h>
#include <linux/crc32.h>
+#include <linux/slab.h>
#include <net/mac80211.h>
#include <asm/unaligned.h>
#include <linux/kernel.h>
#include <linux/rtnetlink.h>
+#include <linux/slab.h>
#include "rate.h"
#include "ieee80211_i.h"
#include "debugfs.h"
#include <linux/debugfs.h>
#include <linux/random.h>
#include <linux/ieee80211.h>
+#include <linux/slab.h>
#include <net/mac80211.h>
#include "rate.h"
#include "rc80211_minstrel.h"
#include <linux/skbuff.h>
#include <linux/debugfs.h>
#include <linux/ieee80211.h>
+#include <linux/slab.h>
#include <net/mac80211.h>
#include "rc80211_minstrel.h"
#include <linux/types.h>
#include <linux/skbuff.h>
#include <linux/debugfs.h>
+#include <linux/slab.h>
#include <net/mac80211.h>
#include "rate.h"
#include "mesh.h"
#include <linux/netdevice.h>
#include <linux/types.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <net/mac80211.h>
#include "rate.h"
*/
#include <linux/jiffies.h>
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/skbuff.h>
#include <linux/netdevice.h>
#include <linux/if_arp.h>
#include <linux/rtnetlink.h>
+#include <linux/slab.h>
#include <net/mac80211.h>
#include "ieee80211_i.h"
void ieee80211_tx_pending(unsigned long data)
{
struct ieee80211_local *local = (struct ieee80211_local *)data;
+ struct ieee80211_sub_if_data *sdata;
unsigned long flags;
int i;
bool txok;
if (!txok)
break;
}
+
+ if (skb_queue_empty(&local->pending[i]))
+ list_for_each_entry_rcu(sdata, &local->interfaces, list)
+ netif_tx_wake_queue(
+ netdev_get_tx_queue(sdata->dev, i));
}
spin_unlock_irqrestore(&local->queue_stop_reason_lock, flags);
/* someone still has this queue stopped */
return;
- if (!skb_queue_empty(&local->pending[queue]))
+ if (skb_queue_empty(&local->pending[queue])) {
+ rcu_read_lock();
+ list_for_each_entry_rcu(sdata, &local->interfaces, list)
+ netif_tx_wake_queue(netdev_get_tx_queue(sdata->dev, queue));
+ rcu_read_unlock();
+ } else
tasklet_schedule(&local->tx_pending_tasklet);
-
- rcu_read_lock();
- list_for_each_entry_rcu(sdata, &local->interfaces, list)
- netif_tx_wake_queue(netdev_get_tx_queue(sdata->dev, queue));
- rcu_read_unlock();
}
void ieee80211_wake_queue_by_reason(struct ieee80211_hw *hw, int queue,
*/
res = drv_start(local);
if (res) {
- WARN(local->suspended, "Harware became unavailable "
- "upon resume. This is could be a software issue"
- "prior to suspend or a hardware issue\n");
+ WARN(local->suspended, "Hardware became unavailable "
+ "upon resume. This could be a software issue "
+ "prior to suspend or a hardware issue.\n");
return res;
}
#include <linux/err.h>
#include <linux/mm.h>
#include <linux/scatterlist.h>
+#include <linux/slab.h>
#include <asm/unaligned.h>
#include <net/mac80211.h>
#include <linux/if_arp.h>
#include <linux/etherdevice.h>
#include <linux/crc32.h>
+#include <linux/slab.h>
#include <net/mac80211.h>
#include <asm/unaligned.h>
#include <linux/netdevice.h>
#include <linux/types.h>
-#include <linux/slab.h>
#include <linux/skbuff.h>
#include <linux/compiler.h>
#include <linux/ieee80211.h>
+#include <linux/gfp.h>
#include <asm/unaligned.h>
#include <net/mac80211.h>
#include <linux/inetdevice.h>
#include <linux/proc_fs.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <net/net_namespace.h>
#include <net/sock.h>
#include <linux/in.h>
#include <linux/ip.h>
#include <linux/netfilter.h>
+#include <linux/slab.h>
#include <net/net_namespace.h>
#include <net/protocol.h>
#include <net/tcp.h>
#include <linux/module.h>
#include <linux/vmalloc.h>
#include <linux/proc_fs.h> /* for proc_net_* */
+#include <linux/slab.h>
#include <linux/seq_file.h>
#include <linux/jhash.h>
#include <linux/random.h>
#include <linux/tcp.h>
#include <linux/sctp.h>
#include <linux/icmp.h>
+#include <linux/slab.h>
#include <net/ip.h>
#include <net/tcp.h>
#include <linux/workqueue.h>
#include <linux/swap.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include <linux/netfilter.h>
#include <linux/netfilter_ipv4.h>
#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
#include <linux/ip.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/skbuff.h>
#include <linux/kernel.h>
#include <linux/jiffies.h>
-#include <linux/slab.h>
#include <linux/types.h>
#include <linux/interrupt.h>
#include <linux/sysctl.h>
#include <linux/in.h>
#include <linux/ip.h>
#include <linux/netfilter.h>
+#include <linux/gfp.h>
#include <net/protocol.h>
#include <net/tcp.h>
#include <asm/unaligned.h>
#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
#include <linux/ip.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/skbuff.h>
#include <linux/skbuff.h>
#include <linux/jiffies.h>
#include <linux/list.h>
+#include <linux/slab.h>
/* for sysctl */
#include <linux/fs.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/skbuff.h>
+#include <linux/gfp.h>
#include <linux/in.h>
#include <linux/ip.h>
#include <net/protocol.h>
#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
#include <linux/ip.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/skbuff.h>
#include <linux/module.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/net.h>
#include <linux/gcd.h>
#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/tcp.h> /* for tcphdr */
#include <net/ip.h>
#include <net/tcp.h> /* for csum_tcpudp_magic */
*/
#include <linux/netfilter.h>
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/moduleparam.h>
#include <linux/in.h>
#include <linux/udp.h>
#include <linux/netfilter.h>
+#include <linux/gfp.h>
#include <net/netfilter/nf_conntrack.h>
#include <net/netfilter/nf_conntrack_expect.h>
#include <linux/percpu.h>
#include <linux/kernel.h>
#include <linux/netdevice.h>
+#include <linux/slab.h>
#include <net/netfilter/nf_conntrack.h>
#include <net/netfilter/nf_conntrack_core.h>
#include <linux/moduleparam.h>
#include <linux/netfilter.h>
#include <linux/ip.h>
+#include <linux/slab.h>
#include <linux/ipv6.h>
#include <linux/ctype.h>
#include <linux/inet.h>
#include <linux/inet.h>
#include <linux/in.h>
#include <linux/ip.h>
+#include <linux/slab.h>
#include <linux/udp.h>
#include <linux/tcp.h>
#include <linux/skbuff.h>
#include <linux/skbuff.h>
#include <linux/vmalloc.h>
#include <linux/stddef.h>
-#include <linux/slab.h>
#include <linux/random.h>
#include <linux/err.h>
#include <linux/kernel.h>
#include <linux/ip.h>
#include <linux/tcp.h>
#include <linux/netfilter.h>
+#include <linux/slab.h>
#include <net/netfilter/nf_conntrack.h>
#include <net/netfilter/nf_conntrack_expect.h>
#include <linux/netlink.h>
#include <linux/spinlock.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <linux/netfilter.h>
#include <net/netlink.h>
nlmsg_failure:
kfree_skb(skb);
errout:
- nfnetlink_set_err(net, 0, group, -ENOBUFS);
+ if (nfnetlink_set_err(net, 0, group, -ENOBUFS) > 0)
+ return -ENOBUFS;
+
return 0;
}
#endif /* CONFIG_NF_CONNTRACK_EVENTS */
#include <linux/types.h>
#include <linux/netfilter.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/mutex.h>
#include <linux/skbuff.h>
#include <linux/vmalloc.h>
#include <linux/spinlock.h>
#include <linux/skbuff.h>
#include <linux/dccp.h>
+#include <linux/slab.h>
#include <net/net_namespace.h>
#include <net/netns/generic.h>
#include <linux/in.h>
#include <linux/netdevice.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <net/dst.h>
#include <net/net_namespace.h>
#include <net/netns/generic.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/netfilter.h>
+#include <linux/slab.h>
#include <linux/in.h>
#include <linux/tcp.h>
#include <net/netfilter/nf_conntrack.h>
#include <linux/types.h>
#include <linux/netfilter.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/skbuff.h>
#include <linux/proc_fs.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/proc_fs.h>
}
EXPORT_SYMBOL_GPL(nfnetlink_send);
-void nfnetlink_set_err(struct net *net, u32 pid, u32 group, int error)
+int nfnetlink_set_err(struct net *net, u32 pid, u32 group, int error)
{
- netlink_set_err(net->nfnl, pid, group, error);
+ return netlink_set_err(net->nfnl, pid, group, error);
}
EXPORT_SYMBOL_GPL(nfnetlink_set_err);
#include <linux/list.h>
#include <linux/jhash.h>
#include <linux/random.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/netfilter/nf_log.h>
#include <net/netfilter/nfnetlink_log.h>
#include <linux/skbuff.h>
#include <linux/init.h>
#include <linux/spinlock.h>
+#include <linux/slab.h>
#include <linux/notifier.h>
#include <linux/netdevice.h>
#include <linux/netfilter.h>
#include <linux/vmalloc.h>
#include <linux/mutex.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include <net/net_namespace.h>
#include <linux/netfilter/x_tables.h>
*/
#include <linux/module.h>
+#include <linux/gfp.h>
#include <linux/skbuff.h>
#include <linux/selinux.h>
#include <linux/netfilter_ipv4/ip_tables.h>
#include <linux/module.h>
#include <linux/skbuff.h>
#include <linux/netfilter/x_tables.h>
+#include <linux/slab.h>
#include <linux/leds.h>
#include <linux/mutex.h>
#include <linux/jhash.h>
#include <linux/rtnetlink.h>
#include <linux/random.h>
+#include <linux/slab.h>
#include <net/gen_stats.h>
#include <net/netlink.h>
#include <linux/module.h>
#include <linux/skbuff.h>
#include <linux/ip.h>
+#include <linux/gfp.h>
#include <linux/ipv6.h>
#include <linux/tcp.h>
#include <net/dst.h>
#include <linux/ip.h>
#include <linux/ipv6.h>
#include <linux/jhash.h>
+#include <linux/slab.h>
#include <linux/list.h>
#include <linux/module.h>
#include <linux/random.h>
#include <linux/module.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <net/ip.h>
#include <linux/dccp.h>
case 64 ... 95:
i[2] = maskl(i[2], p - 64);
i[3] = 0;
+ break;
case 96 ... 127:
i[3] = maskl(i[3], p - 96);
break;
struct xt_hashlimit_htable *htable = s->private;
unsigned int *bucket = (unsigned int *)v;
- kfree(bucket);
+ if (!IS_ERR(bucket))
+ kfree(bucket);
spin_unlock_bh(&htable->lock);
}
* published by the Free Software Foundation.
*/
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/skbuff.h>
#include <linux/spinlock.h>
* Sam Johnston <samj@samj.net>
*/
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/netfilter/x_tables.h>
#include <linux/bitops.h>
#include <linux/skbuff.h>
#include <linux/inet.h>
+#include <linux/slab.h>
#include <net/net_namespace.h>
#include <net/netns/generic.h>
for (i = 0; i < e->nstamps; i++) {
if (info->seconds && time_after(time, e->stamps[i]))
continue;
- if (info->hit_count && ++hits >= info->hit_count) {
+ if (!info->hit_count || ++hits >= info->hit_count) {
ret = !ret;
break;
}
#include <linux/spinlock.h>
#include <linux/skbuff.h>
#include <linux/net.h>
+#include <linux/slab.h>
#include <linux/netfilter/xt_statistic.h>
#include <linux/netfilter/x_tables.h>
* published by the Free Software Foundation.
*/
+#include <linux/gfp.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/skbuff.h>
#include <linux/audit.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/netlink.h>
#include <net/genetlink.h>
#include <linux/spinlock.h>
#include <linux/string.h>
#include <linux/audit.h>
+#include <linux/slab.h>
#include <net/netlabel.h>
#include <net/cipso_ipv4.h>
#include <asm/bug.h>
};
/* Domain hash table */
-/* XXX - updates should be so rare that having one spinlock for the entire
- * hash table should be okay */
+/* updates should be so rare that having one spinlock for the entire hash table
+ * should be okay */
static DEFINE_SPINLOCK(netlbl_domhsh_lock);
+#define netlbl_domhsh_rcu_deref(p) \
+ rcu_dereference_check(p, rcu_read_lock_held() || \
+ lockdep_is_held(&netlbl_domhsh_lock))
static struct netlbl_domhsh_tbl *netlbl_domhsh = NULL;
static struct netlbl_dom_map *netlbl_domhsh_def = NULL;
* Description:
* This is the hashing function for the domain hash table, it returns the
* correct bucket number for the domain. The caller is responsibile for
- * calling the rcu_read_[un]lock() functions.
+ * ensuring that the hash table is protected with either a RCU read lock or the
+ * hash table lock.
*
*/
static u32 netlbl_domhsh_hash(const char *key)
for (iter = 0, val = 0, len = strlen(key); iter < len; iter++)
val = (val << 4 | (val >> (8 * sizeof(u32) - 4))) ^ key[iter];
- return val & (rcu_dereference(netlbl_domhsh)->size - 1);
+ return val & (netlbl_domhsh_rcu_deref(netlbl_domhsh)->size - 1);
}
/**
* Description:
* Searches the domain hash table and returns a pointer to the hash table
* entry if found, otherwise NULL is returned. The caller is responsibile for
- * the rcu hash table locks (i.e. the caller much call rcu_read_[un]lock()).
+ * ensuring that the hash table is protected with either a RCU read lock or the
+ * hash table lock.
*
*/
static struct netlbl_dom_map *netlbl_domhsh_search(const char *domain)
if (domain != NULL) {
bkt = netlbl_domhsh_hash(domain);
- bkt_list = &rcu_dereference(netlbl_domhsh)->tbl[bkt];
+ bkt_list = &netlbl_domhsh_rcu_deref(netlbl_domhsh)->tbl[bkt];
list_for_each_entry_rcu(iter, bkt_list, list)
if (iter->valid && strcmp(iter->domain, domain) == 0)
return iter;
* Searches the domain hash table and returns a pointer to the hash table
* entry if an exact match is found, if an exact match is not present in the
* hash table then the default entry is returned if valid otherwise NULL is
- * returned. The caller is responsibile for the rcu hash table locks
- * (i.e. the caller much call rcu_read_[un]lock()).
+ * returned. The caller is responsibile ensuring that the hash table is
+ * protected with either a RCU read lock or the hash table lock.
*
*/
static struct netlbl_dom_map *netlbl_domhsh_search_def(const char *domain)
entry = netlbl_domhsh_search(domain);
if (entry == NULL) {
- entry = rcu_dereference(netlbl_domhsh_def);
+ entry = netlbl_domhsh_rcu_deref(netlbl_domhsh_def);
if (entry != NULL && !entry->valid)
entry = NULL;
}
struct netlbl_af6list *tmp6;
#endif /* IPv6 */
+ /* XXX - we can remove this RCU read lock as the spinlock protects the
+ * entire function, but before we do we need to fixup the
+ * netlbl_af[4,6]list RCU functions to do "the right thing" with
+ * respect to rcu_dereference() when only a spinlock is held. */
rcu_read_lock();
-
spin_lock(&netlbl_domhsh_lock);
if (entry->domain != NULL)
entry_old = netlbl_domhsh_search(entry->domain);
#include <linux/init.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/audit.h>
#include <linux/in.h>
#include <linux/in6.h>
#include <linux/skbuff.h>
#include <linux/in.h>
#include <linux/in6.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/netlink.h>
#include <net/genetlink.h>
#include <linux/notifier.h>
#include <linux/netdevice.h>
#include <linux/security.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/netlink.h>
#include <net/genetlink.h>
/* updates should be so rare that having one spinlock for the entire
* hash table should be okay */
static DEFINE_SPINLOCK(netlbl_unlhsh_lock);
+#define netlbl_unlhsh_rcu_deref(p) \
+ rcu_dereference_check(p, rcu_read_lock_held() || \
+ lockdep_is_held(&netlbl_unlhsh_lock))
static struct netlbl_unlhsh_tbl *netlbl_unlhsh = NULL;
static struct netlbl_unlhsh_iface *netlbl_unlhsh_def = NULL;
* Description:
* This is the hashing function for the unlabeled hash table, it returns the
* bucket number for the given device/interface. The caller is responsible for
- * calling the rcu_read_[un]lock() functions.
+ * ensuring that the hash table is protected with either a RCU read lock or
+ * the hash table lock.
*
*/
static u32 netlbl_unlhsh_hash(int ifindex)
{
- /* this is taken _almost_ directly from
- * security/selinux/netif.c:sel_netif_hasfn() as they do pretty much
- * the same thing */
- return ifindex & (rcu_dereference(netlbl_unlhsh)->size - 1);
+ return ifindex & (netlbl_unlhsh_rcu_deref(netlbl_unlhsh)->size - 1);
}
/**
* Description:
* Searches the unlabeled connection hash table and returns a pointer to the
* interface entry which matches @ifindex, otherwise NULL is returned. The
- * caller is responsible for calling the rcu_read_[un]lock() functions.
+ * caller is responsible for ensuring that the hash table is protected with
+ * either a RCU read lock or the hash table lock.
*
*/
static struct netlbl_unlhsh_iface *netlbl_unlhsh_search_iface(int ifindex)
struct netlbl_unlhsh_iface *iter;
bkt = netlbl_unlhsh_hash(ifindex);
- bkt_list = &rcu_dereference(netlbl_unlhsh)->tbl[bkt];
+ bkt_list = &netlbl_unlhsh_rcu_deref(netlbl_unlhsh)->tbl[bkt];
list_for_each_entry_rcu(iter, bkt_list, list)
if (iter->valid && iter->ifindex == ifindex)
return iter;
return NULL;
}
-/**
- * netlbl_unlhsh_search_iface_def - Search for a matching interface entry
- * @ifindex: the network interface
- *
- * Description:
- * Searches the unlabeled connection hash table and returns a pointer to the
- * interface entry which matches @ifindex. If an exact match can not be found
- * and there is a valid default entry, the default entry is returned, otherwise
- * NULL is returned. The caller is responsible for calling the
- * rcu_read_[un]lock() functions.
- *
- */
-static struct netlbl_unlhsh_iface *netlbl_unlhsh_search_iface_def(int ifindex)
-{
- struct netlbl_unlhsh_iface *entry;
-
- entry = netlbl_unlhsh_search_iface(ifindex);
- if (entry != NULL)
- return entry;
-
- entry = rcu_dereference(netlbl_unlhsh_def);
- if (entry != NULL && entry->valid)
- return entry;
-
- return NULL;
-}
-
/**
* netlbl_unlhsh_add_addr4 - Add a new IPv4 address entry to the hash table
* @iface: the associated interface entry
* Description:
* Add a new address entry into the unlabeled connection hash table using the
* interface entry specified by @iface. On success zero is returned, otherwise
- * a negative value is returned. The caller is responsible for calling the
- * rcu_read_[un]lock() functions.
+ * a negative value is returned.
*
*/
static int netlbl_unlhsh_add_addr4(struct netlbl_unlhsh_iface *iface,
* Description:
* Add a new address entry into the unlabeled connection hash table using the
* interface entry specified by @iface. On success zero is returned, otherwise
- * a negative value is returned. The caller is responsible for calling the
- * rcu_read_[un]lock() functions.
+ * a negative value is returned.
*
*/
static int netlbl_unlhsh_add_addr6(struct netlbl_unlhsh_iface *iface,
* Description:
* Add a new, empty, interface entry into the unlabeled connection hash table.
* On success a pointer to the new interface entry is returned, on failure NULL
- * is returned. The caller is responsible for calling the rcu_read_[un]lock()
- * functions.
+ * is returned.
*
*/
static struct netlbl_unlhsh_iface *netlbl_unlhsh_add_iface(int ifindex)
if (netlbl_unlhsh_search_iface(ifindex) != NULL)
goto add_iface_failure;
list_add_tail_rcu(&iface->list,
- &rcu_dereference(netlbl_unlhsh)->tbl[bkt]);
+ &netlbl_unlhsh_rcu_deref(netlbl_unlhsh)->tbl[bkt]);
} else {
INIT_LIST_HEAD(&iface->list);
- if (rcu_dereference(netlbl_unlhsh_def) != NULL)
+ if (netlbl_unlhsh_rcu_deref(netlbl_unlhsh_def) != NULL)
goto add_iface_failure;
rcu_assign_pointer(netlbl_unlhsh_def, iface);
}
*
* Description:
* Remove an IP address entry from the unlabeled connection hash table.
- * Returns zero on success, negative values on failure. The caller is
- * responsible for calling the rcu_read_[un]lock() functions.
+ * Returns zero on success, negative values on failure.
*
*/
static int netlbl_unlhsh_remove_addr4(struct net *net,
*
* Description:
* Remove an IP address entry from the unlabeled connection hash table.
- * Returns zero on success, negative values on failure. The caller is
- * responsible for calling the rcu_read_[un]lock() functions.
+ * Returns zero on success, negative values on failure.
*
*/
static int netlbl_unlhsh_remove_addr6(struct net *net,
struct netlbl_unlhsh_iface *iface;
rcu_read_lock();
- iface = netlbl_unlhsh_search_iface_def(skb->skb_iif);
+ iface = netlbl_unlhsh_search_iface(skb->skb_iif);
if (iface == NULL)
+ iface = rcu_dereference(netlbl_unlhsh_def);
+ if (iface == NULL || !iface->valid)
goto unlabel_getattr_nolabel;
switch (family) {
case PF_INET: {
#include <linux/audit.h>
#include <linux/tty.h>
#include <linux/security.h>
+#include <linux/gfp.h>
#include <net/sock.h>
#include <net/netlink.h>
#include <net/genetlink.h>
struct netlink_sock *nlk = nlk_sk(sk);
struct sockaddr_nl *nladdr = (struct sockaddr_nl *)addr;
+ if (alen < sizeof(addr->sa_family))
+ return -EINVAL;
+
if (addr->sa_family == AF_UNSPEC) {
sk->sk_state = NETLINK_UNCONNECTED;
nlk->dst_pid = 0;
struct netlink_set_err_data *p)
{
struct netlink_sock *nlk = nlk_sk(sk);
+ int ret = 0;
if (sk == p->exclude_sk)
goto out;
!test_bit(p->group - 1, nlk->groups))
goto out;
+ if (p->code == ENOBUFS && nlk->flags & NETLINK_RECV_NO_ENOBUFS) {
+ ret = 1;
+ goto out;
+ }
+
sk->sk_err = p->code;
sk->sk_error_report(sk);
out:
- return 0;
+ return ret;
}
/**
* @pid: the PID of a process that we want to skip (if any)
* @groups: the broadcast group that will notice the error
* @code: error code, must be negative (as usual in kernelspace)
+ *
+ * This function returns the number of broadcast listeners that have set the
+ * NETLINK_RECV_NO_ENOBUFS socket option.
*/
-void netlink_set_err(struct sock *ssk, u32 pid, u32 group, int code)
+int netlink_set_err(struct sock *ssk, u32 pid, u32 group, int code)
{
struct netlink_set_err_data info;
struct hlist_node *node;
struct sock *sk;
+ int ret = 0;
info.exclude_sk = ssk;
info.pid = pid;
read_lock(&nl_table_lock);
sk_for_each_bound(sk, node, &nl_table[ssk->sk_protocol].mc_list)
- do_one_set_err(sk, &info);
+ ret += do_one_set_err(sk, &info);
read_unlock(&nl_table_lock);
+ return ret;
}
EXPORT_SYMBOL(netlink_set_err);
#include <linux/module.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/types.h>
#include <linux/socket.h>
#include <linux/types.h>
#include <linux/socket.h>
#include <linux/in.h>
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/sched.h>
#include <linux/timer.h>
#include <linux/fcntl.h>
#include <linux/in.h>
#include <linux/if_ether.h> /* For the statistics structure. */
+#include <linux/slab.h>
#include <asm/system.h>
#include <asm/uaccess.h>
#include <linux/string.h>
#include <linux/sockios.h>
#include <linux/net.h>
+#include <linux/slab.h>
#include <net/ax25.h>
#include <linux/inet.h>
#include <linux/netdevice.h>
* Copyright Tomi Manninen OH2BNS (oh2bns@sral.fi)
*/
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/socket.h>
#include <linux/timer.h>
#include <net/ax25.h>
#include <linux/string.h>
#include <linux/sockios.h>
#include <linux/net.h>
+#include <linux/slab.h>
#include <net/ax25.h>
#include <linux/inet.h>
#include <linux/netdevice.h>
#include <linux/string.h>
#include <linux/sockios.h>
#include <linux/net.h>
+#include <linux/slab.h>
#include <net/ax25.h>
#include <linux/inet.h>
#include <linux/netdevice.h>
#include <linux/string.h>
#include <linux/sockios.h>
#include <linux/net.h>
+#include <linux/slab.h>
#include <net/ax25.h>
#include <linux/inet.h>
#include <linux/netdevice.h>
#include <linux/wireless.h>
#include <linux/kernel.h>
#include <linux/kmod.h>
+#include <linux/slab.h>
#include <net/net_namespace.h>
#include <net/ip.h>
#include <net/protocol.h>
#include <linux/kernel.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <asm/unaligned.h>
#include <net/sock.h>
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/socket.h>
#include <asm/ioctls.h>
#include <net/sock.h>
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/socket.h>
#include <net/sock.h>
#include <net/tcp_states.h>
#include <linux/kernel.h>
#include <linux/net.h>
+#include <linux/slab.h>
#include <linux/netdevice.h>
#include <linux/phonet.h>
#include <linux/proc_fs.h>
#include <linux/kernel.h>
#include <linux/netlink.h>
#include <linux/phonet.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/phonet/pn_dev.h>
* 02110-1301 USA
*/
+#include <linux/gfp.h>
#include <linux/kernel.h>
#include <linux/net.h>
#include <linux/poll.h>
#include <linux/module.h>
#include <linux/errno.h>
#include <linux/kernel.h>
+#include <linux/gfp.h>
#include <linux/in.h>
#include <linux/poll.h>
#include <net/sock.h>
* SOFTWARE.
*
*/
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/rbtree.h>
*/
#include <linux/kernel.h>
#include <linux/list.h>
+#include <linux/slab.h>
#include <net/inet_hashtables.h>
#include "rds.h"
#include <linux/inetdevice.h>
#include <linux/if_arp.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include "rds.h"
#include "ib.h"
*/
#include <linux/kernel.h>
#include <linux/in.h>
+#include <linux/slab.h>
#include <linux/vmalloc.h>
#include "rds.h"
*
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include "rds.h"
#include "rdma.h"
*
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/dma-mapping.h>
#include <rdma/rdma_cm.h>
*/
#include <linux/percpu.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include <linux/proc_fs.h>
#include "rds.h"
#include <linux/inetdevice.h>
#include <linux/if_arp.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include "rds.h"
#include "iw.h"
*/
#include <linux/kernel.h>
#include <linux/in.h>
+#include <linux/slab.h>
#include <linux/vmalloc.h>
#include "rds.h"
*
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include "rds.h"
#include "rdma.h"
*
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/dma-mapping.h>
#include <rdma/rdma_cm.h>
*
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/in.h>
#include "rds.h"
*
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include "rds.h"
#include "rdma.h"
*
*/
#include <linux/highmem.h>
+#include <linux/gfp.h>
#include "rds.h"
*
*/
#include <linux/pagemap.h>
+#include <linux/slab.h>
#include <linux/rbtree.h>
#include <linux/dma-mapping.h> /* for DMA_*_DEVICE */
*
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <linux/in.h>
*
*/
#include <linux/kernel.h>
+#include <linux/gfp.h>
#include <net/sock.h>
#include <linux/in.h>
#include <linux/list.h>
*
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/in.h>
#include <net/tcp.h>
*
*/
#include <linux/kernel.h>
+#include <linux/gfp.h>
#include <linux/in.h>
#include <net/tcp.h>
*
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <net/tcp.h>
#include "rds.h"
#include <linux/wait.h>
#include <linux/poll.h>
#include <linux/fs.h>
+#include <linux/slab.h>
#include "rfkill.h"
#include <linux/types.h>
#include <linux/socket.h>
#include <linux/in.h>
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/sched.h>
#include <linux/spinlock.h>
#include <linux/fcntl.h>
#include <linux/in.h>
#include <linux/if_ether.h>
+#include <linux/slab.h>
#include <asm/system.h>
#include <asm/io.h>
#include <linux/string.h>
#include <linux/sockios.h>
#include <linux/net.h>
+#include <linux/slab.h>
#include <net/ax25.h>
#include <linux/inet.h>
#include <linux/netdevice.h>
* Copyright (C) Jonathan Naylor G4KLX (g4klx@g4klx.demon.co.uk)
*/
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/socket.h>
#include <linux/timer.h>
#include <net/ax25.h>
#include <linux/string.h>
#include <linux/sockios.h>
#include <linux/net.h>
+#include <linux/gfp.h>
#include <net/ax25.h>
#include <linux/inet.h>
#include <linux/netdevice.h>
#include <linux/string.h>
#include <linux/sockios.h>
#include <linux/net.h>
+#include <linux/slab.h>
#include <net/ax25.h>
#include <linux/inet.h>
#include <linux/netdevice.h>
#include <linux/string.h>
#include <linux/sockios.h>
#include <linux/net.h>
+#include <linux/slab.h>
#include <net/ax25.h>
#include <linux/inet.h>
#include <linux/netdevice.h>
#include <linux/module.h>
#include <linux/net.h>
+#include <linux/slab.h>
#include <linux/skbuff.h>
#include <linux/poll.h>
#include <linux/proc_fs.h>
#include <linux/in.h>
#include <linux/in6.h>
#include <linux/icmp.h>
+#include <linux/gfp.h>
#include <net/sock.h>
#include <net/af_rxrpc.h>
#include <net/ip.h>
/* get a notification message to send to the server app */
notification = alloc_skb(0, GFP_NOFS);
+ if (!notification) {
+ _debug("no memory");
+ ret = -ENOMEM;
+ goto error_nofree;
+ }
rxrpc_new_skb(notification);
notification->mark = RXRPC_SKB_MARK_NEW_CALL;
ret = -ECONNREFUSED;
error:
rxrpc_free_skb(notification);
+error_nofree:
_leave(" = %d", ret);
return ret;
}
#include <linux/circ_buf.h>
#include <linux/net.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <linux/udp.h>
#include <net/sock.h>
#include <net/af_rxrpc.h>
* 2 of the License, or (at your option) any later version.
*/
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/circ_buf.h>
#include <net/sock.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/net.h>
#include <linux/skbuff.h>
#include <linux/crypto.h>
#include <linux/in.h>
#include <linux/in6.h>
#include <linux/icmp.h>
+#include <linux/gfp.h>
#include <net/sock.h>
#include <net/af_rxrpc.h>
#include <net/ip.h>
#include <linux/key-type.h>
#include <linux/crypto.h>
#include <linux/ctype.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/af_rxrpc.h>
#include <keys/rxrpc-type.h>
#include <linux/module.h>
#include <linux/net.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/af_rxrpc.h>
#include "ar-internal.h"
*/
#include <linux/net.h>
+#include <linux/gfp.h>
#include <linux/skbuff.h>
#include <linux/circ_buf.h>
#include <net/sock.h>
#include <linux/in.h>
#include <linux/in6.h>
#include <linux/icmp.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/af_rxrpc.h>
#include <net/ip.h>
#include <linux/module.h>
#include <linux/net.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/af_rxrpc.h>
#include "ar-internal.h"
#include <linux/crypto.h>
#include <linux/scatterlist.h>
#include <linux/ctype.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/af_rxrpc.h>
#include <keys/rxrpc-type.h>
module will be called cls_flow.
config NET_CLS_CGROUP
- bool "Control Group Classifier"
+ tristate "Control Group Classifier"
select NET_CLS
depends on CGROUPS
---help---
Say Y here if you want to classify packets based on the control
cgroup of their process.
+ To compile this code as a module, choose M here: the
+ module will be called cls_cgroup.
+
config NET_EMATCH
bool "Extended Matches"
select NET_CLS
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/errno.h>
+#include <linux/slab.h>
#include <linux/skbuff.h>
#include <linux/init.h>
#include <linux/kmod.h>
#include <linux/rtnetlink.h>
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <net/netlink.h>
#include <net/pkt_sched.h>
#include <linux/tc_act/tc_ipt.h>
#include <linux/rtnetlink.h>
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/gfp.h>
#include <net/net_namespace.h>
#include <net/netlink.h>
#include <net/pkt_sched.h>
#include <linux/rtnetlink.h>
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <net/netlink.h>
#include <net/pkt_sched.h>
#include <linux/tc_act/tc_pedit.h>
#include <linux/skbuff.h>
#include <linux/rtnetlink.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <net/act_api.h>
#include <net/netlink.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/skbuff.h>
#include <linux/kmod.h>
#include <linux/netlink.h>
#include <linux/err.h>
+#include <linux/slab.h>
#include <net/net_namespace.h>
#include <net/sock.h>
#include <net/netlink.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/string.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/string.h>
#include <linux/errno.h>
u32 classid;
};
+static struct cgroup_subsys_state *cgrp_create(struct cgroup_subsys *ss,
+ struct cgroup *cgrp);
+static void cgrp_destroy(struct cgroup_subsys *ss, struct cgroup *cgrp);
+static int cgrp_populate(struct cgroup_subsys *ss, struct cgroup *cgrp);
+
+struct cgroup_subsys net_cls_subsys = {
+ .name = "net_cls",
+ .create = cgrp_create,
+ .destroy = cgrp_destroy,
+ .populate = cgrp_populate,
+#ifdef CONFIG_NET_CLS_CGROUP
+ .subsys_id = net_cls_subsys_id,
+#else
+#define net_cls_subsys_id net_cls_subsys.subsys_id
+#endif
+ .module = THIS_MODULE,
+};
+
+
static inline struct cgroup_cls_state *cgrp_cls_state(struct cgroup *cgrp)
{
return container_of(cgroup_subsys_state(cgrp, net_cls_subsys_id),
return cgroup_add_files(cgrp, ss, ss_files, ARRAY_SIZE(ss_files));
}
-struct cgroup_subsys net_cls_subsys = {
- .name = "net_cls",
- .create = cgrp_create,
- .destroy = cgrp_destroy,
- .populate = cgrp_populate,
- .subsys_id = net_cls_subsys_id,
-};
-
struct cls_cgroup_head
{
u32 handle;
static int __init init_cgroup_cls(void)
{
- return register_tcf_proto_ops(&cls_cgroup_ops);
+ int ret = register_tcf_proto_ops(&cls_cgroup_ops);
+ if (ret)
+ return ret;
+ ret = cgroup_load_subsys(&net_cls_subsys);
+ if (ret)
+ unregister_tcf_proto_ops(&cls_cgroup_ops);
+ return ret;
}
static void __exit exit_cgroup_cls(void)
{
unregister_tcf_proto_ops(&cls_cgroup_ops);
+ cgroup_unload_subsys(&net_cls_subsys);
}
module_init(init_cgroup_cls);
#include <linux/ip.h>
#include <linux/ipv6.h>
#include <linux/if_vlan.h>
+#include <linux/slab.h>
#include <net/pkt_cls.h>
#include <net/ip.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/string.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/kernel.h>
#include <linux/skbuff.h>
#include <linux/errno.h>
+#include <linux/slab.h>
#include <net/act_api.h>
#include <net/netlink.h>
#include <net/pkt_cls.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/string.h>
* only available if that subsystem is enabled in the kernel.
*/
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/types.h>
#include <linux/kernel.h>
* Authors: Thomas Graf <tgraf@suug.ch>
*/
+#include <linux/gfp.h>
#include <linux/module.h>
#include <linux/types.h>
#include <linux/kernel.h>
* Authors: Thomas Graf <tgraf@suug.ch>
*/
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/types.h>
#include <linux/kernel.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/list.h>
#include <linux/hrtimer.h>
#include <linux/lockdep.h>
+#include <linux/slab.h>
#include <net/net_namespace.h>
#include <net/sock.h>
/* Written 1998-2000 by Werner Almesberger, EPFL ICA */
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/string.h>
#include <linux/errno.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/string.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/errno.h>
#include <linux/netdevice.h>
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/string.h>
#include <linux/errno.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/init.h>
#include <linux/rcupdate.h>
#include <linux/list.h>
+#include <linux/slab.h>
#include <net/pkt_sched.h>
/* Main transmission queue. */
* For all the glorious comments look at include/net/red.h
*/
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/compiler.h>
#include <linux/rbtree.h>
#include <linux/workqueue.h>
+#include <linux/slab.h>
#include <net/netlink.h>
#include <net/pkt_sched.h>
*/
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/errno.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/string.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/errno.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/ipv6.h>
#include <linux/skbuff.h>
#include <linux/jhash.h>
+#include <linux/slab.h>
#include <net/ip.h>
#include <net/netlink.h>
#include <net/pkt_sched.h>
#include <linux/module.h>
#include <linux/types.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/if_arp.h>
* be incorporated into the next SCTP release.
*/
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/crypto.h>
#include <linux/scatterlist.h>
*/
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/in.h>
#include <net/sock.h>
#include <net/ipv6.h>
#include <linux/net.h>
#include <linux/inet.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/sctp/sctp.h>
#include <net/sctp/sm.h>
#include <linux/socket.h>
#include <linux/ip.h>
#include <linux/time.h> /* For struct timeval */
+#include <linux/slab.h>
#include <net/ip.h>
#include <net/icmp.h>
#include <net/snmp.h>
#include <net/sctp/sctp.h>
#include <net/sctp/sm.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
/* Initialize an SCTP inqueue. */
void sctp_inq_init(struct sctp_inq *queue)
#include <linux/netdevice.h>
#include <linux/init.h>
#include <linux/ipsec.h>
+#include <linux/slab.h>
#include <linux/ipv6.h>
#include <linux/icmpv6.h>
#include <linux/ip.h>
#include <linux/ipv6.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <net/inet_ecn.h>
#include <net/ip.h>
#include <net/icmp.h>
#include <linux/list.h> /* For struct list_head */
#include <linux/socket.h>
#include <linux/ip.h>
+#include <linux/slab.h>
#include <net/sock.h> /* For skb_set_owner_w */
#include <net/sctp/sctp.h>
#include <linux/socket.h>
#include <linux/ip.h>
#include <linux/time.h> /* For struct timeval */
+#include <linux/gfp.h>
#include <net/sock.h>
#include <net/sctp/sctp.h>
#include <net/sctp/sm.h>
#include <linux/bootmem.h>
#include <linux/highmem.h>
#include <linux/swap.h>
+#include <linux/slab.h>
#include <net/net_namespace.h>
#include <net/protocol.h>
#include <net/ip.h>
#include <linux/inet.h>
#include <linux/scatterlist.h>
#include <linux/crypto.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <linux/skbuff.h>
#include <linux/types.h>
#include <linux/socket.h>
#include <linux/ip.h>
+#include <linux/gfp.h>
#include <net/sock.h>
#include <net/sctp/sctp.h>
#include <net/sctp/sm.h>
#include <linux/ipv6.h>
#include <linux/net.h>
#include <linux/inet.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/inet_ecn.h>
#include <linux/skbuff.h>
#include <linux/poll.h>
#include <linux/init.h>
#include <linux/crypto.h>
+#include <linux/slab.h>
#include <net/ip.h>
#include <net/icmp.h>
*/
#include <linux/types.h>
+#include <linux/slab.h>
#include <net/sctp/sctp.h>
#include <net/sctp/sm.h>
* be incorporated into the next SCTP release.
*/
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/random.h>
#include <net/sctp/sctp.h>
* be incorporated into the next SCTP release.
*/
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/bitmap.h>
#include <net/sctp/sctp.h>
* be incorporated into the next SCTP release.
*/
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/skbuff.h>
#include <net/sctp/structs.h>
* be incorporated into the next SCTP release.
*/
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/skbuff.h>
#include <net/sock.h>
#include <linux/wireless.h>
#include <linux/nsproxy.h>
#include <linux/magic.h>
+#include <linux/slab.h>
#include <asm/uaccess.h>
#include <asm/unistd.h>
break;
++datagrams;
+ /* MSG_WAITFORONE turns on MSG_DONTWAIT after one packet */
+ if (flags & MSG_WAITFORONE)
+ flags |= MSG_DONTWAIT;
+
if (timeout) {
ktime_get_ts(timeout);
*timeout = timespec_sub(end_time, *timeout);
#include <net/ipv6.h>
#include <linux/sunrpc/clnt.h>
+#include <linux/slab.h>
#if defined(CONFIG_IPV6) || defined(CONFIG_IPV6_MODULE)
*/
#include <linux/err.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/module.h>
#include <linux/sched.h>
rqstp->rq_release_snd_buf = priv_release_snd_buf;
return 0;
out_free:
- for (i--; i >= 0; i--) {
- __free_page(rqstp->rq_enc_pages[i]);
- }
+ rqstp->rq_enc_pages_num = i;
+ priv_release_snd_buf(rqstp);
out:
return -EAGAIN;
}
#include <linux/types.h>
#include <linux/module.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/sunrpc/sched.h>
#include <linux/sunrpc/gss_asn1.h>
#include <linux/err.h>
#include <linux/types.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/scatterlist.h>
#include <linux/crypto.h>
#include <linux/highmem.h>
*/
#include <linux/types.h>
-#include <linux/slab.h>
#include <linux/jiffies.h>
#include <linux/sunrpc/gss_krb5.h>
#include <linux/random.h>
*/
#include <linux/types.h>
-#include <linux/slab.h>
#include <linux/sunrpc/gss_krb5.h>
#include <linux/crypto.h>
*/
#include <linux/types.h>
-#include <linux/slab.h>
#include <linux/jiffies.h>
#include <linux/sunrpc/gss_krb5.h>
#include <linux/crypto.h>
#include <linux/types.h>
-#include <linux/slab.h>
#include <linux/jiffies.h>
#include <linux/sunrpc/gss_krb5.h>
#include <linux/random.h>
*/
#include <linux/types.h>
-#include <linux/slab.h>
#include <linux/jiffies.h>
#include <linux/sunrpc/gss_spkm3.h>
#include <linux/random.h>
*
*/
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/module.h>
#include <linux/pagemap.h>
* Copyright (C) 1996, Olaf Kirch <okir@monad.swb.de>
*/
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/sched.h>
#include <linux/module.h>
******************************************************************************/
#include <linux/tcp.h>
+#include <linux/slab.h>
#include <linux/sunrpc/xprt.h>
#ifdef RPC_DEBUG
#define RPCDBG_FACILITY RPCDBG_SVCDSP
-void bc_release_request(struct rpc_task *task)
-{
- struct rpc_rqst *req = task->tk_rqstp;
-
- dprintk("RPC: bc_release_request: task= %p\n", task);
-
- /*
- * Release this request only if it's a backchannel
- * preallocated request
- */
- if (!bc_prealloc(req))
- return;
- xprt_free_bc_request(req);
-}
-
/* Empty callback ops */
static const struct rpc_call_ops nfs41_callback_ops = {
};
task = rpc_new_task(&task_setup_data);
if (!task) {
xprt_free_bc_request(req);
+ task = ERR_PTR(-ENOMEM);
goto out;
}
task->tk_rqstp = req;
struct dentry *dentry;
dentry = __rpc_lookup_create(parent, name);
+ if (IS_ERR(dentry))
+ return dentry;
if (dentry->d_inode == NULL)
return dentry;
dput(dentry);
#include <linux/kernel.h>
#include <linux/errno.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <net/ipv6.h>
#include <linux/sunrpc/clnt.h>
#include <linux/compiler.h>
#include <linux/netdevice.h>
+#include <linux/gfp.h>
#include <linux/skbuff.h>
#include <linux/types.h>
#include <linux/pagemap.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/interrupt.h>
#include <linux/module.h>
#include <linux/kthread.h>
+#include <linux/slab.h>
#include <linux/sunrpc/types.h>
#include <linux/sunrpc/xdr.h>
#include <linux/errno.h>
#include <linux/freezer.h>
#include <linux/kthread.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <linux/sunrpc/stats.h>
#include <linux/sunrpc/svc_xprt.h>
#include <linux/seq_file.h>
#include <linux/hash.h>
#include <linux/string.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/ipv6.h>
#include <linux/kernel.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/types.h>
#include <linux/string.h>
#include <linux/kernel.h>
#include <linux/sunrpc/clnt.h>
#include <linux/sunrpc/metrics.h>
+#include <linux/sunrpc/bc_xprt.h>
#include "sunrpc.h"
if (req->rq_release_snd_buf)
req->rq_release_snd_buf(req);
- /*
- * Early exit if this is a backchannel preallocated request.
- * There is no need to have it added to the RPC slot list.
- */
- if (is_bc_request)
- return;
-
- memset(req, 0, sizeof(*req)); /* mark unused */
-
dprintk("RPC: %5u release request %p\n", task->tk_pid, req);
+ if (likely(!is_bc_request)) {
+ memset(req, 0, sizeof(*req)); /* mark unused */
- spin_lock(&xprt->reserve_lock);
- list_add(&req->rq_list, &xprt->free);
- rpc_wake_up_next(&xprt->backlog);
- spin_unlock(&xprt->reserve_lock);
+ spin_lock(&xprt->reserve_lock);
+ list_add(&req->rq_list, &xprt->free);
+ rpc_wake_up_next(&xprt->backlog);
+ spin_unlock(&xprt->reserve_lock);
+ } else
+ xprt_free_bc_request(req);
}
/**
*/
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/fs.h>
#include <linux/sysctl.h>
#include <linux/sunrpc/clnt.h>
#include <linux/sunrpc/debug.h>
#include <linux/sunrpc/rpc_rdma.h>
#include <linux/sched.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <rdma/ib_verbs.h>
#include <rdma/rdma_cm.h>
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/seq_file.h>
#include "xprt_rdma.h"
*/
#include <linux/pci.h> /* for Tavor hack below */
+#include <linux/slab.h>
#include "xprt_rdma.h"
.buf_free = rpc_free,
.send_request = xs_tcp_send_request,
.set_retrans_timeout = xprt_set_retrans_timeout_def,
-#if defined(CONFIG_NFS_V4_1)
- .release_request = bc_release_request,
-#endif /* CONFIG_NFS_V4_1 */
.close = xs_tcp_close,
.destroy = xs_destroy,
.print_stats = xs_tcp_print_stats,
#include <linux/netdevice.h>
#include <linux/in.h>
#include <linux/list.h>
+#include <linux/slab.h>
#include <linux/vmalloc.h>
/*
#include <net/tipc/tipc_bearer.h>
#include <net/tipc/tipc_msg.h>
#include <linux/netdevice.h>
+#include <linux/slab.h>
#include <net/net_namespace.h>
#define MAX_ETH_BEARERS 2
#include <linux/socket.h>
#include <linux/errno.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/poll.h>
#include <linux/fcntl.h>
+#include <linux/gfp.h>
#include <asm/string.h>
#include <asm/atomic.h>
#include <net/sock.h>
#include <linux/un.h>
#include <linux/net.h>
#include <linux/fs.h>
-#include <linux/slab.h>
#include <linux/skbuff.h>
#include <linux/netdevice.h>
#include <linux/file.h>
*/
#include <linux/mm.h>
+#include <linux/slab.h>
#include <linux/sysctl.h>
#include <net/af_unix.h>
* wimax_msg_send()
*/
#include <linux/device.h>
+#include <linux/slab.h>
#include <net/genetlink.h>
#include <linux/netdevice.h>
#include <linux/wimax.h>
* wimax_rfkill_rm()
*/
#include <linux/device.h>
+#include <linux/gfp.h>
#include <net/genetlink.h>
#include <linux/netdevice.h>
#include <linux/wimax.h>
#include <linux/module.h>
#include <linux/err.h>
#include <linux/list.h>
+#include <linux/slab.h>
#include <linux/nl80211.h>
#include <linux/debugfs.h>
#include <linux/notifier.h>
* published by the Free Software Foundation.
*/
+#include <linux/slab.h>
#include "core.h"
#include "debugfs.h"
#include <linux/etherdevice.h>
#include <linux/if_arp.h>
+#include <linux/slab.h>
#include <net/cfg80211.h>
#include "wext-compat.h"
#include "nl80211.h"
#include <linux/module.h>
#include <linux/netdevice.h>
#include <linux/nl80211.h>
+#include <linux/slab.h>
#include <linux/wireless.h>
#include <net/cfg80211.h>
#include <net/iw_handler.h>
#include <linux/if.h>
#include <linux/module.h>
#include <linux/err.h>
+#include <linux/slab.h>
#include <linux/list.h>
#include <linux/if_ether.h>
#include <linux/ieee80211.h>
*
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/list.h>
#include <linux/random.h>
#include <linux/nl80211.h>
};
static LIST_HEAD(reg_regdb_search_list);
-static DEFINE_SPINLOCK(reg_regdb_search_lock);
+static DEFINE_MUTEX(reg_regdb_search_mutex);
static void reg_regdb_search(struct work_struct *work)
{
const struct ieee80211_regdomain *curdom, *regdom;
int i, r;
- spin_lock(®_regdb_search_lock);
+ mutex_lock(®_regdb_search_mutex);
while (!list_empty(®_regdb_search_list)) {
request = list_first_entry(®_regdb_search_list,
struct reg_regdb_search_request,
r = reg_copy_regd(®dom, curdom);
if (r)
break;
- spin_unlock(®_regdb_search_lock);
mutex_lock(&cfg80211_mutex);
set_regdom(regdom);
mutex_unlock(&cfg80211_mutex);
- spin_lock(®_regdb_search_lock);
break;
}
}
kfree(request);
}
- spin_unlock(®_regdb_search_lock);
+ mutex_unlock(®_regdb_search_mutex);
}
static DECLARE_WORK(reg_regdb_work, reg_regdb_search);
memcpy(request->alpha2, alpha2, 2);
- spin_lock(®_regdb_search_lock);
+ mutex_lock(®_regdb_search_mutex);
list_add_tail(&request->list, ®_regdb_search_list);
- spin_unlock(®_regdb_search_lock);
+ mutex_unlock(®_regdb_search_mutex);
schedule_work(®_regdb_work);
}
* Copyright 2008 Johannes Berg <johannes@sipsolutions.net>
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/netdevice.h>
#include <linux/wireless.h>
#include <linux/etherdevice.h>
#include <linux/if_arp.h>
+#include <linux/slab.h>
#include <linux/workqueue.h>
#include <linux/wireless.h>
#include <net/iw_handler.h>
*/
#include <linux/bitops.h>
#include <linux/etherdevice.h>
+#include <linux/slab.h>
#include <net/cfg80211.h>
#include <net/ip.h>
#include "core.h"
#include <linux/nl80211.h>
#include <linux/if_arp.h>
#include <linux/etherdevice.h>
+#include <linux/slab.h>
#include <net/iw_handler.h>
#include <net/cfg80211.h>
#include "wext-compat.h"
#include <linux/kernel.h>
#include <linux/netdevice.h>
#include <linux/rtnetlink.h>
+#include <linux/slab.h>
#include <linux/wireless.h>
#include <linux/uaccess.h>
#include <net/cfg80211.h>
*
* (As all part of the Linux kernel, this file is GPL)
*/
+#include <linux/slab.h>
#include <linux/wireless.h>
#include <linux/netdevice.h>
#include <net/iw_handler.h>
#include <linux/etherdevice.h>
#include <linux/if_arp.h>
+#include <linux/slab.h>
#include <net/cfg80211.h>
#include "wext-compat.h"
#include "nl80211.h"
#include <linux/netdevice.h>
#include <linux/if_arp.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <net/tcp_states.h>
#include <asm/uaccess.h>
#include <linux/kernel.h>
#include <linux/netdevice.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <net/sock.h>
#include <linux/if_arp.h>
#include <net/x25.h>
*/
#include <linux/if_arp.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <net/x25.h>
LIST_HEAD(x25_forward_list);
* i-frames.
*/
+#include <linux/slab.h>
#include <linux/errno.h>
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/kernel.h>
#include <linux/jiffies.h>
#include <linux/timer.h>
+#include <linux/slab.h>
#include <linux/netdevice.h>
#include <linux/skbuff.h>
#include <asm/uaccess.h>
* needed cleaned seq-number fields.
*/
+#include <linux/slab.h>
#include <linux/socket.h>
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/if_arp.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <net/x25.h>
LIST_HEAD(x25_route_list);
* restriction on response.
*/
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/skbuff.h>
#include <linux/crypto.h>
#include <linux/err.h>
-#include <linux/gfp.h>
#include <linux/list.h>
#include <linux/module.h>
#include <linux/mutex.h>
#include <linux/percpu.h>
+#include <linux/slab.h>
#include <linux/smp.h>
#include <linux/vmalloc.h>
#include <net/ip.h>
#include <linux/netdevice.h>
#include <linux/netfilter.h>
#include <linux/skbuff.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <net/dst.h>
#include <net/xfrm.h>
#include <linux/audit.h>
#include <asm/uaccess.h>
#include <linux/ktime.h>
+#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/kernel.h>
#include <linux/sysctl.h>
+#include <linux/slab.h>
#include <net/net_namespace.h>
#include <net/xfrm.h>
#include <linux/kobject.h>
#include <linux/string.h>
#include <linux/sysfs.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/init.h>
my $file_cnt = @files;
my $lastfile;
- open(my $patch, '<', $file)
+ open(my $patch, "< $file")
or die "$P: Can't open $file: $!\n";
while (<$patch>) {
my $patch_line = $_;
$nested =~ s/\/\*.*?\*\///gos;
# strip kmemcheck_bitfield_{begin,end}.*;
$members =~ s/kmemcheck_bitfield_.*?;//gos;
+ # strip attributes
+ $members =~ s/__aligned\s*\(\d+\)//gos;
create_parameterlist($members, ';', $file);
check_sections($file, $declaration_name, "struct", $sectcheck, $struct_actual, $nested);
$prototype =~ s/^noinline +//;
$prototype =~ s/__devinit +//;
$prototype =~ s/__init +//;
+ $prototype =~ s/__init_or_module +//;
$prototype =~ s/^#\s*define\s+//; #ak added
$prototype =~ s/__attribute__\s*\(\([a-z,]*\)\)//;
#include <linux/list.h>
#include <linux/uaccess.h>
#include <linux/seq_file.h>
+#include <linux/slab.h>
#include <linux/rcupdate.h>
#include <linux/mutex.h>
* and store_template.
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include "ima.h"
static const char *IMA_TEMPLATE_NAME = "ima";
*/
#include <linux/fs.h>
+#include <linux/gfp.h>
#include <linux/audit.h>
#include "ima.h"
#include <linux/crypto.h>
#include <linux/scatterlist.h>
#include <linux/err.h>
+#include <linux/slab.h>
#include "ima.h"
static int init_desc(struct hash_desc *desc)
* current measurement list and IMA statistics
*/
#include <linux/fcntl.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/seq_file.h>
#include <linux/rculist.h>
* - cache integrity information associated with an inode
* using a radix tree.
*/
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/spinlock.h>
#include <linux/radix-tree.h>
*/
#include <linux/module.h>
#include <linux/scatterlist.h>
+#include <linux/slab.h>
#include <linux/err.h>
#include "ima.h"
#include <linux/binfmts.h>
#include <linux/mount.h>
#include <linux/mman.h>
+#include <linux/slab.h>
#include "ima.h"
#include <linux/security.h>
#include <linux/magic.h>
#include <linux/parser.h>
+#include <linux/slab.h>
#include "ima.h"
*/
#include <linux/module.h>
#include <linux/rculist.h>
+#include <linux/slab.h>
#include "ima.h"
LIST_HEAD(ima_measurements); /* list of all measurements */
#include <linux/module.h>
#include <linux/init.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/fs.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/sched.h>
-#include <linux/slab.h>
#include <linux/keyctl.h>
#include <linux/fs.h>
#include <linux/err.h>
#include <linux/types.h>
#include <linux/stddef.h>
#include <linux/kernel.h>
+#include <linux/gfp.h>
#include <linux/fs.h>
#include <linux/init.h>
#include <net/sock.h>
*/
#include <linux/init.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/stddef.h>
#include <linux/kernel.h>
#include <linux/list.h>
#include <linux/spinlock.h>
#include <linux/rcupdate.h>
+#include <linux/gfp.h>
#include <linux/ip.h>
#include <linux/ipv6.h>
#include <net/sock.h>
*/
#include <linux/init.h>
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/stddef.h>
#include <linux/kernel.h>
#include <linux/list.h>
#include <linux/types.h>
#include <linux/rcupdate.h>
#include <linux/list.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/in.h>
#include <linux/in6.h>
#include <linux/types.h>
#include <linux/rcupdate.h>
#include <linux/list.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include <linux/in.h>
#include <linux/in6.h>
* Author : Stephen Smalley, <sds@epoch.ncsc.mil>
*/
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/string.h>
#include <linux/errno.h>
#include "symtab.h"
#include <linux/netfilter.h>
#include <linux/netfilter_ipv4.h>
#include <linux/netfilter_ipv6.h>
+#include <linux/slab.h>
#include <linux/ip.h>
#include <linux/tcp.h>
#include <linux/skbuff.h>
*/
#include <linux/types.h>
+#include <linux/slab.h>
#include <linux/fs.h>
#include <linux/sched.h>
#include "smack.h"
#include <linux/ip.h>
#include <linux/tcp.h>
#include <linux/udp.h>
+#include <linux/slab.h>
#include <linux/mutex.h>
#include <linux/pipe_fs_i.h>
#include <net/netlabel.h>
#include <linux/vmalloc.h>
#include <linux/security.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <net/net_namespace.h>
#include <net/netlabel.h>
#include <net/cipso_ipv4.h>
*/
#include <linux/uaccess.h>
+#include <linux/slab.h>
#include <linux/security.h>
#include <linux/hardirq.h>
#include "common.h"
#include "common.h"
#include <linux/binfmts.h>
+#include <linux/slab.h>
/* Variables definitions.*/
*/
#include "common.h"
+#include <linux/slab.h>
/* Keyword array for single path operations. */
static const char *tomoyo_path_keyword[TOMOYO_MAX_PATH_OPERATION] = {
#include "common.h"
#include <linux/kthread.h>
+#include <linux/slab.h>
enum tomoyo_gc_id {
TOMOYO_ID_DOMAIN_INITIALIZER,
#include <linux/fs_struct.h>
#include <linux/hash.h>
#include <linux/magic.h>
+#include <linux/slab.h>
#include "common.h"
/**
*/
#include <linux/delay.h>
#include <linux/module.h>
+#include <linux/slab.h>
MODULE_AUTHOR("Johannes Berg <johannes@sipsolutions.net>");
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("pcm3052 (onyx) codec driver for snd-aoa");
#include <linux/delay.h>
#include <linux/module.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
MODULE_AUTHOR("Johannes Berg <johannes@sipsolutions.net>");
MODULE_LICENSE("GPL");
*/
#include <linux/delay.h>
#include <linux/module.h>
+#include <linux/slab.h>
MODULE_AUTHOR("Johannes Berg <johannes@sipsolutions.net>");
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("toonie codec driver for snd-aoa");
* GPL v2, can be found in COPYING.
*/
+#include <linux/slab.h>
#include <asm/pmac_feature.h>
#include <asm/pmac_pfunc.h>
#include "../aoa.h"
#include <asm/prom.h>
#include <linux/list.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include "../aoa.h"
#include "../soundbus/soundbus.h"
#include <linux/kernel.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <asm/prom.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/interrupt.h>
#include <linux/dma-mapping.h>
#include <asm/io.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <asm/macio.h>
#include <linux/pci.h>
* published by the Free Software Foundation.
*/
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/dma-mapping.h>
if (!rtd->dma_desc_array)
goto err1;
+ rtd->dma_ch = -1;
runtime->private_data = rtd;
return 0;
/* this file included from control.c */
#include <linux/compat.h>
+#include <linux/slab.h>
struct snd_ctl_elem_list32 {
u32 offset;
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/hrtimer.h>
#include <linux/init.h>
#include <linux/time.h>
#include <linux/mm.h>
+#include <linux/slab.h>
#include <linux/smp_lock.h>
#include <linux/string.h>
#include <sound/core.h>
*/
#include <linux/input.h>
+#include <linux/slab.h>
#include <sound/jack.h>
#include <sound/core.h>
#include <linux/init.h>
#include <linux/time.h>
+#include <linux/slab.h>
#include <linux/ioport.h>
#include <sound/core.h>
*
*/
-#include <linux/slab.h>
#include <linux/time.h>
#include <sound/core.h>
#include <sound/pcm.h>
/* This file included from pcm_native.c */
#include <linux/compat.h>
+#include <linux/slab.h>
static int snd_pcm_ioctl_delay_compat(struct snd_pcm_substream *substream,
s32 __user *src)
#define xrun_debug(substream, mask) \
((substream)->pstr->xrun_debug & (mask))
+#else
+#define xrun_debug(substream, mask) 0
+#endif
#define dump_stack_on_xrun(substream) do { \
if (xrun_debug(substream, XRUN_DEBUG_STACK)) \
}
}
+#ifdef CONFIG_SND_PCM_XRUN_DEBUG
#define hw_ptr_error(substream, fmt, args...) \
do { \
if (xrun_debug(substream, XRUN_DEBUG_BASIC)) { \
#else /* ! CONFIG_SND_PCM_XRUN_DEBUG */
-#define xrun_debug(substream, mask) 0
-#define xrun(substream) do { } while (0)
#define hw_ptr_error(substream, fmt, args...) do { } while (0)
#define xrun_log(substream, pos) do { } while (0)
#define xrun_log_show(substream) do { } while (0)
#include <asm/io.h>
#include <linux/time.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/moduleparam.h>
#include <linux/vmalloc.h>
#include <sound/core.h>
#include "seq_oss_event.h"
#include <linux/init.h>
#include <linux/moduleparam.h>
+#include <linux/slab.h>
/*
* common variables
#include <sound/seq_midi_event.h>
#include "../seq_lock.h"
#include <linux/init.h>
+#include <linux/slab.h>
/*
#include <sound/seq_oss_legacy.h>
#include "../seq_lock.h"
#include <linux/wait.h>
+#include <linux/slab.h>
/*
* constants
#include "seq_oss_midi.h"
#include "../seq_lock.h"
#include <linux/init.h>
+#include <linux/slab.h>
/*
* constants
#include "seq_oss_timer.h"
#include "seq_oss_event.h"
#include <sound/seq_oss_legacy.h>
+#include <linux/slab.h>
/*
*/
#include "../seq_lock.h"
#include "../seq_clientmgr.h"
#include <linux/wait.h>
+#include <linux/slab.h>
/*
/* This file included from seq.c */
#include <linux/compat.h>
+#include <linux/slab.h>
struct snd_seq_port_info32 {
struct snd_seq_addr addr; /* client/port numbers */
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include "seq_system.h"
#include "seq_timer.h"
#include <linux/platform_device.h>
#include <linux/ioport.h>
+#include <linux/slab.h>
#include <linux/io.h>
#include <linux/interrupt.h>
#include <linux/interrupt.h>
#include <linux/err.h>
#include <linux/platform_device.h>
-#include <linux/slab.h>
#include <linux/ioport.h>
#include <linux/moduleparam.h>
#include <sound/core.h>
#include <linux/parport.h>
#include <linux/spinlock.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/initval.h>
#include <sound/rawmidi.h>
*/
#include "opl3_voice.h"
-#include <linux/slab.h>
static int snd_opl3_open_seq_oss(struct snd_seq_oss_arg *arg, void *closure);
static int snd_opl3_close_seq_oss(struct snd_seq_oss_arg *arg);
*
*/
+#include <linux/slab.h>
#include <sound/opl3.h>
#include <sound/asound_fm.h>
#include "opl4_local.h"
#include <sound/initval.h>
#include <linux/ioport.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <asm/io.h>
*/
#include <linux/module.h>
+#include <linux/gfp.h>
#include <linux/moduleparam.h>
#include <linux/interrupt.h>
#include <sound/pcm.h>
#include <linux/parport.h>
#include <linux/spinlock.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/initval.h>
#include <sound/rawmidi.h>
#include <linux/device.h>
#include <linux/firmware.h>
+#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <sound/core.h>
#include <sound/hwdep.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/version.h>
#include <sound/core.h>
#include <sound/tea575x-tuner.h>
#include <linux/init.h>
#include <linux/err.h>
#include <linux/isa.h>
-#include <linux/slab.h>
#include <linux/pnp.h>
#include <linux/moduleparam.h>
#include <sound/core.h>
#include <linux/init.h>
#include <linux/err.h>
#include <linux/isa.h>
-#include <linux/slab.h>
#include <linux/pnp.h>
#include <linux/moduleparam.h>
#include <sound/core.h>
#include <linux/init.h>
#include <linux/err.h>
#include <linux/isa.h>
-#include <linux/slab.h>
#include <linux/pnp.h>
#include <linux/isapnp.h>
#include <linux/moduleparam.h>
#include <linux/err.h>
#include <linux/isa.h>
#include <linux/delay.h>
-#include <linux/slab.h>
#include <linux/pnp.h>
#include <linux/moduleparam.h>
#include <asm/dma.h>
*/
#include <linux/io.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/ioport.h>
#include <linux/errno.h>
#include <linux/isa.h>
#include <linux/interrupt.h>
#include <linux/pm.h>
-#include <linux/slab.h>
#include <linux/pnp.h>
#include <linux/moduleparam.h>
#include <sound/core.h>
#include <linux/isa.h>
#include <linux/pnp.h>
#include <linux/delay.h>
-#include <linux/slab.h>
#include <linux/ioport.h>
#include <linux/moduleparam.h>
#include <asm/io.h>
#include <linux/err.h>
#include <linux/isa.h>
#include <linux/delay.h>
-#include <linux/slab.h>
#include <linux/pnp.h>
#include <linux/moduleparam.h>
#include <asm/io.h>
#include "emu8000_local.h"
#include <linux/init.h>
+#include <linux/slab.h>
#include <sound/initval.h>
#include <sound/pcm.h>
#include <asm/dma.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/pnp.h>
#include <linux/err.h>
#include <linux/isa.h>
#include <linux/init.h>
#include <linux/err.h>
#include <linux/isa.h>
-#include <linux/slab.h>
#include <linux/ioport.h>
#include <linux/moduleparam.h>
#include <sound/core.h>
#include <linux/init.h>
#include <linux/interrupt.h>
-#include <linux/slab.h>
#include <linux/err.h>
#include <linux/isa.h>
#include <linux/pnp.h>
#include <linux/init.h>
#include <linux/time.h>
#include <linux/wait.h>
+#include <linux/slab.h>
#include <linux/firmware.h>
#include <sound/core.h>
#include <sound/snd_wavefront.h>
#include <linux/wait.h>
#include <linux/firmware.h>
#include <linux/moduleparam.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/snd_wavefront.h>
#include <sound/initval.h>
#include <linux/dma-mapping.h>
#include <linux/platform_device.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <asm/sgi/hpc3.h>
#include <asm/sgi/ip22.h>
#include <linux/init.h>
#include <linux/delay.h>
#include <linux/spinlock.h>
-#include <linux/gfp.h>
#include <linux/interrupt.h>
#include <linux/dma-mapping.h>
#include <linux/platform_device.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <asm/ip32/ip32_ints.h>
#include <asm/ip32/mace.h>
#include <linux/interrupt.h>
#include <linux/module.h>
#include <linux/stddef.h>
+#include <linux/slab.h>
#include <linux/isapnp.h>
#include <linux/pnp.h>
#include <linux/spinlock.h>
#define SAMPLE_ROUNDUP 0
#include <linux/mm.h>
+#include <linux/gfp.h>
#include "sound_config.h"
#define DMAP_FREE_ON_CLOSE 0
#include <linux/init.h>
#include <linux/module.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include "sound_config.h"
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/spinlock.h>
#include <linux/module.h>
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/types.h>
#include <linux/delay.h>
#include <linux/kernel.h>
#include <linux/module.h>
-#include <linux/slab.h>
#include <linux/types.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/smp_lock.h>
+#include <linux/gfp.h>
#include <asm/irq.h>
#include <asm/io.h>
#include "sound_config.h"
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/delay.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include "sound_config.h"
#include "sb_mixer.h"
#include <linux/module.h>
#include <linux/delay.h>
#include <linux/spinlock.h>
+#include <linux/slab.h>
#include "sound_config.h"
#include "sound_firmware.h"
*/
#include <linux/spinlock.h>
+#include <linux/slab.h>
#include "sound_config.h"
* Stanislav Voronyi <stas@esc.kharkov.com> : Support for AWE 3DSE device (Jun 7 1999)
*/
+#include <linux/slab.h>
+
#include "sound_config.h"
#define __SB_MIXER_C__
#include <asm/dma.h>
#include <asm/io.h>
#include <linux/wait.h>
-#include <linux/slab.h>
#include <linux/ioport.h>
#include <linux/major.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include "sound_config.h"
#include <linux/init.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/spinlock.h>
#include "sound_config.h"
* We currently support a mixer device, but it is currently non-functional.
*/
+#include <linux/gfp.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/kernel.h>
struct audio_operations *adev = audio_devs[dev];
if (enable_bits & PCM_ENABLE_OUTPUT) {
- if (!(adev->flags & DMA_ACTIVE)) {
+ if (!(adev->dmap_out->flags & DMA_ACTIVE)) {
unsigned long flags;
local_irq_save(flags);
/* prevent recusion */
- adev->flags |= DMA_ACTIVE;
+ adev->dmap_out->flags |= DMA_ACTIVE;
dma_interrupt = vidc_audio_dma_interrupt;
vidc_sound_dma_irq(0, NULL);
#include <linux/wait.h>
#include <linux/interrupt.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <asm/visws/cobalt.h>
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/sched.h>
#include <linux/interrupt.h>
#include <linux/delay.h>
0x10140523, /* Thinkpad R40 */
0x10140534, /* Thinkpad X31 */
0x10140537, /* Thinkpad T41p */
+ 0x1014053e, /* Thinkpad R40e */
0x10140554, /* Thinkpad T42p/R50p */
0x10140567, /* Thinkpad T43p 2668-G7U */
0x10140581, /* Thinkpad X41-2527 */
0x10280160, /* Dell Dimension 2400 */
0x104380b0, /* Asus A7V8X-MX */
0x11790241, /* Toshiba Satellite A-15 S127 */
+ 0x1179ff10, /* Toshiba P500 */
0x144dc01a, /* Samsung NP-X20C004/SEG */
0 /* end */
};
*
*/
-#include <linux/slab.h>
#include <linux/mutex.h>
#include <sound/core.h>
#include <asm/io.h>
#include <linux/init.h>
#include <linux/pci.h>
-#include <linux/slab.h>
#include <linux/gameport.h>
#include <linux/moduleparam.h>
#include <linux/dma-mapping.h>
#include <linux/init.h>
#include <linux/pci.h>
-#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/delay.h>
#include <asm/system.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/interrupt.h>
-#include <linux/slab.h>
#include <linux/moduleparam.h>
#include <sound/core.h>
#include <sound/initval.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/interrupt.h>
-#include <linux/slab.h>
#include <linux/moduleparam.h>
#include <sound/core.h>
#include <sound/initval.h>
struct snd_pcm_substream *substream)
{
size_t ptr;
- unsigned int reg;
+ unsigned int reg, rem, tries;
+
if (!rec->running)
return 0;
#if 1 // this seems better..
reg = rec->ch ? CM_REG_CH1_FRAME2 : CM_REG_CH0_FRAME2;
- ptr = rec->dma_size - (snd_cmipci_read_w(cm, reg) + 1);
- ptr >>= rec->shift;
+ for (tries = 0; tries < 3; tries++) {
+ rem = snd_cmipci_read_w(cm, reg);
+ if (rem < rec->dma_size)
+ goto ok;
+ }
+ printk(KERN_ERR "cmipci: invalid PCM pointer: %#x\n", rem);
+ return SNDRV_PCM_POS_XRUN;
+ok:
+ ptr = (rec->dma_size - (rem + 1)) >> rec->shift;
#else
reg = rec->ch ? CM_REG_CH1_FRAME1 : CM_REG_CH0_FRAME1;
ptr = snd_cmipci_read(cm, reg) - rec->offset;
#include <linux/delay.h>
#include <linux/moduleparam.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/sb.h>
#include <sound/initval.h>
*/
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/pci.h>
#include <sound/core.h>
#include <sound/control.h>
*/
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/delay.h>
#include <sound/core.h>
#include "ctdaio.h"
#include "cttimer.h"
#include <linux/delay.h>
+#include <linux/slab.h>
#include <sound/pcm.h>
#include <sound/control.h>
#include <sound/asoundef.h>
#include "ctpcm.h"
#include "cttimer.h"
+#include <linux/slab.h>
#include <sound/pcm.h>
/* Hardware descriptions for playback */
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
-#include <linux/slab.h>
#include <linux/moduleparam.h>
#include <linux/firmware.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/info.h>
#include <sound/control.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
-#include <linux/slab.h>
#include <linux/moduleparam.h>
#include <linux/firmware.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/info.h>
#include <sound/control.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
-#include <linux/slab.h>
#include <linux/moduleparam.h>
#include <linux/firmware.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/info.h>
#include <sound/control.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
-#include <linux/slab.h>
#include <linux/moduleparam.h>
#include <linux/firmware.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/info.h>
#include <sound/control.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
-#include <linux/slab.h>
#include <linux/moduleparam.h>
#include <linux/firmware.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/info.h>
#include <sound/control.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
-#include <linux/slab.h>
#include <linux/moduleparam.h>
#include <linux/firmware.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/info.h>
#include <sound/control.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
-#include <linux/slab.h>
#include <linux/moduleparam.h>
#include <linux/firmware.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/info.h>
#include <sound/control.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
-#include <linux/slab.h>
#include <linux/moduleparam.h>
#include <linux/firmware.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/info.h>
#include <sound/control.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
-#include <linux/slab.h>
#include <linux/moduleparam.h>
#include <linux/firmware.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/info.h>
#include <sound/control.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
-#include <linux/slab.h>
#include <linux/moduleparam.h>
#include <linux/firmware.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/info.h>
#include <sound/control.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
-#include <linux/slab.h>
#include <linux/moduleparam.h>
#include <linux/firmware.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/info.h>
#include <sound/control.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
-#include <linux/slab.h>
#include <linux/moduleparam.h>
#include <linux/firmware.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/info.h>
#include <sound/control.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
-#include <linux/slab.h>
#include <linux/moduleparam.h>
#include <linux/firmware.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/info.h>
#include <sound/control.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
-#include <linux/slab.h>
#include <linux/moduleparam.h>
#include <linux/firmware.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/info.h>
#include <sound/control.h>
*/
#include <linux/pci.h>
+#include <linux/gfp.h>
#include <linux/time.h>
#include <linux/mutex.h>
#include <linux/input.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <linux/workqueue.h>
#include <sound/core.h>
#include "hda_beep.h"
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <asm/unaligned.h>
#include "hda_codec.h"
SND_PCI_QUIRK(0x103c, 0x306d, "HP dv3", POS_FIX_LPIB),
SND_PCI_QUIRK(0x1106, 0x3288, "ASUS M2V-MX SE", POS_FIX_LPIB),
SND_PCI_QUIRK(0x1043, 0x813d, "ASUS P5AD2", POS_FIX_LPIB),
+ SND_PCI_QUIRK(0x1458, 0xa022, "ga-ma770-ud3", POS_FIX_LPIB),
SND_PCI_QUIRK(0x1462, 0x1002, "MSI Wind U115", POS_FIX_LPIB),
SND_PCI_QUIRK(0x1565, 0x820f, "Biostar Microtech", POS_FIX_LPIB),
SND_PCI_QUIRK(0x8086, 0xd601, "eMachines T5212", POS_FIX_LPIB),
#endif
}
spec->vmaster_nid = 0x13;
+
+ switch (codec->subsystem_id >> 16) {
+ case 0x103c:
+ /* HP laptops have really bad sound over 0 dB on NID 0x10.
+ * Fix max PCM level to 0 dB (originally it has 0x1e steps
+ * with 0 dB offset 0x17)
+ */
+ snd_hda_override_amp_caps(codec, 0x10, HDA_INPUT,
+ (0x17 << AC_AMPCAP_OFFSET_SHIFT) |
+ (0x17 << AC_AMPCAP_NUM_STEPS_SHIFT) |
+ (0x05 << AC_AMPCAP_STEP_SIZE_SHIFT) |
+ (1 << AC_AMPCAP_MUTE_SHIFT));
+ break;
+ }
+
return 0;
}
* patch entries
*/
static struct hda_codec_preset snd_hda_preset_nvhdmi[] = {
- { .id = 0x10de0067, .name = "MCP67 HDMI", .patch = patch_nvhdmi_2ch },
- { .id = 0x10de8001, .name = "MCP73 HDMI", .patch = patch_nvhdmi_2ch },
{ .id = 0x10de0002, .name = "MCP77/78 HDMI",
.patch = patch_nvhdmi_8ch_7x },
{ .id = 0x10de0003, .name = "MCP77/78 HDMI",
.patch = patch_nvhdmi_8ch_7x },
{ .id = 0x10de0007, .name = "MCP79/7A HDMI",
.patch = patch_nvhdmi_8ch_7x },
- { .id = 0x10de000c, .name = "MCP89 HDMI",
+ { .id = 0x10de000a, .name = "GT220 HDMI",
.patch = patch_nvhdmi_8ch_89 },
{ .id = 0x10de000b, .name = "GT21x HDMI",
.patch = patch_nvhdmi_8ch_89 },
+ { .id = 0x10de000c, .name = "MCP89 HDMI",
+ .patch = patch_nvhdmi_8ch_89 },
{ .id = 0x10de000d, .name = "GT240 HDMI",
.patch = patch_nvhdmi_8ch_89 },
+ { .id = 0x10de0067, .name = "MCP67 HDMI", .patch = patch_nvhdmi_2ch },
+ { .id = 0x10de8001, .name = "MCP73 HDMI", .patch = patch_nvhdmi_2ch },
{} /* terminator */
};
MODULE_ALIAS("snd-hda-codec-id:10de0005");
MODULE_ALIAS("snd-hda-codec-id:10de0006");
MODULE_ALIAS("snd-hda-codec-id:10de0007");
-MODULE_ALIAS("snd-hda-codec-id:10de0067");
-MODULE_ALIAS("snd-hda-codec-id:10de8001");
-MODULE_ALIAS("snd-hda-codec-id:10de000c");
+MODULE_ALIAS("snd-hda-codec-id:10de000a");
MODULE_ALIAS("snd-hda-codec-id:10de000b");
+MODULE_ALIAS("snd-hda-codec-id:10de000c");
MODULE_ALIAS("snd-hda-codec-id:10de000d");
+MODULE_ALIAS("snd-hda-codec-id:10de0067");
+MODULE_ALIAS("snd-hda-codec-id:10de8001");
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("NVIDIA HDMI HD-audio codec");
return err;
}
- alc_free_kctls(codec); /* no longer needed */
-
/* assign Capture Source enums to NID */
kctl = snd_hda_find_mixer_ctl(codec, "Capture Source");
if (!kctl)
}
}
}
+
+ alc_free_kctls(codec); /* no longer needed */
+
return 0;
}
alc_set_pin_output(codec, nid, pin_type);
if (spec->multiout.dac_nids[dac_idx] == 0x25)
idx = 4;
- else
+ else {
+ if (spec->multiout.num_dacs >= dac_idx)
+ return;
idx = spec->multiout.dac_nids[dac_idx] - 2;
+ }
snd_hda_codec_write(codec, nid, 0, AC_VERB_SET_CONNECT_SEL, idx);
}
"HP HDX", STAC_HP_HDX), /* HDX16 */
SND_PCI_QUIRK_MASK(PCI_VENDOR_ID_HP, 0xfff0, 0x3620,
"HP dv6", STAC_HP_DV5),
+ SND_PCI_QUIRK(PCI_VENDOR_ID_HP, 0x3061,
+ "HP dv6", STAC_HP_DV5), /* HP dv6-1110ax */
SND_PCI_QUIRK_MASK(PCI_VENDOR_ID_HP, 0xfff0, 0x7010,
"HP", STAC_HP_DV5),
SND_PCI_QUIRK(PCI_VENDOR_ID_DELL, 0x0233,
#include <asm/io.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <sound/core.h>
#include <sound/initval.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <sound/core.h>
#include "ice1712.h"
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <sound/core.h>
#include "ice1712.h"
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <sound/core.h>
#include "ice1712.h"
#include <linux/init.h>
#include <linux/pci.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <sound/initval.h>
#include <sound/control.h>
#include <linux/dma-mapping.h>
#include <linux/moduleparam.h>
#include <linux/mutex.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/initval.h>
#include <linux/pci.h>
#include <linux/firmware.h>
#include <linux/vmalloc.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <sound/core.h>
#include "mixart.h"
#include <linux/interrupt.h>
#include <linux/mutex.h>
#include <linux/pci.h>
+#include <linux/slab.h>
#include <sound/ac97_codec.h>
#include <sound/asoundef.h>
#include <sound/core.h>
#include <linux/delay.h>
+#include <linux/gfp.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
-#include <linux/slab.h>
#include <linux/moduleparam.h>
#include <sound/core.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
-#include <linux/slab.h>
#include <linux/moduleparam.h>
#include <sound/core.h>
#include <linux/init.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
-#include <linux/slab.h>
#include <linux/pci.h>
#include <linux/firmware.h>
#include <linux/moduleparam.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/pci.h>
-#include <linux/slab.h>
#include <linux/moduleparam.h>
#include <sound/core.h>
#include <linux/init.h>
#include <linux/pci.h>
#include <linux/time.h>
+#include <linux/slab.h>
#include <linux/moduleparam.h>
#include <linux/interrupt.h>
#include <linux/delay.h>
*/
#include <linux/delay.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/info.h>
#include "pdaudiocf.h"
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
-#include <linux/slab.h>
#include <linux/delay.h>
#include <sound/core.h>
#include <sound/asoundef.h>
#include <linux/init.h>
#include <linux/moduleparam.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include "vxpocket.h"
#include <pcmcia/ciscode.h>
#include <asm/io.h>
#include <linux/init.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <sound/core.h>
#include "pmac.h"
#include <linux/init.h>
#include <linux/i2c.h>
#include <linux/delay.h>
-#include <linux/slab.h>
#include <sound/core.h>
#include "pmac.h"
#include <linux/dma-mapping.h>
#include <linux/dmapool.h>
+#include <linux/gfp.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/io.h>
-#include <linux/slab.h>
#include <sound/asound.h>
#include <sound/control.h>
#include <linux/interrupt.h>
#include <linux/io.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/initval.h>
#include <sound/pcm.h>
#include <linux/init.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/device.h>
#include <linux/delay.h>
#include <linux/mutex.h>
#include <linux/init.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/suspend.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/platform_device.h>
-#include <linux/slab.h>
#include <linux/dma-mapping.h>
+#include <linux/gfp.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <linux/interrupt.h>
#include <linux/wait.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/platform_device.h>
-#include <linux/slab.h>
#include <linux/dma-mapping.h>
+#include <linux/gfp.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <linux/module.h>
#include <linux/init.h>
#include <linux/platform_device.h>
-#include <linux/slab.h>
#include <linux/dma-mapping.h>
+#include <linux/gfp.h>
#include <sound/core.h>
#include <sound/pcm.h>
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/kernel.h>
#include <linux/device.h>
#include <sound/core.h>
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/device.h>
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/device.h>
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/device.h>
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/device.h>
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/device.h>
#include <sound/core.h>
*/
#include <linux/module.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/soc.h>
#include <sound/initval.h>
#include <linux/pm.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/pm.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/init.h>
#include <linux/i2c.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <sound/soc.h>
#include <sound/soc-dapm.h>
#include <sound/initval.h>
#include <linux/module.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/soc.h>
#include <sound/initval.h>
*/
#include <linux/tty.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/initval.h>
#include <linux/pm.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/kernel.h>
#include <linux/device.h>
#include <linux/gpio.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/initval.h>
#include <linux/pm.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/device.h>
#include <sound/core.h>
#include <linux/pm.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/device.h>
#include <linux/sysfs.h>
#include <linux/spi/spi.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/pm.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/interrupt.h>
#include <linux/gpio.h>
#include <linux/regulator/consumer.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
if (dac33->fifo_mode) {
/* Generic for all FIFO modes */
/* 50-51 : ASRC Control registers */
- dac33_write(codec, DAC33_ASRC_CTRL_A, (1 << 4)); /* div=2 */
+ dac33_write(codec, DAC33_ASRC_CTRL_A, DAC33_SRCLKDIV(1));
dac33_write(codec, DAC33_ASRC_CTRL_B, 1); /* ??? */
/* Write registers 0x34 and 0x35 (MSB, LSB) */
case SND_SOC_DAIFMT_DSP_A:
aictrl_a |= DAC33_AFMT_DSP;
aictrl_b &= ~DAC33_DATA_DELAY_MASK;
- aictrl_b |= DAC33_DATA_DELAY(1); /* 1 bit delay */
- break;
- case SND_SOC_DAIFMT_DSP_B:
- aictrl_a |= DAC33_AFMT_DSP;
- aictrl_b &= ~DAC33_DATA_DELAY_MASK; /* No delay */
+ aictrl_b |= DAC33_DATA_DELAY(0);
break;
case SND_SOC_DAIFMT_RIGHT_J:
aictrl_a |= DAC33_AFMT_RIGHT_J;
{
/* 44-46: DAC Control Registers */
/* A : DAC sample rate Fsref/1.5 */
- dac33_write(codec, DAC33_DAC_CTRL_A, DAC33_DACRATE(1));
+ dac33_write(codec, DAC33_DAC_CTRL_A, DAC33_DACRATE(0));
/* B : DAC src=normal, not muted */
dac33_write(codec, DAC33_DAC_CTRL_B, DAC33_DACSRCR_RIGHT |
DAC33_DACSRCL_LEFT);
#include <linux/i2c.h>
#include <linux/gpio.h>
#include <linux/regulator/consumer.h>
+#include <linux/slab.h>
#include <sound/tpa6130a2-plat.h>
#include <sound/soc.h>
#include <sound/soc-dapm.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
#include <linux/i2c/twl.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/module.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <sound/soc.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
#include <linux/debugfs.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/pm.h>
#include <linux/platform_device.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/delay.h>
#include <linux/pm.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
#include <linux/spi/spi.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
#include <linux/regulator/consumer.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
#include <linux/regulator/consumer.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
#include <linux/spi/spi.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/device.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
#include <linux/spi/spi.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/delay.h>
#include <linux/pm.h>
#include <linux/i2c.h>
+#include <linux/slab.h>
#include <linux/platform_device.h>
#include <linux/regulator/consumer.h>
#include <linux/spi/spi.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
#include <linux/spi/spi.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
#include <linux/spi/spi.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
#include <linux/spi/spi.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/pm.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/pm.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
#include <linux/regulator/consumer.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
#include <linux/spi/spi.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
#include <linux/regulator/consumer.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/pm.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/pm.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/pm.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/pm.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/pm.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/i2c.h>
#include <linux/spi/spi.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/pm.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/i2c.h>
#include <linux/regulator/consumer.h>
#include <linux/spi/spi.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
#include <linux/regulator/consumer.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/pm.h>
#include <linux/i2c.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/device.h>
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/device.h>
*/
#include <linux/init.h>
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/device.h>
#include <sound/core.h>
msleep(1);
reg = snd_soc_read(codec, WM8993_DC_SERVO_READBACK_0);
dev_dbg(codec->dev, "DC servo: %x\n", reg);
- } while (reg & WM8993_DCS_DATAPATH_BUSY);
+ } while (reg & WM8993_DCS_DATAPATH_BUSY && count < 400);
if (reg & WM8993_DCS_DATAPATH_BUSY)
dev_err(codec->dev, "Timed out waiting for DC Servo\n");
#include <linux/init.h>
#include <linux/module.h>
#include <linux/device.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/io.h>
#include <linux/clk.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/device.h>
+#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/io.h>
#include <linux/clk.h>
#include <linux/dma-mapping.h>
#include <linux/interrupt.h>
#include <linux/delay.h>
+#include <linux/gfp.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <linux/interrupt.h>
#include <linux/device.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <linux/module.h>
#include <linux/of_device.h>
+#include <linux/slab.h>
#include <sound/soc.h>
* express or implied.
*/
+#include <linux/slab.h>
#include <linux/module.h>
#include <linux/interrupt.h>
#include <linux/of_device.h>
#include <linux/bitops.h>
#include <linux/platform_device.h>
#include <linux/of.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
config SND_IMX_SOC
tristate "SoC Audio for Freescale i.MX CPUs"
- depends on ARCH_MXC && BROKEN
+ depends on ARCH_MXC
select SND_PCM
select FIQ
select SND_SOC_AC97_BUS
#include <linux/interrupt.h>
#include <linux/module.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/initval.h>
#include <linux/interrupt.h>
#include <linux/module.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/initval.h>
#include <linux/interrupt.h>
#include <linux/module.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/initval.h>
#include <linux/device.h>
#include <linux/platform_device.h>
#include <linux/wait.h>
+#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/err.h>
#include <linux/clk.h>
*/
#include <linux/dma-mapping.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/init.h>
#include <linux/module.h>
+#include <linux/slab.h>
#include <linux/platform_device.h>
#include <linux/clk.h>
#include <linux/io.h>
#include <linux/clk.h>
#include <linux/interrupt.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
select DMA_ENGINE
select DMADEVICES
select SH_DMAE
+ select FW_LOADER
##
## Boards
*/
#include <linux/module.h>
+#include <linux/gfp.h>
#include <linux/init.h>
#include <linux/platform_device.h>
#include <linux/dma-mapping.h>
#include <linux/list.h>
#include <linux/pm_runtime.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/initval.h>
#include <linux/delay.h>
#include <linux/firmware.h>
#include <linux/pm_runtime.h>
+#include <linux/slab.h>
#include <asm/clock.h>
#include <asm/siu.h>
#include <linux/interrupt.h>
#include <linux/module.h>
#include <linux/platform_device.h>
-#include <linux/slab.h>
#include <sound/control.h>
#include <sound/core.h>
#include <linux/bitops.h>
#include <linux/debugfs.h>
#include <linux/platform_device.h>
+#include <linux/slab.h>
#include <sound/ac97_codec.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <linux/platform_device.h>
#include <linux/jiffies.h>
#include <linux/debugfs.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/delay.h>
#include <linux/interrupt.h>
#include <linux/io.h>
+#include <linux/gfp.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/soc.h>
#include <linux/init.h>
#include <linux/platform_device.h>
#include <linux/scatterlist.h>
+#include <linux/slab.h>
#include <sound/core.h>
#include <sound/pcm.h>
#include <sound/pcm_params.h>
#include <linux/module.h>
#include <linux/fs.h>
#include <linux/mm.h>
-#include <linux/slab.h>
#include <linux/sched.h>
#include <asm/uaccess.h>
#include "oss/sound_firmware.h"
#include <linux/module.h>
#include <linux/kernel.h>
-#include <linux/slab.h>
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/interrupt.h>
#include <linux/irq.h>
#include <linux/io.h>
#include <linux/dma-mapping.h>
+#include <linux/gfp.h>
#include <sound/core.h>
#include <sound/pcm.h>
*/
#include <linux/wait.h>
-#include <linux/slab.h>
#include <sound/core.h>
#include <sound/emux_synth.h>
#include <sound/info.h>
*/
#include <linux/spinlock.h>
+#include <linux/slab.h>
#include <linux/init.h>
#include <linux/usb.h>
#include <sound/core.h>
#include <linux/interrupt.h>
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/gfp.h>
#include <linux/usb.h>
#include <sound/initval.h>
#include <sound/core.h>
*/
#include <linux/usb.h>
+#include <linux/gfp.h>
#include <sound/rawmidi.h>
#include <sound/core.h>
#include <sound/pcm.h>
* Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/
+#include <linux/slab.h>
#include <linux/usb.h>
#include <linux/usb/audio.h>
#include <sound/core.h>
*/
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include <sound/core.h>
#include <sound/memalloc.h>
*/
#include <linux/usb.h>
+#include <linux/gfp.h>
#include "usb_stream.h"
#include <linux/init.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
+#include <linux/slab.h>
#include <linux/interrupt.h>
#include <linux/usb.h>
#include <sound/core.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <linux/usb.h>
#include <sound/core.h>
#include <sound/info.h>
*/
#include <linux/delay.h>
+#include <linux/gfp.h>
#include "usbusx2yaudio.c"
#if defined(USX2Y_NRPACKS_VARIABLE) || (!defined(USX2Y_NRPACKS_VARIABLE) && USX2Y_NRPACKS == 1)
CFLAGS = -ggdb3 -Wall -Wextra -std=gnu99 -Werror $(CFLAGS_OPTIMIZE) -D_FORTIFY_SOURCE=2 $(EXTRA_WARNINGS) $(EXTRA_CFLAGS)
EXTLIBS = -lpthread -lrt -lelf -lm
-ALL_CFLAGS = $(CFLAGS)
+ALL_CFLAGS = $(CFLAGS) -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64
ALL_LDFLAGS = $(LDFLAGS)
STRIP ?= strip
PTHREAD_LIBS =
endif
-ifeq ($(shell sh -c "(echo '\#include <libelf.h>'; echo 'int main(void) { Elf * elf = elf_begin(0, ELF_C_READ, 0); return (long)elf; }') | $(CC) -x c - $(ALL_CFLAGS) -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -o $(BITBUCKET) $(ALL_LDFLAGS) $(EXTLIBS) "$(QUIET_STDERR)" && echo y"), y)
-ifneq ($(shell sh -c "(echo '\#include <gnu/libc-version.h>'; echo 'int main(void) { const char * version = gnu_get_libc_version(); return (long)version; }') | $(CC) -x c - $(ALL_CFLAGS) -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -o $(BITBUCKET) $(ALL_LDFLAGS) $(EXTLIBS) "$(QUIET_STDERR)" && echo y"), y)
+ifeq ($(shell sh -c "(echo '\#include <libelf.h>'; echo 'int main(void) { Elf * elf = elf_begin(0, ELF_C_READ, 0); return (long)elf; }') | $(CC) -x c - $(ALL_CFLAGS) -o $(BITBUCKET) $(ALL_LDFLAGS) $(EXTLIBS) "$(QUIET_STDERR)" && echo y"), y)
+ifneq ($(shell sh -c "(echo '\#include <gnu/libc-version.h>'; echo 'int main(void) { const char * version = gnu_get_libc_version(); return (long)version; }') | $(CC) -x c - $(ALL_CFLAGS) -o $(BITBUCKET) $(ALL_LDFLAGS) $(EXTLIBS) "$(QUIET_STDERR)" && echo y"), y)
msg := $(error No gnu/libc-version.h found, please install glibc-dev[el]/glibc-static);
endif
- ifneq ($(shell sh -c "(echo '\#include <libelf.h>'; echo 'int main(void) { Elf * elf = elf_begin(0, ELF_C_READ_MMAP, 0); return (long)elf; }') | $(CC) -x c - $(ALL_CFLAGS) -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -o $(BITBUCKET) $(ALL_LDFLAGS) $(EXTLIBS) "$(QUIET_STDERR)" && echo y"), y)
+ ifneq ($(shell sh -c "(echo '\#include <libelf.h>'; echo 'int main(void) { Elf * elf = elf_begin(0, ELF_C_READ_MMAP, 0); return (long)elf; }') | $(CC) -x c - $(ALL_CFLAGS) -o $(BITBUCKET) $(ALL_LDFLAGS) $(EXTLIBS) "$(QUIET_STDERR)" && echo y"), y)
BASIC_CFLAGS += -DLIBELF_NO_MMAP
endif
else
msg := $(error No libelf.h/libelf found, please install libelf-dev/elfutils-libelf-devel and glibc-dev[el]);
endif
-ifneq ($(shell sh -c "(echo '\#include <dwarf.h>'; echo '\#include <libdw.h>'; echo 'int main(void) { Dwarf *dbg; dbg = dwarf_begin(0, DWARF_C_READ); return (long)dbg; }') | $(CC) -x c - $(ALL_CFLAGS) -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/elfutils -ldw -lelf -o $(BITBUCKET) $(ALL_LDFLAGS) $(EXTLIBS) "$(QUIET_STDERR)" && echo y"), y)
+ifneq ($(shell sh -c "(echo '\#include <dwarf.h>'; echo '\#include <libdw.h>'; echo 'int main(void) { Dwarf *dbg; dbg = dwarf_begin(0, DWARF_C_READ); return (long)dbg; }') | $(CC) -x c - $(ALL_CFLAGS) -I/usr/include/elfutils -ldw -lelf -o $(BITBUCKET) $(ALL_LDFLAGS) $(EXTLIBS) "$(QUIET_STDERR)" && echo y"), y)
msg := $(warning No libdw.h found or old libdw.h found, disables dwarf support. Please install elfutils-devel/elfutils-dev);
BASIC_CFLAGS += -DNO_DWARF_SUPPORT
else
#include "util/debug.h"
#include <linux/rbtree.h>
+#include <linux/slab.h>
struct alloc_stat;
typedef int (*sort_fn_t)(struct alloc_stat *, struct alloc_stat *);
#include "util/probe-event.h"
#define MAX_PATH_LEN 256
-#define MAX_PROBES 128
/* Session management structure */
static struct {
struct sym_entry *syme, *n;
struct rb_root tmp = RB_ROOT;
struct rb_node *nd;
- int sym_width = 0, dso_width = 0, max_dso_width;
+ int sym_width = 0, dso_width = 0, dso_short_width = 0;
const int win_width = winsize.ws_col - 1;
samples = userspace_samples = 0;
if (syme->map->dso->long_name_len > dso_width)
dso_width = syme->map->dso->long_name_len;
+ if (syme->map->dso->short_name_len > dso_short_width)
+ dso_short_width = syme->map->dso->short_name_len;
+
if (syme->name_len > sym_width)
sym_width = syme->name_len;
}
printed = 0;
- max_dso_width = winsize.ws_col - sym_width - 29;
- if (dso_width > max_dso_width)
- dso_width = max_dso_width;
+ if (sym_width + dso_width > winsize.ws_col - 29) {
+ dso_width = dso_short_width;
+ if (sym_width + dso_width > winsize.ws_col - 29)
+ sym_width = winsize.ws_col - dso_width - 29;
+ }
putchar('\n');
if (nr_counters == 1)
printf(" samples pcnt");
/* Parse probe point */
parse_perf_probe_probepoint(argv[0], pp);
- if (pp->file || pp->line)
+ if (pp->file || pp->line || pp->lazy_line)
*need_dwarf = true;
/* Copy arguments and ensure return probe has no C argument */
die("%u exceeds max register number.", regn);
if (deref)
- ret = snprintf(pf->buf, pf->len, " %s=+%ju(%s)",
- pf->var, (uintmax_t)offs, regs);
+ ret = snprintf(pf->buf, pf->len, " %s=%+jd(%s)",
+ pf->var, (intmax_t)offs, regs);
else
ret = snprintf(pf->buf, pf->len, " %s=%s", pf->var, regs);
DIE_IF(ret < 0);
if (dwarf_attr(vr_die, DW_AT_location, &attr) == NULL)
goto error;
/* TODO: handle more than 1 exprs */
- ret = dwarf_getlocation_addr(&attr, (pf->addr - pf->cu_base),
- &expr, &nexpr, 1);
+ ret = dwarf_getlocation_addr(&attr, pf->addr, &expr, &nexpr, 1);
if (ret <= 0 || nexpr == 0)
goto error;
/* Get the frame base attribute/ops */
dwarf_attr(sp_die, DW_AT_frame_base, &fb_attr);
- ret = dwarf_getlocation_addr(&fb_attr, (pf->addr - pf->cu_base),
- &pf->fb_ops, &nops, 1);
+ ret = dwarf_getlocation_addr(&fb_attr, pf->addr, &pf->fb_ops, &nops, 1);
if (ret <= 0 || nops == 0)
pf->fb_ops = NULL;
/* *pf->fb_ops will be cached in libdw. Don't free it. */
pf->fb_ops = NULL;
+ if (pp->found == MAX_PROBES)
+ die("Too many( > %d) probe point found.\n", MAX_PROBES);
+
pp->probes[pp->found] = strdup(tmp);
pp->found++;
}
int find_probe_point(int fd, struct probe_point *pp)
{
struct probe_finder pf = {.pp = pp};
- int ret;
Dwarf_Off off, noff;
size_t cuhl;
Dwarf_Die *diep;
pf.fname = NULL;
if (!pp->file || pf.fname) {
- /* Save CU base address (for frame_base) */
- ret = dwarf_lowpc(&pf.cu_die, &pf.cu_base);
- if (ret != 0)
- pf.cu_base = 0;
if (pp->function)
find_probe_point_by_func(&pf);
else if (pp->lazy_line)
/* For variable searching */
Dwarf_Op *fb_ops; /* Frame base attribute */
- Dwarf_Addr cu_base; /* Current CU base address */
const char *var; /* Current variable name */
char *buf; /* Current output buffer */
int len; /* Length of output buffer */
int size __unused,
unsigned long long nsecs, char *comm)
{
- PyObject *handler, *retval, *context, *t;
+ PyObject *handler, *retval, *context, *t, *obj;
static char handler_name[256];
struct format_field *field;
unsigned long long val;
offset &= 0xffff;
} else
offset = field->offset;
- PyTuple_SetItem(t, n++,
- PyString_FromString((char *)data + offset));
+ obj = PyString_FromString((char *)data + offset);
} else { /* FIELD_IS_NUMERIC */
val = read_size(data + field->offset, field->size);
if (field->flags & FIELD_IS_SIGNED) {
- PyTuple_SetItem(t, n++, PyInt_FromLong(val));
+ if ((long long)val >= LONG_MIN &&
+ (long long)val <= LONG_MAX)
+ obj = PyInt_FromLong(val);
+ else
+ obj = PyLong_FromLongLong(val);
} else {
- PyTuple_SetItem(t, n++, PyInt_FromLong(val));
+ if (val <= LONG_MAX)
+ obj = PyInt_FromLong(val);
+ else
+ obj = PyLong_FromUnsignedLongLong(val);
}
}
+ PyTuple_SetItem(t, n++, obj);
}
if (_PyTuple_Resize(&t, n) == -1)
self->long_name_len = strlen(name);
}
+static void dso__set_short_name(struct dso *self, const char *name)
+{
+ if (name == NULL)
+ return;
+ self->short_name = name;
+ self->short_name_len = strlen(name);
+}
+
static void dso__set_basename(struct dso *self)
{
- self->short_name = basename(self->long_name);
+ dso__set_short_name(self, basename(self->long_name));
}
struct dso *dso__new(const char *name)
int i;
strcpy(self->name, name);
dso__set_long_name(self, self->name);
- self->short_name = self->name;
+ dso__set_short_name(self, self->name);
for (i = 0; i < MAP__NR_TYPES; ++i)
self->symbols[i] = self->symbol_names[i] = RB_ROOT;
self->slen_calculated = 0;
struct kmap *kmap = self->kernel ? map__kmap(map) : NULL;
struct map *curr_map = map;
struct dso *curr_dso = self;
- size_t dso_name_len = strlen(self->short_name);
Elf_Data *symstrs, *secstrs;
uint32_t nr_syms;
int err = -1;
char dso_name[PATH_MAX];
if (strcmp(section_name,
- curr_dso->short_name + dso_name_len) == 0)
+ (curr_dso->short_name +
+ self->short_name_len)) == 0)
goto new_symbol;
if (strcmp(section_name, ".text") == 0) {
struct dso *self = dso__new(name ?: "[kernel.kallsyms]");
if (self != NULL) {
- self->short_name = "[kernel]";
+ dso__set_short_name(self, "[kernel]");
self->kernel = 1;
}
u8 sorted_by_name;
u8 loaded;
u8 build_id[BUILD_ID_SIZE];
- u16 long_name_len;
const char *short_name;
char *long_name;
+ u16 long_name_len;
+ u16 short_name_len;
char name[0];
};
#include <linux/spinlock.h>
#include <linux/pci.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include "irq.h"
static struct kvm_assigned_dev_kernel *kvm_find_assigned_dev(struct list_head *head,
#include "iodev.h"
#include <linux/kvm_host.h>
+#include <linux/slab.h>
#include <linux/kvm.h>
#include "coalesced_mmio.h"
#include <linux/list.h>
#include <linux/eventfd.h>
#include <linux/kernel.h>
+#include <linux/slab.h>
#include "iodev.h"
#include <linux/smp.h>
#include <linux/hrtimer.h>
#include <linux/io.h>
+#include <linux/slab.h>
#include <asm/processor.h>
#include <asm/page.h>
#include <asm/current.h>
*/
#include <linux/kvm_host.h>
+#include <linux/slab.h>
#include <trace/events/kvm.h>
#include <asm/msidef.h>
#include <linux/module.h>
#include <linux/errno.h>
#include <linux/percpu.h>
-#include <linux/gfp.h>
#include <linux/mm.h>
#include <linux/miscdevice.h>
#include <linux/vmalloc.h>
#include <linux/compat.h>
#include <linux/srcu.h>
#include <linux/hugetlb.h>
+#include <linux/slab.h>
#include <asm/processor.h>
#include <asm/io.h>
projects / mirror_ubuntu-bionic-kernel.git / commitdiff