Brian Welty [Thu, 13 Jul 2023 01:25:42 +0000 (18:25 -0700)]
drm/xe: Fix lockdep warning from xe_vm_madvise
We need to hold vm->lock before the xe_vm_is_closed_or_banned().
Else we get this splat:
[ 802.555227] ------------[ cut here ]------------
[ 802.555234] WARNING: CPU: 33 PID: 3122 at drivers/gpu/drm/xe/xe_vm.h:60
[ 802.555515] CPU: 33 PID: 3122 Comm: xe_exec_fault_m Tainted:
...
[ 802.555709] Call Trace:
[ 802.555714] <TASK>
[ 802.555720] ? __warn+0x81/0x170
[ 802.555737] ? xe_vm_madvise_ioctl+0x2de/0x440 [xe]
Fixes: 9d858b69b0cf ("drm/xe: Ban a VM if rebind worker hits an error") Reviewed-by: Matthew Brost <matthew.brost@intel.com> Signed-off-by: Brian Welty <brian.welty@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Fixes: b06d47be7c83 ("drm/xe: Port Xe to GPUVA") Reviewed-by: Matthew Brost <matthew.brost@intel.com> Signed-off-by: Brian Welty <brian.welty@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Matthew Auld [Mon, 26 Jun 2023 17:20:40 +0000 (18:20 +0100)]
drm/xe/mmio: update gt_count when probing multi-tile
It looks like the single-tile PVC in CI dies during module load when doing
the pcode init. From the logs we try to access the address 0000000000138124 which doesn't map to anything, however 0x138124 also
looks to be the PCODE_MAILBOX register. So looks like the per-tile
mmio register mapping is NULL.
During probe the tile count is potentially trimmed, since we don't know
the real count until we actually probe the device. This seems to be
the case for single-tile PVC or similar devices. However it looks like
the gt_count is never adjusted to respect this updated tile count. As a
result when later doing some for_each_gt() loop, like we do for the
pcode, we can get back some GT that maps to some non-existent tile
which hasn't been properly set up, leading to crashes.
Try to fix this by adjusting the gt_count after probing the tiles for
real.
v2: Fix typo so it actually builds
References: https://gitlab.freedesktop.org/drm/xe/kernel/-/issues/383 Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Lucas De Marchi <lucas.demarchi@intel.com> Cc: Matt Roper <matthew.d.roper@intel.com> Reviewed-by: Ofir Bitton <obitton@habana.ai> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Matthew Auld [Mon, 10 Jul 2023 09:40:49 +0000 (10:40 +0100)]
drm/xe: handle TLB invalidations from CT fast-path
In various test cases that put the system under a heavy load, we can
sometimes see errors with missed TLB invalidations. In such cases we see
the interrupt arrive for the invalidation from the GuC, however the
actual processing of the completion is pushed onto a workqueue and
handled with all the other CT stuff, which might take longer than
expected. Since we expect TLB invalidations to complete within a
reasonable amount of time (at most ~250ms), and they do seem pretty
critical, allow handling directly from the CT fast-path.
v2 (José):
- Actually use the correct spinlock/unlock_irq, since pending_lock is
grabbed from IRQ.
v3:
- Don't publish the TLB fence on the list until after we fully
initialize it and successfully do the CT send. The list is now only
protected by the spin_lock pending_lock and we can't hold that
across the entire TLB send operation.
v4 (Matt Brost):
- Be careful with racing against fast CT path writing the seqno,
before we have actually published the fence.
References: https://gitlab.freedesktop.org/drm/xe/kernel/-/issues/297
References: https://gitlab.freedesktop.org/drm/xe/kernel/-/issues/320
References: https://gitlab.freedesktop.org/drm/xe/kernel/-/issues/449 Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Matthew Brost <matthew.brost@intel.com> Cc: José Roberto de Souza <jose.souza@intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Matthew Auld [Mon, 10 Jul 2023 09:40:47 +0000 (10:40 +0100)]
drm/xe/tlb: print seqno_recv on fence TLB timeout
To help debugging, sample the current seqno_recv and dump it out if we
encounter a TLB timeout for the fences path.
Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Matthew Brost <matthew.brost@intel.com> Cc: José Roberto de Souza <jose.souza@intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Matthew Auld [Mon, 10 Jul 2023 09:40:46 +0000 (10:40 +0100)]
drm/xe/tlb: also update seqno_recv during reset
We might have various kworkers waiting for TLB flushes to complete which
are not tracked with an explicit TLB fence, however at this stage that
will never happen since the CT is already disabled, so make sure we
signal them here under the assumption that we have completed a full GT
reset.
v2:
- We need to use seqno - 1 here. After acquiring ct->lock the seqno is
actually the next users seqno and not the pending one.
Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Matthew Brost <matthew.brost@intel.com> Cc: José Roberto de Souza <jose.souza@intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Matthew Auld [Mon, 10 Jul 2023 09:40:45 +0000 (10:40 +0100)]
drm/xe/gt: tweak placement for signalling TLB fences after GT reset
Assumption here is that submission is disabled along with CT, and full
GT reset will also nuke TLBs, so should be safe to signal all in-flight
TLB fences, but only after the actual reset so move the placement
slightly.
Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Matthew Brost <matthew.brost@intel.com> Cc: José Roberto de Souza <jose.souza@intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Matthew Auld [Mon, 10 Jul 2023 09:40:44 +0000 (10:40 +0100)]
drm/xe/ct: serialise fast_lock during CT disable
The fast-path CT could be running as we enter a runtime-suspend or
potentially a GT reset, however here we only use the ct->fast_lock and
not the full ct->lock. Before disabling the CT, also serialise against
the fast_lock to ensure any in-progress work finishes before we start
nuking the CT related stuff. Once we disable ct->enabled and drop the
lock, any new work should fail gracefully, and anything that was in
progress should be finished.
Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Matthew Brost <matthew.brost@intel.com> Cc: José Roberto de Souza <jose.souza@intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Matthew Auld [Mon, 10 Jul 2023 09:40:43 +0000 (10:40 +0100)]
drm/xe/tlb: increment next seqno after successful CT send
If we are in the middle of a GT reset or similar the CT might be
disabled, such that the CT send fails. However we already incremented
gt->tlb_invalidation.seqno which might lead to warnings, since we
effectively just skipped a seqno:
Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Matthew Brost <matthew.brost@intel.com> Cc: José Roberto de Souza <jose.souza@intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Matthew Auld [Mon, 10 Jul 2023 09:40:42 +0000 (10:40 +0100)]
drm/xe/ct: hold fast_lock when reserving space for g2h
Reserving and checking for space on the g2h side relies on the
fast_lock, and not the CT lock since we need to release space from the
fast CT path. Make sure we hold it when checking for space and reserving
it. The main concern is calling __g2h_release_space() as we are reserving
something and since the info.space and info.g2h_outstanding operations
are not atomic we can get some nonsense values back.
Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Matthew Brost <matthew.brost@intel.com> Cc: José Roberto de Souza <jose.souza@intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Matthew Auld [Mon, 10 Jul 2023 09:40:41 +0000 (10:40 +0100)]
drm/xe: hold mem_access.ref for CT fast-path
Just checking xe_device_mem_access_ongoing() is not enough, we also need
to hold the reference otherwise the ref can transition from 1 -> 0 as we
enter g2h_read(), leading to warnings. While we can't do a full rpm sync
in the IRQ, we can keep the device awake if the ref is non-zero.
Introduce a new helper for this and set it to work in for the CT
fast-path.
Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Matthew Brost <matthew.brost@intel.com> Cc: José Roberto de Souza <jose.souza@intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Matthew Auld [Mon, 10 Jul 2023 09:40:40 +0000 (10:40 +0100)]
drm/xe/tlb: ensure we access seqno_recv once
Ensure we load gt->tlb_invalidation.seqno_recv once, and use that for
our seqno checking. The gt->tlb_invalidation_seqno_past is a shared
global variable and can potentially change at any point here. However
the checks here need to operate on a stable version of seqno_recv for
this to make any sense.
Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Matthew Brost <matthew.brost@intel.com> Cc: José Roberto de Souza <jose.souza@intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Matthew Auld [Mon, 10 Jul 2023 09:40:39 +0000 (10:40 +0100)]
drm/xe/tlb: drop unnecessary smp_wmb()
wake_up_all() and wait_event_timeout() already have the correct barriers
as per https://www.kernel.org/doc/Documentation/memory-barriers.txt.
This should ensure that the seqno_recv write can't be re-ordered wrt to
the actual wake_up_all() i.e we get woken up but there is no write. The
reader side with wait_event_timeout() also has the correct barriers.
With that drop the hand rolled smp_wmb(), which is anyway missing some
kind of matching barrier on the reader side.
Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Matthew Brost <matthew.brost@intel.com> Cc: José Roberto de Souza <jose.souza@intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Matthew Brost [Sat, 8 Jul 2023 05:23:57 +0000 (22:23 -0700)]
drm/xe: Port Xe to GPUVA
Rather than open coding VM binds and VMA tracking, use the GPUVA
library. GPUVA provides a common infrastructure for VM binds to use mmap
/ munmap semantics and support for VK sparse bindings.
The concepts are:
1) xe_vm inherits from drm_gpuva_manager
2) xe_vma inherits from drm_gpuva
3) xe_vma_op inherits from drm_gpuva_op
4) VM bind operations (MAP, UNMAP, PREFETCH, UNMAP_ALL) call into the
GPUVA code to generate an VMA operations list which is parsed, committed,
and executed.
v2 (CI): Add break after default in case statement.
v3: Rebase
v4: Fix some error handling
v5: Use unlocked version VMA in error paths
v6: Rebase, address some review feedback mainly Thomas H
v7: Fix compile error in xe_vma_op_unwind, address checkpatch
Signed-off-by: Matthew Brost <matthew.brost@intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Matthew Brost [Thu, 22 Jun 2023 19:39:48 +0000 (12:39 -0700)]
drm/xe: Ban a VM if rebind worker hits an error
We cannot recover a VM if a rebind worker hits an error, ban the VM if
happens to ensure we do not attempt to place this VM on the hardware
again.
A follow up will inform the user if this happens.
v2: Return -ECANCELED in exec VM closed or banned, check for closed or
banned within VM lock.
v3: Fix lockdep splat by looking engine outside of vm->lock
v4: Fix error path when engine lookup fails
v5: Add debug message in rebind worker on error, update comments wrt
locking, add xe_vm_close helper
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Signed-off-by: Matthew Brost <matthew.brost@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Matthew Brost [Mon, 10 Jul 2023 14:41:21 +0000 (07:41 -0700)]
drm/xe: Use internal VM flags in xe_vm_create
xe_vm_create used the IOCTL create flags in a few places rather than the
internal VM flags and this just happened to work as these values
matched. This is risky (and incorrect) as the internal flag values are
free to change. Fix this and use the internal VM flag values.
Signed-off-by: Matthew Brost <matthew.brost@intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Alan Previn [Fri, 2 Jun 2023 18:16:50 +0000 (11:16 -0700)]
drm/xe/guc: Fix h2g_write usage of GUC_CTB_MSG_MAX_LEN
In the ABI header, GUC_CTB_MSG_MIN_LEN is '1' because
GUC_CTB_HDR_LEN is 1. This aligns with H2G/G2H CTB specification
where all command formats are defined in units of dwords so that '1'
is a dword. Accordingly, GUC_CTB_MSG_MAX_LEN is 256-1 (i.e. 255
dwords). However, h2g_write was incorrectly assuming that
GUC_CTB_MSG_MAX_LEN was in bytes. Fix this.
v3: Fix nit on #define location.(Matt)
v2: By correctly treating GUC_CTB_MSG_MAX_LEN as dwords, it causes
a local array to consume 4x the stack size. Rework the function
to avoid consuming stack even if the action size is large. (Matt)
Signed-off-by: Alan Previn <alan.previn.teres.alexis@intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Jani Nikula [Tue, 4 Jul 2023 15:32:41 +0000 (18:32 +0300)]
drm/xe/mmio: add xe_mmio_read16()
Little by little, make stuff feature complete.
Signed-off-by: Jani Nikula <jani.nikula@intel.com> Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Thomas Hellström [Thu, 29 Jun 2023 20:51:33 +0000 (22:51 +0200)]
drm/xe: Make page-table updates using the default engine happen in order
If the default engine m->eng was used, there is no check for idle and
a cpu page-table update may thus happen in parallel with a gpu one.
Don't allow CPU page-table updates with the default engine until
the engine is idle.
Matt Roper [Wed, 14 Jun 2023 20:52:02 +0000 (13:52 -0700)]
drm/xe: Enable PCI device earlier
Newer Intel platforms require that inspect the contents of the GMD_ID
registers very early in the driver initialization process to determine
the IP version (and proper init sequences), of the platform. Move the
general PCI device setup and enablement slightly earlier, before we
start trying to peek at the GMD_ID registers.
Matt Roper [Wed, 14 Jun 2023 20:52:00 +0000 (13:52 -0700)]
drm/xe: Don't raise error on fused-off media
It's legitimate for the media GMD_ID register to read back as 0x0 if
media functionality is fused off or otherwise not present on the
platform. Avoid printing an "unknown media version" error message for
this case.
Matt Roper [Wed, 14 Jun 2023 20:51:59 +0000 (13:51 -0700)]
drm/xe: Return GMD_ID revid properly
peek_gmdid() returns the IP version, not the raw value of the GMD_ID
register. Make sure we extract and return the rev_id field as well so
that it can be used to determine the IP steppings properly.
Tejas Upadhyay [Wed, 28 Jun 2023 06:23:16 +0000 (11:53 +0530)]
drm/xe: Make usable size of VRAM readable
Current size member of vram struct does not give
complete information as what "size" contains. Does
it contain reserved portions or not. Name it usable
size and accordingly describe other size members as
well.
Reviewed-by: Matthew Brost <matthew.brost@intel.com> Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com> Signed-off-by: Tejas Upadhyay <tejas.upadhyay@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Tejas Upadhyay [Wed, 28 Jun 2023 06:06:16 +0000 (11:36 +0530)]
drm/xe: Add sysfs entry to report per tile memory size
Add sysfs entry to read per tile physical memory
including stolen memory.
V5:
- rename var name and make it part of vram struct - Lucas
V4:
- %s/addr_range/physical_vram_size_byes, make it
user readable name - Joonas/Aravind
- Display in bytes - Joonas/Aravind
V3:
- Exclude DG1, replace sysfs_create_file/files - Aravind
V2:
- Use DEVICE_ATTR_RO - Aravind
- Dont put kobj on sysfs_file_create fail - Himal
- Skip addr_range sysfs create for non dgfx - Himal
Tejas Upadhyay [Tue, 6 Jun 2023 10:18:38 +0000 (15:48 +0530)]
drm/xe: Add GTs under respective tile sysfs
With the separation of xe_tile and xe_gt, We now consider
a PCI device (xe_device) to contain one or more tiles (struct xe_tile).
Each tile will contain one or more GTs (struct xe_gt).
So lets align sysfs paths accordingly.
TODO: Currently we have gt0 under tile0 and gt1 under tile1
on multi-tile. This GT indexing still under discussion, when
it is concluded we need to revisit this change.
drm/xe: Use nanoseconds instead of jiffies in uapi for user fence
Using jiffies as a timeout from userspace is weird even if
theoretically exists possiblity of acquiring jiffies via getconf.
Unfortunately this method is unreliable and the returned
value may vary from the one configured in the kernel config.
Now timeout is expressed in nanoseconds and its interpretation depends
on setting DRM_XE_UFENCE_WAIT_ABSTIME flag. Relative timeout (flag
is not set) means fence expire at now() + timeout. Absolute timeout
(flag is set) means that the fence expires at exact point of time.
Passing negative timeout means we will wait "forever" by setting
wait time to MAX_SCHEDULE_TIMEOUT.
Paulo Zanoni [Mon, 26 Jun 2023 21:22:21 +0000 (14:22 -0700)]
drm/xe: properly check bounds for xe_wait_user_fence_ioctl()
If !no_engines, then we use copy_from_user to copy to the 'eci' array,
which has XE_HW_ENGINE_MAX_INSTANCE members. The amount of members
copied is given by the user in args->num_engines, so add code to check
that args->num_engines does not exceed XE_HW_ENGINE_MAX_INSTANCE. It's
an unsigned value so there's no need to check for negative values.
Reviewed-by: José Roberto de Souza <jose.souza@intel.com> Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com> Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com> Link: https://lore.kernel.org/r/20230626212221.136640-2-paulo.r.zanoni@intel.com Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Someone is running a program that's definitely submitting random
numbers to this ioctl. If you pass width=65535 and
num_placements=32769 then you get a negative 'len', which avoids the
EINVAL check, leading to the bug.
Switch 'len' to u32. It is the result of the multiplication of two u16
numbers, so it won't be able to overflow back into smaller numbers as
an u32.
v2: Make len u32 instead of checking for <=0 (José).
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com> Reviewed-by: José Roberto de Souza <jose.souza@intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com> Link: https://lore.kernel.org/r/20230626212221.136640-1-paulo.r.zanoni@intel.com Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
drm/xe/slpc: Start SLPC before GuC submission on reset
The SLPC code has a strict 5ms timeout from when the start command is
queued to when we expect the reply to appear in memory. This works if
the CT channel is empty, but if the channel is busy there might be an
extra delay that causes the process to exceeded the timeout. We see
this issue when a reset occurs while userspace keeps submitting,
because the submission code is re-enabled first and it will start using
the channel to service those submissions.
To fix this, we can simply start SLPC before re-enabling submission.
This has also the benefit of not allowing submissions to go through with
an uninitialized SLPC.
Anusha Srivatsa [Tue, 13 Jun 2023 17:47:40 +0000 (10:47 -0700)]
drm/xe: Add missing ADL entries to xe_test_wa
With the fake device creation fix in the previous patch,
adding Alderlake P platform in xe_wa_test.
With this, driver is able to run the kunit test for
ADLP properly.
Cc: Lucas De Marchi <lucas.demarchi@intel.com> Signed-off-by: Anusha Srivatsa <anusha.srivatsa@intel.com> Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com> Link: https://lore.kernel.org/r/20230613174740.786041-2-anusha.srivatsa@intel.com Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Anusha Srivatsa [Tue, 13 Jun 2023 17:47:39 +0000 (10:47 -0700)]
drm/xe/kunit: Handle fake device creation for all platform/subplatform cases
For platform like Alderlake P there are subplatforms and
just Alderlake P. Unlike DG2 in which every flavour is
either a G10,G11 or G12 variant. In this case(Alderlake P/S),
the Kunit test evaluates the subplatform to NONE and is
unable to create a fake device. Removing the condition
in xe_pci_fake_device_init() to support this corner case
so driver can proceed with the unit testing.
Cc: Lucas De Marchi <lucas.demarchi@intel.com> Signed-off-by: Anusha Srivatsa <anusha.srivatsa@intel.com> Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com> Link: https://lore.kernel.org/r/20230613174740.786041-1-anusha.srivatsa@intel.com Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Anshuman Gupta [Wed, 24 May 2023 09:06:53 +0000 (14:36 +0530)]
drm/xe/pm: Disable PM on unbounded pcie parent bridge
Intel Discrete GFX cards gfx may have multiple PCIe endpoints,
they connects to root port via pcie upstream switch port(USP)
and virtual pcie switch port(VSP), sometimes VSP pcie devices
doesn't bind to pcieport driver. Without pcieport driver, pcie PM
comes without any warranty and with unbounded VSP gfx card won't
transition to low power pcie Device and Link states therefore
assert drm_warn on unbounded VSP and disable xe driver
PM support.
v2:
- Disable Xe PCI PM support. [Rodrigo]
v3:
- Changed subject and Rebase.
v4:
- %s/xe_pci_unbounded_bridge_disable_pm/xe_assert_on_unbounded_bridge.
[Rodrigo]
- Use device_set_pm_not_required() instead of dev_pm_ops NULL assignment.
Thomas Hellström [Thu, 25 May 2023 07:41:44 +0000 (09:41 +0200)]
drm/xe: Fix vm refcount races
Fix a race in xe_vm_lookup() where the vm could disappear after
the lookup mutex unlock but before the get. The xe_vm_get() call
must be inside the lookup mutex.
Also fix a vm close race where multiple callers could potentially
succeed in calling xe_vm_close_and_put().
Thomas Hellström [Mon, 26 Jun 2023 18:17:41 +0000 (20:17 +0200)]
drm/xe/bo: Evict VRAM to TT rather than to system
The main difference is that we don't bounce and sync on eviction, allowing
for pipelined eviction. Moving forward we also need to be careful with
dma mappings which can be released in SYSTEM but may remain in TT.
Thomas Hellström [Mon, 26 Jun 2023 18:17:39 +0000 (20:17 +0200)]
drm/xe/bo: Avoid creating a system resource when allocating a fresh VRAM bo
When creating a new bo, on the first move the bo->resource is typically
NULL. Our move callback rejected that instructing TTM to create a system
resource. In addition a struct ttm_tt with a page-vector was created,
although not populated with pages. Similarly when the clearing of VRAM
was complete, the system resource was put on a ghost object and freed
using the TTM delayed destroy mechanism.
This is a lot of pointless work. So avoid creating the system resource and
instead change the code to cope with a NULL bo->resource.
v2:
- Add some code comments (Matthew Brost)
v3:
- Fix a dereference of old_mem which might be NULL.
Thomas Hellström [Mon, 26 Jun 2023 18:17:38 +0000 (20:17 +0200)]
drm/xe/bo: Fix swapin when moving to VRAM
When a source system resource had been swapped out, we incorrectly
assumed that we were lacking source data for a move and therefore
cleared the destination instead of swapping in and copying the
swapped-out data. Fix this.
Riana Tauro [Fri, 23 Jun 2023 05:24:30 +0000 (10:54 +0530)]
drm/xe: add a new sysfs directory for gtidle properties
1) Add a new sysfs directory under devices/gt#/ called gtidle
to contain idle properties of GT such as name, idle_status,
idle_residency_ms
2) Remove forcewake calls for residency counter
v2:
- abstract using function pointers (Anshuman)
- remove forcewake calls for residency counter
- use device_attr (Badal)
- move rc functions to guc_pc
- change name to gt_idle (Rodrigo)
v3:
- return error for drmm_add_action_or_reset
- replace file and functions with gt_idle prefix
to gt_idle_sysfs (Himal)
- use enum for gt idle state
- move multiplier to gt idle and initialize (Anshuman)
- correct doc annotation (Rodrigo)
- remove return variable
- use kobj_gt instead of new gtidle kobj
- move residency_ms to gtidle file
- retain xe_guc_pc prefix for functions in guc_rc file (Michal)
v4:
- fix doc errors in xe_guc_pc file
- change u64 to u32 for reading residency counter
- keep gtidle states generic GT_IDLE_C[0/6] (Anshuman)
v5:
- update commit message to include removal of
forcewake calls (Anshuman)
- return void from sysfs initialization function and add warnings
(Andi)
Matthew Auld [Mon, 19 Jun 2023 11:00:20 +0000 (12:00 +0100)]
drm/xe/bo: consider bo->flags in xe_bo_migrate()
For VRAM allocations the bo->flags can control some characteristics of
the underlying memory, like whether it needs to be contiguous, and in
the future whether it needs to be in the CPU visible portion. Rather use
add_vram() in xe_bo_migrate() which should take care of such things for
us.
Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Cc: José Roberto de Souza <jose.souza@intel.com> Reviewed-by: José Roberto de Souza <jose.souza@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Matthew Auld [Fri, 31 Mar 2023 08:46:25 +0000 (09:46 +0100)]
drm/xe/uapi: add some kernel-doc for region query
Since we need to extend this, we should also take the time to add some
basic kernel-doc here for the existing bits. Note that this is all still
subject to change when upstreaming.
Also convert XE_MEM_REGION_CLASS_* into an enum, so we can more easily
create links to it from other parts of the uapi.
Suggested-by: Gwan-gyeong Mun <gwan-gyeong.mun@intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Cc: Lucas De Marchi <lucas.demarchi@intel.com> Cc: José Roberto de Souza <jose.souza@intel.com> Cc: Filip Hazubski <filip.hazubski@intel.com> Cc: Carl Zhang <carl.zhang@intel.com> Cc: Effie Yu <effie.yu@intel.com> Reviewed-by: Gwan-gyeong Mun <gwan-gyeong.mun@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Matthew Auld [Fri, 31 Mar 2023 08:46:24 +0000 (09:46 +0100)]
drm/xe/uapi: restrict system wide accounting
Since this is considered an info leak (system wide accounting), rather
hide behind perfmon_capable().
v2:
- Without perfmon_capable() it likely makes more sense to report as zero,
instead of reporting as used == total size. This should give similar
behaviour as i915 which rather tracks free instead of used.
Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Cc: Gwan-gyeong Mun <gwan-gyeong.mun@intel.com> Cc: Lucas De Marchi <lucas.demarchi@intel.com> Cc: José Roberto de Souza <jose.souza@intel.com> Cc: Filip Hazubski <filip.hazubski@intel.com> Cc: Carl Zhang <carl.zhang@intel.com> Cc: Effie Yu <effie.yu@intel.com> Cc: Gwan-gyeong Mun <gwan-gyeong.mun@intel.com> Reviewed-by: José Roberto de Souza <jose.souza@intel.com> Reviewed-by: Gwan-gyeong Mun <gwan-gyeong.mun@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Francois Dugast [Thu, 22 Jun 2023 11:59:20 +0000 (13:59 +0200)]
drm/xe: Move defines before relevant fields
Align on same rule in the whole file: defines then doc then relevant
field, with an empty line to separate fields.
v2:
- Rebase on drm-xe-next
- Fix ordering of defines and fields in uAPI (Lucas De Marchi)
v3: Remove useless empty lines (Lucas De Marchi)
v4: Move changelog to commit
v5: Rebase
Reported-by: Oded Gabbay <ogabbay@kernel.org> Link: https://lists.freedesktop.org/archives/intel-xe/2023-May/004704.html Signed-off-by: Francois Dugast <francois.dugast@intel.com> Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Francois Dugast [Fri, 9 Jun 2023 07:37:12 +0000 (07:37 +0000)]
drm/xe: Document structures for device query
This adds documentation to the various structures used to query
memory, GTs, topology, engines, and so on. It includes a functional
code snippet to query engines.
v2:
- Rebase on drm-xe-next
- Also document structures related to drm_xe_device_query, changed
pseudo code to snippet (Lucas De Marchi)
v3:
- Move changelog to commit
- Fix warnings showed only using dim checkpath
Reported-by: Oded Gabbay <ogabbay@kernel.org> Link: https://lists.freedesktop.org/archives/intel-xe/2023-May/004704.html Signed-off-by: Francois Dugast <francois.dugast@intel.com> Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Mika Kuoppala [Fri, 2 Jun 2023 17:27:32 +0000 (20:27 +0300)]
drm/xe: Fix unreffed ptr leak on engine lookup
The engine xarray holds a ref to engine, guarded by the lock.
While we do lookup for engine, we need to take the ref inside
the lock to prevent unreffed pointer escaping and
causing potential use-after-free after.
Lucas De Marchi [Tue, 13 Jun 2023 18:03:55 +0000 (11:03 -0700)]
drm/xe: Skip applying copy engine fuses
Like commit 69a3738ba57f ("drm/i915: Skip applying copy engine fuses"),
do not apply copy engine fuses for platforms where MEML3_EN is not
relevant for determining the presence of the copy engines.
Matthew Auld [Thu, 15 Jun 2023 17:20:52 +0000 (18:20 +0100)]
drm/xe/bo: handle PL_TT -> PL_TT
When moving between PL_VRAM <-> PL_SYSTEM we have to have use PL_TT in
the middle as a temporary resource for the actual copy. In some GL
workloads it can be seen that once the resource has been moved to the
PL_TT we might have to bail out of the ttm_bo_validate(), before
finishing the final hop. If this happens the resource is left as
TTM_PL_FLAG_TEMPORARY, and when the ttm_bo_validate() is restarted the
current placement is always seen as incompatible, requiring us to
complete the move. However if the BO allows PL_TT as a possible
placement we can end up attempting a PL_TT -> PL_TT move (like when
running out of VRAM) which leads to explosions in xe_bo_move(), like
triggering the XE_BUG_ON(!tile).
Going from TTM_PL_FLAG_TEMPORARY with PL_TT -> PL_VRAM should already
work as-is, so it looks like we only need to worry about PL_TT -> PL_TT
and it looks like we can just treat it as a dummy move, since no real
move is needed.
Reported-by: José Roberto de Souza <jose.souza@intel.com> Signed-off-by: Matthew Auld <matthew.auld@intel.com> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Matthew Brost [Tue, 28 Mar 2023 01:34:49 +0000 (18:34 -0700)]
drm/xe: Only try to lock external BOs in VM bind
We only need to try to lock a BO if it's external as non-external BOs
share the dma-resv with the already locked VM. Trying to lock
non-external BOs caused an issue (list corruption) in an uncoming patch
which adds bulk LRU move. Since this code isn't needed, remove it.
Matthew Brost [Mon, 22 May 2023 01:24:20 +0000 (18:24 -0700)]
drm/xe: Long running job update
For long running (LR) jobs with the DRM scheduler we must return NULL in
run_job which results in signaling the job's finished fence immediately.
This prevents LR jobs from creating infinite dma-fences.
Signaling job's finished fence immediately breaks flow controlling ring
with the DRM scheduler. To work around this, the ring is flow controlled
and written in the exec IOCTL. Signaling job's finished fence
immediately also breaks the TDR which is used in reset / cleanup entity
paths so write a new path for LR entities.
v2: Better commit, white space, remove rmb(), better comment next to
emit_job()
v3 (Thomas): Change LR reference counting, fix working in commit
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Signed-off-by: Matthew Brost <matthew.brost@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Matthew Brost [Thu, 15 Jun 2023 18:22:36 +0000 (11:22 -0700)]
drm/xe: NULL binding implementation
Add uAPI and implementation for NULL bindings. A NULL binding is defined
as writes dropped and read zero. A single bit in the uAPI has been added
which results in a single bit in the PTEs being set.
NULL bindings are intendedd to be used to implement VK sparse bindings,
in particular residencyNonResidentStrict property.
v2: Fix BUG_ON shown in VK testing, fix check patch warning, fix
xe_pt_scan_64K, update __gen8_pte_encode to understand NULL bindings,
remove else if vma_addr
Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Suggested-by: Paulo Zanoni <paulo.r.zanoni@intel.com> Signed-off-by: Matthew Brost <matthew.brost@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
The GuC can't access addresses above GUC_GGTT_TOP, so any GuC-accessible
objects can't be mapped above that offset. Instead of checking each
object to see if GuC may access it or not before mapping it, we just
limit the GGTT size to GUC_GGTT_TOP. This wastes a bit of address space
(about ~18 MBs, which is in addition to what already removed at the bottom
of the GGTT), but it is a good tradeoff to keep the code simple.
The in-code comment has also been updated to explain the limitation.
Matt Roper [Thu, 8 Jun 2023 18:12:17 +0000 (11:12 -0700)]
drm/xe/mtl: Add some initial MTL workarounds
This adds a handful of workarounds that apply to production steppings of
MTL:
- Wa_14018575942
- Wa_22016670082
- Wa_14017856879
- Wa_18019271663
Wa_22016670082 is currently only applied to the primary GT at the
moment, but may need to be extended to the media GT in the future if a
pending update to the workaround database gets finalized.
OOB workarounds will need to be implemented separately in future patches
for Wa_14016712196, Wa_16018063123, and Wa_18013179988.
Michał Winiarski [Tue, 23 May 2023 13:50:20 +0000 (15:50 +0200)]
drm/xe: Fix check for platform without geometry pipeline
It's not possible for the condition checking if we're running on
platform without geometry pipeline to ever be true, since
gt->fuse_topo.g_dss_mask is an array.
It also breaks the build:
../drivers/gpu/drm/xe/xe_rtp.c:183:50: error: address of array 'gt->fuse_topo.g_dss_mask' will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion]
Michał Winiarski [Tue, 23 May 2023 13:50:19 +0000 (15:50 +0200)]
drm/xe: Fix uninitialized variables
Using uninitialized variables leads to undefined behavior.
Moreover, it causes the compiler to complain with:
../drivers/gpu/drm/xe/xe_vm.c:3265:40: error: variable 'vma' is uninitialized when used here [-Werror,-Wuninitialized]
../drivers/gpu/drm/xe/xe_rtp.c:118:36: error: variable 'i' is uninitialized when used here [-Werror,-Wuninitialized]
../drivers/gpu/drm/xe/xe_mocs.c:449:3: error: variable 'flags' is uninitialized when used here [-Werror,-Wuninitialized]
Riana Tauro [Tue, 13 Jun 2023 09:42:32 +0000 (15:12 +0530)]
drm/xe: Fix GT looping for standalone media
gt_count is only being incremented when initializing the primary GT;
since the media GT sets the ID directly, gt_count is not incremented
again, resulting in an incorrect count on MTL. Use autoincrement while
assigning the media GTs ID to ensure gt_count is correct on MTL and
other future platforms with standalone media.
Signed-off-by: Riana Tauro <riana.tauro@intel.com> Link: https://lore.kernel.org/r/20230613094232.3703549-1-riana.tauro@intel.com
[mattrope: Tweaked commit message to focus on gt_count importance] Signed-off-by: Matt Roper <matthew.d.roper@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Badal Nilawar [Fri, 9 Jun 2023 02:49:54 +0000 (08:19 +0530)]
drm/xe: Donot apply forcewake while reading actual frequency
RPSTAT1 is an sgunit register and thus doesn't need forcewake.
MTL_MIRROR_TARGET_WP1 is within an "always on" power domain and thus
doesn't require any forcewake to ensure the register is powered
up and usable. When GT is RC6 the actual frequency reported will be 0.
v2:
- Add bspec index (Anshuman)
- %s/GEN12_RPSTAT1/GT_PERF_STATUS as per bspec
v3: Update Fixes tag
Bspec: 51837, 67651 Fixes: dd08ebf6c352 ("drm/xe: Introduce a new DRM driver for Intel GPUs") Signed-off-by: Badal Nilawar <badal.nilawar@intel.com> Reviewed-by: Matt Roper <matthew.d.roper@intel.com> Link: https://lore.kernel.org/r/20230609024954.987039-1-badal.nilawar@intel.com Signed-off-by: Matt Roper <matthew.d.roper@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Lucas De Marchi [Sun, 11 Jun 2023 22:24:45 +0000 (15:24 -0700)]
drm/xe/guc: Normalize error messages with %#x
One of the messages was printed without 0x prefix, so it was not clear
if it was decimal or hex: make sure to add the prefix by using %#x.
While at it, normalize the other messages in the same function to follow
the same pattern.
Lucas De Marchi [Sun, 11 Jun 2023 22:24:43 +0000 (15:24 -0700)]
drm/xe: Rename pte/pde encoding functions
Remove the leftover TODO by renameing the functions to use xe prefix.
Since the static __gen8_pte_encode() already has a double score,
just remove the prefix.
Matthew Brost [Wed, 7 Jun 2023 18:43:52 +0000 (11:43 -0700)]
drm/xe: s/XE_PTE_READ_ONLY/XE_PTE_FLAG_READ_ONLY
This define is for internal PTE flags rather than fields in the hardware
PTEs, rename as such. This will help in an upcoming patch to avoid
further confusion.
Matthew Brost [Fri, 9 Jun 2023 18:19:30 +0000 (11:19 -0700)]
drm/xe: Use Xe ordered workqueue for rebind worker
A mix of the system unbound wq and Xe ordered wq was used for the
rebind, only use the Xe ordered wq. This will ensure only 1 rebind is
occuring at a time providing a somewhat clunky work around for short
comings in TTM wrt to memory contention. Once the TTM memory contention
is resolved we should be able to use a dedicated non-ordered workqueue.
Also add helper to queue rebind worker to avoid using wrong workqueue
going forward.
Reviewed-by: José Roberto de Souza <jose.souza@intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Signed-off-by: Matthew Brost <matthew.brost@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Matthew Brost [Fri, 9 Jun 2023 18:09:37 +0000 (11:09 -0700)]
drm/xe: Handle unmapped userptr in analyze VM
A corner exists where a userptr may have no mapping when analyze VM is
called, handle this case.
Reviewed-by: José Roberto de Souza <jose.souza@intel.com> Reviewed-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Signed-off-by: Matthew Brost <matthew.brost@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
drm/xe: Emit a render cache flush after each rcs/ccs batch
We need to flush render caches before fence signalling, where we might
release the memory for reuse. We can't rely on userspace doing this,
so flush render caches after the batch, but before user fence- and
dma_fence signalling.
Copy the cache flush from i915, but omit PIPE_CONTROL_FLUSH_L3, since it
should be implied by the other flushes. Also omit
PIPE_CONTROL_TLB_INVALIDATE since there should be no apparent need to
invalidate TLB after batch completion.
v2:
- Update Makefile for OOB WA.
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Tested-by: José Roberto de Souza <jose.souza@intel.com> Reviewed-by: José Roberto de Souza <jose.souza@intel.com> #1 Reported-by: José Roberto de Souza <jose.souza@intel.com> Link: https://gitlab.freedesktop.org/drm/xe/kernel/-/issues/291 Closes: https://gitlab.freedesktop.org/drm/xe/kernel/-/issues/291 Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
drm/xe: Invalidate TLB also on bind if in scratch page mode
For scratch table mode we need to cover the case where a scratch PTE might
have been pre-fetched and cached and used instead of that of the newly
bound vma.
For compute vms, invalidate TLB globally using GuC before signalling
bind complete. For !long-running vms, invalidate TLB at batch start.
Also document how TLB invalidation works.
v2:
- Fix a pointer to the comment about TLB invalidation (Jose Souza).
- Add a bool to the vm whether we want to invalidate TLB at batch start.
- Invalidate TLB also on BCS- and video engines at batch start where
needed.
- Use BIT() macro instead of explicit shift.
Signed-off-by: Thomas Hellström <thomas.hellstrom@linux.intel.com> Tested-by: José Roberto de Souza <jose.souza@intel.com> #v1 Reported-by: José Roberto de Souza <jose.souza@intel.com> #v1 Link: https://gitlab.freedesktop.org/drm/xe/kernel/-/issues/291 Closes: https://gitlab.freedesktop.org/drm/xe/kernel/-/issues/291 Acked-by: José Roberto de Souza <jose.souza@intel.com> Reviewed-by: Matthew Brost <matthew.brost@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Matt Roper [Fri, 2 Jun 2023 23:10:54 +0000 (16:10 -0700)]
drm/xe/wa: Extend scope of Wa_14015795083
Wa_14015795083 was already implemented for DG2 and PVC, but the
workaround database has been updated to extend it to more platforms. It
should now apply to all platforms with graphics versions 12.00 - 12.60,
as well as A-step of Xe_LPG (12.70 / 12.71).
Michael J. Ruhl [Mon, 5 Jun 2023 16:08:56 +0000 (12:08 -0400)]
drm/xe: REBAR resize should be best effort
The resizing of the PCI BAR is a best effort feature. If it is
not available, it should not fail the driver probe.
Rework the resize to not exit on failure.
Fixes: 7f075300a318 ("drm/xe: Simplify rebar sizing") Acked-by: Lucas De Marchi <lucas.demarchi@intel.com> Reviewed-by: Matthew Auld <matthew.auld@intel.com> Signed-off-by: Michael J. Ruhl <michael.j.ruhl@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Matt Roper [Fri, 2 Jun 2023 23:52:10 +0000 (16:52 -0700)]
drm/xe: Don't hardcode GuC's MOCS index in register header
Although PVC is currently the only platform that needs us to program a
GuC register with the index of an uncached MOCS entry, it's likely other
platforms will need this in the future. Rather than hardcoding PVC's
index into the register header, we should just pull the appropriate
index from gt->mocs.uc_index to future-proof the code.
Matt Roper [Fri, 2 Jun 2023 23:52:09 +0000 (16:52 -0700)]
drm/xe: Initialize MOCS earlier
xe_mocs_init_early doesn't touch the hardware, it just sets up internal
software state. There's no need to perform this step in the "forcewake
held" region. Moving the init earlier will also make the uc_index
values available earlier which will be important for an upcoming GuC
init patch.
Matt Roper [Fri, 2 Jun 2023 23:52:08 +0000 (16:52 -0700)]
drm/xe: Reformat xe_guc_regs.h
Reformat the GuC register header according to the same rules used by
other register headers:
- Register definitions are ordered by offset
- Value of #define's start on column 49
- Lowercase used for hex values
No functional change.
This header has some things that aren't directly related to register
definitions (e.g., number of doorbells, doorbell info structure, GuC
interrupt vector layout, etc. These items have been moved to the bottom
of the header.
Gustavo Sousa [Thu, 1 Jun 2023 19:44:19 +0000 (16:44 -0300)]
drm/xe: Replace deprecated DRM_ERROR()
DRM_ERROR() has been deprecated in favor of pr_err(). However, we should
prefer to use xe_gt_err() or drm_err() whenever possible so we get gt-
or device-specific output with the error message.
v2:
- Prefer drm_err() over pr_err(). (Matt, Jani)
v3:
- Prefer xe_gt_err() over drm_err() when possible. (Matt)
v4:
- Use the already available dev variable instead of xe->drm as
parameter to drm_err(). (Matt)
Cc: Jani Nikula <jani.nikula@intel.com> Cc: Lucas De Marchi <lucas.demarchi@intel.com> Cc: Haridhar Kalvala <haridhar.kalvala@intel.com> Reviewed-by: Matt Roper <matthew.d.roper@intel.com> Link: https://lore.kernel.org/r/20230601194419.1179609-1-gustavo.sousa@intel.com Signed-off-by: Gustavo Sousa <gustavo.sousa@intel.com> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
projects / mirror_ubuntu-kernels.git / log