drm/nouveau/kms/nv50-: use NVIDIA's headers for core ntfy_init()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for core init()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wndw update()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wndw blend_set()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wndw scale_set()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wndw image_clr()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wndw image_set()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wndw xlut_clr()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wndw xlut_set()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wndw csc_clr()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wndw csc_set()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wndw ntfy_wait_begun()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wndw ntfy_clr()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wndw ntfy_set()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wndw ntfy_reset()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wndw sema_clr()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wndw sema_set()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wimm update()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wimm point()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: use NVIDIA's headers for wimm space()

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/fence: convert sync() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/fence: convert emit() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/gem: convert indirect pushbufs to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/chan: convert nvsw init to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau: convert nvsw init to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv04-: no need for multiple nvsw objects

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv04: convert page_flip() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv04: convert page_flip_emit() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/svm: convert migrate_clear to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/svm: convert migrate_copy to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/bo: convert move move() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/bo: convert move init() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/bo: split buffer move functions into their own source files

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/fbcon: convert copyarea() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/fbcon: convert fillrect() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/fbcon: convert imageblit() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/fbcon: convert accel_init() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau: interop with new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: implement proper push buffer control logic

We had a, what was supposed to be temporary, hack in the KMS code where we'd
completely drain an EVO/NVD channel's push buffer when wrapping to the start
again, instead of treating it as a ring buffer.

Let's fix that, finally.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/kms/nv50-: convert core crc_set_ctx() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core crc_set_src() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core head_or() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core head_procamp() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core head_dither() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core head_ovly() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core head_base() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core head_curs_clr() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core head_curs_set() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core head_core_clr() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core head_core_set() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core head_olut_clr() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core head_olut_set() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core head_mode() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core head_view() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core or_ctrl() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core wndw_owner() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core update() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core caps_init() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert core init() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wndw update() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wndw blend_set() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wndw scale_set() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wndw image_clr() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wndw image_set() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wndw xlut_clr() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wndw xlut_set() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wndw csc_clr() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wndw csc_set() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wndw ntfy_clr() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wndw ntfy_set() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wndw sema_clr() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wndw sema_set() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wimm update() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: convert wimm point() to new push macros

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/nv50-: wrap existing command submission in nvif_push interface

This commit pulls in a bunch of new push buffer macros which are able to
support NVIDIA's class headers, and provide more useful debug output and
error checking (compile-time, where possible) than we had previously.

Will incrementally transition each function over to the unified interfaces.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/nvif: add support for object-level debug output

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/nvif: give every notify object a human-readable name

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/nvif: give every disp object a human-readable identifier

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/nvif: give every usermode object a human-readable identifier

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/nvif: give every vmm object a human-readable identifier

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/nvif: give every mem object a human-readable identifier

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/nvif: give every mmu object a human-readable identifier

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/nvif: give every device object a human-readable identifier

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/nvif: give every object a human-readable identifier

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/nvif: rename client ctor/dtor

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reviewed-by: Lyude Paul <lyude@redhat.com>

drm/nouveau/kms/tu102: set NVC57D_HEAD_SET_HEAD_USAGE_BOUNDS_UPSCALING_ALLOWED to TRUE

Fixes issues when switching between scaling modes.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: Use fallthrough pseudo-keyword

Replace the existing /* fall through */ comments and its variants with
the new pseudo-keyword macro fallthrough[1]. Also, remove unnecessary
fall-through markings when it is the case.

[1] https://www.kernel.org/doc/html/latest/process/deprecated.html?highlight=fallthrough#implicit-switch-case-fall-through

Signed-off-by: Gustavo A. R. Silva <gustavoars@kernel.org>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/vmm/gp100-: fix mapping 2MB sysmem pages

The nvif_object_ioctl() method NVIF_VMM_V0_PFNMAP wasn't correctly
setting the hardware specific GPU page table entries for 2MB sized
pages. Fix this by adding functions to set and clear PD0 GPU page
table entries.

Signed-off-by: Ralph Campbell <rcampbell@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/mmu: make nvkm_vmm_ctor() static

The function nvkm_vmm_ctor() is not called outside of the file defining
it, so make it static.

Signed-off-by: Ralph Campbell <rcampbell@nvidia.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: fix reference count leak in nouveau_debugfs_strap_peek

nouveau_debugfs_strap_peek() calls pm_runtime_get_sync() that
increments the reference count. In case of failure, decrement the
ref count before returning the error.

Signed-off-by: Aditya Pakki <pakki001@umn.edu>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: Fix reference count leak in nouveau_connector_detect

nouveau_connector_detect() calls pm_runtime_get_sync and in turn
increments the reference count. In case of failure, decrement the
ref count before returning the error.

Signed-off-by: Aditya Pakki <pakki001@umn.edu>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: fix reference count leak in nv50_disp_atomic_commit

nv50_disp_atomic_commit() calls calls pm_runtime_get_sync and in turn
increments the reference count. In case of failure, decrement the
ref count before returning the error.

Signed-off-by: Aditya Pakki <pakki001@umn.edu>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: fix multiple instances of reference count leaks

On calling pm_runtime_get_sync() the reference count of the device
is incremented. In case of failure, decrement the
ref count before returning the error.

Signed-off-by: Aditya Pakki <pakki001@umn.edu>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/drm/noveau: fix reference count leak in nouveau_fbcon_open

nouveau_fbcon_open() calls calls pm_runtime_get_sync() that
increments the reference count. In case of failure, decrement the
ref count before returning the error.

Signed-off-by: Aditya Pakki <pakki001@umn.edu>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/sec2/gp102: allow module to load when LSFW is missing

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gm200-: explicitly handle nofw

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/pmu/gm200-: explicitly handle nofw

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/pmu/gm20x: don't pretend we support loading with our custom FW

It technically loads, and runs, but is ultimately pointless outside of
a very narrow window (fanless systems where one wants to attempt using
the, broken for a lot of gm20x, memory reclocking code).

It's also potentially dangerous to override the VBIOS-provided "Pre-OS"
PMU, which would be responsible for fan control otherwise.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/acr: store a mask of LS falcons the controlling LSFW can bootstrap

This will prevent some pain with broken firmware trees, as under some
circumstances the HSFW can fail and leave the GPU in a state we don't
know how to recover from.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/acr: store a mask of LS falcons the HSFW can bootstrap

This will prevent reloading of HS FW where it's pointless, and bypass
hitting some timeouts.

Not a situation one should generally hit, but can occur with a messed
up firmware installation.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>