drm/amd/powerplay: add interface to populate smc pptable (v2)

This patch adds interface to populate smc pptable for smu.
It's to set initialized values (get from vbios) to dpm tables context such as
gfxclk, memclk, dcefclk, and etc. And enable the DPM feature for each type of
clks.

v2: add detailed info to describe this function

Signed-off-by: Huang Rui <ray.huang@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: add interface to parse pptable (v2)

This patch adds interface to parse pptable for smu.
It's to parse pptable format and fill PPTable_t smc_pptable to smu_table_context
structure. And read the smc_dpm_table from vbios, then fill it into smc_pptable.

v2: add detailed info to describe this function

Signed-off-by: Huang Rui <ray.huang@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: add interface to init fb allocations (v2)

This patch adds interface to init fb allocations for smu.
It's to allocate vram bos to store smc table contents.

v2: add detailed info to describe this function

Signed-off-by: Huang Rui <ray.huang@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: add interface to check pptable (v2)

This patch adds interface to check pptable function for smu.
It's to check if the format_revision in vbios is up to pptable header version,
and the structure size is not 0.

v2: add detailed info to describe this function

Signed-off-by: Huang Rui <ray.huang@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: add interface to get vbios bootup values (v2)

This patch adds interface to get vbios bootup values for smu.
It's to get boot_values from vbios to set revision, gfxclk, and etc.

v2: add detailed info to describe this function.

Signed-off-by: Huang Rui <ray.huang@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: add interface to init power (v2)

This patch adds interface to init power function for smu.
It's to create smu_power_context structure, and allocate smu_dpm_context and
context size to fill the smu_power_context data.

v2: add detailed info to describe this function

Signed-off-by: Huang Rui <ray.huang@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: add interface to init smc tables (v2)

This patch adds interface to init smc tables for smu. It is to create smu_table
structure, and init smc tables such as TABLE_PPTABLE, TABLE_WATERMARKS,
TABLE_SMU_METRICS, and etc.

v2: add detailed info to describe this function

Signed-off-by: Huang Rui <ray.huang@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: add placeholder of smu_initialize_pptable

This patch adds placeholder of smu_initialize_pptable for smu.

Signed-off-by: Huang Rui <ray.huang@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: add interface to read pptable from vbios

This patch adds interface to read pptable from vbios for smu.

Signed-off-by: Huang Rui <ray.huang@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: add fw load checking interface

This patch adds fw load checking interface for smu.

Signed-off-by: Huang Rui <ray.huang@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: add firmware loading interface

This patch adds firmware loading interface for smu.

Signed-off-by: Huang Rui <ray.huang@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: add smu11 sub block for SMU IP

This patch add smu11 sub ip block skeleton as placeholder

Signed-off-by: Huang Rui <ray.huang@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: add new smu ip block

This patch add amdgpu_smu ip block skeleton as placeholder.
We plan use new smu ip block from Vega20.

Signed-off-by: Huang Rui <ray.huang@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: remove set but not used variable 'vbi_time_out'

Fixes gcc '-Wunused-but-set-variable' warning:

drivers/gpu/drm/amd/amdgpu/si_dpm.c: In function 'si_program_response_times':
drivers/gpu/drm/amd/amdgpu/si_dpm.c:4101:29: warning:
 variable 'backbias_response_time' set but not used [-Wunused-but-set-variable]

It's never used since introduction in 841686df9f7d ("drm/amdgpu: add SI DPM
support (v4)"), so can be removed

Signed-off-by: YueHaibing <yuehaibing@huawei.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: remove set but not used variables 'vm, bo'

Fixes gcc '-Wunused-but-set-variable' warning:

drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c: In function 'update_gpuvm_pte':
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c:840:20: warning:
 variable 'bo' set but not used [-Wunused-but-set-variable]

drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c:838:20: warning:
 variable 'vm' set but not used [-Wunused-but-set-variable]

They're never used since introduction in a46a2cd103a8 ("drm/amdgpu: Add GPUVM
memory management functions for KFD")

Signed-off-by: YueHaibing <yuehaibing@huawei.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: remove set but not used variable 'bdev'

Fixes gcc '-Wunused-but-set-variable' warning:

drivers/gpu/drm/ttm/ttm_execbuf_util.c: In function 'ttm_eu_fence_buffer_objects':
drivers/gpu/drm/ttm/ttm_execbuf_util.c:191:24: warning:
 variable 'bdev' set but not used [-Wunused-but-set-variable]

It's not used any more and can be removed.

Signed-off-by: YueHaibing <yuehaibing@huawei.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Set VM_L2_CNTL.PDE_FAULT_CLASSIFICATION to 0

This is recommended by HW designers. Previously when it was set to 1,
the PDE walk error in VM fault will be treated as
PERMISSION_OR_INVALID_PAGE_FAULT rather than usually expected OTHER_FAULT.
As a result, the retry control in VM_CONTEXT*_CNTL will change accordingly.

The above behavior is kind of abnormal. Furthermore, the
PDE_FAULT_CLASSIFICATION == 1 feature was targeted for very old ASICs
and it never made it way to production. Therefore, we should set it to 0.

Signed-off-by: Yong Zhao <Yong.Zhao@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Eliminate the set_pde_pte function pointer in amdgpu_gmc_funcs

All the gmc_*_set_pde_pte functions are the same across different ASICs,
so we can eliminate the set_pde_pte function pointer and instead use a
generic function.

Signed-off-by: Yong Zhao <Yong.Zhao@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: Account for kernel allocations in kernel zone only

Don't account for them in other zones such as dma32. The kernel page
allocator has its own heuristics to avoid exhausting special zones
for regular kernel allocations.

Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Acked-by: Thomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm: Use the same mmap-range offset and size for GEM and TTM

GEM defines DRM_FILE_PAGE_OFFSET_{START,SIZE} constants for the
mmap-able range of addresses. TTM can use them as well.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Christian König <christian.koenig@amd.com>
Acked-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: Quick-test mmap offset in ttm_bo_mmap()

A BO's address has to be at least the minimum offset. Sharing this
test in ttm_bo_mmap() removes code from drivers. A full buffer-address
validation is still done within drm_vma_offset_lockup_locked().

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Christian König <christian.koenig@amd.com>
Acked-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: Remove file_page_offset parameter from ttm_bo_device_init()

The parameter file_page_offset is a constant shared by all drivers. Just
replace it with the constant itself.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Christian König <christian.koenig@amd.com>
Acked-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: Define a single DRM_FILE_PAGE_OFFSET constant

Most TTM drivers define the constant DRM_FILE_PAGE_OFFSET of the same
value. The only exception is vboxvideo, which is being converted to the
new offset by this patch. Unifying the constants in a single place
simplifies the driver code.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Christian König <christian.koenig@amd.com>
Acked-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

staging/vboxvideo: Use same BO mmap offset as other drivers

The vboxvideo driver mmaps BOs at 0x10000000 or higher. Changing the
offset to 0x100000000 aligns the driver with all other DRM drivers.

Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
Reviewed-by: Hans de Goede <hdegoede@redhat.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: fix HMM config dependency issue

Only select HMM_MIRROR will get kernel config dependency warnings
if CONFIG_HMM is missing in the config. Add depends on HMM will
solve the issue.

Add conditional compilation to fix compilation errors if HMM_MIRROR
is not enabled as HMM config is not enabled.

Remove unused function amdgpu_ttm_tt_mark_user_pages.

Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: replace get_user_pages with HMM mirror helpers

Use HMM helper function hmm_vma_fault() to get physical pages backing
userptr and start CPU page table update track of those pages. Then use
hmm_vma_range_done() to check if those pages are updated before
amdgpu_cs_submit for gfx or before user queues are resumed for kfd.

If userptr pages are updated, for gfx, amdgpu_cs_ioctl will restart
from scratch, for kfd, restore worker is rescheduled to retry.

HMM simplify the CPU page table concurrent update check, so remove
guptasklock, mmu_invalidations, last_set_pages fields from
amdgpu_ttm_tt struct.

HMM does not pin the page (increase page ref count), so remove related
operations like release_pages(), put_page(), mark_page_dirty().

Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdkfd: avoid HMM change cause circular lock

There is circular lock between gfx and kfd path with HMM change:
lock(dqm) -> bo::reserve -> amdgpu_mn_lock

To avoid this, move init/unint_mqd() out of lock(dqm), to remove nested
locking between mmap_sem and bo::reserve. The locking order
is: bo::reserve -> amdgpu_mn_lock(p->mn)

Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: use HMM callback to replace mmu notifier

Replace our MMU notifier with hmm_mirror_ops.sync_cpu_device_pagetables
callback. Enable CONFIG_HMM and CONFIG_HMM_MIRROR as a dependency in
DRM_AMDGPU_USERPTR Kconfig.

It supports both KFD userptr and gfx userptr paths.

Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: use BACO on vega12 if platform supports it

Use BACO for reset of the platform supports it.

Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/powerplay: split out common smu9 BACO code

Several of the BACO functions are common across smu9-based
asics.  Split the common code out.

Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/powerplay: add BACO support for vega12

This implements BACO (Bus Active, Chip Off) support
for vega12.

Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: 3.2.19

Signed-off-by: Mark McGarrity <mark.mcgarrity@amd.com>
Reviewed-by: Tony Cheng <Tony.Cheng@amd.com>
Acked-by: Leo Li <sunpeng.li@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Add ability to override bounding box in DC construct

Add a dc_bounding_box_overrides struct to define bb overrides. It is
loaded in during DC init.

Signed-off-by: Jun Lei <Jun.Lei@amd.com>
Reviewed-by: Tony Cheng <Tony.Cheng@amd.com>
Acked-by: Leo Li <sunpeng.li@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Make stream commits call into DC only once

[Why]
dc_commit_updates_for_stream is called twice per stream: once
with the flip data and once will all other data. This causes
problems when these DC calls have different numbers of planes

For example, a commit with a pageflip on plane A and a
non-pageflip change on plane B will first call
into DC with just plane A, causing plane B to be
disabled. Then it will call into DC with both planes,
re-enabling plane B

[How]
Merge flip and full into a single bundle

Apart from the single DC call, the logic should not be
changed by this patch

Signed-off-by: David Francis <David.Francis@amd.com>
Reviewed-by: Nicholas Kazlauskas <Nicholas.Kazlauskas@amd.com>
Acked-by: Leo Li <sunpeng.li@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: optionally optimize edp link rate based on timing

[Why]
eDP v1.4 allows panels to report link rates other than RBR/HBR/HBR2, that
may be more optimal for the panel's timing. Power can be saved by using
a link rate closer to the required bandwidth of the panel's timing.

[How]
Scan the table of reported link rates from the panel, and select the
minimum link rate that satisfies the bandwidth requirements of the panel's
timing. Include a flag to make the feature optional.

Signed-off-by: Josip Pavic <Josip.Pavic@amd.com>
Reviewed-by: Harry Wentland <Harry.Wentland@amd.com>
Acked-by: Anthony Koo <Anthony.Koo@amd.com>
Acked-by: Leo Li <sunpeng.li@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Add p_state_change_support flag to dc_clocks

Will be used to signify if P-state change is supported.

Signed-off-by: Jun Lei <Jun.Lei@amd.com>
Reviewed-by: Eric Yang <eric.yang2@amd.com>
Acked-by: Leo Li <sunpeng.li@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Clean up wait on vblank event

[Why]
The wait_for_vblank boolean in commit_tail was passed by reference
into each stream commit, and if that commit was an asynchronous
flip, it would disable vblank waits on all subsequent flips.

This made the behaviour depend on crtc order in a non-intuitive way,
although since the asynchronous pageflip flag is only used by the
legacy IOCTLs at the moment it is never an issue

[How]
Find wait_for_vblank before doing any stream commits

Signed-off-by: David Francis <David.Francis@amd.com>
Reviewed-by: Nicholas Kazlauskas <Nicholas.Kazlauskas@amd.com>
Acked-by: Leo Li <sunpeng.li@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Set flip pending for pipe split

[WHY]
When doing split pipe, if one pipe is pending on flip, the entire
plane's status should be flip pending, otherwise corruption can occur
when OS writes to a surface prematurely.

[HOW]
Clear the flip pending bit before checking pipes, then OR the flip
pending bits from all pipes together to create the flip pending status
of the entire plane.

Signed-off-by: Wesley Chalmers <Wesley.Chalmers@amd.com>
Reviewed-by: Jun Lei <Jun.Lei@amd.com>
Acked-by: Eryk Brol <Eryk.Brol@amd.com>
Acked-by: Leo Li <sunpeng.li@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Fix exception from AUX acquire failure

[Why]
AUX arbitration occurs between SW and FW components.
When AUX acquire fails, it causes engine->ddc to be NULL,
which leads to an exception when we try to release the AUX
engine.

[How]
When AUX engine acquire fails, it should return from the
function without trying to continue the operation.
The upper level will determine if it wants to retry.
i.e. dce_aux_transfer_with_retries will be used and retry.

Signed-off-by: Anthony Koo <Anthony.Koo@amd.com>
Reviewed-by: Aric Cyr <Aric.Cyr@amd.com>
Acked-by: Leo Li <sunpeng.li@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Reset planes that were disabled in init_pipes

[Why]
Seamless boot tries to reuse planes that were enabled for the first
commit applied.

In the case where Raven is booting with two monitors connected and the
first commit contains two streams the screen corruption would occur
because the second stream was trying to re-use a tg and plane that
weren't previously enabled.

The state on the first commit looks something like the following:

TG0: enabled=1
TG1: enabled=0
TG2: enabled=0
TG3: enabled=0

New state: pipe=0, stream=0,    plane=0,       new_tg=0
New state: pipe=1, stream=1,    plane=1,       new_tg=1
New state: pipe=2, stream=NULL, plane=NULL,    new_tg=NULL
New state: pipe=3, stream=NULL, plane=NULL,    new_tg=NULL

Only one plane/tg is setup before we enter accelerated mode so
we really want to disabling everything but that first plane.

[How]

Check if the stream is not NULL and if the tg is enabled before
deciding whether to skip the plane disable.

Also ensure we're also disabling on the current state's pipe_ctx so
we don't overwrite the fields in the new pending state.

Signed-off-by: Nicholas Kazlauskas <nicholas.kazlauskas@amd.com>
Reviewed-by: Anthony Koo <Anthony.Koo@amd.com>
Acked-by: Harry Wentland <Harry.Wentland@amd.com>
Acked-by: Leo Li <sunpeng.li@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Allow for plane-less resource reservation

This change changes dc add plane logic to allow plane-less resource
reservation (pipe split).

If a free pipe_ctx (no plane_state attached) is the head pipe, and is
found with a bottom pipe attached, assign the plane to add on the bottom
pipe.

In addition, prepend dcn10 to dcn10-specific reset_back_end_for_pipe
and reset_hw_ctx_wrap

Signed-off-by: Dmytro Laktyushkin <Dmytro.Laktyushkin@amd.com>
Reviewed-by: Charlene Liu <Charlene.Liu@amd.com>
Acked-by: Leo Li <sunpeng.li@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: dcn add check surface in_use

Driver need to  poll the SURFACE_INUSE register to determine when to
start the new task and write data to the checked surface.

Implement the wait functions, and add the necessary hubbub registers.

Signed-off-by: Charlene Liu <charlene.liu@amd.com>
Reviewed-by: Dmytro Laktyushkin <Dmytro.Laktyushkin@amd.com>
Acked-by: Leo Li <sunpeng.li@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Add DCN_VM aperture registers

[Why]
For later use by the DC VM implementation

Signed-off-by: Eryk Brol <eryk.brol@amd.com>
Reviewed-by: Jun Lei <Jun.Lei@amd.com>
Acked-by: Leo Li <sunpeng.li@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: make seamless boot work generically

[Why]
Seamless boot code not working on all ASICs because of
some underflow issues caused by some uninitialized HW
state.

[How]
Keep some logical and power gating init code in hw_init.
Move some per pipe init code to enable accelerated mode

Signed-off-by: Anthony Koo <Anthony.Koo@amd.com>
Reviewed-by: Aric Cyr <Aric.Cyr@amd.com>
Acked-by: Leo Li <sunpeng.li@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Remove redundant 'else' statement in dcn1_update_clocks

[Why]
DM has impelemented new pp_smu interface. 'Else' is not longer needed.

Signed-off-by: Fatemeh Darbehani <fatemeh.darbehani@amd.com>
Reviewed-by: Eric Yang <eric.yang2@amd.com>
Acked-by: Leo Li <sunpeng.li@amd.com>
Acked-by: Yongqiang Sun <yongqiang.sun@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Move enum gamut_remap_select to hw_shared.h

This enum definition is shared, so move it to a shared location.

Signed-off-by: Eric Bernstein <eric.bernstein@amd.com>
Reviewed-by: Tony Cheng <Tony.Cheng@amd.com>
Acked-by: Leo Li <sunpeng.li@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: 3.2.18

Signed-off-by: mmcgarri <mark.mcgarrity@amd.com>
Reviewed-by: Tony Cheng <Tony.Cheng@amd.com>
Acked-by: Leo Li <sunpeng.li@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: PPLIB Hookup

[Why]
Make dml and integration with pplib clearer.

[How]
Change the way the dml formula is initialized to make its values more
clear. Restructure DC interface with pplib into rv_funcs.
Cap clocks received from pplib.

Signed-off-by: Jun Lei <Jun.Lei@amd.com>
Signed-off-by: Eryk Brol <eryk.brol@amd.com>
Reviewed-by: Dmytro Laktyushkin <Dmytro.Laktyushkin@amd.com>
Acked-by: Leo Li <sunpeng.li@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Add disable triple buffering DC debug option

Added a "disable_tri_buf" DC debug option. When set to 1  feature will
be off.

Signed-off-by: Charlene Liu <charlene.liu@amd.com>
Reviewed-by: Dmytro Laktyushkin <Dmytro.Laktyushkin@amd.com>
Acked-by: Leo Li <sunpeng.li@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Use vrr friendly pageflip throttling in DC.

In VRR mode, keep track of the vblank count of the last
completed pageflip in amdgpu_crtc->last_flip_vblank, as
recorded in the pageflip completion handler after each
completed flip.

Use that count to prevent mmio programming a new pageflip
within the same vblank in which the last pageflip completed,
iow. to throttle pageflips to at most one flip per video
frame, while at the same time allowing to request a flip
not only before start of vblank, but also anywhere within
vblank.

The old logic did the same, and made sense for regular fixed
refresh rate flipping, but in vrr mode it prevents requesting
a flip anywhere inside the possibly huge vblank, thereby
reducing framerate in vrr mode instead of improving it, by
delaying a slightly delayed flip requests up to a maximum
vblank duration + 1 scanout duration. This would limit VRR
usefulness to only help applications with a very high GPU
demand, which can submit the flip request before start of
vblank, but then have to wait long for fences to complete.

With this method a flip can be both requested and - after
fences have completed - executed, ie. it doesn't matter if
the request (amdgpu_dm_do_flip()) gets delayed until deep
into the extended vblank due to cpu execution delays. This
also allows clients which want to regulate framerate within
the vrr range a much more fine-grained control of flip timing,
a feature that might be useful for video playback, and is
very useful for neuroscience/vision research applications.

In regular non-VRR mode, retain the old flip submission
behavior. This to keep flip scheduling for fullscreen X11/GLX
OpenGL clients intact, if they use the GLX_OML_sync_control
extensions glXSwapBufferMscOML(, ..., target_msc,...) function
with a specific target_msc target vblank count.

glXSwapBuffersMscOML() or DRI3/Present PresentPixmap() will
not flip at the proper target_msc for a non-zero target_msc
if VRR mode is active with this patch. They'd often flip one
frame too early. However, this limitation should not matter
much in VRR mode, as scheduling based on vblank counts is
pretty futile/unusable under variable refresh duration
anyway, so no real extra harm is done.

According to some testing already done with this patch by
Nicholas on top of my tests, IGT tests didn't report any
problems. If fixes stuttering and flickering when flipping
at rates below the minimum vrr refresh rate.

Fixes: bb47de736661 ("drm/amdgpu: Set FreeSync state using drm VRR
properties")
Signed-off-by: Mario Kleiner <mario.kleiner.de@gmail.com>
Cc: <stable@vger.kernel.org>
Cc: Harry Wentland <harry.wentland@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Cc: Michel Dänzer <michel@daenzer.net>
Tested-by: Bruno Filipe <bmilreu@gmail.com>
Reviewed-by: Nicholas Kazlauskas <nicholas.kazlauskas@amd.com>
Signed-off-by: Nicholas Kazlauskas <nicholas.kazlauskas@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>

drm/amdgpu: clear PDs/PTs only after initializing them

Clear the VM PDs/PTs only after initializing all the structures.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Pass app_tf by value rather than by reference

Clang warns when an expression that equals zero is used as a null
pointer constant (in lieu of NULL):

drivers/gpu/drm/amd/amdgpu/../display/amdgpu_dm/amdgpu_dm.c:4435:3:
warning: expression which evaluates to zero treated as a null pointer
constant of type 'const enum color_transfer_func *'
[-Wnon-literal-null-conversion]
                TRANSFER_FUNC_UNKNOWN,
                ^~~~~~~~~~~~~~~~~~~~~
1 warning generated.

This warning is caused by commit bb47de736661 ("drm/amdgpu: Set FreeSync
state using drm VRR properties") and it could be solved by using NULL
instead of TRANSFER_FUNC_UNKNOWN or casting TRANSFER_FUNC_UNKNOWN as a
pointer. However, after looking into it, there doesn't appear to be a
good reason to pass app_tf by reference as it is never mutated along the
way. This is the only code path in which app_tf is used:

mod_freesync_build_vrr_infopacket ->
    build_vrr_infopacket_v2 ->
        build_vrr_infopacket_fs2_data

Neither mod_freesync_build_vrr_infopacket or build_vrr_infopacket_v2
modify app_tf's value and build_vrr_infopacket_fs2_data expects just
the value so we can avoid dereferencing anything by just passing in
app_tf's value to mod_freesync_build_vrr_infopacket and
build_vrr_infopacket_v2.

There is no functional change because build_vrr_infopacket_fs2_data
doesn't do anything if TRANSFER_FUNC_UNKNOWN is passed to it, the same
as not calling build_vrr_infopacket_fs2_data at all like before this
change when NULL was used for app_tf.

Reviewed-by: Harry Wentland <harry.wentland@amd.com>
Signed-off-by: Nathan Chancellor <natechancellor@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

Revert "drm/amdgpu: use BACO reset on vega20 if platform support"

This reverts commit 2172b89e7c94605380d8c0dedf543c93f0a0b27c.

Signed-off-by: Candice Li <candice.li@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: show the right override pcie parameters

Instead of the hard-coded ones from VBIOS.

Signed-off-by: Evan Quan <evan.quan@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: honor the OD settings

Set the soft/hard max settings as max possible to
not violate the OD settings.

Signed-off-by: Evan Quan <evan.quan@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: set default fclk for no fclk dpm support case

Set the default fclk as what we got from VBIOS.

Signed-off-by: Evan Quan <evan.quan@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: support retrieving clock information from other sysplls

There will be some needs to retrieve clock information from other
sysplls also except default 0.

Signed-off-by: Evan Quan <evan.quan@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: overwrite ODSettingsMin for UCLK_FMAX feature

For UCLK_FMAX OD feature, SMU overwrites the highest UCLK DPM level freq.
Therefore it can only take values that are greater than the second highest
DPM level freq.

Signed-off-by: Evan Quan <evan.quan@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: force FCLK to highest also for 5K or higher displays

This can fix possible screen freeze on high resolution displays.

Signed-off-by: Evan Quan <evan.quan@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: need to reapply the dpm level settings

As these settings got reset during above phm_apply_clock_adjust_rules.

Signed-off-by: Evan Quan <evan.quan@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: drop redundant soft min/max settings

As these are already set during apply_clocks_adjust_rules.

Signed-off-by: Evan Quan <evan.quan@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdkfd: use init_mqd function to allocate object for hid_mqd (CI)

if use the legacy method to allocate object, when mqd_hiq need to run
uninit code, it will be cause WARNING call trace.

eg: (s3 suspend test)
[   34.918944] Call Trace:
[   34.918948]  [<ffffffff92961dc1>] dump_stack+0x19/0x1b
[   34.918950]  [<ffffffff92297648>] __warn+0xd8/0x100
[   34.918951]  [<ffffffff9229778d>] warn_slowpath_null+0x1d/0x20
[   34.918991]  [<ffffffffc03ce1fe>] uninit_mqd_hiq_sdma+0x4e/0x50 [amdgpu]
[   34.919028]  [<ffffffffc03d0ef7>] uninitialize+0x37/0xe0 [amdgpu]
[   34.919064]  [<ffffffffc03d15a6>] kernel_queue_uninit+0x16/0x30 [amdgpu]
[   34.919086]  [<ffffffffc03d26c2>] pm_uninit+0x12/0x20 [amdgpu]
[   34.919107]  [<ffffffffc03d4915>] stop_nocpsch+0x15/0x20 [amdgpu]
[   34.919129]  [<ffffffffc03c1dce>] kgd2kfd_suspend.part.4+0x2e/0x50 [amdgpu]
[   34.919150]  [<ffffffffc03c2667>] kgd2kfd_suspend+0x17/0x20 [amdgpu]
[   34.919171]  [<ffffffffc03c103a>] amdgpu_amdkfd_suspend+0x1a/0x20 [amdgpu]
[   34.919187]  [<ffffffffc02ec428>] amdgpu_device_suspend+0x88/0x3a0 [amdgpu]
[   34.919189]  [<ffffffff922e22cf>] ? enqueue_entity+0x2ef/0xbe0
[   34.919205]  [<ffffffffc02e8220>] amdgpu_pmops_suspend+0x20/0x30 [amdgpu]
[   34.919207]  [<ffffffff925c56ff>] pci_pm_suspend+0x6f/0x150
[   34.919208]  [<ffffffff925c5690>] ? pci_pm_freeze+0xf0/0xf0
[   34.919210]  [<ffffffff926b45c6>] dpm_run_callback+0x46/0x90
[   34.919212]  [<ffffffff926b49db>] __device_suspend+0xfb/0x2a0
[   34.919213]  [<ffffffff926b4b9f>] async_suspend+0x1f/0xa0
[   34.919214]  [<ffffffff922c918f>] async_run_entry_fn+0x3f/0x130
[   34.919216]  [<ffffffff922b9d4f>] process_one_work+0x17f/0x440
[   34.919217]  [<ffffffff922bade6>] worker_thread+0x126/0x3c0
[   34.919218]  [<ffffffff922bacc0>] ? manage_workers.isra.25+0x2a0/0x2a0
[   34.919220]  [<ffffffff922c1c31>] kthread+0xd1/0xe0
[   34.919221]  [<ffffffff922c1b60>] ? insert_kthread_work+0x40/0x40
[   34.919222]  [<ffffffff92974c1d>] ret_from_fork_nospec_begin+0x7/0x21
[   34.919224]  [<ffffffff922c1b60>] ? insert_kthread_work+0x40/0x40
[   34.919224] ---[ end trace 38cd9f65c963adad ]---

Signed-off-by: Kevin Wang <kevin1.wang@amd.com>
Reviewed-by: Oak Zeng <Oak.Zeng@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: use REG32_PCIE wrapper instead for psp

This patch uses REG32_PCIE wrapper instead of writting pci_index2 and reading
pci_data2 for psp. This sequence should be protected by pcie_idx_lock.

Signed-off-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: use REG32_PCIE wrapper instead for powerplay

This patch uses REG32_PCIE wrapper instead of writting pci_index2 and reading
pci_data2 for powerplay. This sequence should be protected by pcie_idx_lock.

Signed-off-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Fix issue with link_active state not correct for MST

[Why]
For MST, link not disabled until all streams disabled

[How]
Add check for stream_count before setting link_active = false for MST

Signed-off-by: Anthony Koo <Anthony.Koo@amd.com>
Reviewed-by: Wenjing Liu <Wenjing.Liu@amd.com>
Acked-by: Leo Li <sunpeng.li@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Fix reference counting for struct dc_sink.

Reference counting in amdgpu_dm_connector for amdgpu_dm_connector::dc_sink
and amdgpu_dm_connector::dc_em_sink as well as in dc_link::local_sink seems
to be out of shape. Thus make reference counting consistent for these
members and just plain increment the reference count when the variable
gets assigned and decrement when the pointer is set to zero or replaced.
Also simplify reference counting in selected function sopes to be sure the
reference is released in any case. In some cases add NULL pointer check
before dereferencing.
At a hand full of places a comment is placed to stat that the reference
increment happened already somewhere else.

This actually fixes the following kernel bug on my system when enabling
display core in amdgpu. There are some more similar bug reports around,
so it probably helps at more places.

   kernel BUG at mm/slub.c:294!
   invalid opcode: 0000 [#1] SMP PTI
   CPU: 9 PID: 1180 Comm: Xorg Not tainted 5.0.0-rc1+ #2
   Hardware name: Supermicro X10DAi/X10DAI, BIOS 3.0a 02/05/2018
   RIP: 0010:__slab_free+0x1e2/0x3d0
   Code: 8b 54 24 30 48 89 4c 24 28 e8 da fb ff ff 4c 8b 54 24 28 85 c0 0f 85 67 fe ff ff 48 8d 65 d8 5b 41 5c 41 5d 41 5e 41 5f 5d c3 <0f> 0b 49 3b 5c 24 28 75 ab 48 8b 44 24 30 49 89 4c 24 28 49 89 44
   RSP: 0018:ffffb0978589fa90 EFLAGS: 00010246
   RAX: ffff92f12806c400 RBX: 0000000080200019 RCX: ffff92f12806c400
   RDX: ffff92f12806c400 RSI: ffffdd6421a01a00 RDI: ffff92ed2f406e80
   RBP: ffffb0978589fb40 R08: 0000000000000001 R09: ffffffffc0ee4748
   R10: ffff92f12806c400 R11: 0000000000000001 R12: ffffdd6421a01a00
   R13: ffff92f12806c400 R14: ffff92ed2f406e80 R15: ffffdd6421a01a20
   FS:  00007f4170be0ac0(0000) GS:ffff92ed2fb40000(0000) knlGS:0000000000000000
   CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
   CR2: 0000562818aaa000 CR3: 000000045745a002 CR4: 00000000003606e0
   DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
   DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
   Call Trace:
    ? drm_dbg+0x87/0x90 [drm]
    dc_stream_release+0x28/0x50 [amdgpu]
    amdgpu_dm_connector_mode_valid+0xb4/0x1f0 [amdgpu]
    drm_helper_probe_single_connector_modes+0x492/0x6b0 [drm_kms_helper]
    drm_mode_getconnector+0x457/0x490 [drm]
    ? drm_connector_property_set_ioctl+0x60/0x60 [drm]
    drm_ioctl_kernel+0xa9/0xf0 [drm]
    drm_ioctl+0x201/0x3a0 [drm]
    ? drm_connector_property_set_ioctl+0x60/0x60 [drm]
    amdgpu_drm_ioctl+0x49/0x80 [amdgpu]
    do_vfs_ioctl+0xa4/0x630
    ? __sys_recvmsg+0x83/0xa0
    ksys_ioctl+0x60/0x90
    __x64_sys_ioctl+0x16/0x20
    do_syscall_64+0x5b/0x160
    entry_SYSCALL_64_after_hwframe+0x44/0xa9
   RIP: 0033:0x7f417110809b
   Code: 0f 1e fa 48 8b 05 ed bd 0c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 0f 1f 44 00 00 f3 0f 1e fa b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d bd bd 0c 00 f7 d8 64 89 01 48
   RSP: 002b:00007ffdd8d1c268 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
   RAX: ffffffffffffffda RBX: 0000562818a8ebc0 RCX: 00007f417110809b
   RDX: 00007ffdd8d1c2a0 RSI: 00000000c05064a7 RDI: 0000000000000012
   RBP: 00007ffdd8d1c2a0 R08: 0000562819012280 R09: 0000000000000007
   R10: 0000000000000000 R11: 0000000000000246 R12: 00000000c05064a7
   R13: 0000000000000012 R14: 0000000000000012 R15: 00007ffdd8d1c2a0
   Modules linked in: nfsv4 dns_resolver nfs lockd grace fscache fuse vfat fat amdgpu intel_rapl sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass crct10dif_pclmul chash gpu_sched crc32_pclmul snd_hda_codec_realtek ghash_clmulni_intel amd_iommu_v2 iTCO_wdt iTCO_vendor_support ttm snd_hda_codec_generic snd_hda_codec_hdmi ledtrig_audio snd_hda_intel drm_kms_helper snd_hda_codec intel_cstate snd_hda_core drm snd_hwdep snd_seq snd_seq_device intel_uncore snd_pcm intel_rapl_perf snd_timer snd soundcore ioatdma pcspkr intel_wmi_thunderbolt mxm_wmi i2c_i801 lpc_ich pcc_cpufreq auth_rpcgss sunrpc igb crc32c_intel i2c_algo_bit dca wmi hid_cherry analog gameport joydev

This patch is based on agd5f/drm-next-5.1-wip. This patch does not require
all of that, but agd5f/drm-next-5.1-wip contains at least one more dc_sink
counting fix that I could spot.

Signed-off-by: Mathias Fröhlich <Mathias.Froehlich@web.de>
Reviewed-by: Leo Li <sunpeng.li@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/powerplay: add missing breaks in polaris10_smumgr

This was noticed by Gustavo and his -Wimplicit-fallthrough
patches.  However, in this case, I believe we should have breaks
rather than falling though, that said, in practice we should
never fall through in the first place so there should be no
change in behavior.

Reviewed-by: Evan Quan <evan.quan@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: fix the confusing ppfeature mask calculations

Simplify the ppfeature mask calculations.

Signed-off-by: Evan Quan <evan.quan@amd.com>
Reviewed-by: Kenneth Feng <kenneth.feng@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/powerplay: print current clock level when dpm is disabled on vg20

When DPM for the specific clock is disabled, driver should still print out
current clock info for rocm-smi support on vega20

Signed-off-by: shaoyunl <shaoyun.liu@amd.com>
Reviewed-by: Eric Huang <JinhuiEric.Huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

Merge branch 'drm-next-5.1' of git://people.freedesktop.org/~agd5f/linux into drm-next

Fixes for 5.1:
amdgpu:
- Fix missing fw declaration after dropping old CI DPM code
- Fix debugfs access to registers beyond the MMIO bar size
- Fix context priority handling
- Add missing license on some new files
- Various cleanups and bug fixes

radeon:
- Fix missing break in CS parser for evergreen
- Various cleanups and bug fixes

sched:
- Fix entities with 0 run queues

Signed-off-by: Dave Airlie <airlied@redhat.com>
From: Alex Deucher <alexdeucher@gmail.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20190221214134.3308-1-alexander.deucher@amd.com

drm/amdgpu: Bump amdgpu version for context priority override.

Signed-off-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/powerplay: fix typo in BACO header guards

s/BOCO/BACO/g

Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/powerplay: fix return codes in BACO code

Use a proper return code rather than -1.

Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add missing license on baco files

Trivial.

Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

Merge https://gitlab.freedesktop.org/drm/msm into drm-next

On the display side, cleanups and fixes to enabled modifiers
(QCOM_COMPRESSED).  And otherwise mostly misc fixes all around.

Signed-off-by: Dave Airlie <airlied@redhat.com>
From: Rob Clark <robdclark@gmail.com>
Link: https://patchwork.freedesktop.org/patch/msgid/CAF6AEGuZ5uBKpf=fHvKpTiD10nychuEY8rnE+HeRz0QMvtY5_A@mail.gmail.com

Merge branch 'linux-5.1' of git://github.com/skeggsb/linux into drm-next

Various fixes/cleanups, along with initial support for SVM features
utilising HMM address-space mirroring and device memory migration.
There's a lot more work to do in these areas, both in terms of
features and efficiency, but these can slowly trickle in later down
the track.

Signed-off-by: Dave Airlie <airlied@redhat.com>
From: Ben Skeggs <skeggsb@gmail.com>
Link: https://patchwork.freedesktop.org/patch/msgid/CACAvsv5bsB4rRY1Gqa_Bp_KAd-v_q1rGZ4nYmOAQhceL0Nr-Xg@mail.gmail.com

drm/nouveau/dmem: use dma addresses during migration copies

Removes the need for temporary VMM mappings.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/dmem: use physical vram addresses during migration copies

Removes the need for temporary VMM mappings.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/dmem: extend copy function to allow direct use of physical addresses

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/svm: new ioctl to migrate process memory to GPU memory

This add an ioctl to migrate a range of process address space to the
device memory. On platform without cache coherent bus (x86, ARM, ...)
this means that CPU can not access that range directly, instead CPU
will fault which will migrate the memory back to system memory.

This is behind a staging flag so that we can evolve the API.

Signed-off-by: Jérôme Glisse <jglisse@redhat.com>

drm/nouveau/dmem: device memory helpers for SVM

Device memory can be use in SVM, in which case we do not have any of
the existing buffer object. This commit add infrastructure to allow
use of device memory without nouveau_bo. Again this is a temporary
solution until a rework of GPU memory management.

Signed-off-by: Jérôme Glisse <jglisse@redhat.com>

drm/nouveau/svm: initial support for shared virtual memory

This uses HMM to mirror a process' CPU page tables into a channel's page
tables, and keep them synchronised so that both the CPU and GPU are able
to access the same memory at the same virtual address.

While this code also supports Volta/Turing, it's only enabled for Pascal
GPUs currently due to channel recovery being unreliable right now on the
later GPUs.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau: prepare for enabling svm with existing userspace interfaces

For a channel to make use of SVM features, it requires a different GPU MMU
configuration than we would normally use, which is not desirable to switch
to unless a client is actively going to use SVM.

In order to supporting SVM without more extensive changes to the userspace
interfaces, the SVM_INIT ioctl needs to replace the previous configuration
safely.

The only way we can currently do this safely, accounting for some unlikely
failure conditions, is to allocate the new VMM without destroying the last
one, and prioritising the SVM-enabled configuration in the code that cares.

This will get cleaned up again further down the track.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/fault/gv100-: expose VoltaFaultBufferA

This nvclass exposes the replayable fault buffer, which will be used
by SVM to manage GPU page faults.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/fault/gp100: expose MaxwellFaultBufferA

This nvclass exposes the replayable fault buffer, which will be used
by SVM to manage GPU page faults.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/mmu/gp100-: support vmms with gcc/tex replayable faults enabled

Some GPU units are capable of supporting "replayable" page faults, where
the execution unit will wait for SW to fixup GPU page tables rather than
triggering a channel-fatal fault.

This feature isn't useful (it's harmful, even) unless something like HMM
is being used to manage events appearing in the replayable fault buffer,
so, it's disabled by default.

This commit allows a client to request it be enabled.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/mmu/gp100-: add privileged methods for fault replay/cancel

Host methods exist to do at least some of what we need, but we are not
currently pushing replay/cancels through a channel like UVM does as it's
not clear whether it's necessary in our case (UVM also updates PTEs with
the GPU).

UVM also pushes a software method for fault cancels on Pascal, seemingly
because the host methods don't appear to be sufficient.  If/when we want
to push the replay/cancel on the GPU, we can re-purpose the cancellation
code here to implement that swmthd.

Keep it simple for now, until we figure out exactly what we need here.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/mmu: add a privileged method to directly manage PTEs

This provides a somewhat more direct method of manipulating the GPU page
tables, which will be required to support SVM.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/mmu: store mapped flag separately from memory pointer

This will be used to support a privileged client providing PTEs directly,
without a memory object to use as a reference.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/mmu: support initialisation of client-managed address-spaces

NVKM is currently responsible for managing the allocation of a client's
GPU address-space, but there's various use-cases (ie. HMM address-space
mirroring) where giving a client more direct control is desirable.

This commit allows for a VMM to be created where the area allocated for
NVKM is limited to a client-specified window, the remainder of address-
space is controlled directly by the client.

Leaving a window is necessary to support various internal requirements,
but also to support existing allocation interfaces as not all of the HW
is capable of working with a HMM allocation.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: expose method to determine current context

MMU will need access to this info.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: expose fecs methods for pausing ctxsw

MMU will need access to these.

v2. Apply fix from Rhys Kidd to send correct FECS method for STOP_CTXSW.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/falcon: fix a few indentation issues

There are a few statements that are indented incorrectly. Fix these.

Signed-off-by: Colin Ian King <colin.king@canonical.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/mmu/gf100-: virtualise setting pdb base address for invalidation

It appears that Pascal and newer need something different.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/mmu/gf100-: make mmu invalidate function more general

Will want to reuse this for fault replay/cancellation swmthds.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: store fecs/gpccs falcon pointers in substructures

Future changes will want to add some additional things here, keep them
grouped together.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: move fecs bind_pointer into a function

Makes the code somewhat less magic.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: remove some unnecessary reg writes

This is already done during golden context creation.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: move fecs elpg setup into functions

Makes the code somewhat less magic.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>

drm/nouveau/gr/gf100-: move fecs discover_pm_image_size into a function

Makes the code somewhat less magic.

Signed-off-by: Ben Skeggs <bskeggs@redhat.com>