drm/amdgpu: add helper for VM PD/PT allocation parameters v3

Add a helper function to figure them out only once.

v2: fix typo with memset
v3: rebase on kfd changes (Alex)

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: remove extra root PD alignment

Just another leftover from radeon.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Acked-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: fix holding mn_lock while allocating memory

We can't hold the mn_lock while allocating memory.

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: amdgpu_ctx_add_fence can't fail

No more waiting for a fence done here.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Change kiq ring initialize sequence on gfx9

1. initialize kiq before initialize gfx ring.
2. set kiq ring ready immediately when kiq initialize
   successfully.
3. split function gfx_v9_0_kiq_resume into two functions.
     gfx_v9_0_kiq_resume is for kiq initialize.
     gfx_v9_0_kcq_resume is for kcq initialize.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Change kiq initialize/reset sequence on gfx8

1. initialize kiq before initialize gfx ring.
2. set kiq ring ready immediately when kiq initialize
   successfully.
3. split function gfx_v8_0_kiq_resume into two functions.
   gfx_v8_0_kiq_resume is for kiq initialize.
   gfx_v8_0_kcq_resume is for kcq initialize.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Refine gfx_v9_0_kcq_disable function

Send all kcq unmap_queue packets and then wait for
complete.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Remove duplicate code in gfx_v9_0.c

There are no any logical changes here.

1. if kcq can be enabled via kiq, we don't need to
   do kiq ring test.
2. amdgpu_ring_test_ring function can be used to
   sync the ring complete, remove the duplicate code.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Refine gfx_v8_0_kcq_disable function

Send all kcq unmap_queue packets and then wait for
complete.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Remove duplicate code in gfx_v8_0.c

There are no any logical changes here.

1. if kcq can be enabled via kiq, we don't need to
   do kiq ring test.
2. amdgpu_ring_test_ring function can be used to
   sync the ring complete, remove the duplicate code.

v2: alloc 6 (not 7) dws for unmap_queues

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Remove dead code in amdgpu_pm.c

As we have unify powergate_uvd/vce/mmhub to set_powergating_by_smu,
and set_powergating_by_smu was supported by both dpm and powerplay.
so remove the else case.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Power on uvd block when hw_fini

when hw_fini/suspend, smu only need to power on uvd block
if uvd pg is supported, don't need to call uvd to do hw_init.

v2: fix typo in patch descriptions and comments.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Tested-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Update power state at the end of smu hw_init.

For SI/Kv, the power state is managed by function
amdgpu_pm_compute_clocks.

when dpm enabled, we should call amdgpu_pm_compute_clocks
to update current power state instand of set boot state.

this change can fix the oops when kfd driver was enabled on Kv.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Tested-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Fix vce initialize failed on Kaveri/Mullins

Forgot to add vce pg support via smu for Kaveri/Mullins.

Fixes: 561a5c83eadd ("drm/amd/pp: Unify powergate_uvd/vce/mmhub
                       to set_powergating_by_smu")

v2: refine patch descriptions suggested by Michel

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Tested-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Remove duplicated power source update

when ac/dc switch, driver will be notified by acpi event.
then the power source will be updated. so don't need to
get power source when set power state.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Enable/disable gfx PG feature in rlc safe mode

This is required by gfx hw and can fix the rlc hang when
do s3 stree test on Cz/St.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Hang Zhou <hang.zhou@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Adjust the VM size based on system memory size v2

Set the VM size based on system memory size between the ASIC-specific
limits given by min_vm_size and max_bits. GFXv9 GPUs will keep their
default VM size of 256TB (48 bit). Only older GPUs will adjust VM size
depending on system memory size.

This makes more VM space available for ROCm applications on GFXv8 GPUs
that want to map all available VRAM and system memory in their SVM
address space.

v2:
* Clarify comment
* Round up memory size before >> 30
* Round up automatic vm_size to power of two

Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Acked-by: Junwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: implement soft_recovery for GFX9

Try to kill waves on the SQ.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: implement soft_recovery for GFX8 v2

Try to kill waves on the SQ.

v2: only for the GFX ring for now.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: implement soft_recovery for GFX7

Try to kill waves on the SQ.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add ring soft recovery v4

Instead of hammering hard on the GPU try a soft recovery first.

v2: reorder code a bit
v3: increase timeout to 10ms, increment GPU reset counter
v4: squash in compile fix (Christian)

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>

drm/amdgpu: move PD/PT bos on LRU again

The new bulk moving functionality is ready, the overhead of moving PD/PT bos to
LRU is fixed. So move them on LRU again.

Signed-off-by: Huang Rui <ray.huang@amd.com>
Tested-by: Mike Lothian <mike@fireburn.co.uk>
Tested-by: Dieter Nützel <Dieter@nuetzel-hh.de>
Acked-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: use bulk moves for efficient VM LRU handling (v6)

I continue to work for bulk moving that based on the proposal by Christian.

Background:
amdgpu driver will move all PD/PT and PerVM BOs into idle list. Then move all of
them on the end of LRU list one by one. Thus, that cause so many BOs moved to
the end of the LRU, and impact performance seriously.

Then Christian provided a workaround to not move PD/PT BOs on LRU with below
patch:
Commit 0bbf32026cf5ba41e9922b30e26e1bed1ecd38ae ("drm/amdgpu: band aid
validating VM PTs")

However, the final solution should bulk move all PD/PT and PerVM BOs on the LRU
instead of one by one.

Whenever amdgpu_vm_validate_pt_bos() is called and we have BOs which need to be
validated we move all BOs together to the end of the LRU without dropping the
lock for the LRU.

While doing so we note the beginning and end of this block in the LRU list.

Now when amdgpu_vm_validate_pt_bos() is called and we don't have anything to do,
we don't move every BO one by one, but instead cut the LRU list into pieces so
that we bulk move everything to the end in just one operation.

Test data:
+--------------+-----------------+-----------+---------------------------------------+
|              |The Talos        |Clpeak(OCL)|BusSpeedReadback(OCL)                  |
|              |Principle(Vulkan)|           |                                       |
+------------------------------------------------------------------------------------+
|              |                 |           |0.319 ms(1k) 0.314 ms(2K) 0.308 ms(4K) |
| Original     |  147.7 FPS      |  76.86 us |0.307 ms(8K) 0.310 ms(16K)             |
+------------------------------------------------------------------------------------+
| Orignial + WA|                 |           |0.254 ms(1K) 0.241 ms(2K)              |
|(don't move   |  162.1 FPS      |  42.15 us |0.230 ms(4K) 0.223 ms(8K) 0.204 ms(16K)|
|PT BOs on LRU)|                 |           |                                       |
+------------------------------------------------------------------------------------+
| Bulk move    |  163.1 FPS      |  40.52 us |0.244 ms(1K) 0.252 ms(2K) 0.213 ms(4K) |
|              |                 |           |0.214 ms(8K) 0.225 ms(16K)             |
+--------------+-----------------+-----------+---------------------------------------+

After test them with above three benchmarks include vulkan and opencl. We can
see the visible improvement than original, and even better than original with
workaround.

v2: move all BOs include idle, relocated, and moved list to the end of LRU and
put them together.
v3: remove unused parameter and use list_for_each_entry instead of the one with
save entry.
v4: move the amdgpu_vm_move_to_lru_tail after command submission, at that time,
all bo will be back on idle list.
v5: remove amdgpu_vm_move_to_lru_tail_by_list(), use bulk_moveable instread of
validated, and move ttm_bo_bulk_move_lru_tail() also into
amdgpu_vm_move_to_lru_tail().
v6: clean up and fix return value.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Huang Rui <ray.huang@amd.com>
Tested-by: Mike Lothian <mike@fireburn.co.uk>
Tested-by: Dieter Nützel <Dieter@nuetzel-hh.de>
Acked-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: add bulk move function on LRU

This function allow us to bulk move a group of BOs to the tail of their LRU.
The positions of group of BOs are stored on the (first, last) bulk_move_pos
structure.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Huang Rui <ray.huang@amd.com>
Tested-by: Mike Lothian <mike@fireburn.co.uk>
Tested-by: Dieter Nützel <Dieter@nuetzel-hh.de>
Acked-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: revise ttm_bo_move_to_lru_tail to support bulk moves

When move a BO to the end of LRU, it need remember the BO positions.
Make sure all moved bo in between "first" and "last". And they will be bulk
moving together.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Huang Rui <ray.huang@amd.com>
Tested-by: Mike Lothian <mike@fireburn.co.uk>
Tested-by: Dieter Nützel <Dieter@nuetzel-hh.de>
Acked-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: add helper structures for bulk moves on lru list

Add bulk move pos to store the pointer of first and last buffer object.
The list in between will be bulk moved on lru list.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Huang Rui <ray.huang@amd.com>
Tested-by: Mike Lothian <mike@fireburn.co.uk>
Tested-by: Dieter Nützel <Dieter@nuetzel-hh.de>
Acked-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add amdgpu_gmc_pd_addr helper

Add a helper to get the root PD address and remove the workarounds from
the GMC9 code for that.

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: remove gart.table_addr

We can easily figure out the address on the fly.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: rename gart.robj into gart.bo

sed -i "s/gart.robj/gart.bo/" drivers/gpu/drm/amd/amdgpu/*.c
sed -i "s/gart.robj/gart.bo/" drivers/gpu/drm/amd/amdgpu/*.h

Just cleaning up radeon leftovers.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: move setting the GART addr into TTM

Move setting the GART addr for window based copies into the TTM code who
uses it.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: cleanup VM handling in the CS a bit

Add a helper function for getting the root PD addr and cleanup join the
two VM related functions and cleanup the function name.

No functional change.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: validate the VM root PD from the VM code

Preparation for following changes. This validates the root PD twice,
but the overhead of that should be minimal.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: cleanup GPU recovery check a bit (v2)

Check if we should call the function instead of providing the forced
flag.

v2: rebase on KFD changes (Alex)

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: amdgpu_kiq_reg_write_reg_wait() can be static

Fixes: d790449835e6 ("drm/amdgpu: use kiq to do invalidate tlb")
Reviewed-by: Emily Deng <Emily.Deng@amd.com>
Signed-off-by: kbuild test robot <fengguang.wu@intel.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/display: add support for LVDS (v5)

This adds support for LVDS displays.

v2: add support for spread spectrum, sink detect
v3: clean up enable_lvds_output
v4: fix up link_detect
v5: remove assert on 888 format

Bug: https://bugs.freedesktop.org/show_bug.cgi?id=105880
Reviewed-by: Harry Wentland <harry.wentland@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Don't use kiq in gpu reset

When in gpu reset, don't use kiq, it will generate more TDR.

Signed-off-by: Emily Deng <Emily.Deng@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>.
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/scheduler: avoid redundant shifting of the entity v2

do not remove entity from the rq if the current rq is from
the least loaded scheduler.

Signed-off-by: Nayan Deshmukh <nayan26deshmukh@gmail.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: indent an if statement

The if statement isn't indented and it makes static checkers complain.

Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Reviewed-by: Harry Wentland <harry.wentland@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: fix a compile warning

Fix comile warning like,
  CC [M]  drivers/gpu/drm/i915/gvt/execlist.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/instmem/nv50.o
  CC [M]  drivers/gpu/drm/radeon/btc_dpm.o
  CC [M]  drivers/isdn/hisax/avm_a1p.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dcn10/dcn10_dpp.o
drivers/gpu/drm/amd/amdgpu/../display/dc/dcn10/dcn10_hw_sequencer.c: In function ‘dcn10_update_mpcc’:
drivers/gpu/drm/amd/amdgpu/../display/dc/dcn10/dcn10_hw_sequencer.c:1903:9: warning: missing braces around initializer [-Wmissing-braces]
  struct mpcc_blnd_cfg blnd_cfg = {0};
         ^
drivers/gpu/drm/amd/amdgpu/../display/dc/dcn10/dcn10_hw_sequencer.c:1903:9: warning: (near initialization for ‘blnd_cfg.black_color’) [-Wmissing-braces]

Acked-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Wen Yang <wen.yang99@zte.com.cn>
Reviewed-by: Jiang Biao <jiang.biao2@zte.com.cn>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/sriov: Only sriov runtime support use kiq

For sriov, don't use kiq in exclusive mode, as don't know how long time
it will take, some times it will occur exclusive timeout.

Signed-off-by: Emily Deng <Emily.Deng@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: fix sdma doorbell range setting

Use the old doorbell range setting until the driver is
able to support more sdma queues.

Signed-off-by: Evan Quan <evan.quan@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Fix compile warning

In function ‘gfx_v9_0_check_fw_write_wait’:
warning: enumeration value ‘CHIP_TAHITI’ not handled in switch [-Wswitch]

Always add default case in case there is no match

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: rework ctx entity creation

Use a fixed number of entities for each hardware IP.

The number of compute entities is reduced to four, SDMA keeps it two
entities and all other engines just expose one entity.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: cleanup HW_IP query

Move the code into a separate function.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/scheduler: Add stopped flag to drm_sched_entity

The flag will prevent another thread from same process to
reinsert the entity queue into scheduler's rq after it was already
removewd from there by another thread during drm_sched_entity_flush.

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: remove fulll access for suspend phase1

There is no need for gpu full access for suspend phase1
because under virtualization there is no hw register access
for dce block.

Signed-off-by: Yintian Tao <yttao@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: use kiq to do invalidate tlb

To avoid the tlb flush not interrupted by world switch, use kiq and one
command to do tlb invalidate.

v2:
Refine the invalidate lock position.

Signed-off-by: Emily Deng <Emily.Deng@amd.com>
Reviewed-and-Tested-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Remove the sriov checking and add firmware checking

Unify bare metal and sriov, and add firmware checking for
reg write and reg wait unify command.

Signed-off-by: Emily Deng <Emily.Deng@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Reviewed-and-Tested-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: implement DPMS DTN test v2

[why]
Existing DTN infrastructure in driver is hacky.  It uses implicit log
names, and also incorrect escape ID.

[how]
- Implement using generic DTN escape ID.
- Move file logging functionality from driver to to script; driver now outputs to string/buffer
- Move HWSS debug functionality to separate c file
- Add debug functionalty for per-block logging as CSV
- Add pretty print in python

Signed-off-by: Jun Lei <Jun.Lei@amd.com>
Reviewed-by: Tony Cheng <Tony.Cheng@amd.com>
Acked-by: Bhawanpreet Lakha <Bhawanpreet.Lakha@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: move edp fast boot optimization flag to stream

[Why]
During S4/S3 stress test it is possible to resume from S4 without
calling mode set on eDP, meaning high level optimization flag is not
reset. If this is followed by an S3 resume call, driver will see
optimization flag is set and consume it and think backend is powered
on when in fact it is not.

This results in PHY being off in sequence where
S4->Resume->S3->Resume->ApplyOpt->black screen.

[How]
Move optimization flag to stream instead of a DC flag.

Signed-off-by: Anthony Koo <Anthony.Koo@amd.com>
Reviewed-by: Tony Cheng <Tony.Cheng@amd.com>
Acked-by: Bhawanpreet Lakha <Bhawanpreet.Lakha@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Combine dce80 and dce100 i2c hw functions

[Why]
There are two versions of the hw function pointers: one for dce80
and one for all other versions.  These paired functions are
nearly identical.  dce80 and dce100 should not require
different i2c access functions.

[How]
Combine each pair of functions into a single function.  Mostly
the new functions are based on the dce100 versions as those
versions are newer, support more features, and
were more maintained.

Signed-off-by: David Francis <David.Francis@amd.com>
Reviewed-by: Sun peng Li <Sunpeng.Li@amd.com>
Acked-by: Bhawanpreet Lakha <Bhawanpreet.Lakha@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Define registers for dcn10

Define register for dcn10 for future changes

Signed-off-by: Nikola Cornij <nikola.cornij@amd.com>
Reviewed-by: Tony Cheng <Tony.Cheng@amd.com>
Acked-by: Bhawanpreet Lakha <Bhawanpreet.Lakha@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Program csc matrix as part of stream update

Add csc_transform struct to dc_stream_update, and program if set when
updating streams

Signed-off-by: SivapiriyanKumarasamy <sivapiriyan.kumarasamy@amd.com>
Reviewed-by: Anthony Koo <Anthony.Koo@amd.com>
Acked-by: Bhawanpreet Lakha <Bhawanpreet.Lakha@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Create new i2c resource

[Why]
I2C code did not match dc resource model and was generally
unpleasant

[How]
Move code into new svelte dce_i2c files, replacing various i2c
objects with two structs: dce_i2c_sw and dce_i2c_hw.  Fully split
sw and hw code paths.  Remove all redundant declarations.  Use
address lists to distinguish between versions.  Change dce80 code
to newer register access macros.

Signed-off-by: David Francis <David.Francis@amd.com>
Reviewed-by: Tony Cheng <Tony.Cheng@amd.com>
Acked-by: Bhawanpreet Lakha <Bhawanpreet.Lakha@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: Do not evict VRAM on APUs with disabled HIBERNATE

Improve commit d796d844 (drm/radeon/kms: make hibernate work on IGPs) to
only migrate VRAM objects if the Linux kernel is actually built with
support for hibernation (suspend to disk).

Link: https://bugs.freedesktop.org/show_bug.cgi?id=100941
Signed-off-by: Paul Menzel <pmenzel@molgen.mpg.de>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Do not evict VRAM on APUs with disabled HIBERNATE

Improve commit d796d844 (drm/radeon/kms: make hibernate work on IGPs) to
only migrate VRAM objects if the Linux kernel is actually built with
support for hibernation (suspend to disk).

The better solution is to get the information, if this is suspend or
hibernate, from `amdgpu_device_suspend()`, but that’s more involved, so
apply the simple solution first.

Link: https://bugs.freedesktop.org/show_bug.cgi?id=107277
Signed-off-by: Paul Menzel <pmenzel@molgen.mpg.de>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add status checking after fw is loaded

The status field must be 0 after FW is loaded.

Signed-off-by: Huang Rui <ray.huang@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: remove dead codes

These codes are not used.

Signed-off-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: fix VM size reporting on Raven

Raven doesn't have an VCE block and so also no buggy VCE firmware.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Acked-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Set clock ungate state when suspend/fini

After set power ungate state, set clock ungate state
before when suspend or fini.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Set power ungate state when suspend/fini

Unify to set power ungate state at the begin of suspend/fini.
Remove the workaround code for gfx off feature in
amdgpu_device.c.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Refine function name and function args

There are no any logical changes here.

1. change function names:
   amdgpu_device_ip_late_set_pg/cg_state to
   amdgpu_device_set_pg/cg_state.
2. add a function argument cg/pg_state, so
   we can enable/disable cg/pg through those functions

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Program gamut remap as part of stream update

Add gamut remap to dc_stream_update struct, and program if set when updating
streams.

Signed-off-by: SivapiriyanKumarasamy <sivapiriyan.kumarasamy@amd.com>
Reviewed-by: Anthony Koo <Anthony.Koo@amd.com>
Acked-by: Leo Li <sunpeng.li@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: HDR dynamic meta should be treated as stream update

[Why]
Recently we fixed HDR static meta using AFMT registers to be treated as
fast stream update.
Dynamic meta is still being treated as (full) surface update because it
touches HUBP and it travels with pipe data.
Here we change it to be (fast) stream update.
Note, originally we also wanted to redesign here a bit, but without OS
level support for true dynamic meta, it's left the same. We are simply
using HW that can do dynamic meta to send HDR static meta, I still prefer
keeping it in one static meta type then defining dynamic meta types to
hold the same info. Once we know how OS interfaces look like, we can
do proper design.

[How]
Move dyn meta update from update_hubp_dpp to commit_planes_do_stream_update

Signed-off-by: Krunoslav Kovac <Krunoslav.Kovac@amd.com>
Reviewed-by: Tony Cheng <Tony.Cheng@amd.com>
Acked-by: Bhawanpreet Lakha <Bhawanpreet.Lakha@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Handle HDR meta update as fast update

[Why]
Vesa DPMS tool sends different HDR meta in OS flips without changing output
parameters. We don't properly update HDR info frame:
- we label HDR meta update as fast update
- when updating HW info frame, we only do it if full update

[How]
It should still be fast update, so when doing HW infoframe update,
do it always no matter the update type.
Also, don't request passive flip for HDR meta update only without output
transfer function or color space changed.

Signed-off-by: Krunoslav Kovac <Krunoslav.Kovac@amd.com>
Reviewed-by: Tony Cheng <Tony.Cheng@amd.com>
Acked-by: Leo Li <sunpeng.li@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Program vsc_infopacket in commit_planes_for_stream

Signed-off-by: Alvin lee <alvin.lee3@amd.com>
Reviewed-by: Jun Lei <Jun.Lei@amd.com>
Acked-by: Harry Wentland <harry.wentland@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Enable Stereo in Dal3

- program infoframe for Stereo
- program stereo flip control registers properly

v2: Add missing license headers

Signed-off-by: Alvin lee <alvin.lee3@amd.com>
Reviewed-by: Tony Cheng <Tony.Cheng@amd.com>
Acked-by: Harry Wentland <harry.wentland@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Program vline interrupt on FAST update

Signed-off-by: SivapiriyanKumarasamy <sivapiriyan.kumarasamy@amd.com>
Reviewed-by: Tony Cheng <Tony.Cheng@amd.com>
Reviewed-by: Anthony Koo <Anthony.Koo@amd.com>
Acked-by: Harry Wentland <harry.wentland@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: decouple front and backend pgm using dpms_off as backend enable flag

Signed-off-by: Samson Tam <Samson.Tam@amd.com>
Reviewed-by: Anthony Koo <Anthony.Koo@amd.com>
Acked-by: Harry Wentland <harry.wentland@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: add config for sending VSIF

Signed-off-by: Anthony Koo <Anthony.Koo@amd.com>
Reviewed-by: Aric Cyr <Aric.Cyr@amd.com>
Acked-by: Harry Wentland <harry.wentland@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Don't force UPDATE_TYPE_FULL if stream_update has hdr_static_metadata

This was missed when pushing public patch for 3e3a40b03847 (drm/amd/display:
Updated HDR Static Metadata to directly take info packet raw)

This is currently no problem yet since we're not doing HDR on Linux yet.

Signed-off-by: Harry Wentland <harry.wentland@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Correct rounding calcs in mod_freesync_is_valid_range

Signed-off-by: Anthony Koo <Anthony.Koo@amd.com>
Reviewed-by: Aric Cyr <Aric.Cyr@amd.com>
Acked-by: Harry Wentland <harry.wentland@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: refactor vupdate interrupt registration

We only need to register once OS calls the interrupt control.
Also, if we are entering static screen mode, disable after ramping is done.
Disable shall be done via timer of 2 seconds regardless of ramping
complete or not, just to simplify.

Also, ramp to mid instead of min, due to better flicker performance...

Signed-off-by: Anthony Koo <Anthony.Koo@amd.com>
Reviewed-by: Aric Cyr <Aric.Cyr@amd.com>
Acked-by: Harry Wentland <harry.wentland@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: fix LFC tearing at top of screen

Tearing occurred because new VTOTAL MIN/MAX was being programmed
too early.
The flip can happen within the VUPDATE high region, and the new min/max
would take effect immediately. But this means that frame is not variable
anymore, and tearing would occur when the flip actually happens.

The fixed insert duration should be programmed on the first VUPDATE
interrupt instead.

Signed-off-by: Anthony Koo <Anthony.Koo@amd.com>
Reviewed-by: Aric Cyr <Aric.Cyr@amd.com>
Acked-by: Harry Wentland <harry.wentland@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Add back code to allow for rounding error

Signed-off-by: Anthony Koo <Anthony.Koo@amd.com>
Reviewed-by: Aric Cyr <Aric.Cyr@amd.com>
Acked-by: Harry Wentland <harry.wentland@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Fix bug that causes black screen

Ignore MSA bit on DP display is usually set during SetTimings, but
there was a case where the module thought refresh rate was not valid
and ignore MSA bit was not set.

Later, a valid refresh rate range was requested but since ignore MSA bit
not set, it caused black screen.

Issue if with how the module checked for VRR support. Fix up that logic.
DM should call new valid_range function to determine if timing is supported.

Signed-off-by: Anthony Koo <Anthony.Koo@amd.com>
Reviewed-by: Aric Cyr <Aric.Cyr@amd.com>
Acked-by: Harry Wentland <harry.wentland@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Fix bug where refresh rate becomes fixed

This issue occurs if refresh rate range is very small and lfc is not used.
When frame spikes occur, refresh rate becomes fixed and will not restore properly

Signed-off-by: Anthony Koo <Anthony.Koo@amd.com>
Reviewed-by: Aric Cyr <Aric.Cyr@amd.com>
Acked-by: Harry Wentland <harry.wentland@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: add method to check for supported range

Signed-off-by: Anthony Koo <Anthony.Koo@amd.com>
Reviewed-by: Aric Cyr <Aric.Cyr@amd.com>
Acked-by: Harry Wentland <harry.wentland@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/display: Refactor FreeSync module

Remove dependency on internal sink map and instead
use existing stream and plane state

Signed-off-by: Anthony Koo <Anthony.Koo@amd.com>
Signed-off-by: Harry Wentland <harry.wentland@amd.com>
Reviewed-by: Tony Cheng <Tony.Cheng@amd.com>
Acked-by: Harry Wentland <harry.wentland@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: fill in amdgpu_dm_remove_sink_from_freesync_module

Add code to tear down freesync modules when disabled.

Signed-off-by: Harry Wentland <harry.wentland@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:change VEGA booting with firmware loaded by PSP

With PSP firmware loading, TMR mc address is supposed to be used.

Signed-off-by: James Zhu <James.Zhu@amd.com>
Acked-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Feifei Xu <Feifei.Xu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: added support 2nd UVD instance

Added psp fw loading support for vega20 2nd UVD instance.

Signed-off-by: Evan Quan <evan.quan@amd.com>
Acked-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Feifei Xu <Feifei.Xu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/doc: Adapt GPU scheduler references for renamed C file

Fixes: "drm/scheduler: rename gpu_scheduler.c to sched_main.c"
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: use entity instead of ring for CS

Further demangle ring from entity handling.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: move context related stuff to amdgpu_ctx.h

Further unmangle amdgpu.h.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: remove ring lru handling

Not needed any more.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: remove the queue manager

Not needed any more since that is now done by the scheduler.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: use scheduler load balancing for compute CS

Start to use the scheduler load balancing for userspace compute
command submissions.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: use scheduler load balancing for SDMA CS

Start to use the scheduler load balancing for userspace SDMA
command submissions.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: use new scheduler load balancing for VMs

Instead of the fixed round robin use let the scheduler balance the load
of page table updates.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Change VCE booting with firmware loaded by PSP

With PSP firmware loading, TMR mc address is supposed to be used.

Signed-off-by: James Zhu <James.Zhu@amd.com>
Acked-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Feifei Xu <Feifei.Xu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/scheduler: rename gpu_scheduler.c to sched_main.c

Better match the naming of the other components.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/scheduler: cleanup entity coding style

Cleanup coding style in sched_entity.c

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/scheduler: move entity handling into separate file

This is complex enough on it's own. Move it into a separate C file.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/scheduler: trivial error handling fix

Return -ENOMEM when allocating the rq_list fails.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: fix integer overflow test in amdgpu_bo_list_create()

We accidentally left out the size of the amdgpu_bo_list struct.  It
could lead to memory corruption on 32 bit systems.  You'd have to
pick the absolute maximum and set "num_entries == 59652323" then size
would wrap to 16 bytes.

Fixes: 920990cb080a ("drm/amdgpu: allocate the bo_list array after the list")
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Reviewed-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Bas Nieuwenhuizen <basni@chromium.org>
Signed-off-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/pp: OverDrive gfx domain voltage on Tonga

Also ajust the gfx domain voltage on Tonga when user overdriver
the voltage.

For Tonga, Driver do not update user's setting to voltage table
in smu, we only pick up a minimum value from voltage table that
not less than the user's setting.

v2: fix a typo

Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Cancel the delay work when suspend

Cancel the delay work to avoid the corner case that
ib test was not running when suspend

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: remove experimental flag for vega20

Now that PSP and SMU support is in place.

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/psp: Enlarge PSP TMR SIZE from 3M to 4M.

Enlarge the PSP TMR SIZE to 4M for dual UVD fw front-door loading.

Signed-off-by: Feifei Xu <Feifei.Xu@amd.com>
Reviewed-by: Evan Quan <evan.quan@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>