]> git.proxmox.com Git - mirror_qemu.git/log
mirror_qemu.git
3 years agoiotests: Restrict some Python tests to file
Max Reitz [Tue, 27 Oct 2020 19:05:53 +0000 (20:05 +0100)]
iotests: Restrict some Python tests to file

Most Python tests are restricted to the file protocol (without
explicitly saying so), but these are the ones that would break
./check -fuse -qcow2.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20201027190600.192171-14-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
3 years agoiotests/091: Use _cleanup_qemu instad of "wait"
Max Reitz [Tue, 27 Oct 2020 19:05:52 +0000 (20:05 +0100)]
iotests/091: Use _cleanup_qemu instad of "wait"

If the test environment has some other child processes running (like a
storage daemon that provides a FUSE export), then "wait" will never
finish.  Use wait=yes _cleanup_qemu instead.

(We need to discard the output so there is no change to the reference
output.)

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20201027190600.192171-13-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
3 years agoiotests: Derive image names from $TEST_IMG
Max Reitz [Tue, 27 Oct 2020 19:05:51 +0000 (20:05 +0100)]
iotests: Derive image names from $TEST_IMG

Avoid creating images with custom filenames in $TEST_DIR, because
non-file protocols may want to keep $TEST_IMG (and all other test
images) in some other directory.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20201027190600.192171-12-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
3 years agoiotests/046: Avoid renaming images
Max Reitz [Tue, 27 Oct 2020 19:05:50 +0000 (20:05 +0100)]
iotests/046: Avoid renaming images

This generally does not work on non-file protocols.  It is better to
create the image with the final name from the start, and most tests do
this already.  Let 046 follow suit.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20201027190600.192171-11-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
3 years agoiotests: Use convert -n in some cases
Max Reitz [Tue, 27 Oct 2020 19:05:49 +0000 (20:05 +0100)]
iotests: Use convert -n in some cases

qemu-img convert (without -n) can often be replaced by a combination of
_make_test_img + qemu-img convert -n.  Doing so allows converting to
protocols that do not allow direct file creation, such as FUSE exports.
The only problem is that for formats other than qcow2 and qed (qcow1 at
least), this may lead to high disk usage for some reason, so we cannot
do it everywhere.

But we can do it in 028 and 089, so let us do that so they can run on
FUSE exports.  Also, in 028 this allows us to remove a 9-line comment
that used to explain why we cannot safely filter drive-backup's image
creation output.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20201027190600.192171-10-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
3 years agoiotests: Do not pipe _make_test_img
Max Reitz [Tue, 27 Oct 2020 19:05:48 +0000 (20:05 +0100)]
iotests: Do not pipe _make_test_img

Executing _make_test_img as part of a pipe will undo all variable
changes it has done.  As such, this could not work with FUSE (because
we want to remember all of our exports and their qemu instances).

Replace the pipe by a temporary file in 071 and 174 (the two tests that
can run on FUSE).

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20201027190600.192171-9-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
3 years agoiotests: Do not needlessly filter _make_test_img
Max Reitz [Tue, 27 Oct 2020 19:05:47 +0000 (20:05 +0100)]
iotests: Do not needlessly filter _make_test_img

In most cases, _make_test_img does not need a _filter_imgfmt on top.  It
does that by itself.

(The exception is when IMGFMT has been overwritten but TEST_IMG has not.
In such cases, we do need a _filter_imgfmt on top to filter the test's
original IMGFMT from TEST_IMG.)

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20201027190600.192171-8-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
3 years agofuse: Implement hole detection through lseek
Max Reitz [Tue, 27 Oct 2020 19:05:46 +0000 (20:05 +0100)]
fuse: Implement hole detection through lseek

This is a relatively new feature in libfuse (available since 3.8.0,
which was released in November 2019), so we have to add a dedicated
check whether it is available before making use of it.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20201027190600.192171-7-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
3 years agofuse: (Partially) implement fallocate()
Max Reitz [Tue, 27 Oct 2020 19:05:45 +0000 (20:05 +0100)]
fuse: (Partially) implement fallocate()

This allows allocating areas after the (old) EOF as part of a growing
resize, writing zeroes, and discarding.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20201027190600.192171-6-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
3 years agofuse: Allow growable exports
Max Reitz [Tue, 27 Oct 2020 19:05:44 +0000 (20:05 +0100)]
fuse: Allow growable exports

These will behave more like normal files in that writes beyond the EOF
will automatically grow the export size.

As an optimization, keep the RESIZE permission for growable exports so
we do not have to take it for every post-EOF write.  (This permission is
not released when the export is destroyed, because at that point the
BlockBackend is destroyed altogether anyway.)

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20201027190600.192171-5-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
3 years agofuse: Implement standard FUSE operations
Max Reitz [Tue, 27 Oct 2020 19:05:43 +0000 (20:05 +0100)]
fuse: Implement standard FUSE operations

This makes the export actually useful instead of only producing errors
whenever it is accessed.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20201027190600.192171-4-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
3 years agofuse: Allow exporting BDSs via FUSE
Max Reitz [Tue, 27 Oct 2020 19:05:42 +0000 (20:05 +0100)]
fuse: Allow exporting BDSs via FUSE

block-export-add type=fuse allows mounting block graph nodes via FUSE on
some existing regular file.  That file should then appears like a raw
disk image, and accesses to it result in accesses to the exported BDS.

Right now, we only implement the necessary block export functions to set
it up and shut it down.  We do not implement any access functions, so
accessing the mount point only results in errors.  This will be
addressed by a followup patch.

We keep a hash table of exported mount points, because we want to be
able to detect when users try to use a mount point twice.  This is
because we invoke stat() to check whether the given mount point is a
regular file, but if that file is served by ourselves (because it is
already used as a mount point), then this stat() would have to be served
by ourselves, too, which is impossible to do while we (as the caller)
are waiting for it to settle.  Therefore, keep track of mount point
paths to at least catch the most obvious instances of that problem.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20201027190600.192171-3-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
3 years agomeson: Detect libfuse
Max Reitz [Tue, 27 Oct 2020 19:05:41 +0000 (20:05 +0100)]
meson: Detect libfuse

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20201027190600.192171-2-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
3 years agoblock/iscsi: Use lock guard macros
Gan Qixin [Thu, 3 Dec 2020 07:50:55 +0000 (15:50 +0800)]
block/iscsi: Use lock guard macros

Replace manual lock()/unlock() calls with lock guard macros
(QEMU_LOCK_GUARD/WITH_QEMU_LOCK_GUARD) in block/iscsi.

Signed-off-by: Gan Qixin <ganqixin@huawei.com>
Message-Id: <20201203075055.127773-5-ganqixin@huawei.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
3 years agoblock/throttle-groups: Use lock guard macros
Gan Qixin [Thu, 3 Dec 2020 07:50:54 +0000 (15:50 +0800)]
block/throttle-groups: Use lock guard macros

Replace manual lock()/unlock() calls with lock guard macros
(QEMU_LOCK_GUARD/WITH_QEMU_LOCK_GUARD) in block/throttle-groups.

Signed-off-by: Gan Qixin <ganqixin@huawei.com>
Message-Id: <20201203075055.127773-4-ganqixin@huawei.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
3 years agoblock/curl: Use lock guard macros
Gan Qixin [Thu, 3 Dec 2020 07:50:53 +0000 (15:50 +0800)]
block/curl: Use lock guard macros

Replace manual lock()/unlock() calls with lock guard macros
(QEMU_LOCK_GUARD/WITH_QEMU_LOCK_GUARD) in block/curl.

Signed-off-by: Gan Qixin <ganqixin@huawei.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201203075055.127773-3-ganqixin@huawei.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
3 years agoblock/accounting: Use lock guard macros
Gan Qixin [Thu, 3 Dec 2020 07:50:52 +0000 (15:50 +0800)]
block/accounting: Use lock guard macros

Replace manual lock()/unlock() calls with lock guard macros
(QEMU_LOCK_GUARD/WITH_QEMU_LOCK_GUARD) in block/accounting.

Signed-off-by: Gan Qixin <ganqixin@huawei.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201203075055.127773-2-ganqixin@huawei.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
3 years agoMerge remote-tracking branch 'remotes/bonzini-gitlab/tags/for-upstream' into staging
Peter Maydell [Fri, 11 Dec 2020 13:50:35 +0000 (13:50 +0000)]
Merge remote-tracking branch 'remotes/bonzini-gitlab/tags/for-upstream' into staging

* Fix for NULL segments (Bin Meng)
* Support for 32768 CPUs on x86 without IOMMU (David)
* PDEP/PEXT fix and testcase (myself)
* Remove bios_name and ram_size globals (myself)
* qemu_init rationalization (myself)
* Update kernel-doc (myself + upstream patches)
* Propagate MemTxResult across DMA and PCI functions (Philippe)
* Remove master/slave when applicable (Philippe)
* WHPX support for in-kernel irqchip (Sunil)

# gpg: Signature made Thu 10 Dec 2020 17:21:50 GMT
# gpg:                using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg:                issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini-gitlab/tags/for-upstream: (113 commits)
  scripts: kernel-doc: remove unnecessary change wrt Linux
  Revert "docs: temporarily disable the kernel-doc extension"
  scripts: kernel-doc: use :c:union when needed
  scripts: kernel-doc: split typedef complex regex
  scripts: kernel-doc: fix typedef parsing
  Revert "kernel-doc: Handle function typedefs that return pointers"
  Revert "kernel-doc: Handle function typedefs without asterisks"
  scripts: kernel-doc: try to use c:function if possible
  scripts: kernel-doc: fix line number handling
  scripts: kernel-doc: allow passing desired Sphinx C domain dialect
  scripts: kernel-doc: don't mangle with parameter list
  scripts: kernel-doc: fix typedef identification
  scripts: kernel-doc: reimplement -nofunction argument
  scripts: kernel-doc: fix troubles with line counts
  scripts: kernel-doc: use a less pedantic markup for funcs on Sphinx 3.x
  scripts: kernel-doc: make it more compatible with Sphinx 3.x
  Revert "kernel-doc: Use c:struct for Sphinx 3.0 and later"
  Revert "scripts/kerneldoc: For Sphinx 3 use c:macro for macros with arguments"
  scripts: kernel-doc: add support for typedef enum
  kernel-doc: add support for ____cacheline_aligned attribute
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
3 years agoMerge remote-tracking branch 'remotes/ehabkost/tags/machine-next-pull-request' into...
Peter Maydell [Fri, 11 Dec 2020 12:16:33 +0000 (12:16 +0000)]
Merge remote-tracking branch 'remotes/ehabkost/tags/machine-next-pull-request' into staging

Machine queue, 2020-12-10

Some patches that were queued after 5.2 soft freeze.

# gpg: Signature made Thu 10 Dec 2020 22:41:29 GMT
# gpg:                using RSA key 5A322FD5ABC4D3DBACCFD1AA2807936F984DC5A6
# gpg:                issuer "ehabkost@redhat.com"
# gpg: Good signature from "Eduardo Habkost <ehabkost@redhat.com>" [full]
# Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF  D1AA 2807 936F 984D C5A6

* remotes/ehabkost/tags/machine-next-pull-request:
  i386/cpu: Make the Intel PT LIP feature configurable
  sev: add sev-inject-launch-secret
  qom: code hardening - have bound checking while looping with integer value

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
3 years agoi386/cpu: Make the Intel PT LIP feature configurable
Luwei Kang [Wed, 2 Dec 2020 10:10:42 +0000 (18:10 +0800)]
i386/cpu: Make the Intel PT LIP feature configurable

The current implementation will disable the guest Intel PT feature
if the Intel PT LIP feature is supported on the host, but the LIP
feature is comming soon(e.g. SnowRidge and later).

This patch will make the guest LIP feature configurable and Intel
PT feature can be enabled in guest when the guest LIP status same
with the host.

Signed-off-by: Luwei Kang <luwei.kang@intel.com>
Message-Id: <20201202101042.11967-1-luwei.kang@intel.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
3 years agosev: add sev-inject-launch-secret
Tobin Feldman-Fitzthum [Tue, 27 Oct 2020 17:03:03 +0000 (13:03 -0400)]
sev: add sev-inject-launch-secret

AMD SEV allows a guest owner to inject a secret blob
into the memory of a virtual machine. The secret is
encrypted with the SEV Transport Encryption Key and
integrity is guaranteed with the Transport Integrity
Key. Although QEMU facilitates the injection of the
launch secret, it cannot access the secret.

Signed-off-by: Tobin Feldman-Fitzthum <tobin@linux.ibm.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Brijesh Singh <brijesh.singh@amd.com>
Message-Id: <20201027170303.47550-1-tobin@linux.ibm.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
3 years agoqom: code hardening - have bound checking while looping with integer value
Ani Sinha [Mon, 21 Sep 2020 09:33:25 +0000 (15:03 +0530)]
qom: code hardening - have bound checking while looping with integer value

Object property insertion code iterates over an integer to get an unused
index that can be used as an unique name for an object property. This loop
increments the integer value indefinitely. Although very unlikely, this can
still cause an integer overflow.
In this change, we fix the above code by checking against INT16_MAX and making
sure that the interger index does not overflow beyond that value. If no
available index is found, the code would cause an assertion failure. This
assertion failure is necessary because the callers of the function do not check
the return value for NULL.

Signed-off-by: Ani Sinha <ani@anisinha.ca>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200921093325.25617-1-ani@anisinha.ca>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
3 years agoscripts: kernel-doc: remove unnecessary change wrt Linux
Paolo Bonzini [Mon, 30 Nov 2020 12:21:12 +0000 (07:21 -0500)]
scripts: kernel-doc: remove unnecessary change wrt Linux

A comment in kernel-doc mentions QEMU's qatomic_set macro, but since
this code originated in Linux we should just revert it and stay as close
to the kernel's copy of the script as possible.

The change was introduced (more or less unintentionally) in QEMU commit
commit d73415a31547, which did a global search-and-replace of QEMU's
atomic access macros.

Suggested-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoRevert "docs: temporarily disable the kernel-doc extension"
Paolo Bonzini [Mon, 30 Nov 2020 12:17:45 +0000 (07:17 -0500)]
Revert "docs: temporarily disable the kernel-doc extension"

This reverts commit fd68a72875cf318f4310726f842139119c5f45d5.  We're
done with the update of kernel-doc and we can restore kernel-doc's
functionality.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoscripts: kernel-doc: use :c:union when needed
Mauro Carvalho Chehab [Tue, 17 Nov 2020 16:53:12 +0000 (17:53 +0100)]
scripts: kernel-doc: use :c:union when needed

Sphinx C domain code after 3.2.1 will start complaning if :c:struct
would be used for an union type:

.../Documentation/gpu/drm-kms-helpers:352: ../drivers/video/hdmi.c:851: WARNING: C 'identifier' cross-reference uses wrong tag: reference name is 'union hdmi_infoframe' but found name is 'struct hdmi_infoframe'. Full reference name is 'union hdmi_infoframe'. Full found name is 'struct hdmi_infoframe'.

So, let's address this issue too in advance, in order to
avoid future issues.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/6e4ec3eec914df62389a299797a3880ae4490f35.1603791716.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-30-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoscripts: kernel-doc: split typedef complex regex
Mauro Carvalho Chehab [Tue, 17 Nov 2020 16:53:11 +0000 (17:53 +0100)]
scripts: kernel-doc: split typedef complex regex

The typedef regex for function prototypes are very complex.
Split them into 3 separate regex and then join them using
qr.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/3a4af999a0d62d4ab9dfae1cdefdfcad93383356.1603792384.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-29-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoscripts: kernel-doc: fix typedef parsing
Mauro Carvalho Chehab [Tue, 17 Nov 2020 16:53:10 +0000 (17:53 +0100)]
scripts: kernel-doc: fix typedef parsing

The include/linux/genalloc.h file defined this typedef:

typedef unsigned long (*genpool_algo_t)(unsigned long *map,unsigned long size,unsigned long start,unsigned int nr,void *data, struct gen_pool *pool, unsigned long start_addr);

Because it has a type composite of two words (unsigned long),
the parser gets the typedef name wrong:

.. c:macro:: long

   **Typedef**: Allocation callback function type definition

Fix the regex in order to accept composite types when
defining a typedef for a function pointer.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/328e8018041cc44f7a1684e57f8d111230761c4f.1603792384.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-28-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoRevert "kernel-doc: Handle function typedefs that return pointers"
Paolo Bonzini [Tue, 17 Nov 2020 16:53:09 +0000 (17:53 +0100)]
Revert "kernel-doc: Handle function typedefs that return pointers"

This reverts commit 19ab6044be0c55d529e875e3ee16fdd5c3b54d33.
We will replace the commit with the fix from Linux.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-27-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoRevert "kernel-doc: Handle function typedefs without asterisks"
Paolo Bonzini [Tue, 17 Nov 2020 16:53:08 +0000 (17:53 +0100)]
Revert "kernel-doc: Handle function typedefs without asterisks"

This reverts commit 3cd3c5193cde5242e243c25759f85802e267994f.
We will replace the commit with the fix from Linux.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-26-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoscripts: kernel-doc: try to use c:function if possible
Mauro Carvalho Chehab [Tue, 17 Nov 2020 16:53:07 +0000 (17:53 +0100)]
scripts: kernel-doc: try to use c:function if possible

There are a few namespace clashes by using c:macro everywhere:

basically, when using it, we can't have something like:

.. c:struct:: pwm_capture

.. c:macro:: pwm_capture

So, we need to use, instead:

.. c:function:: int pwm_capture (struct pwm_device * pwm, struct pwm_capture * result, unsigned long timeout)

for the function declaration.

The kernel-doc change was proposed by Jakob Lykke Andersen here:

https://github.com/jakobandersen/linux_docs/commit/6fd2076ec001cca7466857493cd678df4dfe4a65

Although I did a different implementation.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-25-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoscripts: kernel-doc: fix line number handling
Mauro Carvalho Chehab [Tue, 17 Nov 2020 16:53:06 +0000 (17:53 +0100)]
scripts: kernel-doc: fix line number handling

Address several issues related to pointing to the wrong line
number:

1) ensure that line numbers will always be initialized

   When section is the default (Description), the line number
   is not initializing, producing this:

$ ./scripts/kernel-doc --enable-lineno ./drivers/media/v4l2-core/v4l2-mem2mem.c|less

**Description**

#define LINENO 0
In case of streamoff or release called on any context,
1] If the context is currently running, then abort job will be called
2] If the context is queued, then the context will be removed from
   the job_queue

  Which is not right. Ensure that the line number will always
  be there. After applied, the result now points to the right location:

**Description**

#define LINENO 410
In case of streamoff or release called on any context,
1] If the context is currently running, then abort job will be called
2] If the context is queued, then the context will be removed from
   the job_queue

2) The line numbers for function prototypes are always + 1,
   because it is taken at the line after handling the prototype.
   Change the logic to point to the next line after the /** */
   block;

3) The "DOC:" line number should point to the same line as this
   markup is found, and not to the next one.

Probably part of the issues were due to a but that was causing
the line number offset to be incremented by one, if --export
were used.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-24-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoscripts: kernel-doc: allow passing desired Sphinx C domain dialect
Mauro Carvalho Chehab [Tue, 17 Nov 2020 16:53:05 +0000 (17:53 +0100)]
scripts: kernel-doc: allow passing desired Sphinx C domain dialect

When kernel-doc is called via kerneldoc.py, there's no need to
auto-detect the Sphinx version, as the Sphinx module already
knows it. So, add an optional parameter to allow changing the
Sphinx dialect.

As kernel-doc can also be manually called, keep the auto-detection
logic if the parameter was not specified. On such case, emit
a warning if sphinx-build can't be found at PATH.

I ended using a suggestion from Joe for using a more readable
regex, instead of using a complex one with a hidden group like:

m/^(\d+)\.(\d+)(?:\.?(\d+)?)/

in order to get the optional <patch> argument.

Thanks-to: Joe Perches <joe@perches.com>
Suggested-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-23-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoscripts: kernel-doc: don't mangle with parameter list
Mauro Carvalho Chehab [Tue, 17 Nov 2020 16:53:04 +0000 (17:53 +0100)]
scripts: kernel-doc: don't mangle with parameter list

While kernel-doc needs to parse parameters in order to
identify its name, it shouldn't be touching the type,
as parsing it is very difficult, and errors happen.

One current error is when parsing this parameter:

const u32 (*tab)[256]

Found at ./lib/crc32.c, on this function:

u32 __pure crc32_be_generic (u32 crc, unsigned char const *p, size_t len, const u32 (*tab)[256], u32 polynomial);

The current logic mangles it, producing this output:

const u32 ( *tab

That's something that it is not recognizeable.

So, instead, let's push the argument as-is, and use it
when printing the function prototype and when describing
each argument.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-22-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoscripts: kernel-doc: fix typedef identification
Mauro Carvalho Chehab [Tue, 17 Nov 2020 16:53:03 +0000 (17:53 +0100)]
scripts: kernel-doc: fix typedef identification

Some typedef expressions are output as normal functions.

As we need to be clearer about the type with Sphinx 3.x,
detect such cases.

While here, fix a wrongly-indented block.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-21-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoscripts: kernel-doc: reimplement -nofunction argument
Mauro Carvalho Chehab [Tue, 17 Nov 2020 16:53:02 +0000 (17:53 +0100)]
scripts: kernel-doc: reimplement -nofunction argument

Right now, the build system doesn't use -nofunction, as
it is pretty much useless, because it doesn't consider
the other output modes (extern, internal), working only
with all.

Also, it is limited to exclude functions.

Re-implement it in order to allow excluding any symbols from
the document output, no matter what mode is used.

The parameter was also renamed to "-nosymbol", as it express
better its meaning.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-20-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoscripts: kernel-doc: fix troubles with line counts
Mauro Carvalho Chehab [Tue, 17 Nov 2020 16:53:01 +0000 (17:53 +0100)]
scripts: kernel-doc: fix troubles with line counts

There's currently a bug with the way kernel-doc script
counts line numbers that can be seen with:

$ ./scripts/kernel-doc -rst  -enable-lineno include/linux/math64.h >all && ./scripts/kernel-doc -rst -internal -enable-lineno include/linux/math64.h >int && diff -U0 int all

--- int 2020-09-28 12:58:08.927486808 +0200
+++ all 2020-09-28 12:58:08.905486845 +0200
@@ -1 +1 @@
-#define LINENO 27
+#define LINENO 26
@@ -3 +3 @@
-#define LINENO 16
+#define LINENO 15
@@ -9 +9 @@
-#define LINENO 17
+#define LINENO 16
...

This is happening with perl version 5.30.3, but I'm not
so sure if this is a perl bug, or if this is due to something
else.

In any case, fixing it is easy. Basically, when "-internal"
parameter is used, the process_export_file() function opens the
handle "IN". This makes the line number to be incremented, as the
handler for the main open is also "IN".

Fix the problem by using a different handler for the
main open().

While here, add a missing close for it.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-19-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoscripts: kernel-doc: use a less pedantic markup for funcs on Sphinx 3.x
Mauro Carvalho Chehab [Tue, 17 Nov 2020 16:53:00 +0000 (17:53 +0100)]
scripts: kernel-doc: use a less pedantic markup for funcs on Sphinx 3.x

Unfortunately, Sphinx 3.x parser for c functions is too pedantic:

https://github.com/sphinx-doc/sphinx/issues/8241

While it could be relaxed with some configurations, there are
several corner cases that it would make it hard to maintain,
and will require teaching conf.py about several macros.

So, let's instead use the :c:macro notation. This will
produce an output that it is not as nice as currently, but it
should still be acceptable, and will provide cross-references,
removing thousands of warnings when building with newer
versions of Sphinx.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-18-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoscripts: kernel-doc: make it more compatible with Sphinx 3.x
Mauro Carvalho Chehab [Tue, 17 Nov 2020 16:52:59 +0000 (17:52 +0100)]
scripts: kernel-doc: make it more compatible with Sphinx 3.x

With Sphinx 3.x, the ".. c:type:" tag was changed to accept either:

.. c:type:: typedef-like declaration
.. c:type:: name

Using it for other types (including functions) don't work anymore.

So, there are newer tags for macro, enum, struct, union, and others,
which doesn't exist on older versions.

Add a check for the Sphinx version and change the produced tags
accordingly.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-17-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoRevert "kernel-doc: Use c:struct for Sphinx 3.0 and later"
Paolo Bonzini [Tue, 17 Nov 2020 16:52:58 +0000 (17:52 +0100)]
Revert "kernel-doc: Use c:struct for Sphinx 3.0 and later"

This reverts commit 152d1967f650f67b7ece3a5dda77d48069d72647.
We will replace the commit with the fix from Linux.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-16-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoRevert "scripts/kerneldoc: For Sphinx 3 use c:macro for macros with arguments"
Paolo Bonzini [Tue, 17 Nov 2020 16:52:57 +0000 (17:52 +0100)]
Revert "scripts/kerneldoc: For Sphinx 3 use c:macro for macros with arguments"

This reverts commit 92bb29f9b2c3d4a98eef5f0db935d4be291eec72.
We will replace the commit with the fix from Linux.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-15-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoscripts: kernel-doc: add support for typedef enum
Mauro Carvalho Chehab [Tue, 17 Nov 2020 16:52:56 +0000 (17:52 +0100)]
scripts: kernel-doc: add support for typedef enum

The PHY kernel-doc markup has gained support for documenting
a typedef enum.

However, right now the parser was not prepared for it.

So, add support for parsing it.

Fixes: 4069a572d423 ("net: phy: Document core PHY structures")
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-14-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agokernel-doc: add support for ____cacheline_aligned attribute
Jonathan Cameron [Tue, 17 Nov 2020 16:52:55 +0000 (17:52 +0100)]
kernel-doc: add support for ____cacheline_aligned attribute

Subroutine dump_struct uses type attributes to check if the struct
syntax is valid. Then, it removes all attributes before using it for
output. `____cacheline_aligned` is an attribute that is
not included in both steps. Add it, since it is used by kernel structs.

Based on previous patch to add ____cacheline_aligned_in_smp.
Motivated by patches to reorder this attribute to before the
variable name.   Whilst we could do that in all cases, that would
be a massive change and it is more common in the kernel to place
this particular attribute after the variable name. A quick grep
suggests approximately 400 instances of which 341 have this
attribute just before a semicolon and hence after the variable name.

Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Cc: Lee Jones <lee.jones@linaro.org>
Link: https://lore.kernel.org/r/20200910185415.653139-1-jic23@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-13-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agokernel-doc: include line numbers for function prototypes
Mauro Carvalho Chehab [Tue, 17 Nov 2020 16:52:54 +0000 (17:52 +0100)]
kernel-doc: include line numbers for function prototypes

This should solve bad error reports like this one:

./include/linux/iio/iio.h:0: WARNING: Unknown target name: "devm".

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/56eed0ba50cd726236acd12b11b55ce54854c5ea.1599660067.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-12-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoscripts/kernel-doc: optionally treat warnings as errors
Pierre-Louis Bossart [Tue, 17 Nov 2020 16:52:53 +0000 (17:52 +0100)]
scripts/kernel-doc: optionally treat warnings as errors

The kbuild bot recently added the W=1 option, which triggered
documentation cleanups to squelch hundreds of kernel-doc warnings.

To make sure new kernel contributions don't add regressions to
kernel-doc descriptors, this patch suggests an option to treat
warnings as errors in CI/automated tests.

A -Werror command-line option is added to the kernel-doc script. When
this option is set, the script will return the number of warnings
found. The caller can then treat this positive return value as an
error and stop the build.

Using this command line option is however not straightforward when the
kernel-doc script is called from other scripts. To align with typical
kernel compilation or documentation generation, the Werror option is
also set by checking the KCFLAGS environment variable, or if
KDOC_WERROR is defined, as in the following examples:

KCFLAGS="-Wall -Werror" make W=1 sound/
KCFLAGS="-Wall -Werror" make W=1 drivers/soundwire/
KDOC_WERROR=1 make htmldocs

Note that in the last example the documentation build does not stop,
only an additional log is provided.

Credits to Randy Dunlap for suggesting the use of environment variables.

Suggested-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
Link: https://lore.kernel.org/r/20200728162040.92467-1-pierre-louis.bossart@linux.intel.com
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-11-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoscripts/kernel-doc: handle function pointer prototypes
Mauro Carvalho Chehab [Tue, 17 Nov 2020 16:52:52 +0000 (17:52 +0100)]
scripts/kernel-doc: handle function pointer prototypes

There are some function pointer prototypes inside the net
includes, like this one:

int (*pcs_config)(struct phylink_config *config, unsigned int mode,
  phy_interface_t interface, const unsigned long *advertising);

There's nothing wrong using it with kernel-doc, but we need to
add a rule for it to parse such kind of prototype.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/fec520dd731a273013ae06b7653a19c7d15b9562.1592895969.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-10-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoscripts/kernel-doc: parse __ETHTOOL_DECLARE_LINK_MODE_MASK
Mauro Carvalho Chehab [Tue, 17 Nov 2020 16:52:51 +0000 (17:52 +0100)]
scripts/kernel-doc: parse __ETHTOOL_DECLARE_LINK_MODE_MASK

The __ETHTOOL_DECLARE_LINK_MODE_MASK macro is a variant of
DECLARE_BITMAP(), used by phylink.h. As we have already a
parser for DECLARE_BITMAP(), let's add one for this macro,
in order to avoid such warnings:

./include/linux/phylink.h:54: warning: Function parameter or member '__ETHTOOL_DECLARE_LINK_MODE_MASK(advertising' not described in 'phylink_link_state'
./include/linux/phylink.h:54: warning: Function parameter or member '__ETHTOOL_DECLARE_LINK_MODE_MASK(lp_advertising' not described in 'phylink_link_state'

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/d1d1dea67a28117c0b0c33271b139c4455fef287.1592895969.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-9-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoReplace HTTP links with HTTPS ones: documentation
Alexander A. Klimov [Tue, 17 Nov 2020 16:52:50 +0000 (17:52 +0100)]
Replace HTTP links with HTTPS ones: documentation

Rationale:
Reduces attack surface on kernel devs opening the links for MITM
as HTTPS traffic is much harder to manipulate.

Deterministic algorithm:
For each file:
  For each line:
    If doesn't contain `\bxmlns\b`:
      For each link, `\bhttp://[^# \t\r\n]*(?:\w|/)`:
        If both the HTTP and HTTPS versions
        return 200 OK and serve the same content:
          Replace HTTP with HTTPS.

Signed-off-by: Alexander A. Klimov <grandmaster@al2klimov.de>
Link: https://lore.kernel.org/r/20200526060544.25127-1-grandmaster@al2klimov.de
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-8-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoscripts: kernel-doc: accept blank lines on parameter description
Mauro Carvalho Chehab [Tue, 17 Nov 2020 16:52:49 +0000 (17:52 +0100)]
scripts: kernel-doc: accept blank lines on parameter description

Sphinx is very pedantic with respect to blank lines. Sometimes,
in order to make it to properly handle something, we need to
add a blank line. However, currently, any blank line inside a
kernel-doc comment like:

/*
 * @foo: bar
         *
 *       foobar
 *
 * some description

will be considered as if "foobar" was part of the description.

This patch changes kernel-doc behavior. After it, foobar will
be considered as part of the parameter text. The description
will only be considered as such if it starts with:

zero spaces after asterisk:

*foo

one space after asterisk:
* foo

or have a explicit Description section:

*   Description:

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/c07d2862792d75a2691d69c9eceb7b89a0164cc0.1586881715.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-7-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoscripts: kernel-doc: accept negation like !@var
Mauro Carvalho Chehab [Tue, 17 Nov 2020 16:52:48 +0000 (17:52 +0100)]
scripts: kernel-doc: accept negation like !@var

On a few places, it sometimes need to indicate a negation of a
parameter, like:

!@fshared

This pattern happens, for example, at:

kernel/futex.c

and it is perfectly valid. However, kernel-doc currently
transforms it into:

!**fshared**

This won't do what it would be expected.

Fortunately, fixing the script is a simple matter of storing
the "!" before "@" and adding it after the bold markup, like:

**!fshared**

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/0314b47f8c3e1f9db00d5375a73dc3cddd8a21f2.1586881715.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-6-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoscripts: kernel-doc: proper handle @foo->bar()
Mauro Carvalho Chehab [Tue, 17 Nov 2020 16:52:47 +0000 (17:52 +0100)]
scripts: kernel-doc: proper handle @foo->bar()

The pattern @foo->bar() is valid, as it can be used by a
function pointer inside a struct passed as a parameter.

Right now, it causes a warning:

./drivers/firewire/core-transaction.c:606: WARNING: Inline strong start-string without end-string.

In this specific case, the kernel-doc markup is:

/**
 * fw_core_remove_address_handler() - unregister an address handler
 * @handler: callback
 *
 * To be called in process context.
 *
 * When fw_core_remove_address_handler() returns, @handler->callback() is
 * guaranteed to not run on any CPU anymore.
 */

With seems valid on my eyes. So, instead of trying to hack
the kernel-doc markup, let's teach it about how to handle
such things. This should likely remove lots of other similar
warnings as well.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/48b46426d7bf6ff7529f20e5718fbf4e9758e62c.1586881715.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-5-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoscripts/kernel-doc: Add support for named variable macro arguments
Jonathan Neuschäfer [Tue, 17 Nov 2020 16:52:46 +0000 (17:52 +0100)]
scripts/kernel-doc: Add support for named variable macro arguments

Currently, when kernel-doc encounters a macro with a named variable
argument[1], such as this:

   #define hlist_for_each_entry_rcu(pos, head, member, cond...)

... it expects the variable argument to be documented as `cond...`,
rather than `cond`. This is semantically wrong, because the name (as
used in the macro body) is actually `cond`.

With this patch, kernel-doc will accept the name without dots (`cond`
in the example above) in doc comments, and warn if the name with dots
(`cond...`) is used and verbose mode[2] is enabled.

The support for the `cond...` syntax can be removed later, when the
documentation of all such macros has been switched to the new syntax.

Testing this patch on top of v5.4-rc6, `make htmldocs` shows a few
changes in log output and HTML output:

 1) The following warnings[3] are eliminated:

   ./include/linux/rculist.h:374: warning:
        Excess function parameter 'cond' description in 'list_for_each_entry_rcu'
   ./include/linux/rculist.h:651: warning:
        Excess function parameter 'cond' description in 'hlist_for_each_entry_rcu'

 2) For list_for_each_entry_rcu and hlist_for_each_entry_rcu, the
    correct description is shown

 3) Named variable arguments are shown without dots

[1]: https://gcc.gnu.org/onlinedocs/cpp/Variadic-Macros.html
[2]: scripts/kernel-doc -v
[3]: See also https://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git/commit/?h=dev&id=5bc4bc0d6153617eabde275285b7b5a8137fdf3c

Signed-off-by: Jonathan Neuschäfer <j.neuschaefer@gmx.net>
Tested-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-4-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agokernel-doc: add support for ____cacheline_aligned_in_smp attribute
André Almeida [Tue, 17 Nov 2020 16:52:45 +0000 (17:52 +0100)]
kernel-doc: add support for ____cacheline_aligned_in_smp attribute

Subroutine dump_struct uses type attributes to check if the struct
syntax is valid. Then, it removes all attributes before using it for
output. `____cacheline_aligned_in_smp` is an attribute that is
not included in both steps. Add it, since it is used by kernel structs.

Signed-off-by: André Almeida <andrealmeid@collabora.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-3-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agokernel-doc: fix processing nested structs with attributes
André Almeida [Tue, 17 Nov 2020 16:52:44 +0000 (17:52 +0100)]
kernel-doc: fix processing nested structs with attributes

The current regular expression for strip attributes of structs (and
for nested ones as well) also removes all whitespaces that may
surround the attribute. After that, the code will split structs and
iterate for each symbol separated by comma at the end of struct
definition (e.g. "} alias1, alias2;"). However, if the nested struct
does not have any alias and has an attribute, it will result in a
empty string at the closing bracket (e.g "};"). This will make the
split return nothing and $newmember will keep uninitialized. Fix
that, by ensuring that the attribute substitution will leave at least
one whitespace.

Signed-off-by: André Almeida <andrealmeid@collabora.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-2-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agodocs: temporarily disable the kernel-doc extension
Paolo Bonzini [Mon, 30 Nov 2020 12:16:11 +0000 (07:16 -0500)]
docs: temporarily disable the kernel-doc extension

Preserve bisectability while we update scripts/kernel-doc from Linux.
Without this patch, building with Sphinx 3 would break while we
revert our own Sphinx 3 support and replace it with Linux's.

Suggested-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoconfig-file: move -set implementation to vl.c
Paolo Bonzini [Thu, 12 Nov 2020 13:16:22 +0000 (08:16 -0500)]
config-file: move -set implementation to vl.c

We want to make it independent of QemuOpts.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agovl: clean up -boot variables
Paolo Bonzini [Fri, 13 Nov 2020 07:41:38 +0000 (02:41 -0500)]
vl: clean up -boot variables

Move more of them into MachineState, in preparation for moving initialization
of the machine out of vl.c.

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agovl: remove serial_max_hds
Paolo Bonzini [Tue, 3 Nov 2020 08:26:05 +0000 (03:26 -0500)]
vl: remove serial_max_hds

serial_hd(i) is NULL if and only if i >= serial_max_hds().  Test
serial_hd(i) instead of bounding the loop at serial_max_hds(),
thus removing one more function that vl.c is expected to export.

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agovl: extract softmmu/rtc.c
Paolo Bonzini [Wed, 28 Oct 2020 11:36:57 +0000 (07:36 -0400)]
vl: extract softmmu/rtc.c

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agovl: extract machine done notifiers
Paolo Bonzini [Wed, 28 Oct 2020 11:36:57 +0000 (07:36 -0400)]
vl: extract machine done notifiers

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agovl: extract softmmu/datadir.c
Paolo Bonzini [Wed, 28 Oct 2020 11:36:57 +0000 (07:36 -0400)]
vl: extract softmmu/datadir.c

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agohmp: introduce cmd_available
Paolo Bonzini [Tue, 27 Oct 2020 08:40:02 +0000 (04:40 -0400)]
hmp: introduce cmd_available

Combine the RUN_STATE_PRECONFIG and cmd_can_preconfig checks
into a single function, to avoid repeating the same expression
(or its negation after applying DeMorgan's rule) over and
over again.

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agovl: start VM via qmp_cont
Paolo Bonzini [Tue, 27 Oct 2020 12:42:04 +0000 (08:42 -0400)]
vl: start VM via qmp_cont

Complement the previous patch by starting the VM with a QMP command.
The plan is that the user will be able to do the same, invoking two
commands "finish-machine-init" and "cont" instead of
"x-exit-preconfig".

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agomigration, vl: start migration via qmp_migrate_incoming
Paolo Bonzini [Tue, 27 Oct 2020 08:22:57 +0000 (04:22 -0400)]
migration, vl: start migration via qmp_migrate_incoming

Make qemu_start_incoming_migration local to migration/migration.c.
By using the runstate instead of a separate flag, vl need not do
anything to setup deferred incoming migration.

qmp_migrate_incoming also does not need the deferred_incoming flag
anymore, because "-incoming PROTOCOL" will clear the "once" flag
before the main loop starts.  Therefore, later invocations of
the migrate-incoming command will fail with the existing
"The incoming migration has already been started" error message.

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agovl: move -global check earlier
Paolo Bonzini [Tue, 27 Oct 2020 08:26:14 +0000 (04:26 -0400)]
vl: move -global check earlier

The check has the same effect here, it only matters that it is performed
once all devices, both builtin and user-specified, have been created.

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agovl: initialize displays before preconfig loop
Paolo Bonzini [Tue, 27 Oct 2020 08:07:30 +0000 (04:07 -0400)]
vl: initialize displays before preconfig loop

Displays should be available before the monitor starts, so that
it is possible to use the graphical console to interact with
the monitor itself.

This patch is quite ugly, but all this is temporary.  The double
call to qemu_init_displays will go away as soon we can unify machine
initialization between the preconfig and "normal" flows, and so will
the preconfig_exit_requested variable (that is only preconfig_requested
remains).

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agovl: separate qemu_resolve_machine_memdev
Paolo Bonzini [Tue, 27 Oct 2020 15:16:18 +0000 (11:16 -0400)]
vl: separate qemu_resolve_machine_memdev

This is a bit nasty: the machine is storing a string and later
resolving it.  We probably want to remove the memdev property
and instead make this a memory-set command.  "-M memdev" can be
handled a legacy option that is special cased by
machine_set_property.

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agovl: separate qemu_apply_machine_options
Paolo Bonzini [Tue, 27 Oct 2020 15:08:04 +0000 (11:08 -0400)]
vl: separate qemu_apply_machine_options

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agovl: separate qemu_create_machine
Paolo Bonzini [Tue, 27 Oct 2020 15:04:37 +0000 (11:04 -0400)]
vl: separate qemu_create_machine

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agovl: separate qemu_create_late_backends
Paolo Bonzini [Tue, 27 Oct 2020 13:36:52 +0000 (09:36 -0400)]
vl: separate qemu_create_late_backends

"Late" backends are created after the machine.

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agovl: separate qemu_create_early_backends
Paolo Bonzini [Wed, 21 Oct 2020 14:41:18 +0000 (10:41 -0400)]
vl: separate qemu_create_early_backends

"Early" backends are created before the machine and can be used as
machine options.

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agovl: move CHECKPOINT_INIT after preconfig
Paolo Bonzini [Wed, 21 Oct 2020 11:19:34 +0000 (07:19 -0400)]
vl: move CHECKPOINT_INIT after preconfig

Move CHECKPOINT_INIT right before the machine initialization is
completed.  Everything before is essentially an extension of
command line parsing.

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agovl: extract default devices to separate functions
Paolo Bonzini [Wed, 21 Oct 2020 10:22:39 +0000 (06:22 -0400)]
vl: extract default devices to separate functions

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agovl: load plugins as late as possible
Paolo Bonzini [Wed, 21 Oct 2020 14:25:06 +0000 (10:25 -0400)]
vl: load plugins as late as possible

There is no need to load plugins in the middle of default device processing,
move -plugin handling just before preconfig is entered.

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agovl: create "-net nic -net user" default earlier
Paolo Bonzini [Wed, 21 Oct 2020 10:01:30 +0000 (06:01 -0400)]
vl: create "-net nic -net user" default earlier

Create it together with other default backends, even though the processing is
done later.

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoqemu-option: restrict qemu_opts_set to merge-lists QemuOpts
Paolo Bonzini [Mon, 9 Nov 2020 09:46:30 +0000 (04:46 -0500)]
qemu-option: restrict qemu_opts_set to merge-lists QemuOpts

qemu_opts_set is used to create default network backends and to
parse sugar options -kernel, -initrd, -append, -bios and -dtb.
These are very different uses:

I would *expect* a function named qemu_opts_set to set an option in a
merge-lists QemuOptsList, such as -kernel, and possibly to set an option
in a non-merge-lists QemuOptsList with non-NULL id, similar to -set.

However, it wouldn't *work* to use qemu_opts_set for the latter
because qemu_opts_set uses fail_if_exists==1. So, for non-merge-lists
QemuOptsList and non-NULL id, the semantics of qemu_opts_set (fail if the
(QemuOptsList, id) pair already exists) are debatable.

On the other hand, I would not expect qemu_opts_set to create a
non-merge-lists QemuOpts with a single option; which it does, though.
For this case of non-merge-lists QemuOptsList and NULL id, qemu_opts_set
hardly adds value over qemu_opts_parse.  It does skip some parsing and
unescaping, but that's not needed when creating default network
backends.

So qemu_opts_set has warty behavior for non-merge-lists QemuOptsList
if id is non-NULL, and it's mostly pointless if id is NULL.  My
solution to keeping the API as simple as possible is to limit
qemu_opts_set to merge-lists QemuOptsList.  For them, it's useful (we
don't want comma-unescaping for -kernel) *and* has sane semantics.
Network backend creation is switched to qemu_opts_parse.

qemu_opts_set is now only used on merge-lists QemuOptsList... except
in the testcase, which is changed to use a merge-list QemuOptsList.

With this change we can also remove the id parameter.  With the
parameter always NULL, we know that qemu_opts_create cannot fail
and can pass &error_abort to it.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agovl: extract various command line desugaring snippets to a new function
Paolo Bonzini [Wed, 21 Oct 2020 14:21:22 +0000 (10:21 -0400)]
vl: extract various command line desugaring snippets to a new function

Keep the machine initialization sequence free of miscellaneous command
line parsing actions.

The only difference is that preallocation will always be done with one
thread if -smp is not provided; previously it was using mc->default_cpus,
which is almost always 1 anyway.

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agovl: preconfig and loadvm are mutually exclusive
Paolo Bonzini [Wed, 21 Oct 2020 14:19:08 +0000 (10:19 -0400)]
vl: preconfig and loadvm are mutually exclusive

Just like -incoming.  Later we will add support for "-incoming defer
-preconfig", because there are cases (Xen, block layer) that want
to look at RUNSTATE_INMIGRATE.  -loadvm will remain mutually exclusive
with preconfig; the plan is to just do loadvm in the monitor, since
the user is already going to interact with it for preconfiguration.

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agovl: extract various command line validation snippets to a new function
Paolo Bonzini [Tue, 3 Nov 2020 08:45:26 +0000 (03:45 -0500)]
vl: extract various command line validation snippets to a new function

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agovl: move prelaunch part of qemu_init to new functions
Paolo Bonzini [Fri, 23 Oct 2020 12:04:29 +0000 (08:04 -0400)]
vl: move prelaunch part of qemu_init to new functions

The final part of qemu_init, starting with the completion of
board init, is already relatively clean.  Split it out of
qemu_init so that qemu_init keeps only the messy parts.

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agovl: extract qemu_init_subsystems
Paolo Bonzini [Wed, 21 Oct 2020 09:33:40 +0000 (05:33 -0400)]
vl: extract qemu_init_subsystems

Group a bunch of subsystem initializations that can be done right
after command line parsing.  Remove initializations that can be done
simply as global variable initializers.

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agovl: move various initialization routines out of qemu_init
Paolo Bonzini [Wed, 21 Oct 2020 09:25:58 +0000 (05:25 -0400)]
vl: move various initialization routines out of qemu_init

Some very simple initialization routines can be nested in existing
subsystem-level functions, do that to simplify qemu_init.

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agovl: split various early command line options to a separate function
Paolo Bonzini [Wed, 21 Oct 2020 09:17:39 +0000 (05:17 -0400)]
vl: split various early command line options to a separate function

Various options affect the global state of QEMU including the rest of
qemu_init, and they need to be called very early.  Group them together
in a function that is called at the beginning.

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agovl: remove bogus check
Paolo Bonzini [Wed, 21 Oct 2020 10:07:55 +0000 (06:07 -0400)]
vl: remove bogus check

There is no reason to prevent -preconfig -daemonize.  Of course if
no monitor is defined there will be no way to start the VM,
but that is a user error.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agovl: extract validation of -smp to machine.c
Paolo Bonzini [Wed, 21 Oct 2020 10:49:31 +0000 (06:49 -0400)]
vl: extract validation of -smp to machine.c

Once smp_parse is done, the validation operates on the MachineState.
There is no reason for that code to be in vl.c.

Reviewed-by: Igor Mammedov <imammedo@redhat.com>
Tested-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agotreewide: do not use short-form boolean options
Paolo Bonzini [Fri, 13 Nov 2020 08:10:52 +0000 (03:10 -0500)]
treewide: do not use short-form boolean options

They are going to be deprecated, avoid warnings on stdout while the
tests run.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agohw/char/serial: Clean up unnecessary code
Philippe Mathieu-Daudé [Fri, 20 Nov 2020 16:19:33 +0000 (17:19 +0100)]
hw/char/serial: Clean up unnecessary code

Since commit 5ec3a23e6c8 ("serial: convert PIO to new memory
api read/write") we don't need to worry about accesses bigger
than 8-bit. Use the extract()/deposit() functions to access
the correct part of the 16-bit 'divider' register.

Reported-by: Jonathan D. Belanger <jbelanger1@rochester.rr.com>
Buglink: https://bugs.launchpad.net/qemu/+bug/1904331
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20201120161933.2514089-1-f4bug@amsat.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agomake ram_size local to vl.c
Paolo Bonzini [Wed, 28 Oct 2020 10:24:22 +0000 (06:24 -0400)]
make ram_size local to vl.c

Use the machine properties for the leftovers too.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agosparc64: do not use ram_size global
Paolo Bonzini [Wed, 28 Oct 2020 10:14:33 +0000 (06:14 -0400)]
sparc64: do not use ram_size global

Use the machine properties instead.

Cc: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agos390x: do not use ram_size global
Paolo Bonzini [Wed, 28 Oct 2020 10:17:07 +0000 (06:17 -0400)]
s390x: do not use ram_size global

Use the machine properties instead.

Cc: Cornelia Huck <cohuck@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoriscv: do not use ram_size global
Paolo Bonzini [Wed, 28 Oct 2020 10:16:22 +0000 (06:16 -0400)]
riscv: do not use ram_size global

Use the machine properties instead.

Cc: Alistair Francis <Alistair.Francis@wdc.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoppc: do not use ram_size global
Paolo Bonzini [Wed, 28 Oct 2020 10:17:31 +0000 (06:17 -0400)]
ppc: do not use ram_size global

Use the machine properties instead.

Cc: qemu-ppc@nongnu.org
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agonios2: do not use ram_size global
Paolo Bonzini [Wed, 28 Oct 2020 10:25:06 +0000 (06:25 -0400)]
nios2: do not use ram_size global

Use the equivalent argument to the function instead.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agomoxie: do not use ram_size global
Paolo Bonzini [Wed, 28 Oct 2020 10:21:08 +0000 (06:21 -0400)]
moxie: do not use ram_size global

Use the loader parameters instead.

Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agomips: do not use ram_size global
Paolo Bonzini [Wed, 28 Oct 2020 10:20:52 +0000 (06:20 -0400)]
mips: do not use ram_size global

Use the machine properties or loader parameters instead.

Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agomicroblaze: do not use ram_size global
Paolo Bonzini [Wed, 28 Oct 2020 10:20:01 +0000 (06:20 -0400)]
microblaze: do not use ram_size global

Use the equivalent argument to the function instead.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agom68k: do not use ram_size global
Paolo Bonzini [Wed, 28 Oct 2020 10:15:40 +0000 (06:15 -0400)]
m68k: do not use ram_size global

Use the machine properties instead.

Cc: Laurent Vivier <lvivier@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoi386: do not use ram_size global
Paolo Bonzini [Wed, 28 Oct 2020 10:22:23 +0000 (06:22 -0400)]
i386: do not use ram_size global

Use the loader parameters instead.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agohppa: do not use ram_size global
Paolo Bonzini [Wed, 28 Oct 2020 10:19:23 +0000 (06:19 -0400)]
hppa: do not use ram_size global

Use the machine properties instead.

Cc: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agocris: do not use ram_size global
Paolo Bonzini [Wed, 28 Oct 2020 10:19:42 +0000 (06:19 -0400)]
cris: do not use ram_size global

Use the machine properties instead.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
3 years agoarm: do not use ram_size global
Paolo Bonzini [Wed, 28 Oct 2020 10:18:20 +0000 (06:18 -0400)]
arm: do not use ram_size global

Use the machine properties instead.

Cc: qemu-ppc@nongnu.org
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>