Daniel Jordan [Fri, 6 Sep 2019 01:40:26 +0000 (21:40 -0400)]
padata, pcrypt: take CPU hotplug lock internally in padata_alloc_possible
With pcrypt's cpumask no longer used, take the CPU hotplug lock inside
padata_alloc_possible.
Useful later in the series for avoiding nested acquisition of the CPU
hotplug lock in padata when padata_alloc_possible is allocating an
unbound workqueue.
Without this patch, this nested acquisition would happen later in the
series:
pcrypt_init_padata
get_online_cpus
alloc_padata_possible
alloc_padata
alloc_workqueue(WQ_UNBOUND) // later in the series
alloc_and_link_pwqs
apply_wqattrs_lock
get_online_cpus // recursive rwsem acquisition
Signed-off-by: Daniel Jordan <daniel.m.jordan@oracle.com> Acked-by: Steffen Klassert <steffen.klassert@secunet.com> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: Lai Jiangshan <jiangshanlai@gmail.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Tejun Heo <tj@kernel.org> Cc: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Daniel Jordan [Fri, 6 Sep 2019 01:40:25 +0000 (21:40 -0400)]
crypto: pcrypt - remove padata cpumask notifier
Now that padata_do_parallel takes care of finding an alternate callback
CPU, there's no need for pcrypt's callback cpumask, so remove it and the
notifier callback that keeps it in sync.
Signed-off-by: Daniel Jordan <daniel.m.jordan@oracle.com> Acked-by: Steffen Klassert <steffen.klassert@secunet.com> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: Lai Jiangshan <jiangshanlai@gmail.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Tejun Heo <tj@kernel.org> Cc: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Daniel Jordan [Fri, 6 Sep 2019 01:40:24 +0000 (21:40 -0400)]
padata: make padata_do_parallel find alternate callback CPU
padata_do_parallel currently returns -EINVAL if the callback CPU isn't
in the callback cpumask.
pcrypt tries to prevent this situation by keeping its own callback
cpumask in sync with padata's and checks that the callback CPU it passes
to padata is valid. Make padata handle this instead.
padata_do_parallel now takes a pointer to the callback CPU and updates
it for the caller if an alternate CPU is used. Overall behavior in
terms of which callback CPUs are chosen stays the same.
Prepares for removal of the padata cpumask notifier in pcrypt, which
will fix a lockdep complaint about nested acquisition of the CPU hotplug
lock later in the series.
Signed-off-by: Daniel Jordan <daniel.m.jordan@oracle.com> Acked-by: Steffen Klassert <steffen.klassert@secunet.com> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: Lai Jiangshan <jiangshanlai@gmail.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Tejun Heo <tj@kernel.org> Cc: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Daniel Jordan [Fri, 6 Sep 2019 01:40:23 +0000 (21:40 -0400)]
workqueue: require CPU hotplug read exclusion for apply_workqueue_attrs
Change the calling convention for apply_workqueue_attrs to require CPU
hotplug read exclusion.
Avoids lockdep complaints about nested calls to get_online_cpus in a
future patch where padata calls apply_workqueue_attrs when changing
other CPU-hotplug-sensitive data structures with the CPU read lock
already held.
Signed-off-by: Daniel Jordan <daniel.m.jordan@oracle.com> Acked-by: Tejun Heo <tj@kernel.org> Acked-by: Steffen Klassert <steffen.klassert@secunet.com> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: Lai Jiangshan <jiangshanlai@gmail.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Stephen Boyd [Thu, 5 Sep 2019 16:41:12 +0000 (09:41 -0700)]
random: Use wait_event_freezable() in add_hwgenerator_randomness()
Sebastian reports that after commit ff296293b353 ("random: Support freezable
kthreads in add_hwgenerator_randomness()") we can call might_sleep() when the
task state is TASK_INTERRUPTIBLE (state=1). This leads to the following warning.
do not call blocking ops when !TASK_RUNNING; state=1 set at [<00000000349d1489>] prepare_to_wait_event+0x5a/0x180
WARNING: CPU: 0 PID: 828 at kernel/sched/core.c:6741 __might_sleep+0x6f/0x80
Modules linked in:
We shouldn't call kthread_freezable_should_stop() from deep within the
wait_event code because the task state is still set as
TASK_INTERRUPTIBLE instead of TASK_RUNNING and
kthread_freezable_should_stop() will try to call into the freezer with
the task in the wrong state. Use wait_event_freezable() instead so that
it calls schedule() in the right place and tries to enter the freezer
when the task state is TASK_RUNNING instead.
Reported-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Tested-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Cc: Keerthy <j-keerthy@ti.com> Fixes: ff296293b353 ("random: Support freezable kthreads in add_hwgenerator_randomness()") Signed-off-by: Stephen Boyd <swboyd@chromium.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: x86/aes-ni - use AES library instead of single-use AES cipher
The RFC4106 key derivation code instantiates an AES cipher transform
to encrypt only a single block before it is freed again. Switch to
the new AES library which is more suitable for such use cases.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: caam - dispose of IRQ mapping only after IRQ is freed
With IRQ requesting being managed by devres we need to make sure that
we dispose of IRQ mapping after and not before it is free'd (otherwise
we'll end up with a warning from the kernel). To achieve that simply
convert IRQ mapping to rely on devres as well.
Fixes: f314f12db65c ("crypto: caam - convert caam_jr_init() to use devres") Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com> Cc: Chris Healy <cphealy@gmail.com> Cc: Lucas Stach <l.stach@pengutronix.de> Cc: Horia Geantă <horia.geanta@nxp.com> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: Iuliana Prodan <iuliana.prodan@nxp.com> Cc: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org Reviewed-by: Horia Geantă <horia.geanta@nxp.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
In order to access IP block's registers we need to enable appropriate
clocks first, otherwise we are risking hanging the CPU.
The problem becomes very apparent when trying to use CAAM driver built
as a kernel module. In that case caam_probe() gets called after
clk_disable_unused() which means all of the necessary clocks are
guaranteed to be disabled.
Coincidentally, this change also fixes iomap leak introduced by early
return (instead of "goto iounmap_ctrl") in commit 41fc54afae70 ("crypto: caam - simplfy clock initialization")
Tested on ZII i.MX6Q+ RDU2
Fixes: 176435ad2ac7 ("crypto: caam - defer probing until QMan is available") Fixes: 41fc54afae70 ("crypto: caam - simplfy clock initialization") Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com> Cc: Chris Healy <cphealy@gmail.com> Cc: Lucas Stach <l.stach@pengutronix.de> Cc: Horia Geantă <horia.geanta@nxp.com> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: Iuliana Prodan <iuliana.prodan@nxp.com> Cc: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org Tested-by: Horia Geantă <horia.geanta@nxp.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: arm/aes-neonbs - implement ciphertext stealing for XTS
Update the AES-XTS implementation based on NEON instructions so that it
can deal with inputs whose size is not a multiple of the cipher block
size. This is part of the original XTS specification, but was never
implemented before in the Linux kernel.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: arm/aes-ce - implement ciphertext stealing for XTS
Update the AES-XTS implementation based on AES instructions so that it
can deal with inputs whose size is not a multiple of the cipher block
size. This is part of the original XTS specification, but was never
implemented before in the Linux kernel.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: arm64/aes-neonbs - implement ciphertext stealing for XTS
Update the AES-XTS implementation based on NEON instructions so that it
can deal with inputs whose size is not a multiple of the cipher block
size. This is part of the original XTS specification, but was never
implemented before in the Linux kernel.
Since the bit slicing driver is only faster if it can operate on at
least 7 blocks of input at the same time, let's reuse the alternate
path we are adding for CTS to process any data tail whose size is
not a multiple of 128 bytes.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: arm64/aes - implement support for XTS ciphertext stealing
Add the missing support for ciphertext stealing in the implementation
of AES-XTS, which is part of the XTS specification but was omitted up
until now due to lack of a need for it.
The asm helpers are updated so they can deal with any input size, as
long as the last full block and the final partial block are presented
at the same time. The glue code is updated so that the common case of
operating on a sector or page is mostly as before. When CTS is needed,
the walk is split up into two pieces, unless the entire input is covered
by a single step.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: arm64/aes-cts-cbc - move request context data to the stack
Since the CTS-CBC code completes synchronously, there is no point in
keeping part of the scratch data it uses in the request context, so
move it to the stack instead.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: skcipher - add the ability to abort a skcipher walk
After starting a skcipher walk, the only way to ensure that all
resources it has tied up are released is to complete it. In some
cases, it will be useful to be able to abort a walk cleanly after
it has started, so add this ability to the skcipher walk API.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: arm64/aes-neon - limit exposed routines if faster driver is enabled
The pure NEON AES implementation predates the bit-slicing one, and is
generally slower, unless the algorithm in question can only execute
sequentially.
So advertising the skciphers that the bit-slicing driver implements as
well serves no real purpose, and we can just disable them. Note that the
bit-slicing driver also has a link time dependency on the pure NEON
driver, for CBC encryption and for XTS tweak calculation, so we still
need both drivers on systems that do not implement the Crypto Extensions.
At the same time, expose those modaliases for the AES instruction based
driver. This is necessary since otherwise, we may end up loading the
wrong driver when any of the skciphers are instantiated before the CPU
capability based module loading has completed.
Finally, add the missing modalias for cts(cbc(aes)) so requests for
this algorithm will autoload the correct module.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
When the ARM AES instruction based crypto driver was introduced, there
were no known implementations that could benefit from a 4-way interleave,
and so a 3-way interleave was used instead. Since we have sufficient
space in the SIMD register file, let's switch to a 4-way interleave to
align with the 64-bit driver, and to ensure that we can reach optimum
performance when running under emulation on high end 64-bit cores.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: arm/aes-ce - yield the SIMD unit between scatterwalk steps
Reduce the scope of the kernel_neon_begin/end regions so that the SIMD
unit is released (and thus preemption re-enabled) if the crypto operation
cannot be completed in a single scatterwalk step. This avoids scheduling
blackouts due to preemption being enabled for unbounded periods, resulting
in a more responsive system.
After this change, we can also permit the cipher_walk infrastructure to
sleep, so set the 'atomic' parameter to skcipher_walk_virt() to false as
well.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Herbert Xu [Fri, 6 Sep 2019 03:13:06 +0000 (13:13 +1000)]
crypto: skcipher - Unmap pages after an external error
skcipher_walk_done may be called with an error by internal or
external callers. For those internal callers we shouldn't unmap
pages but for external callers we must unmap any pages that are
in use.
This patch distinguishes between the two cases by checking whether
walk->nbytes is zero or not. For internal callers, we now set
walk->nbytes to zero prior to the call. For external callers,
walk->nbytes has always been non-zero (as zero is used to indicate
the termination of a walk).
lib/crypto/sha256.c and include/crypto/sha256_base.h define
99% identical functions to init a sha256_state struct for sha224 or
sha256 use.
This commit moves the functions from lib/crypto/sha256.c to
include/crypto/sha.h (making them static inline) and makes the
sha224/256_base_init static inline functions from
include/crypto/sha256_base.h wrappers around the now also
static inline include/crypto/sha.h functions.
Signed-off-by: Hans de Goede <hdegoede@redhat.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Hans de Goede [Sun, 1 Sep 2019 20:35:28 +0000 (22:35 +0200)]
crypto: ccree - Rename arrays to avoid conflict with crypto/sha256.h
Rename the algo_init arrays to cc_algo_init so that they do not conflict
with the functions declared in crypto/sha256.h.
This is a preparation patch for folding crypto/sha256.h into crypto/sha.h.
Signed-off-by: Hans de Goede <hdegoede@redhat.com> Acked-by: Gilad Ben-Yossef <gilad@benyossef.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Daniel Mack [Sat, 31 Aug 2019 11:55:55 +0000 (13:55 +0200)]
hwrng: timeriomem - relax check on memory resource size
The timeriomem_rng driver only accesses the first 4 bytes of the given
memory area and currently, it also forces that memory resource to be
exactly 4 bytes in size.
This, however, is problematic when used with device-trees that are
generated from things like FPGA toolchains, where the minimum size
of an exposed memory block may be something like 4k.
Hence, let's only check for what's needed for the driver to operate
properly; namely that we have enough memory available to read the
random data from.
Signed-off-by: Daniel Mack <daniel@zonque.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: inside-secure - Minor code cleanup and optimizations
Some minor cleanup changing e.g. "if (!x) A else B" to "if (x) B else A",
merging some back-to-back if's with the same condition, collapsing some
back-to-back assignments to the same variable and replacing some weird
assignments with proper symbolics.
Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: inside-secure - Made .cra_priority value a define
Instead of having a fixed value (of 300) all over the place, the value for
for .cra_priority is now made into a define (SAFEXCEL_CRA_PRIORITY).
This makes it easier to play with, e.g. during development.
Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: inside-secure - Only enable algorithms advertised by the hardware
This patch probes the supported algorithms from the hardware and only
registers the ones that the hardware actually supports. This is necessary
because this is a generic driver supposed to run on a wide variety of
engines, which may or may not implement certain algorithms.
Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
ctx->alg and ctx->mode were set from safexcel_send_req through the
various safexcel_encrypt and _decrypt routines, but this makes little
sense as these are static per ciphersuite. So moved to _init instead,
in preparation of adding more ciphersuites.
Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Mao Wenan [Wed, 28 Aug 2019 08:07:40 +0000 (16:07 +0800)]
crypto: hisilicon - select CRYPTO_LIB_DES while compiling SEC driver
When CRYPTO_DEV_HISI_SEC=y, below compilation error is found after
'commit 894b68d8be4b ("crypto: hisilicon/des - switch to new verification routines")':
drivers/crypto/hisilicon/sec/sec_algs.o: In function `sec_alg_skcipher_setkey_des_cbc':
sec_algs.c:(.text+0x11f0): undefined reference to `des_expand_key'
drivers/crypto/hisilicon/sec/sec_algs.o: In function `sec_alg_skcipher_setkey_des_ecb':
sec_algs.c:(.text+0x1390): undefined reference to `des_expand_key'
make: *** [vmlinux] Error 1
This because DES library has been moved to lib/crypto in this commit
'04007b0e6cbb ("crypto: des - split off DES library from generic DES cipher driver")'.
Fix this by selecting CRYPTO_LIB_DES in CRYPTO_DEV_HISI_SEC.
Fixes: 04007b0e6cbb ("crypto: des - split off DES library from generic DES cipher driver") Fixes: 894b68d8be4b ("crypto: hisilicon/des - switch to new verification routines") Signed-off-by: Mao Wenan <maowenan@huawei.com> Acked-by: Jonathan Cameron <Jonathan.Cameron@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Ard Biesheuvel [Thu, 22 Aug 2019 15:47:31 +0000 (18:47 +0300)]
crypto: ccp - invoke fallback for XTS ciphertext stealing
For correctness and compliance with the XTS-AES specification, we are
adding support for ciphertext stealing to XTS implementations, even
though no use cases are known that will be enabled by this.
Since the ccp driver already has a fallback skcipher standby for
dealing with input sizes other than [16, 512, 1024, 2048, 4096],
just drop the check against the block size.
Cc: Tom Lendacky <thomas.lendacky@amd.com> Cc: Gary Hook <gary.hook@amd.com> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
YueHaibing [Thu, 22 Aug 2019 14:46:49 +0000 (22:46 +0800)]
crypto: nx - remove unused variables 'nx_driver_string' and 'nx_driver_version'
drivers/crypto/nx/nx.h:12:19: warning:
nx_driver_string defined but not used [-Wunused-const-variable=]
drivers/crypto/nx/nx.h:13:19: warning:
nx_driver_version defined but not used [-Wunused-const-variable=]
They are never used, so just remove it.
Reported-by: Hulk Robot <hulkci@huawei.com> Signed-off-by: YueHaibing <yuehaibing@huawei.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Andrey Smirnov [Tue, 20 Aug 2019 20:24:00 +0000 (13:24 -0700)]
crypto: caam - always select job ring via RSR on i.MX8MQ
Per feedback from NXP tech support the way to use register based
service interface on i.MX8MQ is to follow the same set of steps
outlined for the case when virtualization is enabled, regardless if it
is. Current version of SRM for i.MX8MQ speaks of DECO DID_MS and DECO
DID_LS registers, but apparently those are not implemented, so the
case when SCFGR[VIRT_EN]=0 should be handled the same as the case when
SCFGR[VIRT_EN]=1
Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com> Cc: Chris Spencer <christopher.spencer@sea.co.uk> Cc: Cory Tusar <cory.tusar@zii.aero> Cc: Chris Healy <cphealy@gmail.com> Cc: Lucas Stach <l.stach@pengutronix.de> Cc: Horia Geantă <horia.geanta@nxp.com> Cc: Aymen Sghaier <aymen.sghaier@nxp.com> Cc: Leonard Crestez <leonard.crestez@nxp.com> Cc: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Andrey Smirnov [Tue, 20 Aug 2019 20:23:59 +0000 (13:23 -0700)]
crypto: caam - select DMA address size at runtime
i.MX8 mScale SoC still use 32-bit addresses in its CAAM implmentation,
so we can't rely on sizeof(dma_addr_t) to detemine CAAM pointer
size. Convert the code to query CTPR and MCFGR for that during driver
probing.
Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com> Cc: Chris Spencer <christopher.spencer@sea.co.uk> Cc: Cory Tusar <cory.tusar@zii.aero> Cc: Chris Healy <cphealy@gmail.com> Cc: Lucas Stach <l.stach@pengutronix.de> Cc: Horia Geantă <horia.geanta@nxp.com> Cc: Aymen Sghaier <aymen.sghaier@nxp.com> Cc: Leonard Crestez <leonard.crestez@nxp.com> Cc: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Andrey Smirnov [Tue, 20 Aug 2019 20:23:58 +0000 (13:23 -0700)]
crypto: caam - don't hardcode inpentry size
Using dma_addr_t for elements of JobR input ring is not appropriate on
all 64-bit SoCs, some of which, like i.MX8MQ, use only 32-bit wide
pointers there. Convert all of the code to use explicit helper
function that can be later extended to support i.MX8MQ. No functional
change intended.
Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com> Cc: Chris Spencer <christopher.spencer@sea.co.uk> Cc: Cory Tusar <cory.tusar@zii.aero> Cc: Chris Healy <cphealy@gmail.com> Cc: Lucas Stach <l.stach@pengutronix.de> Cc: Horia Geantă <horia.geanta@nxp.com> Cc: Aymen Sghaier <aymen.sghaier@nxp.com> Cc: Leonard Crestez <leonard.crestez@nxp.com> Cc: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Andrey Smirnov [Tue, 20 Aug 2019 20:23:57 +0000 (13:23 -0700)]
crypto: caam - drop explicit usage of struct jr_outentry
Using struct jr_outentry to specify the layout of JobR output ring is
not appropriate for all 64-bit SoC, since some of them, like i.MX8MQ,
use 32-bit pointers there which doesn't match 64-bit
dma_addr_t. Convert existing code to use explicit helper functions to
access any of the JobR output ring elements, so that the support for
i.MX8MQ can be added later. No functional change intended.
Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com> Cc: Chris Spencer <christopher.spencer@sea.co.uk> Cc: Cory Tusar <cory.tusar@zii.aero> Cc: Chris Healy <cphealy@gmail.com> Cc: Lucas Stach <l.stach@pengutronix.de> Cc: Horia Geantă <horia.geanta@nxp.com> Cc: Aymen Sghaier <aymen.sghaier@nxp.com> Cc: Leonard Crestez <leonard.crestez@nxp.com> Cc: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Andrey Smirnov [Tue, 20 Aug 2019 20:23:56 +0000 (13:23 -0700)]
crypto: caam - move cpu_to_caam_dma() selection to runtime
Instead of selecting the implementation of
cpu_to_caam_dma()/caam_dma_to_cpu() at build time using the
preprocessor, convert the code to do that at run-time using IS_ENABLED
macro. This is needed to add support for i.MX8MQ. No functional change
intended.
Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com> Cc: Chris Spencer <christopher.spencer@sea.co.uk> Cc: Cory Tusar <cory.tusar@zii.aero> Cc: Chris Healy <cphealy@gmail.com> Cc: Lucas Stach <l.stach@pengutronix.de> Cc: Horia Geantă <horia.geanta@nxp.com> Cc: Aymen Sghaier <aymen.sghaier@nxp.com> Cc: Leonard Crestez <leonard.crestez@nxp.com> Cc: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Andrey Smirnov [Tue, 20 Aug 2019 20:23:55 +0000 (13:23 -0700)]
crypto: caam - make CAAM_PTR_SZ dynamic
In order to be able to configure CAAM pointer size at run-time, which
needed to support i.MX8MQ, which is 64-bit SoC with 32-bit pointer
size, convert CAAM_PTR_SZ to refer to a global variable of the same
name ("caam_ptr_sz") and adjust the rest of the code accordingly. No
functional change intended.
Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com> Cc: Chris Spencer <christopher.spencer@sea.co.uk> Cc: Cory Tusar <cory.tusar@zii.aero> Cc: Chris Healy <cphealy@gmail.com> Cc: Lucas Stach <l.stach@pengutronix.de> Cc: Horia Geantă <horia.geanta@nxp.com> Cc: Aymen Sghaier <aymen.sghaier@nxp.com> Cc: Leonard Crestez <leonard.crestez@nxp.com> Cc: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Andrey Smirnov [Tue, 20 Aug 2019 20:23:52 +0000 (13:23 -0700)]
crypto: caam - use ioread64*_hi_lo in rd_reg64
Following the same transformation logic as outlined in previous commit
converting wr_reg64, convert rd_reg64 to use helpers from
<linux/io-64-nonatomic-hi-lo.h> first. No functional change intended.
Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com> Reviewed-by: Horia Geantă <horia.geanta@nxp.com> Cc: Chris Spencer <christopher.spencer@sea.co.uk> Cc: Cory Tusar <cory.tusar@zii.aero> Cc: Chris Healy <cphealy@gmail.com> Cc: Lucas Stach <l.stach@pengutronix.de> Cc: Horia Geantă <horia.geanta@nxp.com> Cc: Aymen Sghaier <aymen.sghaier@nxp.com> Cc: Leonard Crestez <leonard.crestez@nxp.com> Cc: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Andrey Smirnov [Tue, 20 Aug 2019 20:23:51 +0000 (13:23 -0700)]
crytpo: caam - make use of iowrite64*_hi_lo in wr_reg64
In order to be able to unify 64 and 32 bit implementations of
wr_reg64, let's convert it to use helpers from
<linux/io-64-nonatomic-hi-lo.h> first. Here are the steps of the
transformation:
Andrey Smirnov [Tue, 20 Aug 2019 20:23:50 +0000 (13:23 -0700)]
crypto: caam - request JR IRQ as the last step
In order to avoid any risk of JR IRQ request being handled while some
of the resources used for that are not yet allocated move the code
requesting said IRQ to the endo of caam_jr_init().
Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com> Cc: Chris Spencer <christopher.spencer@sea.co.uk> Cc: Cory Tusar <cory.tusar@zii.aero> Cc: Chris Healy <cphealy@gmail.com> Cc: Lucas Stach <l.stach@pengutronix.de> Cc: Horia Geantă <horia.geanta@nxp.com> Cc: Aymen Sghaier <aymen.sghaier@nxp.com> Cc: Leonard Crestez <leonard.crestez@nxp.com> Cc: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Andrey Smirnov [Tue, 20 Aug 2019 20:23:49 +0000 (13:23 -0700)]
crypto: caam - convert caam_jr_init() to use devres
Use devres to allocate all of the resources in caam_jr_init() (DMA
coherent and regular memory, IRQs) drop calls to corresponding
deallocation routines. No functional change intended.
Andrey Smirnov [Tue, 20 Aug 2019 20:23:47 +0000 (13:23 -0700)]
crypto: caam - move DMA mask selection into a function
Exactly the same code to figure out DMA mask is repeated twice in the
driver code. To avoid repetition, move that logic into a standalone
subroutine in intern.h. While at it re-shuffle the code to make it
more readable with early returns.
Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com> Reviewed-by: Horia Geantă <horia.geanta@nxp.com> Cc: Chris Spencer <christopher.spencer@sea.co.uk> Cc: Cory Tusar <cory.tusar@zii.aero> Cc: Chris Healy <cphealy@gmail.com> Cc: Lucas Stach <l.stach@pengutronix.de> Cc: Horia Geantă <horia.geanta@nxp.com> Cc: Aymen Sghaier <aymen.sghaier@nxp.com> Cc: Leonard Crestez <leonard.crestez@nxp.com> Cc: linux-crypto@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: inside-secure - add support for using the EIP197 without vendor firmware
Until now, the inside-secure driver required a set of firmware images
supplied by the silicon vendor, typically under NDA, to be present in
/lib/firmware/inside-secure in order to be able to function.
This patch removes the dependence on this official vendor firmware by
falling back to generic "mini" FW - developed specifically for this
driver - that can be provided under GPL 2.0 through linux-firmwares.
Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com> Acked-by: Antoine Tenart <antoine.tenart@bootlin.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: inside-secure - add support for PCI based FPGA development board
This patch adds support for a PCIE development board with FPGA from Xilinx,
to facilitate pre-silicon driver development by both Inside Secure and its
IP customers. Since Inside Secure neither produces nor has access to actual
silicon, this is required functionality to allow us to contribute.
Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com> Acked-by: Antoine Tenart <antoine.tenart@bootlin.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: inside-secure - Remove redundant algo to engine mapping code
This removes some code determine which engine has which algorithms which
was effectively redundant (may have been forward-looking?) due to always
enabling all algorithms for all currently supported engines.
A future patch will use a different, more scalable approach to achieve
this. This is removed now because otherwise the next patch will add new
hardware which would otherwise have to be added to all algorithms, so
now is a convenient time to just get rid of this.
Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com> Acked-by: Antoine Tenart <antoine.tenart@bootlin.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
crypto: inside-secure - make driver selectable for non-Marvell hardware
While being a generic EIP97/EIP197 driver, the driver was only selectable
for Marvell Armada hardware. This fix makes the driver selectable for any
Device Tree supporting kernel configuration, allowing it to be used for
other compatible hardware by just adding the correct device tree entry.
It also allows the driver to be selected for PCI(E) supporting kernel con-
figurations, to be able to use it with PCIE based FPGA development boards
for pre-silicon driver development by both Inside Secure and its IP custo-
mers.
Signed-off-by: Pascal van Leeuwen <pvanleeuwen@verimatrix.com> Acked-by: Antoine Tenart <antoine.tenart@bootlin.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Ard Biesheuvel [Mon, 19 Aug 2019 14:22:25 +0000 (17:22 +0300)]
crypto: s5p - deal gracefully with bogus input sizes
The s5p skcipher driver returns -EINVAL for zero length inputs, which
deviates from the behavior of the generic ECB template, and causes fuzz
tests to fail. In cases where the input is not a multiple of the AES
block size (and the chaining mode is not CTR), it prints an error to
the kernel log, which is a thing we usually try to avoid in response
to situations that can be triggered by unprivileged users.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Acked-by: Kamil Konieczny <k.konieczny@partner.samsung.com> Reviewed-by: Krzysztof Kozlowski <krzk@kernel.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add an accelerated version of the 'essiv(cbc(aes),sha256)' skcipher,
which is used by fscrypt or dm-crypt on systems where CBC mode is
signficantly more performant than XTS mode (e.g., when using a h/w
accelerator which supports the former but not the latter) This avoids
a separate call into the AES cipher for every invocation.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Ard Biesheuvel [Mon, 19 Aug 2019 14:17:35 +0000 (17:17 +0300)]
crypto: arm64/aes-cts-cbc - factor out CBC en/decryption of a walk
The plain CBC driver and the CTS one share some code that iterates over
a scatterwalk and invokes the CBC asm code to do the processing. The
upcoming ESSIV/CBC mode will clone that pattern for the third time, so
let's factor it out first.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Ard Biesheuvel [Mon, 19 Aug 2019 14:17:34 +0000 (17:17 +0300)]
crypto: essiv - add tests for essiv in cbc(aes)+sha256 mode
Add a test vector for the ESSIV mode that is the most widely used,
i.e., using cbc(aes) and sha256, in both skcipher and AEAD modes
(the latter is used by tcrypt to encapsulate the authenc template
or h/w instantiations of the same)
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Ard Biesheuvel [Mon, 19 Aug 2019 14:15:00 +0000 (17:15 +0300)]
crypto: arm64/aegis128 - use explicit vector load for permute vectors
When building the new aegis128 NEON code in big endian mode, Clang
complains about the const uint8x16_t permute vectors in the following
way:
crypto/aegis128-neon-inner.c:58:40: warning: vector initializers are not
compatible with NEON intrinsics in big endian mode
[-Wnonportable-vector-initialization]
static const uint8x16_t shift_rows = {
^
crypto/aegis128-neon-inner.c:58:40: note: consider using vld1q_u8() to
initialize a vector from memory, or vcombine_u8(vcreate_u8(), vcreate_u8())
to initialize from integer constants
Since the same issue applies to the uint8x16x4_t loads of the AES Sbox,
update those references as well. However, since GCC does not implement
the vld1q_u8_x4() intrinsic, switch from IS_ENABLED() to a preprocessor
conditional to conditionally include this code.
Hans de Goede [Sat, 17 Aug 2019 14:24:35 +0000 (16:24 +0200)]
crypto: sha256_generic - Switch to the generic lib/crypto/sha256.c lib code
Drop the duplicate generic sha256 (and sha224) implementation from
crypto/sha256_generic.c and use the implementation from
lib/crypto/sha256.c instead.
"diff -u lib/crypto/sha256.c sha256_generic.c" shows that the core
sha256_transform function from both implementations is identical and
the other code is functionally identical too.
Suggested-by: Eric Biggers <ebiggers@kernel.org> Signed-off-by: Hans de Goede <hdegoede@redhat.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Hans de Goede [Sat, 17 Aug 2019 14:24:34 +0000 (16:24 +0200)]
crypto: sha256 - Add sha224 support to sha256 library code
Add sha224 support to the lib/crypto/sha256 library code. This will allow
us to replace both the sha256 and sha224 parts of crypto/sha256_generic.c
when we remove the code duplication in further patches in this series.
Suggested-by: Eric Biggers <ebiggers@kernel.org> Signed-off-by: Hans de Goede <hdegoede@redhat.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Hans de Goede [Sat, 17 Aug 2019 14:24:33 +0000 (16:24 +0200)]
crypto: sha256 - Make lib/crypto/sha256.c suitable for generic use
Before this commit lib/crypto/sha256.c has only been used in the s390 and
x86 purgatory code, make it suitable for generic use:
* Export interesting symbols
* Add -D__DISABLE_EXPORTS to CFLAGS_sha256.o for purgatory builds to
avoid the exports for the purgatory builds
* Add to lib/crypto/Makefile and crypto/Kconfig
Signed-off-by: Hans de Goede <hdegoede@redhat.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Hans de Goede [Sat, 17 Aug 2019 14:24:32 +0000 (16:24 +0200)]
crypto: sha256 - Use get/put_unaligned_be32 to get input, memzero_explicit
Use get/put_unaligned_be32 in lib/crypto/sha256.c to load / store data
so that it can be used with unaligned buffers too, making it more generic.
And use memzero_explicit for better clearing of sensitive data.
Note unlike other patches in this series this commit actually makes
functional changes to the sha256 code as used by the purgatory code.
This fully aligns the lib/crypto/sha256.c sha256 implementation with the
one from crypto/sha256_generic.c allowing us to remove the latter in
further patches in this series.
Signed-off-by: Hans de Goede <hdegoede@redhat.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Hans de Goede [Sat, 17 Aug 2019 14:24:31 +0000 (16:24 +0200)]
crypto: sha256 - Move lib/sha256.c to lib/crypto
Generic crypto implementations belong under lib/crypto not directly in
lib, likewise the header should be in include/crypto, not include/linux.
Note that the code in lib/crypto/sha256.c is not yet available for
generic use after this commit, it is still only used by the s390 and x86
purgatory code. Making it suitable for generic use is done in further
patches in this series.
Signed-off-by: Hans de Goede <hdegoede@redhat.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
projects / mirror_ubuntu-kernels.git / log