]> git.proxmox.com Git - wasi-libc.git/log
wasi-libc.git
4 months agoNew upstream version 0.0~git20230821.ec4566b
Fabian Grünbichler [Mon, 27 May 2024 12:47:45 +0000 (14:47 +0200)]
New upstream version 0.0~git20230821.ec4566b

5 months agoNew upstream version 0.0~git20230621.7018e24
Fabian Grünbichler [Fri, 17 May 2024 10:04:56 +0000 (12:04 +0200)]
New upstream version 0.0~git20230621.7018e24

13 months agoFix recursive mutex (#433)
YAMAMOTO Takashi [Mon, 21 Aug 2023 17:21:37 +0000 (02:21 +0900)]
Fix recursive mutex (#433)

the robust mutex logic in musl seems to assume that
the bit 29 of TIDs is always zero for some reasons.

from https://git.musl-libc.org/cgit/musl/commit/?id=099b89d3840c30d7dd962e18668c2e6d39f0c626
> note that the kernel ABI also reserves bit 29
> not to appear in any tid,

i'm not sure if the assumption is true or not, given that
FUTEX_TID_MASK is 0x3fffffff.

anyway, when using non-default type of mutex like recursive mutex,
it causes problems as we actually use TID 0x3fffffff for the main thread.

as we don't support robust mutex anyway, this commit simply
comments out the problematic condition.

fixes: https://github.com/bytecodealliance/wasm-micro-runtime/issues/2466

15 months agoAdd definitions for PF_INET, PF_INET6 and PF_UNSPEC (#426)
Marcin Kolny [Thu, 13 Jul 2023 16:18:08 +0000 (17:18 +0100)]
Add definitions for PF_INET, PF_INET6 and PF_UNSPEC (#426)

Given there are already AF_* definitions, and they are (now) essentially
synonyms, we add those definitions to enable compilation of code that
already use PF_* macros.

15 months agoAcquire the global lock before initializing malloc (#410)
Moritz Sichert [Tue, 11 Jul 2023 22:33:07 +0000 (00:33 +0200)]
Acquire the global lock before initializing malloc (#410)

In a multi-threaded execution we need to make sure that only exactly one
thread initializes malloc. The function try_init_allocator() can't
easily be made thread-safe, so just move the call to
try_init_allocator() inside the block that holds the lock.

15 months agodlmalloc: require __heap_end (#394)
YAMAMOTO Takashi [Tue, 11 Jul 2023 22:32:08 +0000 (07:32 +0900)]
dlmalloc: require __heap_end (#394)

This commit effectively drops the support of older wasm-ld. (LLVM <15.0.7).

We have two relevant use cases:

* `memory.grow` use outside of malloc
  (eg. used by polyfill preview1 binaries)

* `--init-memory` to somehow preallocate heap
  (eg. avoid dynamic allocations, especially on small environments)

While https://github.com/WebAssembly/wasi-libc/pull/377
fixed the former, it broke the latter if you are using
an older LLVM, which doesn't provide the `__heap_end` symbol,
to link your module.

As we couldn't come up with a solution which satisfies all parties,
this commit simply makes it require new enough LLVM which provides
`__heap_end`. After all, a link-time failure is more friendly to users
than failing later in a subtle way.

15 months agoUse -fno-strict-aliasing for emmalloc (#424)
Josh Stone [Fri, 23 Jun 2023 20:24:52 +0000 (13:24 -0700)]
Use -fno-strict-aliasing for emmalloc (#424)

15 months agoImprove `README.md` (#425)
Andrew Brown [Fri, 23 Jun 2023 20:24:08 +0000 (13:24 -0700)]
Improve `README.md` (#425)

This changes the front-page documentation to:
- use `wasi-libc` instead of "WASI Libc"
- explain how to build the pthreads-enabled `wasm32-wasi-threads` target

15 months agoFix a use-after-free bug for detached threads (#420)
YAMAMOTO Takashi [Wed, 21 Jun 2023 23:45:57 +0000 (08:45 +0900)]
Fix a use-after-free bug for detached threads (#420)

* Fix a use-after-free bug for detached threads

the issue was pointed out by @alexcrichton in
https://github.com/WebAssembly/wasi-libc/issues/405

* Rename map_base_lazy_free_queue as it only keeps a single item

Also, align the comment style with musl.

16 months agoAdjust Makefile for LLVM trunk (17) as of 2023-06-18 (#422)
Mike Hommey [Tue, 20 Jun 2023 17:47:19 +0000 (02:47 +0900)]
Adjust Makefile for LLVM trunk (17) as of 2023-06-18 (#422)

https://github.com/llvm/llvm-project/commit/7dd387d2971d7759cadfffeb2082439f6c7ddd49
added a bunch of __FPCLASS_* macros.

16 months agoFix races around pthread exit and join (#409)
YAMAMOTO Takashi [Wed, 7 Jun 2023 16:51:07 +0000 (01:51 +0900)]
Fix races around pthread exit and join (#409)

17 months agoConvert preopen initialization to be lazy. (#408)
Dan Gohman [Wed, 3 May 2023 19:44:21 +0000 (12:44 -0700)]
Convert preopen initialization to be lazy. (#408)

* Convert preopen initialization to be lazy.

Insteead of eagerly initializing the preopens in a static constructor,
perform preopen initialization the first time it's needed, or before a
close or a renumber which might disrupt the file descriptor space.

And, use a weak symbol with a stub function for use by `close` or
`fd_renumber`, we that they can trigger preopen initialization only
if it's actually needed.

This way, if a program doesn't contain any calls to any function that
needs preopens, it can avoid linking in the preopen initialization code.
And if it contains calls but doesn't execute them at runtime, it can
avoid executing the preopen intiailization code.

A downside here is that this may cause problems for users that call
`__wasi_fd_close` or `__wasi_fd_renumber` directly and close over
overwrite some preopens before libc has a chance to scan them. To
partially address this, this PR does add a declaration for
`__wasilibc_populate_preopens` to <wasi/libc.h> so that users can call
it manually if they need to.

* Fix calling `internal_register_preopened_fd` with the lock held.

Factor out the lock acquisition from the implementation of
`internal_register_preopened_fd` so that we can call it from
`__wasilibc_populate_preopens` with the lock held.

17 months agoUpdate README regarding the Arch Linux package (#412)
Jiri Pospisil [Thu, 27 Apr 2023 17:11:07 +0000 (19:11 +0200)]
Update README regarding the Arch Linux package (#412)

The Arch Linux package is now official and there are other WASI related
packages as well.

18 months agoFix debug build's `predefined-macros.txt` (#407)
Alex Crichton [Wed, 5 Apr 2023 19:13:39 +0000 (14:13 -0500)]
Fix debug build's `predefined-macros.txt` (#407)

This commit fixes the ability to build `wasi-libc` with `-g` options and
possibly without `-O2` options as well. I've found this useful when
debugging issues as historically that the build fails when `-g` is
passed or optimizations are removed due to the checks against these
expectation files. This commit adds more filters to the list of macros
to ensure that optimization/debug related ones are all removed from the
expectation lists.

18 months agoFix a_store operation in atomic.h (#403)
Wenyong Huang [Fri, 24 Mar 2023 07:34:15 +0000 (15:34 +0800)]
Fix a_store operation in atomic.h (#403)

19 months agoUse __builtin_ctz and __builtin_clz in dlmalloc (#401)
Cheng Shao [Mon, 6 Mar 2023 16:37:38 +0000 (17:37 +0100)]
Use __builtin_ctz and __builtin_clz in dlmalloc (#401)

Co-authored-by: Ben Smith <binjimin@gmail.com>
19 months agoSort the object list passed to ar in the Makefile. (#399)
Pier Angelo Vendrame [Fri, 3 Mar 2023 17:29:24 +0000 (18:29 +0100)]
Sort the object list passed to ar in the Makefile. (#399)

This makes builds reproducible.

19 months agoAvoid using absolute pathnames in Makefile. NFC (#400)
Sam Clegg [Thu, 2 Mar 2023 01:14:42 +0000 (17:14 -0800)]
Avoid using absolute pathnames in Makefile. NFC (#400)

This makes the output of the build a lot more concise and easy to read.

The only real change here is to build each of the crt1 startup files
individually instead to trying to build them all in a single clang
invocation (that latter doesn't allow for -o to be specified which is
a pretty severe limitation, so its best avoided anyway).

It also reduces the size of the `ar` command line for libc itself from
78017 to 43609 (on my machine), which sadly is still tool long for win32
I believe.

19 months agothreads: enable `PTHREAD_{MUTEX,RWLOCK,COND}_INITIALIZER`. (#397)
Catherine [Fri, 24 Feb 2023 15:12:37 +0000 (15:12 +0000)]
threads: enable `PTHREAD_{MUTEX,RWLOCK,COND}_INITIALIZER`. (#397)

19 months agosetup_default_stack_size: set __default_stacksize unconditionally (#396)
YAMAMOTO Takashi [Wed, 22 Feb 2023 15:02:50 +0000 (00:02 +0900)]
setup_default_stack_size: set __default_stacksize unconditionally (#396)

If a user specifies a small stack size for the main,
maybe it's reasonable to use the same size for threads as well.

20 months agoFix MSG_TRUNC (#391)
YAMAMOTO Takashi [Sat, 4 Feb 2023 15:45:56 +0000 (00:45 +0900)]
Fix MSG_TRUNC (#391)

20 months agoAdd a check to reactor modules to ensure _initialize is only called once (#388)
Cheng Shao [Mon, 30 Jan 2023 12:30:02 +0000 (13:30 +0100)]
Add a check to reactor modules to ensure _initialize is only called once (#388)

Calling _initialize multiple times is undefined behavior, since the
ctors are not guaranteed to be idempotent. We should have this safety
check which is similar to #329.

20 months agothreads: implement support for spinlock (#324)
Marcin Kolny [Thu, 26 Jan 2023 17:42:48 +0000 (17:42 +0000)]
threads: implement support for spinlock (#324)

20 months agoRename thread_spawn import (#387)
YAMAMOTO Takashi [Wed, 25 Jan 2023 01:14:03 +0000 (10:14 +0900)]
Rename thread_spawn import (#387)

Following the wit-defined ABI:
https://github.com/WebAssembly/wasi-threads/pull/26

21 months agothreads: change `wasm32-wasi-pthread` to `wasm32-wasi-threads` (#381)
Andrew Brown [Fri, 13 Jan 2023 02:56:52 +0000 (18:56 -0800)]
threads: change `wasm32-wasi-pthread` to `wasm32-wasi-threads` (#381)

* Change `wasm32-wasi-pthread` to `wasm32-wasi-threads`

After some thought, I think that we should rename the `THREAD_MODEL=posix` build to avoid confusion. Why? Though in this project the use of this target does involve pthreads, it will not be so in other standard libraries or languages (see, e.g., https://github.com/rust-lang/compiler-team/issues/574). I think it would be preferable to emphasize the "threads" Wasm-level proposal and the "wasi-threads" proposal rather than the specific details of which threading API is being exposed.

* fix: rename the `expected` output directory as well

21 months agothreads: add `pthread_attr_setdetachstate` (#382)
Andrew Brown [Thu, 12 Jan 2023 03:47:44 +0000 (19:47 -0800)]
threads: add `pthread_attr_setdetachstate` (#382)

This API may not make a lot of sense in a WebAssembly world but it
seemed helpful to include it, even if it doesn't have much effect.

21 months agoRemove hacks for clang 8 (#384)
Mike Hommey [Thu, 12 Jan 2023 02:09:45 +0000 (11:09 +0900)]
Remove hacks for clang 8 (#384)

__FLOAT128__ has been defined since clang 9
__FLT16_* were defined until clang 9

21 months agoAdjust Makefile for LLVM trunk (16) as of 2023-01-05 (#379)
Mike Hommey [Tue, 10 Jan 2023 00:21:58 +0000 (09:21 +0900)]
Adjust Makefile for LLVM trunk (16) as of 2023-01-05 (#379)

https://github.com/llvm/llvm-project/commit/d227c3b68cf5c236902c9ff4fdf8b719c9a3dd26
added __GCC_HAVE_SYNC_COMPARE_AND_SWAP macros.

21 months agoUse __BIGGEST_ALIGNMENT__ instead of max_align_t (#375)
YAMAMOTO Takashi [Mon, 9 Jan 2023 16:33:58 +0000 (01:33 +0900)]
Use __BIGGEST_ALIGNMENT__ instead of max_align_t (#375)

Also, remove no longer necessary __need_STDDEF_H_misc stuff.

References:
https://github.com/WebAssembly/wasi-libc/pull/335
https://github.com/WebAssembly/wasi-sdk/issues/111
https://github.com/llvm/llvm-project/blob/2e999b7dd1934a44d38c3a753460f1e5a217e9a5/clang/lib/Headers/stddef.h#L106-L113

21 months agoDon't use sbrk(0) to determine the initial heap size (#377)
Alex Crichton [Mon, 9 Jan 2023 16:33:05 +0000 (10:33 -0600)]
Don't use sbrk(0) to determine the initial heap size (#377)

* Don't use sbrk(0) to determine the initial heap size

This commit changes the `try_init_allocator` function as part of
dlmalloc to not use `sbrk(0)` to determine the initial heap size. The
purpose of this function is to use the extra memory at the end of linear
memory for the initial allocation heap before `memory.grow` is used to
allocate more memory. To learn the extent of this region the code
previously would use `sbrk(0)` to find the current size of linear
memory. This does not work, however, when other systems have called
`memory.grow` before this function is called. For example if another
allocator is used or if another component of a wasm binary grows memory
for its own purposes then that memory will be incorrectly claimed to be
owned by dlmalloc.

Instead this commit rounds up the `__heap_base` address to the nearest
page size, since that must be allocatable. Otherwise anything above this
rounded address is assumed to be used by something else, even if it's
addressable.

* Use `__heap_end` if defined

* Move mstate initialization earlier

21 months ago__init_tp: Initialize TID to non-zero value (#360)
YAMAMOTO Takashi [Fri, 6 Jan 2023 19:35:27 +0000 (04:35 +0900)]
__init_tp: Initialize TID to non-zero value (#360)

This fixes TID-based locking used within libc.

Also, initialize detach_state.

cf. https://github.com/WebAssembly/wasi-threads/pull/16

21 months agoImplement the critical part of wasi_thread_start in asm (#376)
YAMAMOTO Takashi [Fri, 6 Jan 2023 19:34:22 +0000 (04:34 +0900)]
Implement the critical part of wasi_thread_start in asm (#376)

* Implement the critical part of wasi_thread_start in asm

It's fragile to set up the critical part of C environment in C.

* Specify --target for asm files as well

* wasi_thread_start: Move __tls_base initialization to asm as well

21 months agoEnable pthread_equal function definition (#374)
Shengyun Zhou [Tue, 27 Dec 2022 10:34:03 +0000 (18:34 +0800)]
Enable pthread_equal function definition (#374)

21 months agoUse a separate OBJDIR for each TARGET_TRIPLE (#373)
YAMAMOTO Takashi [Mon, 26 Dec 2022 10:19:11 +0000 (19:19 +0900)]
Use a separate OBJDIR for each TARGET_TRIPLE (#373)

To make it easier to create a sysroot with both triples.

Eg.
```
make -j4 CC=/opt/wasi-sdk-16.0/bin/clang
make -j4 CC=/opt/wasi-sdk-16.0/bin/clang THREAD_MODEL=posix
```

21 months agowasi_thread_start: add a comment (#371)
YAMAMOTO Takashi [Fri, 23 Dec 2022 14:14:21 +0000 (23:14 +0900)]
wasi_thread_start: add a comment (#371)

21 months agoUse `ENOENT` rather than `ENOTCAPABLE` for missing preopens. (#370)
Dan Gohman [Wed, 21 Dec 2022 23:47:53 +0000 (15:47 -0800)]
Use `ENOENT` rather than `ENOTCAPABLE` for missing preopens. (#370)

When a user calls `open` with a path that does not have a corresponding
preopen, set errno to `ENOENT` rather than `ENOTCAPABLE`. This
conceptually represents an attempt to open a path which has not been
provided within the sandbox, so it's more accurately represented as
"not present" rather than "insufficient capabilities".

21 months agoDisable pthread_exit for now (#366)
YAMAMOTO Takashi [Wed, 21 Dec 2022 01:24:09 +0000 (10:24 +0900)]
Disable pthread_exit for now (#366)

The current wasi-threads has no thread-exit functionality.
Thus it isn't straightforward to implement pthread_exit
without leaking thread context. This commit simply disables
pthread_exit for now.

Also, instead of abusing `wasi_proc_exit` for thread exit,
make `wasi_thread_start` return.

Note: `wasi_proc_exit` is supposed to terminate all threads
in the "process", not only the calling thread.

Note: Depending on the conclusion of the discussion about
`wasi_thread_exit`, we might revisit this change later.

References:
https://github.com/WebAssembly/wasi-threads/issues/7
https://github.com/WebAssembly/wasi-threads/pull/17

22 months agothreads: Retrieve default stack size from __heap_base/__data_end (#350)
Marcin Kolny [Mon, 19 Dec 2022 12:18:19 +0000 (12:18 +0000)]
threads: Retrieve default stack size from __heap_base/__data_end (#350)

When compiling with `-z stack-size` flag, only the main thread's stack
size is set to the specified value and other threads use musl's default value.
That's inconsistent with LLD's `-Wl,-stack_size`.

I think we can make it similar to MUSL's behavior, where thread's stack
size can be set via `PT_GNU_STACK` program header (via `-Wl,-z,stack-size`
flag).

Configuring stack size through `pthread_attr_t` still work as expected and
overrides the defaults ([pthread_create.c](https://github.com/WebAssembly/wasi-libc/blob/be1ffd6a9eba1704085987482557c2a32724227f/libc-top-half/musl/src/thread/pthread_create.c#L362))
default settings.

22 months agoReduce over-allocation of stack (#365)
YAMAMOTO Takashi [Mon, 19 Dec 2022 12:16:54 +0000 (21:16 +0900)]
Reduce over-allocation of stack (#365)

* Disable stack guard
* Stop rounding up stack size to PAGE_SIZE

22 months agoAdd a separate install target for threaded libc (#331)
Petr Penzin [Wed, 14 Dec 2022 01:57:57 +0000 (17:57 -0800)]
Add a separate install target for threaded libc (#331)

Produce a different sysroot directory for threaded target. Restructure
`expected` directory to correspond to the target.

22 months ago__wait.c: fix a timeout (#361)
YAMAMOTO Takashi [Tue, 13 Dec 2022 20:07:33 +0000 (05:07 +0900)]
__wait.c: fix a timeout (#361)

Note: The typical symptom of this bug is a busy waiting on a lock.

Note: 0 means immediate timeout. a negative value means no timeout.

22 months agoEnable flockfile and friends (#362)
YAMAMOTO Takashi [Tue, 13 Dec 2022 17:56:45 +0000 (02:56 +0900)]
Enable flockfile and friends (#362)

22 months agothreads: enable access to `pthread_barrier_*` functions (#358)
Andrew Brown [Tue, 13 Dec 2022 17:19:32 +0000 (09:19 -0800)]
threads: enable access to `pthread_barrier_*` functions (#358)

In building some `libc-test` tests, I found these functions were not
compiled in. This change adds `pthread_barrier_init`,
`pthread_barrier_wait`, and `pthread_barrier_destroy` to the
`THREAD_MODEL=posix` build. As has been done with previous pthreads PRs,
this PR skips any inter-process locking by removing any calls to
`__vm_lock` and friends. If in the future WASI gains the "process"
concept, then these locations (and the pre-existing ones) will need to
be modified.

22 months agoEnable pthread_detach (#359)
YAMAMOTO Takashi [Fri, 9 Dec 2022 17:58:46 +0000 (02:58 +0900)]
Enable pthread_detach (#359)

22 months agothreads: enable access to `pthread_attr_get` functions (#357)
Andrew Brown [Thu, 8 Dec 2022 22:55:42 +0000 (14:55 -0800)]
threads: enable access to `pthread_attr_get` functions (#357)

The pthreads API exposes functions for querying the attributes of a
thread. This change allows these functions to be compiled in the
`THREAD_MODEL=posix` build. Some functions are skipped (and documented);
they can be added if/when needed. This change is motivated by a
`libc-test` test that uses these functions.

22 months agoDon't define `_POSIX_THREADS` unless threads are enabled. (#356)
Dan Gohman [Wed, 7 Dec 2022 21:54:19 +0000 (13:54 -0800)]
Don't define `_POSIX_THREADS` unless threads are enabled. (#356)

* Don't define `_POSIX_THREADS` unless threads are enabled.

Fixes #355.

* Remove `_POSIX_THREADS` from predefined-macros.txt.

22 months agowasi_thread_start: remove a useless cast (#354)
YAMAMOTO Takashi [Wed, 7 Dec 2022 17:31:41 +0000 (02:31 +0900)]
wasi_thread_start: remove a useless cast (#354)

22 months ago__wasi_thread_spawn: stop truncating the return value (#353)
YAMAMOTO Takashi [Wed, 7 Dec 2022 17:18:28 +0000 (02:18 +0900)]
__wasi_thread_spawn: stop truncating the return value (#353)

as __wasi_errno_t is uint16_t, with the current coding,
__pthread_create will never see negative return values from
wasi:thread_spawn.
eg. (int)(uint16_t)-1 == 65535.

22 months agoAdjust Makefile for LLVM trunk (16) as of 2022-11-08 (#344)
Mike Hommey [Wed, 7 Dec 2022 16:27:06 +0000 (01:27 +0900)]
Adjust Makefile for LLVM trunk (16) as of 2022-11-08 (#344)

https://github.com/llvm/llvm-project/commit/1e4e2433bcd1a0296ef1043c462252f0d087d90c
enabled sign-ext and mutable-globals by default, which adds
corresponding __wasm_-prefixed #defines.

https://github.com/llvm/llvm-project/commit/9e956995db1fc7e792e3dfb3a465a52626195557
changed the definition of __GNUC_VA_LIST to match that of GCC headers,
leaving it without a value.

22 months agoAdd `-fstack-protector` support to wasi-libc (#351)
Yuta Saito [Tue, 6 Dec 2022 17:19:49 +0000 (02:19 +0900)]
Add `-fstack-protector` support to wasi-libc (#351)

Inlcude `__stack_chk_fail.c` and initialize `__stack_chk_guard` in ctor.

```
$ cat main.c
char input[] = "0123456789012345";
int main(void) {
    char buf[8];

    for (char *sp = input, *dp = buf; *sp != '\0'; sp++, dp++) {
        *dp = *sp;
    }
    return 0;
}

$ clang main.c -fstack-protector
$ wasmtime ./a.out
Error: failed to run main module `./a.out`

Caused by:
    0: failed to invoke command default
    1: wasm trap: wasm `unreachable` instruction executed
       wasm backtrace:
           0:  0x258 - <unknown>!__stack_chk_fail
           1:  0x21e - <unknown>!__original_main
           2:   0xca - <unknown>!_start
```

22 months agoFix logic errors in the zero-inode path. (#352)
Dan Gohman [Tue, 6 Dec 2022 17:16:45 +0000 (09:16 -0800)]
Fix logic errors in the zero-inode path. (#352)

I've now tested the zero-inode path on a Wasm engine specially-modified
to have `fd_readdir` set inode numbers to zero. Fix two bugs this turned up:
 - Increment `buffer_processed`, as noticed by @yamt
 - Don't do an `fstatat` on "..", because that references a path outside
   of the directory, which gets a permission-denied error.

22 months agotest: run a subset of tests using `libc-test` (#346)
Andrew Brown [Sat, 3 Dec 2022 02:27:42 +0000 (18:27 -0800)]
test: run a subset of tests using `libc-test` (#346)

* test: run a subset of tests from `libc-test`

This change introduces a `test` directory that retrieves the `libc-test`
suite, compiles a subset of the tests using `wasi-libc`, and runs them
with Wasmtime.

* ci: run tests during CI

This change includes some fixups to the filesystem to place Clang's
runtime library for `wasm32-wasi` in the right location. Note that this
CI action is limited to a single OS--Linux.

22 months agoIf `fd_readdir` returns a zero inode, call `fstatat` to get the inode value. (#345)
Dan Gohman [Fri, 2 Dec 2022 01:44:22 +0000 (17:44 -0800)]
If `fd_readdir` returns a zero inode, call `fstatat` to get the inode value. (#345)

* If `fd_readdir` returns a zero inode, call `fstatat` to get the inode value.

On some systems, `fd_readdir` may not implement the `d_ino` field and
may set it to zero. When this happens, have wasi-libc call `fstatat` to
get the inode number.

See the discussion in
https://github.com/WebAssembly/wasi-filesystem/issues/65 for details.

* Update the `d_type` field too, in case it changes.

22 months agoEnable a few more pthread* files (pthread_key*, pthread_once) (#348)
Marcin Kolny [Wed, 30 Nov 2022 22:16:56 +0000 (22:16 +0000)]
Enable a few more pthread* files (pthread_key*, pthread_once) (#348)

* threads: enable TSD functions

* threads: Enable pthread_once

22 months agoInstall libtinfo5 on ubuntu. (#349)
Dan Gohman [Wed, 30 Nov 2022 19:17:08 +0000 (11:17 -0800)]
Install libtinfo5 on ubuntu. (#349)

We download LLVM releases built for Ubuntu 18 because LLVM doesn't
always have builds for different versions, but the builds we use
depend on libtinfo5 which isn't installed on Ubuntu 20 by default.
So install it.

22 months agoDefine an `__errno_location` function. (#347)
Dan Gohman [Mon, 28 Nov 2022 21:50:04 +0000 (13:50 -0800)]
Define an `__errno_location` function. (#347)

This function returns the address of `errno`, which makes it easier to
access from non-C languages since `errno` is a thread-local variable
which requires a special ABI.

23 months agoNew upstream version 0.0~git20220510.9886d3d
Fabian Grünbichler [Tue, 15 Nov 2022 19:02:10 +0000 (20:02 +0100)]
New upstream version 0.0~git20220510.9886d3d

23 months agothreads: implement init of TLS and stack pointer (#342)
韩朴宇 [Thu, 10 Nov 2022 22:12:53 +0000 (06:12 +0800)]
threads: implement init of TLS and stack pointer (#342)

* threads: implement init of TLS and stack pointer

* fix: rename wasi_snapshot_preview2_thread_spawn to wasi_thread_spawn

Signed-off-by: Harald Hoyer <harald@profian.com>
* fix: change signature of wasi_thread_start

Signed-off-by: Harald Hoyer <harald@profian.com>
* fix: pthread_exit for WASI

Can't use `exit()` because it is too high level.
Have to unlock the thread list.

Signed-off-by: Harald Hoyer <harald@profian.com>
* fix: initialize struct pthread for the main thread

Signed-off-by: Harald Hoyer <harald@profian.com>
* fix: store the aligned stack minus `struct start_args`

Signed-off-by: Harald Hoyer <harald@profian.com>
Signed-off-by: Harald Hoyer <harald@profian.com>
Co-authored-by: Harald Hoyer <harald@profian.com>
23 months agoSupport threads in the new crt1-command.c ctor check. (#339)
Dan Gohman [Tue, 8 Nov 2022 21:37:44 +0000 (13:37 -0800)]
Support threads in the new crt1-command.c ctor check. (#339)

Use an atomic compare-and-swap for checking whether constructors have
been run, when threads are enabled.

23 months agoPort emmalloc to wasi-libc.
Dan Gohman [Fri, 14 Oct 2022 23:09:55 +0000 (16:09 -0700)]
Port emmalloc to wasi-libc.

 - Avoid using Emscripten-specific functions
 - Avoid using a constructor.
 - Add support for allocating memory at `__heap_base`.
 - Adjust the max-align value.
 - Disable functions that wasi-libc doesn't currently publish.
 - Add `__libc_` aliases.

23 months agoBeginning porting Emscripten's emmalloc to wasi-libc.
Dan Gohman [Fri, 14 Oct 2022 23:01:00 +0000 (16:01 -0700)]
Beginning porting Emscripten's emmalloc to wasi-libc.

emmalloc.c is derived from Emscripten at this path here:

https://github.com/emscripten-core/emscripten/blob/main/system/lib/emmalloc.c

2 years agoAdd a check to command modules to ensure that they're only started once. (#329)
Dan Gohman [Fri, 14 Oct 2022 00:58:14 +0000 (17:58 -0700)]
Add a check to command modules to ensure that they're only started once. (#329)

* Add a check to command modules to ensure that they're only started once.

Wasm command modules should only be called once per instance, because
the programming model doesn't leave linear memory in a reusable state
when the program exits. As use cases arise for loading wasm modules in
environments that want to treat them like reactors, add a safety check
to ensure that command modules are used according to their
expectations.

2 years agothreads: enable pthread_join (#336)
Marcin Kolny [Mon, 10 Oct 2022 22:25:56 +0000 (23:25 +0100)]
threads: enable pthread_join (#336)

2 years agothreads: Enable __lock file for posix thread model (#337)
Marcin Kolny [Mon, 10 Oct 2022 22:19:25 +0000 (23:19 +0100)]
threads: Enable __lock file for posix thread model (#337)

2 years agoAvoid depending on `max_align_t` in <__struct_sockaddr.h>.
Dan Gohman [Mon, 10 Oct 2022 17:35:30 +0000 (10:35 -0700)]
Avoid depending on `max_align_t` in <__struct_sockaddr.h>.

`max_align_t` is not declared in C99 mode, so use an aligned attribute
and `__BIGGEST_ALIGNMENT__` instead.

Fixes WebAssembly/wasi-sdk#111.

2 years agoFix missing errno assignments.
Dan Gohman [Thu, 6 Oct 2022 18:48:54 +0000 (11:48 -0700)]
Fix missing errno assignments.

In PR #294 I removed a little too much code; we still need to assign to
`errno` in the code in question here.

2 years agoAdd braces to `if` statements whose bodies have multiple statements.
Dan Gohman [Thu, 6 Oct 2022 17:43:42 +0000 (10:43 -0700)]
Add braces to `if` statements whose bodies have multiple statements.

Add braces to `if` statements so that the whole intended body is covered
by the conditional.

2 years agothreads: implement `pthread_create` (#325)
Andrew Brown [Tue, 4 Oct 2022 14:21:18 +0000 (07:21 -0700)]
threads: implement `pthread_create` (#325)

* threads: implement `pthread_create`

As described in the [`wasi-threads`] proposal, this change implements
`pthread_create` using the new `wasi_thread_spawn(void *arg)` API. As
described there, `wasi-libc` exports the thread entry point with the
expected name, `wasi_thread_start`, and then unwraps the passed argument
`struct` to invoke the user function with the user argument `struct`.

[`wasi-threads`]: https://github.com/WebAssembly/wasi-threads/pull/5

Previously, the TID was only passed to the child thread entry point; the
parent thread was forced to wait until the child thread set the TID in
the pthread structure. With this change, the TID will be passed not only
to the child thread but also returned to the parent thread, so that
either side can make progress. The `i32.store` becomes an
`i32.atomic.store` to avoid concurrent writes.

2 years agoUse quotes instead of angle brackets.
Dan Gohman [Wed, 28 Sep 2022 21:32:52 +0000 (14:32 -0700)]
Use quotes instead of angle brackets.

2 years agoDon't run static constructors on arbitrary user exports.
Dan Gohman [Fri, 23 Sep 2022 18:46:59 +0000 (11:46 -0700)]
Don't run static constructors on arbitrary user exports.

Previously, "new-style commmands" considered every user-defined
export to be a potential command entrypoint, so wasi-libc and wasm-ld
cooperated to run the user's static constructors on each entrypoint.

This form of new-style command turned out not to be useful, and it
interferes with some use cases, so disable it.

This is done by making an explicit call to `__wasm_call_ctors`, which
tells wasm-ld that it shouldn't synthesize any calls to
`__wasm_call_ctors` on its own.

2 years agomake `__get_tp()` a static function (#327)
Marcin Kolny [Mon, 19 Sep 2022 20:53:39 +0000 (21:53 +0100)]
make `__get_tp()` a static function (#327)

I'm not sure if the function really has to be exported. If so, we should
probably move it to a separate compilation unit, otherwise it will be defined
multiple times (e.g. in `strerror.o` and `__lctrans.o`) causing linker errors.
However, I don't see a reason (at least for now) to export this function,
therefore making it static in this PR.

2 years agothreads: implement support for conditional variables (#323)
Marcin Kolny [Tue, 6 Sep 2022 21:21:06 +0000 (23:21 +0200)]
threads: implement support for conditional variables (#323)

The implementation is not as efficient as for native Linux platform due
to lack of FUTEX_REQUEUE-like system call in WASI.

For now we wake all the waiters which is inefficient; if that becomes
a bottleneck, I suggest we'll revisit the implementation.

2 years agoImplement support for pthread_rwlock (#321)
Marcin Kolny [Tue, 6 Sep 2022 17:14:07 +0000 (19:14 +0200)]
Implement support for pthread_rwlock (#321)

* threads: implement support for pthread_rwlockattr

* threads: implement support for pthread_rwlock

2 years agothreads: implement support for pthread_condattr (#320)
Marcin Kolny [Tue, 6 Sep 2022 17:13:21 +0000 (19:13 +0200)]
threads: implement support for pthread_condattr (#320)

2 years agothreads: implement support for unnamed semaphores (#316)
Andrew Brown [Mon, 22 Aug 2022 21:22:09 +0000 (14:22 -0700)]
threads: implement support for unnamed semaphores (#316)

[POSIX semaphores] come in two forms: named and unnamed. Roughly, named
semaphores use files to implement locking across processes; unnamed
semaphores use a shared memory region to implement locking across
threads in the same process. Since WASI currently has no process concept
(and it is relatively unclear how to map the WASI files as shared
memory), only the unnamed semaphores are supported by this changed. This
means that `sem_open`, `sem_close`, and `sem_unlink` will not available
to programs compiled with a threads-enabled `wasi-libc`.

[POSIX semaphores]: https://man7.org/linux/man-pages/man7/sem_overview.7.html

2 years agothreads: implement support for pthread mutexes (#315)
Andrew Brown [Mon, 22 Aug 2022 15:39:44 +0000 (08:39 -0700)]
threads: implement support for pthread mutexes (#315)

This change adds pthread's mutex support to the `THREAD_MODEL=posix`
build of wasi-libc. Some less-common features are unsupported and
documented here:
- mutex robust lists are disabled and their use will return a runtime
  error; currently WASI does not support the concept of multiple
  processes so maintaining robust mutexes across processes does not yet
  make sense in wasi-libc
- timed locks with priority inheritance (PI) are disabled and will act
  as any other mutex; this feature is related to task priorities which
  is not yet relevant in the WASI ecosystem (see [priority-inheritance
  futexes](https://man7.org/linux/man-pages/man2/futex.2.html))
- thread cancellation is ignored; this feature is difficult to support
  and @sunfishcode would likely want to discuss this before adding it at
  some later time.

2 years agoFix `make THREAD_MODEL=posix` (#311)
Andrew Brown [Tue, 9 Aug 2022 15:08:37 +0000 (08:08 -0700)]
Fix `make THREAD_MODEL=posix` (#311)

* Fixes for the THREAD_MODEL=posix build

* Fix expected symbols from previous commit

* Enable `lock` in `random.c` when threads are enabled

This uses the `_REENTRANT` definition to indicate when the `lock` should
be available.

* Disable `aio.h` when compiling for threads

In talking to @sunfishcode about `aio.h`, this functionality is not yet
a primary concern (it was already disabled in the default,
single-threaded mode). Additionally, this change adds expectation lines
for the new symbols/includes added and removed by `pthread.h`.

This change was reached by running:
```console
$ git diff --no-index expected/wasm32-wasi sysroot/share/wasm32-wasi > patch.diff
# replace "sysroot/share" with "expected" in `patch.diff`
$ git apply patch.diff --reject
# manually fix any rejections
```

* Specify the TLS model until LLVM 15 is released

The `-ftls-model` configuration can be removed once https://reviews.llvm.org/D130053 makes its way into an upstream release.

* Rename `__wasi_libc_pthread_self` to `__wasilibc_pthread_self`

The symbol is still undefined, though.

* Add different sets of expected output based on THREAD_MODEL

* Re-add trailing whitespace to `predefined-macros.txt`

@sbc100 wanted to retain the whitespace trailing after certain
predefined macro lines. This change restores that whitespace from
upstream and re-generates the POSIX version using the following command:

```console
$ git diff --no-index expected/wasm32-wasi/posix/predefined-macros.txt sysroot/share/wasm32-wasi/predefined-macros.txt | sed 's/sysroot\/share\/wasm32-wasi/expected\/wasm32-wasi\/posix/' | git apply
```

* Protect `preopens.c` against concurrent access

* Only build thread-capable wasi-libc on latest version of Clang

* Use `thrd_sleep` from MUSL instead of aliasing `nanosleep`

* Define `pthread_setcancelstate` in `THREAD_MODEL=posix` builds

There are other options here (e.g., always define the `pthread_*`
symbols with stubs) but until we discuss that this is an intermediate
working step.

* Define a Wasm global to store `pthread_self`

* Remove `g_needs_dynamic_alloc` global

* Document the state of pthread support

* review: de-duplicate symbols based on #314

* review: only define `__wasilibc_cwd_{un}lock` when needed

* review: add #ifdefs to `__pthread_setcancelstate`

* review: add additional #ifdefs to `pthread_self.c`

* review: put lock definition behind #ifdef _REENTRANT

* review: remove pthread_setcancelstate.c

* review: re-fix indentation

* review: alias __clock_nanosleep in bottom half

* review: remove extra line

Co-authored-by: Sam Clegg <sbc@chromium.org>
2 years agoSort lists of input files (#313)
Bernhard M. Wiedemann [Tue, 9 Aug 2022 04:54:37 +0000 (06:54 +0200)]
Sort lists of input files (#313)

For reproducible .a files in spite of non-deterministic
filesystem readdir order

Without this patch, find returned files in filesystem order
and llvm-ar used that order to create .a files.

2 years agoRemove duplicates from `defined-symbols.txt` (#314)
Andrew Brown [Mon, 1 Aug 2022 22:52:16 +0000 (15:52 -0700)]
Remove duplicates from `defined-symbols.txt` (#314)

In #311, it became apparent that duplicate symbol definitions were
becoming unwieldy. This change merges all duplicates using `uniq`.

2 years agoMove `weak` attribute to front (#310)
Andrew Brown [Tue, 26 Jul 2022 23:12:17 +0000 (16:12 -0700)]
Move `weak` attribute to front (#310)

This is a follow-up based on @sbc100's comments in #306.

2 years agoFix typo
Andrew Brown [Tue, 26 Jul 2022 22:57:48 +0000 (15:57 -0700)]
Fix typo

2 years agoUse MUSL's `weak*` feature in bottom half (#306)
Andrew Brown [Tue, 26 Jul 2022 21:15:12 +0000 (14:15 -0700)]
Use MUSL's `weak*` feature in bottom half (#306)

This change extracts the `weak*`-related parts of #303 as a separate PR.
Note that this is slightly strange in that it uses some top-half MUSL
headers in the bottom-half code, but discussion around this led me to
believe that the advantages of, e.g., `LOCK` made this worthwhile.
Beyond just changing uses of `weak` to `__weak__`, we also MUSL's `weak`
and `weak_alias` macros in a few more places.

2 years agoDelete several blocks of unused code. (#294)
Dan Gohman [Wed, 20 Jul 2022 22:46:06 +0000 (15:46 -0700)]
Delete several blocks of unused code. (#294)

* Delete several blocks of unused code.

Delete several pieces of code from libc-bottom-half/cloudlibc that aren't in
use on wasi-libc.

* Delete more of `_CLOCK_PROCESS_CPUTIME_ID` or `_CLOCK_THREAD_CPUTIME_ID`.

2 years agoUpdate README and add CI-tests for minimal supported LLVM-version (10) (#302)
msirringhaus [Wed, 13 Jul 2022 15:49:17 +0000 (17:49 +0200)]
Update README and add CI-tests for minimal supported LLVM-version (10) (#302)

Fixes: #301
2 years agoAdd a `getpagesize` function. (#300)
Dan Gohman [Mon, 11 Jul 2022 15:27:39 +0000 (08:27 -0700)]
Add a `getpagesize` function. (#300)

* Add a `getpagesize` function.

This adds a `getpagesize` function. This interface is deprecated in POSIX,
but it's sufficiently widely used and not problematic in practice.

* Use musl's `getpagesize`.

* Enable the `getpagesize` declaration in unistd.h.

2 years agoFix `gettimeofday` to correctly handle a null argument.
Dan Gohman [Wed, 25 May 2022 03:55:41 +0000 (20:55 -0700)]
Fix `gettimeofday` to correctly handle a null argument.

`gettimeofday` is defined to do nothing if passed NULL.

2 years agoRemove support for `__original_main`. (#295)
Dan Gohman [Wed, 1 Jun 2022 15:51:20 +0000 (08:51 -0700)]
Remove support for `__original_main`. (#295)

* Remove support for `__original_main`.

This was used in old LLVM versions. Recent LLVM versions all emit either
`__main_void` or `__main_argv_argc`, so we can use those directly.

* Update the CI to use LLVM 14.0.0.

2 years agoUpdate to musl 1.2.3.
Dan Gohman [Wed, 25 May 2022 02:31:23 +0000 (19:31 -0700)]
Update to musl 1.2.3.

* Update to musl 1.2.3.

See the WHATSNEW file for details.

2 years agoFix utimensat to avoid passing uninitialized values into WASI calls.
Dan Gohman [Tue, 24 May 2022 20:56:33 +0000 (13:56 -0700)]
Fix utimensat to avoid passing uninitialized values into WASI calls.

Previously, utimensat would leave the mtim and/or atim timestamps
uninitialized when the `MTIM_NOW` or `ATIM_NOW` were in use, because
that means the respective timestamps are not used.

However, clang now automatically adds `noundef` to the arguments in
functions like `__wasi_path_filestat_set_times`, and there are cases
where simplifycfg can see paths where the uninitialized values are
passed to those `noundef` arguments.

To fix this, change the utimens code to zero out the timestamps when
they aren't in use, to avoid passing uninitialized arguments.

2 years agoImplement BULK_MEMORY_THRESHOLD
Cheng Shao [Wed, 18 May 2022 19:19:27 +0000 (19:19 +0000)]
Implement BULK_MEMORY_THRESHOLD

2 years agoAdd TODO pointing to LLVM bug report
Cheng Shao [Sat, 11 Dec 2021 15:04:49 +0000 (15:04 +0000)]
Add TODO pointing to LLVM bug report

2 years agoOnly apply -mbulk-memory to specific files
Cheng Shao [Sat, 4 Dec 2021 05:52:11 +0000 (05:52 +0000)]
Only apply -mbulk-memory to specific files

2 years agoDon't use __builtin version of wmem*
Cheng Shao [Sat, 4 Dec 2021 04:27:20 +0000 (04:27 +0000)]
Don't use __builtin version of wmem*

2 years agoUse bulk memory opcodes when possible
Cheng Shao [Sat, 4 Dec 2021 03:56:45 +0000 (03:56 +0000)]
Use bulk memory opcodes when possible

2 years agoFix test artifacts
Cheng Shao [Sat, 4 Dec 2021 03:33:42 +0000 (03:33 +0000)]
Fix test artifacts

2 years ago-mbulk-memory go brrr
Cheng Shao [Sat, 4 Dec 2021 03:13:34 +0000 (03:13 +0000)]
-mbulk-memory go brrr

2 years agoFix `find_relpath` to handle large buffers correctly.
Dan Gohman [Tue, 10 May 2022 01:16:05 +0000 (18:16 -0700)]
Fix `find_relpath` to handle large buffers correctly.

As a follow-up to #247, fix `find_relpath` to handle large buffers
correctly.

Co-Authored-by: Yuta Saito <kateinoigakukun@gmail.com>
2 years agoExclude C++ headers from the generated include-all.c program.
Dan Gohman [Fri, 6 May 2022 01:04:14 +0000 (18:04 -0700)]
Exclude C++ headers from the generated include-all.c program.

The order that things happen in appears to have changed, and the
include-all.c script is now generated at a time when there can be
C++ headers in the sysroot, so adjust the script to exclude C++
headers.

2 years agoOverride Make's default for CC and AR.
Mike Hommey [Thu, 5 May 2022 05:06:22 +0000 (14:06 +0900)]
Override Make's default for CC and AR.

2 years agofeat: add support for accept and accept4
Harald Hoyer [Mon, 2 May 2022 10:21:42 +0000 (12:21 +0200)]
feat: add support for accept and accept4

Since the socket address of the accepted socket is unknown,
all bytes are set to zero and the length is truncated to the size
of the generic `struct sockaddr`.

Signed-off-by: Harald Hoyer <harald@profian.com>