!buildbot wasi

🤖 New build scheduled with the buildbot fleet by@vstinner for commit2736f25 🤖

The command will test the builders whose names match following regular expression:wasi

The builders matched are:

• wasm32-wasi PR
• wasm32-wasi Non-Debug PR
• wasm32 WASI 8Core PR

The 8 MiB limit (8 388 608 bytes) is already used by thewasmtime command:

test.pythoninfo:

sysconfig[HOSTRUNNER]: wasmtime run --wasm max-wasm-stack=8388608 --wasi preview2 --env PYTHONPATH=/$(shell realpath --relative-to /opt/buildbot/kushaldas-wasm/3.x.kushaldas-wasi.wasi.nondebug/build/build_oot/host/../.. /opt/buildbot/kushaldas-wasm/3.x.kushaldas-wasi.wasi.nondebug/build/build_oot/host)/$(shell cat pybuilddir.txt):/Lib --dir ../..::/

The value is set at:

$ grep 8388608 Tools/wasm/ -RTools/wasm/wasi.py:                        # The 8388608 value comes from `ulimit -s` under LinuxTools/wasm/wasi.py:                        "--wasm max-wasm-stack=8388608 "Tools/wasm/wasm_build.py:            "--wasm max-wasm-stack=8388608 "

I'm not sure of the difference between the linker-z stack-size=8388608 option andwasmtime optionmax-wasm-stack=8388608. According to the issue, the stack doesn't seem to grow from 512 KiB to 8 MiB automatically:#117645 (comment)

With this change, test_dynamic works as expected (on a release build) with 8 MiB stack.

@brettcannon: Would you mind to review this change? Are you ok to increase the initial stack from 512 KiB to 8 MiB?

test_dynamic pass again on "buildbot/wasm32 WASI 8Core PR" (built in release mode). Previously, test_dynamic was failing on this buildbot.

Note: I also tried toleave the initial stack unchanged (512 KiB), and only increase the memory size (to 20 MiB):-z stack-size=524288 -Wl,--stack-first -Wl,--initial-memory=20971520.test_dynamic fails in this case (release build). We need to increase the stack size in the linker if we want test_dynamic to pass: what this PR does.

I'm not convinced that wasmtime can grow the stack and/or the total memory size. So I changed the configure.ac comment.

I think this definitely needs a WASI expert. It looks a bit like tuning generic build settings for the specific buildbot.
@ericsnowcurrently, do you want to take a look?

I did another test:

• 512 KiB initial stack
• 40 MiB initial memory

test_dynamicfails with:

    2: memory fault at wasm address 0x1000005b0 in linear memory of size 0x2800000    3: wasm trap: out of bounds memory access

It confirms that thewasmtimecannot grow the stack: we have to set it to 8 MiB directly.

But it can grow the "total"memory (address space). Example on a build with 40 MiB initial memory:

# cross-build/wasm32-wasi/python.sh Python 3.13.0a5+ (heads/wasi_stack-dirty:a69ffa75eb, Apr 15 2024, 13:36:08) [Clang 16.0.0 ] on wasiType "help", "copyright", "credits" or "license" for more information.>>> x=b'x' * (1024 * 1024 * 60)>>> len(x)62914560

I can allocate 60 MiB without any issue.

It seems like nobody is available to review this change. I plan tomerge my PR at the end of the week, unless someone comes and wants to review it :-)

The change reenables a test_dynamic test which is currently always skipped on WASI. IMO it will unlock other use cases which are currently limited by the maximum stack memory.

IMO a stack of 512 KiB is too small for Python, and sadly--wasm max-wasm-stack=8388608 option towasmtime doesn't work: it fails to grow the stack.

Merged, thanks for the review@ericsnowcurrently.