feat(QA-200): Automated pythonindex package tests #2
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
baalimago
force-pushed
the
qa-209-convert-build-scripts-into-a-python-test-project
branch
from
503999d to
26265fb
Compare
baalimago
commented
Sep 17, 2025
baalimago
commented
Sep 17, 2025
baalimago
force-pushed
the
qa-209-convert-build-scripts-into-a-python-test-project
branch
from
304df25 to
a943e0c
Compare
Also added stuff so that the pipeline is triggered without deploying, also on this branch so that I can debug the flow
baalimago
force-pushed
the
qa-209-convert-build-scripts-into-a-python-test-project
branch
from
de01f92 to
79b03a3
Compare
baalimago
force-pushed
the
qa-209-convert-build-scripts-into-a-python-test-project
branch
from
b987678 to
232ca57
Compare
A bit liberal in that commit summary, but it's not entirely wrong. We don't yet have support for pytest, which was the original plan. So instead I've re-engineered Lennart's solution, but in python (since I couldn't get his solution to work). What we get is a python script which runs all the tests found in ./tests (or as specified as TEST_DIR). It also tracks success/failures, so it does the same things as pytest, but bare minimum for our usecase.
baalimago
force-pushed
the
qa-209-convert-build-scripts-into-a-python-test-project
branch
from
232ca57 to
ecba365
Compare
This way we can call the checks whenever we want. And hopefully shipit can do whatever magic it does to make it #justwork when deployed to edge
Although the <api>/check endpoint worked just fine locally, it times out the workload on edge. So we need to be able to call each test one by one. So in addition to this, I added a <api>/list endpoint which lists all tests, then a /check/<test> endpoint which calls a singular test. This allows us to run just a specific test, if need be. Also added instructions on how this works to README.md
baalimago
force-pushed
the
qa-209-convert-build-scripts-into-a-python-test-project
branch
from
729addf to
24467a3
Compare
zebreus
requested changes
Sep 18, 2025
Comment on lines
+9
to
+17
| def _has_network() -> bool: | ||
| try: | ||
| socket.getaddrinfo("httpbin.org", 443) | ||
| return True | ||
| except Exception: | ||
| return False | ||
|
|
||
|
|
||
| HAS_NET = _has_network() |
There was a problem hiding this comment.
Hmm, I noticed that it only works with pytest, which we don't support (yet!). I wrote this when I had the intent of using pytest, instead of needing to write my own version
So it always fails on this, and requests aren't tested. But perhaps it's fine?
|
|
||
| Run tests: | ||
|
|
||
| - `uv pip compile pyproject.toml --python-version=3.13 --universal --extra-index-url https://pythonindex.wasix.org/simple --index-url=https://pypi.org/simple --emit-index-url --only-binary :all: -o wasmer-requirements.txt` |
There was a problem hiding this comment.
I think you may be able to convert the test script to a PEP 723 script with URL based specifiers. That way we should be able to just uv run the tests.
There was a problem hiding this comment.
I don't really know what that means or how it would work. These preconditionals are just copy-pasted from what Syrus had in the fastapi template.
I vote for out of scope/future improvement!
Co-authored-by: Zebreus <zebreus@zebre.us>
baalimago
force-pushed
the
qa-209-convert-build-scripts-into-a-python-test-project
branch
from
3ec07f7 to
be1323e
Compare
baalimago
force-pushed
the
qa-209-convert-build-scripts-into-a-python-test-project
branch
from
be1323e to
953ffe4
Compare
baalimago
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR will setup testing for all the python native packages which have a test defined.
Read the README section! TL;DR:
./testing/run-all-tests-via-api.py --host https://build-scripts.wasmer.<env> --port 443to run all tests on edge