- Notifications
You must be signed in to change notification settings - Fork27
Add e2e tests#331
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
base:staging
Are you sure you want to change the base?
Add e2e tests#331
Uh oh!
There was an error while loading.Please reload this page.
Conversation
Initialize Playwright e2e test architecture
overcut-aibot commentedDec 10, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
Completed Working on "Code Review"✅ Review submitted: REQUEST_CHANGES. Total comments: 14 across 7 files. |
github-actionsbot commentedDec 10, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.OpenSSF ScorecardScorecard details
Scanned Files
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the ✨ Finishing touches🧪 Generate unit tests (beta)
Thanks for usingCodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Blocking issues remain: the CI workflow never boots the QueryWeaver server before running Playwright, the new API helpers leak/discard authentication contexts, and the graph upload helper never actually sends file contents. Please address these along with the supporting fixes noted inline.
| run:npm ci | ||
| -name:Install Playwright Browsers | ||
| run:npx playwright install --with-deps | ||
| -name:Run Playwright tests |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
[blocker]: Playwright CI never launches backend. The Playwright workflow installs dependencies and immediately runsnpx playwright test, but it neither starts the QueryWeaver backend/frontend nor setsPLAYWRIGHT_BASE_URL, so every navigation/API helper will hit an empty localhost port and time out on CI. Add a step (or PlaywrightwebServer config) that starts the API/frontend, waits for it to become reachable, and exportsPLAYWRIGHT_BASE_URL before the tests run.
| /* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */ | ||
| use:{ | ||
| /* Base URL to use in actions like `await page.goto('')`. */ | ||
| baseURL:process.env.BASE_URL||'http://localhost:5000', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
[major]: Base URL left undefined.use.baseURL is commented out, so helpers likepage.goto('/') guess the host and CI cannot override it. Without a deterministic base URL the suite cannot talk to either localhost or staging. Setuse: { baseURL: process.env.PLAYWRIGHT_BASE_URL ?? 'http://localhost:5000', ... } (and pass the env in CI) so tests share a consistent origin.
| }, | ||
| "devDependencies": { | ||
| "@playwright/test":"^1.57.0", | ||
| "@types/node":"^24.10.2", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
[major]: Node typings out of sync with runtime.@types/node was bumped to ^24.x while Docker/CI still execute Node 22, so TypeScript will now allow APIs that crash under the shipped runtime when running the Playwright tooling. Either upgrade every runtime to Node 24 or pin@types/node to the 22.x series so type definitions match the deployed Node version.
e2e/infra/ui/browserWrapper.ts Outdated
| } | ||
| } | ||
| asynccloseBrowser(){ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
[major]: Wrapper reuses handles after closing.closeBrowser() closes the page/browser but leavesthis.browser andthis.context populated, so the nextcreateNewPage() reuses handles Playwright has already torn down and throws target-closed errors. After closing, dispose the context and set boththis.browser andthis.context to null (or recreate them) so later calls relaunch cleanly.
| import{APIRequestContext,request}from"@playwright/test" | ||
| constgetRequest=async(url:string,headers?:Record<string,string>,body?:any,availableRequest?:APIRequestContext)=>{ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
[major]: API request contexts leak. Each helper creates arequest.newContext() when one isn’t supplied but never disposes it, so repeated calls leak HTTP sessions/processes and eventually hang the runner. Track whether the helper created the context (e.g.,const shouldDispose = !availableRequest) andawait requestContext.dispose() in a finally block when true.
| }, | ||
| "devDependencies": { | ||
| "@playwright/test":"^1.57.0", | ||
| "@types/node":"^24.10.2", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
[major]@types/node is bumped to 24.10 here while the project runtime is still Node 22 (see Dockerfile’ssetup_22.x install and the workflow’snode-version: lts/*). TypeScript will now permit APIs that only exist in Node 24, which will crash when the Playwright tooling runs under Node 22 inside Docker/CI. Please either keep the types on the 22.x line or upgrade the runtime everywhere so the type definitions match the actual Node version.
e2e/infra/ui/browserWrapper.ts Outdated
| } | ||
| } | ||
| asynccloseBrowser(){ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
[major]closeBrowser() only closes the page and then the raw browser handle, but it never closes the activeBrowserContext or resetthis.browser/this.context tonull. After one test finishes those fields keep pointing at Playwright objects that have already been torn down, so the nextcreateNewPage will try to reuse a dead context and throwbrowserContext.newPage: Protocol error: Target closed. Pleaseawait this.context?.close() and null the fields (browser/context/page) when shutting down so a subsequent run can relaunch cleanly.
| headers:headers||undefined, | ||
| }; | ||
| constrequestContext=availableRequest||(awaitrequest.newContext()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
[major] Each helper here spins up a freshAPIRequestContext when one isn’t provided, but none of them ever dispose it. In a longer suite these stacks of orphaned contexts leak HTTP sessions/processes until Playwright hangs or hits the browser limit. Please track whether the helper created the context (e.g.const shouldDispose = !availableRequest) and callawait requestContext.dispose() in afinally block when appropriate so we don’t leak resources between API calls.
| headers:headers||undefined, | ||
| }; | ||
| constrequestContext=awaitrequest.newContext(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
[major] Unlike the GET/POST/PATCH helpers,deleteRequest always creates a brand‑newAPIRequestContext, so callers such asdeleteGraph/deleteToken drop the authenticated session they just established and hit the endpoint without cookies/headers. Please accept the optionalavailableRequest parameter (mirroring the other helpers) and reuse it so DELETE calls keep the same auth context as the rest of the API interactions.
e2e/logic/api/apiCalls.ts Outdated
| ):Promise<GraphUploadResponse>{ | ||
| try{ | ||
| constbaseUrl=getBaseUrl(); | ||
| constformData:Record<string,string>={ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
[major]uploadGraph builds a plain object{ file: filePath } and then forcesContent-Type: multipart/form-data. Playwright will serialize that object as JSON ({"file":"/tmp/foo.json"}) and the backend never receives any file bytes or multipart boundary, so uploads will always fail. Use Playwright’s multipart support instead—e.g.requestContext.post(url, { multipart: { file: await fs.createReadStream(filePath), database, description } })—and let Playwright set the header so the graph file actually reaches the server.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Thanks for the comprehensive E2E scaffolding work. I left 14 comments (2 blockers, 12 majors) covering three critical areas:
- CI workflow never boots the backend/API service, so the newly added Playwright suite will always run against an empty port and fail. Please ensure the workflow provisions/starts the backend before executing the browser and API tests.
- The Playwright configuration/UI helpers rely on
page.goto()/API calls with an undefined base URL and leak Playwright resources (browser contexts + APIRequestContext instances). We need a guaranteed base URL (wired through CI) and proper teardown/reuse of contexts so runs do not leak and hang. - Test-only dependencies are now part of the runtime package (e.g.,
playwrightunderdependencies), inflating production bundles, while API helpers mishandle authentication for DELETE/uploads. Move the heavy tooling back to devDependencies and address the helper issues called out inline so E2E runs can actually exercise the app.
Once those blockers and the associated comments are addressed, I’ll re-review. Thanks!
| @@ -1,8 +1,12 @@ | |||
| { | |||
| "dependencies": { | |||
| "playwright":"^1.57.0", | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
[major]playwright is added underdependencies, which means we now ship the browser binaries to every production install/deployment even though they are only needed for local/CI testing. This roughly gigabyte-sized dependency bloats Lambda/container images, slows prod deploys, and is unused at runtime. Please moveplaywright (and its transitive install) todevDependencies so only development/test environments pull it in.
| timeout-minutes:60 | ||
| runs-on:ubuntu-latest | ||
| steps: | ||
| -uses:actions/checkout@v4 | ||
| # Setup Python | ||
| -name:Set up Python | ||
| uses:actions/setup-python@v5 | ||
| with: | ||
| python-version:${{ env.PYTHON_VERSION }} | ||
| # Setup Node.js | ||
| -uses:actions/setup-node@v4 | ||
| with: | ||
| node-version:${{ env.NODE_VERSION }} | ||
| # Install pipenv | ||
| -name:Install pipenv | ||
| run:pip install pipenv | ||
| # Install Python dependencies | ||
| -name:Install Python dependencies | ||
| run:pipenv sync --dev | ||
| # Install Node dependencies (frontend) | ||
| -name:Install frontend dependencies | ||
| run:npm ci | ||
| # Build frontend | ||
| -name:Build frontend | ||
| run:npm run build | ||
| working-directory:./app | ||
| # Start Docker Compose services (test databases + FalkorDB) | ||
| -name:Start test databases | ||
| run:| | ||
| docker compose -f e2e/docker-compose.test.yml up -d | ||
| # Wait for databases to be healthy | ||
| echo "Waiting for databases to be ready..." | ||
| sleep 10 | ||
| docker ps -a | ||
| # Start FalkorDB (Redis graph database) | ||
| -name:Start FalkorDB | ||
| run:| | ||
| docker run -d --name falkordb-test -p 6379:6379 falkordb/falkordb:latest | ||
| sleep 5 | ||
| docker ps -a | ||
| # Start the FastAPI application in background | ||
| -name:Start FastAPI application | ||
| run:| | ||
| pipenv run uvicorn api.index:app --host localhost --port 5000 & | ||
| # Wait for app to start | ||
| echo "Waiting for application to start..." | ||
| sleep 10 | ||
| curl -f http://localhost:5000/ || (echo "Failed to start application" && exit 1) | ||
| env: | ||
| PYTHONUNBUFFERED:1 | ||
| FASTAPI_SECRET_KEY:test-secret-key-for-ci | ||
| APP_ENV:development | ||
| FASTAPI_DEBUG:False | ||
| FALKORDB_HOST:localhost | ||
| FALKORDB_PORT:6379 | ||
| DISABLE_MCP:true | ||
| # Install Playwright browsers | ||
| -name:Install Playwright Browsers | ||
| run:npx playwright install --with-deps chromium firefox | ||
| # Run Playwright tests | ||
| -name:Run Playwright tests | ||
| run:npx playwright test --reporter=list | ||
| env: | ||
| CI:true | ||
| # Upload test results on failure | ||
| -uses:actions/upload-artifact@v4 | ||
| if:failure() | ||
| with: | ||
| name:playwright-report | ||
| path:playwright-report/ | ||
| retention-days:30 | ||
| # Upload test screenshots on failure | ||
| -uses:actions/upload-artifact@v4 | ||
| if:failure() | ||
| with: | ||
| name:test-results | ||
| path:test-results/ | ||
| retention-days:30 | ||
| # Cleanup - Stop all containers | ||
| -name:Cleanup Docker containers | ||
| if:always() | ||
| run:| | ||
| docker compose -f e2e/docker-compose.test.yml down -v ||true | ||
| docker stop falkordb-test ||true | ||
| docker rm falkordb-test ||true |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium
Show autofix suggestionHide autofix suggestion
Copilot Autofix
AI 1 day ago
To fix the problem, add an explicitpermissions block to the workflow, restricting the GITHUB_TOKEN to the minimum necessary privileges. Since this workflow does not perform any modifications to the repository contents or interact with issues or pull requests, the minimal permissioncontents: read is sufficient. The best place to add this is at the top level of the workflow (after or beforeenv:), so it applies to all jobs unless overridden. Therefore, add:
permissions:contents:read
Adding this at the root guarantees that jobs cannot escalate privileges unexpectedly, and no permissions are granted for writing, merging PRs, or modifying issues. No further code changes or new imports are needed.
| @@ -5,6 +5,9 @@ | ||
| pull_request: | ||
| branches:[ main, staging ] | ||
| permissions: | ||
| contents:read | ||
| env: | ||
| PYTHON_VERSION:'3.12' | ||
| NODE_VERSION:'lts/*' |
No description provided.