Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

fix: resolve dotenv loading in examples and tests#1155

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Open
shamsharoon wants to merge41 commits intobrowserbase:main
base:main
Choose a base branch
Loading
fromshamsharoon:fix/dotenv-loading-and-update-tests

Conversation

@shamsharoon
Copy link

@shamsharoonshamsharoon commentedOct 22, 2025
edited
Loading

why

Examples were failing with "OpenAI API key is missing" errors despite havingOPENAI_API_KEY in the.env file. The issue was thatdotenv.config() instagehand.config.ts was looking for.env in the current working directory (process.cwd()), which pointed toexamples/ when running viapnpm run example, not the project root where.env actually exists.

Related Issue:dotenv peerDependency

what changed

  • stagehand.config.ts: Updateddotenv.config() to usepath.resolve(__dirname, ".env") instead of relying onprocess.cwd(). This ensures.env is loaded relative to the config file's location, working correctly whether running from source or compiled code.

  • examples/2048.ts: Updated to import and useStagehandConfig instead of hardcoding configuration, aligning with other examples in the repo.

  • evals/deterministic/tests/Errors/apiKeyError.test.ts: Fixed tests that verify error handling when API keys are missing:

    • AddedbeforeAll/afterAll hooks to temporarily clear and restoreOPENAI_API_KEY
    • Added config overrides (llmClient: undefined,modelClientOptions: undefined) to properly test error conditions
    • Ensures tests don't interfere with other test files

test plan

  • ✅ Ranpnpm run example 2048 - example executes successfully without API key errors
  • ✅ Ranpnpm run e2e:local - all 43 tests pass
  • ✅ Specifically testedapiKeyError.test.ts - all 3 tests pass and correctly throw expected errors
  • ✅ VerifiedafterAll hook restores environment for subsequent test files

greptile-apps[bot] reacted with thumbs up emoji
tkattkatand others added30 commitsSeptember 10, 2025 13:40
# whysolvesbrowserbase#1060 patch regression of playwright arguments being removed from agentexecute response# what changedagent.execute now returns playwright arguments in its response # test plantested locally
…ms to docs (browserbase#1065)# whyreflect project id changes in docs# what changedadvanced configuration comments# test planreviewed via mintlify on localhost
# whyEasier to use for Custom LLM Clients and keep users up to date with ouraisdk file# what changedadded export of aisdk to lib/index.ts# test planbuild local stagehand, import local AISdkClient, run Azure Stagehandsession
…onfigu… (browserbase#1073)…ration settings# whyUpdated docs to match the new fingerprint params in the Browserbase docshere:https://docs.browserbase.com/guides/stealth-customization#customization-options# what changedUpdate browser configuration docs to reflect the docs changes. # test plan
# whyUpdating docs to reflect aisdk can be imported directly# what changedThe model page# test planReviewed page with mintlify dev locally
# whyCurrently, we do not support stagehand agent within the api# what changedWhen api is enabled, stagehand agent now routes through the api # test planTested locally
# whyCurrently, using playwright screenshot command is not available when theexecution environment is Stagehand. A customer has indicated they wouldprefer to use Playwright's native screenshot command instead of CDP whenusing Browserbase as CDP screenshot causes unexpected behavior for theirtarget site.# what changed- added a StagehandScreenshotOptions type with useCDP argument added- extended page type to accept custom stagehand screeenshot options- update screenshot proxy to default useCDP to true if the env isbrowserbase and use playwright screenshot if false- added eval for screenshot with and without cdp# test plan- tested and confirmed functionality with eval and external examplescript (not committed)
…rowserbase#1057)# whyWe want to build a best in class agent in stagehand.Therefore, we need more eval benchmarks.# what changed- Added Web-bench evals dataset- Added a subset of OS World evals - those that can be run in a chromebrowser (desktop-based tasks omitted)- added LICENSE noticed to the copied evals tasks- Added ground truth / expected result to some WebVoyager tasks usingreference_answer.json from Browser Use public evals repo.Improvements to `pnpm run evals -man` to better describe how to runevals.# test planEvals should run locally and bb for these new benchmarks.
# whyInitial instructions didn't mention uv or pip prerequisites and alsodidn't mention venv. Fix reduces friction on first timers.# what changed- added link to install uv- added details for initializing venv- adjusted code example respectively # test plandocs change
# why- webpage structure changed, needed to update the xpath in the expectedlocator
… with LanguageModelV1 + LiteLLM works for python (browserbase#1086)# why1. aisdk not yet available through npm package2. customLLM provider only works with LanguageModelV13. LiteLLM compatible providers are supported in python# what changed1. change docs to install stagehand from git repo2. pin versions that use LanguageModelV1# test planlocal test
# whycurrently we pass stagehand page to agent, this results in our pagemanagement having issues when facing new tabs# what changedthe stagehand object is now passed instead of stagehandPage# test plantested locally
# whyOur existing screenshot service is a dummy time-based triggered service.It also does not trigger based on any actions of the agent.# what changedAdded img hash diff algo (quick check with MSE, verify with SSIM algo)to see if there was an actual UI change and only store ss in the bufferif that is so.Added ss interceptor which copies each screenshot the agent is taking toa buffer (if different enough from the previous ss) to be later used forevals.- There's also a small refactor of the agent initialization config toenable the screenshot collector service to be attached# test planTests pass locally---------Co-authored-by: Miguel <36487034+miguelg719@users.noreply.github.com>Co-authored-by: miguel <miguelg71921@gmail.com>
# whyTo help make sense of eval test cases and results# what changedAdded metadata to eval runs, cleaned deprecated code# test plan
# why# what changed# test plan
# whyanthropic released a new sota computer use model# what changedadded claude-sonnet-4-5-20250929 as a model to the list# test planran evals
…ase#1103)WhyCustom AI SDK tools and MCP integrations weren't working properly withAnthropic CUA - parameters were empty {} and tools weren't tracked.What Changed- Convert Zod schemas to JSON Schema before sending to Anthropic (usingzodToJsonSchema)- Track custom tool calls in the actions array- Silence "Unknown tool name" warnings for custom toolsTest PlanTested with examples file. Parameters passed correctly ({"city":"San Francisco"} instead of {})Custom tools execute and appear in actions arrayNo warnings
# whyTo improve context# what changedAdded current page and url to the system prompt# test plan
# whyTo inform the user throughout the agent execution process# what changedAdded logs to tool calls, and on the stagehand agent handler# test plan- [x] tested locally
PR to make clearer the dependencies for `extract` (for those who haven'tused zod or pydantic before)---------Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
# why- before this change, when we convert `z.string().url()` to an ID, if itwas inside a `z.array()`, it was not getting converted back into a URL- this meant that if you defined a schema like this:```tsschema: z.object({  records: z.array(z.string().url()),})```you would receive an array like this:```{  records: [    '0-302', '0-309',    '0-316', '0-323',    '0-330', '0-337',    '0-344', '0-351',    '0-358', '0-365'  ]}```- with this change, you will now receive the actual URLs, ie:```{  records: [    'https://www.archives.gov/files/research/jfk/releases/2025/0318/104-10003-10041.pdf',    'https://www.archives.gov/files/research/jfk/releases/2025/0318/104-10004-10143%20(C06932208).pdf',    'https://www.archives.gov/files/research/jfk/releases/2025/0318/104-10004-10143.pdf',    'https://www.archives.gov/files/research/jfk/releases/2025/0318/104-10004-10156.pdf',    'https://www.archives.gov/files/research/jfk/releases/2025/0318/104-10004-10213.pdf',    'https://www.archives.gov/files/research/jfk/releases/2025/0318/104-10005-10321.pdf',    'https://www.archives.gov/files/research/jfk/releases/2025/0318/104-10006-10247.pdf',    'https://www.archives.gov/files/research/jfk/releases/2025/0318/104-10007-10345.pdf',    'https://www.archives.gov/files/research/jfk/releases/2025/0318/104-10009-10021.pdf',    'https://www.archives.gov/files/research/jfk/releases/2025/0318/104-10009-10222.pdf'  ]}```# what changed- updated the `injectUrls` function so that when it hits an array andthere is not deeper path, it loops through the array and injects theURLs# test plan- evals
# whyAdding support for Gemini's new Computer Use model# what changedWe partnered with Google Deepmind to help integrate and test their newComputer Use models.<img width="1238" height="655" alt="Screenshot 2025-10-07 at 1 14 44 PM"src="https://github.com/user-attachments/assets/af0d854a-8e55-4937-a071-10335497f686"/>The new model tag `gemini-2.5-pro-computer-use-preview-10-2025` isavailable for Stagehand Agent. You can try it today with the example`cua-example.ts`To learn more, check out the blog post[https://www.browserbase.com/blog/evaluating-browser-agents](https://www.browserbase.com/blog/evaluating-browser-agents)---------Co-authored-by: tkattkat <tkat@tkat.net>Co-authored-by: Kylejeong2 <kylejeong21@gmail.com>Co-authored-by: Sameel <sameel.m.arif@gmail.com>
# why# what changed# test plan
This PR was opened by the [Changesetsrelease](https://github.com/changesets/action) GitHub action. Whenyou're ready to do a release, you can merge this and the packages willbe published to npm automatically. If you're not ready to do a releaseyet, that's fine, whenever you add more changesets to main, this PR willbe updated.# Releases## @browserbasehq/stagehand@2.5.1### Patch Changes- [browserbase#1082](browserbase#1082)[`8c0fd01`](browserbase@8c0fd01)Thanks [@tkattkat](https://github.com/tkattkat)! - Pass stagehand objectto agent instead of stagehand page- [browserbase#1104](browserbase#1104)[`a1ad06c`](browserbase@a1ad06c)Thanks [@miguelg719](https://github.com/miguelg719)! - Fix logging forstagehand agent- [browserbase#1066](browserbase#1066)[`9daa584`](browserbase@9daa584)Thanks [@tkattkat](https://github.com/tkattkat)! - Add playwrightarguments to agent execute response- [browserbase#1077](browserbase#1077)[`7f38b3a`](browserbase@7f38b3a)Thanks [@tkattkat](https://github.com/tkattkat)! - adds support forstagehand agent in the api- [browserbase#1032](browserbase#1032)[`bf2d0e7`](browserbase@bf2d0e7)Thanks [@miguelg719](https://github.com/miguelg719)! - Fix for zod peerdependency support- [browserbase#1014](browserbase#1014)[`6966201`](browserbase@6966201)Thanks [@tkattkat](https://github.com/tkattkat)! - Replace operatorhandler with base of new agent- [browserbase#1089](browserbase#1089)[`536f366`](browserbase@536f366)Thanks [@miguelg719](https://github.com/miguelg719)! - Fixed info logson api session create- [browserbase#1103](browserbase#1103)[`889cb6c`](browserbase@889cb6c)Thanks [@tkattkat](https://github.com/tkattkat)! - patch custom toolsupport in anthropic cua client- [browserbase#1056](browserbase#1056)[`6a002b2`](browserbase@6a002b2)Thanks [@chrisreadsf](https://github.com/chrisreadsf)! - remove need forduplicate project id if already passed to Stagehand- [browserbase#1090](browserbase#1090)[`8ff5c5a`](browserbase@8ff5c5a)Thanks [@miguelg719](https://github.com/miguelg719)! - Improve failedact error logs- [browserbase#1014](browserbase#1014)[`6966201`](browserbase@6966201)Thanks [@tkattkat](https://github.com/tkattkat)! - replace operatoragent with scaffold for new stagehand agent- [browserbase#1107](browserbase#1107)[`3ccf335`](browserbase@3ccf335)Thanks [@seanmcguire12](https://github.com/seanmcguire12)! - fix: urlextraction not working inside an array- [browserbase#1102](browserbase#1102)[`a99aa48`](browserbase@a99aa48)Thanks [@miguelg719](https://github.com/miguelg719)! - Add current pageand date context to agent- [browserbase#1110](browserbase#1110)[`dda52f1`](browserbase@dda52f1)Thanks [@miguelg719](https://github.com/miguelg719)! - Add support fornew Gemini Computer Use models## @browserbasehq/stagehand-evals@1.1.0### Minor Changes- [browserbase#1057](browserbase#1057)[`b7be89e`](browserbase@b7be89e)Thanks [@filip-michalsky](https://github.com/filip-michalsky)! - addedweb voyager ground truth (optional), added web bench, and subset ofOSWorld evals which run on a browser### Patch Changes- [browserbase#1072](browserbase#1072)[`dc2d420`](browserbase@dc2d420)Thanks [@filip-michalsky](https://github.com/filip-michalsky)! - improveevals screenshot service - add img hashing diff to add screenshots andchange to screenshot intercepts from the agent- Updated dependencies\[[`8c0fd01`](browserbase@8c0fd01),[`a1ad06c`](browserbase@a1ad06c),[`9daa584`](browserbase@9daa584),[`7f38b3a`](browserbase@7f38b3a),[`bf2d0e7`](browserbase@bf2d0e7),[`6966201`](browserbase@6966201),[`536f366`](browserbase@536f366),[`889cb6c`](browserbase@889cb6c),[`6a002b2`](browserbase@6a002b2),[`8ff5c5a`](browserbase@8ff5c5a),[`6966201`](browserbase@6966201),[`3ccf335`](browserbase@3ccf335),[`a99aa48`](browserbase@a99aa48),[`dda52f1`](browserbase@dda52f1)]:    -   @browserbasehq/stagehand@2.5.1## @browserbasehq/stagehand-examples@1.0.10### Patch Changes- Updated dependencies\[[`8c0fd01`](browserbase@8c0fd01),[`a1ad06c`](browserbase@a1ad06c),[`9daa584`](browserbase@9daa584),[`7f38b3a`](browserbase@7f38b3a),[`bf2d0e7`](browserbase@bf2d0e7),[`6966201`](browserbase@6966201),[`536f366`](browserbase@536f366),[`889cb6c`](browserbase@889cb6c),[`6a002b2`](browserbase@6a002b2),[`8ff5c5a`](browserbase@8ff5c5a),[`6966201`](browserbase@6966201),[`3ccf335`](browserbase@3ccf335),[`a99aa48`](browserbase@a99aa48),[`dda52f1`](browserbase@dda52f1)]:    -   @browserbasehq/stagehand@2.5.1Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
# whyThe original example used JavaScript destructuring syntax [table] whichdoesn't work in Python. Fixed to use proper Python array indexing.# what changedfixed example to proper python syntax# test planCo-authored-by: Steven Bryan <steven@mac.local.meter>
# why- need to set default viewport when running on browserbase. previously,we only defined the default inside the exported `StagehandConfig`# what changed- set default viewport to 1288 * 711 when running on browserbase# test plan- tested locally,- regression evals
This PR was opened by the [Changesetsrelease](https://github.com/changesets/action) GitHub action. Whenyou're ready to do a release, you can merge this and the packages willbe published to npm automatically. If you're not ready to do a releaseyet, that's fine, whenever you add more changesets to main, this PR willbe updated.# Releases## @browserbasehq/stagehand@2.5.2### Patch Changes- [browserbase#1114](browserbase#1114)[`c0fbc51`](browserbase@c0fbc51)Thanks [@seanmcguire12](https://github.com/seanmcguire12)! - configuredefault viewport when running on browserbase## @browserbasehq/stagehand-evals@1.1.1### Patch Changes- Updated dependencies\[[`c0fbc51`](browserbase@c0fbc51)]:    -   @browserbasehq/stagehand@2.5.2## @browserbasehq/stagehand-examples@1.0.11### Patch Changes- Updated dependencies\[[`c0fbc51`](browserbase@c0fbc51)]:    -   @browserbasehq/stagehand@2.5.2Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
renland others added11 commitsOctober 9, 2025 17:18
Updated link in the Getting Started section to point to the correctQuickstart Guide.# whyQuickstart link in README leads to a non-existent page.<img width="1556" height="763" alt="image"src="https://github.com/user-attachments/assets/20a1a5b5-8534-43b4-89d5-e3a062b3965a"/># what changedUpdated quickstart link in README to the correct quickstart address`https://docs.stagehand.dev/first-steps/quickstart`# test planAccess new link to quickstart
# whycurrently, for openai cua agent, we are handling keypress actionsincorrectlycurrently, there is no way to pass a custom system prompt to the Googlecua agent# what changed- All key actions, are now ran through mapKeyToPlaywright function toensure we are properly mapping the agents actions to valid playwrightkeys- Custom system prompts now override the default system prompt forGoogle Cua agent# test plantested locally with google & openai cua agentsFixesbrowserbase#1122
# whycurrently when using stagehand agent through api, it returns earlywithout executing# what changedwe now properly handle the options when none are present # test plantested locally, and tested across other cua agents to ensure no breakingchanges
# whyNew model dropped# what changedAdded support for haiku 4.5# test plan
…ocs (browserbase#1140)# whyWe recently shipped updates to our MCP server that should be reflectedin the documentation.# what changedupdate tools list for MCP update, removing mentions of multisession,adding experimental flag + get url toolrelated PR:browserbase/mcp-server-browserbase#123# test plann/a
# whyBroken links. # what changedFixed broken links# test planThis PR.---------Co-authored-by: GG <guergabo@mac.local.meter>
# whycurrently, we have no sense of what url an action was taken on and whattime it was taken when using agentthis is useful to have, because in the dashboard it will allow us tofilter the agents actions by url, and display timestamps of theindividual actions# what changed- added pageUrl to every action - added timestamp to every actionthe url is grabbed prior to the action being taken. This is because ifwe do it after the action is taken, there is a chance the action couldhave caused a navigation, which would result in the incorrect url forthe action# test plantested locally
# whyMake it easier to parse/filter/group evals# what changedEvals tagged with more granular metadata and error parsing# test plan---------Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
# why- there was previously no logger parameter in the external aisdk client# what changed- added logger param# test plan- evals, - install from alpha & smoke test---------Co-authored-by: Miguel <36487034+miguelg719@users.noreply.github.com>
**Summary**This PR fixes the default viewport mentioned in our docs.
- Update stagehand.config.ts to use __dirname for reliable .env path- Update 2048 example to use StagehandConfig- Fix API key error tests to properly test without env keys- Add beforeAll/afterAll hooks to manage test environment isolation
@changeset-bot
Copy link

⚠️ No Changeset found

Latest commit:a2ad8d0

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go.If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Copy link
Contributor

@greptile-appsgreptile-appsbot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Greptile Overview

Summary

Fixed environment variable loading in examples and tests by updatingstagehand.config.ts to usepath.resolve(__dirname, ".env") instead of relying onprocess.cwd(), ensuring.env is loaded correctly regardless of execution context.

Key Changes:

  • stagehand.config.ts: Changeddotenv.config() to use__dirname-based path resolution for reliable.env loading
  • examples/2048.ts: Updated to import and use sharedStagehandConfig for consistency
  • evals/deterministic/tests/Errors/apiKeyError.test.ts: AddedbeforeAll/afterAll hooks and explicit config overrides (llmClient: undefined,modelClientOptions: undefined) to properly isolate tests and verify error handling

Impact:
Examples now run successfully from any directory without "OpenAI API key is missing" errors, and API key error tests are properly isolated and won't interfere with other test files.

Confidence Score: 5/5

  • This PR is safe to merge with minimal risk
  • The changes are well-tested (43 tests passing), address a clear bug with a straightforward fix, and improve test isolation without introducing new dependencies or complex logic
  • No files require special attention

Important Files Changed

File Analysis

FilenameScoreOverview
stagehand.config.ts5/5Fixed dotenv path resolution to use__dirname instead ofprocess.cwd() for reliable.env loading
examples/2048.ts5/5Updated to use sharedStagehandConfig for consistency with other examples
evals/deterministic/tests/Errors/apiKeyError.test.ts5/5Added proper test isolation withbeforeAll/afterAll hooks and explicit config overrides to test error conditions

Sequence Diagram

sequenceDiagram    participant User    participant ExampleScript as examples/2048.ts    participant Config as stagehand.config.ts    participant Dotenv as dotenv    participant FS as File System        User->>ExampleScript: pnpm run example 2048    Note over User,ExampleScript: CWD = examples/        ExampleScript->>Config: import StagehandConfig    activate Config        Config->>Dotenv: dotenv.config({ path: path.resolve(__dirname, ".env") })    Note over Config,Dotenv: __dirname points to project root<br/>(where config file is located)        Dotenv->>FS: Read .env from project root    FS-->>Dotenv: Environment variables loaded        Dotenv-->>Config: Success    Config-->>ExampleScript: Config with OPENAI_API_KEY    deactivate Config        ExampleScript->>ExampleScript: new Stagehand({ ...StagehandConfig })    Note over ExampleScript: API key properly loaded
Loading

3 files reviewed, no comments

Edit Code Review Agent Settings |Greptile

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

1 more reviewer

@greptile-appsgreptile-apps[bot]greptile-apps[bot] left review comments

Reviewers whose approvals may not affect merge requirements

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

13 participants

@shamsharoon@tkattkat@chrisreadsf@Kylejeong2@miguelg719@derekmeegan@filip-michalsky@rsbryan@seanmcguire12@victlue@jay-sahnan@renl@guergabo

[8]ページ先頭

©2009-2025 Movatter.jp