- Notifications
You must be signed in to change notification settings - Fork746
Skip credential validation for disabled providers in relay mode#5283
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
base:main
Are you sure you want to change the base?
Uh oh!
There was an error while loading.Please reload this page.
Conversation
By default, all model providers have credential validation skippedwhen the gateway is running in relay mode (since they will not be used).We still run credential validation for providers that are childrenof a 'skip_relay' model
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Pull request overview
This PR implements conditional credential validation for model providers based on relay mode configuration. When the gateway runs in relay mode, credential validation is skipped for model providers that won't be used (since requests are forwarded to the downstream gateway), but validation is still performed for providers used by models marked withskip_relay=true.
Key changes:
- Introduced
relay_modeparameter to model loading logic to conditionally skip credential validation - Added
with_skip_credential_validationhelper function to wrap async futures with skipped validation - Separated e2e test credential validation behavior from general credential validation logic via
e2e_skip_credential_validation
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tensorzero-core/src/model.rs | Adds relay_mode parameter to model loading and conditionally wraps provider config loading with skip credential validation for relay models |
| tensorzero-core/src/config/mod.rs | Introduceswith_skip_credential_validation async helper ande2e_skip_credential_validation function; changes SKIP_CREDENTIAL_VALIDATION to private; updates config loading to use new helpers |
| tensorzero-core/src/model_table.rs | Refactors credential loading to usee2e_skip_credential_validation for conditional warning logs instead of cfg guards |
| tensorzero-core/src/providers/gcp_vertex_anthropic.rs | Updates GCP error handling to usee2e_skip_credential_validation for conditional warnings |
| tensorzero-core/src/providers/azure.rs | Refactors tests to use newwith_skip_credential_validation helper instead of direct task_local scope |
| tensorzero-core/src/config/tests.rs | Adds test for nestedwith_skip_credential_validation calls to verify proper scope handling |
| gateway/tests/relay_credential_validation.rs | Adds comprehensive e2e tests for relay mode credential validation behavior including skip_relay models |
| tensorzero-optimizers/tests/common/mod.rs | Updates model config loading call to passrelay_mode=false parameter |
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
By default, all model providers have credential validation skipped when the gateway is running in relay mode (since they will not be used). We still run credential validation for providers that are children of a 'skip_relay' model
Fixes#5277
Important
Skip credential validation for model providers in relay mode, except for 'skip_relay' models, with tests added for validation behavior.
UninitializedModelConfig::load()inmodel.rs.with_skip_credential_validation()function inconfig/mod.rsto manage credential validation state.relay_credential_validation.rsto test credential validation behavior in relay mode.test_nested_skip_credential_validation()inconfig/tests.rsto test nested credential validation skipping.load_credential()inmodel_table.rsto usee2e_skip_credential_validation()for logging warnings during tests.AzureProviderandGCPVertexAnthropicProviderto usewith_skip_credential_validation()for credential loading inazure.rsandgcp_vertex_anthropic.rs.This description was created by
foraaceecb. You cancustomize this summary. It will automatically update as commits are pushed.