- Notifications
You must be signed in to change notification settings - Fork6.6k
fix(model_garden): add deepseek example#13444
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
base:main
Are you sure you want to change the base?
Uh oh!
There was an error while loading.Please reload this page.
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Summary of Changes
Hello@msampathkumar, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request adds a new Python sample to the Model Garden section, specifically for the DeepSeek model. It provides a functional example of how to programmatically interact with the DeepSeek model via Vertex AI's OpenAPI endpoint, including necessary authentication, dependency management, and a dedicated test case.
Highlights
- New DeepSeek Model Example: This PR introduces a new Python example demonstrating how to interact with the DeepSeek model hosted in Vertex AI Model Garden. The example showcases text generation by making an API call to the model's OpenAPI endpoint.
- Authentication and API Interaction: The core example (
textgen_with_txt.py) includes a helper function to obtain Google Cloud ADC (Application Default Credentials) and a bearer token, which are then used to authenticate and make a POST request to the Vertex AI endpoint for chat completions. - Testing and Dependencies: Accompanying the example are new
requirements.txtandrequirements-test.txtfiles to manage runtime and testing dependencies (likerequestsandpytest), along with anoxfile_config.pyfor build configuration and atest_examples.pyto verify the example's functionality.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on ourdocumentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either/gemini <command> or@gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review | Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary | Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help | Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a.gemini/ folder in the base of the repository. Detailed instructions can be foundhere.
Limitations & Feedback
Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on@gemini-code-assist comments to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with theGemini Code Assist IDE Extension.
Footnotes
Review thePrivacy Notices,Generative AI Prohibited Use Policy,Terms of Service, and learn how to configure Gemini Code Assist in GitHubhere. Gemini can make mistakes, so double check it anduse code with caution.↩
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Code Review
This pull request introduces a new Python example for interacting with a DeepSeek model via Vertex AI Model Garden. The changes include the core script for API interaction, unit tests, Nox configuration, and dependency files. My review focuses on several key areas:
- Correctness: Ensuring type hints match actual return types and that the primary function returns the expected generated content.
- Error Handling: Adding checks for HTTP errors to make the script more robust.
- Maintainability (PEP 8): Recommending standard import placement for better code organization.
Overall, the example is a good starting point, and the suggested changes aim to enhance its reliability and adherence to Python best practices. The review comments reference PEP 8, Python's official style guide, particularly concerning import statement placement.
| # limitations under the License. | ||
| def get_bearer_token() -> str: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
The functionget_bearer_token is type-hinted to returnstr, but it actually returns a tuple(project_name, bearer_token). This mismatch can lead to type errors and incorrect assumptions by callers of this function.
| defget_bearer_token()->str: | |
| defget_bearer_token()->tuple[str,str]: |
| } | ||
| # Send the request | ||
| response = requests.post(url, headers=headers, data=json.dumps(data)) | ||
| print(f"Request Response: {response.status_code}") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
The HTTP response status is printed, but errors (like 4xx or 5xx client/server errors) are not explicitly handled before attempting to process the response. If the API call fails,json.loads(response.content) could raise an exception on unexpected content, or the script might proceed with invalid data. It's robust practice to check for HTTP errors immediately after the request.
Consider usingresponse.raise_for_status(), which will raise anrequests.exceptions.HTTPError if the HTTP request returned an unsuccessful status code.
| print(f"Request Response:{response.status_code}") | |
| response.raise_for_status()# Will raise an HTTPError for bad responses (4XX or 5XX) | |
| print(f"Request Response:{response.status_code}") |
| # The sky appears blue due to a phenomenon called **Rayleigh scattering**, | ||
| # which occurs when sunlight passes through Earth's atmosphere and | ||
| # ... | ||
| return response.content |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
The functiongenerate_content is type-hinted to returnstr and its name suggests it returns the generated textual content from the model. However, it currently returnsresponse.content, which is of typebytes (the raw response body).
To align with the type hint and function's intent, it should parse the JSON response and return the actual generated text string, typically found within thechoices part of the response. Accessing nested dictionary keys should also be done safely (e.g., using.get()).
| returnresponse.content | |
| # Extract andreturnthe assistant's message if available | |
| ifmodel_response.get("choices")andlen(model_response["choices"])>0: | |
| message=model_response["choices"][0].get("message", {}) | |
| content=message.get("content") | |
| ifcontentisnotNone: | |
| returncontent | |
| return""# Return empty string or raise an error if content not found/as appropriate |
| import google.auth | ||
| from google.auth.transport.requests import Request |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Importsgoogle.auth andgoogle.auth.transport.requests are defined locally within theget_bearer_token function. According to PEP 81, imports should generally be at the top of the file (after module docstrings and comments, before globals and constants) for better readability, maintainability, and discoverability of dependencies. Please move these imports to the module level.
Style Guide References
| importgoogle.auth | |
| fromgoogle.auth.transport.requestsimportRequest | |
| # These imports should be moved to the top of the file: | |
| # import google.auth | |
| # from google.auth.transport.requests import Request |
Footnotes
PEP 8: Imports are always put at the top of the file, just after any module comments and docstrings, and before module globals and constants. This improves clarity and makes it easier to see what modules a script requires.↩
| import requests | ||
| import json |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Importsrequests andjson are defined locally within thegenerate_content function. Similar to the previous comment, PEP 81 recommends placing imports at the top of the file. Please move these to the module level.
Style Guide References
| importrequests | |
| importjson | |
| # These imports should be moved to the top of the file: | |
| # import requests | |
| # import json |
Footnotes
PEP 8: Imports are always put at the top of the file, just after any module comments and docstrings, and before module globals and constants. This improves clarity and makes it easier to see what modules a script requires.↩
Description
Add new samples for Model Garden.
Note - Need to enable the Model!
Note: Before submitting a pull request, please open an issue for discussion if you are not associated with Google.
Checklist
nox -s py-3.9(seeTest Environment Setup)nox -s lint(seeTest Environment Setup)