- Notifications
You must be signed in to change notification settings - Fork468
add snippets tests & response_format support#1524
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
base:main
Are you sure you want to change the base?
Uh oh!
There was an error while loading.Please reload this page.
Conversation
Hey@TGlide let me know if you want any guidance about inference snippets or a review :) |
Hi@Wauplin sorry for the delay! Yes, I would love a review, specially to see if I've gone about this the right way. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Hey@TGlide , understood! I've had a first look at it, let me know if you have extra questions :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
sorry about all the work done on this module but tests are actually defined in thetasks-gen package (see./tasks-gen/scripts/generate-snippets-fixtures.ts) and fixtures are auto-generated + committed in./packages/tasks-gen/snippets-fixtures. This way we can easily spot in a PR what has changed and if the snippets are looking good.
| @@ -3,6 +3,9 @@ | |||
| * | |||
| * Using src/scripts/inference-codegen | |||
| */ | |||
| import type { ChatCompletionInputGrammarType } from "../chat-completion/inference.js"; | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
this is not needed in ASR types I believe
| temperature?: GenerationParameters["temperature"]; | ||
| max_tokens?: GenerationParameters["max_new_tokens"]; | ||
| top_p?: GenerationParameters["top_p"]; | ||
| response_format?: Record<string, unknown>; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I think there is a confusion between the Chat Completion API that accepts aresponse_format and the Text Generation API that accepts agrammar input.
| @@ -408,6 +436,46 @@ function formatBody(obj: object, format: "curl" | "json" | "python" | "ts"): str | |||
| } | |||
| } | |||
| function formatPythonValue(obj: unknown, depth?: number): string { | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I start to really think we should first generate the snippets and then format it. Seems that there are a few solutions although not very popular (blackjs,prettier/plugin-python). For now, let's keep it like this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Out of curiosity, was this code auto-generate or written manually? (if yes, better to mention it in docstring)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
(note to myself, maybe a single-file python formatter would be enough given our small requirements.Here's an example)
No description provided.