- Notifications
You must be signed in to change notification settings - Fork328
Mistral AI and GGUF and Qwen support#1125
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
| @@ -197,7 +197,7 @@ def create_pipeline(task): | |||
| lower = model_name.lower() | |||
| else: | |||
| lower = None | |||
| if lower and "-ggml" in lower: | |||
| if lower and("-ggml" in lower or "-gguf" in lower): | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
@kczimm these run on the CPU, so could be useful for local dev on a M1. I'll try on my end just to be sure (on my desktop right now)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Worked on my M1 after uninstallingdeepspeed.
Uh oh!
There was an error while loading.Please reload this page.