- Notifications
You must be signed in to change notification settings - Fork2.4k
Closed as not planned
Description
Running auto-gptq-0.5.0:
I get the following error:
CUDA extension not installed.CUDA extension not installed.exllama_kernels not installed.Traceback (most recent call last): File "C:\Users\Owner\Desktop\Naby_AI\ai.py", line 18, in <module> from auto_gptq import AutoGPTQForCausalLM, BaseQuantizeConfig File "C:\Users\Owner\AppData\Local\Programs\Python\Python310\lib\site-packages\auto_gptq\__init__.py", line 4, in <module> from .utils.peft_utils import get_gptq_peft_model File "C:\Users\Owner\AppData\Local\Programs\Python\Python310\lib\site-packages\auto_gptq\utils\peft_utils.py", line 20, in <module> from ..nn_modules.qlinear.qlinear_exllama import QuantLinear as QuantLinearExllama File "C:\Users\Owner\AppData\Local\Programs\Python\Python310\lib\site-packages\auto_gptq\nn_modules\qlinear\qlinear_exllama.py", line 14, in <module> from exllama_kernels import make_q4, q4_matmulImportError: DLL load failed while importing exllama_kernels: The specified module could not be found.
Running auto-gptq-0.4.2 works just fine no errors.
Metadata
Metadata
Assignees
Labels
No labels