- Notifications
You must be signed in to change notification settings - Fork227
Update VitisAIQuantization to use Quark#1715
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
base:main
Are you sure you want to change the base?
Uh oh!
There was an error while loading.Please reload this page.
Conversation
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
lintrunner found more than 20 potential problems in the proposed changes. Check theFiles changed tab for more details.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
please improve the example to refer to Quark documentation:https://quark.docs.amd.com/latest/supported_accelerators/ryzenai/index.html |
Uh oh!
There was an error while loading.Please reload this page.
/azp run |
Azure Pipelines successfully started running 1 pipeline(s). |
Uh oh!
There was an error while loading.Please reload this page.
olive/cache.py Outdated
@@ -40,6 +40,7 @@ class CacheSubDirs: | |||
evaluations: Path | |||
resources: Path | |||
mlflow: Path | |||
vitis_ai: Path |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Can you explain more about how will you use this folder? The cache folder is designed to be pass-agnostic so i want to double confirm the use case here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
The folder will be created at the beginning of the evaluation step, upon the creation of a VitisAIExecutionProvider inference session (used as model cache by EP). Is evaluation considered an Olive pass?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
No, if VitisAIEP will need to cache a model for evaluation, can we create a temporal folder for it? (and it will be deleted after all. I assume this model cache is not needed when the workflow finish.). We can create a temporary folder in cache.evaluations like temp_model_cache or something.
Update the entry in |
Added links of Quark documentation on quantization configurations |
Uh oh!
There was an error while loading.Please reload this page.
For a Ryzen AI example, please use XINT8 as the example instead of INT8_TRANSFORMER_ACCURATE. Just checking — does this example currently runnable on Olive? |
Uh oh!
There was an error while loading.Please reload this page.
@vortex-captain please create a copy of your branch directly in this repo and open a new PR to be able to run the CI without the login issue. |
Updated example in description. And yes, such an example (BERT text model) is runnable on Olive, but in evaluation, the output model cannot run on NPU (all nodes assigned to CPU), unlike the ResNet examples. Any insights? |
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
elif provider == "VitisAIExecutionProvider": | ||
import os | ||
apu_type = get_vai_apu_type() | ||
set_vai_environment_variable(apu_type) | ||
install_dir = Path(os.environ["RYZEN_AI_INSTALLATION_PATH"]) | ||
provider_options[idx]["config_file"] = str(install_dir / "voe-4.0-win_amd64" / "vaip_config.json") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
This is adding dependency to a specific version. I don't think we should add this here.
Uh oh!
There was an error while loading.Please reload this page.
Description
Example usage in Olive workflow json:
Please refer tohttps://quark.docs.amd.com/latest/onnx/user_guide_config_description.html for the complete list of
config_template
options. All the other quantization options are listed inhttps://quark.docs.amd.com/latest/onnx/appendix_full_quant_config_features.html .Examples
2 ResNet examples are added to
examples/vai
, which convert the models using Quark then evaluate onVitisAIExecutionProvider
(run on NPU, RyzenAI 1.3.1, onnxruntime-vitisai 1.19).Checklist before requesting a review
lintrunner -a
(Optional) Issue link