- Notifications
You must be signed in to change notification settings - Fork845
[AI Evaluation] Add versioning to reporting and caching#6070
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSharp/ScenarioRunResult.cs OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
src/Libraries/Microsoft.Extensions.AI/ChatCompletion/DistributedCachingChatClient.csShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
src/Libraries/Microsoft.Extensions.AI/ChatCompletion/DistributedCachingChatClient.cs OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
…edCachingChatClient.cs
peterwald commentedMar 11, 2025
@dotnet/dotnet-extensions-ai This change is ready to merge if you approve. |
src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSharp/ScenarioRunResult.cs OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSharp/Defaults.cs OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
…arp/ScenarioRunResult.csCo-authored-by: Igor Velikorossov <RussKie@users.noreply.github.com>
…arp/Defaults.csCo-authored-by: Igor Velikorossov <RussKie@users.noreply.github.com>
src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSharp/ScenarioRunResult.cs OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
src/Libraries/Microsoft.Extensions.AI/ChatCompletion/DistributedCachingChatClient.csShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSharp/Defaults.cs OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSharp/ScenarioRunResult.cs OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSharp/ScenarioRunResult.cs OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSharp/ScenarioRunResult.cs OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSharp/ScenarioRunResult.cs OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSharp/Defaults.cs OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
peterwald commentedMar 13, 2025
Updated based on your feedback@shyamnamboodiripad@stephentoub |
src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSharp/ScenarioRunResult.cs OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
test/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting.Tests/ScenarioRunResultTests.csShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSharp/ScenarioRunResult.cs OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
e393f33 intomainUh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
These are basically independent escape mechanisms to handle future changes in formats.
ReportFormatVersion- Will record which version of the format was used to persist the scenario run, as well as exposing which version the current library will write. This should allow us to convert/adapt to reporting format changes in the future if needed.CacheVersion- Will change all the caching keys causing the existing cache entries to be no longer valid.@stephentoub@SteveSandersonMSHopefully neither of these will ever be needed.
Microsoft Reviewers:Open in CodeFlow