- Notifications
You must be signed in to change notification settings - Fork845
Replace JSON vector store with SQLite#6438
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
....AI.Templates/src/ChatWithCustomData/ChatWithCustomData-CSharp.Web/Services/IngestedChunk.csShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
stephentoub commentedMay 13, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
This will add over a mb of binaries to the git history. Are we ok with that? Or did you mean you'll update the pr before it merges? |
jeffhandley commentedMay 14, 2025
I applied * NO MERGE * |
jeffhandley left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
This looks great to me,@MackinnonBuck. Nice work! Let's get a review from at least one of@roji or@SteveSandersonMS though, and wait for the NuGet packages to be available before merging.
roji left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Thanks for pinging me, see below for a few comments.
Note that there will still be a few API changes for GA; some of my comments can already be addressed now, but others can't. So we can do another sync/iteration on this PR (hopefully by Friday everything will be in), and in general only merge after the GA nugets are fully available on nuget.org.
...tensions.AI.Templates/src/ChatWithCustomData/ChatWithCustomData-CSharp.Web/Program.Aspire.cs OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
...soft.Extensions.AI.Templates/src/ChatWithCustomData/ChatWithCustomData-CSharp.Web/Program.cs OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
.../Snapshots/aichatweb.BasicAspire.verified/aichatweb/aichatweb.Web/Services/SemanticSearch.cs OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
SteveSandersonMS left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Looks superb - great job!
jeffhandley left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Just the patch version bum in my comments, and then once@roji approves we're good to go. Nice work,@MackinnonBuck!
...src/ChatWithCustomData/ChatWithCustomData-CSharp.Web/ChatWithCustomData-CSharp.Web.csproj.inShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
roji left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Looks great@MackinnonBuck! See the one long comment below about doing automatic embedding generation for upsert as well, not just for search. Other than that I think it's ready for merging.
...tweb.AzureOpenAI_Qdrant_Aspire.verified/aichatweb/aichatweb.Web/Services/IngestedDocument.cs OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
...s.IntegrationTests/Snapshots/aichatweb.Basic.verified/aichatweb/Services/IngestedDocument.cs OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
...chatweb.AzureOpenAI_Qdrant_Aspire.verified/aichatweb/aichatweb.Web/Services/IngestedChunk.cs OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
Co-authored-by: Jeff Handley <jeffhandley@users.noreply.github.com>
....Templates/src/ChatWithCustomData/ChatWithCustomData-CSharp.Web/Services/IngestedDocument.csShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
...zureOpenAI_Qdrant_Aspire.verified/aichatweb/aichatweb.Web/Services/Ingestion/DataIngestor.csShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
roji left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Looks great@MackinnonBuck, well done!
712d9aa intomainUh oh!
There was an error while loading.Please reload this page.
Co-authored-by: Jeff Handley <jeffhandley@users.noreply.github.com>
Uh oh!
There was an error while loading.Please reload this page.
This PR removes the JSON vector store and replaces it with a SQLite implementation using
Microsoft.SemanticKernel.Connectors.SqliteVec.Since
Microsoft.SemanticKernel.Connectors.SqliteVecisn't shipping publicly yet, I've builtMicrosoft.SemanticKernellocally and included the generated.nupkgs directly in the repo. This change (41bd61e) will get reverted after the updatedMicrosoft.SemanticKernelpackages ship.Microsoft Reviewers:Open in CodeFlow