NotificationsYou must be signed in to change notification settings
Fork328
Star6.4k

Added blog post semantic search in postgres in 15 minutes#1535

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Jump to bottom

Merged

SilasMarvin merged 21 commits intomasterfromsilas-semantic-search-in-postgres-in-15-minutes

Jun 18, 2024

Merged

Added blog post semantic search in postgres in 15 minutes#1535

SilasMarvin merged 21 commits intomasterfromsilas-semantic-search-in-postgres-in-15-minutes

Jun 18, 2024

Conversation

Copy link

Contributor

SilasMarvin commentedJun 17, 2024

No description provided.

SilasMarvin added4 commits

June 11, 2024 10:57

Preliminary draft of semantic search in postgres in 15 minutes

18f8f44

Cleanups

00bd75d

Ready for review

068af92

Cleanup first paragraph

a9148da

SilasMarvin requested review frommontanalow andlevkk

June 17, 2024 21:45

levkkand others added4 commits

June 17, 2024 15:25

A few suggestions (#1536)

3e0fa33

Add reason on why to use semantic search

c71fcd2

Clean up spelling errors

9b6e75f

Fix more small spelling errors

b451c9b

montanalow reviewed

Jun 18, 2024

View reviewed changes

pgml-cms/blog/semantic-search-in-postgres-in-15-minutes.md OutdatedShow resolvedHide resolved

pgml-cms/blog/semantic-search-in-postgres-in-15-minutes.md Outdated


		We used the [pgml.embed](/docs/api/sql-extension/pgml.embed) PostresML function to generate an embedding of the sentence "Generating embeddings in Postgres is fun!" using the [mixedbread-ai/mxbai-embed-large-v1](https://huggingface.co/mixedbread-ai/mxbai-embed-large-v1) model from mixedbread.ai.

		The output size of the vector varies per model, and in _mxbai-embed-large-v1_ outputs vectors with 1024 dimensions: each vector contains 1024 floating point numbers.

Copy link

Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Suggested change

	The output size of the vector varies per model, and in_mxbai-embed-large-v1_ outputs vectors with 1024 dimensions: each vector contains 1024 floating point numbers.
	The output size of the vector varies per model, and in`mxbai-embed-large-v1` outputs vectors with 1024 dimensions: each vector contains 1024 floating point numbers.

I think we should use backticks for all identifiers, rather than italics.

Copy link

Contributor

levkkJun 18, 2024•
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

They don't render very nicely in our docs imo:

A few options here:

Increase line height, may help with the rendering.
Remove the border from the code blocks, making them less jarring.
Not use inline code blocks at all, since they don't really add to the readability at the moment.

Copy link

Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I agree that we could make some design improvements here. It'd be good to bring this up with the design team. We should still consistently use semantic markup, so that when the style is fixed, we get the benefits.

pgml-cms/blog/semantic-search-in-postgres-in-15-minutes.md OutdatedShow resolvedHide resolved

pgml-cms/blog/semantic-search-in-postgres-in-15-minutes.md


		!!!

		Now trying our search engine again:

Copy link

Contributor

montanalowJun 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This would be better illustrated with a graph at 10, 100, 1000, 10000, 100k, 1M. Show that Big O.

pgml-cms/blog/semantic-search-in-postgres-in-15-minutes.md OutdatedShow resolvedHide resolved

SilasMarvinand others added13 commits

June 18, 2024 08:52

Finish timings

d418deb