- Notifications
You must be signed in to change notification settings - Fork3
⚡ Blazingly fast query engine
License
NotificationsYou must be signed in to change notification settings
rekki/go-query
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
Blazingly fast query engine
Used to build and execute queries such as:
n:=10// total docs in indexAnd(Term(n,"name:hello", []int32{4,5}),Term(n,"name:world", []int32{4,100}),Or(Term(n,"country:nl", []int32{20,30}),Term(n,"country:uk", []int32{4,30}), ))
- scoring: only
idfscore (for now) - supported queries:
or,and,and_not,dis_max,constant,term normalizers: space_between_digits, lowercase, trim, cleanup, etctokenizers: left edge, custom, charngram, unique, soundex etcgo-query-index: useful example of how to build more complex search engine with the library
Usually when you have inverted index you endup having something like:
data:= []*Document{}index:=map[string][]int32{}fordocId,d:=rangedocument {for_,token:=rangetokenize(normalize(d.Name)) {index[token]=append(index[token],docId) }}
then from documents like{hello world},{hello},{new york},{new world} you get inverted index in the form of:
{"hello": [0,1],"world": [0,3],"new": [2,3],"york": [2]}anyway, if you want to read more on those check out the IR-book
This package helps you query indexes of this form, in fairly efficient way, keep in mind it expects the[]int32 slices to besorted. Example:
package mainimport ("fmt""github.com/rekki/go-query")funcmain() {totalDocumentsInIndex:=10q:=query.And(query.Or(query.Term(totalDocumentsInIndex,"a", []int32{1,2,8,9}),query.Term(totalDocumentsInIndex,"b", []int32{3,9,8}), ),query.AndNot(query.Or(query.Term(totalDocumentsInIndex,"c", []int32{4,5}),query.Term(totalDocumentsInIndex,"c", []int32{4,100}), ),query.Or(query.Term(totalDocumentsInIndex,"d", []int32{1,2,3,4,5,6,7,8,9,10}),query.Term(totalDocumentsInIndex,"e", []int32{1,2,3,4,5,6,7,8,9,10}), ), ), )// q.String() is {{a OR b} AND {{d OR e} -({c OR x})}}forq.Next()!=query.NO_MORE {did:=q.GetDocId()score:=q.Score()fmt.Printf("matching %d, score: %f\n",did,score) }}
will print:
matching 1, score: 2.639057matching 2, score: 2.639057matching 3, score: 2.852632matching 8, score: 2.639057matching 9, score: 4.105394