Repository files navigation

1. Install .NET

2. Download this Project.

3. Run

cd FTServerdotnet run -c Release

4. Openhttp://127.0.0.1:5066/

5. Press [Ctrl+C] to shut down.

Input a Full URL to index the Page, then search.

Move page forward by re-indexing the page.

[Word1 Word2 Word3] => text hasWord1 andWord2 andWord3

["Word1 Word2 Word3"] => text has"Word1 Word2 Word3" as a whole

Search [https] or [http] => get almost all pages

Download Visual Studio Code

iBoxDB

AngleSharp

Semantic-UI

The results order based on theid() number inclass PageText, descending order.

A Page has many PageTexts. if don't need multiple Texts, modifyHtml.getDefaultTexts(Page), returns only one PageText (the page description text only,Config.DescriptionOnly=true ).

the Page.GetRandomContent() method is used to keep the Search-Page-Content always changing, doesn't affect the real PageText order.

Use the ID number to control the order instead of loading all pages to memory.

search (... String keywords, longstartId, longcount)

startId => which ID(the id when you created PageText) to start,use (startId=Long.MaxValue) to read from the top, descending order

count => records to read,important parameter, the search speed depends on this parameter, not how big the data is.

set the startId as the last id from the results of search minus one

startId=search("keywords",startId,count);nextpage_startId=startId-1// this 'minus one' has done inside search()...//read next pagesearch("keywords",nextpage_startId,count)

mostly, the nextpage_startId is posted from client browser when user reached the end of webpage,and set the default nextpage_startId=Long.MaxValue,in javascript the big number have to write as String ("'" + nextpage_startId + "'")

Open

publicPageHtml.Get(Stringurl);

Set your private WebSite text

Pagepage=newPage();page.url=url;page.title=title;page.text=bodyTextpage...= ...return page;

Setting Index Readonly Cache (Readonly_MaxDBCount) fromFTServer/Code/Config.cs .

[user@localhost~]$ cat /proc/sys/fs/file-max803882[user@localhost~]$ulimit -a| grep filesopen files                      (-n) 500000[user@localhost~]$ulimit -Hn500000[user@localhost~]$ulimit -Sn500000[user@localhost~]$ $ vi /etc/security/limits.conf*         hard    nofile      500000*         soft    nofile      500000root      hard    nofile      500000root      soft    nofile      500000[user@localhost~]$ firewall-cmd --add-port=5066/tcp --permanent

Why does Tracker consume resources on my PC?

[user@localhost~]$ tracker daemon -k[user@localhost project]$ tracker reset --hard

Transplant from Full Text Search Java JSP Version