Repository files navigation

Retrack - track changes in a web page, API, or file.

Before running the Retrack server, you need to configure the database connection. If you don't have a PostgreSQL serverrunning,youcan run it locally with the following Docker Compose file:

docker-compose -f ./dev/docker/postgres.yml --env-file ./.env up --build --force-recreate

To remove everything and start from scratch, run:

docker-compose -f ./dev/docker/postgres.yml down --volumes --remove-orphans

Make sure to replacePOSTGRES_HOST_AUTH_METHOD=trust in Docker Compose file with a more secure authentication methodif you'replanning to use a local database for an extended period. For the existing database, you'll need to provide connectiondetails in theTOML configuration file as explained below.

Once all services are configured, you can start the Retrack server withcargo run. By default, theserver will be accessible viahttp://localhost:7676. Usecurl to verify that the server is up and running:

curl -XGET http://localhost:7676/api/status---{"version":"0.0.1"}

The server can be configured with a TOML configuration file. See the example below for a basic configuration:

port =7676[db]name ='retrack'host ='localhost'port =5432username ='postgres'password ='password'# Connection details for Web Scraper service.[components]web_scraper_url ='http://localhost:7272/'# SMTP server configuration used to send emails (signup emails, notifications etc.).[smtp]address ="xxx"username ="xxx"password ="xxx"# Trackers specific configuration.[trackers]max_revisions =10min_schedule_interval =600_000schedules = ["@","@hourly","@daily","@weekly","@monthly","@@"]

If you saved your configuration to a file namedretrack.toml, you can start the server with the following command:

cargo run -- -c retrack.toml

You can also use.env file to specify the location of the configuration file and database connection details requiredfor development and testing:

# Refer to https://github.com/launchbadge/sqlx for more details.DATABASE_URL=postgres://postgres@localhost/retrack# Path to the configuration file.RETRACK_CONFIG=${PWD}/retrack.toml

If you plan to interact with the web page trackers, you'll also need to have Chromium installed and start the Web Scraper component:

npm run watch -w components/retrack-web-scraper

To run with the debug output from Playwright, use the following command:

DEBUG=pw:protocol npm run watch -w components/retrack-web-scraper

You can also use.env file to configure Web Scraper:

# A path to the Chromium/Chrome executable.RETRACK_WEB_SCRAPER_BROWSER_EXECUTABLE_PATH="/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"# An amount of time in seconds to cache the web page content by the Web Scraper component.RETRACK_WEB_SCRAPER_CACHE_TTL_SEC=5