Repository files navigation

A quick scraper application with a functioning example.

Install dependencies and runyarn develop.

Create your admin credentials on Strapi and create the first Scraper:

name: Jamstack.orgenabled: truefrequency: * * * * *

The cron tries to run the scraping code every minute, and:

• check if the Scraper isenabled before running;
• respects thefrequency set;

The scraping is done with puppeteer and cherio and is at/scripts/scrapers. The content is saved to theSite generators model.

You can turn it into a generic website scraper or just modify the index.js file to your needs. Possibilities are endless...