Folders and files

Name		Name	Last commit message	Last commit date
Latest commit History 84 Commits
.github		.github
Formula		Formula
.gitignore		.gitignore
.golangci.yml		.golangci.yml
.goreleaser.yml		.goreleaser.yml
LICENSE.md		LICENSE.md
Makefile		Makefile
README.md		README.md
cli.go		cli.go
go.mod		go.mod
go.sum		go.sum
icon.svg		icon.svg
main.go		main.go
output.go		output.go
scrape.go		scrape.go
snapcraft.yaml		snapcraft.yaml

Repository files navigation

Scrape

CLI utility to scrape emails from websites

Features

Asynchronous scraping
Recursive link follow
External link follow
Cloudflare email obfuscation decoding
Client side rendered pages support through headlesschromium load awaits
Simple, grepable output

Install

MacOS:

brew tap lawzava/scrape https://github.com/lawzava/scrapebrew install scrape

Linux:
```
sudo snap install scrape
```

Usage

Sample call:

scrape -w https://lawzava.com

Depends onchromium orgoogle-chrome being available in path if--js is used

Parameters:

      --async             Scrape website pages asynchronously (default true)      --debug             Print debug logs  -d, --depth int         Max depth to follow when scraping recursively (default 3)      --follow-external   Follow external 3rd party links within website  -h, --help              help for scrape      --js                Enables EnableJavascript execution await      --output string     Output type to use (default 'plain', supported: 'csv', 'json') (default "plain")      --output-with-url   Adds URL to output with each email      --recursively       Scrape website recursively (default true)      --timeout int       If > 0, specify a timeout (seconds) for js execution await  -w, --website string    Website to scrape (default "https://lawzava.com")

Note about scraper package

For those that are looking forscraper package - this repository was intended as a cli-use only thus the scraper package was moved tolawzava/emailscraper.Thescrape utility will be maintained as a CLI implementation ofemailscraper package.