Repository files navigation

This program can search Snapchat's publicSnap Mapat multiple locations and download all stories for later investigation andcategorization. A lot is unknown about how Snapchat decides which videos showup at which "point" on the map, so the scraper can also randomize the geolocationwhere it scrapes from by a few hundred meters to hopefully convince the APIto give you a few more relevant videos.

This has been tested on Linux, but should work on Windows/Mac as well witha video viewer installed.

git clone https://github.com/nemec/snapchat-map-scraper.gitcd snapchat-map-scraper/python3 -m venv env# create virtual environmentsource env/bin/activate# activate virtual environmentpip3 install -r requirements.txt

Follow each step in order. Also, ensure you have activated your virtualenvironment, otherwise the packages will be missing.

This database holds data related to one group of search queries. Since SQLiteproduces database files with little overhead, you should create a new databaseeach time you want to sample data.

python3 story_downloader.py create snap.db

The add command will add a new location to the database. The program will laterscrape and log snaps from all of the added locations so that you can track multiplespots at once. Find the latitude and longitude of your point of interest (thiscan easily be found on Snap Map, as seen below) and replace them in the commandbelow. The label makes it easier to remember where a point is.

python3 story_downloader.py add --database snap.db --label"downtown dallas" 32.783038 -96.796388# Added downtown dallas to database

The scrape command iterates through all of the added locations and downloadssnaps at that location. Previews, videos, and overlays are all downloadedbut the review command can only deal with video at the moment.

Optional arguments:

• --randomize will randomize the geo location within 500m, intended to promptSnapchat's API to send new videos that it hasn't yet served. I have no ideawhy Snapchat only sends a subset of the total videos for a single point,probably to make the service more scalable.
• --repeat will cause the application to loop infinitely looking for new videos.Leave it running overnight to collect as much as possible.
• --sleep will modify how long the application sleeps before repeating. Defaultsto 120 seconds and only takes effect if--repeat is included.
• A label can also be added as a positional argument to only scrape one location.Example:python3 story_downloader.py scrape "downtown dallas"

python3 story_downloader.py scrape --database snap.db --randomize --repeat# Scraped 16 media from location downtown dallas# Sleeping for 120 seconds...

The review command looks through all unreviewed videos and opens each one ina video player for review. Once you have closed the video player process,the CLI will prompt you for a "classification". This can be any text describingthe situation in the video or a tag, etc. If the video is irrelevant, leavethe classification blank and just hit "Enter". Once all remaining videos areclassified, the application will exit. Depending on your operating system,

python3 story_downloader.py review --database snap.db# Classify or leave blank: police protesters# 15 remaining# Classify or leave blank:# 14 remaining# Classify or leave blank: fire# 13 remaining# ...

The export command will copy all videos which are "classified" to a new folderfor further review/editing/publishing.

python3 story_downloader.pyexport --database snap.db export/# 3 video(s) exportedls export/# 2020-05-29-04:26:30-fire.mp4# 2020-05-30-12:26:57-crowd.mp4# '2020-05-30-21:16:22-line of people.mp4'