- Notifications
You must be signed in to change notification settings - Fork55
Amazon S3 bucket finder and crawler.
License
clarketm/s3recon
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
Amazon S3 bucket finder and crawler.
NOTE: s3recon requires python version>=3.6
$ pip install s3recon
usage: s3recon [-h] [-o file] [-d] [-p] [-t seconds] [-v] [-c num] word_list [word_list ...]positional arguments: word_list read words from one or more <word-list> filesoptional arguments: -h, --help show this help message and exit -o file, --output file write output to <file> -d, --db write output to database -p, --public only include 'public' buckets in the output -t seconds, --timeout seconds http request timeout in <seconds> (default: 30) -v, --version show program's version number and exit -c num, --concurrency num maximum <num> of concurrent requests (default: # of lcpus)TheSecLists repository has a multitude of word-lists to choose from. For this example, let's download the sample word-list included in this repository.
$ curl -sSfL -o"word-list.txt""https://raw.githubusercontent.com/clarketm/s3recon/master/data/words.txt"
Executes3recon using theword-list.txt file and output thepublic S3 buckets to a json file namedresults.json.
$ s3recon"word-list.txt" -o"results.json" --public- PRIVATE https://s3.sa-east-1.amazonaws.com/test-lyft- PRIVATE https://s3.ap-south-1.amazonaws.com/test.amazon+ PUBLIC https://walmart-dev.s3.us-east-1.amazonaws.com- PRIVATE https://s3.ap-southeast-1.amazonaws.com/apple-prod- PRIVATE https://walmart.s3.ap-southeast-1.amazonaws.com...
Check theresults.json output file to view the S3 buckets you have discovered!
$ cat"results.json"{"public": {"total":12,"hits": ["https://walmart-dev.s3.us-east-1.amazonaws.com","https://apple-production.s3.ap-southeast-1.amazonaws.com",... ] }}Note: to include
privatebuckets in the results omit the-p, --publicflag from the command.
Enumerate the static files located in each bucket and record the findings.
Coming soon!
TheSecLists repository has a multitude of word-lists to choose from. For this example, let's download the sample word-list included in this repository.
$ curl -sSfL -o"word-list.txt""https://raw.githubusercontent.com/clarketm/s3recon/master/data/words.txt"
$ docker run --name "mongo" -p 27017:27017 -v "mongodb_data:/data/db" -v "mongodb_config:/data/configdb" -d mongoExecutes3recon using theword-list.txt file and output to MongoDB instance.
$ s3recon"word-list.txt" --db- PRIVATE https://s3.sa-east-1.amazonaws.com/test-lyft- PRIVATE https://s3.ap-south-1.amazonaws.com/test.amazon+ PUBLIC https://walmart-dev.s3.us-east-1.amazonaws.com- PRIVATE https://s3.ap-southeast-1.amazonaws.com/apple-prod- PRIVATE https://walmart.s3.ap-southeast-1.amazonaws.com...Check the MongoDB database:s3recon collection:hits to view the S3 buckets you have discovered!
$ mongo"s3recon" --quiet --eval'db.hits.find({}, {"url": 1, "access": 1, "_id": 0}).limit(5)'
{"url" :"https://s3.us-east-2.amazonaws.com/apple","access" :"private" }{"url" :"https://s3.us-west-1.amazonaws.com/microsoft-dev","access" :"private" }{"url" :"https://s3.us-west-1.amazonaws.com/dev-microsoft","access" :"private" }{"url" :"https://s3.us-east-2.amazonaws.com/amazon","access" :"private" }{"url" :"https://s3.us-east-1.amazonaws.com/dev-amazon","access" :"private" }Enumerate the static files located in each bucket and record the findings.
Coming soon!
s3recon can be configure using a yaml configuration file located in either the current working directory (e.g../s3recon.yml) or your home diretory (e.g.~/s3recon.yml).
The following is the list of configurable values:
# s3recon.ymldatabase:{ host: "0.0.0.0", ... }separators:["-", "_", "."]environments:["", "backup", "backups", ...]regions:["ap-northeast-1", "ap-northeast-2", ...]
To see the full list of configurable values (and theirdefaults) please refer to thes3recon.yml file in this repository.
The AWSregions can be altered by setting theregions array in yours3recon.yml configuration file.
# s3recon.ymlregions:[ "us-west-2", ...]
Theenvironments are modifiers permuted with each item of theword-list (and theseparator) to construct the bucket value in request.The value can be altered by setting theenvironments array in yours3recon.yml configuration file.
For example, to only search lines from the word-listverbatim (i.e. without modification) you can set this value to an empty array.
# s3recon.ymlenvironments:[]
The databasehost andport can be configured by altering thedatabase map in yours3recon.yml configuration file.
For example,host andport can be set directly inside thedatabase map
# s3recon.ymldatabase:{host:"0.0.0.0",port:27017}
Sorry, at the moment only MongoDB is supported.
- Create
crawlcommand to crawl public/private buckets found infindstage. - Separate out
findandcrawlas subcommands. - Store discovered buckets in a NoSQL database.
This tools is distributed for educational and security purposes. I take no responsibility and assume no liability for the manner in which this tool is used.
MIT ©Travis Clarke
About
Amazon S3 bucket finder and crawler.