- Notifications
You must be signed in to change notification settings - Fork0
Redis Image with WhoisDS Domain Updates: Automatically fetches and stores newly registered domains in a Redis database.
License
StrackVibes/NRD-db
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
Welcome to the NRD-db (Newly Registered Domains with Redis) GitHub repository! NRD-db is a Docker image designed to automatically fetch and store newly registered domains in a Redis database. It simplifies the process of populating a Redis database with up-to-date domain information, making it a great fit for use with Arkime's WISE tagging.
·Report Bug ·Request Feature
Table of Contents
The primary objective of NRD-db is to provide an automated solution for keeping your Redis database up-to-date with newly registered domains. Searching through local text files for specific domains can be inefficient, and that's where NRD-db comes in. It fetches domain data from the WhoisDS service and stores it in a Redis database, allowing you to access this information efficiently.
To get a local copy up and running follow these simple example steps.
Before you begin, ensure that you have the following dependencies installed:
- DockerNOTE: To avoid using sudo for docker activities, add your username to the Docker Group
sudo apt install docker-ce -g
sudo usermod -aG docker${USER}
You can build and run the NRD-db Docker container using the following commands:
- Clone the repo
git clone https://github.com/StrackVibes/NRD-db.git
- Build the Docker image
cd NRD-dbdocker build -t nrd-db.
You can customize the NRD fetching and storage process by setting environment variables with the docker '--env' argument or permanently in the NRD.sh script. Here are the available variables:
NAME | DEFAULT VALUE | NOTES |
---|---|---|
DIR | /opt/nrd | The directory where NRD-db stores files and data. |
DAY_RANGE | 1 | The number of days you want to fetch newly registered domains for. |
DAILY_DIR | /tmp/daily | The directory where NRD-db stores temporary daily domain data files. |
TEMP_FILE | /tmp/nrd | The path to the temporary file used during domain data retrieval. |
PAID_WHOISDS_USERNAME | Your WhoisDS username for accessing paid data (if applicable) | |
PAID_WHOISDS_PASSWORD | Your WhoisDS password for accessing paid data (if applicable) | |
BASE_URL_FREE | Free | The base URL for fetching newly registered domain data for free. |
BASE_URL_PAID | Paid | The base URL for fetching newly registered domain data with your WhoisDS paid credentials. |
After configuring the environment variables, simply run the NRD-db Docker container, and it will start fetching newly registered domains based on the default variables in nrd.sh.
docker run -d nrd-db
By default, NRD-db is set to fetch NRD data for the last 1 day. You can adjust theDAY_RANGE variable to specify a different day range.
docker run -d nrd-db --env DAY_RANGE=10
You can use thePAID_WHOISDS_USERNAME andPAID_WHOISDS_PASSWORD variables if you have a paid WhoisDS subscription. If not, the tool will use the free data source by default.
docker run -d nrd-db --env PAID_WHOISDS_USERNAME=ThreatHunter --env PAID_WHOISDS_PASSWORD=NeRD
With thisdocker-compose.yml example, you can easily launch the NRD-db service with following:
version:'3'services:nrd: image: nrd build: ./Dockerfile container_name: nrd restart: always ports: -"6379:6379" volumes: - ./nrd/:/root/redis - ./nrd/redis.conf:/usr/local/etc/redis/redis.conf - ./nrd/collection/:/opt/nrd/ environment: - REDIS_PASSWORD=my-password - REDIS_PORT=6379 - REDIS_DATABASES=1
- Scheduled Updates
- Improved Logging
- Retireve ...
- DNS Record(s) Information
- IP2ASN Information
- WHOIS Information
- Reverse WHOIS (by Name) Information
- Certficates
- VirusTotal Information
See theopen issues for a full list of proposed features (and known issues).
Contributions are what makes the open-source community such an amazing place to learn, inspire, and create. Any contributions you make aregreatly appreciated.
If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement".Don't forget to give the project a star! Thanks again!
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/AmazingFeature
) - Commit your Changes (
git commit -m 'Add some AmazingFeature'
) - Push to the Branch (
git push origin feature/AmazingFeature
) - Open a Pull Request
Distributed under the Chicken Dance License. SeeLICENSE.md
for more information.
Shane Strack -@inshane09
Project Link:https://github.com/StrackVibes/NRD-db