Repository files navigation

Framework for implementing Intrusion Detection Systems (NIDS) aimed at identifying anomalies in network flows using Machine Learning models.

To create the experimental environment, execute the following commands:

python -m venv .env

Note: The virtual environment must have this name because the scripts use this name to load the needed libraries.fl-unsup-nids/blob/anomaly-flow-baseline/utils/load_data.pyAfter creating the Virtual environment, use the following command to install the necessary python packages:

pip install -r requirements.txt

To run the local experiments, use the following command:

source experiments.sh

To run the Federated Learning experiments, use the following command:

source run.sh

To run the experiments, we need to have the CIC-IDS2018, BoT-IoT and ToN-IoT in a folder called datasets in the root of the project, e.g. :

|- anomaly-flow    |- datasets         |- NF-CSE-CIC-IDS2018-v2-DDoS.csv.gz        |- NF-BoT-IoT-v2-DDoS.csv.gz        |- NF-ToN-IoT-v2-DDoS.csv.gz

In our case we used derived datasets containing only benign and DDoS samples filtered from the originals datasets.

To generate Synthetic Data for a specific dataset use the scriptmain.ipynb.

To run the external and simple model scripts, please install the custom pip package for the EFC algorithm:

1. Make sure the scripts have run privileges:

chmod +x ./auxiliary_scripts/install-efc.sh

2. Run the script (Git and the anomaly-flow environment named.env required):

./auxiliary_scripts/install-efc.sh

The baselines for a multi-domain DDoS detection using federated learning are available here:

• Autoencoder
• Logistic Regression
• FLAD

The data used to carry out the experiments can be obtained from:

Machine Learning-Based NIDS Datasets (Netflow V2 Datasets)

The following projects were used as reference for the preparation of this work:

GANomaly

Anomaly-Toolbox Project

Energy-based Flow Classifier

   Leonardo Henrique de Melo

   Gustavo de Carvalho Bertoli

   Michele Nogueira

   Aldri Luiz dos Santos

   Lourenço Alves Pereira Junior