Repository files navigation

WARN: This is a highly unstable experiment at the moment.

Deduplicates scans which were already executed too recently.

This allows to build up setups with cascading scans where you have "discovery" scans which are executed often, e.g. every hour, which discover targets. (e.g.)

This allows to build up setups with cascading scans which perform "discovery" scans very often, but then only trigger compute heavy subsequent scans in a less frequent interval. E.g. scan for hosts in a network every 10m, port-scan identified hosts every 1h and only trigger resource intensive nuclei / zap scans every week.

The scan-deduplicator will automatically deduplicate scans which have ascan-deduplicator.securecodebox.io/min-time-interval annotation set.If a identical scan (based on a hash of the scan spec) was already started (in the same cluster & namespace), it the deduplicator will prevent it from being created on the cluster.

apiVersion:"execution.securecodebox.io/v1"kind:ScheduledScanmetadata:name:"nmap-scanme-nmap-org"annotations:scan-deduplicator.securecodebox.io/min-time-interval:4hspec:interval:5m# will actually only be started every 4hours, because of the deduplicationscanSpec:scanType:"nmap"parameters:      -"scanme.nmap.org"

One of the primary use cases of cascading scans is to deduplicate cascading scans.This allows to run the discovery scans earlier in the cascade with a higher frequency and then onjly run the more expensive scans later in the cascade less often.

apiVersion:"cascading.securecodebox.io/v1"kind:CascadingRulemetadata:name:"nuclei-http"labels:securecodebox.io/invasive:non-invasivesecurecodebox.io/intensive:lightspec:scanAnnotations:scan-deduplicator.securecodebox.io/min-time-interval:24hmatches:anyOf:      -category:"Open Port"attributes:service:"http"state:openscanSpec:scanType:"nuclei"parameters:# Target domain name of the finding and start a nuclei scan      -"-u"      -"http://{{$.hostOrIP}}:{{attributes.port}}"---apiVersion:"execution.securecodebox.io/v1"kind:ScheduledScanmetadata:name:"nmap-local-network"spec:interval:30mscanSpec:scanType:"nmap"parameters:      --p80,8080      -"192.168.178.0/24"cascades:{}

The scan-deduplicator works via avalidating webhook configuration in Kubernetes.Before a scan is created in the cluster, kubernetes reaches out to the scan-deduplicator via a http request.The scan-deduplicator then checks if the scan was already executed recently.If it was executed recently, the webhook marks the scan as invalid and kubernetes will not create the scan.

Deploys the scan-deduplicator, including avalkey instance for a persistent cache.

NOTE: This requires the cluster to havecert-manger installed in the cluster to generate a tls certificate for the validating webhook.

kubectl create namespace scan-deduplicator||truekubectl create --namespace scan-deduplicator secret generic scan-deduplicator-cache-credentials --from-literal="password=$(uuidgen)"||truekubectl apply --namespace scan-deduplicator -f deploy/