Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

IPFIX flow exporter with DPDK support capable of bi-directional flows, per-packet-information statistics, and extensibility via processing plugins (e.g., for application layer parsers).

License

NotificationsYou must be signed in to change notification settings

CESNET/ipfixprobe

Repository files navigation


The ipfixprobe is a high-performance, modular flow exporter that processes packets into bidirectional flows and exports them via a selected output plugin.It supports a wide range of application-layer protocol parsers, including TLS, QUIC, HTTP, DNS, and many others. These protocol-specific extensions can be enabled via process plugins in the configuration.

Need more details? Check out ourdocumentation for a full list of supported protocols and usage examples.

Coverity ScanGitHub top language

✨ Key Features

  • Modular input–parser–output architecture
  • High-speed processing (DPDK, multi-threaded, NUMA-aware)
  • Built-in protocol parsers: TLS, QUIC, HTTP, DNS, …
  • Bidirectional flow (biflow) support
  • Real-time telemetry and statistics

📦 Installation

If you are running a RHEL system or one of its derivatives (e.g. Oracle Linux, Rocky Linux, CentOS Stream), the easiest way to installipfixprobe is from ourcopr repository.

$ dnf install dnf-plugins-core# Extra step necessary on some systems$ dnf coprenable @CESNET/ipfixprobe$ dnf install ipfixprobe

This installs the mainipfixprobe binary along with core functionality.

🗃️ Available Packages

The Copr repository provides modular RPM packages, so you can install only what you need.The following packages are available and can be installed individually as needed:

Package NameDescription
ipfixprobeCore binary with common process/output plugins.
ipfixprobe-msecCore binary with common process/output plugins. Uses millisecond timestamps (compatible with Flowmon collector)
ipfixprobe-input-pcapInput plugin for PCAP files and live capture
ipfixprobe-input-dpdkHigh-speed input plugin using DPDK
ipfixprobe-input-nfbInput plugin for CESNET NFB/NDP cards
ipfixprobe-process-experimentalExtra (possibly unstable) process plugins

For other systems, follow the build instructions below.

🛠️ Build

You can build ipfixprobe from source using standard CMake.This lets you customize the build by enabling optional plugins and features as needed.

Note: Some plugins may require additional dependencies beyond the basic requirements.

RHEL/CentOS:

🧰 Requirements

$ dnf install epel-release git make cmake gcc-c++ rpm-build$ dnf install libunwind-devel lz4-devel openssl-devel fuse3-devel# for RHEL 8/9$ dnf install gcc-toolset-14-libatomic-devel# for RHEL 10+$ dnf install libatomic

Debian/Ubuntu:

🧰 Requirements

$ apt install git make cmake g++ pkg-config rpm$ apt install libunwind-dev liblz4-dev libssl-dev libfuse3-dev libatomic1

🧱 Build steps

git clone https://github.com/CESNET/ipfixprobe.gitcd ipfixprobemkdir build&&cd buildcmake ..make -j$(nproc)# make install

⚙️ Optional build flags

You can enable or disable optional plugins and features via CMake flags:

FlagDefaultDescription
-DENABLE_MILLISECONDS_TIMESTAMP=ONOFFUse millisecond precision timestamps (for Flowmon compatibility)
-DENABLE_INPUT_PCAP=ONOFFEnable PCAP input plugin (live & file) (requireslibpcap)
-DENABLE_INPUT_DPDK=ONOFFEnable high-speed DPDK input plugin (requiresdpdk-devel)
-DENABLE_INPUT_NFB=ONOFFEnable input plugin for CESNET NFB/NDP cards (requiresnetcope-common)
-DENABLE_PROCESS_EXPERIMENTAL=ONOFFEnable experimental process plugins
-DENABLE_NEMEA=ONOFFEnable support for NEMEA modules (requiresnemea-framework-devel )

Run the command to view all available build options:

cmake -LAH

Example

To build with DPDK and PCAP input support, and install to /usr:

cmake .. \  -DCMAKE_INSTALL_PREFIX=/usr \  -DENABLE_INPUT_PCAP=ON \  -DENABLE_INPUT_DPDK=ON

🧩 Available Plugins

Input Plugins

List of input plugins with estimated performance and configuration complexity.

PluginMax ThroughputUsage ComplexityDescription
pcap_live~1 GbpsEasycaptures packets from a live network interface
pcap_file~1 GbpsEasyreads packets from an offline PCAP file
raw~1 GbpsEasycaptures packets using a raw socket
ndp400 GbpsMediumuses CESNET NFB/NDP hardware for packet input
dpdk400 GbpsComplexreceives packets via high-performance DPDK
dpdk-ring400 GbpsComplexreceives packets from a shared DPDK memory ring

Process Plugins

These plugins extract protocol-specific or behavioral information from packets and enrich flow records with metadata.

PluginDescription
basicextracts basic L3/L4 flow fields (IPs, ports, protocol)
icmpextracts ICMP type/code and related metadata
httpextracts HTTP methods, hosts, URIs, status codes
tlsextracts TLS handshake info (SNI, version, JA3, etc.)
ovpnextracts metadata from OpenVPN tunnels
wgparses WireGuard handshake and endpoint metadata
quicparses QUIC protocol including SNI, versions, ALPN
basicplusadds common L3/L4 flow fields (e.g., ports, IPs, TCP flags)
bstatsbasic flow statistics (packet/byte counters, duration, ...)
dnsextracts DNS queries, responses, and domains
dnssdparses DNS Service Discovery (mDNS) traffic
flowHashextracts a flow hash
idpContentparses IDP content in flows
mplsextracts MPLS labels and encapsulation metadata
mqttparses MQTT protocol traffic (IoT messaging)
netbiosextracts NetBIOS session and name service info
nettisaparses NETTISA related metadata (experimental)
ntpextracts NTP timestamps and server info
osqueryparses osquery-generated data streams
passiveDnsgenerates passive DNS entries from observed DNS traffic
phistsparses phishing-related signatures (heuristic)
pstatsadvanced packet statistics (e.g., inter-packet gaps)
rtspextracts RTSP stream metadata
sipparses SIP call setup, headers, and codecs
smtpextracts SMTP envelope data (from, to, subject, etc.)
ssaDetectorperforms simple anomaly detection based on traffic patterns
ssdpparses SSDP (UPnP discovery) protocol
vlanextracts VLAN IDs and QinQ encapsulation

Output Plugins

These plugins export flow records to various formats and external systems.

PluginDescription
ipfixexports flow records in IPFIX format to a remote collector (UDP/TCP)
textwrites flow records in human-readable text to a file or stdout
unirecexports flow records using the UniRec format for NEMEA/TRAP ecosystem

📘 Usage

Theipfixprobe processing pipeline can be launched directly or via theipfixprobed wrapper — a lightweight runtime designed to simplify plugin configuration using YAML files.This method is preferred for production deployments due to its clarity and flexibility.

✅ Recommended (YAML-based configuration)

/usr/bin/ipfixprobed<CONFIG_NAME> [LIBRARY_PATH]
  • <CONFIG_NAME> – Name of the YAML configuration file (without the .conf extension).The full path must be/etc/ipfixprobe/<CONFIG_NAME>.conf.

  • [LIBRARY_PATH] – (Optional) Path to the ipfixprobe plugin library directory.If not provided, the default/usr/lib64/ipfixprobe is used.

Example:

/usr/bin/ipfixprobed example

This will runipfixprobe using the YAML configuration from/etc/ipfixprobe/example.conf.

👉 See the full YAML configuration reference here:https://github.com/CESNET/ipfixprobe/blob/master/init/link0.conf.example

⚠️ Legacy usage (CLI parameters)

You may also run the processing pipeline using theipfixprobe binary directly, with CLI parameters.This method is not recommended for production use, as it lacks flexibility and clarity.

/usr/bin/ipfixprobe [OPTIONS]

Available options:

  • -i ARGS Activate input plugin (-h input for help)
  • -s ARGS Activate storage plugin (-h storage for help)
  • -o ARGS Activate output plugin (-h output for help)
  • -p ARGS Activate processing plugin (-h process for help)
  • -q SIZE Size of queue between input and storage plugins
  • -b SIZE Size of input queue packet block
  • -Q SIZE Size of queue between storage and output plugins
  • -B SIZE Size of packet buffer
  • -f NUM Export max flows per second
  • -c SIZE Quit after number of packets are processed on each interface
  • -P FILE Create a PID file
  • -t PATH Mount point of AppFs telemetry directory
  • -d Run as a standalone process
  • -h [PLUGIN] Print help text. Supported help for input, storage, output and process plugins
  • -V Show version and exit

📖 Examples

Below are practical examples showcasing common plugin configurations using both CLI and YAML formats.

🔹 Basic Interface Capture

Capture network traffic from thewlp2s0 interface using a raw socket. Flow records are printed in plain-text format to the console.

🧪 Command-line usage:

/usr/bin/ipfixprobe -i'raw;ifc=wlp2s0' -o'text'

📄 Equivalent YAML configuration:

input_plugin:raw:interface:'wlp2s0'output_plugin:text:{}

🔹 PCAP File → IPFIX Collector

Process packets from a.pcap file and export flows via IPFIX over UDP

🧪 Command-line usage:

/usr/bin/ipfixprobe -i'pcap;file=/data/capture.pcap' -o'ipfix;udp;host=collector.example.com;port=4739'

📄 Equivalent YAML configuration:

input_plugin:pcap_file:file:'/data/capture.pcap'output_plugin:ipfix:collector:host:collector.example.comport:4739protocol:udp:{}

🔹Live Capture with Cache Configuration and Telemetry

Capture frometh0 interface using libpcap, split biflows into uniflows and use active/inactive timeouts, print flows to console.Telemetry data are exposed via the appFs library in/var/run/ipfixprobe directory.

🧪 Command-line usage:

/usr/bin/ipfixprobe -i'pcap;ifc=eth0' -s'cache;split;active=300;inactive=60' -o'text' -t /var/run/ipfixprobe

📄 Equivalent YAML configuration:

input_plugin:pcap_live:interface:'eth0'storage:cache:{}timeouts:active:300inactive:60split_biflow:trueoutput_plugin:text:{}telemetry:appfs:enabled:truemount_point:/var/run/ipfixprobe

🔹 High-speed DPDK Capture with HTTP, TLS, and QUIC Processing

Capture packets using DPDK from port 0 with 2 queues bound to a specific PCI device (0000:17:00.0), enable HTTP, TLS, and QUIC process plugins, and export flows via IPFIX to a local collector at 127.0.0.1.

🧪 Command-line usage:

/usr/bin/ipfixprobe -i"dpdk;p=0;q=2;e=-a 0000:17:00.0" -i dpdk -p http -p tls -p quic -o"ipfix;h=127.0.0.1"

📄 Equivalent YAML configuration:

input_plugin:dpdk:allowed_nics:"0000:17:00.0"rx_queues:2process_plugins:  -http  -tls  -quicoutput_plugin:ipfix:collector:host:'127.0.0.1'port:4739protocol:udp:{}

📊 Telemetry

🧪 Testing & Validation

🧰 FAQ

About

IPFIX flow exporter with DPDK support capable of bi-directional flows, per-packet-information statistics, and extensibility via processing plugins (e.g., for application layer parsers).

Topics

Resources

License

Stars

Watchers

Forks

Contributors19


[8]ページ先頭

©2009-2025 Movatter.jp