Repository files navigation

nom is a parser combinators library written in Rust. Its goal is to provide toolsto build safe parsers without compromising the speed or memory consumption. Tothat end, it uses extensively Rust'sstrong typing andmemory safety to producefast and correct parsers, and provides functions, macros and traits to abstract most of theerror prone plumbing.

nom will happily take a byte out of your files :)

• Example
• Documentation
• Why use nom?
  • Binary format parsers
  • Text format parsers
  • Programming language parsers
  • Streaming formats
• Parser combinators
• Technical features
• Rust version requirements
• Installation
• Related projects
• Parsers written with nom
• Contributors

Hexadecimal color parser:

use nom::{  bytes::complete::{tag, take_while_m_n},  combinator::map_res,  sequence::Tuple,IResult,Parser,};#[derive(Debug,PartialEq)]pubstructColor{pubred:u8,pubgreen:u8,pubblue:u8,}fnfrom_hex(input:&str) ->Result<u8, std::num::ParseIntError>{  u8::from_str_radix(input,16)}fnis_hex_digit(c:char) ->bool{  c.is_digit(16)}fnhex_primary(input:&str) ->IResult<&str,u8>{map_res(take_while_m_n(2,2, is_hex_digit),    from_hex).parse(input)}fnhex_color(input:&str) ->IResult<&str,Color>{let(input, _) =tag("#")(input)?;let(input,(red, green, blue)) =(hex_primary, hex_primary, hex_primary).parse(input)?;Ok((input,Color{ red, green, blue}))}fnmain(){println!("{:?}", hex_color("#2F14DF"))}#[test]fnparse_color(){assert_eq!(    hex_color("#2F14DF"),Ok(("",Color{        red:47,        green:20,        blue:223,})));}

• Reference documentation
• The Nominomicon: A Guide To Using Nom
• Various design documents and tutorials
• List of combinators and their behaviour

If you need any help developing your parsers, please pinggeal on IRC (Libera, Geeknode, OFTC), go to#nom-parsers on Libera IRC, or on theGitter chat room.

If you want to write:

nom was designed to properly parse binary formats from the beginning. Comparedto the usual handwritten C parsers, nom parsers are just as fast, free frombuffer overflow vulnerabilities, and handle common patterns for you:

• TLV
• Bit level parsing
• Hexadecimal viewer in the debugging macros for easy data analysis
• Streaming parsers for network formats and huge files

Example projects:

• FLV parser
• Matroska parser
• tar parser

While nom was made for binary format at first, it soon grew to work just aswell with text formats. From line based formats like CSV, to more complex, nestedformats such as JSON, nom can manage it, and provides you with useful tools:

• Fast case insensitive comparison
• Recognizers for escaped strings
• Regular expressions can be embedded in nom parsers to represent complex character patterns succinctly
• Special care has been given to managing non ASCII characters properly

Example projects:

• HTTP proxy
• TOML parser

While programming language parsers are usually written manually for moreflexibility and performance, nom can be (and has been successfully) usedas a prototyping parser for a language.

nom will get you started quickly with powerful custom error types, that youcan leverage withnom_locate topinpoint the exact line and column of the error. No need for separatetokenizing, lexing and parsing phases: nom can automatically handle whitespaceparsing, and construct an AST in place.

Example projects:

• PHP VM
• xshade shading language

While a lot of formats (and the code handling them) assume that they can fitthe complete data in memory, there are formats for which we only get a partof the data at once, like network formats, or huge files.nom has been designed for a correct behaviour with partial data: If there isnot enough data to decide, nom will tell you it needs more instead of silentlyreturning a wrong result. Whether your data comes entirely or in chunks, theresult should be the same.

It allows you to build powerful, deterministic state machines for your protocols.

Example projects:

• HTTP proxy
• Using nom with generators

Parser combinators are an approach to parsers that is very different fromsoftware likelex andyacc. Instead of writing the grammarin a separate file and generating the corresponding code, you use verysmall functions with very specific purpose, like "take 5 bytes", or"recognize the word 'HTTP'", and assemble them in meaningful patternslike "recognize 'HTTP', then a space, then a version".The resulting code is small, and looks like the grammar you would havewritten with other parser approaches.

This has a few advantages:

• The parsers are small and easy to write
• The parsers components are easy to reuse (if they're general enough, please add them to nom!)
• The parsers components are easy to test separately (unit tests and property-based tests)
• The parser combination code looks close to the grammar you would have written
• You can build partial parsers, specific to the data you need at the moment, and ignore the rest

nom parsers are for:

• byte-oriented: The basic type is&[u8] and parsers will work as much as possible on byte array slices (but are not limited to them)
• bit-oriented: nom can address a byte slice as a bit stream
• string-oriented: The same kind of combinators can apply on UTF-8 strings as well
• zero-copy: If a parser returns a subset of its input data, it will return a slice of that input, without copying
• streaming: nom can work on partial data and detect when it needs more data to produce a correct result
• descriptive errors: The parsers can aggregate a list of error codes with pointers to the incriminated input slice. Those error lists can be pattern matched to provide useful messages.
• custom error types: You can provide a specific type to improve errors returned by parsers
• safe parsing: nom leverages Rust's safe memory handling and powerful types, and parsers are routinely fuzzed and tested with real world data. So far, the only flaws found by fuzzing were in code written outside of nom
• speed: Benchmarks have shown that nom parsers often outperform many parser combinators library like Parsec and attoparsec, some regular expression engines and even handwritten C parsers

Some benchmarks are available onGitHub.

The 8.0 series of nom supportsRustc version 1.65 or greater.

The current policy is that this will only be updated in the next major nom release.

nom is available oncrates.io and can be included in your Cargo enabled project like this:

[dependencies]nom ="8"

There are a few compilation features:

• alloc: (activated by default) if disabled, nom can work inno_std builds without memory allocators. If enabled, combinators that allocate (likemany0) will be available
• std: (activated by default, activatesalloc too) if disabled, nom can work inno_std builds

You can configure those features like this:

[dependencies.nom]version ="8"default-features =falsefeatures = ["alloc"]

• Get line and column info in nom's input type
• Using nom as lexer and parser

Here is a (non exhaustive) list of known projects using nom:

• Text file formats:Ceph Crush,Cronenberg,Email,XFS Runtime Stats,CSV,FASTA,FASTQ,INI,ISO 8601 dates,libconfig-like configuration file format,Web archive,PDB,proto files,Fountain screenplay markup,vimwiki,vimwiki_macros,Kconfig language,Askama templates,LP files
• Programming languages:PHP,Basic Calculator,GLSLLua,Python,SQL,Elm,SystemVerilog,Turtle,CSML,Wasm,Pseudocode,Filter for MeiliSearch,PotterScript,R
• Interface definition formats:Thrift
• Audio, video and image formats:GIF,MagicaVoxel .vox,MIDI,SWF,WAVE,Matroska (MKV),Exif/Metadata parser for JPEG/HEIF/HEIC/MOV/MP4
• Document formats:TAR,GZ,GDSII
• Cryptographic formats:X.509
• Network protocol formats:Bencode,D-Bus,DHCP,HTTP,URI,IMAP (alt),IRC,Pcap-NG,Pcap,Pcap + PcapNG,IKEv2,NTP,SNMP,Kerberos v5,DER,TLS,V5, V7, V9, IPFIX / Netflow v10,GTP,SIP,SMTP,Prometheus,DNS
• Language specifications:BNF
• Misc formats:Game Boy ROM,ANT FIT,Version Numbers,Telcordia/Bellcore SR-4731 SOR OTDR files,MySQL binary log,URI,Furigana,Wordle Result,NBT

Want to create a new parser usingnom? A list of not yet implemented formats is availablehere.

Want to add your parser here? Create a pull request for it!

nom is the fruit of the work of many contributors over the years, many thanks for your help!