Repository files navigation

Concise data definition language (RFC 8610) parser implementation in Python.

CDDL expresses Concise Binary Object Representation (CBOR) data structures (RFC 7049). Its main goal is to provide an easy and unambiguous way to express structures for protocol messages and data formats that use CBOR or JSON.

This Python implementation provides a CDDL parser suitable for producing marked up serializations of the CDDL. It is intended to be used in spec authoring tools to add cross-referencing logic within CDDL blocks.

Clone the repository:

git clone https//github.com/tidoust/cddlparser

Runcddlparser.py, passing in the path to a CDDL file as parameter

python cddlparser.py tests/__fixtures__/example.cddl

That should print a serialization of theAbstract Syntax Tree (AST) produced by the parser, followed by a re-serialization of the AST, which should match the original file.

The parser is available as aPypi package. To install the package, run:

pip install cddlparser

You should then be able to write code such as:

fromcddlparserimportparseast=parse('''  person = {      identity,                         ; an identity      employer: tstr,                   ; some employer  }''')print('The Abstract syntax tree:')print(ast)print()print('Re-serialization:')print(ast.serialize())

To create markup during serialization, you need to pass an object that subclasses theMarker class (see inline notes for a bit of documentation).

fromcddlparserimportparsefromcddlparser.astimportCDDLNode,Marker,Markup,RuleclassStrongNameMarker(Marker):defserializeName(self,name:str,node:CDDLNode)->str:return'<b>'+name+'</b>'defmarkupFor(self,node:CDDLNode)->Markup:ifisinstance(node,Rule):return ('<div>','</div>')returnsuper().markupFor(node)ast=parse('''person = {  identity,  employer: tstr,}''')print(ast.serialize(StrongNameMarker()))

This should produce:

<divclass="rule"><b>person</b> = {<b>identity</b>,<b>employer</b>:<b>tstr</b>,}</div>

You may run tests from a local copy of the code:

python tests.py

Parser tests compare the AST produced by the parser with a serialized snapshot of the expected AST. If you make changes to the parser and need to refresh a snapshot, delete the correspondingtests/__snapshots__/[test].snap file and run tests again.

Parser tests also compare the result of serializing the AST with the initial input.

The test files are a combination of the test files used in the other CDDL parser projects mentioned:

• Test files from cddl-rs.
• Test files from cddl, with a couple of fixes.

The code uses static types. To validate types and code, installmypy,ruff,black andpylint if not already done and run:

mypyruff checkblack.pylint cddlparser

• Updates to the CDDL grammar defined inRFC 9862 are not supported.
• The parser validates the CDDL syntax against the CDDL grammar. It also validates that there are no obvious type/group inconsistencies. The parser does not validate the CDDL beyond that. For example, the parser does not choke if two rules have the same name but define different types.
• The only logic that exists in the AST for now is the serialization logic. There are no facilities to import CDDL modules, resolve references, inline groups, validate CBOR, etc.
• The parser does not fully understand when a rule defines a type and when it defines a group. It may represent the right hand side of a type definition as aGroupEntry node, instead of as aType node.
• Overall, the AST is verbose and could be simplified.

Thiscddlparser Python module merely came into existence because I needed a CDDL parser in Python that I could leverage to add CDDL support inBikeshed (not done yet!) and could not find any. I took inspiration from existing CDDL parsers written in other languages:

• cddl: a JavaScript implementation of a CDDL parser for Node.js, released under an MIT license, written by @christian-bromann.cddlparser started as a direct port of the JavaScript code, and the lexer remains similar to the JavaScript one. Testing structures and main test files also come fromcddl. The parser incddlparser is completely different though, given the need to preserve the original formatting (including whitespaces and comments) to re-serialize the AST back into a string.
• cddl-rs: a Rust implementation of a CDDL parser, released under an MIT license, written by @anweiss, that features a CDDL validator. The parser incddlparser follows a similar "close to the CDDL grammar" logic. Thecddlparser test suite also contains test files from thecddl-rs project.
• cddlc: A set of CDDL utilities written in Ruby by @cabo, along with CDDL extracts from IETF RFCs. Thecddlparser test suite makes sure that CDDL extracts in thecddlc repository can be parsed and serialized again.