Repository files navigation

👉Note: this is a new crate that reuses an old name.The old crate (0.3.0 and lower) has a bunch of problems.Make sure to use the new crate, currently in alpha at1.0.0-alpha.23.

CommonMark compliant markdown parser in Rust with ASTs and extensions.

• compliant (100% to CommonMark)
• extensions (100% GFM, 100% MDX, frontmatter, math)
• safe (100% safe Rust, also 100% safe HTML by default)
• robust (2300+ tests, 100% coverage, fuzz testing)
• ast (mdast)

• If youjust want to turn markdown into HTML (with maybe a few extensions)
• If you want to doreally complex things with markdown

markdown-rs is an open source markdown parser written in Rust.It’s implemented as a state machine (#![no_std] +alloc) that emitsconcrete tokens, so that every byte is accounted for, with positional info.The API then exposes this information as an AST, which is easier to work with,or it compiles directly to HTML.

While most markdown parsers work towards compliancy with CommonMark (or GFM),this project goes further by following how the reference parsers (cmark,cmark-gfm) work, which is confirmed with thousands of extra tests.

Other than CommonMark and GFM, this project also supports common extensionsto markdown such as MDX, math, and frontmatter.

This Rust crate has a sibling project in #"https://github.com/micromark/micromark">micromark(andmdast-util-from-markdown for the AST).

P.S. if you want tocompile MDX, usemdxjs-rs.

• to learn markdown, see thischeatsheet and tutorial
• for the API, see thecrate docs
• for questions, seeDiscussions
• to help, seecontribute orsponsor below

• Install
• Use
• API
• Extensions
• Project
  • Overview
  • File structure
  • Test
  • Version
  • Security
  • Contribute
  • Sponsor
  • Thanks
• Related
• License

WithRust (rust edition 2018+, ±version 1.56+), install withcargo:

cargo add markdown@1.0.0-alpha.23

👉Note: this is a new crate that reuses an old name.The old crate (0.3.0 and lower) has a bunch of problems.Make sure to use the new crate, currently in alpha at1.0.0-alpha.23.

fnmain(){println!("{}", markdown::to_html("## Hello, *world*!"));}

Yields:

<h2>Hello,<em>world</em>!</h2>

Extensions (in this case GFM):

fnmain() ->Result<(), markdown::message::Message>{println!("{}",        markdown::to_html_with_options("* [x] contact@example.com ~~strikethrough~~",&markdown::Options::gfm())?);Ok(())}

Yields:

<ul><li><inputchecked=""disabled=""type="checkbox"/><ahref="mailto:contact@example.com">contact@example.com</a><del>strikethrough</del></li></ul>

Syntax tree (mdast):

fnmain() ->Result<(), markdown::message::Message>{println!("{:?}",        markdown::to_mdast("# Hey, *you*!",&markdown::ParseOptions::default())?);Ok(())}

Yields:

Root { children: [Heading { children: [Text { value: "Hey, ", position: Some(1:3-1:8 (2-7)) }, Emphasis { children: [Text { value: "you", position: Some(1:9-1:12 (8-11)) }], position: Some(1:8-1:13 (7-12)) }, Text { value: "!", position: Some(1:13-1:14 (12-13)) }], position: Some(1:1-1:14 (0-13)), depth: 1 }], position: Some(1:1-1:14 (0-13)) }

markdown-rs exposesto_html,to_html_with_options,to_mdast,Options,and a few other structs and enums.

See thecrate docs for more info.

markdown-rs supports extensions toCommonMark.These extensions are maintained in this project.They are not enabled by default but can be turned on with options.

• frontmatter
• GFM
  • autolink literal
  • footnote
  • strikethrough
  • table
  • tagfilter
  • task list item
• math
• MDX
  • ESM
  • expressions
  • JSX

It is not a goal of this project to support lots of different extensions.It’s instead a goal to support very common and mostly standardized extensions.

markdown-rs is maintained as a single monolithic crate.

The process to parse markdown looks like this:

                    markdown-rs+-------------------------------------------------+|            +-------+         +---------+--html-|| -markdown->+ parse +-events->+ compile +||            +-------+         +---------+-mdast-|+-------------------------------------------------+

The files insrc/ are as follows:

• construct/*.rs— CommonMark, GFM, and other extension constructs used in markdown
• util/*.rs— helpers often needed when parsing markdown
• event.rs— things with meaning happening somewhere
• lib.rs— public API
• mdast.rs— syntax tree
• parser.rs— turn a string of markdown into events
• resolve.rs— steps to process events
• state.rs— steps of the state machine
• subtokenize.rs— handle content in other content
• to_html.rs— turns events into a string of HTML
• to_mdast.rs— turns events into a syntax tree
• tokenizer.rs— glue the states of the state machine together
• unist.rs— point and position, used in mdast

markdown-rs is tested with the ~650 CommonMark tests and more than 1k extratests confirmed with CM reference parsers.Then there’s even more tests for GFM and other extensions.These tests reach all branches in the code, which means that this project has100% code coverage.Fuzz testing is used to check for things that might fall through coverage.

The following bash scripts are useful when working on this project:

• generate code (latest CM tests and Unicode info):

  cargo run --manifest-path generate/Cargo.toml

• run examples:

  RUST_BACKTRACE=1 RUST_LOG=trace cargo run --example lib --features log

• format:

  cargo fmt&& cargo fix --all-features --all-targets --workspace

• lint:

  cargo fmt --check&& cargo clippy --all-features --all-targets --workspace

• test:

  RUST_BACKTRACE=1 cargotest --all-features --workspace

• docs:

  cargo doc --document-private-items --examples --workspace

• fuzz:

  cargo install cargo-fuzzcargo install honggfuzzcargo +nightly fuzz run markdown_libfuzzcargo hfuzz run markdown_honggfuzz

markdown-rs followsSemVer.

The typical security aspect discussed for markdown iscross-site scripting(XSS) attacks.Markdown itself is safe if it does not include embedded HTML or dangerousprotocols in links/images (such as#"auto">Turning on theallow_dangerous_html orallow_dangerous_protocol options foruser-provided markdown opens you up to XSS attacks.

Additionnally, you should be able to setallow_any_img_src safely.The default is to allow onlyhttp:,https:, and relative images,which is what GitHub does. But it should be safe to allow any value onsrc.

TheHTML specification prohibits dangerous scripts inimages and all modern browsers respect this and are thus safe.Opera 12 (from 2012) is a notable browser that did not respect this.

An aspect related to XSS for security is syntax errors: markdown itself has nosyntax errors.Some syntax extensions (specifically, only MDX) do include syntax errors.For that reason,to_html_with_options returnsResult<String, Message>, ofwhich the error is a struct indicating where the problem happened, whatoccurred, and what was expected instead.Make sure to handle your errors when using MDX.

Another security aspect is DDoS attacks.For example, an attacker could throw a 100mb file atmarkdown-rs, in whichcase it’s going to take a long while to finish.It is also possible to crashmarkdown-rs with smaller payloads, notably whenthousands of links, images, emphasis, or strong are opened but not closed.It is wise to cap the accepted size of input (500kb can hold a big book) and toprocess content in a different thread so that it can be stopped when needed.

For more information on markdown sanitation, seeimproper-markup-sanitization.md by@chalker.

Seecontributing.md for ways to help.Seesupport.md for ways to get help.Seecode-of-conduct.md for how to communicate in and around thisproject.

Support this effort and give back by sponsoring:

• GitHub Sponsors(personal; monthly or one-time)
• OpenCollective orGitHub Sponsors(unified; monthly or one-time)

Special thanks go out to:

• Vercel for funding the initial development
• @Murderlon for the design of the logo
• @johannhof for the crate name

• micromark— same asmarkdown-rs but in JavaScript
• mdxjs-rs— wrapsmarkdown-rs tocompile MDX to JavaScript

MIT ©Titus Wormer