Rustls: memory safety for TLS

This article brought to you by LWN subscribers

Subscribers to LWN.net made this article — and everything that surrounds it — possible. If you appreciate our content, pleasebuy a subscription and make the next set of articles possible.

The movement toward usingmemory-safelanguages, and Rust in particular, has picked up a lot of steam over the past year or two. Removing thepossibility of buffer overflows, use-after-free bugs, and other woes associatedwith unmanaged pointers is an attractive feature, especially given thatthe majority of today's vulnerabilities stem from memory-safetyissues. On April 20, theInternet Security ResearchGroup (ISRG)announceda funding initiative targeting theRustls TLS library in order toprepare it for more widespread adoption—including by ISRG'sLet's Encrypt project.

Google has provided the funds that allowed ISRG to contract Dirkjan Ochtmanto make some improvements to the library. Two of the items listedin the announcement are aimed at making Rustls integrate more easily withthe large body of C code in use today; most of those programs use the C-basedOpenSSL library for their TLS needs. As might be expected, ISRG and itsexecutive director, Josh Aas, who authored the announcement, are ratherexcited by the possibilities of Rust and Rustls going forward:

Rustls is an excellent alternative to OpenSSL and similar libraries. Muchof its critical code is written in Rust so it’s largely memory-safe withoutsacrificing performance. It hasbeenaudited and found to be a high quality implementation. Here’s one of our favorite lines from the report:

“Using the type system to statically encode properties such as the TLSstate transition function is one just one example of great defense-in-depthdesign decisions.”

[...] We currently live in a world where deploying a few million lines of Ccode on a network edge to handle requests is standard practice, despite allof the evidence we have that such behavior is unsafe. Our industry needs toget to a place where deploying code that isn’t memory safe to handlenetwork traffic is widely understood to be dangerous andirresponsible. People need memory safe software that suits their needs tobe available to them though, and that’s why we’re getting to work.

The Rustls project—the name is pronounced "rustles" incidentally—has beenaround since 2016, but the pace of releases has picked up over thelast year or so. It is based on thering cryptographic library,which is written in Rust, C, and assembly, and the relatedwebpki library forcertificate verification that is also written in Rust. The Rustls project ismaking a serious effort to leave mistakes in earlier TLS implementationsbehind (including outdated protocol versions and ciphers) and to forgeahead with a robust library that avoids many of the pitfalls that exist forcode of this nature, which handles untrusted input from the internet at large.

According to the announcement, ISRG has already started work on integratingRustlsinto curlandintothe Apache web server. For the latter, ISRG said that having a fullweb server in a memory-safe language would be beneficial, but would take anenormous effort; instead,allowing the TLS handling to be done by Rustls is a good incrementalstep. Google has provided funding for Stefan Eissing to dothe work on the web server, while ISRG is funding curl author and maintainer DanielStenberg to work on making curl more memory safe.Beyond allowing the use of Rustls, the curl refit is also adding support for thehyper Rust-based HTTP library. As can be seen, there is a growing ecosystem of memory-safelibraries for these kinds of tasks.

ISRG is targeting several different improvements to Rustls as part ofOchtman's work, including working toeliminate functions thatpanic, which effectively cause the program to crash. As the Rustdocumentationpointsout, usingpanic!() precludes callers from recovering from anerror, which is probably not appropriate in a lot of Rust code. Thepresence of panics in the Rust code making its way toward the Linuxkernel washighlighted as a problem byLinus Torvalds as part of his review. The Rustls project is working toward eliminatingpanics in its own code as well as eliminating calls into library routinesthat might panic, so that it provides a panic-free API for applications.

TheC API forRustls is also a focus of the work being sponsored. Since so much oftoday's TLS-using landscape is written in C, it is important for thelibrary to interface well with that language. The announcement mentionsmoving the API into the main Rustls repository, though theGitHub issue trackingthe plan shows no overwhelming preference for making that move. Thedevelopers do seem to be leaning in that direction, however.

Another improvement targeted by ISRG is support for IP addresses inSubjectAlternative Name (SAN) fields in certificates. SAN is an extension totheX.509 format used forpublic-key certificates (such as SSL/TLS certificates used by web sites).Some environments use IP addresses, rather than domain names, because thereis no DNS being used by the hosts on the network.The final feature listed is adding support for configuring the serverconnection based on information provided by the client. This is usedfor handling theServer NameIndication (SNI) andApplication-LayerProtocol Negotiation (ALPN) TLS extensions that allow clients toindicate their preferences to the server.

Handling user-controlled (or attacker-controlled) input coming across thenetwork is obviously one of the bigger areas of concern from a securityperspective. One of the earliest "attacks" on the internet, theMorris worm in 1988,relied in part on a buffer overflow in a network service; many other flawsof that sort have been seen in the decades since. Those kinds of problemscan be avoided by using a memory-safe language like Rust, though there willstill be opportunities for mistakes, as a look at the C API shows.

The mismatch between how C and Rust handle memory is still a potentialproblem area when the two languages are being used together. For example, the Rustls CAPI provides a mechanism toallocate andfree structures for use in calling into the library, but there are somesignificant caveats on their use. Programs need to ensure that only onethread is accessing the structure at any given time (by use of a mutex orother exclusion mechanism) and that the structure is freed only once.Those kinds of restrictions are not a surprise for C programmers, but theydo show the (probably obvious) limits of the memory-safety guarantees when callingRust from C.

Over the course of the next few years, we will get multiple opportunities to see howwell this hybrid approach to dealing with the lack of memory safety inC-language programs plays out. There is a lot of promise in the approach,and the Rust language has garnered a fair bit of optimism for being a majorpart of our, hopefully, memory-safe future. It will be interesting to seeif the reality is as grand as the vision—and, to a certain extent, hype.