- Notifications
You must be signed in to change notification settings - Fork0
A Rust crate to detect, normalize, and convert line endings across platforms. Ensures consistent handling of LF, CRLF, and CR line endings in text processing.
License
jzombie/rust-line-ending
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
| OS | Status |
|---|---|
| Ubuntu-latest | &f=jpg&w=240) |
| macOS-latest | &f=jpg&w=240) |
| Windows-latest | &f=jpg&w=240) |
A Rust crate to detect, normalize, and convert line endings across platforms, including support for character streams. Ensures consistent handling ofLF,CRLF, andCR line endings in text processing.
cargo add line-ending
Line endings can be auto-detected for the current platform.
use line_ending::LineEnding;let detected =LineEnding::from_current_platform();#[cfg(target_os ="windows")]assert_eq!(detected,LineEnding::CRLF,"Windows should detect CRLF");#[cfg(target_family ="unix")]assert_eq!(detected,LineEnding::LF,"Unix/macOS should detect LF");#[cfg(target_family ="wasm")]assert_eq!(detected,LineEnding::LF,"WASM should default to LF");
Split a string into a vector of strings using the auto-detected line ending parsed from the string.
use line_ending::LineEnding;let crlf =LineEnding::split("first\r\nsecond\r\nthird");let cr =LineEnding::split("first\rsecond\rthird");let lf =LineEnding::split("first\nsecond\nthird");let expected =vec!["first","second","third"];assert_eq!(crlf, expected);assert_eq!(cr, expected);assert_eq!(lf, expected);
Join a vector of strings using the specified line ending.
use line_ending::LineEnding;let lines =vec!["first".to_string(),"second".to_string(),"third".to_string(),];assert_eq!(LineEnding::CRLF.join(lines.clone()),"first\r\nsecond\r\nthird");assert_eq!(LineEnding::CR.join(lines.clone()),"first\rsecond\rthird");assert_eq!(LineEnding::LF.join(lines.clone()),"first\nsecond\nthird");
Apply a specific line ending type to an existing string.
use line_ending::LineEnding;let mixed_text ="first line\r\nsecond line\rthird line\nfourth line\n";assert_eq!(LineEnding::CRLF.apply(mixed_text),"first line\r\nsecond line\r\nthird line\r\nfourth line\r\n");assert_eq!(LineEnding::CR.apply(mixed_text),"first line\rsecond line\rthird line\rfourth line\r");assert_eq!(LineEnding::LF.apply(mixed_text),"first line\nsecond line\nthird line\nfourth line\n");
Detect the predominant line ending style used in the input string.
use line_ending::LineEnding;let crlf ="first line\r\nsecond line\r\nthird line";let cr ="first line\rsecond line\rthird line";let lf ="first line\nsecond line\nthird line";assert_eq!(LineEnding::from(crlf),LineEnding::CRLF);assert_eq!(LineEnding::from(cr),LineEnding::CR);assert_eq!(LineEnding::from(lf),LineEnding::LF);
Convert all line endings in a string toLF (\n) for consistent processing.
use line_ending::LineEnding;let crlf ="first\r\nsecond\r\nthird";let cr ="first\rsecond\rthird";let lf ="first\nsecond\nthird";assert_eq!(LineEnding::normalize(crlf), lf);assert_eq!(LineEnding::normalize(cr), lf);assert_eq!(LineEnding::normalize(lf), lf);
Restore line endings in a string to the specified type.
use line_ending::LineEnding;let lf ="first\nsecond\nthird";let crlf_restored =LineEnding::CRLF.denormalize(lf);let cr_restored =LineEnding::CR.denormalize(lf);let lf_restored =LineEnding::LF.denormalize(lf);assert_eq!(crlf_restored,"first\r\nsecond\r\nthird");assert_eq!(cr_restored,"first\rsecond\rthird");assert_eq!(lf_restored,"first\nsecond\nthird");
When a string contains multiple types of line endings (LF,CRLF, andCR), theLineEnding::from method will detect the most frequent line ending type and return it as the dominant one. This ensures a consistent approach to mixed-line-ending detection.
use line_ending::LineEnding;let mixed_type ="line1\nline2\r\nline3\nline4\nline5\r\n";assert_eq!(LineEnding::from(mixed_type),LineEnding::LF);// `LF` is the most common
The detection algorithm works as follows:
- Counts occurrences of each line ending type (
LF,CRLF,CR). - Selects the most frequent one as the detected line ending.
- Defaults to
CRLFif all are equally present or if the input is empty.
use line_ending::LineEnding;let mostly_crlf ="line1\r\nline2\r\nline3\nline4\r\nline5\r\n";assert_eq!(LineEnding::from(mostly_crlf),LineEnding::CRLF);// `CRLF` is the most commonlet mostly_cr ="line1\rline2\rline3\nline4\rline5\r";assert_eq!(LineEnding::from(mostly_cr),LineEnding::CR);// `CR` is the most common
IfLF,CRLF, andCR all appear the same number of times, the function will returnCRLF as a tie-breaker.
use line_ending::LineEnding;let equal_mixed ="line1\r\nline2\nline3\rline4\r\nline5\nline6\r";assert_eq!(LineEnding::from(equal_mixed),LineEnding::CRLF);// `CRLF` > `CR` > `LF`
CRLF is chosen as a tie-breaker because it represents bothCR andLF, making it the most inclusive option.
If a single line contains different line endings, the function still chooses the most frequent across the entire string.
use line_ending::LineEnding;let mixed_on_one_line ="line1\r\nline2\rline3\r\nline4\r\nline5\r";assert_eq!(LineEnding::from(mixed_on_one_line),LineEnding::CRLF);// `CRLF` appears the most overall
use line_ending::LineEnding;let empty_text ="";assert_eq!(LineEnding::from(empty_text),LineEnding::CRLF);// Defaults to `CRLF`
Count occurrences of each line ending type in the given string.
use line_ending::{LineEnding,LineEndingScores};// `LineEndingScores` is a hash map that associates each line ending type with// its occurrence count.let mostly_lf ="line1\nline2\r\nline3\rline4\nline5\nline6\n";assert_eq!(LineEnding::from(mostly_lf),LineEnding::LF);assert_eq!(LineEnding::score_mixed_types(mostly_lf,),[(LineEnding::CRLF,1),(LineEnding::CR,1),(LineEnding::LF,4),].into_iter().collect::<LineEndingScores>());
If you want to forcefully split by a certain type.
use line_ending::{LineEnding};let mostly_lf ="line1\nline2\r\nline3\rline4\nline5\nline6\n";let split_crlf =LineEnding::CRLF.split_with(mostly_lf);assert_eq!(split_crlf, vec!["line1\nline2","line3\rline4\nline5\nline6\n"]);
When processing text from a stream (for example, when reading from a file), you often work with aPeekable iterator over characters. Manually checking for a newline (such as '\n') isn’t enough to handle all platforms, because Windows uses a two‑character sequence (\r\n) and some older systems use just\r.
This crate provides a trait extension (via thePeekableLineEndingExt trait) that adds a consume_line_ending() method to aPeekable<Chars> iterator. This method automatically detects and consumes the full line break sequence (whether it’s LF, CR, or CRLF) from the stream.
The following example demonstrates how to split a character stream into lines without having to manually handle each line-ending case:
use line_ending::{LineEnding,PeekableLineEndingExt};let text ="line1\r\nline2\nline3\rline4";letmut it = text.chars().peekable();letmut lines =Vec::new();letmut current_line =String::new();while it.peek().is_some(){// consume_line_ending() will automatically consume the full line break (CR, LF, or CRLF)if it.consume_line_ending().is_some(){ lines.push(current_line); current_line =String::new();}else{ current_line.push(it.next().unwrap());}}lines.push(current_line);assert_eq!(lines, vec!["line1","line2","line3","line4"]);
Note: Mixed-type line-ending character streams are automatically handled.
Rust treats\\n as a literal sequence rather than an actual newline. This behavior ensures that escaped sequences are not mistakenly interpreted as real line breaks.
For example:
use line_ending::LineEnding;let lf_with_escaped ="First\\nSecond\nThird";let result =LineEnding::split(lf_with_escaped);assert_eq!(result, vec!["First\\nSecond","Third"]);// Escaped `\\n` remains intactlet lf ="First\nSecond\nThird";let result_actual =LineEnding::split(lf);assert_eq!(result_actual, vec!["First","Second","Third"]);// Actual `\n` splits
Licensed underMIT. SeeLICENSE for details.
About
A Rust crate to detect, normalize, and convert line endings across platforms. Ensures consistent handling of LF, CRLF, and CR line endings in text processing.
Topics
Resources
License
Uh oh!
There was an error while loading.Please reload this page.
Stars
Watchers
Forks
Releases
Packages0
Uh oh!
There was an error while loading.Please reload this page.