Contributing to coreutils

Hi! Welcome to uutils/coreutils!

Thanks for wanting to contribute to this project! This document explainseverything you need to know to contribute. Before you start make sure to alsocheck out these documents:

• Our community'sCODE_OF_CONDUCT.md.
• DEVELOPMENT.md for setting up your developmentenvironment.

Now follows a very important warning:

[!WARNING]uutils is original code and cannot contain any code from GNU orother implementations. This means thatwe cannot accept any changes based onthe GNU source code. To make sure that cannot happen,you cannot link tothe GNU source code either. It is however possible to look at other implementationsunder a BSD or MIT license likeApple's implementationorOpenBSD.

Finally, feel free to join ourDiscord!

Getting Oriented

uutils is a big project consisting of many parts. Here are the most importantparts for getting started:

• src/uu: The code for all utilities
• src/uucore: Crate containing all the shared code betweenthe utilities.
• tests/by-util: The tests for all utilities.
• src/bin/coreutils.rs: Code for the multicallbinary.
• docs: the documentation for the website
• tests/uutests/:Crate implementing the various functions to test uutils commands.

Each utility is defined as a separate crate. The structure of each of thesecrates is as follows:

• Cargo.toml
• src/main.rs: contains only a single macro call
• src/<util name>.rs: the actual code for the utility
• <util name>.md: the documentation for the utility

We have separated repositories for crates that we maintain but also publish foruse by others:

• uutils-term-grid
• parse_datetime

Design Goals

We have the following goals with our development:

• Compatible: The utilities should be a drop-in replacement for the GNUcoreutils.
• Cross-platform: All utilities should run on as many of the supportedplatforms as possible.
• Reliable: The utilities should never unexpectedly fail.
• Performant: Our utilities should be written in fast idiomatic Rust. We aimto match or exceed the performance of the GNU utilities.
• Well-tested: We should have a lot of tests to be able to guaranteereliability and compatibility.

How to Help

There are several ways to help and writing code is just one of them. Reportingissues and writing documentation are just as important as writing code.

Reporting Issues

We can't fix bugs we don't know about, so good issues are super helpful! Hereare some tips for writing good issues:

• If you find a bug, make sure it's still a problem on themain branch.
• Search through the existing issues to see whether it has already beenreported.
• Make sure to include all relevant information, such as:
  • Which version of uutils did you check?
  • Which version of GNU coreutils are you comparing with?
  • What platform are you on?
• Provide a way to reliably reproduce the issue.
• Be as specific as possible!

Writing Documentation

There's never enough documentation. If you come across any documentation thatcould be improved, feel free to submit a PR for it!

Writing Code

If you want to submit a PR, make sure that you've discussed the solution withthe maintainers beforehand. We want to avoid situations where you put a lot ofwork into a fix that we can't merge! If there's no issue for what you're tryingto fix yet, make onebefore you start working on the PR.

Generally, we try to follow what GNU is doing in terms of options and behavior.It is recommended to look at the GNU coreutils manual(on the web,or locally usinginfo <utility>). It is more in depth than the man pages andprovides a good description of available features and their implementationdetails. But remember, you cannot look at the GNU source code!

Also remember that we can only merge PRs which pass our test suite, followrustfmt, and do not have any warnings from clippy. SeeDEVELOPMENT.md for more information. Be sure to also readabout ourRust style.

Our Rust Style

We want uutils to be written in idiomatic Rust, so here are some guidelines tofollow. Some of these are aspirational, meaning that we don't do them correctlyeverywhere in the code. If you find violations of the advice below, feel free tosubmit a patch!

Don'tpanic!

The coreutils should be very reliable. This means that we should neverpanic!.Therefore, you should avoid using.unwrap() andpanic!. Sometimes the use ofunreachable! can be justified with a comment explaining why that code isunreachable.

Don'texit

We want uutils to be embeddable in other programs. This means that no functionin uutils should exit the program. Doing so would also lead to code with moreconfusing control flow. Avoid thereforestd::process::exit and similarfunctions which exit the program early.

unsafe

uutils cannot be entirely safe, because we have to call out tolibc and dosyscalls. However, we still want to limit our use ofunsafe. We generally onlyacceptunsafe for FFI, with very few exceptions. Note that performance is veryrarely a valid argument for usingunsafe.

If you still need to write code withunsafe, make sure to read theRustonomicon and annotate thecalls with// SAFETY: comments explaining why the use ofunsafe is sound.

Macros

Macros can be a great tool, but they are also usually hard to understand. Theyshould be used sparingly. Make sure to explore simpler options before you reachfor a solution involving macros.

str,OsStr &Path

Rust has many string-like types, and sometimes it's hard to choose the rightone. It's tempting to usestr (andString) for everything, but that is notalways the right choice for uutils, because we need to support invalid UTF-8,just like the GNU coreutils. For example, paths on Linux might not be validUTF-8! Whenever we are dealing with paths, we should therefore stick withOsStr andPath. Make sure that you only convert tostr/String if youknow that something is always valid UTF-8. If you need more operations onOsStr, you can use thebstr crate.

Doc-comments

We use rustdoc for our documentation, so it's best to followrustdoc's guidelines.Make sure that your documentation is not just repeating the name of thefunction, but actually giving more useful information. Rustdoc recommends thefollowing structure:

[short sentence explaining what it is][more detailed explanation][at least one code example that users can copy/paste to try it][even more advanced explanations if necessary]

Other comments

Comments should be written toexplain the code, not todescribe the code.Try to focus on explainingwhy the code is the way it is. If you feel like youhave to describe the code, that's usually a sign that you could improve thenaming of variables and functions.

If you edit a piece of code, make sure to update any comments that need tochange as a result. The only thing worse than having no comments is havingoutdated comments!

Git Etiquette

To ensure easy collaboration, we have guidelines for using Git and GitHub.

Commits

• Make small and atomic commits.
• Keep a clean history of commits.
• Write informative commit messages.
• Annotate your commit message with the component you're editing. For example:cp: do not overwrite on with -i oruucore: add support for FreeBSD.
• Do not unnecessarily move items around in the code. This makes the changesmuch harder to review. If you do need to move things around, do that in aseparate commit.

Commit messages

You can readthis section in the Git book to learn how to write good commitmessages.

In addition, here are a few examples for a summary line when committing touutils:

• commit for a single utility

nohup: cleanup and refactor

• commit for a utility's tests

tests/rm: test new feature

Beyond changes to an individual utility or its tests, other summary lines fornon-utility modules include:

README: add helpuucore: add new modulesuutils: add new utilitygitignore: add temporary files

PRs

• Make the titles of PRs descriptive.
  • This means describing the problem you solve. For example, do not writeFix #1234, butls: fix version sort order.
  • You can prefix the title with the utility the PR concerns.
• Keep PRs small and self-contained. A set of small PRs is much more likely toget merged quickly than one large PR.
• Make sure the CI passes (up to intermittently failing tests).
• You know your code best, that's why it's best if you can solve merge conflictson your branch yourself.
  • It's up to you whether you want to usegit merge main orgit rebase main.
  • Feel free to ask for help with merge conflicts.
• You do not need to ping maintainers to request a review, but it's fine to doso if you don't get a response within a few days.

Platforms

We take pride in supporting many operating systems and architectures. Any codeyou contribute must at least compile without warnings for all platforms in theCI. However, you can use#[cfg(...)] attributes to create platform dependentfeatures.

Tip: For Windows, Microsoft provides some images (VMWare, Hyper-V,VirtualBox and Parallels) for developmenton their official download page.

Improving the GNU compatibility

Please make sure you have installedGNU utils and prerequisites andcan execute commands described inComparing with GNU section ofDEVELOPMENT.md

The Python script./util/remaining-gnu-error.py shows the list of failingtests in the CI.

To improve the GNU compatibility, the following process is recommended:

1. Identify a test (the smaller, the better) on a program that you understand oris easy to understand. You can use the./util/remaining-gnu-error.py scriptto help with this decision.
2. Build both the GNU and Rust coreutils using:bash util/build-gnu.sh
3. Run the test withbash util/run-gnu-test.sh <your test>
4. Start to modify<your test> to understand what is wrong. Examples:
   1. Addset -v to have the bash verbose mode
   2. Addecho $? where needed
   3. When the variablefail is used in the test,echo $fail to see when thetest started to fail
   4. Bump the content of the output (ex:cat err)
   5. ...
5. Or, if the test is simple, extract the relevant information to create a newtest case running both GNU & Rust implementation
6. Start to modify the Rust implementation to match the expected behavior
7. Add a test to make sure that we don't regress (our test suite is super quick)

Code coverage

To generate code coverage report locally please followCode coverage report section ofDEVELOPMENT.md

Other implementations

The Coreutils have different implementations, with different levels ofcompletions:

• GNU's
• OpenBSD
• Busybox
• Toybox (Android)
• Mac OS
• V lang
• SerenityOS
• Initial Unix
• Perl Power Tools

However, when reimplementing the tools/options in Rust, don't read their sourcecodes when they are using reciprocal licenses (ex: GNU GPL, GNU LGPL, etc).

Licensing

uutils is distributed under the terms of the MIT License; see theLICENSE filefor details. This is a permissive license, which allows the software to be usedwith few restrictions.

Copyrights in the uutils project are retained by their contributors, and nocopyright assignment is required to contribute.

If you wish to add or change dependencies as part of a contribution to theproject, a tool likecargo-license can be used to show their license details.The following types of license are acceptable:

• MIT License
• Dual- or tri-license with an MIT License option ("Apache-2.0 or MIT" is apopular combination)
• "MIT equivalent" license (2-clause BSD, 3-clause BSD, ISC)
• License less restrictive than the MIT License (CC0 1.0 Universal)
• Apache License version 2.0

Licenses we will not use:

• An ambiguous license, or no license
• Strongly reciprocal licenses (GNU GPL, GNU LGPL)

If you wish to add a reference but it doesn't meet these requirements, pleaseraise an issue to describe the dependency.