185

I know nothing about Sed but need this command (which works fine on Ubuntu) to work on a Mac OSX:

sed -i "/ $domain .*#drupalpro/d" /etc/hosts

I'm getting:

sed: 1: "/etc/hosts": extra characters at the end of h command
mklement0's user avatar
mklement0
452k68 gold badges727 silver badges986 bronze badges
askedMay 25, 2013 at 3:03
Michelle Williamson's user avatar
4
  • @AnthonySottile: The suggested duplicate is definitely closely related, except that - unlike this question - it specifically asks for a solution that works withboth the BSD/macOS and GNU implementations ofsed. Thus, people interested in a BSD/macOS-only solution may find more focused answers here.CommentedJul 3, 2017 at 3:16
  • @mklement0 yeah the question isslightly different, but the answers on both are nearly identical -- imo not worth having two questionsshrugsCommentedJul 3, 2017 at 4:07
  • 1
    @AnthonySottile: It was just a suggestion - either way is probably fine - let's let future close-voters decide.CommentedJul 3, 2017 at 4:15
  • May be,BashX project can help to you with this kind of problems.CommentedApr 23, 2021 at 18:55

4 Answers4

250

Ubuntu ships with GNUsed, where the suffix for the-i option is optional. OS X ships with BSDsed, where the suffix is mandatory. Trysed -i ''

answeredMay 25, 2013 at 3:15
microtherion's user avatar
Sign up to request clarification or add additional context in comments.

5 Comments

So just follow the sed -i with single quotes, sed -i '1i export PATH="$HOME/.composer/vendor/bin:$PATH"' $HOME/.bashrc Still does not work for me
@pal4life the point is that you need aseparate set of quotesbefore you start the command, so something likesed -i '' '1i export PATH="$HOME/.composer/vendor/bin:$PATH"' $HOME/.bashrc
If you need the same script to work on both macOS (OS X, BSD) and Linux et al, then you must use a backup suffix (e.g..bak) and you must attach it to the-i option —sed -i.bak …. If you don't do this, the script will not operate correctly on one of the two platforms. GNUsed does not like the empty argument that the BSDsed requires to edit in place; you can't attach the empty argument to-i because it is empty and indistinguishable from-i. Also, thanks to contradictory designs, the-i option won't be standardized as-i by POSIX; it can't reconcile the irreconcileable.
Oh, and if you don't want the backup, runrm -f /etc/hosts.bak after running thesed. That's simpler than trying to work out which variant ofsed you have installed or other gizmos I've seen suggested. I hope you keep the file under version control (or otherwise backed up) so that you can recover if there's a bug in yoursed script. Of course, the.bak file gives you that protection.
May be,BashX project can help to you with this kind of problems.
203

To complementmicrotherion's helpful, to-the-point answer:

  • with a portable solution
  • with background information

tl;dr:

The equivalent of thisGNUsed (standard on mostLinux distros) command:

sed -i    's/foo/bar/' file

is thisBSD/macOSsed command:

sed -i '' 's/foo/bar/' file  # Note the '' as a *separate argument*

WithBSD/macOSsed, the following commandsdonot work at all or not as intended:

sed -i    's/foo/bar/' file  # Breaks; script is misinterpreted as backup-file suffixsed -i''  's/foo/bar/' file  # Dittosed -i -e 's/foo/bar/' file  # -e is misinterpreted as backup-file suffix

For a discussion ofall the differences between GNUsed and BSD/macOSsed, seethis answer of mine.

Portable approach:

Note:Portable here means that the command works with both implementations discussed. It is not portable in aPOSIX sense, becausethe-i option is not POSIX-compliant.

# Works with both GNU and BSD/macOS Sed, due to a *non-empty* option-argument:# Create a backup file *temporarily* and remove it on success.sed -i.bak 's/foo/bar/' file && rm file.bak

For an explanation, see below; for alternative solutions, including a POSIX-compliant one, seethis related answer of mine.


Background information

InGNUsed (standard on most Linux distros) andBSD/macOSsed,the-i option, which performsin-place updating[1] of its input files, accepts anoption-argument that specifies whatsuffix (filename extension) to use for thebackup file of the file being updated.

E.g., inboth implementations, the following keeps the original file,file, as backup filefile.bak:

sed -i.bak 's/foo/bar/' file  # Keep original as 'file.bak'; NO SPACE between -i and .bak

Even thoughwithGNUsed the suffix argument isoptional, whereas withBSD/macOSsed it ismandatory, the above syntax works withboth implementations, becausedirectly adjoining the option-argument (.bak) to the option (-i) --i.bak, as opposed to-i .bak - works both as anoptional and amandatory option-argument:

  • Syntax-i.bak is theonly form that works for anoptional option-argument.
  • Syntax-i.bakalso works as amandatory option-argument, as analternative to-i .bak, i.e., specifying the option and its argumentseparately.

Not specifying a suffix - which is often the case - means thatno backup file should be kept, and that is wherethe incompatibility arises:

  • WithGNUsed, not specifying a suffix means simply using-iby itself.

  • WithBSD/macOSsed, not specifying a suffix means specifyingthe empty string as the - mandatory - suffix, andfortechnical reasons, the empty string can only be passed as aseparate argument: that is,-i ''not-i''.

-i'' doesn't work, because tosed it is indistinguishable from just-i, because theshell effectivelyremoves the empty quotes (it concatenates-i and'' and removes quotes with syntactical function), and passes just-i inboth cases.

With (effectively) just-i specified, it is thenext argument that is interpreted as the option-argument:

sed -i 's/foo/bar/' file # BREAKS with BSD/macOS Sed

's/foo/bar/' - intended to the the Sedscript (command) - is now interpreted as thesuffix, and the wordfile is interpreted as the script.
Interpreting such a word as script then leads to obscure error message such as
sed: 1: "file": invalid command code f,
because thef is interpreted as a Sed command (function).

Similarly, with:

sed -i -e 's/foo/bar/' file # CREATES BACKUP FILE 'file-e'

-e is interpreted as thesuffix argument, and NOT as Sed's-e option (which can be used to specifymultiple commands, if needed).
As as result,instead of keeping NO backup, you get a backup file with suffix-e.

That this command doesn't work as intended is less obvious, because the in-place updating does succeed, given that the syntax requirement of the suffix argument is satisfied by the-e argument.

That the accidental creation of these backup files easily goes unnoticed is the likeliest explanation forCrt's incorrect answer andthis incorrect answer to a similar question having received so many up-votes (as of this writing).


[1] Strictly speaking, a temporary file is created behind the scenes that thenreplaces the original file; this approach can be problematic: see the bottom half ofthis answer of mine.

answeredJul 1, 2017 at 18:10
mklement0's user avatar

3 Comments

This is the correct and most informative answer to the question. Not sure who downvoted it but no reasons have been expressed for the downvote while I have many reasons to accept this over the other answers.
I'm not an expert here, but, to keep POSIX compliance, wouldmv file file.bak && sed 's/foo/bar/' file.bak > file && rm file.bak work?
@CássioRenan: In principle, yes, butsed 's/foo/bar' file > file.bak && mv file.bak file (as shown inone linked answer) is simpler. If-i is available and portability is not a concern, its use is preferable, because it makes some effort to preserve the original file's attributes, though it notably still destroys a symlink - see the 2nd half ofanother linked answer. The POSIX-compliant approach not only also has the symlink problem, but simply creates a new file with the default permissions, ...
24

man is your friend.

OS X

 -i extension         Edit files in-place, saving backups with the specified extension.         If a zero-length extension is given, no backup will be saved.  It         is not recommended to give a zero-length extension when in-place         editing files, as you risk corruption or partial content in situ-         ations where disk space is exhausted, etc.
answeredMay 25, 2013 at 3:14
Diego Torres Milano's user avatar

Comments

14

In OS X, you can use the GNU version of sed:gsed.

# if using brewbrew install gnu-sed#if using portssudo port install gsed

Then, if your script should be portable, depending on your OS you can define which command to use.

SED=sedunamestr=`uname`if [[ "$unamestr" == "Darwin" ]] ; then    SED=gsed    type $SED >/dev/null 2>&1 || {        echo >&2 "$SED it's not installed. Try: brew install gnu-sed" ;        exit 1;    }fi# here your sed command, e.g.:$SED -i "/ $domain .*#drupalpro/d" /etc/hosts
mklement0's user avatar
mklement0
452k68 gold badges727 silver badges986 bronze badges
answeredJun 30, 2017 at 0:36
jcarballo's user avatar

3 Comments

This is the one that saved me. I upgraded my install and added /usr/local/opt/gnu-sed/libexec/gnubin to my path. I'm on OSX 10.12.6.
this should be accepted solution
Nice handling of dependency on gnu SED, very helpful!

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.