Last Updated: 2024-04-18

How to pass dynamic line-based arguments when using xargs

• Pass the -I option to set a placeholder and use that ({} here)

$ ag "function " resources/js | xargs -I {} ag {} resources/views

How to run multiple commands on each arg with xargs

• use xargs sh -c "$COMMANDS"

$ ag "function " resources/js |  xargs -I {}  sh -c "echo {}\n; ag {} resources/views && echo 'found'"

Think about whether you want to pipe within each xargs argument (map) or afterwards (reducer)

Imaging you want the number of lines changed for each file in a repo:

You start with this:

git ls-files | xargs -I {} git log --follow --format=oneline {} | wc -l

Unfortunately this gives the wrong answer -- just a single number. You want one number per file

Instead you want to run word count on the result of git log on each file from ls-files

git ls-files | xargs -I {} sh -c 'echo $1 $(git log --follow --format=oneline $1 | wc -l)' sh {}

How to continue xargs loop even though one iteration failed

By default this happens except for exist code 255. To get around this exception, do the following:

$ xargs sh -c "somecommand || true"

How to prevent xargs from continuing even though one iteration failed

Exit with 255 in a sh -c process using || after the risky command

find . -name "package.json" -type f -maxdepth 2 |  xargs -I{} sh -c '(git -C {} pull || exit 255)'

Though, it might be easier to use find -exec where applicable

$ find app/views -iname *html.erb -exec htmlbeautifier -e -b 1 {} \;

Remember: Xargs is greedy

When I ran wc on the piped output of xargs like so

$ find . -name "*.md" | xargs wc

64     416    2745 ./audio-checklist.md
23     111     776 ./README.md
59     382    2307 ./preparation-checklist.md
2890   15333   96605 total

I saw in the output both the word counts for individual files and a total.

The individual entries are expected. But why the total?

The reason is because xargs passes the "input command" (wc here) as many arguments as the input command can take. Therefore this is equivalent to wc **/*.md, which produces EXACTLY the same output. I.e. the totaling comes from wc, not xargs. I.e. xargs takes as many args as possible (vs. 1 at a time)

To get rid of the total, tell xargs to pass one argument at a time

find . -name "*.md" | xargs -n 1 wc

How to use shell substitution

Running ${FILE#**.} normally would convert file (e.g. selector.jsx) into jsx.

But running it with normal xargs is difficult

git ls-files | xargs -I {} echo "${{}#**.}"

"zsh: bad substitution"

A workaround is to pass into sh invocation that can make use of $1 etc.

git ls-files | xargs -I {} sh -c 'echo ${1#**.}' sh {} | uniq

It can be useful to increase processor count

xargs -P 16 ...

You can see its effect by running top. You can find out the number of cpus on your system with nproc --all (for example)