How to Use xargs Command
xargs command is a useful Linux utility that converts input data into a string of arguments. It is available on all Linux distributions and is mostly used to chain commands together. For instance,
xargs can take standard input from an initial command and format it into arguments that subsequent commands can use. This guide explains how to use
xargs, when to use it, and provides examples of how it is used.
Linux xargs: An Overview
Linux operating systems have three standard data streams: stdin, stderr, and stdout. These streams handle data in the same way regardless of whether it comes from a file, program output, or user input. Stdin is the standard input stream, which accepts text from either the user or a file. Programs or commands send their results to stdout, while directing any error messages to stderr.
In isolation, the relationship between programs, streams, and commands is very simple. However, when one program must send its results to another, it becomes more complicated. Linux provides the pipe command, represented by the
| symbol, to redirect the standard output from one command to a second command. This allows you to “chain” commands together and assemble a command pipeline. For example, you can send the results of the
cat command to the
grep command to search a file for a particular keyword. Only the lines that match the keyword are displayed.
cat example.com.conf | grep ErrorLog
Unfortunately, not all commands can accept data from the pipe command as input. Certain commands, such as
mkdir, must receive their input directly as command arguments. This presents a problem for anyone who wants to use them as part of a sequence.
xargs command resolves this issue. It accepts piped input from any source, including a file or a command, and converts it into a string. It then calls the target command and provides it with the string, which represents the arguments for the command. This means that the target command receives the arguments in the format it expects. If no target command is specified,
To use the
xargs command, use the
| symbol to pipe the results of the initial command to
xargs. Follow the
xargs keyword with any optional parameters as well as the target command. In the following example, the results of the
find command are piped to
xargs, which sends this input to
rm as arguments.
find /path -type f -name '*.txt' | xargs rm
xargs receives piped input, it converts it into a list or series of lists, where each list contains one or more elements. The number of elements is constrained by the system limit and the values of the
xargs invokes the target command once for every list and provides the list as an argument.
For example, if
xargs generates the following command:
rm file1.txt file2.txt
-n option is given a value of
xargs would generate two commands instead. It keeps calling
rm with only one file until the list of files from
find is completely consumed.
rm file1.txt rm file2.txt
Common xargs Use Cases
The most frequent use of
xargs is to allow a command to act on the output of another command. It is extremely common to see
xargs used together to process a long list of files. However,
xargs has some other handy uses. Many of these cases are covered in more depth in the How To Use xargs section.
xargshelps you work around maximum argument lengths for commands. For example, certain kernels limit the maximum size of an argument array. Additionally, some commands might impose even more stringent limits on the number of arguments.
xargscan divide the results of
findinto smaller lists and feed these shorter lists to the second command.
- You can use
xargsin conjunction with the
maxprocsoption to run operations in parallel. This allows
xargsto launch many processes at once, each working on separate data.
xargsallows you to implement command substitution through the use of the
Ioption. In this case,
xargsreplaces all instances of the substitution string with the list of arguments.
xargscan call a shell script that uses
sh. This allows for a much higher degree of complexity in the target command.
- Certain options allow file names with special characters such as
,to be processed. Newline characters can also be gracefully handled. Typically the
print0option is added to the initial
xargsis specified using the
xargscommand is frequently a more efficient alternative to
execwhen either option could be used.
execruns more slowly due to the way that processes are created.
xargs Command-Line Options
xargs with different command-line options can dramatically change the behavior of the utility. Here is a list of some of the most common options.
xargsto use the null character as a separator instead of separating on newline characters and spaces. It is typically used with the
print0option, which adds a null character to the end of each string that it passes to
xargs. All characters are accepted literally with no backslash substitutions. This is useful when dealing with unwieldy or non-standard file names. However, not all applications support null-terminated strings. The
nullflag has the same meaning as
aalong with the name of a file reads data from the file rather than standard input.
dspecifies a character to use as a string delimiter. A backslash or escape character such as
\nmight be used instead.
Ifollowed by a placeholder string is used to implement substitutions. Every time the placeholder occurs in the target command,
xargsreplaces it with the data from standard input. In the command
find /path -name '*.txt' | xargs -I % cp % ~/backups, whenever the
%symbol occurs in the target
xargsreplaces it with the input from
find. Multiple substitutions can occur in the same command. This command sets
1and only processes one line at a time.
Lspecifies the maximum number of non-blank input lines to use in each list.
nindicates the maximum number of arguments supplied each time the target command is invoked. The size option
stakes precedence over
xargs -n 1is used with the
findcommand, one filename is used in each invocation of the target command.
xargscommand with the input console. This is useful for running
xargsin an interactive context.
maxprocsoption. This causes
xargsto run up to this many processes simultaneously. The default is
xargsto run as many processes as the system allows.
pprompts the user before running each command. It is a good idea to use this option if the target command is potentially dangerous.
sspecifies the maximum number of characters supplied to each command, including the command itself and any terminating nulls.
tprints each command that is executed to standard output.
How To Use xargs
The following examples demonstrate several common use cases for the
xargs command. In most cases, the
-t option has been added solely to illustrate the resulting commands.
Using xargs Without Any Options
The simplest way to use
xargs is without any options. The following example displays the word count of every file in the current directory.
xargs converts the
ls results into a string of arguments for
ls | xargs -t wc
wc test1.txt test2.txt 1 10 42 test1.txt 1 10 42 test2.txt 2 20 84 total
Using xargs and Command Substitution
In the following example,
xargs is used to call
cp once for each
.txt file in the
/path directory. When the
% symbol is encountered in the target command, it gets replaced with the name of the file. This allows for the insertion of the filename in someplace other than at the end of the entire target command.
find ./xargstest -type f -name '*.txt' | xargs -t -I % cp -a % ~/backups
cp -a ./xargstest/test2.txt /home/user/backups cp -a ./xargstest/test1.txt /home/user/backups
0 Option to Deal With Complex File Names
In the following example, all directories in the
/path directory are removed. The combination of the
-print0 option for
find and the
-0 option for
xargs handles directory names with spaces or special characters. The
find command inserts a null character after each entry it finds, while
xargs uses the null character to distinguish the entries.
find ./xargstest/test1 -type d -name '*_*' -print0 | xargs -t -0 rmdir
rmdir ./xargstest/test1/test_1 ./xargstest/test1/test_2
Spawning Multiple Processes
-P option allows a larger number of processes to run simultaneously. For example, you can unzip four archives at a time using this option. When one of the processes finishes, another one is launched, as long as more zip files remain. To launch the target commands properly, you must also use the
-L option. In the following example, at most two processes run at any given time. For more information, refer to the documentation from the GNU organization.
find ./xargstest -name '*.zip' | xargs -t -P 2 -L 1 -I % unzip -u %
unzip -u ./xargstest/test4.zip unzip -u ./xargstest/test5.zip Archive: ./xargstest/test4.zip unzip -u ./xargstest/test2.zip Archive: ./xargstest/test5.zip unzip -u ./xargstest/test1.zip Archive: ./xargstest/test2.zip unzip -u ./xargstest/test3.zip Archive: ./xargstest/test1.zip Archive: ./xargstest/test3.zip
Invoke xargs to a Shell Script
To execute complex or multiple commands, a useful trick is to use
sh as the target command. This feeds the input data as input to the shell script. Command substitution can be used to translate the input data into the proper sequence. In this example, the shell runs the
word count program on each text file and then copies it to the
/archive directory. For more information on using
xargs to call
sh, visit the GNU site.
ls *.txt* | xargs -t -I % sh -c 'wc %; cp % archive'
sh -c 'wc test1.txt; cp test1.txt archive' 1 10 42 test1.txt sh -c 'wc test2.txt; cp test2.txt archive' 1 10 42 test2.txt
Call a Command Using a Subset of the Results
-n option limits the number of arguments
xargs sends to the target command. On the other hand, the
-L option selects many lines. If
-n is set to
1, then the command only selects the next entry. For example,
xargs can take the next two text files from the
find command and send them to
diff for comparison.
find ./xargstest/archive -type f -name '*.txt*' -print0 | xargs -0 -t -n 2 diff
diff ./xargstest/archive/test2.txt ./xargstest/archive/test1.txt 1c1 < this is test 1. It has a number of words. --- > this is test 1. It has a number of words and more.
n were set to
1, the command would
diff the file against its parent. In some cases, the
-L 1 option could also be used to achieve the same results.
noption enables an efficient method of copying files to multiple directories at the same time.
echocommand passes in a list of destination directories to
xargs, while the target command describes the specific details of the
- Adding the
n 1option instructs
xargsto pass in each destination directory one at a time.
The following example copies each text file in the current directory sequentially to
echo ~/xargstest/archive/ ~/xargstest/backup/ | xargs -t -n 1 cp -v ./*.txt
cp -v ./test1.txt ./test2.txt /home/user/xargstest/archive/ './test1.txt' -> '/home/user/xargstest/archive/test1.txt' './test2.txt' -> '/home/user/xargstest/archive/test2.txt' cp -v ./test1.txt ./test2.txt /home/user/xargstest/backup/ './test1.txt' -> '/home/user/xargstest/backup/test1.txt' './test2.txt' -> '/home/user/xargstest/backup/test2.txt'
Use xargs Debug Options
-t options are used to debug
xargs commands. The
-t option prints the full command line to the standard error output before running it. The
-p option prompts the user to approve each command by entering
Y before executing it. For example, the following command prompts you to approve each potential delete operation before
rm does anything.
find ./xargstest/archive -type f -print0 | xargs -0 -n 1 -p rm
rm ./xargstest/archive/test2.txt ?...y rm ./xargstest/archive/test1.txt ?...y
Small world. Big idea!
- Welcome to visit the knowledge base of SRE and DevOps!
- License under CC BY-NC 4.0
- No personal information is collected
- Made with Material for MkDocs and generative AI tools
- Copyright issue feedback me#imzye.com, replace # with @
- Get latest SRE news and discuss on Discord Channel