2012-01-24 20:53:25 +00:00
|
|
|
\input texinfo
|
|
|
|
@setfilename parallel.info
|
|
|
|
|
|
|
|
@documentencoding utf-8
|
|
|
|
|
|
|
|
@settitle parallel - build and execute shell command lines from standard input in parallel
|
|
|
|
|
|
|
|
@node Top
|
|
|
|
@top parallel
|
|
|
|
|
|
|
|
@chapter NAME
|
|
|
|
@anchor{NAME}
|
|
|
|
|
|
|
|
parallel - build and execute shell command lines from standard input in parallel
|
|
|
|
|
|
|
|
@chapter SYNOPSIS
|
|
|
|
@anchor{SYNOPSIS}
|
|
|
|
|
|
|
|
@strong{parallel} [options] [@emph{command} [arguments]] < list_of_arguments
|
|
|
|
|
|
|
|
@strong{parallel} [options] [@emph{command} [arguments]] ( @strong{:::} arguments |
|
|
|
|
@strong{::::} argfile(s) ) ...
|
|
|
|
|
|
|
|
@strong{parallel} --semaphore [options] @emph{command}
|
|
|
|
|
|
|
|
@strong{#!/usr/bin/parallel} --shebang [options] [@emph{command} [arguments]]
|
|
|
|
|
|
|
|
@chapter DESCRIPTION
|
|
|
|
@anchor{DESCRIPTION}
|
|
|
|
|
|
|
|
GNU @strong{parallel} is a shell tool for executing jobs in parallel using
|
2012-03-12 22:38:38 +00:00
|
|
|
one or more computers. A job can be a single command or a small
|
2012-01-24 20:53:25 +00:00
|
|
|
script that has to be run for each of the lines in the input. The
|
|
|
|
typical input is a list of files, a list of hosts, a list of users, a
|
|
|
|
list of URLs, or a list of tables. A job can also be a command that
|
|
|
|
reads from a pipe. GNU @strong{parallel} can then split the input into
|
|
|
|
blocks and pipe a block into each command in parallel.
|
|
|
|
|
|
|
|
If you use xargs and tee today you will find GNU @strong{parallel} very easy to
|
|
|
|
use as GNU @strong{parallel} is written to have the same options as xargs. If
|
|
|
|
you write loops in shell, you will find GNU @strong{parallel} may be able to
|
|
|
|
replace most of the loops and make them run faster by running several
|
|
|
|
jobs in parallel.
|
|
|
|
|
|
|
|
GNU @strong{parallel} makes sure output from the commands is the same output as
|
|
|
|
you would get had you run the commands sequentially. This makes it
|
|
|
|
possible to use output from GNU @strong{parallel} as input for other programs.
|
|
|
|
|
|
|
|
For each line of input GNU @strong{parallel} will execute @emph{command} with
|
|
|
|
the line as arguments. If no @emph{command} is given, the line of input is
|
|
|
|
executed. Several lines will be run in parallel. GNU @strong{parallel} can
|
|
|
|
often be used as a substitute for @strong{xargs} or @strong{cat | bash}.
|
|
|
|
|
|
|
|
@section Reader's guide
|
|
|
|
@anchor{Reader's guide}
|
|
|
|
|
|
|
|
Before looking at the options you may want to check out the @strong{EXAMPLE}s
|
|
|
|
after the list of options. That will give you an idea of what GNU
|
|
|
|
@strong{parallel} is capable of.
|
|
|
|
|
|
|
|
You can also watch the intro video for a quick introduction:
|
|
|
|
http://tinyogg.com/watch/TORaR/ http://tinyogg.com/watch/hfxKj/ and
|
|
|
|
http://tinyogg.com/watch/YQuXd/ or
|
|
|
|
http://www.youtube.com/playlist?list=PL284C9FF2488BC6D1
|
|
|
|
|
|
|
|
@chapter OPTIONS
|
|
|
|
@anchor{OPTIONS}
|
|
|
|
|
|
|
|
@table @asis
|
|
|
|
@item @emph{command}
|
|
|
|
@anchor{@emph{command}}
|
|
|
|
|
|
|
|
Command to execute. If @emph{command} or the following arguments contain
|
|
|
|
replacement strings (such as @strong{@{@}}) every instance will be substituted
|
|
|
|
with the input.
|
|
|
|
|
2012-03-15 20:23:53 +00:00
|
|
|
If @emph{command} is given, GNU @strong{parallel} solve the same tasks as
|
|
|
|
@strong{xargs}. If @emph{command} is not given GNU @strong{parallel} will behave
|
|
|
|
similar to @strong{cat | sh}.
|
2012-01-24 20:53:25 +00:00
|
|
|
|
2012-02-20 00:48:28 +00:00
|
|
|
The @emph{command} must be an executable, a script, a composed command, or
|
|
|
|
a function. If it is a function you need to @strong{export -f} the function
|
|
|
|
first. An alias will, however, not work (see why
|
2012-01-24 20:53:25 +00:00
|
|
|
http://www.perlmonks.org/index.pl?node_id=484296).
|
|
|
|
|
|
|
|
@item @strong{@{@}}
|
|
|
|
@anchor{@strong{@{@}}}
|
|
|
|
|
|
|
|
Input line. This replacement string will be replaced by a full line
|
|
|
|
read from the input source. The input source is normally stdin
|
|
|
|
(standard input), but can also be given with @strong{-a}, @strong{:::}, or
|
|
|
|
@strong{::::}.
|
|
|
|
|
|
|
|
The replacement string @strong{@{@}} can be changed with @strong{-I}.
|
|
|
|
|
|
|
|
If the command line contains no replacement strings then @strong{@{@}} will be
|
|
|
|
appended to the command line.
|
|
|
|
|
|
|
|
@item @strong{@{.@}}
|
|
|
|
@anchor{@strong{@{.@}}}
|
|
|
|
|
|
|
|
Input line without extension. This replacement string will be replaced
|
|
|
|
by the input with the extension removed. If the input line contains
|
|
|
|
@strong{.} after the last @strong{/} the last @strong{.} till the end of the string will
|
|
|
|
be removed and @strong{@{.@}} will be replaced with the
|
|
|
|
remaining. E.g. @emph{foo.jpg} becomes @emph{foo}, @emph{subdir/foo.jpg} becomes
|
|
|
|
@emph{subdir/foo}, @emph{sub.dir/foo.jpg} becomes @emph{sub.dir/foo},
|
|
|
|
@emph{sub.dir/bar} remains @emph{sub.dir/bar}. If the input line does not
|
|
|
|
contain @strong{.} it will remain unchanged.
|
|
|
|
|
|
|
|
The replacement string @strong{@{.@}} can be changed with @strong{--er}.
|
|
|
|
|
|
|
|
To understand replacement strings see @strong{@{@}}.
|
|
|
|
|
|
|
|
@item @strong{@{/@}}
|
|
|
|
@anchor{@strong{@{/@}}}
|
|
|
|
|
|
|
|
Basename of input line. This replacement string will be replaced by
|
|
|
|
the input with the directory part removed.
|
|
|
|
|
|
|
|
The replacement string @strong{@{/@}} can be changed with
|
|
|
|
@strong{--basenamereplace}.
|
|
|
|
|
|
|
|
To understand replacement strings see @strong{@{@}}.
|
|
|
|
|
|
|
|
@item @strong{@{//@}}
|
|
|
|
@anchor{@strong{@{//@}}}
|
|
|
|
|
|
|
|
Dirname of input line. This replacement string will be replaced by the
|
|
|
|
dir of the input line. See @strong{dirname}(1).
|
|
|
|
|
|
|
|
The replacement string @strong{@{//@}} can be changed with
|
|
|
|
@strong{--dirnamereplace}.
|
|
|
|
|
|
|
|
To understand replacement strings see @strong{@{@}}.
|
|
|
|
|
|
|
|
@item @strong{@{/.@}}
|
|
|
|
@anchor{@strong{@{/.@}}}
|
|
|
|
|
|
|
|
Basename of input line without extension. This replacement string will
|
|
|
|
be replaced by the input with the directory and extension part
|
|
|
|
removed. It is a combination of @strong{@{/@}} and @strong{@{.@}}.
|
|
|
|
|
|
|
|
The replacement string @strong{@{/.@}} can be changed with
|
|
|
|
@strong{--basenameextensionreplace}.
|
|
|
|
|
|
|
|
To understand replacement strings see @strong{@{@}}.
|
|
|
|
|
|
|
|
@item @strong{@{#@}}
|
|
|
|
@anchor{@strong{@{#@}}}
|
|
|
|
|
|
|
|
Sequence number of the job to run. This replacement string will be
|
|
|
|
replaced by the sequence number of the job being run. It contains the
|
|
|
|
same number as $PARALLEL_SEQ.
|
|
|
|
|
|
|
|
The replacement string @strong{@{#@}} can be changed with @strong{--seqreplace}.
|
|
|
|
|
|
|
|
To understand replacement strings see @strong{@{@}}.
|
|
|
|
|
|
|
|
@item @strong{@{}@emph{n}@strong{@}}
|
|
|
|
@anchor{@strong{@{}@emph{n}@strong{@}}}
|
|
|
|
|
|
|
|
Argument from input source @emph{n} or the @emph{n}'th argument. This
|
|
|
|
positional replacement string will be replaced by the input from input
|
|
|
|
source @emph{n} (when used with @strong{-a} or @strong{::::}) or with the @emph{n}'th
|
|
|
|
argument (when used with @strong{-N}).
|
|
|
|
|
|
|
|
To understand replacement strings see @strong{@{@}}.
|
|
|
|
|
|
|
|
@item @strong{@{}@emph{n}.@strong{@}}
|
|
|
|
@anchor{@strong{@{}@emph{n}.@strong{@}}}
|
|
|
|
|
|
|
|
Argument from input source @emph{n} or the @emph{n}'th argument without
|
|
|
|
extension. It is a combination of @strong{@{}@emph{n}@strong{@}} and @strong{@{.@}}.
|
|
|
|
|
|
|
|
This positional replacement string will be replaced by the input from
|
|
|
|
input source @emph{n} (when used with @strong{-a} or @strong{::::}) or with the
|
|
|
|
@emph{n}'th argument (when used with @strong{-N}). The input will have the
|
|
|
|
extension removed.
|
|
|
|
|
|
|
|
To understand positional replacement strings see @strong{@{}@emph{n}@strong{@}}.
|
|
|
|
|
|
|
|
@item @strong{@{}@emph{n}/@strong{@}}
|
|
|
|
@anchor{@strong{@{}@emph{n}/@strong{@}}}
|
|
|
|
|
|
|
|
Basename of argument from input source @emph{n} or the @emph{n}'th argument.
|
|
|
|
It is a combination of @strong{@{}@emph{n}@strong{@}} and @strong{@{/@}}.
|
|
|
|
|
|
|
|
This positional replacement string will be replaced by the input from
|
|
|
|
input source @emph{n} (when used with @strong{-a} or @strong{::::}) or with the
|
|
|
|
@emph{n}'th argument (when used with @strong{-N}). The input will have the
|
|
|
|
directory (if any) removed.
|
|
|
|
|
|
|
|
To understand positional replacement strings see @strong{@{}@emph{n}@strong{@}}.
|
|
|
|
|
2012-03-15 20:23:53 +00:00
|
|
|
@item @strong{@{}@emph{n}//@strong{@}}
|
|
|
|
@anchor{@strong{@{}@emph{n}//@strong{@}}}
|
|
|
|
|
|
|
|
Dirname of argument from input source @emph{n} or the @emph{n}'th argument.
|
|
|
|
It is a combination of @strong{@{}@emph{n}@strong{@}} and @strong{@{//@}}.
|
|
|
|
|
|
|
|
This positional replacement string will be replaced by the dir of the
|
|
|
|
input from input source @emph{n} (when used with @strong{-a} or @strong{::::}) or with
|
|
|
|
the @emph{n}'th argument (when used with @strong{-N}). See @strong{dirname}(1).
|
|
|
|
|
|
|
|
To understand positional replacement strings see @strong{@{}@emph{n}@strong{@}}.
|
|
|
|
|
2012-01-24 20:53:25 +00:00
|
|
|
@item @strong{@{}@emph{n}/.@strong{@}}
|
|
|
|
@anchor{@strong{@{}@emph{n}/.@strong{@}}}
|
|
|
|
|
|
|
|
Basename of argument from input source @emph{n} or the @emph{n}'th argument
|
|
|
|
without extension. It is a combination of @strong{@{}@emph{n}@strong{@}}, @strong{@{/@}}, and
|
|
|
|
@strong{@{.@}}.
|
|
|
|
|
|
|
|
This positional replacement string will be replaced by the input from
|
|
|
|
input source @emph{n} (when used with @strong{-a} or @strong{::::}) or with the
|
|
|
|
@emph{n}'th argument (when used with @strong{-N}). The input will have the
|
|
|
|
directory (if any) and extension removed.
|
|
|
|
|
|
|
|
To understand positional replacement strings see @strong{@{}@emph{n}@strong{@}}.
|
|
|
|
|
|
|
|
@item @strong{:::} @emph{arguments}
|
|
|
|
@anchor{@strong{:::} @emph{arguments}}
|
|
|
|
|
|
|
|
Use arguments from the command line as input source instead of stdin
|
|
|
|
(standard input). Unlike other options for GNU @strong{parallel} @strong{:::} is
|
|
|
|
placed after the @emph{command} and before the arguments.
|
|
|
|
|
|
|
|
The following are equivalent:
|
|
|
|
|
|
|
|
@verbatim
|
|
|
|
(echo file1; echo file2) | parallel gzip
|
|
|
|
parallel gzip ::: file1 file2
|
|
|
|
parallel gzip {} ::: file1 file2
|
|
|
|
parallel --arg-sep ,, gzip {} ,, file1 file2
|
|
|
|
parallel --arg-sep ,, gzip ,, file1 file2
|
|
|
|
parallel ::: "gzip file1" "gzip file2"
|
|
|
|
@end verbatim
|
|
|
|
|
|
|
|
To avoid treating @strong{:::} as special use @strong{--arg-sep} to set the
|
|
|
|
argument separator to something else. See also @strong{--arg-sep}.
|
|
|
|
|
|
|
|
stdin (standard input) will be passed to the first process run.
|
|
|
|
|
|
|
|
If multiple @strong{:::} are given, each group will be treated as an input
|
|
|
|
source, and all combinations of input sources will be
|
|
|
|
generated. E.g. ::: 1 2 ::: a b c will result in the combinations
|
|
|
|
(1,a) (1,b) (1,c) (2,a) (2,b) (2,c). This is useful for replacing
|
|
|
|
nested for-loops.
|
|
|
|
|
|
|
|
@strong{:::} and @strong{::::} can be mixed. So these are equivalent:
|
|
|
|
|
|
|
|
@verbatim
|
|
|
|
parallel echo {1} {2} {3} ::: 6 7 ::: 4 5 ::: 1 2 3
|
|
|
|
parallel echo {1} {2} {3} :::: <(seq 6 7) <(seq 4 5) :::: <(seq 1 3)
|
|
|
|
parallel -a <(seq 6 7) echo {1} {2} {3} :::: <(seq 4 5) :::: <(seq 1 3)
|
|
|
|
parallel -a <(seq 6 7) -a <(seq 4 5) echo {1} {2} {3} ::: 1 2 3
|
|
|
|
seq 6 7 | parallel -a - -a <(seq 4 5) echo {1} {2} {3} ::: 1 2 3
|
|
|
|
seq 4 5 | parallel echo {1} {2} {3} :::: <(seq 6 7) - ::: 1 2 3
|
|
|
|
@end verbatim
|
|
|
|
|
|
|
|
@item @strong{::::} @emph{argfiles}
|
|
|
|
@anchor{@strong{::::} @emph{argfiles}}
|
|
|
|
|
|
|
|
Another way to write @strong{-a} @emph{argfile1} @strong{-a} @emph{argfile2} ...
|
|
|
|
|
|
|
|
@strong{:::} and @strong{::::} can be mixed.
|
|
|
|
|
|
|
|
See @strong{-a}, @strong{:::} and @strong{--xapply}.
|
|
|
|
|
|
|
|
@item @strong{--null}
|
|
|
|
@anchor{@strong{--null}}
|
|
|
|
|
|
|
|
@item @strong{-0}
|
|
|
|
@anchor{@strong{-0}}
|
|
|
|
|
|
|
|
Use NUL as delimiter. Normally input lines will end in \n
|
|
|
|
(newline). If they end in \0 (NUL), then use this option. It is useful
|
|
|
|
for processing arguments that may contain \n (newline).
|
|
|
|
|
|
|
|
@item @strong{--arg-file} @emph{input-file}
|
|
|
|
@anchor{@strong{--arg-file} @emph{input-file}}
|
|
|
|
|
|
|
|
@item @strong{-a} @emph{input-file}
|
|
|
|
@anchor{@strong{-a} @emph{input-file}}
|
|
|
|
|
|
|
|
Use @emph{input-file} as input source. If you use this option, stdin
|
|
|
|
(standard input) is given to the first process run. Otherwise, stdin
|
|
|
|
(standard input) is redirected from /dev/null.
|
|
|
|
|
|
|
|
If multiple @strong{-a} are given, each @emph{input-file} will be treated as an
|
|
|
|
input source, and all combinations of input sources will be
|
|
|
|
generated. E.g. The file @strong{foo} contains @strong{1 2}, the file @strong{bar}
|
|
|
|
contains @strong{a b c}. @strong{-a foo} @strong{-a bar} will result in the combinations
|
|
|
|
(1,a) (1,b) (1,c) (2,a) (2,b) (2,c). This is useful for replacing
|
|
|
|
nested for-loops.
|
|
|
|
|
|
|
|
See also @strong{--xapply} and @strong{@{}@emph{n}@strong{@}}.
|
|
|
|
|
|
|
|
@item @strong{--arg-file-sep} @emph{sep-str}
|
|
|
|
@anchor{@strong{--arg-file-sep} @emph{sep-str}}
|
|
|
|
|
|
|
|
Use @emph{sep-str} instead of @strong{::::} as separator string between command
|
|
|
|
and argument files. Useful if @strong{::::} is used for something else by the
|
|
|
|
command.
|
|
|
|
|
|
|
|
See also: @strong{::::}.
|
|
|
|
|
|
|
|
@item @strong{--arg-sep} @emph{sep-str}
|
|
|
|
@anchor{@strong{--arg-sep} @emph{sep-str}}
|
|
|
|
|
|
|
|
Use @emph{sep-str} instead of @strong{:::} as separator string. Useful if @strong{:::}
|
|
|
|
is used for something else by the command.
|
|
|
|
|
|
|
|
Also useful if you command uses @strong{:::} but you still want to read
|
|
|
|
arguments from stdin (standard input): Simply change @strong{--arg-sep} to a
|
|
|
|
string that is not in the command line.
|
|
|
|
|
|
|
|
See also: @strong{:::}.
|
|
|
|
|
|
|
|
@item @strong{--basefile} @emph{file}
|
|
|
|
@anchor{@strong{--basefile} @emph{file}}
|
|
|
|
|
|
|
|
@item @strong{--bf} @emph{file}
|
|
|
|
@anchor{@strong{--bf} @emph{file}}
|
|
|
|
|
|
|
|
@emph{file} will be transferred to each sshlogin before a jobs is
|
|
|
|
started. It will be removed if @strong{--cleanup} is active. The file may be
|
|
|
|
a script to run or some common base data needed for the jobs.
|
|
|
|
Multiple @strong{--bf} can be specified to transfer more basefiles. The
|
|
|
|
@emph{file} will be transferred the same way as @strong{--transfer}.
|
|
|
|
|
|
|
|
@item @strong{--basenamereplace} @emph{replace-str}
|
|
|
|
@anchor{@strong{--basenamereplace} @emph{replace-str}}
|
|
|
|
|
|
|
|
@item @strong{--bnr} @emph{replace-str}
|
|
|
|
@anchor{@strong{--bnr} @emph{replace-str}}
|
|
|
|
|
|
|
|
Use the replacement string @emph{replace-str} instead of @strong{@{/@}} for
|
|
|
|
basename of input line.
|
|
|
|
|
|
|
|
@item @strong{--basenameextensionreplace} @emph{replace-str}
|
|
|
|
@anchor{@strong{--basenameextensionreplace} @emph{replace-str}}
|
|
|
|
|
|
|
|
@item @strong{--bner} @emph{replace-str}
|
|
|
|
@anchor{@strong{--bner} @emph{replace-str}}
|
|
|
|
|
|
|
|
Use the replacement string @emph{replace-str} instead of @strong{@{/.@}} for basename of input line without extension.
|
|
|
|
|
|
|
|
@item @strong{--bg}
|
|
|
|
@anchor{@strong{--bg}}
|
|
|
|
|
|
|
|
Run command in background thus GNU @strong{parallel} will not wait for
|
|
|
|
completion of the command before exiting. This is the default if
|
|
|
|
@strong{--semaphore} is set.
|
|
|
|
|
|
|
|
See also: @strong{--fg}, @strong{man sem}
|
|
|
|
|
|
|
|
Implies @strong{--semaphore}.
|
|
|
|
|
|
|
|
@item @strong{--bibtex}
|
|
|
|
@anchor{@strong{--bibtex}}
|
|
|
|
|
|
|
|
Print the BibTeX entry for GNU @strong{parallel}.
|
|
|
|
|
|
|
|
@item @strong{--block} @emph{size}
|
|
|
|
@anchor{@strong{--block} @emph{size}}
|
|
|
|
|
|
|
|
@item @strong{--block-size} @emph{size}
|
|
|
|
@anchor{@strong{--block-size} @emph{size}}
|
|
|
|
|
|
|
|
Size of block in bytes. The size can be postfixed with K, M, G, T, P,
|
|
|
|
k, m, g, t, or p which would multiply the size with 1024, 1048576,
|
|
|
|
1073741824, 1099511627776, 1125899906842624, 1000, 1000000,
|
|
|
|
1000000000, 1000000000000, or 1000000000000000 respectively.
|
|
|
|
|
|
|
|
GNU @strong{parallel} tries to meet the block size but can be off by the
|
|
|
|
length of one record.
|
|
|
|
|
|
|
|
@emph{size} defaults to 1M.
|
|
|
|
|
|
|
|
See @strong{--pipe} for use of this.
|
|
|
|
|
|
|
|
@item @strong{--cleanup}
|
|
|
|
@anchor{@strong{--cleanup}}
|
|
|
|
|
|
|
|
Remove transferred files. @strong{--cleanup} will remove the transferred files
|
|
|
|
on the remote computer after processing is done.
|
|
|
|
|
|
|
|
@verbatim
|
|
|
|
find log -name '*gz' | parallel \
|
|
|
|
--sshlogin server.example.com --transfer --return {.}.bz2 \
|
|
|
|
--cleanup "zcat {} | bzip -9 >{.}.bz2"
|
|
|
|
@end verbatim
|
|
|
|
|
|
|
|
With @strong{--transfer} the file transferred to the remote computer will be
|
|
|
|
removed on the remote computer. Directories created will not be removed
|
|
|
|
- even if they are empty.
|
|
|
|
|
|
|
|
With @strong{--return} the file transferred from the remote computer will be
|
|
|
|
removed on the remote computer. Directories created will not be removed
|
|
|
|
- even if they are empty.
|
|
|
|
|
|
|
|
@strong{--cleanup} is ignored when not used with @strong{--transfer} or @strong{--return}.
|
|
|
|
|
|
|
|
@item @strong{--colsep} @emph{regexp}
|
|
|
|
@anchor{@strong{--colsep} @emph{regexp}}
|
|
|
|
|
|
|
|
@item @strong{-C} @emph{regexp}
|
|
|
|
@anchor{@strong{-C} @emph{regexp}}
|
|
|
|
|
|
|
|
Column separator. The input will be treated as a table with @emph{regexp}
|
|
|
|
separating the columns. The n'th column can be access using
|
|
|
|
@strong{@{}@emph{n}@strong{@}} or @strong{@{}@emph{n}.@strong{@}}. E.g. @strong{@{3@}} is the 3rd column.
|
|
|
|
|
|
|
|
@strong{--colsep} implies @strong{--trim rl}.
|
|
|
|
|
|
|
|
@emph{regexp} is a Perl Regular Expression:
|
|
|
|
http://perldoc.perl.org/perlre.html
|
|
|
|
|
|
|
|
@item @strong{--delimiter} @emph{delim}
|
|
|
|
@anchor{@strong{--delimiter} @emph{delim}}
|
|
|
|
|
|
|
|
@item @strong{-d} @emph{delim}
|
|
|
|
@anchor{@strong{-d} @emph{delim}}
|
|
|
|
|
|
|
|
Input items are terminated by the specified character. Quotes and
|
|
|
|
backslash are not special; every character in the input is taken
|
|
|
|
literally. Disables the end-of-file string, which is treated like any
|
|
|
|
other argument. This can be used when the input consists of simply
|
|
|
|
newline-separated items, although it is almost always better to design
|
|
|
|
your program to use --null where this is possible. The specified
|
|
|
|
delimiter may be a single character, a C-style character escape such
|
|
|
|
as \n, or an octal or hexadecimal escape code. Octal and
|
|
|
|
hexadecimal escape codes are understood as for the printf command.
|
|
|
|
Multibyte characters are not supported.
|
|
|
|
|
|
|
|
@item @strong{--dirnamereplace} @emph{replace-str}
|
|
|
|
@anchor{@strong{--dirnamereplace} @emph{replace-str}}
|
|
|
|
|
|
|
|
@item @strong{--dnr} @emph{replace-str}
|
|
|
|
@anchor{@strong{--dnr} @emph{replace-str}}
|
|
|
|
|
|
|
|
Use the replacement string @emph{replace-str} instead of @strong{@{//@}} for
|
|
|
|
dirname of input line.
|
|
|
|
|
|
|
|
@item @strong{-E} @emph{eof-str}
|
|
|
|
@anchor{@strong{-E} @emph{eof-str}}
|
|
|
|
|
|
|
|
Set the end of file string to eof-str. If the end of file string
|
|
|
|
occurs as a line of input, the rest of the input is ignored. If
|
|
|
|
neither @strong{-E} nor @strong{-e} is used, no end of file string is used.
|
|
|
|
|
|
|
|
@item @strong{--dry-run}
|
|
|
|
@anchor{@strong{--dry-run}}
|
|
|
|
|
|
|
|
Print the job to run on stdout (standard output), but do not run the
|
|
|
|
job. Use @strong{-v -v} to include the ssh/rsync wrapping if the job would
|
|
|
|
be run on a remote computer. Do not count on this literaly, though, as
|
|
|
|
the job may be scheduled on another computer or the local computer if
|
|
|
|
: is in the list.
|
|
|
|
|
|
|
|
@item @strong{--eof}[=@emph{eof-str}]
|
|
|
|
@anchor{@strong{--eof}[=@emph{eof-str}]}
|
|
|
|
|
|
|
|
@item @strong{-e}[@emph{eof-str}]
|
|
|
|
@anchor{@strong{-e}[@emph{eof-str}]}
|
|
|
|
|
|
|
|
This option is a synonym for the @strong{-E} option. Use @strong{-E} instead,
|
|
|
|
because it is POSIX compliant for @strong{xargs} while this option is not.
|
|
|
|
If @emph{eof-str} is omitted, there is no end of file string. If neither
|
|
|
|
@strong{-E} nor @strong{-e} is used, no end of file string is used.
|
|
|
|
|
|
|
|
@item @strong{--eta}
|
|
|
|
@anchor{@strong{--eta}}
|
|
|
|
|
|
|
|
Show the estimated number of seconds before finishing. This forces GNU
|
|
|
|
@strong{parallel} to read all jobs before starting to find the number of
|
|
|
|
jobs. GNU @strong{parallel} normally only reads the next job to run.
|
|
|
|
Implies @strong{--progress}.
|
|
|
|
|
|
|
|
@item @strong{--fg}
|
|
|
|
@anchor{@strong{--fg}}
|
|
|
|
|
|
|
|
Run command in foreground thus GNU @strong{parallel} will wait for
|
|
|
|
completion of the command before exiting.
|
|
|
|
|
|
|
|
See also: @strong{--bg}, @strong{man sem}
|
|
|
|
|
|
|
|
Implies @strong{--semaphore}.
|
|
|
|
|
|
|
|
@item @strong{--gnu}
|
|
|
|
@anchor{@strong{--gnu}}
|
|
|
|
|
|
|
|
Behave like GNU @strong{parallel}. If @strong{--tollef} and @strong{--gnu} are both set,
|
|
|
|
@strong{--gnu} takes precedence.
|
|
|
|
|
|
|
|
@item @strong{--group}
|
|
|
|
@anchor{@strong{--group}}
|
|
|
|
|
|
|
|
Group output. Output from each jobs is grouped together and is only
|
|
|
|
printed when the command is finished. stderr (standard error) first
|
|
|
|
followed by stdout (standard output). This takes some CPU time. In
|
|
|
|
rare situations GNU @strong{parallel} takes up lots of CPU time and if it is
|
|
|
|
acceptable that the outputs from different commands are mixed
|
|
|
|
together, then disabling grouping with @strong{-u} can speedup GNU
|
|
|
|
@strong{parallel} by a factor of 10.
|
|
|
|
|
|
|
|
@strong{--group} is the default. Can be reversed with @strong{-u}.
|
|
|
|
|
|
|
|
@item @strong{--help}
|
|
|
|
@anchor{@strong{--help}}
|
|
|
|
|
|
|
|
@item @strong{-h}
|
|
|
|
@anchor{@strong{-h}}
|
|
|
|
|
|
|
|
Print a summary of the options to GNU @strong{parallel} and exit.
|
|
|
|
|
|
|
|
@item @strong{--halt-on-error} <0|1|2>
|
|
|
|
@anchor{@strong{--halt-on-error} <0|1|2>}
|
|
|
|
|
|
|
|
@item @strong{--halt} <0|1|2>
|
|
|
|
@anchor{@strong{--halt} <0|1|2>}
|
|
|
|
|
|
|
|
@table @asis
|
|
|
|
@item 0
|
|
|
|
@anchor{0}
|
|
|
|
|
|
|
|
Do not halt if a job fails. Exit status will be the number of jobs
|
|
|
|
failed. This is the default.
|
|
|
|
|
|
|
|
@item 1
|
|
|
|
@anchor{1}
|
|
|
|
|
|
|
|
Do not start new jobs if a job fails, but complete the running jobs
|
|
|
|
including cleanup. The exit status will be the exit status from the
|
|
|
|
last failing job.
|
|
|
|
|
|
|
|
@item 2
|
|
|
|
@anchor{2}
|
|
|
|
|
|
|
|
Kill off all jobs immediately and exit without cleanup. The exit
|
|
|
|
status will be the exit status from the failing job.
|
|
|
|
|
|
|
|
@end table
|
|
|
|
|
2012-03-22 23:29:46 +00:00
|
|
|
@item @strong{--header} @emph{regexp}
|
|
|
|
@anchor{@strong{--header} @emph{regexp}}
|
2012-01-24 20:53:25 +00:00
|
|
|
|
|
|
|
Use upto regexp as header. For normal usage the matched header
|
|
|
|
(typically the first line: @strong{--header '\n'}) will be split using
|
|
|
|
@strong{--colsep} (which will default to '\t') and column names can be used
|
|
|
|
as replacement variables: @strong{@{column name@}}. For @strong{--pipe} the matched
|
|
|
|
header will be prepended to each output.
|
|
|
|
|
|
|
|
@strong{--header :} is an alias for @strong{--header '\n'}.
|
|
|
|
|
|
|
|
@item @strong{-I} @emph{replace-str}
|
|
|
|
@anchor{@strong{-I} @emph{replace-str}}
|
|
|
|
|
|
|
|
Use the replacement string @emph{replace-str} instead of @{@}.
|
|
|
|
|
|
|
|
@item @strong{--replace}[=@emph{replace-str}]
|
|
|
|
@anchor{@strong{--replace}[=@emph{replace-str}]}
|
|
|
|
|
|
|
|
@item @strong{-i}[@emph{replace-str}]
|
|
|
|
@anchor{@strong{-i}[@emph{replace-str}]}
|
|
|
|
|
|
|
|
This option is a synonym for @strong{-I}@emph{replace-str} if @emph{replace-str} is
|
|
|
|
specified, and for @strong{-I}@{@} otherwise. This option is deprecated;
|
|
|
|
use @strong{-I} instead.
|
|
|
|
|
2012-05-24 00:03:57 +00:00
|
|
|
@item @strong{--joblog} @emph{logfile} (alpha testing)
|
|
|
|
@anchor{@strong{--joblog} @emph{logfile} (alpha testing)}
|
2012-01-24 20:53:25 +00:00
|
|
|
|
2012-03-12 22:38:38 +00:00
|
|
|
Logfile for executed jobs. Save a list of the executed jobs to
|
2012-01-24 20:53:25 +00:00
|
|
|
@emph{logfile} in the following TAB separated format: sequence number,
|
|
|
|
sshlogin, start time as seconds since epoch, run time in seconds,
|
2012-06-03 20:33:00 +00:00
|
|
|
bytes in files transferred, bytes in files returned, exit status,
|
2012-01-24 20:53:25 +00:00
|
|
|
and command run.
|
|
|
|
|
|
|
|
To convert the times into ISO-8601 strict do:
|
|
|
|
|
|
|
|
@strong{perl -a -F"\t" -ne 'chomp($F[2]=`date -d \@@$F[2] +%FT%T`); print join("\t",@@F)'}
|
|
|
|
|
|
|
|
See also @strong{--resume}.
|
|
|
|
|
|
|
|
@item @strong{--jobs} @emph{N}
|
|
|
|
@anchor{@strong{--jobs} @emph{N}}
|
|
|
|
|
|
|
|
@item @strong{-j} @emph{N}
|
|
|
|
@anchor{@strong{-j} @emph{N}}
|
|
|
|
|
|
|
|
@item @strong{--max-procs} @emph{N}
|
|
|
|
@anchor{@strong{--max-procs} @emph{N}}
|
|
|
|
|
|
|
|
@item @strong{-P} @emph{N}
|
|
|
|
@anchor{@strong{-P} @emph{N}}
|
|
|
|
|
|
|
|
Number of jobslots. Run up to N jobs in parallel. 0 means as many as
|
|
|
|
possible. Default is 100% which will run one job per CPU core.
|
|
|
|
|
|
|
|
If @strong{--semaphore} is set default is 1 thus making a mutex.
|
|
|
|
|
|
|
|
@item @strong{--jobs} @emph{+N}
|
|
|
|
@anchor{@strong{--jobs} @emph{+N}}
|
|
|
|
|
|
|
|
@item @strong{-j} @emph{+N}
|
|
|
|
@anchor{@strong{-j} @emph{+N}}
|
|
|
|
|
|
|
|
@item @strong{--max-procs} @emph{+N}
|
|
|
|
@anchor{@strong{--max-procs} @emph{+N}}
|
|
|
|
|
|
|
|
@item @strong{-P} @emph{+N}
|
|
|
|
@anchor{@strong{-P} @emph{+N}}
|
|
|
|
|
2012-03-15 20:23:53 +00:00
|
|
|
Add N to the number of CPU cores. Run this many jobs in parallel.
|
|
|
|
See also @strong{--use-cpus-instead-of-cores}.
|
2012-01-24 20:53:25 +00:00
|
|
|
|
|
|
|
@item @strong{--jobs} @emph{-N}
|
|
|
|
@anchor{@strong{--jobs} @emph{-N}}
|
|
|
|
|
|
|
|
@item @strong{-j} @emph{-N}
|
|
|
|
@anchor{@strong{-j} @emph{-N}}
|
|
|
|
|
|
|
|
@item @strong{--max-procs} @emph{-N}
|
|
|
|
@anchor{@strong{--max-procs} @emph{-N}}
|
|
|
|
|
|
|
|
@item @strong{-P} @emph{-N}
|
|
|
|
@anchor{@strong{-P} @emph{-N}}
|
|
|
|
|
|
|
|
Subtract N from the number of CPU cores. Run this many jobs in parallel.
|
|
|
|
If the evaluated number is less than 1 then 1 will be used. See also
|
|
|
|
@strong{--use-cpus-instead-of-cores}.
|
|
|
|
|
|
|
|
@item @strong{--jobs} @emph{N}%
|
|
|
|
@anchor{@strong{--jobs} @emph{N}%}
|
|
|
|
|
|
|
|
@item @strong{-j} @emph{N}%
|
|
|
|
@anchor{@strong{-j} @emph{N}%}
|
|
|
|
|
|
|
|
@item @strong{--max-procs} @emph{N}%
|
|
|
|
@anchor{@strong{--max-procs} @emph{N}%}
|
|
|
|
|
|
|
|
@item @strong{-P} @emph{N}%
|
|
|
|
@anchor{@strong{-P} @emph{N}%}
|
|
|
|
|
|
|
|
Multiply N% with the number of CPU cores. Run this many jobs in parallel.
|
|
|
|
If the evaluated number is less than 1 then 1 will be used. See also
|
|
|
|
@strong{--use-cpus-instead-of-cores}.
|
|
|
|
|
|
|
|
@item @strong{--jobs} @emph{procfile}
|
|
|
|
@anchor{@strong{--jobs} @emph{procfile}}
|
|
|
|
|
|
|
|
@item @strong{-j} @emph{procfile}
|
|
|
|
@anchor{@strong{-j} @emph{procfile}}
|
|
|
|
|
|
|
|
@item @strong{--max-procs} @emph{procfile}
|
|
|
|
@anchor{@strong{--max-procs} @emph{procfile}}
|
|
|
|
|
|
|
|
@item @strong{-P} @emph{procfile}
|
|
|
|
@anchor{@strong{-P} @emph{procfile}}
|
|
|
|
|
|
|
|
Read parameter from file. Use the content of @emph{procfile} as parameter
|
|
|
|
for @emph{-j}. E.g. @emph{procfile} could contain the string 100% or +2 or
|
|
|
|
10. If @emph{procfile} is changed when a job completes, @emph{procfile} is
|
|
|
|
read again and the new number of jobs is computed. If the number is
|
|
|
|
lower than before, running jobs will be allowed to finish but new jobs
|
|
|
|
will not be started until the wanted number of jobs has been reached.
|
|
|
|
This makes it possible to change the number of simultaneous running
|
|
|
|
jobs while GNU @strong{parallel} is running.
|
|
|
|
|
|
|
|
@item @strong{--keep-order}
|
|
|
|
@anchor{@strong{--keep-order}}
|
|
|
|
|
|
|
|
@item @strong{-k}
|
|
|
|
@anchor{@strong{-k}}
|
|
|
|
|
|
|
|
Keep sequence of output same as the order of input. Normally the
|
|
|
|
output of a job will be printed as soon as the job completes. Try this
|
|
|
|
to see the difference:
|
|
|
|
|
|
|
|
@verbatim
|
|
|
|
parallel -j4 sleep {}\; echo {} ::: 2 1 4 3
|
|
|
|
parallel -j4 -k sleep {}\; echo {} ::: 2 1 4 3
|
|
|
|
@end verbatim
|
|
|
|
|
|
|
|
@item @strong{-L} @emph{max-lines}
|
|
|
|
@anchor{@strong{-L} @emph{max-lines}}
|
|
|
|
|
2012-06-14 21:13:11 +00:00
|
|
|
When used with @strong{--pipe}: Read records of @emph{max-lines}.
|
|
|
|
|
|
|
|
When used otherwise: Use at most @emph{max-lines} nonblank input lines per
|
|
|
|
command line. Trailing blanks cause an input line to be logically
|
|
|
|
continued on the next input line.
|
2012-01-24 20:53:25 +00:00
|
|
|
|
|
|
|
@strong{-L 0} means read one line, but insert 0 arguments on the command
|
|
|
|
line.
|
|
|
|
|
2012-06-14 21:13:11 +00:00
|
|
|
Implies @strong{-X} unless @strong{-m}, @strong{--xargs}, or @strong{--pipe} is set.
|
2012-01-24 20:53:25 +00:00
|
|
|
|
|
|
|
@item @strong{--max-lines}[=@emph{max-lines}]
|
|
|
|
@anchor{@strong{--max-lines}[=@emph{max-lines}]}
|
|
|
|
|
|
|
|
@item @strong{-l}[@emph{max-lines}]
|
|
|
|
@anchor{@strong{-l}[@emph{max-lines}]}
|
|
|
|
|
2012-06-14 21:13:11 +00:00
|
|
|
When used with @strong{--pipe}: Read records of @emph{max-lines}.
|
|
|
|
|
|
|
|
When used otherwise: Synonym for the @strong{-L} option. Unlike @strong{-L}, the
|
|
|
|
@emph{max-lines} argument is optional. If @emph{max-lines} is not specified,
|
|
|
|
it defaults to one. The @strong{-l} option is deprecated since the POSIX
|
|
|
|
standard specifies @strong{-L} instead.
|
2012-01-24 20:53:25 +00:00
|
|
|
|
|
|
|
@strong{-l 0} is an alias for @strong{-l 1}.
|
|
|
|
|
|
|
|
Implies @strong{-X} unless @strong{-m} is set.
|
|
|
|
|
2012-05-24 00:03:57 +00:00
|
|
|
@item @strong{--load} @emph{max-load} (alpha testing)
|
|
|
|
@anchor{@strong{--load} @emph{max-load} (alpha testing)}
|
2012-01-24 20:53:25 +00:00
|
|
|
|
|
|
|
Do not start new jobs on a given computer unless the load is less than
|
|
|
|
@emph{max-load}. @emph{max-load} uses the same syntax as @strong{--jobs}, so @emph{100%}
|
2012-03-15 20:23:53 +00:00
|
|
|
for one per CPU is a valid setting. Only difference is 0 which is
|
|
|
|
interpreted as 0.01.
|
2012-01-24 20:53:25 +00:00
|
|
|
|
2012-03-15 20:23:53 +00:00
|
|
|
The load average is only sampled every 10 seconds using @strong{uptime} to
|
|
|
|
avoid stressing small computers. Only the first (1 minute) load is
|
|
|
|
used.
|
2012-01-24 20:53:25 +00:00
|
|
|
|
|
|
|
@item @strong{--controlmaster} (experimental)
|
|
|
|
@anchor{@strong{--controlmaster} (experimental)}
|
|
|
|
|
|
|
|
@item @strong{-M} (experimental)
|
|
|
|
@anchor{@strong{-M} (experimental)}
|
|
|
|
|
|
|
|
Use ssh's ControlMaster to make ssh connections faster. Useful if jobs
|
|
|
|
run remote and are very fast to run. This is disabled for sshlogins
|
|
|
|
that specify their own ssh command.
|
|
|
|
|
|
|
|
@item @strong{--xargs}
|
|
|
|
@anchor{@strong{--xargs}}
|
|
|
|
|
|
|
|
Multiple arguments. Insert as many arguments as the command line
|
|
|
|
length permits.
|
|
|
|
|
|
|
|
If @strong{@{@}} is not used the arguments will be appended to the
|
|
|
|
line. If @strong{@{@}} is used multiple times each @strong{@{@}} will be replaced
|
|
|
|
with all the arguments.
|
|
|
|
|
|
|
|
Support for @strong{--xargs} with @strong{--sshlogin} is limited and may fail.
|
|
|
|
|
|
|
|
See also @strong{-X} for context replace. If in doubt use @strong{-X} as that will
|
|
|
|
most likely do what is needed.
|
|
|
|
|
|
|
|
@item @strong{-m}
|
|
|
|
@anchor{@strong{-m}}
|
|
|
|
|
|
|
|
Multiple arguments. Insert as many arguments as the command line
|
|
|
|
length permits. If multiple jobs are being run in parallel: distribute
|
|
|
|
the arguments evenly among the jobs. Use @strong{-j1} to avoid this.
|
|
|
|
|
|
|
|
If @strong{@{@}} is not used the arguments will be appended to the
|
|
|
|
line. If @strong{@{@}} is used multiple times each @strong{@{@}} will be replaced
|
|
|
|
with all the arguments.
|
|
|
|
|
|
|
|
Support for @strong{-m} with @strong{--sshlogin} is limited and may fail.
|
|
|
|
|
|
|
|
See also @strong{-X} for context replace. If in doubt use @strong{-X} as that will
|
|
|
|
most likely do what is needed.
|
|
|
|
|
|
|
|
@item @strong{--minversion} @emph{version}
|
|
|
|
@anchor{@strong{--minversion} @emph{version}}
|
|
|
|
|
|
|
|
Print the version GNU @strong{parallel} and exit. If the current version of
|
|
|
|
GNU @strong{parallel} is less than @emph{version} the exit code is
|
|
|
|
255. Otherwise it is 0.
|
|
|
|
|
|
|
|
This is useful for scripts that depend on features only available from
|
|
|
|
a certain version of GNU @strong{parallel}.
|
|
|
|
|
|
|
|
@item @strong{--nonall}
|
|
|
|
@anchor{@strong{--nonall}}
|
|
|
|
|
|
|
|
@strong{--onall} with no arguments. Run the command on all computers given
|
|
|
|
with @strong{--sshlogin} but take no arguments. GNU @strong{parallel} will log
|
|
|
|
into @strong{--jobs} number of computers in parallel and run the job on the
|
|
|
|
computer. @strong{-j} adjusts how many computers to log into in parallel.
|
|
|
|
|
|
|
|
This is useful for running the same command (e.g. uptime) on a list of
|
|
|
|
servers.
|
|
|
|
|
|
|
|
@item @strong{--onall}
|
|
|
|
@anchor{@strong{--onall}}
|
|
|
|
|
|
|
|
Run all the jobs on all computers given with @strong{--sshlogin}. GNU
|
|
|
|
@strong{parallel} will log into @strong{--jobs} number of computers in parallel
|
|
|
|
and run one job at a time on the computer. The order of the jobs will
|
|
|
|
not be changed, but some computers may finish before others. @strong{-j}
|
|
|
|
adjusts how many computers to log into in parallel.
|
|
|
|
|
|
|
|
When using @strong{--group} the output will be grouped by each server, so
|
|
|
|
all the output from one server will be grouped together.
|
|
|
|
|
|
|
|
@item @strong{--output-as-files}
|
|
|
|
@anchor{@strong{--output-as-files}}
|
|
|
|
|
|
|
|
@item @strong{--outputasfiles}
|
|
|
|
@anchor{@strong{--outputasfiles}}
|
|
|
|
|
|
|
|
@item @strong{--files}
|
|
|
|
@anchor{@strong{--files}}
|
|
|
|
|
|
|
|
Instead of printing the output to stdout (standard output) the output
|
|
|
|
of each job is saved in a file and the filename is then printed.
|
|
|
|
|
2012-05-24 00:03:57 +00:00
|
|
|
@item @strong{--pipe} (alpha testing)
|
|
|
|
@anchor{@strong{--pipe} (alpha testing)}
|
2012-01-24 20:53:25 +00:00
|
|
|
|
2012-05-24 00:03:57 +00:00
|
|
|
@item @strong{--spreadstdin} (alpha testing)
|
|
|
|
@anchor{@strong{--spreadstdin} (alpha testing)}
|
2012-01-24 20:53:25 +00:00
|
|
|
|
|
|
|
Spread input to jobs on stdin (standard input). Read a block of data
|
|
|
|
from stdin (standard input) and give one block of data as input to one
|
|
|
|
job.
|
|
|
|
|
|
|
|
The block size is determined by @strong{--block}. The strings @strong{--recstart}
|
|
|
|
and @strong{--recend} tell GNU @strong{parallel} how a record starts and/or
|
|
|
|
ends. The block read will have the final partial record removed before
|
|
|
|
the block is passed on to the job. The partial record will be
|
|
|
|
prepended to next block.
|
|
|
|
|
|
|
|
If @strong{--recstart} is given this will be used to split at record start.
|
|
|
|
|
|
|
|
If @strong{--recend} is given this will be used to split at record end.
|
|
|
|
|
|
|
|
If both @strong{--recstart} and @strong{--recend} are given both will have to
|
|
|
|
match to find a split position.
|
|
|
|
|
|
|
|
If neither @strong{--recstart} nor @strong{--recend} are given @strong{--recend}
|
|
|
|
defaults to '\n'. To have no record separator use @strong{--recend ""}.
|
|
|
|
|
|
|
|
@strong{--files} is often used with @strong{--pipe}.
|
|
|
|
|
|
|
|
@item @strong{--progress}
|
|
|
|
@anchor{@strong{--progress}}
|
|
|
|
|
|
|
|
Show progress of computations. List the computers involved in the task
|
|
|
|
with number of CPU cores detected and the max number of jobs to
|
|
|
|
run. After that show progress for each computer: number of running
|
|
|
|
jobs, number of completed jobs, and percentage of all jobs done by
|
|
|
|
this computer. The percentage will only be available after all jobs
|
|
|
|
have been scheduled as GNU @strong{parallel} only read the next job when
|
|
|
|
ready to schedule it - this is to avoid wasting time and memory by
|
|
|
|
reading everything at startup.
|
|
|
|
|
|
|
|
By sending GNU @strong{parallel} SIGUSR2 you can toggle turning on/off
|
|
|
|
@strong{--progress} on a running GNU @strong{parallel} process.
|
|
|
|
|
|
|
|
See also: @strong{--eta}
|
|
|
|
|
|
|
|
@item @strong{--max-args}=@emph{max-args}
|
|
|
|
@anchor{@strong{--max-args}=@emph{max-args}}
|
|
|
|
|
|
|
|
@item @strong{-n} @emph{max-args}
|
|
|
|
@anchor{@strong{-n} @emph{max-args}}
|
|
|
|
|
|
|
|
Use at most @emph{max-args} arguments per command line. Fewer than
|
|
|
|
@emph{max-args} arguments will be used if the size (see the @strong{-s} option)
|
|
|
|
is exceeded, unless the @strong{-x} option is given, in which case
|
|
|
|
GNU @strong{parallel} will exit.
|
|
|
|
|
|
|
|
@strong{-n 0} means read one argument, but insert 0 arguments on the command
|
|
|
|
line.
|
|
|
|
|
|
|
|
Implies @strong{-X} unless @strong{-m} is set.
|
|
|
|
|
|
|
|
@item @strong{--max-replace-args}=@emph{max-args}
|
|
|
|
@anchor{@strong{--max-replace-args}=@emph{max-args}}
|
|
|
|
|
|
|
|
@item @strong{-N} @emph{max-args}
|
|
|
|
@anchor{@strong{-N} @emph{max-args}}
|
|
|
|
|
|
|
|
Use at most @emph{max-args} arguments per command line. Like @strong{-n} but
|
|
|
|
also makes replacement strings @strong{@{1@}} .. @strong{@{}@emph{max-args}@strong{@}} that
|
|
|
|
represents argument 1 .. @emph{max-args}. If too few args the @strong{@{}@emph{n}@strong{@}} will
|
|
|
|
be empty.
|
|
|
|
|
|
|
|
@strong{-N 0} means read one argument, but insert 0 arguments on the command
|
|
|
|
line.
|
|
|
|
|
|
|
|
This will set the owner of the homedir to the user:
|
|
|
|
|
|
|
|
@strong{tr ':' '\n' < /etc/passwd | parallel -N7 chown @{1@} @{6@}}
|
|
|
|
|
|
|
|
Implies @strong{-X} unless @strong{-m} or @strong{--pipe} is set.
|
|
|
|
|
|
|
|
When used with @strong{--pipe} @strong{-N} is the number of records to read. This
|
|
|
|
is much slower than @strong{--block} so avoid it if performance is
|
|
|
|
important.
|
|
|
|
|
|
|
|
@item @strong{--max-line-length-allowed}
|
|
|
|
@anchor{@strong{--max-line-length-allowed}}
|
|
|
|
|
2012-03-12 22:38:38 +00:00
|
|
|
Print the maximal number of characters allowed on the command line and
|
2012-01-24 20:53:25 +00:00
|
|
|
exit (used by GNU @strong{parallel} itself to determine the line length
|
|
|
|
on remote computers).
|
|
|
|
|
|
|
|
@item @strong{--number-of-cpus}
|
|
|
|
@anchor{@strong{--number-of-cpus}}
|
|
|
|
|
|
|
|
Print the number of physical CPUs and exit (used by GNU @strong{parallel}
|
|
|
|
itself to determine the number of physical CPUs on remote computers).
|
|
|
|
|
|
|
|
@item @strong{--number-of-cores}
|
|
|
|
@anchor{@strong{--number-of-cores}}
|
|
|
|
|
|
|
|
Print the number of CPU cores and exit (used by GNU @strong{parallel} itself
|
|
|
|
to determine the number of CPU cores on remote computers).
|
|
|
|
|
|
|
|
@item @strong{--nice} @emph{niceness}
|
|
|
|
@anchor{@strong{--nice} @emph{niceness}}
|
|
|
|
|
|
|
|
Run the command at this niceness. For simple commands you can just add
|
|
|
|
@strong{nice} in front of the command. But if the command consists of more
|
|
|
|
sub commands (Like: ls|wc) then prepending @strong{nice} will not always
|
|
|
|
work. @strong{--nice} will make sure all sub commands are niced.
|
|
|
|
|
|
|
|
@item @strong{--interactive}
|
|
|
|
@anchor{@strong{--interactive}}
|
|
|
|
|
|
|
|
@item @strong{-p}
|
|
|
|
@anchor{@strong{-p}}
|
|
|
|
|
|
|
|
Prompt the user about whether to run each command line and read a line
|
|
|
|
from the terminal. Only run the command line if the response starts
|
|
|
|
with 'y' or 'Y'. Implies @strong{-t}.
|
|
|
|
|
|
|
|
@item @strong{--profile} @emph{profilename}
|
|
|
|
@anchor{@strong{--profile} @emph{profilename}}
|
|
|
|
|
|
|
|
@item @strong{-J} @emph{profilename}
|
|
|
|
@anchor{@strong{-J} @emph{profilename}}
|
|
|
|
|
|
|
|
Use profile @emph{profilename} for options. This is useful if you want to
|
|
|
|
have multiple profiles. You could have one profile for running jobs in
|
|
|
|
parallel on the local computer and a different profile for running jobs
|
|
|
|
on remote computers. See the section PROFILE FILES for examples.
|
|
|
|
|
|
|
|
@emph{profilename} corresponds to the file ~/.parallel/@emph{profilename}.
|
|
|
|
|
|
|
|
You can give multiple profiles by repeating @strong{--profile}. If parts of
|
|
|
|
the profiles conflict, the later ones will be used.
|
|
|
|
|
|
|
|
Default: config
|
|
|
|
|
|
|
|
@item @strong{--quote}
|
|
|
|
@anchor{@strong{--quote}}
|
|
|
|
|
|
|
|
@item @strong{-q}
|
|
|
|
@anchor{@strong{-q}}
|
|
|
|
|
|
|
|
Quote @emph{command}. This will quote the command line so special
|
|
|
|
characters are not interpreted by the shell. See the section
|
|
|
|
QUOTING. Most people will never need this. Quoting is disabled by
|
|
|
|
default.
|
|
|
|
|
|
|
|
@item @strong{--no-run-if-empty}
|
|
|
|
@anchor{@strong{--no-run-if-empty}}
|
|
|
|
|
|
|
|
@item @strong{-r}
|
|
|
|
@anchor{@strong{-r}}
|
|
|
|
|
|
|
|
If the stdin (standard input) only contains whitespace, do not run the command.
|
|
|
|
|
|
|
|
If used with @strong{--pipe} this is slow.
|
|
|
|
|
|
|
|
@item @strong{--recstart} @emph{startstring}
|
|
|
|
@anchor{@strong{--recstart} @emph{startstring}}
|
|
|
|
|
|
|
|
@item @strong{--recend} @emph{endstring}
|
|
|
|
@anchor{@strong{--recend} @emph{endstring}}
|
|
|
|
|
|
|
|
If @strong{--recstart} is given @emph{startstring} will be used to split at record start.
|
|
|
|
|
|
|
|
If @strong{--recend} is given @emph{endstring} will be used to split at record end.
|
|
|
|
|
|
|
|
If both @strong{--recstart} and @strong{--recend} are given the combined string
|
|
|
|
@emph{endstring}@emph{startstring} will have to match to find a split
|
|
|
|
position. This is useful if either @emph{startstring} or @emph{endstring}
|
|
|
|
match in the middle of a record.
|
|
|
|
|
|
|
|
If neither @strong{--recstart} nor @strong{--recend} are given then @strong{--recend}
|
|
|
|
defaults to '\n'. To have no record separator use @strong{--recend ""}.
|
|
|
|
|
|
|
|
@strong{--recstart} and @strong{--recend} are used with @strong{--pipe}.
|
|
|
|
|
|
|
|
Use @strong{--regexp} to interpret @strong{--recstart} and @strong{--recend} as regular
|
|
|
|
expressions. This is slow, however.
|
|
|
|
|
|
|
|
@item @strong{--regexp}
|
|
|
|
@anchor{@strong{--regexp}}
|
|
|
|
|
|
|
|
Use @strong{--regexp} to interpret @strong{--recstart} and @strong{--recend} as regular
|
|
|
|
expressions. This is slow, however.
|
|
|
|
|
|
|
|
@item @strong{--remove-rec-sep}
|
|
|
|
@anchor{@strong{--remove-rec-sep}}
|
|
|
|
|
|
|
|
@item @strong{--removerecsep}
|
|
|
|
@anchor{@strong{--removerecsep}}
|
|
|
|
|
|
|
|
@item @strong{--rrs}
|
|
|
|
@anchor{@strong{--rrs}}
|
|
|
|
|
|
|
|
Remove the text matched by @strong{--recstart} and @strong{--recend} before piping
|
|
|
|
it to the command.
|
|
|
|
|
|
|
|
Only used with @strong{--pipe}.
|
|
|
|
|
2012-03-22 23:29:46 +00:00
|
|
|
@item @strong{--resume}
|
|
|
|
@anchor{@strong{--resume}}
|
2012-01-24 20:53:25 +00:00
|
|
|
|
|
|
|
Resumes from the last unfinished job. By reading @strong{--joblog} GNU
|
|
|
|
@strong{parallel} will figure out the last unfinished job and continue from
|
|
|
|
there. As GNU @strong{parallel} only looks at the sequence numbers in
|
|
|
|
@strong{--joblog} then the input, the command, and @strong{--joblog} all have to
|
|
|
|
remain unchanged; otherwise GNU @strong{parallel} may run wrong commands.
|
|
|
|
|
|
|
|
See also: @strong{--joblog}.
|
|
|
|
|
|
|
|
@item @strong{--retries} @emph{n}
|
|
|
|
@anchor{@strong{--retries} @emph{n}}
|
|
|
|
|
|
|
|
If a job fails, retry it on another computer. Do this @emph{n} times. If
|
|
|
|
there are fewer than @emph{n} computers in @strong{--sshlogin} GNU @strong{parallel} will
|
|
|
|
re-use the computers. This is useful if some jobs fail for no apparent
|
|
|
|
reason (such as network failure).
|
|
|
|
|
|
|
|
@item @strong{--return} @emph{filename}
|
|
|
|
@anchor{@strong{--return} @emph{filename}}
|
|
|
|
|
|
|
|
Transfer files from remote computers. @strong{--return} is used with
|
|
|
|
@strong{--sshlogin} when the arguments are files on the remote computers. When
|
|
|
|
processing is done the file @emph{filename} will be transferred
|
|
|
|
from the remote computer using @strong{rsync} and will be put relative to
|
|
|
|
the default login dir. E.g.
|
|
|
|
|
|
|
|
@verbatim
|
|
|
|
echo foo/bar.txt | parallel \
|
|
|
|
--sshlogin server.example.com --return {.}.out touch {.}.out
|
|
|
|
@end verbatim
|
|
|
|
|
|
|
|
This will transfer the file @emph{$HOME/foo/bar.out} from the computer
|
|
|
|
@emph{server.example.com} to the file @emph{foo/bar.out} after running
|
|
|
|
@strong{touch foo/bar.out} on @emph{server.example.com}.
|
|
|
|
|
|
|
|
@verbatim
|
|
|
|
echo /tmp/foo/bar.txt | parallel \
|
|
|
|
--sshlogin server.example.com --return {.}.out touch {.}.out
|
|
|
|
@end verbatim
|
|
|
|
|
|
|
|
This will transfer the file @emph{/tmp/foo/bar.out} from the computer
|
|
|
|
@emph{server.example.com} to the file @emph{/tmp/foo/bar.out} after running
|
|
|
|
@strong{touch /tmp/foo/bar.out} on @emph{server.example.com}.
|
|
|
|
|
|
|
|
Multiple files can be transferred by repeating the options multiple
|
|
|
|
times:
|
|
|
|
|
|
|
|
@verbatim
|
|
|
|
echo /tmp/foo/bar.txt | \
|
|
|
|
parallel --sshlogin server.example.com \
|
|
|
|
--return {.}.out --return {.}.out2 touch {.}.out {.}.out2
|
|
|
|
@end verbatim
|
|
|
|
|
|
|
|
@strong{--return} is often used with @strong{--transfer} and @strong{--cleanup}.
|
|
|
|
|
|
|
|
@strong{--return} is ignored when used with @strong{--sshlogin :} or when not used
|
|
|
|
with @strong{--sshlogin}.
|
|
|
|
|
|
|
|
@item @strong{--max-chars}=@emph{max-chars}
|
|
|
|
@anchor{@strong{--max-chars}=@emph{max-chars}}
|
|
|
|
|
|
|
|
@item @strong{-s} @emph{max-chars}
|
|
|
|
@anchor{@strong{-s} @emph{max-chars}}
|
|
|
|
|
|
|
|
Use at most @emph{max-chars} characters per command line, including the
|
|
|
|
command and initial-arguments and the terminating nulls at the ends of
|
|
|
|
the argument strings. The largest allowed value is system-dependent,
|
|
|
|
and is calculated as the argument length limit for exec, less the size
|
|
|
|
of your environment. The default value is the maximum.
|
|
|
|
|
|
|
|
Implies @strong{-X} unless @strong{-m} is set.
|
|
|
|
|
|
|
|
@item @strong{--show-limits}
|
|
|
|
@anchor{@strong{--show-limits}}
|
|
|
|
|
|
|
|
Display the limits on the command-line length which are imposed by the
|
|
|
|
operating system and the @strong{-s} option. Pipe the input from /dev/null
|
|
|
|
(and perhaps specify --no-run-if-empty) if you don't want GNU @strong{parallel}
|
|
|
|
to do anything.
|
|
|
|
|
|
|
|
@item @strong{--semaphore}
|
|
|
|
@anchor{@strong{--semaphore}}
|
|
|
|
|
|
|
|
Work as a counting semaphore. @strong{--semaphore} will cause GNU
|
|
|
|
@strong{parallel} to start @emph{command} in the background. When the number of
|
|
|
|
simultaneous jobs is reached, GNU @strong{parallel} will wait for one of
|
|
|
|
these to complete before starting another command.
|
|
|
|
|
|
|
|
@strong{--semaphore} implies @strong{--bg} unless @strong{--fg} is specified.
|
|
|
|
|
|
|
|
@strong{--semaphore} implies @strong{--semaphorename `tty`} unless
|
|
|
|
@strong{--semaphorename} is specified.
|
|
|
|
|
|
|
|
Used with @strong{--fg}, @strong{--wait}, and @strong{--semaphorename}.
|
|
|
|
|
|
|
|
The command @strong{sem} is an alias for @strong{parallel --semaphore}.
|
|
|
|
|
|
|
|
See also: @strong{man sem}
|
|
|
|
|
|
|
|
@item @strong{--semaphorename} @emph{name}
|
|
|
|
@anchor{@strong{--semaphorename} @emph{name}}
|
|
|
|
|
|
|
|
@item @strong{--id} @emph{name}
|
|
|
|
@anchor{@strong{--id} @emph{name}}
|
|
|
|
|
|
|
|
Use @strong{name} as the name of the semaphore. Default is the name of the
|
|
|
|
controlling tty (output from @strong{tty}).
|
|
|
|
|
|
|
|
The default normally works as expected when used interactively, but
|
|
|
|
when used in a script @emph{name} should be set. @emph{$$} or @emph{my_task_name}
|
|
|
|
are often a good value.
|
|
|
|
|
|
|
|
The semaphore is stored in ~/.parallel/semaphores/
|
|
|
|
|
|
|
|
Implies @strong{--semaphore}.
|
|
|
|
|
|
|
|
See also: @strong{man sem}
|
|
|
|
|
|
|
|
@item @strong{--semaphoretimeout} @emph{secs} (not implemented)
|
|
|
|
@anchor{@strong{--semaphoretimeout} @emph{secs} (not implemented)}
|
|
|
|
|
|
|
|
If the semaphore is not released within secs seconds, take it anyway.
|
|
|
|
|
|
|
|
Implies @strong{--semaphore}.
|
|
|
|
|
|
|
|
See also: @strong{man sem}
|
|
|
|
|
|
|
|
@item @strong{--seqreplace} @emph{replace-str}
|
|
|
|
@anchor{@strong{--seqreplace} @emph{replace-str}}
|
|
|
|
|
|
|
|
Use the replacement string @emph{replace-str} instead of @strong{@{#@}} for
|
|
|
|
job sequence number.
|
|
|
|
|
|
|
|
@item @strong{--shellquote}
|
|
|
|
@anchor{@strong{--shellquote}}
|
|
|
|
|
|
|
|
Does not run the command but quotes it. Useful for making quoted
|
|
|
|
composed commands for GNU @strong{parallel}.
|
|
|
|
|
|
|
|
@item @strong{--skip-first-line}
|
|
|
|
@anchor{@strong{--skip-first-line}}
|
|
|
|
|
|
|
|
Do not use the first line of input (used by GNU @strong{parallel} itself
|
|
|
|
when called with @strong{--shebang}).
|
|
|
|
|
|
|
|
@item @strong{-S} @emph{[ncpu/]sshlogin[,[ncpu/]sshlogin[,...]]}
|
|
|
|
@anchor{@strong{-S} @emph{[ncpu/]sshlogin[@comma{}[ncpu/]sshlogin[@comma{}...]]}}
|
|
|
|
|
|
|
|
@item @strong{--sshlogin} @emph{[ncpu/]sshlogin[,[ncpu/]sshlogin[,...]]}
|
|
|
|
@anchor{@strong{--sshlogin} @emph{[ncpu/]sshlogin[@comma{}[ncpu/]sshlogin[@comma{}...]]}}
|
|
|
|
|
|
|
|
Distribute jobs to remote computers. The jobs will be run on a list of
|
|
|
|
remote computers. GNU @strong{parallel} will determine the number of CPU
|
|
|
|
cores on the remote computers and run the number of jobs as specified by
|
|
|
|
@strong{-j}. If the number @emph{ncpu} is given GNU @strong{parallel} will use this
|
|
|
|
number for number of CPU cores on the host. Normally @emph{ncpu} will not
|
|
|
|
be needed.
|
|
|
|
|
|
|
|
An @emph{sshlogin} is of the form:
|
|
|
|
|
|
|
|
@verbatim
|
|
|
|
[sshcommand [options]][username@]hostname
|
|
|
|
@end verbatim
|
|
|
|
|
|
|
|
The sshlogin must not require a password.
|
|
|
|
|
|
|
|
The sshlogin ':' is special, it means 'no ssh' and will therefore run
|
|
|
|
on the local computer.
|
|
|
|
|
|
|
|
The sshlogin '..' is special, it read sshlogins from ~/.parallel/sshloginfile
|
|
|
|
|
|
|
|
The sshlogin '-' is special, too, it read sshlogins from stdin
|
|
|
|
(standard input).
|
|
|
|
|
|
|
|
To specify more sshlogins separate the sshlogins by comma or repeat
|
|
|
|
the options multiple times.
|
|
|
|
|
|
|
|
For examples: see @strong{--sshloginfile}.
|
|
|
|
|
|
|
|
The remote host must have GNU @strong{parallel} installed.
|
|
|
|
|
|
|
|
@strong{--sshlogin} is known to cause problems with @strong{-m} and @strong{-X}.
|
|
|
|
|
|
|
|
@strong{--sshlogin} is often used with @strong{--transfer}, @strong{--return},
|
|
|
|
@strong{--cleanup}, and @strong{--trc}.
|
|
|
|
|
2012-03-22 23:29:46 +00:00
|
|
|
@item @strong{--sshloginfile} @emph{filename}
|
|
|
|
@anchor{@strong{--sshloginfile} @emph{filename}}
|
2012-01-24 20:53:25 +00:00
|
|
|
|
2012-03-22 23:29:46 +00:00
|
|
|
@item @strong{--slf} @emph{filename}
|
|
|
|
@anchor{@strong{--slf} @emph{filename}}
|
2012-01-24 20:53:25 +00:00
|
|
|
|
|
|
|
File with sshlogins. The file consists of sshlogins on separate
|
|
|
|
lines. Empty lines and lines starting with '#' are ignored. Example:
|
|
|
|
|
|
|
|
@verbatim
|
|
|
|
server.example.com
|
|
|
|
username@server2.example.com
|
|
|
|
8/my-8-core-server.example.com
|
|
|
|
2/my_other_username@my-dualcore.example.net
|
|
|
|
# This server has SSH running on port 2222
|
|
|
|
ssh -p 2222 server.example.net
|
|
|
|
4/ssh -p 2222 quadserver.example.net
|
|
|
|
# Use a different ssh program
|
|
|
|
myssh -p 2222 -l myusername hexacpu.example.net
|
|
|
|
# Use a different ssh program with default number of cores
|
|
|
|
//usr/local/bin/myssh -p 2222 -l myusername hexacpu.example.net
|
|
|
|
# Use a different ssh program with 6 cores
|
|
|
|
6//usr/local/bin/myssh -p 2222 -l myusername hexacpu.example.net
|
|
|
|
# Assume 16 cores on the local computer
|
|
|
|
16/:
|
|
|
|
@end verbatim
|
|
|
|
|
|
|
|
When using a different ssh program the last argument must be the hostname.
|
|
|
|
|
|
|
|
Multiple @strong{--sshloginfile} are allowed.
|
|
|
|
|
|
|
|
The sshloginfile '..' is special, it read sshlogins from
|
|
|
|
~/.parallel/sshloginfile
|
|
|
|
|
|
|
|
The sshloginfile '.' is special, it read sshlogins from
|
|
|
|
/etc/parallel/sshloginfile
|
|
|
|
|
|
|
|
The sshloginfile '-' is special, too, it read sshlogins from stdin
|
|
|
|
(standard input).
|
|
|
|
|
|
|
|
@item @strong{--noswap}
|
|
|
|
@anchor{@strong{--noswap}}
|
|
|
|
|
|
|
|
Do not start new jobs on a given computer if there is both swap-in and
|
|
|
|
swap-out activity.
|
|
|
|
|
|
|
|
The swap activity is only sampled every 10 seconds as the sampling
|
|
|
|
takes 1 second to do.
|
|
|
|
|
|
|
|
Swap activity is computed as (swap-in)*(swap-out) which in practice is
|
|
|
|
a good value: swapping out is not a problem, swapping in is not a
|
|
|
|
problem, but both swapping in and out usually indicates a problem.
|
|
|
|
|
|
|
|
@item @strong{--silent}
|
|
|
|
@anchor{@strong{--silent}}
|
|
|
|
|
|
|
|
Silent. The job to be run will not be printed. This is the default.
|
|
|
|
Can be reversed with @strong{-v}.
|
|
|
|
|
|
|
|
@item @strong{--tty}
|
|
|
|
@anchor{@strong{--tty}}
|
|
|
|
|
|
|
|
Open terminal tty. If GNU @strong{parallel} is used for starting an
|
|
|
|
interactive program then this option may be needed. It will start only
|
|
|
|
one job at a time (i.e. @strong{-j1}), not buffer the output (i.e. @strong{-u}),
|
|
|
|
and it will open a tty for the job. When the job is done, the next job
|
|
|
|
will get the tty.
|
|
|
|
|
|
|
|
@item @strong{--tag}
|
|
|
|
@anchor{@strong{--tag}}
|
|
|
|
|
|
|
|
Tag lines with arguments. Each output line will be prepended with the
|
|
|
|
arguments and TAB (\t). When combined with @strong{--onall} or @strong{--nonall}
|
|
|
|
the lines will be prepended with the sshlogin instead.
|
|
|
|
|
|
|
|
@strong{--tag} is ignored when using @strong{-u}.
|
|
|
|
|
2012-05-06 10:43:40 +00:00
|
|
|
@item @strong{--tagstring} @emph{str}
|
|
|
|
@anchor{@strong{--tagstring} @emph{str}}
|
|
|
|
|
|
|
|
Tag lines with a string. Each output line will be prepended with
|
|
|
|
@emph{str} and TAB (\t). @emph{str} can contain replacement strings such as
|
|
|
|
@{@}.
|
|
|
|
|
|
|
|
@strong{--tagstring} is ignored when using @strong{-u}, @strong{--onall}, and @strong{--nonall}.
|
|
|
|
|
2012-01-24 20:53:25 +00:00
|
|
|
@item @strong{--tmpdir} @emph{dirname}
|
|
|
|
@anchor{@strong{--tmpdir} @emph{dirname}}
|
|
|
|
|
|
|
|
Directory for temporary files. GNU @strong{parallel} normally buffers output
|
|
|
|
into temporary files in /tmp. By setting @strong{--tmpdir} you can use a
|
|
|
|
different dir for the files. Setting @strong{--tmpdir} is equivalent to
|
|
|
|
setting $TMPDIR.
|
|
|
|
|
2012-02-20 23:40:47 +00:00
|
|
|
@item @strong{--timeout} @emph{sec}
|
|
|
|
@anchor{@strong{--timeout} @emph{sec}}
|
2012-01-24 20:53:25 +00:00
|
|
|
|
|
|
|
Time out for command. If the command runs for longer than @emph{sec}
|
|
|
|
seconds it will get killed with SIGTERM, followed by SIGTERM 200 ms
|
|
|
|
later, followed by SIGKILL 200 ms later.
|
|
|
|
|
|
|
|
@item @strong{--tollef}
|
|
|
|
@anchor{@strong{--tollef}}
|
|
|
|
|
2012-05-27 15:51:09 +00:00
|
|
|
Make GNU @strong{parallel} behave more like Tollef's parallel command. It
|
|
|
|
activates @strong{-u}, @strong{-q}, and @strong{--arg-sep --}. It also causes @strong{-l} to
|
|
|
|
change meaning to @strong{--load}.
|
|
|
|
|
|
|
|
Not giving '--' is unsupported.
|
|
|
|
|
|
|
|
@strong{Do not use --tollef unless you know what you are doing}.
|
|
|
|
|
|
|
|
To override use @strong{--gnu}.
|
2012-01-24 20:53:25 +00:00
|
|
|
|
|
|
|
@item @strong{--verbose}
|
|
|
|
@anchor{@strong{--verbose}}
|
|
|
|
|
|
|
|
@item @strong{-t}
|
|
|
|
@anchor{@strong{-t}}
|
|
|
|
|
|
|
|
Print the job to be run on stderr (standard error).
|
|
|
|
|
|
|
|
See also @strong{-v} and @strong{-p}.
|
|
|
|
|
|
|
|
@item @strong{--transfer}
|
|
|
|
@anchor{@strong{--transfer}}
|
|
|
|
|
|
|
|
Transfer files to remote computers. @strong{--transfer} is used with
|
|
|
|
@strong{--sshlogin} when the arguments are files and should be transferred to
|
|
|
|
the remote computers. The files will be transferred using @strong{rsync} and
|
|
|
|
will be put relative to the default login dir. E.g.
|
|
|
|
|
|
|
|
@verbatim
|
|
|
|
echo foo/bar.txt | parallel \
|
|
|
|
--sshlogin server.example.com --transfer wc
|
|
|
|
@end verbatim
|
|
|
|
|
|
|
|
This will transfer the file @emph{foo/bar.txt} to the computer
|
|
|
|
@emph{server.example.com} to the file @emph{$HOME/foo/bar.txt} before running
|
|
|
|
@strong{wc foo/bar.txt} on @emph{server.example.com}.
|
|
|
|
|
|
|
|
@verbatim
|
|
|
|
echo /tmp/foo/bar.txt | parallel \
|
|
|
|
--sshlogin server.example.com --transfer wc
|
|
|
|
@end verbatim
|
|
|
|
|
|
|
|
This will transfer the file @emph{foo/bar.txt} to the computer
|
|
|
|
@emph{server.example.com} to the file @emph{/tmp/foo/bar.txt} before running
|
|
|
|
@strong{wc /tmp/foo/bar.txt} on @emph{server.example.com}.
|
|
|
|
|
|
|
|
@strong{--transfer} is often used with @strong{--return} and @strong{--cleanup}.
|
|
|
|
|
|
|
|
@strong{--transfer} is ignored when used with @strong{--sshlogin :} or when not used with @strong{--sshlogin}.
|
|
|
|
|
|
|
|
@item @strong{--trc} @emph{filename}
|
|
|
|
@anchor{@strong{--trc} @emph{filename}}
|
|
|
|
|
|
|
|
Transfer, Return, Cleanup. Short hand for:
|
|
|
|
|
|
|
|
@strong{--transfer} @strong{--return} @emph{filename} @strong{--cleanup}
|
|
|
|
|
|
|
|
@item @strong{--trim} <n|l|r|lr|rl>
|
|
|
|
@anchor{@strong{--trim} <n|l|r|lr|rl>}
|
|
|
|
|
|
|
|
Trim white space in input.
|
|
|
|
|
|
|
|
@table @asis
|
|
|
|
@item n
|
|
|
|
@anchor{n}
|
|
|
|
|
|
|
|
No trim. Input is not modified. This is the default.
|
|
|
|
|
|
|
|
@item l
|
|
|
|
@anchor{l}
|
|
|
|
|
|
|
|
Left trim. Remove white space from start of input. E.g. " a bc " -> "a bc ".
|
|
|
|
|
|
|
|
@item r
|
|
|
|
@anchor{r}
|
|
|
|
|
|
|
|
Right trim. Remove white space from end of input. E.g. " a bc " -> " a bc".
|
|
|
|
|
|
|
|
@item lr
|
|
|
|
@anchor{lr}
|
|
|
|
|
|
|
|
@item rl
|
|
|
|
@anchor{rl}
|
|
|
|
|
|
|
|
Both trim. Remove white space from both start and end of input. E.g. "
|
|
|
|
a bc " -> "a bc". This is the default if @strong{--colsep} is used.
|
|
|
|
|
|
|
|
@end table
|
|
|
|
|
|
|
|
@item @strong{--ungroup}
|
|
|
|
@anchor{@strong{--ungroup}}
|
|
|
|
|
|
|
|
@item @strong{-u}
|
|
|
|
@anchor{@strong{-u}}
|
|
|
|
|
2012-05-06 10:43:40 +00:00
|
|
|
Ungroup output. Output is printed as soon as possible and by passes
|
2012-05-13 14:03:41 +00:00
|
|
|
GNU @strong{parallel} internal processing. This may cause output from
|
|
|
|
different commands to be mixed thus should only be used if you do not
|
|
|
|
care about the output. Compare these:
|
2012-05-06 10:43:40 +00:00
|
|
|
|
|
|
|
@strong{parallel -j0 'sleep @{@};echo -n start@{@};sleep @{@};echo @{@}end' ::: 1 2 3 4}
|
|
|
|
|
|
|
|
@strong{parallel -u -j0 'sleep @{@};echo -n start@{@};sleep @{@};echo @{@}end' ::: 1 2 3 4}
|
|
|
|
|
|
|
|
It also disables @strong{--tag}. GNU @strong{parallel} runs faster with @strong{-u}. Can
|
|
|
|
be reversed with @strong{--group}.
|
2012-01-24 20:53:25 +00:00
|
|
|
|
|
|
|
@item @strong{--extensionreplace} @emph{replace-str}
|
|
|
|
@anchor{@strong{--extensionreplace} @emph{replace-str}}
|
|
|
|
|
|
|
|
@item @strong{--er} @emph{replace-str}
|
|
|
|
@anchor{@strong{--er} @emph{replace-str}}
|
|
|
|
|
|
|
|
Use the replacement string @emph{replace-str} instead of @{.@} for input line without extension.
|
|
|
|
|
|
|
|
@item @strong{--use-cpus-instead-of-cores}
|
|
|
|
@anchor{@strong{--use-cpus-instead-of-cores}}
|
|
|
|
|
|
|
|
Count the number of physical CPUs instead of CPU cores. When computing
|
|
|
|
how many jobs to run simultaneously relative to the number of CPU cores
|
|
|
|
you can ask GNU @strong{parallel} to instead look at the number of physical
|
|
|
|
CPUs. This will make sense for computers that have hyperthreading as
|
|
|
|
two jobs running on one CPU with hyperthreading will run slower than
|
|
|
|
two jobs running on two physical CPUs. Some multi-core CPUs can run
|
|
|
|
faster if only one thread is running per physical CPU. Most users will
|
|
|
|
not need this option.
|
|
|
|
|
|
|
|
@item @strong{-v}
|
|
|
|
@anchor{@strong{-v}}
|
|
|
|
|
|
|
|
Verbose. Print the job to be run on stdout (standard output). Can be reversed
|
|
|
|
with @strong{--silent}. See also @strong{-t}.
|
|
|
|
|
|
|
|
Use @strong{-v} @strong{-v} to print the wrapping ssh command when running remotely.
|
|
|
|
|
|
|
|
@item @strong{--version}
|
|
|
|
@anchor{@strong{--version}}
|
|
|
|
|
|
|
|
@item @strong{-V}
|
|
|
|
@anchor{@strong{-V}}
|
|
|
|
|
|
|
|
Print the version GNU @strong{parallel} and exit.
|
|
|
|
|
2012-04-21 22:14:20 +00:00
|
|
|
@item @strong{--workdir} @emph{mydir}
|
|
|
|
@anchor{@strong{--workdir} @emph{mydir}}
|
2012-01-24 20:53:25 +00:00
|
|
|
|
2012-04-21 22:14:20 +00:00
|
|
|
@item @strong{--wd} @emph{mydir}
|
|
|
|
@anchor{@strong{--wd} @emph{mydir}}
|
2012-01-24 20:53:25 +00:00
|
|
|
|
|
|
|
Files transferred using @strong{--transfer} and @strong{--return} will be relative
|
|
|
|
to @emph{mydir} on remote computers, and the command will be executed in
|
2012-02-20 00:48:28 +00:00
|
|
|
the dir @emph{mydir}.
|
|
|
|
|
|
|
|
The special @emph{mydir} value @strong{...} will create working dirs under
|
|
|
|
@strong{~/.parallel/tmp/} on the remote computers. If @strong{--cleanup} is given
|
|
|
|
these dirs will be removed.
|
|
|
|
|
|
|
|
The special @emph{mydir} value @strong{.} uses the current working dir. If the
|
|
|
|
current working dir is beneath your home dir, the value @strong{.} is
|
|
|
|
treated as the relative path to your home dir. This means that if your
|
|
|
|
home dir is different on remote computers (e.g. if your login is
|
|
|
|
different) the relative path will still be relative to your home dir.
|
2012-01-24 20:53:25 +00:00
|
|
|
|
|
|
|
@item @strong{--wait}
|
|
|
|
@anchor{@strong{--wait}}
|
|
|
|
|
|
|
|
Wait for all commands to complete.
|
|
|
|
|
|
|
|
Implies @strong{--semaphore}.
|
|
|
|
|
|
|
|
See also: @strong{man sem}
|
|
|
|
|
|
|
|
@item @strong{-X}
|
|
|
|
@anchor{@strong{-X}}
|
|
|
|
|
|
|
|
Multiple arguments with context replace. Insert as many arguments as
|
|
|
|
the command line length permits. If multiple jobs are being run in
|
|
|
|
parallel: distribute the arguments evenly among the jobs. Use @strong{-j1}
|
|
|
|
to avoid this.
|
|
|
|
|
|
|
|
If @strong{@{@}} is not used the arguments will be appended to the line. If
|
|
|
|
@strong{@{@}} is used as part of a word (like @emph{pic@{@}.jpg}) then the whole
|
|
|
|
word will be repeated. If @strong{@{@}} is used multiple times each @strong{@{@}} will
|
|
|
|
be replaced with the arguments.
|
|
|
|
|
|
|
|
Normally @strong{-X} will do the right thing, whereas @strong{-m} can give
|
|
|
|
unexpected results if @strong{@{@}} is used as part of a word.
|
|
|
|
|
|
|
|
Support for @strong{-X} with @strong{--sshlogin} is limited and may fail.
|
|
|
|
|
|
|
|
See also @strong{-m}.
|
|
|
|
|
|
|
|
@item @strong{--exit}
|
|
|
|
@anchor{@strong{--exit}}
|
|
|
|
|
|
|
|
@item @strong{-x}
|
|
|
|
@anchor{@strong{-x}}
|
|
|
|
|
|
|
|
Exit if the size (see the @strong{-s} option) is exceeded.
|
|
|
|
|
|
|
|
@item @strong{--xapply}
|
|
|
|
@anchor{@strong{--xapply}}
|
|
|
|
|
|
|
|
Read multiple input sources like @strong{xapply}. If multiple input sources
|
|
|
|
are given, one argument will be read from each of the input
|
|
|
|
sources. The arguments can be accessed in the command as @strong{@{1@}}
|
|
|
|
.. @strong{@{}@emph{n}@strong{@}}, so @strong{@{1@}} will be a line from the first input source, and
|
|
|
|
@strong{@{6@}} will refer to the line with the same line number from the 6th
|
|
|
|
input source.
|
|
|
|
|
|
|
|
Compare these two:
|
|
|
|
|
|
|
|
@verbatim
|
|
|
|
parallel echo {1} {2} ::: 1 2 3 ::: a b c
|
2012-06-14 21:13:11 +00:00
|
|
|
parallel --xapply e
|