FIXED BUG: Negaive time: time parallel -j+0 --eta -Sserver1,server2,server3,: \

+--transfer --return {.}.bz2 --cleanup 'zcat {} | bzip2 -9 > {.}.bz2' ::: *gz.
FIXED BUG: echo "foo,bar,baz" | parallel -d, -L 1 echo.
This commit is contained in:
Ole Tange 2010-07-13 21:24:03 +02:00
parent ff229d0bd4
commit e33b8e0d01
2 changed files with 90 additions and 12 deletions

View file

@ -1,3 +1,42 @@
FIXED BUG: time parallel -j+0 --eta -Sserver1,server2,server3,: \
--transfer --return {.}.bz2 --cleanup 'zcat {} | bzip2 -9 > {.}.bz2' ::: *gz
Time is negative
FIXED BUG: echo "foo,bar,baz" | parallel -d, -L 1 echo
Gave warning.
xapply compatability
xapply -p/dev/null -f 'diff %1 %2' manifest1 checklist1
BUG: parallel diff {1} {2} :::: manifest1 checklist1
tr ':' '\012' < /etc/passwd | xapply -7 -nf 'chown %1 %6' - - - - - - -
BUG: tr ':' '\012' < /etc/passwd | parallel -N7 chown {1} {6}
xapply '(cd %1 && make all)' */
parallel 'cd {} && make all' ::: *
xapply -f 'diff %1 ../version5/%1' manifest | more
parallel diff {} ../version5/{} < manifest | more
xapply 'indent' *.c
parallel indent ::: *.c
find ~ksb/bin -type f ! -perm -111 -print | xapply -f -v 'chmod a+x' -
find ~ksb/bin -type f ! -perm -111 -print | parallel -v chmod a+x
find ... | xapply -f -5 -i /dev/tty 'vi' - - - - -
sh <(find ... |parallel -n5 echo vi)
xapply -fn "" /etc/passwd
parallel -k echo /etc/passwd
xapply '[ -d %1/RCS ] || echo %1' */
parallel '[ -d {}/RCS ] || echo {}' ::: *
xapply -f '[ -f %1 ] && echo %1' List | ...
parallel '[ -f {} ] && echo {}' < List
# Hvordan udregnes system limits på remote systems hvis jeg ikke ved, hvormange # Hvordan udregnes system limits på remote systems hvis jeg ikke ved, hvormange
# argumenter, der er? Lav system limits lokalt og lad det være max # argumenter, der er? Lav system limits lokalt og lad det være max

View file

@ -6,7 +6,9 @@ parallel - build and execute shell command lines from standard input in parallel
=head1 SYNOPSIS =head1 SYNOPSIS
B<parallel> [options] [I<command> [arguments]] [< list_of_arguments] B<parallel> [options] [I<command> [arguments]] < list_of_arguments
B<parallel> [options] [I<command> [arguments]] B<:::> arguments
=head1 DESCRIPTION =head1 DESCRIPTION
@ -20,12 +22,13 @@ If you use B<xargs> today you will find GNU B<parallel> very easy to
use as GNU B<parallel> is written to have the same options as use as GNU B<parallel> is written to have the same options as
B<xargs>. If you write loops in shell, you will find GNU B<parallel> B<xargs>. If you write loops in shell, you will find GNU B<parallel>
may be able to replace most of the loops and make them run faster by may be able to replace most of the loops and make them run faster by
running several jobs in parallel. If you use B<ppss> or B<pexec> you will find running several jobs in parallel. If you use B<ppss> or B<pexec> you
GNU B<parallel> will often make the command easier to read. will find GNU B<parallel> will often make the command easier to read.
GNU B<parallel> makes sure output from the commands is the same output as GNU B<parallel> makes sure output from the commands is the same output
you would get had you run the commands sequentially. This makes it as you would get had you run the commands sequentially. This makes it
possible to use output from GNU B<parallel> as input for other programs. possible to use output from GNU B<parallel> as input for other
programs.
For each line of input GNU B<parallel> will execute I<command> with For each line of input GNU B<parallel> will execute I<command> with
the line as arguments. If no I<command> is given, the line of input is the line as arguments. If no I<command> is given, the line of input is
@ -77,9 +80,9 @@ B<{.}> can be changed with B<-U>.
=item B<:::> (beta testing) =item B<:::> (beta testing)
Use arguments on the command line as input instead of stdin (standard Use arguments from the command line as input instead of from stdin
input). Unlike other options for GNU B<parallel> B<:::> is placed (standard input). Unlike other options for GNU B<parallel> B<:::> is
after the command and before the arguments. placed after the I<command> and before the arguments.
The following are equivalent: The following are equivalent:
@ -98,6 +101,14 @@ stdin (standard input) will be passed to the first process run.
If B<--arg-file> is set arguments from that file will be appended. If B<--arg-file> is set arguments from that file will be appended.
=item B<::::> (unimplemented)
Use arguments from the command line as files for input. One line will
be read from each of the files. The arguments can be access in the
command as {1} .. {n}, so {1} will be a line from the first file, and
{6} will refer to the corresponding line from the 6th file.
=item B<--null> =item B<--null>
=item B<-0> =item B<-0>
@ -334,7 +345,7 @@ end in the sequence 3 1 4 2 the output will still be 1 2 3 4.
Use at most I<max-lines> nonblank input lines per command line. Use at most I<max-lines> nonblank input lines per command line.
Trailing blanks cause an input line to be logically continued on the Trailing blanks cause an input line to be logically continued on the
next input line. Implies B<-x>. next input line.
Implies B<-X> unless B<-m> is set. Implies B<-X> unless B<-m> is set.
@ -387,6 +398,21 @@ GNU B<parallel> will exit.
Implies B<-X> unless B<-m> is set. Implies B<-X> unless B<-m> is set.
=item B<--max-replace-args>=I<max-args> (unimplemented)
=item B<-N> I<max-args> (unimplemented)
Use at most I<max-args> arguments per command line. Like B<-n> but
also makes replacement strings {1} .. {I<max-args>} that represents
argument 1 .. I<max-args>. If too few args the {number} will be empty.
This will set the owner of the homedir to the user:
B<tr ':' '\012' < /etc/passwd | parallel -N7 chown {1} {6}>
Implies B<-X> unless B<-m> is set.
=item B<--max-line-length-allowed> =item B<--max-line-length-allowed>
Print the maximal number characters allowed on the command line and Print the maximal number characters allowed on the command line and
@ -778,6 +804,10 @@ To put the output in a file called <name>.dir:
B<ls | parallel '(echo -n {}" "; ls {}|wc -l) >> B<{}.dir'> B<ls | parallel '(echo -n {}" "; ls {}|wc -l) >> B<{}.dir'>
Even small shell scripts can be run by GNU B<parallel>:
B<find . | parallel 'a={}; name=${a##*/}; upper=$(echo "$name" | tr "[:lower:]" "[:upper:]"); echo "$name - $upper"'>
=head1 EXAMPLE: Removing file extension when processing files =head1 EXAMPLE: Removing file extension when processing files
@ -1023,6 +1053,14 @@ With the file I<mycomputers> containing the list of computers it becomes:
--trc {.}.bz2 "zcat {} | bzip2 -9 >{.}.bz2" --trc {.}.bz2 "zcat {} | bzip2 -9 >{.}.bz2"
=head1 EXAMPLE: Distributing work to local and remote computers
Convert *.mp3 to *.ogg running one process per CPU core on local computer and server2:
parallel --trc {.}.ogg -j+0 -S server2,: \
'mpg321 -w - {} | oggenc -q0 - -o {.}.ogg' ::: *.mp3
=head1 EXAMPLE: Working as cat | sh. Ressource inexpensive jobs and evaluation =head1 EXAMPLE: Working as cat | sh. Ressource inexpensive jobs and evaluation
GNU B<parallel> can work similar to B<cat | sh>. GNU B<parallel> can work similar to B<cat | sh>.
@ -2765,7 +2803,7 @@ sub get_next_arg {
return get_next_arg(); return get_next_arg();
} }
} }
if($Global::max_lines) { if($Global::max_lines and more_arguments()) {
if($arg =~ /\s$/) { if($arg =~ /\s$/) {
# Trailing space => continued on next line # Trailing space => continued on next line
$arg .= get_next_arg(); $arg .= get_next_arg();
@ -3502,7 +3540,8 @@ sub die_usage {
sub usage { sub usage {
# Returns: N/A # Returns: N/A
print "Usage:\n"; print "Usage:\n";
print "$Global::progname [options] [command [arguments]] < list_of_arguments\n"; print "$Global::progname [options] [command [arguments]] < list_of_arguments)\n";
print "$Global::progname [options] [command [arguments]] ::: arguments\n";
print "\n"; print "\n";
print "See 'man $Global::progname' for the options\n"; print "See 'man $Global::progname' for the options\n";
} }