mirror of
https://git.savannah.gnu.org/git/parallel.git
synced 2024-12-22 12:47:54 +00:00
parallel: test -E for ::: and ::::
This commit is contained in:
parent
24dc6e55c4
commit
b0a7234152
|
@ -4,8 +4,9 @@ So far GNU Parallel has been focused on replacing a single
|
|||
for-loop. The Pakistan release introduces a way to replace nested
|
||||
loops.
|
||||
|
||||
As example I will use the image manipulation program 'convert'. This
|
||||
will convert foo.png to jpg with a size of 800 and JPEG-quality of 95.
|
||||
As example I will use the image manipulation program 'convert' from
|
||||
ImageMagick. This will convert foo.png to jpg with a size of 800 and
|
||||
JPEG-quality of 95.
|
||||
|
||||
convert -size 800 -quality 95 foo.png foo_800_q95.jpg
|
||||
|
||||
|
@ -13,7 +14,7 @@ With a for-loop it can be done on a list of files:
|
|||
|
||||
time \
|
||||
for file in *.png ; do
|
||||
convert -size 800 -quality 95 $file ${file##.JPG}_800_q95.jpg
|
||||
convert -size 800 -quality 95 $file ${file%.*}_800_q95.jpg
|
||||
done
|
||||
|
||||
Using GNU Parallel it looks like this:
|
||||
|
@ -26,7 +27,7 @@ To get the images in 3 different JPEG-qualities you can use a nested for-loop:
|
|||
time \
|
||||
for qual in 25 50 95 ; do
|
||||
for file in *.png ; do
|
||||
convert -size 800 -quality $qual $file ${file##.JPG}_800_q${qual}.jpg
|
||||
convert -size 800 -quality $qual $file ${file%.*}_800_q${qual}.jpg
|
||||
done
|
||||
done
|
||||
|
||||
|
@ -34,14 +35,14 @@ With GNU Parallel 'Pakistan' you can do this:
|
|||
|
||||
time parallel convert -size 800 -quality 95 {1} {1.}_800_q{2}.jpg ::: *.png ::: 25 50 95
|
||||
|
||||
To get the 3 different JPEG-qualities in 2 different sizes you can use
|
||||
a nest the for-loop even further:
|
||||
To get the 3 different JPEG-qualities in 2 different sizes you can
|
||||
nest the for-loop even further:
|
||||
|
||||
time \
|
||||
for size in 800 30 ; do
|
||||
for qual in 25 50 95 ; do
|
||||
for file in *.png ; do
|
||||
convert -size $size -quality $qual $file ${file##.JPG}_${size}_q${qual}.jpg
|
||||
convert -size $size -quality $qual $file ${file%.*}_${size}_q${qual}.jpg
|
||||
done
|
||||
done
|
||||
done
|
||||
|
@ -68,24 +69,6 @@ The special file '-' reads from standard input. This will do the same as above:
|
|||
ls *.png | time parallel convert -size {3} -quality {2} {1} {1.}_{3}_q{2}.jpg :::: - ::: 25 50 95 :::: sizes
|
||||
|
||||
|
||||
Test with first argument == END_OF_FILE string
|
||||
|
||||
unittest til xapply og uden.
|
||||
|
||||
parallel echo {1} {2} {3} ::: a b c :::: myfile ::: X Y
|
||||
|
||||
parallel oggenc -q {1} {2} {2.}_{1}.ogg ::: 1 5 10 :::: wavfiles
|
||||
|
||||
or
|
||||
|
||||
parallel oggenc -q {1} {2} {2.}_{1}.ogg ::: 1 5 10 ::: *.wav
|
||||
|
||||
or
|
||||
|
||||
ls *.wav | parallel oggenc -q {1} {2} {2.}_{1}.ogg ::: 1 5 10 :::: -
|
||||
|
||||
|
||||
|
||||
Postkort:
|
||||
- Forside kun figur
|
||||
- Bagside:
|
||||
|
|
|
@ -167,24 +167,30 @@ This is a major release. So far GNU Parallel has been focused on
|
|||
replacing a single for-loop. The Pakistan release introduces ways to
|
||||
replace nested loops.
|
||||
|
||||
If you are using the {1} {2} syntax for multiple input sources, then you
|
||||
need to read about --xapply as the function has changed.
|
||||
|
||||
New in this release:
|
||||
|
||||
* Multiple ::: can be put on the command line and will be treated
|
||||
similar to contents from multiple -a's.
|
||||
|
||||
* ::: and :::: can now be mixed on the command line. Use {1} .. {n} to
|
||||
refer to the arguments from the different arguments.
|
||||
refer to inputs from the different input sources.
|
||||
|
||||
* --xapply is now needed to get xapply's behaviour of reading one line
|
||||
from arguments from each of multiple input files.
|
||||
from each of the input sources.
|
||||
|
||||
* Multiple input source will cause all combinations of arguments from
|
||||
the sources to be generated. E.g. ::: 1 2 ::: a b c will result in
|
||||
the combinations (1,a) (1,b) (1,c) (2,a) (2,b) (2,c). This can often
|
||||
replace nested loops.
|
||||
* Multiple input sources will cause all combinations of arguments from
|
||||
the sources to be generated. E.g. 'parallel echo {1}+{2} ::: 1 2 :::
|
||||
a b c' will print 1+a 1+b 1+c 2+a 2+b 2+c. This can often replace
|
||||
nested loops.
|
||||
|
||||
* Implemented {//} for the input line with the basename removed (dirname).
|
||||
|
||||
* Article about GNU SQL in USENIX Magazine ;login: (print)
|
||||
http://www.usenix.org/publications/login/2011-04/
|
||||
|
||||
* Using GNU Parallel with EC2. Thanks to Kevin Wu.
|
||||
http://blog.kevinformatics.com/post/4970574713/interested-in-your-experience-using-gnu-parallel-in
|
||||
|
||||
|
@ -203,6 +209,15 @@ New in this release:
|
|||
* Short article about using GNU Parallel with lame:
|
||||
http://loopkid.net/articles/2011/04/30/accelerate-lame-mp3-conversion
|
||||
|
||||
* BBC Research & Development uses GNU Parallel:
|
||||
http://www.bbc.co.uk/blogs/researchanddevelopment/2010/11/prototyping-weeknotes-41-26112.shtml
|
||||
|
||||
* Short article about using GNU Parallel on RHEL. Thanks to Rishi Deshpande.
|
||||
http://nuclear-imaging.info/site_content/2011/05/11/gnu-parallel/
|
||||
|
||||
* Using GNU Parallel for FLAC->MP3 conversion. Thanks to Derek Marcotte.
|
||||
http://derek.chezmarcotte.ca/?p=286
|
||||
|
||||
* Bug fixes and man page updates.
|
||||
|
||||
|
||||
|
@ -220,7 +235,7 @@ If you use xargs and tee today you will find GNU Parallel very easy to
|
|||
use as GNU Parallel is written to have the same options as xargs. If
|
||||
you write loops in shell, you will find GNU Parallel may be able to
|
||||
replace most of the loops and make them run faster by running several
|
||||
jobs in parallel.
|
||||
jobs in parallel. GNU Parallel can even replace nested loops.
|
||||
|
||||
GNU Parallel makes sure output from the commands is the same output as
|
||||
you would get had you run the commands sequentially. This makes it
|
||||
|
@ -236,4 +251,22 @@ When using GNU Parallel for a publication please cite:
|
|||
|
||||
O. Tange (2011): GNU Parallel - The Command-Line Power Tool, ;login:
|
||||
The USENIX Magazine, February 2011:42-47.
|
||||
|
||||
|
||||
= About GNU SQL =
|
||||
|
||||
GNU sql aims to give a simple, unified interface for accessing
|
||||
databases through all the different databases' command line
|
||||
clients. So far the focus has been on giving a common way to specify
|
||||
login information (protocol, username, password, hostname, and port
|
||||
number), size (database and table size), and running queries.
|
||||
|
||||
The database is addressed using a DBURL. If commands are left out you
|
||||
will get that database's interactive shell.
|
||||
|
||||
When using GNU SQL for a publication please cite:
|
||||
|
||||
O. Tange (2011): GNU SQL - A Command Line Tool for Accessing Different
|
||||
Databases Using DBURLs, ;login: The USENIX Magazine, April
|
||||
2011.
|
||||
>>>>>
|
||||
|
|
|
@ -100,7 +100,7 @@ B<{/}> can be used the same places as B<{}>. The replacement string
|
|||
B<{/}> can be changed with B<--basenamereplace>.
|
||||
|
||||
|
||||
=item B<{//}>
|
||||
=item B<{//}> (alpha testing)
|
||||
|
||||
Dirname of input line. This is a specialized replacement string
|
||||
containing the dir of the input. See B<dirname>(1).
|
||||
|
@ -128,7 +128,7 @@ The replacement string B<{#}> can be changed with B<--seqreplace>.
|
|||
|
||||
=item B<{>I<n>B<}>
|
||||
|
||||
Argument from argument file I<n> or the I<n>'th argument. See B<-a>
|
||||
Argument from input source I<n> or the I<n>'th argument. See B<-a>
|
||||
and B<-N>.
|
||||
|
||||
B<{>I<n>B<}> can be used the same places as B<{}>.
|
||||
|
@ -136,7 +136,7 @@ B<{>I<n>B<}> can be used the same places as B<{}>.
|
|||
|
||||
=item B<{>I<n>.B<}>
|
||||
|
||||
Argument from argument file I<n> or the I<n>'th argument without
|
||||
Argument from input source I<n> or the I<n>'th argument without
|
||||
extension. It is a combination of B<{>I<n>B<}> and B<{.}>.
|
||||
|
||||
B<{>I<n>.B<}> can be used the same places as B<{>I<n>B<}>.
|
||||
|
@ -144,7 +144,7 @@ B<{>I<n>.B<}> can be used the same places as B<{>I<n>B<}>.
|
|||
|
||||
=item B<{>I<n>/B<}>
|
||||
|
||||
Basename of argument from argument file I<n> or the I<n>'th argument.
|
||||
Basename of argument from input source I<n> or the I<n>'th argument.
|
||||
It is a combination of B<{>I<n>B<}> and B<{/}>. See B<-a> and B<-N>.
|
||||
|
||||
B<{>I<n>/B<}> can be used the same places as B<{>I<n>B<}>.
|
||||
|
@ -152,7 +152,7 @@ B<{>I<n>/B<}> can be used the same places as B<{>I<n>B<}>.
|
|||
|
||||
=item B<{>I<n>/.B<}>
|
||||
|
||||
Basename of argument from argument file I<n> or the I<n>'th argument
|
||||
Basename of argument from input source I<n> or the I<n>'th argument
|
||||
without extension. It is a combination of B<{>I<n>B<}>, B<{/}>, and
|
||||
B<{.}>. See B<-a> and B<-N>.
|
||||
|
||||
|
@ -161,9 +161,9 @@ B<{>I<n>/.B<}> can be used the same places as B<{>I<n>B<}>.
|
|||
|
||||
=item B<:::> I<arguments> (alpha testing)
|
||||
|
||||
Use arguments from the command line as input instead of from stdin
|
||||
(standard input). Unlike other options for GNU B<parallel> B<:::> is
|
||||
placed after the I<command> and before the arguments.
|
||||
Use arguments from the command line as input source instead of from
|
||||
stdin (standard input). Unlike other options for GNU B<parallel>
|
||||
B<:::> is placed after the I<command> and before the arguments.
|
||||
|
||||
The following are equivalent:
|
||||
|
||||
|
@ -179,10 +179,11 @@ argument separator to something else. See also B<--arg-sep>.
|
|||
|
||||
stdin (standard input) will be passed to the first process run.
|
||||
|
||||
If multiple B<:::> are given, all combinations of group of arguments
|
||||
will be generated. E.g. ::: 1 2 ::: a b c will result in the
|
||||
combinations (1,a) (1,b) (1,c) (2,a) (2,b) (2,c). This is useful for
|
||||
replacing nested for-loops.
|
||||
If multiple B<:::> are given, each group will be treated as an input
|
||||
source, and all combinations of input sources will be
|
||||
generated. E.g. ::: 1 2 ::: a b c will result in the combinations
|
||||
(1,a) (1,b) (1,c) (2,a) (2,b) (2,c). This is useful for replacing
|
||||
nested for-loops.
|
||||
|
||||
B<:::> and B<::::> can be mixed. So these are equivalent:
|
||||
|
||||
|
@ -198,6 +199,8 @@ B<:::> and B<::::> can be mixed. So these are equivalent:
|
|||
|
||||
Another way to write B<-a> I<argfile1> B<-a> I<argfile2> ...
|
||||
|
||||
B<:::> and B<::::> can be mixed.
|
||||
|
||||
See B<-a>.
|
||||
|
||||
|
||||
|
@ -214,21 +217,20 @@ for processing arguments that may contain \n (newline).
|
|||
|
||||
=item B<-a> I<input-file>
|
||||
|
||||
Read items from the file I<input-file> instead of stdin (standard input). If
|
||||
you use this option, stdin is given to the first process run.
|
||||
Otherwise, stdin is redirected from /dev/null.
|
||||
Use I<input-file> as input source. If you use this option, stdin is
|
||||
given to the first process run. Otherwise, stdin is redirected from
|
||||
/dev/null.
|
||||
|
||||
If multiple B<-a> are given, all combinations of all lines will be
|
||||
generated. E.g. 1 2 and a b c will result in the combinations (1,a)
|
||||
(1,b) (1,c) (2,a) (2,b) (2,c). This is useful for replacing nested
|
||||
for-loops.
|
||||
|
||||
The arguments can be accessed in the command as B<{1}>
|
||||
.. B<{>I<n>B<}>, so B<{1}> will be a line from the first file, and
|
||||
B<{6}> will refer to a line from the 6th file.
|
||||
If multiple B<-a> are given, each I<input-file> will be treated as an
|
||||
input source, and all combinations of input sources will be
|
||||
generated. E.g. The file B<foo> contains B<1 2>, the file B<bar>
|
||||
contains B<a b c>. B<-a foo> B<-a bar> will result in the combinations
|
||||
(1,a) (1,b) (1,c) (2,a) (2,b) (2,c). This is useful for replacing
|
||||
nested for-loops.
|
||||
|
||||
See also B<--xapply>
|
||||
|
||||
|
||||
=item B<--arg-file-sep> I<sep-str>
|
||||
|
||||
Use I<sep-str> instead of B<::::> as separator string between command
|
||||
|
|
|
@ -18,3 +18,15 @@ seq 6 7 | parallel -k -a - -a <(seq 4 5) echo {1} {2} {3} ::: 1 2 3
|
|||
|
||||
echo '### Test :::: < - :::'
|
||||
seq 4 5 | parallel -k echo {1} {2} {3} :::: <(seq 6 7) - ::: 1 2 3
|
||||
|
||||
echo '### Test -E'
|
||||
seq 1 100 | parallel -k -E 5 echo :::: - ::: 2 3 4 5 6 7 8 9 10 :::: <(seq 3 11)
|
||||
|
||||
echo '### Test -E one empty'
|
||||
seq 1 100 | parallel -k -E 3 echo :::: - ::: 2 3 4 5 6 7 8 9 10 :::: <(seq 3 11)
|
||||
|
||||
echo '### Test -E 2 empty'
|
||||
seq 1 100 | parallel -k -E 3 echo :::: - ::: 3 4 5 6 7 8 9 10 :::: <(seq 3 11)
|
||||
|
||||
echo '### Test -E all empty'
|
||||
seq 3 100 | parallel -k -E 3 echo :::: - ::: 3 4 5 6 7 8 9 10 :::: <(seq 3 11)
|
||||
|
|
Loading…
Reference in a new issue