parallel: test -E for ::: and ::::

doc/FUTURE_IDEAS

@@ -4,8 +4,9 @@ So far GNU Parallel has been focused on replacing a single
 for-loop. The Pakistan release introduces a way to replace nested
 loops.
 
-As example I will use the image manipulation program 'convert'. This
+As example I will use the image manipulation program 'convert' from
-will convert foo.png to jpg with a size of 800 and JPEG-quality of 95.
+ImageMagick. This will convert foo.png to jpg with a size of 800 and
+JPEG-quality of 95.
 
 convert -size 800 -quality 95 foo.png foo_800_q95.jpg
 
@@ -13,7 +14,7 @@ With a for-loop it can be done on a list of files:
 
 time \
 for file in *.png ; do
-  convert -size 800 -quality 95 $file ${file##.JPG}_800_q95.jpg
+  convert -size 800 -quality 95 $file ${file%.*}_800_q95.jpg
 done
 
 Using GNU Parallel it looks like this:
@@ -26,7 +27,7 @@ To get the images in 3 different JPEG-qualities you can use a nested for-loop:
 time \
 for qual in 25 50 95 ; do
   for file in *.png ; do
-    convert -size 800 -quality $qual $file ${file##.JPG}_800_q${qual}.jpg
+    convert -size 800 -quality $qual $file ${file%.*}_800_q${qual}.jpg
   done
 done
 
@@ -34,14 +35,14 @@ With GNU Parallel 'Pakistan' you can do this:
 
 time parallel convert -size 800 -quality 95 {1} {1.}_800_q{2}.jpg ::: *.png ::: 25 50 95
 
-To get the 3 different JPEG-qualities in 2 different sizes you can use
+To get the 3 different JPEG-qualities in 2 different sizes you can
-a nest the for-loop even further:
+nest the for-loop even further:
 
 time \
 for size in 800 30 ; do
   for qual in 25 50 95 ; do
     for file in *.png ; do
-      convert -size $size -quality $qual $file ${file##.JPG}_${size}_q${qual}.jpg
+      convert -size $size -quality $qual $file ${file%.*}_${size}_q${qual}.jpg
     done
   done
 done
@@ -68,24 +69,6 @@ The special file '-' reads from standard input. This will do the same as above:
 ls *.png | time parallel convert -size {3} -quality {2} {1} {1.}_{3}_q{2}.jpg :::: - ::: 25 50 95 :::: sizes
 
 
-Test with first argument == END_OF_FILE string
-
-unittest til xapply og uden.
-
-parallel echo {1} {2} {3} ::: a b c :::: myfile ::: X Y
-
-parallel oggenc -q {1} {2} {2.}_{1}.ogg ::: 1 5 10 :::: wavfiles
-
-or
-
-parallel oggenc -q {1} {2} {2.}_{1}.ogg ::: 1 5 10 ::: *.wav
-
-or
-
-ls *.wav | parallel oggenc -q {1} {2} {2.}_{1}.ogg ::: 1 5 10 :::: -
-
-
-
 Postkort:
 - Forside kun figur
 - Bagside:

doc/release_new_version

@@ -167,24 +167,30 @@ This is a major release. So far GNU Parallel has been focused on
 replacing a single for-loop. The Pakistan release introduces ways to
 replace nested loops.
 
+If you are using the {1} {2} syntax for multiple input sources, then you
+need to read about --xapply as the function has changed.
+
 New in this release:
 
 * Multiple ::: can be put on the command line and will be treated
   similar to contents from multiple -a's.
 
 * ::: and :::: can now be mixed on the command line. Use {1} .. {n} to
-  refer to the arguments from the different arguments.
+  refer to inputs from the different input sources.
 
 * --xapply is now needed to get xapply's behaviour of reading one line
-  from arguments from each of multiple input files.
+  from each of the input sources.
 
-* Multiple input source will cause all combinations of arguments from
+* Multiple input sources will cause all combinations of arguments from
-  the sources to be generated. E.g. ::: 1 2 ::: a b c will result in
+  the sources to be generated. E.g. 'parallel echo {1}+{2} ::: 1 2 :::
-  the combinations (1,a) (1,b) (1,c) (2,a) (2,b) (2,c). This can often
+  a b c' will print 1+a 1+b 1+c 2+a 2+b 2+c. This can often replace
-  replace nested loops.
+  nested loops.
 
 * Implemented {//} for the input line with the basename removed (dirname).
 
+* Article about GNU SQL in USENIX Magazine ;login: (print)
+  http://www.usenix.org/publications/login/2011-04/
+
 * Using GNU Parallel with EC2. Thanks to Kevin Wu.
   http://blog.kevinformatics.com/post/4970574713/interested-in-your-experience-using-gnu-parallel-in
 
@@ -203,6 +209,15 @@ New in this release:
 * Short article about using GNU Parallel with lame:
   http://loopkid.net/articles/2011/04/30/accelerate-lame-mp3-conversion
 
+* BBC Research & Development uses GNU Parallel:
+  http://www.bbc.co.uk/blogs/researchanddevelopment/2010/11/prototyping-weeknotes-41-26112.shtml
+
+* Short article about using GNU Parallel on RHEL. Thanks to Rishi Deshpande.
+  http://nuclear-imaging.info/site_content/2011/05/11/gnu-parallel/
+
+* Using GNU Parallel for FLAC->MP3 conversion. Thanks to Derek Marcotte.
+  http://derek.chezmarcotte.ca/?p=286
+
 * Bug fixes and man page updates.
 
 
@@ -220,7 +235,7 @@ If you use xargs and tee today you will find GNU Parallel very easy to
 use as GNU Parallel is written to have the same options as xargs. If
 you write loops in shell, you will find GNU Parallel may be able to
 replace most of the loops and make them run faster by running several
-jobs in parallel.
+jobs in parallel. GNU Parallel can even replace nested loops.
 
 GNU Parallel makes sure output from the commands is the same output as
 you would get had you run the commands sequentially. This makes it
@@ -236,4 +251,22 @@ When using GNU Parallel for a publication please cite:
 
 O. Tange (2011): GNU Parallel - The Command-Line Power Tool, ;login:
 The USENIX Magazine, February 2011:42-47.
+
+
+= About GNU SQL =
+
+GNU sql aims to give a simple, unified interface for accessing
+databases through all the different databases' command line
+clients. So far the focus has been on giving a common way to specify
+login information (protocol, username, password, hostname, and port
+number), size (database and table size), and running queries.
+
+The database is addressed using a DBURL. If commands are left out you
+will get that database's interactive shell.
+
+When using GNU SQL for a publication please cite:
+
+O. Tange (2011): GNU SQL - A Command Line Tool for Accessing Different
+Databases Using DBURLs, ;login: The USENIX Magazine, April
+2011.
 >>>>>

src/parallel.pod

@@ -100,7 +100,7 @@ B<{/}> can be used the same places as B<{}>. The replacement string
 B<{/}> can be changed with B<--basenamereplace>.
 
 
-=item B<{//}>
+=item B<{//}> (alpha testing)
 
 Dirname of input line. This is a specialized replacement string
 containing the dir of the input. See B<dirname>(1).
@@ -128,7 +128,7 @@ The replacement string B<{#}> can be changed with B<--seqreplace>.
 
 =item B<{>I<n>B<}>
 
-Argument from argument file I<n> or the I<n>'th argument. See B<-a>
+Argument from input source I<n> or the I<n>'th argument. See B<-a>
 and B<-N>.
 
 B<{>I<n>B<}> can be used the same places as B<{}>.
@@ -136,7 +136,7 @@ B<{>I<n>B<}> can be used the same places as B<{}>.
 
 =item B<{>I<n>.B<}>
 
-Argument from argument file I<n> or the I<n>'th argument without
+Argument from input source I<n> or the I<n>'th argument without
 extension. It is a combination of B<{>I<n>B<}> and B<{.}>.
 
 B<{>I<n>.B<}> can be used the same places as B<{>I<n>B<}>.
@@ -144,7 +144,7 @@ B<{>I<n>.B<}> can be used the same places as B<{>I<n>B<}>.
 
 =item B<{>I<n>/B<}>
 
-Basename of argument from argument file I<n> or the I<n>'th argument.
+Basename of argument from input source I<n> or the I<n>'th argument.
 It is a combination of B<{>I<n>B<}> and B<{/}>.  See B<-a> and B<-N>.
 
 B<{>I<n>/B<}> can be used the same places as B<{>I<n>B<}>.
@@ -152,7 +152,7 @@ B<{>I<n>/B<}> can be used the same places as B<{>I<n>B<}>.
 
 =item B<{>I<n>/.B<}>
 
-Basename of argument from argument file I<n> or the I<n>'th argument
+Basename of argument from input source I<n> or the I<n>'th argument
 without extension.  It is a combination of B<{>I<n>B<}>, B<{/}>, and
 B<{.}>.  See B<-a> and B<-N>.
 
@@ -161,9 +161,9 @@ B<{>I<n>/.B<}> can be used the same places as B<{>I<n>B<}>.
 
 =item B<:::> I<arguments> (alpha testing)
 
-Use arguments from the command line as input instead of from stdin
+Use arguments from the command line as input source instead of from
-(standard input). Unlike other options for GNU B<parallel> B<:::> is
+stdin (standard input). Unlike other options for GNU B<parallel>
-placed after the I<command> and before the arguments.
+B<:::> is placed after the I<command> and before the arguments.
 
 The following are equivalent:
 
@@ -179,10 +179,11 @@ argument separator to something else. See also B<--arg-sep>.
 
 stdin (standard input) will be passed to the first process run.
 
-If multiple B<:::> are given, all combinations of group of arguments
+If multiple B<:::> are given, each group will be treated as an input
-will be generated. E.g. ::: 1 2 ::: a b c will result in the
+source, and all combinations of input sources will be
-combinations (1,a) (1,b) (1,c) (2,a) (2,b) (2,c). This is useful for
+generated. E.g. ::: 1 2 ::: a b c will result in the combinations
-replacing nested for-loops.
+(1,a) (1,b) (1,c) (2,a) (2,b) (2,c). This is useful for replacing
+nested for-loops.
 
 B<:::> and B<::::> can be mixed. So these are equivalent:
 
@@ -198,6 +199,8 @@ B<:::> and B<::::> can be mixed. So these are equivalent:
 
 Another way to write B<-a> I<argfile1> B<-a> I<argfile2> ...
 
+B<:::> and B<::::> can be mixed.
+
 See B<-a>.
 
 
@@ -214,21 +217,20 @@ for processing arguments that may contain \n (newline).
 
 =item B<-a> I<input-file>
 
-Read items from the file I<input-file> instead of stdin (standard input).  If
+Use I<input-file> as input source. If you use this option, stdin is
-you use this option, stdin is given to the first process run.
+given to the first process run.  Otherwise, stdin is redirected from
-Otherwise, stdin is redirected from /dev/null.
+/dev/null.
 
-If multiple B<-a> are given, all combinations of all lines will be
+If multiple B<-a> are given, each I<input-file> will be treated as an
-generated. E.g. 1 2 and a b c will result in the combinations (1,a)
+input source, and all combinations of input sources will be
-(1,b) (1,c) (2,a) (2,b) (2,c). This is useful for replacing nested
+generated. E.g. The file B<foo> contains B<1 2>, the file B<bar>
-for-loops.
+contains B<a b c>.  B<-a foo> B<-a bar> will result in the combinations
-
+(1,a) (1,b) (1,c) (2,a) (2,b) (2,c). This is useful for replacing
-The arguments can be accessed in the command as B<{1}>
+nested for-loops.
-.. B<{>I<n>B<}>, so B<{1}> will be a line from the first file, and
-B<{6}> will refer to a line from the 6th file.
 
 See also B<--xapply>
 
+
 =item B<--arg-file-sep> I<sep-str>
 
 Use I<sep-str> instead of B<::::> as separator string between command

testsuite/tests-to-run/test58.sh

@@ -18,3 +18,15 @@ seq 6 7 | parallel -k -a - -a <(seq 4 5) echo {1} {2} {3} ::: 1 2 3
 
 echo '### Test :::: < - :::'
 seq 4 5 | parallel -k echo {1} {2} {3} :::: <(seq 6 7) - ::: 1 2 3
+
+echo '### Test -E'
+seq 1 100 | parallel -k -E 5 echo :::: - ::: 2 3 4 5 6 7 8 9 10 :::: <(seq 3 11)
+
+echo '### Test -E one empty'
+seq 1 100 | parallel -k -E 3 echo :::: - ::: 2 3 4 5 6 7 8 9 10 :::: <(seq 3 11)
+
+echo '### Test -E 2 empty'
+seq 1 100 | parallel -k -E 3 echo :::: - ::: 3 4 5 6 7 8 9 10 :::: <(seq 3 11)
+
+echo '### Test -E all empty'
+seq 3 100 | parallel -k -E 3 echo :::: - ::: 3 4 5 6 7 8 9 10 :::: <(seq 3 11)