tange/parallel
1
0
Fork 0
mirror of https://git.savannah.gnu.org/git/parallel.git synced 2024-10-18 04:49:40 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

parallel: Fixed bug #50590: Support systems where setpgrp fails.

Browse source
This commit is contained in:
Ole Tange 2017-04-01 02:59:07 +02:00
parent e135fb6fd3
commit 1de0d2be9b
24 changed files with 670 additions and 993 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

36
doc/release_new_version
View file

@ -127,7 +127,7 @@ git diff
YYYYMMDD=`yyyymmdd`
TAG=MyTag
echo "Released as $YYYYMMDD ('$TAG')."
echo "Released as $YYYYMMDD ('$TAG')." | grep MyTag && (STOP;STOP;STOP)
git commit -a -m "Released as $YYYYMMDD ('$TAG')"
git tag -s -u 88888888 -m "Released as $YYYYMMDD ('$TAG')" $TAG
@ -193,9 +193,9 @@ file:///home/tange/privat/parallel/doc/release_new_version
from:tange@gnu.org
to:parallel@gnu.org, bug-parallel@gnu.org
Subject: GNU Parallel 20170322 ('TRAPPIST-1') released <<[stable]>>
Subject: GNU Parallel 20170422 ('') released <<[stable]>>
GNU Parallel 20170322 ('TRAPPIST-1') <<[stable]>> has been released. It is available for download at: http://ftpmirror.gnu.org/parallel/
GNU Parallel 20170422 ('') <<[stable]>> has been released. It is available for download at: http://ftpmirror.gnu.org/parallel/
<<No new functionality was introduced so this is a good candidate for a stable release.>>
@ -206,40 +206,18 @@ Haiku of the month:
New in this release:
* --rpl can now take arguments by adding '(regexp)' in the replacement string.
* Vote for GNU Parallel's community ads on
http://meta.unix.stackexchange.com/a/4356/2972
http://meta.askubuntu.com/a/16750/22307
http://meta.serverfault.com/a/9040/45704
* GNU Parallel was cited in: shmlast: An improved implementation of Conditional Reciprocal Best Hits with LAST and Python https://joss.theoj.org/papers/3cde54de7dfbcada7c0fc04f569b36c7
http://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-017-1593-0
* GNU Parallel was cited in: Prediction of relativistic electron flux in the Earth’s outer radiation belt at geostationary orbit by adaptive methods https://link.springer.com/article/10.1134/S0016793217010108
* GNU Parallel was cited in: Proper experimental design requires randomization/balancing of molecular ecology experiments http://biorxiv.org/content/biorxiv/early/2017/02/17/109280.full.pdf
* GNU Parallel was cited in: Higher likelihood of multiple bit-flips due to neutron-induced strikes on logic gates https://arxiv.org/pdf/1612.08239.pdf
* GNU Parallel was cited in: A Monte Carlo Resampling Approach for the Calculation of Hybrid Classical and Quantum Free Energies http://pubs.acs.org/doi/pdfplus/10.1021/acs.jctc.6b00506
* GNU Parallel was cited in: Learning string distance with smoothing for OCR spelling correction https://link.springer.com/article/10.1007/s11042-016-4185-5
* GNU Parallel was cited in: The Breakdown of String Perturbation Theory for Many External Particles https://arxiv.org/pdf/1611.08003.pdf
* GNU Parallel was cited in: Large-scale benchmarking reveals false discoveries and count transformation sensitivity in 16S rRNA gene amplicon data analysis methods used in microbiome studies https://microbiomejournal.biomedcentral.com/articles/10.1186/s40168-016-0208-8
* GNU Parallel was cited in: Operations Research Applications in Nuclear Energy https://dspace.library.colostate.edu/bitstream/handle/11124/170687/Johnson_mines_0052E_11207.pdf?sequence=1&isAllowed=y
* GNU Parallel was cited in: Parallel Computing: GNU Parallel http://www.blopig.com/blog/2017/02/parallel-computing-gnu-parallel/
* Stig Sandbeck Mathisen: Change All The Passwords (Again) http://garf.us/2017/02/stig-sandbeck-mathisen-change-all-the-passwords-again/
* Trabalhando como Paralelismo no Bash usando o GNU Parallel http://www.beyeler.com.br/2017/03/trabalhando-como-paralelismo-no-bash-usando-o-gnu-parallel/
* GNU parallel と SQL*Loader 組み合わせ実行例https://blogs.oracle.com/LetTheSunShineIn/entry/gnu_parallel_%E3%81%A8_sql_loader
https://128.84.21.199/pdf/1703.09026.pdf
https://github.com/lucascbeyeler/zmbackup
http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0174575#references
* <<Possibly http://link.springer.com/chapter/10.1007%2F978-3-319-22053-6_46>>

17
packager/obs/home:tange/parallel/.osc/_files
View file

@ -1,13 +1,6 @@
<directory name="parallel" rev="199" srcmd5="d215e332e8e0d5d9886c7045abed4819" vrev="1">
<entry md5="e6710d334a82f776e34c07bc276abdf0" mtime="1483271525" name="parallel-20161222.tar.bz2" size="1449836" />
<entry md5="f1d1774a1904db762c292ffa4b542658" mtime="1485019214" name="parallel-20170122.tar.bz2" size="1459419" />
<entry md5="98cb0dfdc6fc764110a71ec591b24bb1" mtime="1487719474" name="parallel-20170222.tar.bz2" size="1479077" />
<entry md5="62b6c8aa976dfd7dcc275fb8ad3f44c4" mtime="1487719474" name="parallel.spec" size="4313" />
<entry md5="3ab86fea7800fb5f7cb87269a8df7aa8" mtime="1483271526" name="parallel_20161122.dsc" size="556" />
<entry md5="5cfd9e7aac93d0d254293b4e6c284873" mtime="1483271527" name="parallel_20161122.tar.gz" size="1576406" />
<entry md5="aca39e894b20e489d439ddaca9ad3281" mtime="1482416160" name="parallel_20161222.tar.gz" size="1603168" />
<entry md5="ec73cfb6f7e961f209e94a30e3e4a57e" mtime="1485019214" name="parallel_20170122.dsc" size="556" />
<entry md5="8790199a8f2e5d07e86ae9cf00c08a4c" mtime="1485019215" name="parallel_20170122.tar.gz" size="1635064" />
<entry md5="58be920398402a5503f92d56b0343662" mtime="1487719475" name="parallel_20170222.dsc" size="556" />
<entry md5="aaef171df3933dbeb3d15fd0dfb90b46" mtime="1487719475" name="parallel_20170222.tar.gz" size="1661902" />
<directory name="parallel" rev="200" srcmd5="ef8094ec2de88060bf1e32d83bfc1a1f" vrev="1">
<entry md5="0910ea89703cd4fee443f5a2403325bb" mtime="1490135663" name="parallel-20170322.tar.bz2" size="1490362" />
<entry md5="c80f0634d4841dd85133b077b421146e" mtime="1490135663" name="parallel.spec" size="4314" />
<entry md5="e9080ce05862bd69bc2d9a3fdfc0766b" mtime="1490135664" name="parallel_20170322.dsc" size="556" />
<entry md5="af85898a49a8311de8bf11718d4ab4b2" mtime="1490135664" name="parallel_20170322.tar.gz" size="1670238" />
</directory>

3
packager/obs/home:tange/parallel/.osc/parallel.spec
View file

@ -1,6 +1,7 @@
Summary: Shell tool for executing jobs in parallel
Name: parallel
Version: 20170222
Version: 20170322
Release: 1.1
License: GPL
Group: Productivity/File utilities

3
packager/obs/home:tange/parallel/parallel.spec
View file

@ -1,6 +1,7 @@
Summary: Shell tool for executing jobs in parallel
Name: parallel
Version: 20170222
Version: 20170322
Release: 1.1
License: GPL
Group: Productivity/File utilities

2
src/niceload
View file

@ -24,7 +24,7 @@
use strict;
use Getopt::Long;
$Global::progname="niceload";
$Global::version = 20170322;
$Global::version = 20170323;
Getopt::Long::Configure("bundling","require_order");
get_options_from_array(\@ARGV) || die_usage();
if($opt::version) {

9
src/parallel
View file

@ -399,6 +399,7 @@ sub cat_partial {
my @start_len = map {
if(++$i % 2) { $start = $_; } else { $_-$start }
} @start_end;
# This can read 7 GB/s using a single core
my $script = spacefree
(0,
q{
@ -1361,7 +1362,7 @@ sub check_invalid_option_combinations {
sub init_globals {
# Defaults:
$Global::version = 20170322;
$Global::version = 20170331;
$Global::progname = 'parallel';
$Global::infinity = 2**31;
$Global::debug = 0;
@ -7454,7 +7455,7 @@ sub sshlogin_wrap {
$pid = fork;
unless($pid) {
# Make own process group to be able to kill HUP it later
setpgrp;
eval { setpgrp };
eval { setpriority(0,0,$nice) };
exec $shell, "-c", ($bashfunc."@ARGV");
die "exec: $!\n";
@ -7899,7 +7900,7 @@ sub start {
eval {
if(not $pid = ::open3($stdin_fh, ">&OUT", ">&ERR", "-")) {
# Each child gets its own process group to make it safe to killall
setpgrp(0,0) || ::die_bug("setpgrp failed");
eval{setpgrp(0,0)};
exec($Global::shell,"-c",$command)
|| ::die_bug("open3-$stdin_fh $command");
}
@ -7925,7 +7926,7 @@ sub start {
my $pid;
my @setpgrp_wrap =
('perl','-e',
"setpgrp\;eval\{setpriority\(0,0,$opt::nice\)\}\;".
"eval\{setpgrp\}\;eval\{setpriority\(0,0,$opt::nice\)\}\;".
"exec '$Global::shell', '-c', \@ARGV");
# The eval is needed to catch exception from open3
eval {

8
src/parallel.pod
View file

@ -1245,7 +1245,7 @@ control on the command line (used by GNU B<parallel> internally when
called with B<--sshlogin>).
=item B<--plus>
=item B<--plus> (alpha testing)
Activate additional replacement strings: {+/} {+.} {+..} {+...} {..}
{...} {/..} {/...} {##}. The idea being that '{+foo}' matches the opposite of
@ -1481,9 +1481,9 @@ it to the command.
Only used with B<--pipe>.
=item B<--results> I<name> (beta testing)
=item B<--results> I<name>
=item B<--res> I<name> (beta testing)
=item B<--res> I<name>
Save the output into files.
@ -1758,7 +1758,7 @@ impossible to track which input block corresponds to which output.
B<--round-robin> implies B<--pipe>, except if B<--pipepart> is given.
=item B<--rpl> 'I<tag> I<perl expression>'
=item B<--rpl> 'I<tag> I<perl expression>' (alpha testing)
Use I<tag> as a replacement string for I<perl expression>. This makes
it possible to define your own replacement strings. GNU B<parallel>'s

5
src/parallel_alternatives.pod
View file

@ -863,6 +863,11 @@ Rush (https://github.com/shenwei356/rush) is written in Go and based
on gargs.
=head2 DIFFERENCES BETWEEN machma AND GNU Parallel
Todo. Requires Go > 1.6.
=head2 DIFFERENCES BETWEEN ClusterSSH AND GNU Parallel
ClusterSSH solves a different problem than GNU B<parallel>.

397
src/parallel_tutorial.pod
View file

@ -20,7 +20,8 @@ To run this tutorial you must have the following:
Install the newest version using your package manager (recommended for
security reasons), the way described in README, or with this command:
(wget -O - pi.dk/3 || curl pi.dk/3/ || fetch -o - http://pi.dk/3) | bash
(wget -O - pi.dk/3 || curl pi.dk/3/ || \
fetch -o - http://pi.dk/3) | bash
This will also install the newest version of the tutorial which you
can see by running this:
@ -88,7 +89,14 @@ The file can be generated by this command:
The file can be generated by this command:
(echo %head1; echo %head2; perl -e 'for(1..10){print "$_\n"}') > num_%header
(echo %head1; echo %head2; \
perl -e 'for(1..10){print "$_\n"}') > num_%header
=item fixedlen
The file can be generated by this command:
perl -e 'print "HHHHAAABBBCCC"' > fixedlen
=item For remote running: ssh login on 2 servers with no password in
$SERVER1 and $SERVER2 must work.
@ -544,7 +552,8 @@ Output:
B<@arg> contains the input source variables:
parallel echo {= 'if($arg[1]==$arg[2]) { skip() }' =} ::: {1..3} ::: {1..3}
parallel echo {= 'if($arg[1]==$arg[2]) { skip() }' =} \
::: {1..3} ::: {1..3}
Output:
@ -557,7 +566,8 @@ Output:
If the strings B<{=> and B<=}> cause problems they can be replaced with B<--parens>:
parallel --parens ,,,, echo ',, s:\.[^.]+$::;s:\.[^.]+$::; ,,' ::: foo.tar.gz
parallel --parens ,,,, echo ',, s:\.[^.]+$::;s:\.[^.]+$::; ,,' \
::: foo.tar.gz
Output:
@ -565,14 +575,16 @@ Output:
To define a shorthand replacement string use B<--rpl>:
parallel --rpl '.. s:\.[^.]+$::;s:\.[^.]+$::;' echo '..' ::: foo.tar.gz
parallel --rpl '.. s:\.[^.]+$::;s:\.[^.]+$::;' echo '..' \
::: foo.tar.gz
Output: Same as above.
If the shorthand starts with B<{> it can be used as a positional
replacement string, too:
parallel --rpl '{..} s:\.[^.]+$::;s:\.[^.]+$::;' echo '{..}' ::: foo.tar.gz
parallel --rpl '{..} s:\.[^.]+$::;s:\.[^.]+$::;' echo '{..}'
::: foo.tar.gz
Output: Same as above.
@ -595,7 +607,8 @@ Here we give the two arguments B<tar.gz> and B<zip> to the replacement
string B<{/I<string1>/I<string2>}> which replaces I<string1> with
I<string2>:
parallel --rpl '{/(.+?)/(.*?)} s/$$1/$$2/;' echo {/tar.gz/zip} ::: foo.tar.gz
parallel --rpl '{/(.+?)/(.*?)} s/$$1/$$2/;' echo {/tar.gz/zip} \
::: foo.tar.gz
Output:
@ -608,14 +621,15 @@ GNU B<parallel>'s 7 replacement strings are implemented as this:
--rpl '{#} $_=$job->seq()'
--rpl '{%} $_=$job->slot()'
--rpl '{/} s:.*/::'
--rpl '{//} $Global::use{"File::Basename"} ||= eval "use File::Basename; 1;"; $_ = dirname($_);'
--rpl '{//} $Global::use{"File::Basename"} ||=
eval "use File::Basename; 1;"; $_ = dirname($_);'
--rpl '{/.} s:.*/::; s:\.[^/.]+$::;'
--rpl '{.} s:\.[^/.]+$::'
=head3 Positional replacement strings
With multiple input sources the argument from the individual input
sources can be accessed with B<{>numberB<}>:
sources can be accessed with S<< B<{>numberB<}> >>:
parallel echo {1} and {2} ::: A B ::: C D
@ -638,7 +652,8 @@ Output (the order may be different):
If a position is negative, it will refer to the input source counted
from behind:
parallel echo 1={1} 2={2} 3={3} -1={-1} -2={-2} -3={-3} ::: A B ::: C D ::: E F
parallel echo 1={1} 2={2} 3={3} -1={-1} -2={-2} -3={-3} \
::: A B ::: C D ::: E F
Output (the order may be different):
@ -657,7 +672,8 @@ Output (the order may be different):
To use a perl expression as a positional replacement string simply
prepend the perl expression with number and space:
parallel echo '{=2 s:\.[^.]+$::;s:\.[^.]+$::; =} {1}' ::: bar ::: foo.tar.gz
parallel echo '{=2 s:\.[^.]+$::;s:\.[^.]+$::; =} {1}' \
::: bar ::: foo.tar.gz
Output:
@ -666,7 +682,8 @@ Output:
If a shorthand defined using B<--rpl> starts with B<{> it can be used as
a positional replacement string, too:
parallel --rpl '{..} s:\.[^.]+$::;s:\.[^.]+$::;' echo '{2..} {1}' ::: bar ::: foo.tar.gz
parallel --rpl '{..} s:\.[^.]+$::;s:\.[^.]+$::;' echo '{2..} {1}' \
::: bar ::: foo.tar.gz
Output: Same as above.
@ -701,32 +718,33 @@ Output (the order may be different):
It is useful with B<--colsep> for processing files with TAB separated values:
parallel --header : --colsep '\t' echo f1={f1} f2={f2} :::: tsv-file.tsv
parallel --header : --colsep '\t' echo f1={f1} f2={f2} \
:::: tsv-file.tsv
Output (the order may be different):
f1=A f2=B
f1=C f2=D
=head3 More pre-defined replacement strings
=head3 More pre-defined replacement strings with --plus
B<--plus> adds the replacement strings B<{+/} {+.} {+..} {+...} {..} {...}
{/..} {/...} {##}>. The idea being that B<{+foo}> matches the opposite of B<{foo}>
and B<{}> = B<{+/}>/B<{/}> = B<{.}>.B<{+.}> = B<{+/}>/B<{/.}>.B<{+.}> = B<{..}>.B<{+..}> =
B<{+/}>/B<{/..}>.B<{+..}> = B<{...}>.B<{+...}> = B<{+/}>/B<{/...}>.B<{+...}>.
parallel --plus echo {} ::: dir/sub/file.ext1.ext2.ext3
parallel --plus echo {+/}/{/} ::: dir/sub/file.ext1.ext2.ext3
parallel --plus echo {.}.{+.} ::: dir/sub/file.ext1.ext2.ext3
parallel --plus echo {+/}/{/.}.{+.} ::: dir/sub/file.ext1.ext2.ext3
parallel --plus echo {..}.{+..} ::: dir/sub/file.ext1.ext2.ext3
parallel --plus echo {+/}/{/..}.{+..} ::: dir/sub/file.ext1.ext2.ext3
parallel --plus echo {...}.{+...} ::: dir/sub/file.ext1.ext2.ext3
parallel --plus echo {+/}/{/...}.{+...} ::: dir/sub/file.ext1.ext2.ext3
parallel --plus echo {} ::: dir/sub/file.ex1.ex2.ex3
parallel --plus echo {+/}/{/} ::: dir/sub/file.ex1.ex2.ex3
parallel --plus echo {.}.{+.} ::: dir/sub/file.ex1.ex2.ex3
parallel --plus echo {+/}/{/.}.{+.} ::: dir/sub/file.ex1.ex2.ex3
parallel --plus echo {..}.{+..} ::: dir/sub/file.ex1.ex2.ex3
parallel --plus echo {+/}/{/..}.{+..} ::: dir/sub/file.ex1.ex2.ex3
parallel --plus echo {...}.{+...} ::: dir/sub/file.ex1.ex2.ex3
parallel --plus echo {+/}/{/...}.{+...} ::: dir/sub/file.ex1.ex2.ex3
Output:
dir/sub/file.ext1.ext2.ext3
dir/sub/file.ex1.ex2.ex3
B<{##}> is simply the number of jobs:
@ -740,12 +758,59 @@ Output:
Job 4 of 5
Job 5 of 5
=head4 Dynamic replacement strings
=head3 Dynamic replacement strings with --plus
B<--plus> also defines B<{:-I<string>}>, B<{:I<number>}>,
B<{:I<number1>:I<number2>}>, B<{#I<string>}>, B<{%I<string>}>,
B<{/I<string1>/I<string2>}>, B<{^I<string>}>, B<{^^I<string>}>,
B<{,I<string>}>, and B<{,,I<string>}>. They are inspired from B<Bash>:
B<--plus> also defines these dynamic replacement strings:
=over 19
=item B<{:-I<string>}>
Default value is I<string> if the argument is empty.
=item B<{:I<number>}>
Substring from I<number> till end of string.
=item B<{:I<number1>:I<number2>}>
Substring from I<number1> to I<number2>.
=item B<{#I<string>}>
If the argument starts with I<string>, remove it.
=item B<{%I<string>}>
If the argument ends with I<string>, remove it.
=item B<{/I<string1>/I<string2>}>
Replace I<string1> with I<string2>.
=item B<{^I<string>}>
If the argument starts with I<string>, upper case it. I<string> must
be a single letter.
=item B<{^^I<string>}>
If the argument contains I<string>, upper case it. I<string> must be a
single letter.
=item B<{,I<string>}>
If the argument starts with I<string>, lower case it. I<string> must
be a single letter.
=item B<{,,I<string>}>
If the argument contains I<string>, lower case it. I<string> must be a
single letter.
=back
They are inspired from B<Bash>:
unset myvar
echo ${myvar:-myval}
@ -956,11 +1021,11 @@ Output (the order may be different):
GNU B<parallel> can also \-quote full lines. Simply run this:
parallel --shellquote
parallel: Warning: Input is read from the terminal. You either know what you
parallel: Warning: are doing (in which case: YOU ARE AWESOME!) or you forgot
parallel: Warning: ::: or :::: or to pipe data into parallel. If so
parallel: Warning: consider going through the tutorial: man parallel_tutorial
parallel: Warning: Press CTRL-D to exit.
Warning: Input is read from the terminal. You either know what you
Warning: are doing (in which case: YOU ARE AWESOME!) or you forgot
Warning: ::: or :::: or to pipe data into parallel. If so
Warning: consider going through the tutorial: man parallel_tutorial
Warning: Press CTRL-D to exit.
perl -e 'print "@ARGV\n"'
[CTRL-D]
@ -1078,7 +1143,8 @@ Output (the order may be different):
GNU B<parallel> will postpone the output until the command completes:
parallel -j2 'printf "%s-start\n%s" {} {};sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1
parallel -j2 'printf "%s-start\n%s" {} {};
sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1
Output:
@ -1094,7 +1160,8 @@ Output:
To get the output immediately use B<--ungroup>:
parallel -j2 --ungroup 'printf "%s-start\n%s" {} {};sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1
parallel -j2 --ungroup 'printf "%s-start\n%s" {} {};
sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1
Output:
@ -1109,12 +1176,13 @@ Output:
4-end
B<--ungroup> is fast, but can cause half a line from one job to be mixed
with half a line of another job. That has happend in the second line,
with half a line of another job. That has happened in the second line,
where the line '4-middle' is mixed with '2-start'.
To avoid this use B<--linebuffer>:
parallel -j2 --linebuffer 'printf "%s-start\n%s" {} {};sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1
parallel -j2 --linebuffer 'printf "%s-start\n%s" {} {};
sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1
Output:
@ -1130,7 +1198,8 @@ Output:
To force the output in the same order as the arguments use B<--keep-order>/B<-k>:
parallel -j2 -k 'printf "%s-start\n%s" {} {};sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1
parallel -j2 -k 'printf "%s-start\n%s" {} {};
sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1
Output:
@ -1364,7 +1433,8 @@ Output:
Computers / CPU cores / Max jobs to run
1:local / 2 / 2
Computer:jobs running/jobs completed/%of started jobs/Average seconds to complete
Computer:jobs running/jobs completed/%of started jobs/
Average seconds to complete
ETA: 2s 0left 1.11avg local:0/9/100%/1.1s
GNU B<parallel> can give progress information with B<--progress>:
@ -1376,7 +1446,8 @@ Output:
Computers / CPU cores / Max jobs to run
1:local / 2 / 2
Computer:jobs running/jobs completed/%of started jobs/Average seconds to complete
Computer:jobs running/jobs completed/%of started jobs/
Average seconds to complete
local:0/9/100%/1.1s
A progress bar can be shown with B<--bar>:
@ -1385,7 +1456,8 @@ A progress bar can be shown with B<--bar>:
And a graphic bar can be shown with B<--bar> and B<zenity>:
seq 1000 | parallel -j10 --bar '(echo -n {};sleep 0.1)' 2> >(zenity --progress --auto-kill --auto-close)
seq 1000 | parallel -j10 --bar '(echo -n {};sleep 0.1)' \
2> >(zenity --progress --auto-kill --auto-close)
A logfile of the jobs completed so far can be generated with B<--joblog>:
@ -1394,11 +1466,11 @@ A logfile of the jobs completed so far can be generated with B<--joblog>:
Output:
Seq Host Starttime Runtime Send Receive Exitval Signal Command
1 : 1376577364.974 0.008 0 0 1 0 exit 1
2 : 1376577364.982 0.013 0 0 2 0 exit 2
3 : 1376577364.990 0.013 0 0 3 0 exit 3
4 : 1376577365.003 0.003 0 0 0 0 exit 0
Seq Host Starttime Runtime Send Receive Exitval Signal Command
1 : 1376577364.974 0.008 0 0 1 0 exit 1
2 : 1376577364.982 0.013 0 0 2 0 exit 2
3 : 1376577364.990 0.013 0 0 3 0 exit 3
4 : 1376577365.003 0.003 0 0 0 0 exit 0
The log contains the job sequence, which host the job was run on, the
start time and run time, how much data was transferred, the exit
@ -1416,19 +1488,19 @@ unchanged.
Output:
Seq Host Starttime Runtime Send Receive Exitval Signal Command
1 : 1376580069.544 0.008 0 0 1 0 exit 1
2 : 1376580069.552 0.009 0 0 2 0 exit 2
3 : 1376580069.560 0.012 0 0 3 0 exit 3
4 : 1376580069.571 0.005 0 0 0 0 exit 0
Seq Host Starttime Runtime Send Receive Exitval Signal Command
1 : 1376580069.544 0.008 0 0 1 0 exit 1
2 : 1376580069.552 0.009 0 0 2 0 exit 2
3 : 1376580069.560 0.012 0 0 3 0 exit 3
4 : 1376580069.571 0.005 0 0 0 0 exit 0
Seq Host Starttime Runtime Send Receive Exitval Signal Command
1 : 1376580069.544 0.008 0 0 1 0 exit 1
2 : 1376580069.552 0.009 0 0 2 0 exit 2
3 : 1376580069.560 0.012 0 0 3 0 exit 3
4 : 1376580069.571 0.005 0 0 0 0 exit 0
5 : 1376580070.028 0.009 0 0 0 0 exit 0
6 : 1376580070.038 0.007 0 0 0 0 exit 0
Seq Host Starttime Runtime Send Receive Exitval Signal Command
1 : 1376580069.544 0.008 0 0 1 0 exit 1
2 : 1376580069.552 0.009 0 0 2 0 exit 2
3 : 1376580069.560 0.012 0 0 3 0 exit 3
4 : 1376580069.571 0.005 0 0 0 0 exit 0
5 : 1376580070.028 0.009 0 0 0 0 exit 0
6 : 1376580070.038 0.007 0 0 0 0 exit 0
Note how the start time of the last 2 jobs is clearly different from the second run.
@ -1439,16 +1511,16 @@ With B<--resume-failed> GNU B<parallel> will re-run the jobs that failed:
Output:
Seq Host Starttime Runtime Send Receive Exitval Signal Command
1 : 1376580069.544 0.008 0 0 1 0 exit 1
2 : 1376580069.552 0.009 0 0 2 0 exit 2
3 : 1376580069.560 0.012 0 0 3 0 exit 3
4 : 1376580069.571 0.005 0 0 0 0 exit 0
5 : 1376580070.028 0.009 0 0 0 0 exit 0
6 : 1376580070.038 0.007 0 0 0 0 exit 0
1 : 1376580154.433 0.010 0 0 1 0 exit 1
2 : 1376580154.444 0.022 0 0 2 0 exit 2
3 : 1376580154.466 0.005 0 0 3 0 exit 3
Seq Host Starttime Runtime Send Receive Exitval Signal Command
1 : 1376580069.544 0.008 0 0 1 0 exit 1
2 : 1376580069.552 0.009 0 0 2 0 exit 2
3 : 1376580069.560 0.012 0 0 3 0 exit 3
4 : 1376580069.571 0.005 0 0 0 0 exit 0
5 : 1376580070.028 0.009 0 0 0 0 exit 0
6 : 1376580070.038 0.007 0 0 0 0 exit 0
1 : 1376580154.433 0.010 0 0 1 0 exit 1
2 : 1376580154.444 0.022 0 0 2 0 exit 2
3 : 1376580154.466 0.005 0 0 3 0 exit 3
Note how seq 1 2 3 have been repeated because they had exit value
different from 0.
@ -1463,19 +1535,19 @@ command line and reruns the commands mentioned in the joblog.
Output:
Seq Host Starttime Runtime Send Receive Exitval Signal Command
1 : 1376580069.544 0.008 0 0 1 0 exit 1
2 : 1376580069.552 0.009 0 0 2 0 exit 2
3 : 1376580069.560 0.012 0 0 3 0 exit 3
4 : 1376580069.571 0.005 0 0 0 0 exit 0
5 : 1376580070.028 0.009 0 0 0 0 exit 0
6 : 1376580070.038 0.007 0 0 0 0 exit 0
1 : 1376580154.433 0.010 0 0 1 0 exit 1
2 : 1376580154.444 0.022 0 0 2 0 exit 2
3 : 1376580154.466 0.005 0 0 3 0 exit 3
1 : 1376580164.633 0.010 0 0 1 0 exit 1
2 : 1376580164.644 0.022 0 0 2 0 exit 2
3 : 1376580164.666 0.005 0 0 3 0 exit 3
Seq Host Starttime Runtime Send Receive Exitval Signal Command
1 : 1376580069.544 0.008 0 0 1 0 exit 1
2 : 1376580069.552 0.009 0 0 2 0 exit 2
3 : 1376580069.560 0.012 0 0 3 0 exit 3
4 : 1376580069.571 0.005 0 0 0 0 exit 0
5 : 1376580070.028 0.009 0 0 0 0 exit 0
6 : 1376580070.038 0.007 0 0 0 0 exit 0
1 : 1376580154.433 0.010 0 0 1 0 exit 1
2 : 1376580154.444 0.022 0 0 2 0 exit 2
3 : 1376580154.466 0.005 0 0 3 0 exit 3
1 : 1376580164.633 0.010 0 0 1 0 exit 1
2 : 1376580164.644 0.022 0 0 2 0 exit 2
3 : 1376580164.666 0.005 0 0 3 0 exit 3
=head2 Termination
@ -1491,11 +1563,10 @@ Output:
0
0
1
parallel: Starting no more jobs. Waiting for 2 jobs to finish. This job failed:
parallel: This job failed:
echo 1; exit 1
parallel: Starting no more jobs. Waiting for 1 jobs to finish.
2
parallel: Starting no more jobs. Waiting for 1 jobs to finish. This job failed:
echo 2; exit 2
With B<--halt now,fail=1> the running jobs will be killed immediately:
@ -1512,7 +1583,8 @@ Output:
If B<--halt> is given a percentage this percentage of the jobs must fail
before GNU B<parallel> stops spawning more jobs:
parallel -j2 --halt soon,fail=20% echo {}\; exit {} ::: 0 1 2 3 4 5 6 7 8 9
parallel -j2 --halt soon,fail=20% echo {}\; exit {} \
::: 0 1 2 3 4 5 6 7 8 9
Output:
@ -1545,7 +1617,8 @@ Output:
GNU B<parallel> can retry the command with B<--retries>. This is useful if a
command fails for unknown reasons now and then.
parallel -k --retries 3 'echo tried {} >>/tmp/runs; echo completed {}; exit {}' ::: 1 2 0
parallel -k --retries 3 \
'echo tried {} >>/tmp/runs; echo completed {}; exit {}' ::: 1 2 0
cat /tmp/runs
Output:
@ -1573,10 +1646,14 @@ B<SIGTERM>, waiting 50 ms, then a B<SIGKILL>, finally waiting 25 ms
before giving up. It looks like this:
show_signals() {
perl -e 'for(keys %SIG) { $SIG{$_} = eval "sub { print \"Got $_\\n\"; }";} while(1){sleep 1}'
perl -e 'for(keys %SIG) {
$SIG{$_} = eval "sub { print \"Got $_\\n\"; }";
}
while(1){sleep 1}'
}
export -f show_signals
echo | parallel --termseq TERM,200,TERM,100,TERM,50,KILL,25 -u --timeout 1 show_signals
echo | parallel --termseq TERM,200,TERM,100,TERM,50,KILL,25 \
-u --timeout 1 show_signals
Output:
@ -1592,14 +1669,16 @@ Output: Same as above.
You can change this to B<SIGINT>, B<SIGTERM>, B<SIGKILL>:
echo | parallel --termseq INT,200,TERM,100,KILL,25 -u --timeout 1 show_signals
echo | parallel --termseq INT,200,TERM,100,KILL,25 \
-u --timeout 1 show_signals
Output:
Got INT
Got TERM
The B<SIGKILL> does not show because it cannot be caught, and thus the child dies.
The B<SIGKILL> does not show because it cannot be caught, and thus the
child dies.
=head2 Limiting the resources
@ -1738,8 +1817,8 @@ Servers can be put into groups by prepending I<@groupname> to the
server and the group can then be selected by appending I<@groupname> to
the argument if using B<--hostgroup>:
parallel --hostgroup -S @grp1/$SERVER1 -S @grp2/$SERVER2 echo {} ::: \
run_on_grp1@grp1 run_on_grp2@grp2
parallel --hostgroup -S @grp1/$SERVER1 -S @grp2/$SERVER2 echo {} \
::: run_on_grp1@grp1 run_on_grp2@grp2
Output:
@ -1750,8 +1829,8 @@ A host can be in multiple groups by separating the groups with B<+>, and
you can force GNU B<parallel> to limit the groups on which the command
can be run with B<-S> I<@groupname>:
parallel -S @grp1 -S @grp1+grp2/$SERVER1 -S @grp2/SERVER2 echo {} ::: \
run_on_grp1 also_grp1
parallel -S @grp1 -S @grp1+grp2/$SERVER1 -S @grp2/SERVER2 echo {} \
::: run_on_grp1 also_grp1
Output:
@ -1774,7 +1853,8 @@ If the files are processed into another file, the resulting file can be
transferred back:
echo This is input_file > input_file
parallel -S $SERVER1 --transferfile {} --return {}.out cat {} ">"{}.out ::: input_file
parallel -S $SERVER1 --transferfile {} --return {}.out \
cat {} ">"{}.out ::: input_file
cat input_file.out
Output: Same as above.
@ -1782,7 +1862,8 @@ Output: Same as above.
To remove the input and output file on the remote server use B<--cleanup>:
echo This is input_file > input_file
parallel -S $SERVER1 --transferfile {} --return {}.out --cleanup cat {} ">"{}.out ::: input_file
parallel -S $SERVER1 --transferfile {} --return {}.out --cleanup \
cat {} ">"{}.out ::: input_file
cat input_file.out
Output: Same as above.
@ -1800,7 +1881,8 @@ transfer that using B<--basefile> which will transfer the file before the
first job:
echo common data > common_file
parallel --basefile common_file -S $SERVER1 cat common_file\; echo {} ::: foo
parallel --basefile common_file -S $SERVER1 \
cat common_file\; echo {} ::: foo
Output:
@ -2024,7 +2106,8 @@ that it is only useful for debugging:
echo in my_func $1 > $1.out
}
export -f my_func3
parallel -vv --workdir ... --nice 17 --env _ --trc {}.out -S $SERVER1 my_func3 {} ::: abc-file
parallel -vv --workdir ... --nice 17 --env _ --trc {}.out \
-S $SERVER1 my_func3 {} ::: abc-file
Output will be similar to:
@ -2074,14 +2157,16 @@ the output each in their own column.
The simplest is to use a CSV file as the storage table:
parallel --sqlandworker csv:////%2Ftmp%2Flog.csv seq ::: 10 ::: 12 13 14
parallel --sqlandworker csv:////%2Ftmp%2Flog.csv \
seq ::: 10 ::: 12 13 14
cat /tmp/log.csv
Note how '/' in the path must be written as %2F.
Output will be similar to:
Seq,Host,Starttime,JobRuntime,Send,Receive,Exitval,_Signal,Command,V1,V2,Stdout,Stderr
Seq,Host,Starttime,JobRuntime,Send,Receive,Exitval,_Signal,
Command,V1,V2,Stdout,Stderr
1,:,1458254498.254,0.069,0,9,0,0,"seq 10 12",10,12,"10
11
12
@ -2103,15 +2188,20 @@ format correctly - even with fields containing newlines as above.
If the output is big you may want to put it into files using B<--results>:
parallel --results outdir --sqlandworker csv:////%2Ftmp%2Flog2.csv seq ::: 10 ::: 12 13 14
parallel --results outdir --sqlandworker csv:////%2Ftmp%2Flog2.csv \
seq ::: 10 ::: 12 13 14
cat /tmp/log2.csv
Output will be similar to:
Seq,Host,Starttime,JobRuntime,Send,Receive,Exitval,_Signal,Command,V1,V2,Stdout,Stderr
1,:,1458824738.287,0.029,0,9,0,0,"seq 10 12",10,12,outdir/1/10/2/12/stdout,outdir/1/10/2/12/stderr
2,:,1458824738.298,0.025,0,12,0,0,"seq 10 13",10,13,outdir/1/10/2/13/stdout,outdir/1/10/2/13/stderr
3,:,1458824738.309,0.026,0,15,0,0,"seq 10 14",10,14,outdir/1/10/2/14/stdout,outdir/1/10/2/14/stderr
Seq,Host,Starttime,JobRuntime,Send,Receive,Exitval,_Signal,
Command,V1,V2,Stdout,Stderr
1,:,1458824738.287,0.029,0,9,0,0,
"seq 10 12",10,12,outdir/1/10/2/12/stdout,outdir/1/10/2/12/stderr
2,:,1458824738.298,0.025,0,12,0,0,
"seq 10 13",10,13,outdir/1/10/2/13/stdout,outdir/1/10/2/13/stderr
3,:,1458824738.309,0.026,0,15,0,0,
"seq 10 14",10,14,outdir/1/10/2/14/stdout,outdir/1/10/2/14/stderr
=head2 DBURL as table
@ -2144,7 +2234,8 @@ To see the result:
Output will be similar to:
Seq|Host|Starttime|JobRuntime|Send|Receive|Exitval|_Signal|Command|V1|V2|Stdout|Stderr
Seq|Host|Starttime|JobRuntime|Send|Receive|Exitval|_Signal|
Command|V1|V2|Stdout|Stderr
1|:|1451619638.903|0.806||8|0|0|echo foo baz|foo|baz|foo baz
|
2|:|1451619639.265|1.54||9|0|0|echo foo quuz|foo|quuz|foo quuz
@ -2287,6 +2378,30 @@ instead of passing full lines to B<wc> it passes full 75 lines at a
time. This of course does not hold for the last job (which in this
case got 25 lines).
=head2 Fixed length records
Fixed length records can be processed by setting B<--recend ''> and
B<--block I<recordsize>>. A header of size I<n> can be processed with
B<--header .{I<n>}>.
Here is how to process a file with a 4-byte header and a 3-byte record
size:
cat fixedlen | parallel --pipe --header .{4} --block 3 --recend '' \
'echo start; cat; echo'
Output:
start
HHHHAAA
start
HHHHCCC
start
HHHHBBB
It may be more efficient to increase B<--block> to a multiplum of the
record size.
=head2 Record separators
GNU B<parallel> uses separators to determine where two records split.
@ -2300,7 +2415,8 @@ string.
Here the B<--recend> is set to B<', '>:
echo /foo, bar/, /baz, qux/, | parallel -kN1 --recend ', ' --pipe echo JOB{#}\;cat\;echo END
echo /foo, bar/, /baz, qux/, | \
parallel -kN1 --recend ', ' --pipe echo JOB{#}\;cat\;echo END
Output:
@ -2316,7 +2432,8 @@ Output:
Here the B<--recstart> is set to B</>:
echo /foo, bar/, /baz, qux/, | parallel -kN1 --recstart / --pipe echo JOB{#}\;cat\;echo END
echo /foo, bar/, /baz, qux/, | \
parallel -kN1 --recstart / --pipe echo JOB{#}\;cat\;echo END
Output:
@ -2332,7 +2449,9 @@ Output:
Here both B<--recend> and B<--recstart> are set:
echo /foo, bar/, /baz, qux/, | parallel -kN1 --recend ', ' --recstart / --pipe echo JOB{#}\;cat\;echo END
echo /foo, bar/, /baz, qux/, | \
parallel -kN1 --recend ', ' --recstart / --pipe \
echo JOB{#}\;cat\;echo END
Output:
@ -2344,9 +2463,12 @@ Output:
Note the difference between setting one string and setting both strings.
With B<--regexp> the B<--recend> and B<--recstart> will be treated as a regular expression:
With B<--regexp> the B<--recend> and B<--recstart> will be treated as
a regular expression:
echo foo,bar,_baz,__qux, | parallel -kN1 --regexp --recend ,_+ --pipe echo JOB{#}\;cat\;echo END
echo foo,bar,_baz,__qux, | \
parallel -kN1 --regexp --recend ,_+ --pipe \
echo JOB{#}\;cat\;echo END
Output:
@ -2358,9 +2480,12 @@ Output:
qux,
END
GNU B<parallel> can remove the record separators with B<--remove-rec-sep>/B<--rrs>:
GNU B<parallel> can remove the record separators with
B<--remove-rec-sep>/B<--rrs>:
echo foo,bar,_baz,__qux, | parallel -kN1 --rrs --regexp --recend ,_+ --pipe echo JOB{#}\;cat\;echo END
echo foo,bar,_baz,__qux, | \
parallel -kN1 --rrs --regexp --recend ,_+ --pipe \
echo JOB{#}\;cat\;echo END
Output:
@ -2378,7 +2503,8 @@ If the input data has a header, the header can be repeated for each
job by matching the header with B<--header>. If headers start with
B<%> you can do this:
cat num_%header | parallel --header '(%.*\n)*' --pipe -N3 echo JOB{#}\;cat
cat num_%header | \
parallel --header '(%.*\n)*' --pipe -N3 echo JOB{#}\;cat
Output (the order may be different):
@ -2677,11 +2803,11 @@ would run at a time.
To control which semaphore is used, use
B<--semaphorename>/B<--id>. Run this in one terminal:
sem --id my_id -u 'echo First started; sleep 10; echo The first finished'
sem --id my_id -u 'echo First started; sleep 10; echo First done'
and simultaneously this in another terminal:
sem --id my_id -u 'echo Second started; sleep 10; echo The second finished'
sem --id my_id -u 'echo Second started; sleep 10; echo Second done'
Note how the second will only be started when the first has finished.
@ -2692,25 +2818,25 @@ else will have to wait. A counting semaphore is like having multiple
toilets: Several people can use the toilets, but when they all are in
use, everyone else will have to wait.
B<sem> can emulate a counting semaphore. Use B<--jobs> to set the number of
toilets like this:
B<sem> can emulate a counting semaphore. Use B<--jobs> to set the
number of toilets like this:
sem --jobs 3 --id my_id -u 'echo First started; sleep 5; echo The first finished' &&
sem --jobs 3 --id my_id -u 'echo Second started; sleep 6; echo The second finished' &&
sem --jobs 3 --id my_id -u 'echo Third started; sleep 7; echo The third finished' &&
sem --jobs 3 --id my_id -u 'echo Fourth started; sleep 8; echo The fourth finished' &&
sem --jobs 3 --id my_id -u 'echo Start 1; sleep 5; echo 1 done' &&
sem --jobs 3 --id my_id -u 'echo Start 2; sleep 6; echo 2 done' &&
sem --jobs 3 --id my_id -u 'echo Start 3; sleep 7; echo 3 done' &&
sem --jobs 3 --id my_id -u 'echo Start 4; sleep 8; echo 4 done' &&
sem --wait --id my_id
Output:
First started
Second started
Third started
The first finished
Fourth started
The second finished
The third finished
The fourth finished
Start 1
Start 2
Start 3
1 done
Start 4
2 done
3 done
4 done
=head2 Timeout
@ -2718,17 +2844,17 @@ With B<--semaphoretimeout> you can force running the command anyway after
a period (postive number) or give up (negative number):
sem --id foo -u 'echo Slow started; sleep 5; echo Slow ended' &&
sem --id foo --semaphoretimeout 1 'echo Force this running after 1 sec' &&
sem --id foo --semaphoretimeout -2 'echo Give up after 1 sec'
sem --id foo --semaphoretimeout 1 'echo Forced running after 1 sec' &&
sem --id foo --semaphoretimeout -2 'echo Give up after 2 secs'
sem --id foo --wait
Output:
Slow started
parallel: Warning: Semaphore timed out. Stealing the semaphore.
Force this running after 1 sec
Slow ended
Forced running after 1 sec
parallel: Warning: Semaphore timed out. Exiting.
Slow ended
Note how the 'Give up' was not run.
@ -2801,7 +2927,8 @@ Output:
In scripts B<--minversion> can be used to ensure the user has at least
this version:
parallel --minversion 20130722 && echo Your version is at least 20130722.
parallel --minversion 20130722 && \
echo Your version is at least 20130722.
Output:

2
src/sql
View file

@ -576,7 +576,7 @@ $Global::Initfile && unlink $Global::Initfile;
exit ($err);
sub parse_options {
$Global::version = 20170322;
$Global::version = 20170323;
$Global::progname = 'sql';
# This must be done first as this may exec myself

65
testsuite/Start.sh
View file

@ -8,34 +8,39 @@ SHFILE=/tmp/unittest-parallel.sh
MAX_SEC_PER_TEST=900
export TIMEOUT=$MAX_SEC_PER_TEST
run_test() {
script="$1"
base=`basename "$script" .sh`
export TMPDIR=/tmp/"$base"-tmpdir
mkdir -p "$TMPDIR"
# Clean before. May be owned by other users
sudo rm -f /tmp/*.{tmx,pac,arg,all,log,swp,loa,ssh,df,pip,tmb,chr,tms,par}
if [ "$TRIES" = "3" ] ; then
# Try 3 times
bash "$script" > actual-results/"$base"
run_once() {
script=$1
base=`basename "$script" .sh`
diff -Naur wanted-results/"$base" actual-results/"$base" >/dev/null ||
bash "$script" > actual-results/"$base"
diff -Naur wanted-results/"$base" actual-results/"$base" >/dev/null ||
bash "$script" > actual-results/"$base"
diff -Naur wanted-results/"$base" actual-results/"$base" ||
(touch "$script" && echo touch "$script")
else
# Run only once
bash "$script" > actual-results/"$base"
diff -Naur wanted-results/"$base" actual-results/"$base" ||
(touch "$script" && echo touch "$script")
fi
bash "$script" | perl -pe 's:'$HOME':~:g' > actual-results/"$base"
}
export -f run_once
# Check if it was cleaned up
find /tmp -maxdepth 1 |
perl -ne '/\.(tmx|pac|arg|all|log|swp|loa|ssh|df|pip|tmb|chr|tms|par)$/ and ++$a and print "TMP NOT CLEAN. FOUND: $_".`touch '$script'`;'
# May be owned by other users
sudo rm -f /tmp/*.{tmx,pac,arg,all,log,swp,loa,ssh,df,pip,tmb,chr,tms,par}
run_test() {
script="$1"
base=`basename "$script" .sh`
export TMPDIR=/tmp/"$base"-tmpdir
mkdir -p "$TMPDIR"
# Clean before. May be owned by other users
sudo rm -f /tmp/*.{tmx,pac,arg,all,log,swp,loa,ssh,df,pip,tmb,chr,tms,par}
# Force running once
echo >> actual-results/"$base"
if [ "$TRIES" = "3" ] ; then
# Try 2 times
run_once $script
run_once $script
fi
run_once $script
diff -Naur wanted-results/"$base" actual-results/"$base" ||
(touch "$script" && echo touch "$script")
# Check if it was cleaned up
find /tmp -maxdepth 1 |
perl -ne '/\.(tmx|pac|arg|all|log|swp|loa|ssh|df|pip|tmb|chr|tms|par)$/ and
++$a and
print "TMP NOT CLEAN. FOUND: $_".`touch '$script'`;'
# May be owned by other users
sudo rm -f /tmp/*.{tmx,pac,arg,all,log,swp,loa,ssh,df,pip,tmb,chr,tms,par}
}
export -f run_test
@ -52,10 +57,10 @@ ls -t tests-to-run/*${1}*.sh | egrep -v "${2}" |
parallel --tty -tj1 run_test | tee testsuite.log
# If testsuite.log contains @@ then there is a diff
if grep -q '@@' testsuite.log ; then
false
false
else
# No @@'s: So everything worked: Copy the source
rm -rf src-passing-testsuite
cp -a ../src src-passing-testsuite
# No @@'s: So everything worked: Copy the source
rm -rf src-passing-testsuite
cp -a ../src src-passing-testsuite
fi
date

16
testsuite/tests-to-run/parallel-local-10s.sh
View file

@ -83,22 +83,6 @@ par_halt_on_error() {
parallel -j0 -k --tag mytest ::: -2 -1 0 1 2 ::: true false ::: true false
}
par_print_before_halt_on_error() {
echo '### What is printed before the jobs are killed'
mytest() {
HALT=$1
(echo 0;
echo 3;
seq 0 7;
echo 0;
echo 8) |
parallel --tag -j10 -kq --halt $HALT perl -e 'sleep $ARGV[0];print STDERR @ARGV,"\n"; '$HALT' > 0 ? exit shift : exit not shift;';
echo exit code $?
}
export -f mytest
parallel -j0 -k --tag mytest ::: -2 -1 0 1 2
}
par_first_print_halt_on_error_1() {
echo '### Test first dying print --halt-on-error 1';
(echo 0; echo 3; seq 0 7;echo 0; echo 8) | parallel -j10 -kq --halt 1 perl -e 'sleep $ARGV[0];print STDERR @ARGV,"\n"; exit shift';

16
testsuite/tests-to-run/parallel-local-30s.sh
View file

@ -4,22 +4,6 @@
# Each should be taking 30-100s and be possible to run in parallel
# I.e.: No race conditions, no logins
par_testhalt() {
testhalt() {
echo '### testhalt --halt '$1;
(yes 0 | head -n 10; seq 10) |
stdout parallel -kj4 --halt $1 'sleep {= $_=0.3*($_+1+seq()) =}; exit {}'; echo $?;
(seq 10; yes 0 | head -n 10) |
stdout parallel -kj4 --halt $1 'sleep {= $_=0.3*($_+1+seq()) =}; exit {}'; echo $?;
};
export -f testhalt;
stdout parallel -kj0 testhalt {1},{2}={3} \
::: now soon ::: fail success ::: 0 1 2 30% 70% |
# Remove lines that only show up now and then
perl -ne '/Starting no more jobs./ or print'
}
par_race_condition1() {
echo '### Test race condition on 8 CPU (my laptop)'
seq 1 5000000 > /tmp/parallel_race_cond

42
testsuite/tests-to-run/parallel-local-race01.sh Executable file
View file

@ -0,0 +1,42 @@
#!/bin/bash
par_print_before_halt_on_error() {
echo '### What is printed before the jobs are killed'
mytest() {
HALT=$1
(echo 0;
echo 3;
seq 0 7;
echo 0;
echo 8) |
parallel --tag -j10 -kq --halt $HALT perl -e 'sleep $ARGV[0];print STDERR @ARGV,"\n"; '$HALT' > 0 ? exit shift : exit not shift;';
echo exit code $?
}
export -f mytest
parallel -j0 -k --tag mytest ::: -2 -1 0 1 2
}
par_testhalt() {
testhalt_false() {
echo '### testhalt --halt '$1;
(yes 0 | head -n 10; seq 10) |
stdout parallel -kj4 --halt $1 \
'sleep {= $_=0.3*($_+1+seq()) =}; exit {}'; echo $?;
}
testhalt_true() {
(seq 10; yes 0 | head -n 10) |
stdout parallel -kj4 --halt $1 \
'sleep {= $_=0.3*($_+1+seq()) =}; exit {}'; echo $?;
};
export -f testhalt_false;
export -f testhalt_true;
stdout parallel -kj0 testhalt_{4} {1},{2}={3} \
::: now soon ::: fail success ::: 0 1 2 30% 70% ::: true false |
# Remove lines that only show up now and then
perl -ne '/Starting no more jobs./ or print'
}
export -f $(compgen -A function | grep par_)
compgen -A function | grep par_ | sort |
parallel --joblog /tmp/jl-`basename $0` -j10 --tag -k '{} 2>&1'

1
testsuite/tests-to-run/parallel-local-ssh4.sh
View file

@ -1,6 +1,7 @@
#!/bin/bash
unset run_test
unset run_once
# SSH only allowed to localhost/lo
# --retries if ssh dies

14
testsuite/wanted-results/parallel-install
View file

@ -1,24 +1,24 @@
### Test installation missing pod2*
make[0]: Entering directory '/mnt/4tb/home/tange/privat/parallel'
make[0]: Entering directory '~/privat/parallel'
make dist-gzip am__post_remove_distdir='@:'
make[0]: Entering directory '/mnt/4tb/home/tange/privat/parallel'
make[0]: Entering directory '~/privat/parallel'
if test -d "parallel-00000000"; then find "parallel-00000000" -type d ! -perm -200 -exec chmod u+w {} ';' && rm -rf "parallel-00000000" || { sleep 5 && rm -rf "parallel-00000000"; }; else :; fi
test -d "parallel-00000000" || mkdir "parallel-00000000"
(cd src && make top_distdir=../parallel-00000000 distdir=../parallel-00000000/src \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[0]: Entering directory '/mnt/4tb/home/tange/privat/parallel/src'
make[0]: Leaving directory '/mnt/4tb/home/tange/privat/parallel/src'
make[0]: Entering directory '~/privat/parallel/src'
make[0]: Leaving directory '~/privat/parallel/src'
test -n "" \
|| find "parallel-00000000" -type d ! -perm -755 \
-exec chmod u+rwx,go+rx {} \; -o \
! -type d ! -perm -444 -links 1 -exec chmod a+r {} \; -o \
! -type d ! -perm -400 -exec chmod a+r {} \; -o \
! -type d ! -perm -444 -exec /bin/bash /mnt/4tb/home/tange/privat/parallel/install-sh -c -m a+r {} {} \; \
! -type d ! -perm -444 -exec /bin/bash ~/privat/parallel/install-sh -c -m a+r {} {} \; \
|| chmod -R a+r "parallel-00000000"
tardir=parallel-00000000 && ${TAR-tar} chof - "$tardir" | GZIP=--best gzip -c >parallel-00000000.tar.gz
make[0]: Leaving directory '/mnt/4tb/home/tange/privat/parallel'
make[0]: Leaving directory '~/privat/parallel'
if test -d "parallel-00000000"; then find "parallel-00000000" -type d ! -perm -200 -exec chmod u+w {} ';' && rm -rf "parallel-00000000" || { sleep 5 && rm -rf "parallel-00000000"; }; else :; fi
make[0]: Leaving directory '/mnt/4tb/home/tange/privat/parallel'
make[0]: Leaving directory '~/privat/parallel'
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p

10
testsuite/wanted-results/parallel-local-0.3s
View file

@ -297,23 +297,23 @@ bug #45993: --wd ... should also work when run locally
parallel: Error: Cannot change into non-executable dir /bi: No such file or directory
parallel --wd /bin 'pwd; echo $OLDPWD; echo' ::: OK
/bin
/mnt/4tb/home/tange/privat/parallel/testsuite
~/privat/parallel/testsuite
OK
parallel --wd / 'pwd; echo $OLDPWD; echo' ::: OK
/
/mnt/4tb/home/tange/privat/parallel/testsuite
~/privat/parallel/testsuite
OK
parallel --wd /tmp 'pwd; echo $OLDPWD; echo' ::: OK
/tmp
/mnt/4tb/home/tange/privat/parallel/testsuite
~/privat/parallel/testsuite
OK
parallel --wd ... 'pwd; echo $OLDPWD; echo' ::: OK | perl -pe 's/\d+/0/g'
/mnt/0tb/home/tange/.parallel/tmp/aspire-0-0
/mnt/0tb/home/tange/privat/parallel/testsuite
OK
parallel --wd . 'pwd; echo $OLDPWD; echo' ::: OK
/mnt/4tb/home/tange/privat/parallel/testsuite
/mnt/4tb/home/tange/privat/parallel/testsuite
~/privat/parallel/testsuite
~/privat/parallel/testsuite
OK
echo '**'
**

100
testsuite/wanted-results/parallel-local-10s
View file

@ -564,106 +564,6 @@ par_plus_dyn_repl abcaaadef
par_plus_dyn_repl abcaaadef
par_plus_dyn_repl abcaaadef
par_plus_dyn_repl abcaaadef
par_print_before_halt_on_error ### What is printed before the jobs are killed
par_print_before_halt_on_error -2 exit code 0
par_print_before_halt_on_error -2 0 0
par_print_before_halt_on_error -2 parallel: This job succeeded:
par_print_before_halt_on_error -2 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ -2\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 1
par_print_before_halt_on_error -1 exit code 0
par_print_before_halt_on_error -1 0 0
par_print_before_halt_on_error -1 parallel: This job succeeded:
par_print_before_halt_on_error -1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ -1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 1
par_print_before_halt_on_error -1 parallel: Starting no more jobs. Waiting for 8 jobs to finish.
par_print_before_halt_on_error -1 parallel: This job succeeded:
par_print_before_halt_on_error -1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ -1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 2
par_print_before_halt_on_error -1 parallel: Starting no more jobs. Waiting for 7 jobs to finish.
par_print_before_halt_on_error -1 3 3
par_print_before_halt_on_error -1 0 0
par_print_before_halt_on_error -1 1 1
par_print_before_halt_on_error -1 2 2
par_print_before_halt_on_error -1 parallel: This job succeeded:
par_print_before_halt_on_error -1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ -1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 3
par_print_before_halt_on_error -1 parallel: Starting no more jobs. Waiting for 6 jobs to finish.
par_print_before_halt_on_error -1 3 3
par_print_before_halt_on_error -1 parallel: This job succeeded:
par_print_before_halt_on_error -1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ -1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 3
par_print_before_halt_on_error -1 parallel: Starting no more jobs. Waiting for 5 jobs to finish.
par_print_before_halt_on_error -1 4 4
par_print_before_halt_on_error -1 parallel: This job succeeded:
par_print_before_halt_on_error -1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ -1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 4
par_print_before_halt_on_error -1 parallel: Starting no more jobs. Waiting for 4 jobs to finish.
par_print_before_halt_on_error -1 5 5
par_print_before_halt_on_error -1 parallel: This job succeeded:
par_print_before_halt_on_error -1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ -1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 5
par_print_before_halt_on_error -1 parallel: Starting no more jobs. Waiting for 3 jobs to finish.
par_print_before_halt_on_error -1 6 6
par_print_before_halt_on_error -1 parallel: This job succeeded:
par_print_before_halt_on_error -1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ -1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 6
par_print_before_halt_on_error -1 parallel: Starting no more jobs. Waiting for 2 jobs to finish.
par_print_before_halt_on_error -1 7 7
par_print_before_halt_on_error -1 0 0
par_print_before_halt_on_error -1 parallel: This job succeeded:
par_print_before_halt_on_error -1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ -1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 7
par_print_before_halt_on_error -1 parallel: Starting no more jobs. Waiting for 1 jobs to finish.
par_print_before_halt_on_error -1 8 8
par_print_before_halt_on_error -1 parallel: This job succeeded:
par_print_before_halt_on_error -1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ -1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 8
par_print_before_halt_on_error 0 exit code 3
par_print_before_halt_on_error 0 0 0
par_print_before_halt_on_error 0 3 3
par_print_before_halt_on_error 0 0 0
par_print_before_halt_on_error 0 1 1
par_print_before_halt_on_error 0 2 2
par_print_before_halt_on_error 0 3 3
par_print_before_halt_on_error 0 4 4
par_print_before_halt_on_error 0 5 5
par_print_before_halt_on_error 0 6 6
par_print_before_halt_on_error 0 7 7
par_print_before_halt_on_error 0 0 0
par_print_before_halt_on_error 0 8 8
par_print_before_halt_on_error 1 exit code 1
par_print_before_halt_on_error 1 0 0
par_print_before_halt_on_error 1 parallel: This job failed:
par_print_before_halt_on_error 1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ 1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 1
par_print_before_halt_on_error 1 parallel: Starting no more jobs. Waiting for 8 jobs to finish.
par_print_before_halt_on_error 1 parallel: This job failed:
par_print_before_halt_on_error 1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ 1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 2
par_print_before_halt_on_error 1 parallel: Starting no more jobs. Waiting for 7 jobs to finish.
par_print_before_halt_on_error 1 3 3
par_print_before_halt_on_error 1 0 0
par_print_before_halt_on_error 1 1 1
par_print_before_halt_on_error 1 2 2
par_print_before_halt_on_error 1 parallel: This job failed:
par_print_before_halt_on_error 1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ 1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 3
par_print_before_halt_on_error 1 parallel: Starting no more jobs. Waiting for 6 jobs to finish.
par_print_before_halt_on_error 1 3 3
par_print_before_halt_on_error 1 parallel: This job failed:
par_print_before_halt_on_error 1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ 1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 3
par_print_before_halt_on_error 1 parallel: Starting no more jobs. Waiting for 5 jobs to finish.
par_print_before_halt_on_error 1 4 4
par_print_before_halt_on_error 1 parallel: This job failed:
par_print_before_halt_on_error 1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ 1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 4
par_print_before_halt_on_error 1 parallel: Starting no more jobs. Waiting for 4 jobs to finish.
par_print_before_halt_on_error 1 5 5
par_print_before_halt_on_error 1 parallel: This job failed:
par_print_before_halt_on_error 1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ 1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 5
par_print_before_halt_on_error 1 parallel: Starting no more jobs. Waiting for 3 jobs to finish.
par_print_before_halt_on_error 1 6 6
par_print_before_halt_on_error 1 parallel: This job failed:
par_print_before_halt_on_error 1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ 1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 6
par_print_before_halt_on_error 1 parallel: Starting no more jobs. Waiting for 2 jobs to finish.
par_print_before_halt_on_error 1 7 7
par_print_before_halt_on_error 1 0 0
par_print_before_halt_on_error 1 parallel: This job failed:
par_print_before_halt_on_error 1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ 1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 7
par_print_before_halt_on_error 1 parallel: Starting no more jobs. Waiting for 1 jobs to finish.
par_print_before_halt_on_error 1 8 8
par_print_before_halt_on_error 1 parallel: This job failed:
par_print_before_halt_on_error 1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ 1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 8
par_print_before_halt_on_error 2 exit code 1
par_print_before_halt_on_error 2 0 0
par_print_before_halt_on_error 2 parallel: This job failed:
par_print_before_halt_on_error 2 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ 2\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 1
par_results_compress 0
par_results_compress 1
par_results_csv bug #: --results csv

472
testsuite/wanted-results/parallel-local-30s
View file

@ -20,478 +20,6 @@ par_race_condition1 7
par_race_condition1 8
par_race_condition1 9
par_race_condition1 10
par_testhalt ### testhalt --halt now,fail=0
par_testhalt parallel: This job failed:
par_testhalt sleep 3.9; exit 1
par_testhalt 0
par_testhalt parallel: This job failed:
par_testhalt sleep 0.9; exit 1
par_testhalt 0
par_testhalt ### testhalt --halt now,fail=1
par_testhalt parallel: This job failed:
par_testhalt sleep 3.9; exit 1
par_testhalt 1
par_testhalt parallel: This job failed:
par_testhalt sleep 0.9; exit 1
par_testhalt 1
par_testhalt ### testhalt --halt now,fail=2
par_testhalt parallel: This job failed:
par_testhalt sleep 3.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 4.5; exit 2
par_testhalt 2
par_testhalt parallel: This job failed:
par_testhalt sleep 0.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 1.5; exit 2
par_testhalt 2
par_testhalt ### testhalt --halt now,fail=30%
par_testhalt parallel: This job failed:
par_testhalt sleep 3.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 4.5; exit 2
par_testhalt parallel: This job failed:
par_testhalt sleep 5.1; exit 3
par_testhalt parallel: This job failed:
par_testhalt sleep 5.7; exit 4
par_testhalt parallel: This job failed:
par_testhalt sleep 6.3; exit 5
par_testhalt parallel: This job failed:
par_testhalt sleep 6.9; exit 6
par_testhalt 30
par_testhalt parallel: This job failed:
par_testhalt sleep 0.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 1.5; exit 2
par_testhalt parallel: This job failed:
par_testhalt sleep 2.1; exit 3
par_testhalt parallel: This job failed:
par_testhalt sleep 2.7; exit 4
par_testhalt parallel: This job failed:
par_testhalt sleep 3.3; exit 5
par_testhalt parallel: This job failed:
par_testhalt sleep 3.9; exit 6
par_testhalt 30
par_testhalt ### testhalt --halt now,fail=70%
par_testhalt parallel: This job failed:
par_testhalt sleep 3.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 4.5; exit 2
par_testhalt parallel: This job failed:
par_testhalt sleep 5.1; exit 3
par_testhalt parallel: This job failed:
par_testhalt sleep 5.7; exit 4
par_testhalt parallel: This job failed:
par_testhalt sleep 6.3; exit 5
par_testhalt parallel: This job failed:
par_testhalt sleep 6.9; exit 6
par_testhalt parallel: This job failed:
par_testhalt sleep 7.5; exit 7
par_testhalt parallel: This job failed:
par_testhalt sleep 8.1; exit 8
par_testhalt parallel: This job failed:
par_testhalt sleep 8.7; exit 9
par_testhalt parallel: This job failed:
par_testhalt sleep 9.3; exit 10
par_testhalt 50
par_testhalt parallel: This job failed:
par_testhalt sleep 0.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 1.5; exit 2
par_testhalt parallel: This job failed:
par_testhalt sleep 2.1; exit 3
par_testhalt parallel: This job failed:
par_testhalt sleep 2.7; exit 4
par_testhalt parallel: This job failed:
par_testhalt sleep 3.3; exit 5
par_testhalt parallel: This job failed:
par_testhalt sleep 3.9; exit 6
par_testhalt parallel: This job failed:
par_testhalt sleep 4.5; exit 7
par_testhalt parallel: This job failed:
par_testhalt sleep 5.1; exit 8
par_testhalt parallel: This job failed:
par_testhalt sleep 5.7; exit 9
par_testhalt parallel: This job failed:
par_testhalt sleep 6.3; exit 10
par_testhalt 50
par_testhalt ### testhalt --halt now,success=0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.6; exit 0
par_testhalt 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.6; exit 0
par_testhalt 0
par_testhalt ### testhalt --halt now,success=1
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.6; exit 0
par_testhalt 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.6; exit 0
par_testhalt 0
par_testhalt ### testhalt --halt now,success=2
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.9; exit 0
par_testhalt 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.9; exit 0
par_testhalt 0
par_testhalt ### testhalt --halt now,success=30%
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.9; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.2; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.5; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.8; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 2.1; exit 0
par_testhalt 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.9; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.2; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.5; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.8; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 5.1; exit 0
par_testhalt 0
par_testhalt ### testhalt --halt now,success=70%
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.9; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.2; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.5; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.8; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 2.1; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 2.4; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 2.7; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.3; exit 0
par_testhalt 50
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.9; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.2; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.5; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.8; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 5.1; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 5.4; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 5.7; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 6.3; exit 0
par_testhalt 50
par_testhalt ### testhalt --halt soon,fail=0
par_testhalt parallel: This job failed:
par_testhalt sleep 3.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 4.5; exit 2
par_testhalt parallel: This job failed:
par_testhalt sleep 5.1; exit 3
par_testhalt parallel: This job failed:
par_testhalt sleep 5.7; exit 4
par_testhalt 0
par_testhalt parallel: This job failed:
par_testhalt sleep 0.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 1.5; exit 2
par_testhalt parallel: This job failed:
par_testhalt sleep 2.1; exit 3
par_testhalt parallel: This job failed:
par_testhalt sleep 2.7; exit 4
par_testhalt 0
par_testhalt ### testhalt --halt soon,fail=1
par_testhalt parallel: This job failed:
par_testhalt sleep 3.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 4.5; exit 2
par_testhalt parallel: This job failed:
par_testhalt sleep 5.1; exit 3
par_testhalt parallel: This job failed:
par_testhalt sleep 5.7; exit 4
par_testhalt 1
par_testhalt parallel: This job failed:
par_testhalt sleep 0.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 1.5; exit 2
par_testhalt parallel: This job failed:
par_testhalt sleep 2.1; exit 3
par_testhalt parallel: This job failed:
par_testhalt sleep 2.7; exit 4
par_testhalt 1
par_testhalt ### testhalt --halt soon,fail=2
par_testhalt parallel: This job failed:
par_testhalt sleep 3.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 4.5; exit 2
par_testhalt parallel: This job failed:
par_testhalt sleep 5.1; exit 3
par_testhalt parallel: This job failed:
par_testhalt sleep 5.7; exit 4
par_testhalt parallel: This job failed:
par_testhalt sleep 6.3; exit 5
par_testhalt 2
par_testhalt parallel: This job failed:
par_testhalt sleep 0.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 1.5; exit 2
par_testhalt parallel: This job failed:
par_testhalt sleep 2.1; exit 3
par_testhalt parallel: This job failed:
par_testhalt sleep 2.7; exit 4
par_testhalt parallel: This job failed:
par_testhalt sleep 3.3; exit 5
par_testhalt 2
par_testhalt ### testhalt --halt soon,fail=30%
par_testhalt parallel: This job failed:
par_testhalt sleep 3.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 4.5; exit 2
par_testhalt parallel: This job failed:
par_testhalt sleep 5.1; exit 3
par_testhalt parallel: This job failed:
par_testhalt sleep 5.7; exit 4
par_testhalt parallel: This job failed:
par_testhalt sleep 6.3; exit 5
par_testhalt parallel: This job failed:
par_testhalt sleep 6.9; exit 6
par_testhalt parallel: This job failed:
par_testhalt sleep 7.5; exit 7
par_testhalt parallel: This job failed:
par_testhalt sleep 8.1; exit 8
par_testhalt parallel: This job failed:
par_testhalt sleep 8.7; exit 9
par_testhalt 30
par_testhalt parallel: This job failed:
par_testhalt sleep 0.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 1.5; exit 2
par_testhalt parallel: This job failed:
par_testhalt sleep 2.1; exit 3
par_testhalt parallel: This job failed:
par_testhalt sleep 2.7; exit 4
par_testhalt parallel: This job failed:
par_testhalt sleep 3.3; exit 5
par_testhalt parallel: This job failed:
par_testhalt sleep 3.9; exit 6
par_testhalt parallel: This job failed:
par_testhalt sleep 4.5; exit 7
par_testhalt parallel: This job failed:
par_testhalt sleep 5.1; exit 8
par_testhalt parallel: This job failed:
par_testhalt sleep 5.7; exit 9
par_testhalt 30
par_testhalt ### testhalt --halt soon,fail=70%
par_testhalt parallel: This job failed:
par_testhalt sleep 3.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 4.5; exit 2
par_testhalt parallel: This job failed:
par_testhalt sleep 5.1; exit 3
par_testhalt parallel: This job failed:
par_testhalt sleep 5.7; exit 4
par_testhalt parallel: This job failed:
par_testhalt sleep 6.3; exit 5
par_testhalt parallel: This job failed:
par_testhalt sleep 6.9; exit 6
par_testhalt parallel: This job failed:
par_testhalt sleep 7.5; exit 7
par_testhalt parallel: This job failed:
par_testhalt sleep 8.1; exit 8
par_testhalt parallel: This job failed:
par_testhalt sleep 8.7; exit 9
par_testhalt parallel: This job failed:
par_testhalt sleep 9.3; exit 10
par_testhalt 50
par_testhalt parallel: This job failed:
par_testhalt sleep 0.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 1.5; exit 2
par_testhalt parallel: This job failed:
par_testhalt sleep 2.1; exit 3
par_testhalt parallel: This job failed:
par_testhalt sleep 2.7; exit 4
par_testhalt parallel: This job failed:
par_testhalt sleep 3.3; exit 5
par_testhalt parallel: This job failed:
par_testhalt sleep 3.9; exit 6
par_testhalt parallel: This job failed:
par_testhalt sleep 4.5; exit 7
par_testhalt parallel: This job failed:
par_testhalt sleep 5.1; exit 8
par_testhalt parallel: This job failed:
par_testhalt sleep 5.7; exit 9
par_testhalt parallel: This job failed:
par_testhalt sleep 6.3; exit 10
par_testhalt 50
par_testhalt ### testhalt --halt soon,success=0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.9; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.2; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.5; exit 0
par_testhalt 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.9; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.2; exit 0
par_testhalt 0
par_testhalt ### testhalt --halt soon,success=1
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.9; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.2; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.5; exit 0
par_testhalt 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.9; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.2; exit 0
par_testhalt 0
par_testhalt ### testhalt --halt soon,success=2
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.9; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.2; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.5; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.8; exit 0
par_testhalt 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.9; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.2; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.5; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.8; exit 0
par_testhalt 0
par_testhalt ### testhalt --halt soon,success=30%
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.9; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.2; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.5; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.8; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 2.1; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 2.4; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 2.7; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3; exit 0
par_testhalt 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.9; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.2; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.5; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.8; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 5.1; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 5.4; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 5.7; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 6; exit 0
par_testhalt 0
par_testhalt ### testhalt --halt soon,success=70%
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.9; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.2; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.5; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.8; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 2.1; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 2.4; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 2.7; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.3; exit 0
par_testhalt 50
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.9; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.2; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.5; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.8; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 5.1; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 5.4; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 5.7; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 6.3; exit 0
par_testhalt 50
par_tmp_full ### Test --tmpdir running full. bug #40733 was caused by this
par_tmp_full parallel: Error: Output is incomplete. Cannot append to buffer file in /tmp/shm/parallel. Is the disk full?
par_tmp_full parallel: Error: Change $TMPDIR with --tmpdir or use --compress.

10
testsuite/wanted-results/parallel-local-ssh1
View file

@ -20,22 +20,22 @@ echo '### bug #41805: Idea: propagate --env for parallel --number-of-cores'
** test_zsh
FOO=test_zsh parallel --env FOO,HOME -S zsh@lo -N0 env ::: "" |sort|egrep 'FOO|^HOME'
FOO=test_zsh
HOME=/mnt/4tb/home/tange
HOME=~
echo '** test_zsh_filter'
** test_zsh_filter
FOO=test_zsh_filter parallel --filter-hosts --env FOO,HOME -S zsh@lo -N0 env ::: "" |sort|egrep 'FOO|^HOME'
FOO=test_zsh_filter
HOME=/mnt/4tb/home/tange
HOME=~
echo '** test_csh'
** test_csh
FOO=test_csh parallel --env FOO,HOME -S csh@lo -N0 env ::: "" |sort|egrep 'FOO|^HOME'
FOO=test_csh
HOME=/mnt/4tb/home/tange
HOME=~
echo '** test_csh_filter'
** test_csh_filter
FOO=test_csh_filter parallel --filter-hosts --env FOO,HOME -S csh@lo -N0 env ::: "" |sort|egrep 'FOO|^HOME'
FOO=test_csh_filter
HOME=/mnt/4tb/home/tange
HOME=~
echo '** bug #41805 done'
** bug #41805 done
echo '### Deal with long command lines on remote servers'
@ -64,7 +64,7 @@ echo '### bug #40001: --joblog and --nonall seem not to work together:'
echo '### bug #40132: FreeBSD: --workdir . gives warning if . == $HOME'
### bug #40132: FreeBSD: --workdir . gives warning if . == $HOME
cd && parallel --workdir . -S lo pwd ::: ""
/mnt/4tb/home/tange
~
echo '### use function as $PARALLEL_SSH'
### use function as $PARALLEL_SSH
foossh() { echo "FOOSSH" >&2; ssh "$@"; }; export -f foossh; PARALLEL_SSH=foossh parallel -S 1/lo echo ::: 'Run through FOOSSH?'

2
testsuite/wanted-results/parallel-local22
View file

@ -88,7 +88,7 @@ echo '### bug #42902: profiles containing arguments with space'
echo /bin/bash\=/bin/bash
/bin/bash=/bin/bash
PARALLEL="--rpl 'FULLPATH chomp(\$_=\"/bin/bash=\".\`readlink -f \$_\`);' perl -e \'print \\\"@ARGV\\\n\\\"\' " parallel With script in \\\$PARALLEL FULLPATH ::: .
With script in $PARALLEL /bin/bash=/mnt/4tb/home/tange/privat/parallel/testsuite
With script in $PARALLEL /bin/bash=~/privat/parallel/testsuite
echo '### bug #42892: parallel -a nonexiting --pipepart'
### bug #42892: parallel -a nonexiting --pipepart
parallel --pipepart -a nonexisting wc

12
testsuite/wanted-results/parallel-local4
View file

@ -1,15 +1,15 @@
bug #46120: Suspend should suspend (at least local) children
it should burn 1.9 CPU seconds, but no more than that
The 5 second sleep will make it be killed by timeout when it fgs
1024 SHA256:SXgag2Z2L91JsrT5WjNBcARD1EpyCCj4JctVJ6Zpzm0 /mnt/4tb/home/tange/.ssh/id_dsa (DSA)
8192 SHA256:lYn04AefJq/5r0e4FftceviJ7JVnq9NGKY3CW9XMpO8 /mnt/4tb/home/tange/.ssh/id_rsa (RSA)
4096 SHA256:jUQ9ysfprs7UOckttjjVb+j3qikUmKWDEWC+eEJkbDQ /mnt/4tb/home/tange/.ssh/id_rsa_openindiana (RSA)
1024 SHA256:SXgag2Z2L91JsrT5WjNBcARD1EpyCCj4JctVJ6Zpzm0 ~/.ssh/id_dsa (DSA)
8192 SHA256:lYn04AefJq/5r0e4FftceviJ7JVnq9NGKY3CW9XMpO8 ~/.ssh/id_rsa (RSA)
4096 SHA256:jUQ9ysfprs7UOckttjjVb+j3qikUmKWDEWC+eEJkbDQ ~/.ssh/id_rsa_openindiana (RSA)
stdout /usr/bin/time -f CPUTIME=%U parallel --timeout 5 burnP6 ::: 1 | grep -q CPUTIME=1
Zero=OK 0
1024 SHA256:SXgag2Z2L91JsrT5WjNBcARD1EpyCCj4JctVJ6Zpzm0 /mnt/4tb/home/tange/.ssh/id_dsa (DSA)
8192 SHA256:lYn04AefJq/5r0e4FftceviJ7JVnq9NGKY3CW9XMpO8 /mnt/4tb/home/tange/.ssh/id_rsa (RSA)
4096 SHA256:jUQ9ysfprs7UOckttjjVb+j3qikUmKWDEWC+eEJkbDQ /mnt/4tb/home/tange/.ssh/id_rsa_openindiana (RSA)
1024 SHA256:SXgag2Z2L91JsrT5WjNBcARD1EpyCCj4JctVJ6Zpzm0 ~/.ssh/id_dsa (DSA)
8192 SHA256:lYn04AefJq/5r0e4FftceviJ7JVnq9NGKY3CW9XMpO8 ~/.ssh/id_rsa (RSA)
4096 SHA256:jUQ9ysfprs7UOckttjjVb+j3qikUmKWDEWC+eEJkbDQ ~/.ssh/id_rsa_openindiana (RSA)
echo 1 | stdout /usr/bin/time -f CPUTIME=%U parallel --timeout 5 burnP6 | grep -q CPUTIME=1
Zero=OK 0

419
testsuite/wanted-results/parallel-tutorial
View file

@ -1,4 +1,8 @@
### test parallel_tutorial
fetch -o - http://pi.dk/3) | bash
/bin/bash: -c: line 0: syntax error near unexpected token `)'
/bin/bash: -c: line 0: ` fetch -o - http://pi.dk/3) | bash'
parallel -k echo ::: A B C > abc-file
parallel -k echo ::: D E F > def-file
perl -e 'printf "A\0B\0C\0"' > abc0-file
@ -8,7 +12,9 @@
perl -e 'for(1..128){print "$_\n"}' > num128
perl -e 'for(1..30000){print "$_\n"}' > num30000
perl -e 'for(1..1000000){print "$_\n"}' > num1000000
(echo %head1; echo %head2; perl -e 'for(1..10){print "$_\n"}') > num_%header
(echo %head1; echo %head2; \
sleep .3
/bin/bash: -c: line 2: syntax error: unexpected end of file
sleep .3
sleep .3
sleep .3
@ -18,8 +24,10 @@ sleep .3
sleep .3
sleep .3
sleep .3
sleep .3
perl -e 'for(1..10){print "$_\n"}') > num_%header
/bin/bash: -c: line 0: syntax error near unexpected token `)'
/bin/bash: -c: line 0: ` perl -e 'for(1..10){print "$_\n"}') > num_%header'
perl -e 'print "HHHHAAABBBCCC"' > fixedlen
parallel echo ::: A B C
A
B
@ -154,14 +162,14 @@ abc-file
abc0-file
abc_-file
def-file
fixedlen
num1000000
num128
num30000
num8
num_%header
tsv-file.tsv
foo
/mnt/4tb/home/tange/privat/parallel/testsuite/tmp
~/privat/parallel/testsuite/tmp
my_func() {
echo in my_func $1
}
@ -223,33 +231,43 @@ Job 5 of 5
9
9
9
parallel echo {= 'if($arg[1]==$arg[2]) { skip() }' =} ::: {1..3} ::: {1..3}
parallel echo {= 'if($arg[1]==$arg[2]) { skip() }' =} \
::: {1..3} ::: {1..3}
1 2
1 3
2 1
2 3
3 1
3 2
parallel --parens ,,,, echo ',, s:\.[^.]+$::;s:\.[^.]+$::; ,,' ::: foo.tar.gz
parallel --parens ,,,, echo ',, s:\.[^.]+$::;s:\.[^.]+$::; ,,' \
::: foo.tar.gz
foo
parallel --rpl '.. s:\.[^.]+$::;s:\.[^.]+$::;' echo '..' ::: foo.tar.gz
foo
parallel --rpl '{..} s:\.[^.]+$::;s:\.[^.]+$::;' echo '{..}' ::: foo.tar.gz
parallel --rpl '.. s:\.[^.]+$::;s:\.[^.]+$::;' echo '..' \
::: foo.tar.gz
foo
parallel --rpl '{..} s:\.[^.]+$::;s:\.[^.]+$::;' echo '{..}'
::: foo.tar.gz
/bin/bash: line 1: :::: command not found
parallel --rpl '{%(.+?)} s/$$1$//;' echo {%.tar.gz}.zip ::: foo.tar.gz
foo.zip
parallel --rpl '{/(.+?)/(.*?)} s/$$1/$$2/;' echo {/tar.gz/zip} \
::: foo.tar.gz
foo.zip
--rpl '{} '
--rpl '{#} $_=$job->seq()'
--rpl '{%} $_=$job->slot()'
--rpl '{/} s:.*/::'
--rpl '{//} $Global::use{"File::Basename"} ||= eval "use File::Basename; 1;"; $_ = dirname($_);'
--rpl '{//} $Global::use{"File::Basename"} ||=
eval "use File::Basename; 1;"; $_ = dirname($_);'
--rpl '{/.} s:.*/::; s:\.[^/.]+$::;'
--rpl '{.} s:\.[^/.]+$::'
/bin/bash: --rpl: command not found
/bin/bash: line 1: --rpl: command not found
/bin/bash: line 2: --rpl: command not found
/bin/bash: line 3: --rpl: command not found
/bin/bash: line 4: --rpl: command not found
/bin/bash: line 5: --rpl: command not found
/bin/bash: line 6: --rpl: command not found
/bin/bash: line 7: --rpl: command not found
parallel echo {1} and {2} ::: A B ::: C D
A and C
A and D
@ -258,7 +276,8 @@ B and D
parallel echo /={1/} //={1//} /.={1/.} .={1.} ::: A/B.C D/E.F
/=B.C //=A /.=B .=A/B
/=E.F //=D /.=E .=D/E
parallel echo 1={1} 2={2} 3={3} -1={-1} -2={-2} -3={-3} ::: A B ::: C D ::: E F
parallel echo 1={1} 2={2} 3={3} -1={-1} -2={-2} -3={-3} \
::: A B ::: C D ::: E F
1=A 2=C 3=E -1=E -2=C -3=A
1=A 2=C 3=F -1=F -2=C -3=A
1=A 2=D 3=E -1=E -2=D -3=A
@ -267,9 +286,11 @@ B and D
1=B 2=C 3=F -1=F -2=C -3=B
1=B 2=D 3=E -1=E -2=D -3=B
1=B 2=D 3=F -1=F -2=D -3=B
parallel echo '{=2 s:\.[^.]+$::;s:\.[^.]+$::; =} {1}' ::: bar ::: foo.tar.gz
parallel echo '{=2 s:\.[^.]+$::;s:\.[^.]+$::; =} {1}' \
::: bar ::: foo.tar.gz
foo bar
parallel --rpl '{..} s:\.[^.]+$::;s:\.[^.]+$::;' echo '{2..} {1}' ::: bar ::: foo.tar.gz
parallel --rpl '{..} s:\.[^.]+$::;s:\.[^.]+$::;' echo '{2..} {1}' \
::: bar ::: foo.tar.gz
foo bar
parallel --colsep '\t' echo 1={1} 2={2} :::: tsv-file.tsv
1=f1 2=f2
@ -280,31 +301,83 @@ f1=A f2=C
f1=A f2=D
f1=B f2=C
f1=B f2=D
parallel --header : --colsep '\t' echo f1={f1} f2={f2} :::: tsv-file.tsv
parallel --header : --colsep '\t' echo f1={f1} f2={f2} \
:::: tsv-file.tsv
f1=A f2=B
f1=C f2=D
parallel --plus echo {} ::: dir/sub/file.ext1.ext2.ext3
parallel --plus echo {+/}/{/} ::: dir/sub/file.ext1.ext2.ext3
parallel --plus echo {.}.{+.} ::: dir/sub/file.ext1.ext2.ext3
parallel --plus echo {+/}/{/.}.{+.} ::: dir/sub/file.ext1.ext2.ext3
parallel --plus echo {..}.{+..} ::: dir/sub/file.ext1.ext2.ext3
parallel --plus echo {+/}/{/..}.{+..} ::: dir/sub/file.ext1.ext2.ext3
parallel --plus echo {...}.{+...} ::: dir/sub/file.ext1.ext2.ext3
parallel --plus echo {+/}/{/...}.{+...} ::: dir/sub/file.ext1.ext2.ext3
dir/sub/file.ext1.ext2.ext3
dir/sub/file.ext1.ext2.ext3
dir/sub/file.ext1.ext2.ext3
dir/sub/file.ext1.ext2.ext3
dir/sub/file.ext1.ext2.ext3
dir/sub/file.ext1.ext2.ext3
dir/sub/file.ext1.ext2.ext3
dir/sub/file.ext1.ext2.ext3
parallel --plus echo {} ::: dir/sub/file.ex1.ex2.ex3
parallel --plus echo {+/}/{/} ::: dir/sub/file.ex1.ex2.ex3
parallel --plus echo {.}.{+.} ::: dir/sub/file.ex1.ex2.ex3
parallel --plus echo {+/}/{/.}.{+.} ::: dir/sub/file.ex1.ex2.ex3
parallel --plus echo {..}.{+..} ::: dir/sub/file.ex1.ex2.ex3
parallel --plus echo {+/}/{/..}.{+..} ::: dir/sub/file.ex1.ex2.ex3
parallel --plus echo {...}.{+...} ::: dir/sub/file.ex1.ex2.ex3
parallel --plus echo {+/}/{/...}.{+...} ::: dir/sub/file.ex1.ex2.ex3
dir/sub/file.ex1.ex2.ex3
dir/sub/file.ex1.ex2.ex3
dir/sub/file.ex1.ex2.ex3
dir/sub/file.ex1.ex2.ex3
dir/sub/file.ex1.ex2.ex3
dir/sub/file.ex1.ex2.ex3
dir/sub/file.ex1.ex2.ex3
dir/sub/file.ex1.ex2.ex3
parallel --plus echo Job {#} of {##} ::: {1..5}
Job 1 of 5
Job 2 of 5
Job 3 of 5
Job 4 of 5
Job 5 of 5
unset myvar
echo ${myvar:-myval}
parallel --plus echo {:-myval} ::: "$myvar"
myval
myval
myvar=abcAaAdef
echo ${myvar:2}
parallel --plus echo {:2} ::: "$myvar"
cAaAdef
cAaAdef
echo ${myvar:2:3}
parallel --plus echo {:2:3} ::: "$myvar"
echo ${myvar#bc}
parallel --plus echo {#bc} ::: "$myvar"
echo ${myvar#abc}
parallel --plus echo {#abc} ::: "$myvar"
echo ${myvar%de}
parallel --plus echo {%de} ::: "$myvar"
echo ${myvar%def}
parallel --plus echo {%def} ::: "$myvar"
echo ${myvar/def/ghi}
parallel --plus echo {/def/ghi} ::: "$myvar"
echo ${myvar^a}
parallel --plus echo {^a} ::: "$myvar"
echo ${myvar^^a}
parallel --plus echo {^^a} ::: "$myvar"
myvar=AbcAaAdef
echo ${myvar,A}
parallel --plus echo '{,A}' ::: "$myvar"
echo ${myvar,,A}
parallel --plus echo '{,,A}' ::: "$myvar"
abcAaAdef
abcAaAdef
abcaaadef
abcaaadef
cat num30000 | parallel --xargs echo | wc -l
9
cat num30000 | parallel --xargs -s 10000 echo | wc -l
@ -337,15 +410,15 @@ foo
A
parallel perl -e 'print "@ARGV\n"' ::: This wont work
parallel: Warning: Input is read from the terminal. You either know what you
parallel: Warning: are doing (in which case: YOU ARE AWESOME!) or you forgot
parallel: Warning: ::: or :::: or to pipe data into parallel. If so
parallel: Warning: Press CTRL-D to exit.
Warning: Input is read from the terminal. You either know what you
Warning: are doing (in which case: YOU ARE AWESOME!) or you forgot
Warning: ::: or :::: or to pipe data into parallel. If so
Warning: Press CTRL-D to exit.
perl -e 'print "@ARGV\n"'
[CTRL-D]
/bin/bash: line 1: parallel:: command not found
/bin/bash: line 1: Warning:: command not found
/bin/bash: -c: line 2: syntax error near unexpected token `('
/bin/bash: -c: line 2: ` parallel: Warning: are doing (in which case: YOU ARE AWESOME!) or you forgot'
/bin/bash: -c: line 2: ` Warning: are doing (in which case: YOU ARE AWESOME!) or you forgot'
parallel --trim r echo pre-{}-post ::: ' A '
pre- A-post
@ -367,6 +440,8 @@ pre-A-post
/bin/bash: -c: line 0: syntax error near unexpected token `)'
/bin/bash: -c: line 0: `set a="def-file"; if( { test -d "$a" } ) echo "$a is a dir"'
/bin/bash: -c: line 0: syntax error near unexpected token `)'
/bin/bash: -c: line 0: `set a="fixedlen"; if( { test -d "$a" } ) echo "$a is a dir"'
/bin/bash: -c: line 0: syntax error near unexpected token `)'
/bin/bash: -c: line 0: `set a="num1000000"; if( { test -d "$a" } ) echo "$a is a dir"'
/bin/bash: -c: line 0: syntax error near unexpected token `)'
/bin/bash: -c: line 0: `set a="num128"; if( { test -d "$a" } ) echo "$a is a dir"'
@ -375,8 +450,6 @@ pre-A-post
/bin/bash: -c: line 0: syntax error near unexpected token `)'
/bin/bash: -c: line 0: `set a="num8"; if( { test -d "$a" } ) echo "$a is a dir"'
/bin/bash: -c: line 0: syntax error near unexpected token `)'
/bin/bash: -c: line 0: `set a="num_%header"; if( { test -d "$a" } ) echo "$a is a dir"'
/bin/bash: -c: line 0: syntax error near unexpected token `)'
/bin/bash: -c: line 0: `set a="tsv-file.tsv"; if( { test -d "$a" } ) echo "$a is a dir"'
parallel --tag echo foo-{} ::: A B C
A foo-A
@ -397,7 +470,8 @@ C
echo A
echo B
echo C
parallel -j2 'printf "%s-start\n%s" {} {};sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1
parallel -j2 'printf "%s-start\n%s" {} {};
sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1
4-start
2-start
2-middle
@ -405,7 +479,8 @@ echo C
1-start
1-middle
1-end
parallel -j2 --ungroup 'printf "%s-start\n%s" {} {};sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1
parallel -j2 --ungroup 'printf "%s-start\n%s" {} {};
sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1
4-start
42-start
2-middle
@ -414,7 +489,8 @@ echo C
1-middle
1-end
-middle
parallel -j2 --linebuffer 'printf "%s-start\n%s" {} {};sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1
parallel -j2 --linebuffer 'printf "%s-start\n%s" {} {};
sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1
4-start
2-start
2-middle
@ -422,7 +498,8 @@ echo C
1-start
1-middle
1-end
parallel -j2 -k 'printf "%s-start\n%s" {} {};sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1
parallel -j2 -k 'printf "%s-start\n%s" {} {};
sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1
4-start
2-start
2-middle
@ -566,7 +643,8 @@ Computers / CPU cores / Max jobs to run
Computer:jobs running/jobs completed/%of started jobs/Average seconds to complete
seq 1000 | parallel -j10 --bar '(echo -n {};sleep 0.1)' 2> >(zenity --timeout=15 --progress --auto-kill --auto-close)
seq 1000 | parallel -j10 --bar '(echo -n {};sleep 0.1)' \
2> >(zenity --timeout=15 --progress --auto-kill --auto-close)
BASE64-Message: GtkDialog mapped without a transient parent. This is discouraged.
parallel --joblog /tmp/log exit ::: 1 2 3 0
cat /tmp/log;
@ -621,19 +699,17 @@ Seq Host Starttime JobRuntime Send Receive Exitval Signal Command
9
9
9
9
parallel: This job failed:
echo X; exit X
parallel: Starting no more jobs. Waiting for 1 jobs to finish.
parallel: This job failed:
echo X; exit X
parallel -j2 --halt now,fail=1 echo {}\; exit {} ::: 0 0 1 2 3
9
9
9
parallel: This job failed:
echo X; exit X
parallel -j2 --halt soon,fail=20% echo {}\; exit {} ::: 0 1 2 3 4 5 6 7 8 9
parallel -j2 --halt soon,fail=20% echo {}\; exit {} \
::: 0 1 2 3 4 5 6 7 8 9
9
9
9
@ -652,7 +728,8 @@ echo X; exit X
9
parallel: This job succeeded:
echo X; exit X
parallel -k --retries 3 'echo tried {} >>/tmp/runs; echo completed {}; exit {}' ::: 1 2 0
parallel -k --retries 3 \
'echo tried {} >>/tmp/runs; echo completed {}; exit {}' ::: 1 2 0
cat /tmp/runs
completed 1
completed 2
@ -662,11 +739,16 @@ tried 0
tried 1
tried 1
show_signals() {
perl -e 'for(keys %SIG) { $SIG{$_} = eval "sub { print \"Got $_\\n\"; }";} while(1){sleep 1}'
perl -e 'for(keys %SIG) {
$SIG{$_} = eval "sub { print \"Got $_\\n\"; }";
}
while(1){sleep 1}'
}
export -f show_signals
echo | parallel --termseq TERM,200,TERM,100,TERM,50,KILL,25 -u --timeout 1 show_signals
echo | parallel --termseq INT,200,TERM,100,KILL,25 -u --timeout 1 show_signals
echo | parallel --termseq TERM,200,TERM,100,TERM,50,KILL,25 \
-u --timeout 1 show_signals
echo | parallel --termseq INT,200,TERM,100,KILL,25 \
-u --timeout 1 show_signals
/bin/bash: show_signals: command not found
parallel --load 100% echo load is less than {} job per cpu ::: 1
load is less than 1 job per cpu
@ -704,19 +786,21 @@ more
hosts
parallel -S 4/$SERVER1 echo force {} cpus on server ::: 4
force 4 cpus on server
parallel --hostgroup -S @grp1/$SERVER1 -S @grp2/$SERVER2 echo {} ::: \
run_on_grp1@grp1 run_on_grp2@grp2
parallel --hostgroup -S @grp1/$SERVER1 -S @grp2/$SERVER2 echo {} \
::: run_on_grp1@grp1 run_on_grp2@grp2
run_on_grp1
run_on_grp2
echo This is input_file > input_file
parallel -S $SERVER1 --transferfile {} cat ::: input_file
This is input_file
echo This is input_file > input_file
parallel -S $SERVER1 --transferfile {} --return {}.out cat {} ">"{}.out ::: input_file
parallel -S $SERVER1 --transferfile {} --return {}.out \
cat {} ">"{}.out ::: input_file
cat input_file.out
This is input_file
echo This is input_file > input_file
parallel -S $SERVER1 --transferfile {} --return {}.out --cleanup cat {} ">"{}.out ::: input_file
parallel -S $SERVER1 --transferfile {} --return {}.out --cleanup \
cat {} ">"{}.out ::: input_file
cat input_file.out
This is input_file
echo This is input_file > input_file
@ -724,7 +808,8 @@ This is input_file
cat input_file.out
This is input_file
echo common data > common_file
parallel --basefile common_file -S $SERVER1 cat common_file\; echo {} ::: foo
parallel --basefile common_file -S $SERVER1 \
cat common_file\; echo {} ::: foo
common data
foo
parallel -S $SERVER1 pwd ::: ""
@ -763,42 +848,42 @@ Unknown option: :::
Unknown option: green
env_parallel only works if it is a function. Do the below and restart your shell.
bash: Put this in /mnt/4tb/home/tange/.bashrc: . /usr/local/bin/env_parallel.bash
E.g. by doing: echo '. /usr/local/bin/env_parallel.bash' >> /mnt/4tb/home/tange/.bashrc
bash: Put this in ~/.bashrc: . /usr/local/bin/env_parallel.bash
E.g. by doing: echo '. /usr/local/bin/env_parallel.bash' >> ~/.bashrc
Supports: aliases, functions, variables, arrays
zsh: Put this in /mnt/4tb/home/tange/.zshrc: . /usr/local/bin/env_parallel.zsh
E.g. by doing: echo '. /usr/local/bin/env_parallel.zsh' >> /mnt/4tb/home/tange/.zshenv
zsh: Put this in ~/.zshrc: . /usr/local/bin/env_parallel.zsh
E.g. by doing: echo '. /usr/local/bin/env_parallel.zsh' >> ~/.zshenv
Supports: functions, variables, arrays
fish: Put this in /mnt/4tb/home/tange/.config/fish/config.fish:
fish: Put this in ~/.config/fish/config.fish:
. (which env_parallel.fish)
E.g. by doing:
echo '. (which env_parallel.fish)' >> /mnt/4tb/home/tange/.config/fish/config.fish
echo '. (which env_parallel.fish)' >> ~/.config/fish/config.fish
Supports: aliases, functions, variables, arrays
ksh: Put this in /mnt/4tb/home/tange/.kshrc: source /usr/local/bin/env_parallel.ksh
E.g. by doing: echo 'source /usr/local/bin/env_parallel.ksh' >> /mnt/4tb/home/tange/.kshrc
ksh: Put this in ~/.kshrc: source /usr/local/bin/env_parallel.ksh
E.g. by doing: echo 'source /usr/local/bin/env_parallel.ksh' >> ~/.kshrc
Supports: aliases, functions, variables, arrays
pdksh: Put this in /mnt/4tb/home/tange/.profile: source /usr/local/bin/env_parallel.pdksh
E.g. by doing: echo '. /usr/local/bin/env_parallel.pdksh' >> /mnt/4tb/home/tange/.profile
pdksh: Put this in ~/.profile: source /usr/local/bin/env_parallel.pdksh
E.g. by doing: echo '. /usr/local/bin/env_parallel.pdksh' >> ~/.profile
Supports: aliases, functions, variables, arrays
ash: Put this in /mnt/4tb/home/tange/.profile: . /usr/local/bin/env_parallel.ash
E.g. by doing: echo '. /usr/local/bin/env_parallel.ash' >> /mnt/4tb/home/tange/.profile
ash: Put this in ~/.profile: . /usr/local/bin/env_parallel.ash
E.g. by doing: echo '. /usr/local/bin/env_parallel.ash' >> ~/.profile
Supports: aliases, variables
dash: Put this in /mnt/4tb/home/tange/.profile: . /usr/local/bin/env_parallel.dash
E.g. by doing: echo '. /usr/local/bin/env_parallel.dash' >> /mnt/4tb/home/tange/.profile
dash: Put this in ~/.profile: . /usr/local/bin/env_parallel.dash
E.g. by doing: echo '. /usr/local/bin/env_parallel.dash' >> ~/.profile
Supports: aliases, variables
csh: Put this in /mnt/4tb/home/tange/.cshrc: source /usr/local/bin/env_parallel.csh
E.g. by doing: echo 'source /usr/local/bin/env_parallel.csh' >> /mnt/4tb/home/tange/.cshrc
csh: Put this in ~/.cshrc: source /usr/local/bin/env_parallel.csh
E.g. by doing: echo 'source /usr/local/bin/env_parallel.csh' >> ~/.cshrc
Supports: aliases, variables, arrays with no special chars
tcsh: Put this in /mnt/4tb/home/tange/.tcshrc: source /usr/local/bin/env_parallel.tcsh
E.g. by doing: echo 'source /usr/local/bin/env_parallel.tcsh' >> /mnt/4tb/home/tange/.tcshrc
tcsh: Put this in ~/.tcshrc: source /usr/local/bin/env_parallel.tcsh
E.g. by doing: echo 'source /usr/local/bin/env_parallel.tcsh' >> ~/.tcshrc
Supports: aliases, variables, arrays with no special chars
To install in all shells run:
@ -819,6 +904,7 @@ foo bar baz
in my_func baz
parallel --record-env
cat ~/.parallel/ignored_vars|sort
BASH_FUNC_run_once%%
BASH_FUNC_run_test%%
COLORFGBG
DBUS_SESSION_BUS_ADDRESS
@ -913,6 +999,65 @@ _
parallel --env _ -S $SERVER1 'echo $VAR; my_func2' ::: bar
/bin/bash: my_func2: command not found
NOT='not exported var'
alias myecho=echo
not_ex() {
myecho in not_exported_func $NOT $1
}
env_parallel --env _ -S $SERVER1 'echo $NOT; not_ex' ::: bar
Unknown option: --env
Unknown option: _
Unknown option: -S
Unknown option: parallel@lo
Unknown option: echo $NOT; not_ex
Unknown option: :::
Unknown option: bar
env_parallel only works if it is a function. Do the below and restart your shell.
bash: Put this in ~/.bashrc: . /usr/local/bin/env_parallel.bash
E.g. by doing: echo '. /usr/local/bin/env_parallel.bash' >> ~/.bashrc
Supports: aliases, functions, variables, arrays
zsh: Put this in ~/.zshrc: . /usr/local/bin/env_parallel.zsh
E.g. by doing: echo '. /usr/local/bin/env_parallel.zsh' >> ~/.zshenv
Supports: functions, variables, arrays
fish: Put this in ~/.config/fish/config.fish:
. (which env_parallel.fish)
E.g. by doing:
echo '. (which env_parallel.fish)' >> ~/.config/fish/config.fish
Supports: aliases, functions, variables, arrays
ksh: Put this in ~/.kshrc: source /usr/local/bin/env_parallel.ksh
E.g. by doing: echo 'source /usr/local/bin/env_parallel.ksh' >> ~/.kshrc
Supports: aliases, functions, variables, arrays
pdksh: Put this in ~/.profile: source /usr/local/bin/env_parallel.pdksh
E.g. by doing: echo '. /usr/local/bin/env_parallel.pdksh' >> ~/.profile
Supports: aliases, functions, variables, arrays
ash: Put this in ~/.profile: . /usr/local/bin/env_parallel.ash
E.g. by doing: echo '. /usr/local/bin/env_parallel.ash' >> ~/.profile
Supports: aliases, variables
dash: Put this in ~/.profile: . /usr/local/bin/env_parallel.dash
E.g. by doing: echo '. /usr/local/bin/env_parallel.dash' >> ~/.profile
Supports: aliases, variables
csh: Put this in ~/.cshrc: source /usr/local/bin/env_parallel.csh
E.g. by doing: echo 'source /usr/local/bin/env_parallel.csh' >> ~/.cshrc
Supports: aliases, variables, arrays with no special chars
tcsh: Put this in ~/.tcshrc: source /usr/local/bin/env_parallel.tcsh
E.g. by doing: echo 'source /usr/local/bin/env_parallel.tcsh' >> ~/.tcshrc
Supports: aliases, variables, arrays with no special chars
To install in all shells run:
env_parallel --install
For details: see man env_parallel
parallel -vv --pipepart --block 1M wc :::: num30000
<num30000 perl -e 'while(@ARGV){sysseek(STDIN,shift,0)||die;$left=shift;while($read=sysread(STDIN,$buf,$left>131072?131072:$left)){$left-=$read;syswrite(STDOUT,$buf);}}' 0 0 0 168894 | (wc)
30000 30000 168894
@ -920,9 +1065,11 @@ _
echo in my_func $1 > $1.out
}
export -f my_func3
parallel -vv --workdir ... --nice 17 --env _ --trc {}.out -S $SERVER1 my_func3 {} ::: abc-file
parallel -vv --workdir ... --nice 17 --env _ --trc {}.out \
-S $SERVER1 my_func3 {} ::: abc-file
ssh -l parallel lo -- mkdir -p ./.TMPWORKDIR && rsync --protocol 30 -rlDzR -essh\ -l\ parallel ./abc-file lo:./.TMPWORKDIR;ssh -l parallel lo -- exec perl -e @GNU_Parallel\\\=split/_/,\\\"use_IPC::Open3\\\;_use_MIME::Base64\\\"\\\;eval\\\"@GNU_Parallel\\\"\\\;\\\$chld\\\=\\\$SIG\\\{CHLD\\\}\\\;\\\$SIG\\\{CHLD\\\}\\\=\\\"IGNORE\\\"\\\;my\\\$zip\\\=\\\(grep\\\{-x\\\$_\\\}\\\"/usr/local/bin/bzip2\\\"\\\)\\\[0\\\]\\\|\\\|\\\"bzip2\\\"\\\;open3\\\(\\\$in,\\\$out,\\\"\\\>\\\&STDERR\\\",\\\$zip,\\\"-dc\\\"\\\)\\\;if\\\(my\\\$perlpid\\\=fork\\\)\\\{close\\\$in\\\;\\\$eval\\\=join\\\"\\\",\\\<\\\$out\\\>\\\;close\\\$out\\\;\\\}else\\\{close\\\$out\\\;print\\\$in\\\(decode_base64\\\(join\\\"\\\",@ARGV\\\)\\\)\\\;close\\\$in\\\;exit\\\;\\\}wait\\\;\\\$SIG\\\{CHLD\\\}\\\=\\\$chld\\\;eval\\\$eval\\\; BASE64;_EXIT_status=$?; mkdir -p ./. && rsync --protocol 30 --rsync-path=cd\ ./.TMPWORKDIR/./.\;\ rsync -rlDzR -essh\ -l\ parallel lo:./abc-file.out ./.;ssh -l parallel lo -- rm\ -f\ ./.TMPWORKDIR/abc-file\;\ sh\ -c\ rmdir\\\ ./.TMPWORKDIR/\\\ ./.parallel/tmp/\\\ ./.parallel/\\\ 2\\\>/dev/null\\\;rm\\\ -rf\\\ ./.TMPWORKDIR\\\;;ssh -l parallel lo -- rm\ -f\ ./.TMPWORKDIR/abc-file.out\;\ sh\ -c\ rmdir\\\ ./.TMPWORKDIR/\\\ ./.parallel/tmp/\\\ ./.parallel/\\\ 2\\\>/dev/null\\\;rm\\\ -rf\\\ ./.TMPWORKDIR\\\;;ssh -l parallel lo -- rm -rf .TMPWORKDIR; exit $_EXIT_status;
parallel --sqlandworker csv:////%2Ftmp%2Flog.csv seq ::: 10 ::: 12 13 14
parallel --sqlandworker csv:////%2Ftmp%2Flog.csv \
seq ::: 10 ::: 12 13 14
cat /tmp/log.csv
99
99
@ -952,7 +1099,8 @@ Seq,Host,Starttime,JobRuntime,Send,Receive,Exitval,_Signal,Command,V1,V2,Stdout,
99
99
",
parallel --results outdir --sqlandworker csv:////%2Ftmp%2Flog2.csv seq ::: 10 ::: 12 13 14
parallel --results outdir --sqlandworker csv:////%2Ftmp%2Flog2.csv \
seq ::: 10 ::: 12 13 14
cat /tmp/log2.csv
99
99
@ -1028,7 +1176,16 @@ parallel: Error: echo is not a valid DBURL
999999 999999 9999999
85500 85500 598500
25 25 176
echo /foo, bar/, /baz, qux/, | parallel -kN1 --recend ', ' --pipe echo JOB{#}\;cat\;echo END
cat fixedlen | parallel --pipe --header .{4} --block 3 --recend '' \
'echo start; cat; echo'
start
HHHHAAA
start
HHHHBBB
start
HHHHCCC
echo /foo, bar/, /baz, qux/, | \
parallel -kN1 --recend ', ' --pipe echo JOB{#}\;cat\;echo END
JOB1
/foo, END
JOB2
@ -1038,7 +1195,8 @@ JOB3
JOB4
qux/,
END
echo /foo, bar/, /baz, qux/, | parallel -kN1 --recstart / --pipe echo JOB{#}\;cat\;echo END
echo /foo, bar/, /baz, qux/, | \
parallel -kN1 --recstart / --pipe echo JOB{#}\;cat\;echo END
JOB1
/foo, barEND
JOB2
@ -1048,13 +1206,17 @@ JOB3
JOB4
/,
END
echo /foo, bar/, /baz, qux/, | parallel -kN1 --recend ', ' --recstart / --pipe echo JOB{#}\;cat\;echo END
echo /foo, bar/, /baz, qux/, | \
parallel -kN1 --recend ', ' --recstart / --pipe \
echo JOB{#}\;cat\;echo END
JOB1
/foo, bar/, END
JOB2
/baz, qux/,
END
echo foo,bar,_baz,__qux, | parallel -kN1 --regexp --recend ,_+ --pipe echo JOB{#}\;cat\;echo END
echo foo,bar,_baz,__qux, | \
parallel -kN1 --regexp --recend ,_+ --pipe \
echo JOB{#}\;cat\;echo END
JOB1
foo,bar,_END
JOB2
@ -1062,7 +1224,9 @@ baz,__END
JOB3
qux,
END
echo foo,bar,_baz,__qux, | parallel -kN1 --rrs --regexp --recend ,_+ --pipe echo JOB{#}\;cat\;echo END
echo foo,bar,_baz,__qux, | \
parallel -kN1 --rrs --regexp --recend ,_+ --pipe \
echo JOB{#}\;cat\;echo END
JOB1
foo,barEND
JOB2
@ -1070,52 +1234,11 @@ bazEND
JOB3
qux,
END
cat num_%header | parallel --header '(%.*\n)*' --pipe -N3 echo JOB{#}\;cat
JOB1
%head1
%head2
9
9
9
JOB2
%head1
%head2
9
9
9
JOB3
%head1
%head2
9
9
9
JOB4
%head1
%head2
99
cat num_%header | \
parallel --header '(%.*\n)*' --pipe -N3 echo JOB{#}\;cat
cat: num_%header: No such file or directory
cat num_%header | parallel --header 2 --pipe -N3 echo JOB{#}\;cat
JOB1
%head1
%head2
9
9
9
JOB2
%head1
%head2
9
9
9
JOB3
%head1
%head2
9
9
9
JOB4
%head1
%head2
99
cat: num_%header: No such file or directory
parallel --pipepart -a num1000000 --block 3m wc
999999 999999 9999999
999999 999999 9999999
@ -1259,27 +1382,30 @@ The second is now running in the background
The first finished
The first finished running in the foreground
The second finished running in the foreground
sem --id my_id -u 'echo First started; sleep 10; echo The first finished'
sem --id my_id -u 'echo First started; sleep 10; echo First done'
First started
The first finished
sem --id my_id -u 'echo Second started; sleep 10; echo The second finished'
sem --jobs 3 --id my_id -u 'echo First started; sleep 5; echo The first finished' &&
sem --jobs 3 --id my_id -u 'echo Second started; sleep 6; echo The second finished' &&
sem --jobs 3 --id my_id -u 'echo Third started; sleep 7; echo The third finished' &&
sem --jobs 3 --id my_id -u 'echo Fourth started; sleep 8; echo The fourth finished' &&
First done
sem --id my_id -u 'echo Second started; sleep 10; echo Second done'
Second done
sem --jobs 3 --id my_id -u 'echo Start 1; sleep 5; echo 1 done' &&
sem --jobs 3 --id my_id -u 'echo Start 2; sleep 6; echo 2 done' &&
sem --jobs 3 --id my_id -u 'echo Start 3; sleep 7; echo 3 done' &&
sem --jobs 3 --id my_id -u 'echo Start 4; sleep 8; echo 4 done' &&
sem --wait --id my_id
First started
The first finished
Third started
Fourth started
The third finished
The fourth finished
Start 1
Start 2
1 done
Start 3
2 done
Start 4
3 done
4 done
sem --id foo -u 'echo Slow started; sleep 5; echo Slow ended' &&
sem --id foo --semaphoretimeout 1 'echo Force this running after 1 sec' &&
sem --id foo --semaphoretimeout -2 'echo Give up after 1 sec'
sem --id foo --semaphoretimeout 1 'echo Forced running after 1 sec' &&
sem --id foo --semaphoretimeout -2 'echo Give up after 2 secs'
sem --id foo --wait
Slow started
Force this running after 1 sec
Forced running after 1 sec
Slow ended
parallel: Warning: Semaphore timed out. Stealing the semaphore.
parallel: Warning: Semaphore timed out. Exiting.
@ -1333,7 +1459,8 @@ Web site: http://www.gnu.org/software/parallel
When using programs that use GNU Parallel to process data for publication
please cite as described in 'parallel --citation'.
parallel --minversion VERSION && echo Your version is at least VERSION.
parallel --minversion VERSION && \
echo Your version is at least VERSION.
VERSION
Your version is at least VERSION.
parallel --citation

2
testsuite/wanted-results/sql03
View file

@ -46,7 +46,7 @@ Error:
sql [-hnr] [--table-size] [--db-size] [-p pass-through] [-s string] dburl [command]
### Test dburl :
Error:
: is not defined in /mnt/4tb/home/tange/.sql/aliases /mnt/4tb/home/tange/.dburl.aliases /etc/sql/aliases /usr/local/bin/dburl.aliases /usr/local/bin/dburl.aliases.dist
: is not defined in ~/.sql/aliases ~/.dburl.aliases /etc/sql/aliases /usr/local/bin/dburl.aliases /usr/local/bin/dburl.aliases.dist
sql [-hnr] [--table-size] [--db-size] [-p pass-through] [-s string] dburl [command]
### Test oracle with multiple arguments on the command line