tange/parallel
1
0
Fork 0
mirror of https://git.savannah.gnu.org/git/parallel.git synced 2024-11-22 05:57:54 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

parallel: Fixed bug #50590: Support systems where setpgrp fails.

Browse source
This commit is contained in:
Ole Tange 2017-04-01 02:59:07 +02:00
parent e135fb6fd3
commit 1de0d2be9b
24 changed files with 670 additions and 993 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

36
doc/release_new_version
View file

@ -127,7 +127,7 @@ git diff
YYYYMMDD=`yyyymmdd` YYYYMMDD=`yyyymmdd`
TAG=MyTag TAG=MyTag
echo "Released as $YYYYMMDD ('$TAG')." echo "Released as $YYYYMMDD ('$TAG')." | grep MyTag && (STOP;STOP;STOP)
git commit -a -m "Released as $YYYYMMDD ('$TAG')" git commit -a -m "Released as $YYYYMMDD ('$TAG')"
git tag -s -u 88888888 -m "Released as $YYYYMMDD ('$TAG')" $TAG git tag -s -u 88888888 -m "Released as $YYYYMMDD ('$TAG')" $TAG
@ -193,9 +193,9 @@ file:///home/tange/privat/parallel/doc/release_new_version
from:tange@gnu.org from:tange@gnu.org
to:parallel@gnu.org, bug-parallel@gnu.org to:parallel@gnu.org, bug-parallel@gnu.org
Subject: GNU Parallel 20170322 ('TRAPPIST-1') released <<[stable]>> Subject: GNU Parallel 20170422 ('') released <<[stable]>>
GNU Parallel 20170322 ('TRAPPIST-1') <<[stable]>> has been released. It is available for download at: http://ftpmirror.gnu.org/parallel/ GNU Parallel 20170422 ('') <<[stable]>> has been released. It is available for download at: http://ftpmirror.gnu.org/parallel/
<<No new functionality was introduced so this is a good candidate for a stable release.>> <<No new functionality was introduced so this is a good candidate for a stable release.>>
@ -206,40 +206,18 @@ Haiku of the month:
New in this release: New in this release:
* --rpl can now take arguments by adding '(regexp)' in the replacement string.
* Vote for GNU Parallel's community ads on * Vote for GNU Parallel's community ads on
http://meta.unix.stackexchange.com/a/4356/2972 http://meta.unix.stackexchange.com/a/4356/2972
http://meta.askubuntu.com/a/16750/22307 http://meta.askubuntu.com/a/16750/22307
http://meta.serverfault.com/a/9040/45704 http://meta.serverfault.com/a/9040/45704
* GNU Parallel was cited in: shmlast: An improved implementation of Conditional Reciprocal Best Hits with LAST and Python https://joss.theoj.org/papers/3cde54de7dfbcada7c0fc04f569b36c7 http://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-017-1593-0
* GNU Parallel was cited in: Prediction of relativistic electron flux in the Earth’s outer radiation belt at geostationary orbit by adaptive methods https://link.springer.com/article/10.1134/S0016793217010108 https://128.84.21.199/pdf/1703.09026.pdf
* GNU Parallel was cited in: Proper experimental design requires randomization/balancing of molecular ecology experiments http://biorxiv.org/content/biorxiv/early/2017/02/17/109280.full.pdf
* GNU Parallel was cited in: Higher likelihood of multiple bit-flips due to neutron-induced strikes on logic gates https://arxiv.org/pdf/1612.08239.pdf
* GNU Parallel was cited in: A Monte Carlo Resampling Approach for the Calculation of Hybrid Classical and Quantum Free Energies http://pubs.acs.org/doi/pdfplus/10.1021/acs.jctc.6b00506
* GNU Parallel was cited in: Learning string distance with smoothing for OCR spelling correction https://link.springer.com/article/10.1007/s11042-016-4185-5
* GNU Parallel was cited in: The Breakdown of String Perturbation Theory for Many External Particles https://arxiv.org/pdf/1611.08003.pdf
* GNU Parallel was cited in: Large-scale benchmarking reveals false discoveries and count transformation sensitivity in 16S rRNA gene amplicon data analysis methods used in microbiome studies https://microbiomejournal.biomedcentral.com/articles/10.1186/s40168-016-0208-8
* GNU Parallel was cited in: Operations Research Applications in Nuclear Energy https://dspace.library.colostate.edu/bitstream/handle/11124/170687/Johnson_mines_0052E_11207.pdf?sequence=1&isAllowed=y
* GNU Parallel was cited in: Parallel Computing: GNU Parallel http://www.blopig.com/blog/2017/02/parallel-computing-gnu-parallel/
* Stig Sandbeck Mathisen: Change All The Passwords (Again) http://garf.us/2017/02/stig-sandbeck-mathisen-change-all-the-passwords-again/
* Trabalhando como Paralelismo no Bash usando o GNU Parallel http://www.beyeler.com.br/2017/03/trabalhando-como-paralelismo-no-bash-usando-o-gnu-parallel/
* GNU parallel と SQL*Loader 組み合わせ実行例https://blogs.oracle.com/LetTheSunShineIn/entry/gnu_parallel_%E3%81%A8_sql_loader
https://github.com/lucascbeyeler/zmbackup
http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0174575#references
* <<Possibly http://link.springer.com/chapter/10.1007%2F978-3-319-22053-6_46>> * <<Possibly http://link.springer.com/chapter/10.1007%2F978-3-319-22053-6_46>>

17
packager/obs/home:tange/parallel/.osc/_files
View file

@ -1,13 +1,6 @@
<directory name="parallel" rev="199" srcmd5="d215e332e8e0d5d9886c7045abed4819" vrev="1"> <directory name="parallel" rev="200" srcmd5="ef8094ec2de88060bf1e32d83bfc1a1f" vrev="1">
<entry md5="e6710d334a82f776e34c07bc276abdf0" mtime="1483271525" name="parallel-20161222.tar.bz2" size="1449836" /> <entry md5="0910ea89703cd4fee443f5a2403325bb" mtime="1490135663" name="parallel-20170322.tar.bz2" size="1490362" />
<entry md5="f1d1774a1904db762c292ffa4b542658" mtime="1485019214" name="parallel-20170122.tar.bz2" size="1459419" /> <entry md5="c80f0634d4841dd85133b077b421146e" mtime="1490135663" name="parallel.spec" size="4314" />
<entry md5="98cb0dfdc6fc764110a71ec591b24bb1" mtime="1487719474" name="parallel-20170222.tar.bz2" size="1479077" /> <entry md5="e9080ce05862bd69bc2d9a3fdfc0766b" mtime="1490135664" name="parallel_20170322.dsc" size="556" />
<entry md5="62b6c8aa976dfd7dcc275fb8ad3f44c4" mtime="1487719474" name="parallel.spec" size="4313" /> <entry md5="af85898a49a8311de8bf11718d4ab4b2" mtime="1490135664" name="parallel_20170322.tar.gz" size="1670238" />
<entry md5="3ab86fea7800fb5f7cb87269a8df7aa8" mtime="1483271526" name="parallel_20161122.dsc" size="556" />
<entry md5="5cfd9e7aac93d0d254293b4e6c284873" mtime="1483271527" name="parallel_20161122.tar.gz" size="1576406" />
<entry md5="aca39e894b20e489d439ddaca9ad3281" mtime="1482416160" name="parallel_20161222.tar.gz" size="1603168" />
<entry md5="ec73cfb6f7e961f209e94a30e3e4a57e" mtime="1485019214" name="parallel_20170122.dsc" size="556" />
<entry md5="8790199a8f2e5d07e86ae9cf00c08a4c" mtime="1485019215" name="parallel_20170122.tar.gz" size="1635064" />
<entry md5="58be920398402a5503f92d56b0343662" mtime="1487719475" name="parallel_20170222.dsc" size="556" />
<entry md5="aaef171df3933dbeb3d15fd0dfb90b46" mtime="1487719475" name="parallel_20170222.tar.gz" size="1661902" />
</directory> </directory>

3
packager/obs/home:tange/parallel/.osc/parallel.spec
View file

@ -1,6 +1,7 @@
Summary: Shell tool for executing jobs in parallel Summary: Shell tool for executing jobs in parallel
Name: parallel Name: parallel
Version: 20170222 Version: 20170322
Release: 1.1 Release: 1.1
License: GPL License: GPL
Group: Productivity/File utilities Group: Productivity/File utilities

3
packager/obs/home:tange/parallel/parallel.spec
View file

@ -1,6 +1,7 @@
Summary: Shell tool for executing jobs in parallel Summary: Shell tool for executing jobs in parallel
Name: parallel Name: parallel
Version: 20170222 Version: 20170322
Release: 1.1 Release: 1.1
License: GPL License: GPL
Group: Productivity/File utilities Group: Productivity/File utilities

2
src/niceload
View file

@ -24,7 +24,7 @@
use strict; use strict;
use Getopt::Long; use Getopt::Long;
$Global::progname="niceload"; $Global::progname="niceload";
$Global::version = 20170322; $Global::version = 20170323;
Getopt::Long::Configure("bundling","require_order"); Getopt::Long::Configure("bundling","require_order");
get_options_from_array(\@ARGV) || die_usage(); get_options_from_array(\@ARGV) || die_usage();
if($opt::version) { if($opt::version) {

9
src/parallel
View file

@ -399,6 +399,7 @@ sub cat_partial {
my @start_len = map { my @start_len = map {
if(++$i % 2) { $start = $_; } else { $_-$start } if(++$i % 2) { $start = $_; } else { $_-$start }
} @start_end; } @start_end;
# This can read 7 GB/s using a single core
my $script = spacefree my $script = spacefree
(0, (0,
q{ q{
@ -1361,7 +1362,7 @@ sub check_invalid_option_combinations {
sub init_globals { sub init_globals {
# Defaults: # Defaults:
$Global::version = 20170322; $Global::version = 20170331;
$Global::progname = 'parallel'; $Global::progname = 'parallel';
$Global::infinity = 2**31; $Global::infinity = 2**31;
$Global::debug = 0; $Global::debug = 0;
@ -7454,7 +7455,7 @@ sub sshlogin_wrap {
$pid = fork; $pid = fork;
unless($pid) { unless($pid) {
# Make own process group to be able to kill HUP it later # Make own process group to be able to kill HUP it later
setpgrp; eval { setpgrp };
eval { setpriority(0,0,$nice) }; eval { setpriority(0,0,$nice) };
exec $shell, "-c", ($bashfunc."@ARGV"); exec $shell, "-c", ($bashfunc."@ARGV");
die "exec: $!\n"; die "exec: $!\n";
@ -7899,7 +7900,7 @@ sub start {
eval { eval {
if(not $pid = ::open3($stdin_fh, ">&OUT", ">&ERR", "-")) { if(not $pid = ::open3($stdin_fh, ">&OUT", ">&ERR", "-")) {
# Each child gets its own process group to make it safe to killall # Each child gets its own process group to make it safe to killall
setpgrp(0,0) || ::die_bug("setpgrp failed"); eval{setpgrp(0,0)};
exec($Global::shell,"-c",$command) exec($Global::shell,"-c",$command)
|| ::die_bug("open3-$stdin_fh $command"); || ::die_bug("open3-$stdin_fh $command");
} }
@ -7925,7 +7926,7 @@ sub start {
my $pid; my $pid;
my @setpgrp_wrap = my @setpgrp_wrap =
('perl','-e', ('perl','-e',
"setpgrp\;eval\{setpriority\(0,0,$opt::nice\)\}\;". "eval\{setpgrp\}\;eval\{setpriority\(0,0,$opt::nice\)\}\;".
"exec '$Global::shell', '-c', \@ARGV"); "exec '$Global::shell', '-c', \@ARGV");
# The eval is needed to catch exception from open3 # The eval is needed to catch exception from open3
eval { eval {

8
src/parallel.pod
View file

@ -1245,7 +1245,7 @@ control on the command line (used by GNU B<parallel> internally when
called with B<--sshlogin>). called with B<--sshlogin>).
=item B<--plus> =item B<--plus> (alpha testing)
Activate additional replacement strings: {+/} {+.} {+..} {+...} {..} Activate additional replacement strings: {+/} {+.} {+..} {+...} {..}
{...} {/..} {/...} {##}. The idea being that '{+foo}' matches the opposite of {...} {/..} {/...} {##}. The idea being that '{+foo}' matches the opposite of
@ -1481,9 +1481,9 @@ it to the command.
Only used with B<--pipe>. Only used with B<--pipe>.
=item B<--results> I<name> (beta testing) =item B<--results> I<name>
=item B<--res> I<name> (beta testing) =item B<--res> I<name>
Save the output into files. Save the output into files.
@ -1758,7 +1758,7 @@ impossible to track which input block corresponds to which output.
B<--round-robin> implies B<--pipe>, except if B<--pipepart> is given. B<--round-robin> implies B<--pipe>, except if B<--pipepart> is given.
=item B<--rpl> 'I<tag> I<perl expression>' =item B<--rpl> 'I<tag> I<perl expression>' (alpha testing)
Use I<tag> as a replacement string for I<perl expression>. This makes Use I<tag> as a replacement string for I<perl expression>. This makes
it possible to define your own replacement strings. GNU B<parallel>'s it possible to define your own replacement strings. GNU B<parallel>'s

5
src/parallel_alternatives.pod
View file

@ -863,6 +863,11 @@ Rush (https://github.com/shenwei356/rush) is written in Go and based
on gargs. on gargs.
=head2 DIFFERENCES BETWEEN machma AND GNU Parallel
Todo. Requires Go > 1.6.
=head2 DIFFERENCES BETWEEN ClusterSSH AND GNU Parallel =head2 DIFFERENCES BETWEEN ClusterSSH AND GNU Parallel
ClusterSSH solves a different problem than GNU B<parallel>. ClusterSSH solves a different problem than GNU B<parallel>.

317
src/parallel_tutorial.pod
View file

@ -20,7 +20,8 @@ To run this tutorial you must have the following:
Install the newest version using your package manager (recommended for Install the newest version using your package manager (recommended for
security reasons), the way described in README, or with this command: security reasons), the way described in README, or with this command:
(wget -O - pi.dk/3 || curl pi.dk/3/ || fetch -o - http://pi.dk/3) | bash (wget -O - pi.dk/3 || curl pi.dk/3/ || \
fetch -o - http://pi.dk/3) | bash
This will also install the newest version of the tutorial which you This will also install the newest version of the tutorial which you
can see by running this: can see by running this:
@ -88,7 +89,14 @@ The file can be generated by this command:
The file can be generated by this command: The file can be generated by this command:
(echo %head1; echo %head2; perl -e 'for(1..10){print "$_\n"}') > num_%header (echo %head1; echo %head2; \
perl -e 'for(1..10){print "$_\n"}') > num_%header
=item fixedlen
The file can be generated by this command:
perl -e 'print "HHHHAAABBBCCC"' > fixedlen
=item For remote running: ssh login on 2 servers with no password in =item For remote running: ssh login on 2 servers with no password in
$SERVER1 and $SERVER2 must work. $SERVER1 and $SERVER2 must work.
@ -544,7 +552,8 @@ Output:
B<@arg> contains the input source variables: B<@arg> contains the input source variables:
parallel echo {= 'if($arg[1]==$arg[2]) { skip() }' =} ::: {1..3} ::: {1..3} parallel echo {= 'if($arg[1]==$arg[2]) { skip() }' =} \
::: {1..3} ::: {1..3}
Output: Output:
@ -557,7 +566,8 @@ Output:
If the strings B<{=> and B<=}> cause problems they can be replaced with B<--parens>: If the strings B<{=> and B<=}> cause problems they can be replaced with B<--parens>:
parallel --parens ,,,, echo ',, s:\.[^.]+$::;s:\.[^.]+$::; ,,' ::: foo.tar.gz parallel --parens ,,,, echo ',, s:\.[^.]+$::;s:\.[^.]+$::; ,,' \
::: foo.tar.gz
Output: Output:
@ -565,14 +575,16 @@ Output:
To define a shorthand replacement string use B<--rpl>: To define a shorthand replacement string use B<--rpl>:
parallel --rpl '.. s:\.[^.]+$::;s:\.[^.]+$::;' echo '..' ::: foo.tar.gz parallel --rpl '.. s:\.[^.]+$::;s:\.[^.]+$::;' echo '..' \
::: foo.tar.gz
Output: Same as above. Output: Same as above.
If the shorthand starts with B<{> it can be used as a positional If the shorthand starts with B<{> it can be used as a positional
replacement string, too: replacement string, too:
parallel --rpl '{..} s:\.[^.]+$::;s:\.[^.]+$::;' echo '{..}' ::: foo.tar.gz parallel --rpl '{..} s:\.[^.]+$::;s:\.[^.]+$::;' echo '{..}'
::: foo.tar.gz
Output: Same as above. Output: Same as above.
@ -595,7 +607,8 @@ Here we give the two arguments B<tar.gz> and B<zip> to the replacement
string B<{/I<string1>/I<string2>}> which replaces I<string1> with string B<{/I<string1>/I<string2>}> which replaces I<string1> with
I<string2>: I<string2>:
parallel --rpl '{/(.+?)/(.*?)} s/$$1/$$2/;' echo {/tar.gz/zip} ::: foo.tar.gz parallel --rpl '{/(.+?)/(.*?)} s/$$1/$$2/;' echo {/tar.gz/zip} \
::: foo.tar.gz
Output: Output:
@ -608,14 +621,15 @@ GNU B<parallel>'s 7 replacement strings are implemented as this:
--rpl '{#} $_=$job->seq()' --rpl '{#} $_=$job->seq()'
--rpl '{%} $_=$job->slot()' --rpl '{%} $_=$job->slot()'
--rpl '{/} s:.*/::' --rpl '{/} s:.*/::'
--rpl '{//} $Global::use{"File::Basename"} ||= eval "use File::Basename; 1;"; $_ = dirname($_);' --rpl '{//} $Global::use{"File::Basename"} ||=
eval "use File::Basename; 1;"; $_ = dirname($_);'
--rpl '{/.} s:.*/::; s:\.[^/.]+$::;' --rpl '{/.} s:.*/::; s:\.[^/.]+$::;'
--rpl '{.} s:\.[^/.]+$::' --rpl '{.} s:\.[^/.]+$::'
=head3 Positional replacement strings =head3 Positional replacement strings
With multiple input sources the argument from the individual input With multiple input sources the argument from the individual input
sources can be accessed with B<{>numberB<}>: sources can be accessed with S<< B<{>numberB<}> >>:
parallel echo {1} and {2} ::: A B ::: C D parallel echo {1} and {2} ::: A B ::: C D
@ -638,7 +652,8 @@ Output (the order may be different):
If a position is negative, it will refer to the input source counted If a position is negative, it will refer to the input source counted
from behind: from behind:
parallel echo 1={1} 2={2} 3={3} -1={-1} -2={-2} -3={-3} ::: A B ::: C D ::: E F parallel echo 1={1} 2={2} 3={3} -1={-1} -2={-2} -3={-3} \
::: A B ::: C D ::: E F
Output (the order may be different): Output (the order may be different):
@ -657,7 +672,8 @@ Output (the order may be different):
To use a perl expression as a positional replacement string simply To use a perl expression as a positional replacement string simply
prepend the perl expression with number and space: prepend the perl expression with number and space:
parallel echo '{=2 s:\.[^.]+$::;s:\.[^.]+$::; =} {1}' ::: bar ::: foo.tar.gz parallel echo '{=2 s:\.[^.]+$::;s:\.[^.]+$::; =} {1}' \
::: bar ::: foo.tar.gz
Output: Output:
@ -666,7 +682,8 @@ Output:
If a shorthand defined using B<--rpl> starts with B<{> it can be used as If a shorthand defined using B<--rpl> starts with B<{> it can be used as
a positional replacement string, too: a positional replacement string, too:
parallel --rpl '{..} s:\.[^.]+$::;s:\.[^.]+$::;' echo '{2..} {1}' ::: bar ::: foo.tar.gz parallel --rpl '{..} s:\.[^.]+$::;s:\.[^.]+$::;' echo '{2..} {1}' \
::: bar ::: foo.tar.gz
Output: Same as above. Output: Same as above.
@ -701,32 +718,33 @@ Output (the order may be different):
It is useful with B<--colsep> for processing files with TAB separated values: It is useful with B<--colsep> for processing files with TAB separated values:
parallel --header : --colsep '\t' echo f1={f1} f2={f2} :::: tsv-file.tsv parallel --header : --colsep '\t' echo f1={f1} f2={f2} \
:::: tsv-file.tsv
Output (the order may be different): Output (the order may be different):
f1=A f2=B f1=A f2=B
f1=C f2=D f1=C f2=D
=head3 More pre-defined replacement strings =head3 More pre-defined replacement strings with --plus
B<--plus> adds the replacement strings B<{+/} {+.} {+..} {+...} {..} {...} B<--plus> adds the replacement strings B<{+/} {+.} {+..} {+...} {..} {...}
{/..} {/...} {##}>. The idea being that B<{+foo}> matches the opposite of B<{foo}> {/..} {/...} {##}>. The idea being that B<{+foo}> matches the opposite of B<{foo}>
and B<{}> = B<{+/}>/B<{/}> = B<{.}>.B<{+.}> = B<{+/}>/B<{/.}>.B<{+.}> = B<{..}>.B<{+..}> = and B<{}> = B<{+/}>/B<{/}> = B<{.}>.B<{+.}> = B<{+/}>/B<{/.}>.B<{+.}> = B<{..}>.B<{+..}> =
B<{+/}>/B<{/..}>.B<{+..}> = B<{...}>.B<{+...}> = B<{+/}>/B<{/...}>.B<{+...}>. B<{+/}>/B<{/..}>.B<{+..}> = B<{...}>.B<{+...}> = B<{+/}>/B<{/...}>.B<{+...}>.
parallel --plus echo {} ::: dir/sub/file.ext1.ext2.ext3 parallel --plus echo {} ::: dir/sub/file.ex1.ex2.ex3
parallel --plus echo {+/}/{/} ::: dir/sub/file.ext1.ext2.ext3 parallel --plus echo {+/}/{/} ::: dir/sub/file.ex1.ex2.ex3
parallel --plus echo {.}.{+.} ::: dir/sub/file.ext1.ext2.ext3 parallel --plus echo {.}.{+.} ::: dir/sub/file.ex1.ex2.ex3
parallel --plus echo {+/}/{/.}.{+.} ::: dir/sub/file.ext1.ext2.ext3 parallel --plus echo {+/}/{/.}.{+.} ::: dir/sub/file.ex1.ex2.ex3
parallel --plus echo {..}.{+..} ::: dir/sub/file.ext1.ext2.ext3 parallel --plus echo {..}.{+..} ::: dir/sub/file.ex1.ex2.ex3
parallel --plus echo {+/}/{/..}.{+..} ::: dir/sub/file.ext1.ext2.ext3 parallel --plus echo {+/}/{/..}.{+..} ::: dir/sub/file.ex1.ex2.ex3
parallel --plus echo {...}.{+...} ::: dir/sub/file.ext1.ext2.ext3 parallel --plus echo {...}.{+...} ::: dir/sub/file.ex1.ex2.ex3
parallel --plus echo {+/}/{/...}.{+...} ::: dir/sub/file.ext1.ext2.ext3 parallel --plus echo {+/}/{/...}.{+...} ::: dir/sub/file.ex1.ex2.ex3
Output: Output:
dir/sub/file.ext1.ext2.ext3 dir/sub/file.ex1.ex2.ex3
B<{##}> is simply the number of jobs: B<{##}> is simply the number of jobs:
@ -740,12 +758,59 @@ Output:
Job 4 of 5 Job 4 of 5
Job 5 of 5 Job 5 of 5
=head4 Dynamic replacement strings =head3 Dynamic replacement strings with --plus
B<--plus> also defines B<{:-I<string>}>, B<{:I<number>}>, B<--plus> also defines these dynamic replacement strings:
B<{:I<number1>:I<number2>}>, B<{#I<string>}>, B<{%I<string>}>,
B<{/I<string1>/I<string2>}>, B<{^I<string>}>, B<{^^I<string>}>, =over 19
B<{,I<string>}>, and B<{,,I<string>}>. They are inspired from B<Bash>:
=item B<{:-I<string>}>
Default value is I<string> if the argument is empty.
=item B<{:I<number>}>
Substring from I<number> till end of string.
=item B<{:I<number1>:I<number2>}>
Substring from I<number1> to I<number2>.
=item B<{#I<string>}>
If the argument starts with I<string>, remove it.
=item B<{%I<string>}>
If the argument ends with I<string>, remove it.
=item B<{/I<string1>/I<string2>}>
Replace I<string1> with I<string2>.
=item B<{^I<string>}>
If the argument starts with I<string>, upper case it. I<string> must
be a single letter.
=item B<{^^I<string>}>
If the argument contains I<string>, upper case it. I<string> must be a
single letter.
=item B<{,I<string>}>
If the argument starts with I<string>, lower case it. I<string> must
be a single letter.
=item B<{,,I<string>}>
If the argument contains I<string>, lower case it. I<string> must be a
single letter.
=back
They are inspired from B<Bash>:
unset myvar unset myvar
echo ${myvar:-myval} echo ${myvar:-myval}
@ -956,11 +1021,11 @@ Output (the order may be different):
GNU B<parallel> can also \-quote full lines. Simply run this: GNU B<parallel> can also \-quote full lines. Simply run this:
parallel --shellquote parallel --shellquote
parallel: Warning: Input is read from the terminal. You either know what you Warning: Input is read from the terminal. You either know what you
parallel: Warning: are doing (in which case: YOU ARE AWESOME!) or you forgot Warning: are doing (in which case: YOU ARE AWESOME!) or you forgot
parallel: Warning: ::: or :::: or to pipe data into parallel. If so Warning: ::: or :::: or to pipe data into parallel. If so
parallel: Warning: consider going through the tutorial: man parallel_tutorial Warning: consider going through the tutorial: man parallel_tutorial
parallel: Warning: Press CTRL-D to exit. Warning: Press CTRL-D to exit.
perl -e 'print "@ARGV\n"' perl -e 'print "@ARGV\n"'
[CTRL-D] [CTRL-D]
@ -1078,7 +1143,8 @@ Output (the order may be different):
GNU B<parallel> will postpone the output until the command completes: GNU B<parallel> will postpone the output until the command completes:
parallel -j2 'printf "%s-start\n%s" {} {};sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1 parallel -j2 'printf "%s-start\n%s" {} {};
sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1
Output: Output:
@ -1094,7 +1160,8 @@ Output:
To get the output immediately use B<--ungroup>: To get the output immediately use B<--ungroup>:
parallel -j2 --ungroup 'printf "%s-start\n%s" {} {};sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1 parallel -j2 --ungroup 'printf "%s-start\n%s" {} {};
sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1
Output: Output:
@ -1109,12 +1176,13 @@ Output:
4-end 4-end
B<--ungroup> is fast, but can cause half a line from one job to be mixed B<--ungroup> is fast, but can cause half a line from one job to be mixed
with half a line of another job. That has happend in the second line, with half a line of another job. That has happened in the second line,
where the line '4-middle' is mixed with '2-start'. where the line '4-middle' is mixed with '2-start'.
To avoid this use B<--linebuffer>: To avoid this use B<--linebuffer>:
parallel -j2 --linebuffer 'printf "%s-start\n%s" {} {};sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1 parallel -j2 --linebuffer 'printf "%s-start\n%s" {} {};
sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1
Output: Output:
@ -1130,7 +1198,8 @@ Output:
To force the output in the same order as the arguments use B<--keep-order>/B<-k>: To force the output in the same order as the arguments use B<--keep-order>/B<-k>:
parallel -j2 -k 'printf "%s-start\n%s" {} {};sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1 parallel -j2 -k 'printf "%s-start\n%s" {} {};
sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1
Output: Output:
@ -1364,7 +1433,8 @@ Output:
Computers / CPU cores / Max jobs to run Computers / CPU cores / Max jobs to run
1:local / 2 / 2 1:local / 2 / 2
Computer:jobs running/jobs completed/%of started jobs/Average seconds to complete Computer:jobs running/jobs completed/%of started jobs/
Average seconds to complete
ETA: 2s 0left 1.11avg local:0/9/100%/1.1s ETA: 2s 0left 1.11avg local:0/9/100%/1.1s
GNU B<parallel> can give progress information with B<--progress>: GNU B<parallel> can give progress information with B<--progress>:
@ -1376,7 +1446,8 @@ Output:
Computers / CPU cores / Max jobs to run Computers / CPU cores / Max jobs to run
1:local / 2 / 2 1:local / 2 / 2
Computer:jobs running/jobs completed/%of started jobs/Average seconds to complete Computer:jobs running/jobs completed/%of started jobs/
Average seconds to complete
local:0/9/100%/1.1s local:0/9/100%/1.1s
A progress bar can be shown with B<--bar>: A progress bar can be shown with B<--bar>:
@ -1385,7 +1456,8 @@ A progress bar can be shown with B<--bar>:
And a graphic bar can be shown with B<--bar> and B<zenity>: And a graphic bar can be shown with B<--bar> and B<zenity>:
seq 1000 | parallel -j10 --bar '(echo -n {};sleep 0.1)' 2> >(zenity --progress --auto-kill --auto-close) seq 1000 | parallel -j10 --bar '(echo -n {};sleep 0.1)' \
2> >(zenity --progress --auto-kill --auto-close)
A logfile of the jobs completed so far can be generated with B<--joblog>: A logfile of the jobs completed so far can be generated with B<--joblog>:
@ -1491,11 +1563,10 @@ Output:
0 0
0 0
1 1
parallel: Starting no more jobs. Waiting for 2 jobs to finish. This job failed: parallel: This job failed:
echo 1; exit 1 echo 1; exit 1
parallel: Starting no more jobs. Waiting for 1 jobs to finish.
2 2
parallel: Starting no more jobs. Waiting for 1 jobs to finish. This job failed:
echo 2; exit 2
With B<--halt now,fail=1> the running jobs will be killed immediately: With B<--halt now,fail=1> the running jobs will be killed immediately:
@ -1512,7 +1583,8 @@ Output:
If B<--halt> is given a percentage this percentage of the jobs must fail If B<--halt> is given a percentage this percentage of the jobs must fail
before GNU B<parallel> stops spawning more jobs: before GNU B<parallel> stops spawning more jobs:
parallel -j2 --halt soon,fail=20% echo {}\; exit {} ::: 0 1 2 3 4 5 6 7 8 9 parallel -j2 --halt soon,fail=20% echo {}\; exit {} \
::: 0 1 2 3 4 5 6 7 8 9
Output: Output:
@ -1545,7 +1617,8 @@ Output:
GNU B<parallel> can retry the command with B<--retries>. This is useful if a GNU B<parallel> can retry the command with B<--retries>. This is useful if a
command fails for unknown reasons now and then. command fails for unknown reasons now and then.
parallel -k --retries 3 'echo tried {} >>/tmp/runs; echo completed {}; exit {}' ::: 1 2 0 parallel -k --retries 3 \
'echo tried {} >>/tmp/runs; echo completed {}; exit {}' ::: 1 2 0
cat /tmp/runs cat /tmp/runs
Output: Output:
@ -1573,10 +1646,14 @@ B<SIGTERM>, waiting 50 ms, then a B<SIGKILL>, finally waiting 25 ms
before giving up. It looks like this: before giving up. It looks like this:
show_signals() { show_signals() {
perl -e 'for(keys %SIG) { $SIG{$_} = eval "sub { print \"Got $_\\n\"; }";} while(1){sleep 1}' perl -e 'for(keys %SIG) {
$SIG{$_} = eval "sub { print \"Got $_\\n\"; }";
}
while(1){sleep 1}'
} }
export -f show_signals export -f show_signals
echo | parallel --termseq TERM,200,TERM,100,TERM,50,KILL,25 -u --timeout 1 show_signals echo | parallel --termseq TERM,200,TERM,100,TERM,50,KILL,25 \
-u --timeout 1 show_signals
Output: Output:
@ -1592,14 +1669,16 @@ Output: Same as above.
You can change this to B<SIGINT>, B<SIGTERM>, B<SIGKILL>: You can change this to B<SIGINT>, B<SIGTERM>, B<SIGKILL>:
echo | parallel --termseq INT,200,TERM,100,KILL,25 -u --timeout 1 show_signals echo | parallel --termseq INT,200,TERM,100,KILL,25 \
-u --timeout 1 show_signals
Output: Output:
Got INT Got INT
Got TERM Got TERM
The B<SIGKILL> does not show because it cannot be caught, and thus the child dies. The B<SIGKILL> does not show because it cannot be caught, and thus the
child dies.
=head2 Limiting the resources =head2 Limiting the resources
@ -1738,8 +1817,8 @@ Servers can be put into groups by prepending I<@groupname> to the
server and the group can then be selected by appending I<@groupname> to server and the group can then be selected by appending I<@groupname> to
the argument if using B<--hostgroup>: the argument if using B<--hostgroup>:
parallel --hostgroup -S @grp1/$SERVER1 -S @grp2/$SERVER2 echo {} ::: \ parallel --hostgroup -S @grp1/$SERVER1 -S @grp2/$SERVER2 echo {} \
run_on_grp1@grp1 run_on_grp2@grp2 ::: run_on_grp1@grp1 run_on_grp2@grp2
Output: Output:
@ -1750,8 +1829,8 @@ A host can be in multiple groups by separating the groups with B<+>, and
you can force GNU B<parallel> to limit the groups on which the command you can force GNU B<parallel> to limit the groups on which the command
can be run with B<-S> I<@groupname>: can be run with B<-S> I<@groupname>:
parallel -S @grp1 -S @grp1+grp2/$SERVER1 -S @grp2/SERVER2 echo {} ::: \ parallel -S @grp1 -S @grp1+grp2/$SERVER1 -S @grp2/SERVER2 echo {} \
run_on_grp1 also_grp1 ::: run_on_grp1 also_grp1
Output: Output:
@ -1774,7 +1853,8 @@ If the files are processed into another file, the resulting file can be
transferred back: transferred back:
echo This is input_file > input_file echo This is input_file > input_file
parallel -S $SERVER1 --transferfile {} --return {}.out cat {} ">"{}.out ::: input_file parallel -S $SERVER1 --transferfile {} --return {}.out \
cat {} ">"{}.out ::: input_file
cat input_file.out cat input_file.out
Output: Same as above. Output: Same as above.
@ -1782,7 +1862,8 @@ Output: Same as above.
To remove the input and output file on the remote server use B<--cleanup>: To remove the input and output file on the remote server use B<--cleanup>:
echo This is input_file > input_file echo This is input_file > input_file
parallel -S $SERVER1 --transferfile {} --return {}.out --cleanup cat {} ">"{}.out ::: input_file parallel -S $SERVER1 --transferfile {} --return {}.out --cleanup \
cat {} ">"{}.out ::: input_file
cat input_file.out cat input_file.out
Output: Same as above. Output: Same as above.
@ -1800,7 +1881,8 @@ transfer that using B<--basefile> which will transfer the file before the
first job: first job:
echo common data > common_file echo common data > common_file
parallel --basefile common_file -S $SERVER1 cat common_file\; echo {} ::: foo parallel --basefile common_file -S $SERVER1 \
cat common_file\; echo {} ::: foo
Output: Output:
@ -2024,7 +2106,8 @@ that it is only useful for debugging:
echo in my_func $1 > $1.out echo in my_func $1 > $1.out
} }
export -f my_func3 export -f my_func3
parallel -vv --workdir ... --nice 17 --env _ --trc {}.out -S $SERVER1 my_func3 {} ::: abc-file parallel -vv --workdir ... --nice 17 --env _ --trc {}.out \
-S $SERVER1 my_func3 {} ::: abc-file
Output will be similar to: Output will be similar to:
@ -2074,14 +2157,16 @@ the output each in their own column.
The simplest is to use a CSV file as the storage table: The simplest is to use a CSV file as the storage table:
parallel --sqlandworker csv:////%2Ftmp%2Flog.csv seq ::: 10 ::: 12 13 14 parallel --sqlandworker csv:////%2Ftmp%2Flog.csv \
seq ::: 10 ::: 12 13 14
cat /tmp/log.csv cat /tmp/log.csv
Note how '/' in the path must be written as %2F. Note how '/' in the path must be written as %2F.
Output will be similar to: Output will be similar to:
Seq,Host,Starttime,JobRuntime,Send,Receive,Exitval,_Signal,Command,V1,V2,Stdout,Stderr Seq,Host,Starttime,JobRuntime,Send,Receive,Exitval,_Signal,
Command,V1,V2,Stdout,Stderr
1,:,1458254498.254,0.069,0,9,0,0,"seq 10 12",10,12,"10 1,:,1458254498.254,0.069,0,9,0,0,"seq 10 12",10,12,"10
11 11
12 12
@ -2103,15 +2188,20 @@ format correctly - even with fields containing newlines as above.
If the output is big you may want to put it into files using B<--results>: If the output is big you may want to put it into files using B<--results>:
parallel --results outdir --sqlandworker csv:////%2Ftmp%2Flog2.csv seq ::: 10 ::: 12 13 14 parallel --results outdir --sqlandworker csv:////%2Ftmp%2Flog2.csv \
seq ::: 10 ::: 12 13 14
cat /tmp/log2.csv cat /tmp/log2.csv
Output will be similar to: Output will be similar to:
Seq,Host,Starttime,JobRuntime,Send,Receive,Exitval,_Signal,Command,V1,V2,Stdout,Stderr Seq,Host,Starttime,JobRuntime,Send,Receive,Exitval,_Signal,
1,:,1458824738.287,0.029,0,9,0,0,"seq 10 12",10,12,outdir/1/10/2/12/stdout,outdir/1/10/2/12/stderr Command,V1,V2,Stdout,Stderr
2,:,1458824738.298,0.025,0,12,0,0,"seq 10 13",10,13,outdir/1/10/2/13/stdout,outdir/1/10/2/13/stderr 1,:,1458824738.287,0.029,0,9,0,0,
3,:,1458824738.309,0.026,0,15,0,0,"seq 10 14",10,14,outdir/1/10/2/14/stdout,outdir/1/10/2/14/stderr "seq 10 12",10,12,outdir/1/10/2/12/stdout,outdir/1/10/2/12/stderr
2,:,1458824738.298,0.025,0,12,0,0,
"seq 10 13",10,13,outdir/1/10/2/13/stdout,outdir/1/10/2/13/stderr
3,:,1458824738.309,0.026,0,15,0,0,
"seq 10 14",10,14,outdir/1/10/2/14/stdout,outdir/1/10/2/14/stderr
=head2 DBURL as table =head2 DBURL as table
@ -2144,7 +2234,8 @@ To see the result:
Output will be similar to: Output will be similar to:
Seq|Host|Starttime|JobRuntime|Send|Receive|Exitval|_Signal|Command|V1|V2|Stdout|Stderr Seq|Host|Starttime|JobRuntime|Send|Receive|Exitval|_Signal|
Command|V1|V2|Stdout|Stderr
1|:|1451619638.903|0.806||8|0|0|echo foo baz|foo|baz|foo baz 1|:|1451619638.903|0.806||8|0|0|echo foo baz|foo|baz|foo baz
| |
2|:|1451619639.265|1.54||9|0|0|echo foo quuz|foo|quuz|foo quuz 2|:|1451619639.265|1.54||9|0|0|echo foo quuz|foo|quuz|foo quuz
@ -2287,6 +2378,30 @@ instead of passing full lines to B<wc> it passes full 75 lines at a
time. This of course does not hold for the last job (which in this time. This of course does not hold for the last job (which in this
case got 25 lines). case got 25 lines).
=head2 Fixed length records
Fixed length records can be processed by setting B<--recend ''> and
B<--block I<recordsize>>. A header of size I<n> can be processed with
B<--header .{I<n>}>.
Here is how to process a file with a 4-byte header and a 3-byte record
size:
cat fixedlen | parallel --pipe --header .{4} --block 3 --recend '' \
'echo start; cat; echo'
Output:
start
HHHHAAA
start
HHHHCCC
start
HHHHBBB
It may be more efficient to increase B<--block> to a multiplum of the
record size.
=head2 Record separators =head2 Record separators
GNU B<parallel> uses separators to determine where two records split. GNU B<parallel> uses separators to determine where two records split.
@ -2300,7 +2415,8 @@ string.
Here the B<--recend> is set to B<', '>: Here the B<--recend> is set to B<', '>:
echo /foo, bar/, /baz, qux/, | parallel -kN1 --recend ', ' --pipe echo JOB{#}\;cat\;echo END echo /foo, bar/, /baz, qux/, | \
parallel -kN1 --recend ', ' --pipe echo JOB{#}\;cat\;echo END
Output: Output:
@ -2316,7 +2432,8 @@ Output:
Here the B<--recstart> is set to B</>: Here the B<--recstart> is set to B</>:
echo /foo, bar/, /baz, qux/, | parallel -kN1 --recstart / --pipe echo JOB{#}\;cat\;echo END echo /foo, bar/, /baz, qux/, | \
parallel -kN1 --recstart / --pipe echo JOB{#}\;cat\;echo END
Output: Output:
@ -2332,7 +2449,9 @@ Output:
Here both B<--recend> and B<--recstart> are set: Here both B<--recend> and B<--recstart> are set:
echo /foo, bar/, /baz, qux/, | parallel -kN1 --recend ', ' --recstart / --pipe echo JOB{#}\;cat\;echo END echo /foo, bar/, /baz, qux/, | \
parallel -kN1 --recend ', ' --recstart / --pipe \
echo JOB{#}\;cat\;echo END
Output: Output:
@ -2344,9 +2463,12 @@ Output:
Note the difference between setting one string and setting both strings. Note the difference between setting one string and setting both strings.
With B<--regexp> the B<--recend> and B<--recstart> will be treated as a regular expression: With B<--regexp> the B<--recend> and B<--recstart> will be treated as
a regular expression:
echo foo,bar,_baz,__qux, | parallel -kN1 --regexp --recend ,_+ --pipe echo JOB{#}\;cat\;echo END echo foo,bar,_baz,__qux, | \
parallel -kN1 --regexp --recend ,_+ --pipe \
echo JOB{#}\;cat\;echo END
Output: Output:
@ -2358,9 +2480,12 @@ Output:
qux, qux,
END END
GNU B<parallel> can remove the record separators with B<--remove-rec-sep>/B<--rrs>: GNU B<parallel> can remove the record separators with
B<--remove-rec-sep>/B<--rrs>:
echo foo,bar,_baz,__qux, | parallel -kN1 --rrs --regexp --recend ,_+ --pipe echo JOB{#}\;cat\;echo END echo foo,bar,_baz,__qux, | \
parallel -kN1 --rrs --regexp --recend ,_+ --pipe \
echo JOB{#}\;cat\;echo END
Output: Output:
@ -2378,7 +2503,8 @@ If the input data has a header, the header can be repeated for each
job by matching the header with B<--header>. If headers start with job by matching the header with B<--header>. If headers start with
B<%> you can do this: B<%> you can do this:
cat num_%header | parallel --header '(%.*\n)*' --pipe -N3 echo JOB{#}\;cat cat num_%header | \
parallel --header '(%.*\n)*' --pipe -N3 echo JOB{#}\;cat
Output (the order may be different): Output (the order may be different):
@ -2677,11 +2803,11 @@ would run at a time.
To control which semaphore is used, use To control which semaphore is used, use
B<--semaphorename>/B<--id>. Run this in one terminal: B<--semaphorename>/B<--id>. Run this in one terminal:
sem --id my_id -u 'echo First started; sleep 10; echo The first finished' sem --id my_id -u 'echo First started; sleep 10; echo First done'
and simultaneously this in another terminal: and simultaneously this in another terminal:
sem --id my_id -u 'echo Second started; sleep 10; echo The second finished' sem --id my_id -u 'echo Second started; sleep 10; echo Second done'
Note how the second will only be started when the first has finished. Note how the second will only be started when the first has finished.
@ -2692,25 +2818,25 @@ else will have to wait. A counting semaphore is like having multiple
toilets: Several people can use the toilets, but when they all are in toilets: Several people can use the toilets, but when they all are in
use, everyone else will have to wait. use, everyone else will have to wait.
B<sem> can emulate a counting semaphore. Use B<--jobs> to set the number of B<sem> can emulate a counting semaphore. Use B<--jobs> to set the
toilets like this: number of toilets like this:
sem --jobs 3 --id my_id -u 'echo First started; sleep 5; echo The first finished' && sem --jobs 3 --id my_id -u 'echo Start 1; sleep 5; echo 1 done' &&
sem --jobs 3 --id my_id -u 'echo Second started; sleep 6; echo The second finished' && sem --jobs 3 --id my_id -u 'echo Start 2; sleep 6; echo 2 done' &&
sem --jobs 3 --id my_id -u 'echo Third started; sleep 7; echo The third finished' && sem --jobs 3 --id my_id -u 'echo Start 3; sleep 7; echo 3 done' &&
sem --jobs 3 --id my_id -u 'echo Fourth started; sleep 8; echo The fourth finished' && sem --jobs 3 --id my_id -u 'echo Start 4; sleep 8; echo 4 done' &&
sem --wait --id my_id sem --wait --id my_id
Output: Output:
First started Start 1
Second started Start 2
Third started Start 3
The first finished 1 done
Fourth started Start 4
The second finished 2 done
The third finished 3 done
The fourth finished 4 done
=head2 Timeout =head2 Timeout
@ -2718,17 +2844,17 @@ With B<--semaphoretimeout> you can force running the command anyway after
a period (postive number) or give up (negative number): a period (postive number) or give up (negative number):
sem --id foo -u 'echo Slow started; sleep 5; echo Slow ended' && sem --id foo -u 'echo Slow started; sleep 5; echo Slow ended' &&
sem --id foo --semaphoretimeout 1 'echo Force this running after 1 sec' && sem --id foo --semaphoretimeout 1 'echo Forced running after 1 sec' &&
sem --id foo --semaphoretimeout -2 'echo Give up after 1 sec' sem --id foo --semaphoretimeout -2 'echo Give up after 2 secs'
sem --id foo --wait sem --id foo --wait
Output: Output:
Slow started Slow started
parallel: Warning: Semaphore timed out. Stealing the semaphore. parallel: Warning: Semaphore timed out. Stealing the semaphore.
Force this running after 1 sec Forced running after 1 sec
Slow ended
parallel: Warning: Semaphore timed out. Exiting. parallel: Warning: Semaphore timed out. Exiting.
Slow ended
Note how the 'Give up' was not run. Note how the 'Give up' was not run.
@ -2801,7 +2927,8 @@ Output:
In scripts B<--minversion> can be used to ensure the user has at least In scripts B<--minversion> can be used to ensure the user has at least
this version: this version:
parallel --minversion 20130722 && echo Your version is at least 20130722. parallel --minversion 20130722 && \
echo Your version is at least 20130722.
Output: Output:

2
src/sql
View file

@ -576,7 +576,7 @@ $Global::Initfile && unlink $Global::Initfile;
exit ($err); exit ($err);
sub parse_options { sub parse_options {
$Global::version = 20170322; $Global::version = 20170323;
$Global::progname = 'sql'; $Global::progname = 'sql';
# This must be done first as this may exec myself # This must be done first as this may exec myself

33
testsuite/Start.sh
View file

@ -8,6 +8,14 @@ SHFILE=/tmp/unittest-parallel.sh
MAX_SEC_PER_TEST=900 MAX_SEC_PER_TEST=900
export TIMEOUT=$MAX_SEC_PER_TEST export TIMEOUT=$MAX_SEC_PER_TEST
run_once() {
script=$1
base=`basename "$script" .sh`
diff -Naur wanted-results/"$base" actual-results/"$base" >/dev/null ||
bash "$script" | perl -pe 's:'$HOME':~:g' > actual-results/"$base"
}
export -f run_once
run_test() { run_test() {
script="$1" script="$1"
base=`basename "$script" .sh` base=`basename "$script" .sh`
@ -15,25 +23,22 @@ run_test() {
mkdir -p "$TMPDIR" mkdir -p "$TMPDIR"
# Clean before. May be owned by other users # Clean before. May be owned by other users
sudo rm -f /tmp/*.{tmx,pac,arg,all,log,swp,loa,ssh,df,pip,tmb,chr,tms,par} sudo rm -f /tmp/*.{tmx,pac,arg,all,log,swp,loa,ssh,df,pip,tmb,chr,tms,par}
# Force running once
echo >> actual-results/"$base"
if [ "$TRIES" = "3" ] ; then if [ "$TRIES" = "3" ] ; then
# Try 3 times # Try 2 times
bash "$script" > actual-results/"$base" run_once $script
diff -Naur wanted-results/"$base" actual-results/"$base" >/dev/null || run_once $script
bash "$script" > actual-results/"$base"
diff -Naur wanted-results/"$base" actual-results/"$base" >/dev/null ||
bash "$script" > actual-results/"$base"
diff -Naur wanted-results/"$base" actual-results/"$base" ||
(touch "$script" && echo touch "$script")
else
# Run only once
bash "$script" > actual-results/"$base"
diff -Naur wanted-results/"$base" actual-results/"$base" ||
(touch "$script" && echo touch "$script")
fi fi
run_once $script
diff -Naur wanted-results/"$base" actual-results/"$base" ||
(touch "$script" && echo touch "$script")
# Check if it was cleaned up # Check if it was cleaned up
find /tmp -maxdepth 1 | find /tmp -maxdepth 1 |
perl -ne '/\.(tmx|pac|arg|all|log|swp|loa|ssh|df|pip|tmb|chr|tms|par)$/ and ++$a and print "TMP NOT CLEAN. FOUND: $_".`touch '$script'`;' perl -ne '/\.(tmx|pac|arg|all|log|swp|loa|ssh|df|pip|tmb|chr|tms|par)$/ and
++$a and
print "TMP NOT CLEAN. FOUND: $_".`touch '$script'`;'
# May be owned by other users # May be owned by other users
sudo rm -f /tmp/*.{tmx,pac,arg,all,log,swp,loa,ssh,df,pip,tmb,chr,tms,par} sudo rm -f /tmp/*.{tmx,pac,arg,all,log,swp,loa,ssh,df,pip,tmb,chr,tms,par}
} }

16
testsuite/tests-to-run/parallel-local-10s.sh
View file

@ -83,22 +83,6 @@ par_halt_on_error() {
parallel -j0 -k --tag mytest ::: -2 -1 0 1 2 ::: true false ::: true false parallel -j0 -k --tag mytest ::: -2 -1 0 1 2 ::: true false ::: true false
} }
par_print_before_halt_on_error() {
echo '### What is printed before the jobs are killed'
mytest() {
HALT=$1
(echo 0;
echo 3;
seq 0 7;
echo 0;
echo 8) |
parallel --tag -j10 -kq --halt $HALT perl -e 'sleep $ARGV[0];print STDERR @ARGV,"\n"; '$HALT' > 0 ? exit shift : exit not shift;';
echo exit code $?
}
export -f mytest
parallel -j0 -k --tag mytest ::: -2 -1 0 1 2
}
par_first_print_halt_on_error_1() { par_first_print_halt_on_error_1() {
echo '### Test first dying print --halt-on-error 1'; echo '### Test first dying print --halt-on-error 1';
(echo 0; echo 3; seq 0 7;echo 0; echo 8) | parallel -j10 -kq --halt 1 perl -e 'sleep $ARGV[0];print STDERR @ARGV,"\n"; exit shift'; (echo 0; echo 3; seq 0 7;echo 0; echo 8) | parallel -j10 -kq --halt 1 perl -e 'sleep $ARGV[0];print STDERR @ARGV,"\n"; exit shift';

16
testsuite/tests-to-run/parallel-local-30s.sh
View file

@ -4,22 +4,6 @@
# Each should be taking 30-100s and be possible to run in parallel # Each should be taking 30-100s and be possible to run in parallel
# I.e.: No race conditions, no logins # I.e.: No race conditions, no logins
par_testhalt() {
testhalt() {
echo '### testhalt --halt '$1;
(yes 0 | head -n 10; seq 10) |
stdout parallel -kj4 --halt $1 'sleep {= $_=0.3*($_+1+seq()) =}; exit {}'; echo $?;
(seq 10; yes 0 | head -n 10) |
stdout parallel -kj4 --halt $1 'sleep {= $_=0.3*($_+1+seq()) =}; exit {}'; echo $?;
};
export -f testhalt;
stdout parallel -kj0 testhalt {1},{2}={3} \
::: now soon ::: fail success ::: 0 1 2 30% 70% |
# Remove lines that only show up now and then
perl -ne '/Starting no more jobs./ or print'
}
par_race_condition1() { par_race_condition1() {
echo '### Test race condition on 8 CPU (my laptop)' echo '### Test race condition on 8 CPU (my laptop)'
seq 1 5000000 > /tmp/parallel_race_cond seq 1 5000000 > /tmp/parallel_race_cond

42
testsuite/tests-to-run/parallel-local-race01.sh Executable file
View file

@ -0,0 +1,42 @@
#!/bin/bash
par_print_before_halt_on_error() {
echo '### What is printed before the jobs are killed'
mytest() {
HALT=$1
(echo 0;
echo 3;
seq 0 7;
echo 0;
echo 8) |
parallel --tag -j10 -kq --halt $HALT perl -e 'sleep $ARGV[0];print STDERR @ARGV,"\n"; '$HALT' > 0 ? exit shift : exit not shift;';
echo exit code $?
}
export -f mytest
parallel -j0 -k --tag mytest ::: -2 -1 0 1 2
}
par_testhalt() {
testhalt_false() {
echo '### testhalt --halt '$1;
(yes 0 | head -n 10; seq 10) |
stdout parallel -kj4 --halt $1 \
'sleep {= $_=0.3*($_+1+seq()) =}; exit {}'; echo $?;
}
testhalt_true() {
(seq 10; yes 0 | head -n 10) |
stdout parallel -kj4 --halt $1 \
'sleep {= $_=0.3*($_+1+seq()) =}; exit {}'; echo $?;
};
export -f testhalt_false;
export -f testhalt_true;
stdout parallel -kj0 testhalt_{4} {1},{2}={3} \
::: now soon ::: fail success ::: 0 1 2 30% 70% ::: true false |
# Remove lines that only show up now and then
perl -ne '/Starting no more jobs./ or print'
}
export -f $(compgen -A function | grep par_)
compgen -A function | grep par_ | sort |
parallel --joblog /tmp/jl-`basename $0` -j10 --tag -k '{} 2>&1'

1
testsuite/tests-to-run/parallel-local-ssh4.sh
View file

@ -1,6 +1,7 @@
#!/bin/bash #!/bin/bash
unset run_test unset run_test
unset run_once
# SSH only allowed to localhost/lo # SSH only allowed to localhost/lo
# --retries if ssh dies # --retries if ssh dies

14
testsuite/wanted-results/parallel-install
View file

@ -1,24 +1,24 @@
### Test installation missing pod2* ### Test installation missing pod2*
make[0]: Entering directory '/mnt/4tb/home/tange/privat/parallel' make[0]: Entering directory '~/privat/parallel'
make dist-gzip am__post_remove_distdir='@:' make dist-gzip am__post_remove_distdir='@:'
make[0]: Entering directory '/mnt/4tb/home/tange/privat/parallel' make[0]: Entering directory '~/privat/parallel'
if test -d "parallel-00000000"; then find "parallel-00000000" -type d ! -perm -200 -exec chmod u+w {} ';' && rm -rf "parallel-00000000" || { sleep 5 && rm -rf "parallel-00000000"; }; else :; fi if test -d "parallel-00000000"; then find "parallel-00000000" -type d ! -perm -200 -exec chmod u+w {} ';' && rm -rf "parallel-00000000" || { sleep 5 && rm -rf "parallel-00000000"; }; else :; fi
test -d "parallel-00000000" || mkdir "parallel-00000000" test -d "parallel-00000000" || mkdir "parallel-00000000"
(cd src && make top_distdir=../parallel-00000000 distdir=../parallel-00000000/src \ (cd src && make top_distdir=../parallel-00000000 distdir=../parallel-00000000/src \
am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir) am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir)
make[0]: Entering directory '/mnt/4tb/home/tange/privat/parallel/src' make[0]: Entering directory '~/privat/parallel/src'
make[0]: Leaving directory '/mnt/4tb/home/tange/privat/parallel/src' make[0]: Leaving directory '~/privat/parallel/src'
test -n "" \ test -n "" \
|| find "parallel-00000000" -type d ! -perm -755 \ || find "parallel-00000000" -type d ! -perm -755 \
-exec chmod u+rwx,go+rx {} \; -o \ -exec chmod u+rwx,go+rx {} \; -o \
! -type d ! -perm -444 -links 1 -exec chmod a+r {} \; -o \ ! -type d ! -perm -444 -links 1 -exec chmod a+r {} \; -o \
! -type d ! -perm -400 -exec chmod a+r {} \; -o \ ! -type d ! -perm -400 -exec chmod a+r {} \; -o \
! -type d ! -perm -444 -exec /bin/bash /mnt/4tb/home/tange/privat/parallel/install-sh -c -m a+r {} {} \; \ ! -type d ! -perm -444 -exec /bin/bash ~/privat/parallel/install-sh -c -m a+r {} {} \; \
|| chmod -R a+r "parallel-00000000" || chmod -R a+r "parallel-00000000"
tardir=parallel-00000000 && ${TAR-tar} chof - "$tardir" | GZIP=--best gzip -c >parallel-00000000.tar.gz tardir=parallel-00000000 && ${TAR-tar} chof - "$tardir" | GZIP=--best gzip -c >parallel-00000000.tar.gz
make[0]: Leaving directory '/mnt/4tb/home/tange/privat/parallel' make[0]: Leaving directory '~/privat/parallel'
if test -d "parallel-00000000"; then find "parallel-00000000" -type d ! -perm -200 -exec chmod u+w {} ';' && rm -rf "parallel-00000000" || { sleep 5 && rm -rf "parallel-00000000"; }; else :; fi if test -d "parallel-00000000"; then find "parallel-00000000" -type d ! -perm -200 -exec chmod u+w {} ';' && rm -rf "parallel-00000000" || { sleep 5 && rm -rf "parallel-00000000"; }; else :; fi
make[0]: Leaving directory '/mnt/4tb/home/tange/privat/parallel' make[0]: Leaving directory '~/privat/parallel'
checking for a BSD-compatible install... /usr/bin/install -c checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p checking for a thread-safe mkdir -p... /bin/mkdir -p

10
testsuite/wanted-results/parallel-local-0.3s
View file

@ -297,23 +297,23 @@ bug #45993: --wd ... should also work when run locally
parallel: Error: Cannot change into non-executable dir /bi: No such file or directory parallel: Error: Cannot change into non-executable dir /bi: No such file or directory
parallel --wd /bin 'pwd; echo $OLDPWD; echo' ::: OK parallel --wd /bin 'pwd; echo $OLDPWD; echo' ::: OK
/bin /bin
/mnt/4tb/home/tange/privat/parallel/testsuite ~/privat/parallel/testsuite
OK OK
parallel --wd / 'pwd; echo $OLDPWD; echo' ::: OK parallel --wd / 'pwd; echo $OLDPWD; echo' ::: OK
/ /
/mnt/4tb/home/tange/privat/parallel/testsuite ~/privat/parallel/testsuite
OK OK
parallel --wd /tmp 'pwd; echo $OLDPWD; echo' ::: OK parallel --wd /tmp 'pwd; echo $OLDPWD; echo' ::: OK
/tmp /tmp
/mnt/4tb/home/tange/privat/parallel/testsuite ~/privat/parallel/testsuite
OK OK
parallel --wd ... 'pwd; echo $OLDPWD; echo' ::: OK | perl -pe 's/\d+/0/g' parallel --wd ... 'pwd; echo $OLDPWD; echo' ::: OK | perl -pe 's/\d+/0/g'
/mnt/0tb/home/tange/.parallel/tmp/aspire-0-0 /mnt/0tb/home/tange/.parallel/tmp/aspire-0-0
/mnt/0tb/home/tange/privat/parallel/testsuite /mnt/0tb/home/tange/privat/parallel/testsuite
OK OK
parallel --wd . 'pwd; echo $OLDPWD; echo' ::: OK parallel --wd . 'pwd; echo $OLDPWD; echo' ::: OK
/mnt/4tb/home/tange/privat/parallel/testsuite ~/privat/parallel/testsuite
/mnt/4tb/home/tange/privat/parallel/testsuite ~/privat/parallel/testsuite
OK OK
echo '**' echo '**'
** **

100
testsuite/wanted-results/parallel-local-10s
View file

@ -564,106 +564,6 @@ par_plus_dyn_repl abcaaadef
par_plus_dyn_repl abcaaadef par_plus_dyn_repl abcaaadef
par_plus_dyn_repl abcaaadef par_plus_dyn_repl abcaaadef
par_plus_dyn_repl abcaaadef par_plus_dyn_repl abcaaadef
par_print_before_halt_on_error ### What is printed before the jobs are killed
par_print_before_halt_on_error -2 exit code 0
par_print_before_halt_on_error -2 0 0
par_print_before_halt_on_error -2 parallel: This job succeeded:
par_print_before_halt_on_error -2 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ -2\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 1
par_print_before_halt_on_error -1 exit code 0
par_print_before_halt_on_error -1 0 0
par_print_before_halt_on_error -1 parallel: This job succeeded:
par_print_before_halt_on_error -1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ -1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 1
par_print_before_halt_on_error -1 parallel: Starting no more jobs. Waiting for 8 jobs to finish.
par_print_before_halt_on_error -1 parallel: This job succeeded:
par_print_before_halt_on_error -1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ -1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 2
par_print_before_halt_on_error -1 parallel: Starting no more jobs. Waiting for 7 jobs to finish.
par_print_before_halt_on_error -1 3 3
par_print_before_halt_on_error -1 0 0
par_print_before_halt_on_error -1 1 1
par_print_before_halt_on_error -1 2 2
par_print_before_halt_on_error -1 parallel: This job succeeded:
par_print_before_halt_on_error -1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ -1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 3
par_print_before_halt_on_error -1 parallel: Starting no more jobs. Waiting for 6 jobs to finish.
par_print_before_halt_on_error -1 3 3
par_print_before_halt_on_error -1 parallel: This job succeeded:
par_print_before_halt_on_error -1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ -1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 3
par_print_before_halt_on_error -1 parallel: Starting no more jobs. Waiting for 5 jobs to finish.
par_print_before_halt_on_error -1 4 4
par_print_before_halt_on_error -1 parallel: This job succeeded:
par_print_before_halt_on_error -1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ -1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 4
par_print_before_halt_on_error -1 parallel: Starting no more jobs. Waiting for 4 jobs to finish.
par_print_before_halt_on_error -1 5 5
par_print_before_halt_on_error -1 parallel: This job succeeded:
par_print_before_halt_on_error -1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ -1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 5
par_print_before_halt_on_error -1 parallel: Starting no more jobs. Waiting for 3 jobs to finish.
par_print_before_halt_on_error -1 6 6
par_print_before_halt_on_error -1 parallel: This job succeeded:
par_print_before_halt_on_error -1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ -1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 6
par_print_before_halt_on_error -1 parallel: Starting no more jobs. Waiting for 2 jobs to finish.
par_print_before_halt_on_error -1 7 7
par_print_before_halt_on_error -1 0 0
par_print_before_halt_on_error -1 parallel: This job succeeded:
par_print_before_halt_on_error -1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ -1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 7
par_print_before_halt_on_error -1 parallel: Starting no more jobs. Waiting for 1 jobs to finish.
par_print_before_halt_on_error -1 8 8
par_print_before_halt_on_error -1 parallel: This job succeeded:
par_print_before_halt_on_error -1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ -1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 8
par_print_before_halt_on_error 0 exit code 3
par_print_before_halt_on_error 0 0 0
par_print_before_halt_on_error 0 3 3
par_print_before_halt_on_error 0 0 0
par_print_before_halt_on_error 0 1 1
par_print_before_halt_on_error 0 2 2
par_print_before_halt_on_error 0 3 3
par_print_before_halt_on_error 0 4 4
par_print_before_halt_on_error 0 5 5
par_print_before_halt_on_error 0 6 6
par_print_before_halt_on_error 0 7 7
par_print_before_halt_on_error 0 0 0
par_print_before_halt_on_error 0 8 8
par_print_before_halt_on_error 1 exit code 1
par_print_before_halt_on_error 1 0 0
par_print_before_halt_on_error 1 parallel: This job failed:
par_print_before_halt_on_error 1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ 1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 1
par_print_before_halt_on_error 1 parallel: Starting no more jobs. Waiting for 8 jobs to finish.
par_print_before_halt_on_error 1 parallel: This job failed:
par_print_before_halt_on_error 1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ 1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 2
par_print_before_halt_on_error 1 parallel: Starting no more jobs. Waiting for 7 jobs to finish.
par_print_before_halt_on_error 1 3 3
par_print_before_halt_on_error 1 0 0
par_print_before_halt_on_error 1 1 1
par_print_before_halt_on_error 1 2 2
par_print_before_halt_on_error 1 parallel: This job failed:
par_print_before_halt_on_error 1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ 1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 3
par_print_before_halt_on_error 1 parallel: Starting no more jobs. Waiting for 6 jobs to finish.
par_print_before_halt_on_error 1 3 3
par_print_before_halt_on_error 1 parallel: This job failed:
par_print_before_halt_on_error 1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ 1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 3
par_print_before_halt_on_error 1 parallel: Starting no more jobs. Waiting for 5 jobs to finish.
par_print_before_halt_on_error 1 4 4
par_print_before_halt_on_error 1 parallel: This job failed:
par_print_before_halt_on_error 1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ 1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 4
par_print_before_halt_on_error 1 parallel: Starting no more jobs. Waiting for 4 jobs to finish.
par_print_before_halt_on_error 1 5 5
par_print_before_halt_on_error 1 parallel: This job failed:
par_print_before_halt_on_error 1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ 1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 5
par_print_before_halt_on_error 1 parallel: Starting no more jobs. Waiting for 3 jobs to finish.
par_print_before_halt_on_error 1 6 6
par_print_before_halt_on_error 1 parallel: This job failed:
par_print_before_halt_on_error 1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ 1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 6
par_print_before_halt_on_error 1 parallel: Starting no more jobs. Waiting for 2 jobs to finish.
par_print_before_halt_on_error 1 7 7
par_print_before_halt_on_error 1 0 0
par_print_before_halt_on_error 1 parallel: This job failed:
par_print_before_halt_on_error 1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ 1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 7
par_print_before_halt_on_error 1 parallel: Starting no more jobs. Waiting for 1 jobs to finish.
par_print_before_halt_on_error 1 8 8
par_print_before_halt_on_error 1 parallel: This job failed:
par_print_before_halt_on_error 1 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ 1\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 8
par_print_before_halt_on_error 2 exit code 1
par_print_before_halt_on_error 2 0 0
par_print_before_halt_on_error 2 parallel: This job failed:
par_print_before_halt_on_error 2 perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ 2\ \>\ 0\ \?\ exit\ shift\ :\ exit\ not\ shift\; 1
par_results_compress 0 par_results_compress 0
par_results_compress 1 par_results_compress 1
par_results_csv bug #: --results csv par_results_csv bug #: --results csv

472
testsuite/wanted-results/parallel-local-30s
View file

@ -20,478 +20,6 @@ par_race_condition1 7
par_race_condition1 8 par_race_condition1 8
par_race_condition1 9 par_race_condition1 9
par_race_condition1 10 par_race_condition1 10
par_testhalt ### testhalt --halt now,fail=0
par_testhalt parallel: This job failed:
par_testhalt sleep 3.9; exit 1
par_testhalt 0
par_testhalt parallel: This job failed:
par_testhalt sleep 0.9; exit 1
par_testhalt 0
par_testhalt ### testhalt --halt now,fail=1
par_testhalt parallel: This job failed:
par_testhalt sleep 3.9; exit 1
par_testhalt 1
par_testhalt parallel: This job failed:
par_testhalt sleep 0.9; exit 1
par_testhalt 1
par_testhalt ### testhalt --halt now,fail=2
par_testhalt parallel: This job failed:
par_testhalt sleep 3.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 4.5; exit 2
par_testhalt 2
par_testhalt parallel: This job failed:
par_testhalt sleep 0.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 1.5; exit 2
par_testhalt 2
par_testhalt ### testhalt --halt now,fail=30%
par_testhalt parallel: This job failed:
par_testhalt sleep 3.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 4.5; exit 2
par_testhalt parallel: This job failed:
par_testhalt sleep 5.1; exit 3
par_testhalt parallel: This job failed:
par_testhalt sleep 5.7; exit 4
par_testhalt parallel: This job failed:
par_testhalt sleep 6.3; exit 5
par_testhalt parallel: This job failed:
par_testhalt sleep 6.9; exit 6
par_testhalt 30
par_testhalt parallel: This job failed:
par_testhalt sleep 0.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 1.5; exit 2
par_testhalt parallel: This job failed:
par_testhalt sleep 2.1; exit 3
par_testhalt parallel: This job failed:
par_testhalt sleep 2.7; exit 4
par_testhalt parallel: This job failed:
par_testhalt sleep 3.3; exit 5
par_testhalt parallel: This job failed:
par_testhalt sleep 3.9; exit 6
par_testhalt 30
par_testhalt ### testhalt --halt now,fail=70%
par_testhalt parallel: This job failed:
par_testhalt sleep 3.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 4.5; exit 2
par_testhalt parallel: This job failed:
par_testhalt sleep 5.1; exit 3
par_testhalt parallel: This job failed:
par_testhalt sleep 5.7; exit 4
par_testhalt parallel: This job failed:
par_testhalt sleep 6.3; exit 5
par_testhalt parallel: This job failed:
par_testhalt sleep 6.9; exit 6
par_testhalt parallel: This job failed:
par_testhalt sleep 7.5; exit 7
par_testhalt parallel: This job failed:
par_testhalt sleep 8.1; exit 8
par_testhalt parallel: This job failed:
par_testhalt sleep 8.7; exit 9
par_testhalt parallel: This job failed:
par_testhalt sleep 9.3; exit 10
par_testhalt 50
par_testhalt parallel: This job failed:
par_testhalt sleep 0.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 1.5; exit 2
par_testhalt parallel: This job failed:
par_testhalt sleep 2.1; exit 3
par_testhalt parallel: This job failed:
par_testhalt sleep 2.7; exit 4
par_testhalt parallel: This job failed:
par_testhalt sleep 3.3; exit 5
par_testhalt parallel: This job failed:
par_testhalt sleep 3.9; exit 6
par_testhalt parallel: This job failed:
par_testhalt sleep 4.5; exit 7
par_testhalt parallel: This job failed:
par_testhalt sleep 5.1; exit 8
par_testhalt parallel: This job failed:
par_testhalt sleep 5.7; exit 9
par_testhalt parallel: This job failed:
par_testhalt sleep 6.3; exit 10
par_testhalt 50
par_testhalt ### testhalt --halt now,success=0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.6; exit 0
par_testhalt 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.6; exit 0
par_testhalt 0
par_testhalt ### testhalt --halt now,success=1
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.6; exit 0
par_testhalt 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.6; exit 0
par_testhalt 0
par_testhalt ### testhalt --halt now,success=2
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.9; exit 0
par_testhalt 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.9; exit 0
par_testhalt 0
par_testhalt ### testhalt --halt now,success=30%
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.9; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.2; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.5; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.8; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 2.1; exit 0
par_testhalt 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.9; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.2; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.5; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.8; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 5.1; exit 0
par_testhalt 0
par_testhalt ### testhalt --halt now,success=70%
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.9; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.2; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.5; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.8; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 2.1; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 2.4; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 2.7; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.3; exit 0
par_testhalt 50
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.9; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.2; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.5; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.8; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 5.1; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 5.4; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 5.7; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 6.3; exit 0
par_testhalt 50
par_testhalt ### testhalt --halt soon,fail=0
par_testhalt parallel: This job failed:
par_testhalt sleep 3.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 4.5; exit 2
par_testhalt parallel: This job failed:
par_testhalt sleep 5.1; exit 3
par_testhalt parallel: This job failed:
par_testhalt sleep 5.7; exit 4
par_testhalt 0
par_testhalt parallel: This job failed:
par_testhalt sleep 0.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 1.5; exit 2
par_testhalt parallel: This job failed:
par_testhalt sleep 2.1; exit 3
par_testhalt parallel: This job failed:
par_testhalt sleep 2.7; exit 4
par_testhalt 0
par_testhalt ### testhalt --halt soon,fail=1
par_testhalt parallel: This job failed:
par_testhalt sleep 3.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 4.5; exit 2
par_testhalt parallel: This job failed:
par_testhalt sleep 5.1; exit 3
par_testhalt parallel: This job failed:
par_testhalt sleep 5.7; exit 4
par_testhalt 1
par_testhalt parallel: This job failed:
par_testhalt sleep 0.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 1.5; exit 2
par_testhalt parallel: This job failed:
par_testhalt sleep 2.1; exit 3
par_testhalt parallel: This job failed:
par_testhalt sleep 2.7; exit 4
par_testhalt 1
par_testhalt ### testhalt --halt soon,fail=2
par_testhalt parallel: This job failed:
par_testhalt sleep 3.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 4.5; exit 2
par_testhalt parallel: This job failed:
par_testhalt sleep 5.1; exit 3
par_testhalt parallel: This job failed:
par_testhalt sleep 5.7; exit 4
par_testhalt parallel: This job failed:
par_testhalt sleep 6.3; exit 5
par_testhalt 2
par_testhalt parallel: This job failed:
par_testhalt sleep 0.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 1.5; exit 2
par_testhalt parallel: This job failed:
par_testhalt sleep 2.1; exit 3
par_testhalt parallel: This job failed:
par_testhalt sleep 2.7; exit 4
par_testhalt parallel: This job failed:
par_testhalt sleep 3.3; exit 5
par_testhalt 2
par_testhalt ### testhalt --halt soon,fail=30%
par_testhalt parallel: This job failed:
par_testhalt sleep 3.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 4.5; exit 2
par_testhalt parallel: This job failed:
par_testhalt sleep 5.1; exit 3
par_testhalt parallel: This job failed:
par_testhalt sleep 5.7; exit 4
par_testhalt parallel: This job failed:
par_testhalt sleep 6.3; exit 5
par_testhalt parallel: This job failed:
par_testhalt sleep 6.9; exit 6
par_testhalt parallel: This job failed:
par_testhalt sleep 7.5; exit 7
par_testhalt parallel: This job failed:
par_testhalt sleep 8.1; exit 8
par_testhalt parallel: This job failed:
par_testhalt sleep 8.7; exit 9
par_testhalt 30
par_testhalt parallel: This job failed:
par_testhalt sleep 0.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 1.5; exit 2
par_testhalt parallel: This job failed:
par_testhalt sleep 2.1; exit 3
par_testhalt parallel: This job failed:
par_testhalt sleep 2.7; exit 4
par_testhalt parallel: This job failed:
par_testhalt sleep 3.3; exit 5
par_testhalt parallel: This job failed:
par_testhalt sleep 3.9; exit 6
par_testhalt parallel: This job failed:
par_testhalt sleep 4.5; exit 7
par_testhalt parallel: This job failed:
par_testhalt sleep 5.1; exit 8
par_testhalt parallel: This job failed:
par_testhalt sleep 5.7; exit 9
par_testhalt 30
par_testhalt ### testhalt --halt soon,fail=70%
par_testhalt parallel: This job failed:
par_testhalt sleep 3.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 4.5; exit 2
par_testhalt parallel: This job failed:
par_testhalt sleep 5.1; exit 3
par_testhalt parallel: This job failed:
par_testhalt sleep 5.7; exit 4
par_testhalt parallel: This job failed:
par_testhalt sleep 6.3; exit 5
par_testhalt parallel: This job failed:
par_testhalt sleep 6.9; exit 6
par_testhalt parallel: This job failed:
par_testhalt sleep 7.5; exit 7
par_testhalt parallel: This job failed:
par_testhalt sleep 8.1; exit 8
par_testhalt parallel: This job failed:
par_testhalt sleep 8.7; exit 9
par_testhalt parallel: This job failed:
par_testhalt sleep 9.3; exit 10
par_testhalt 50
par_testhalt parallel: This job failed:
par_testhalt sleep 0.9; exit 1
par_testhalt parallel: This job failed:
par_testhalt sleep 1.5; exit 2
par_testhalt parallel: This job failed:
par_testhalt sleep 2.1; exit 3
par_testhalt parallel: This job failed:
par_testhalt sleep 2.7; exit 4
par_testhalt parallel: This job failed:
par_testhalt sleep 3.3; exit 5
par_testhalt parallel: This job failed:
par_testhalt sleep 3.9; exit 6
par_testhalt parallel: This job failed:
par_testhalt sleep 4.5; exit 7
par_testhalt parallel: This job failed:
par_testhalt sleep 5.1; exit 8
par_testhalt parallel: This job failed:
par_testhalt sleep 5.7; exit 9
par_testhalt parallel: This job failed:
par_testhalt sleep 6.3; exit 10
par_testhalt 50
par_testhalt ### testhalt --halt soon,success=0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.9; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.2; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.5; exit 0
par_testhalt 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.9; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.2; exit 0
par_testhalt 0
par_testhalt ### testhalt --halt soon,success=1
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.9; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.2; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.5; exit 0
par_testhalt 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.9; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.2; exit 0
par_testhalt 0
par_testhalt ### testhalt --halt soon,success=2
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.9; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.2; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.5; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.8; exit 0
par_testhalt 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.9; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.2; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.5; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.8; exit 0
par_testhalt 0
par_testhalt ### testhalt --halt soon,success=30%
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.9; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.2; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.5; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.8; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 2.1; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 2.4; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 2.7; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3; exit 0
par_testhalt 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.9; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.2; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.5; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.8; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 5.1; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 5.4; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 5.7; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 6; exit 0
par_testhalt 0
par_testhalt ### testhalt --halt soon,success=70%
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 0.9; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.2; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.5; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 1.8; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 2.1; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 2.4; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 2.7; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.3; exit 0
par_testhalt 50
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 3.9; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.2; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.5; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 4.8; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 5.1; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 5.4; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 5.7; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 6; exit 0
par_testhalt parallel: This job succeeded:
par_testhalt sleep 6.3; exit 0
par_testhalt 50
par_tmp_full ### Test --tmpdir running full. bug #40733 was caused by this par_tmp_full ### Test --tmpdir running full. bug #40733 was caused by this
par_tmp_full parallel: Error: Output is incomplete. Cannot append to buffer file in /tmp/shm/parallel. Is the disk full? par_tmp_full parallel: Error: Output is incomplete. Cannot append to buffer file in /tmp/shm/parallel. Is the disk full?
par_tmp_full parallel: Error: Change $TMPDIR with --tmpdir or use --compress. par_tmp_full parallel: Error: Change $TMPDIR with --tmpdir or use --compress.

10
testsuite/wanted-results/parallel-local-ssh1
View file

@ -20,22 +20,22 @@ echo '### bug #41805: Idea: propagate --env for parallel --number-of-cores'
** test_zsh ** test_zsh
FOO=test_zsh parallel --env FOO,HOME -S zsh@lo -N0 env ::: "" |sort|egrep 'FOO|^HOME' FOO=test_zsh parallel --env FOO,HOME -S zsh@lo -N0 env ::: "" |sort|egrep 'FOO|^HOME'
FOO=test_zsh FOO=test_zsh
HOME=/mnt/4tb/home/tange HOME=~
echo '** test_zsh_filter' echo '** test_zsh_filter'
** test_zsh_filter ** test_zsh_filter
FOO=test_zsh_filter parallel --filter-hosts --env FOO,HOME -S zsh@lo -N0 env ::: "" |sort|egrep 'FOO|^HOME' FOO=test_zsh_filter parallel --filter-hosts --env FOO,HOME -S zsh@lo -N0 env ::: "" |sort|egrep 'FOO|^HOME'
FOO=test_zsh_filter FOO=test_zsh_filter
HOME=/mnt/4tb/home/tange HOME=~
echo '** test_csh' echo '** test_csh'
** test_csh ** test_csh
FOO=test_csh parallel --env FOO,HOME -S csh@lo -N0 env ::: "" |sort|egrep 'FOO|^HOME' FOO=test_csh parallel --env FOO,HOME -S csh@lo -N0 env ::: "" |sort|egrep 'FOO|^HOME'
FOO=test_csh FOO=test_csh
HOME=/mnt/4tb/home/tange HOME=~
echo '** test_csh_filter' echo '** test_csh_filter'
** test_csh_filter ** test_csh_filter
FOO=test_csh_filter parallel --filter-hosts --env FOO,HOME -S csh@lo -N0 env ::: "" |sort|egrep 'FOO|^HOME' FOO=test_csh_filter parallel --filter-hosts --env FOO,HOME -S csh@lo -N0 env ::: "" |sort|egrep 'FOO|^HOME'
FOO=test_csh_filter FOO=test_csh_filter
HOME=/mnt/4tb/home/tange HOME=~
echo '** bug #41805 done' echo '** bug #41805 done'
** bug #41805 done ** bug #41805 done
echo '### Deal with long command lines on remote servers' echo '### Deal with long command lines on remote servers'
@ -64,7 +64,7 @@ echo '### bug #40001: --joblog and --nonall seem not to work together:'
echo '### bug #40132: FreeBSD: --workdir . gives warning if . == $HOME' echo '### bug #40132: FreeBSD: --workdir . gives warning if . == $HOME'
### bug #40132: FreeBSD: --workdir . gives warning if . == $HOME ### bug #40132: FreeBSD: --workdir . gives warning if . == $HOME
cd && parallel --workdir . -S lo pwd ::: "" cd && parallel --workdir . -S lo pwd ::: ""
/mnt/4tb/home/tange ~
echo '### use function as $PARALLEL_SSH' echo '### use function as $PARALLEL_SSH'
### use function as $PARALLEL_SSH ### use function as $PARALLEL_SSH
foossh() { echo "FOOSSH" >&2; ssh "$@"; }; export -f foossh; PARALLEL_SSH=foossh parallel -S 1/lo echo ::: 'Run through FOOSSH?' foossh() { echo "FOOSSH" >&2; ssh "$@"; }; export -f foossh; PARALLEL_SSH=foossh parallel -S 1/lo echo ::: 'Run through FOOSSH?'

2
testsuite/wanted-results/parallel-local22
View file

@ -88,7 +88,7 @@ echo '### bug #42902: profiles containing arguments with space'
echo /bin/bash\=/bin/bash echo /bin/bash\=/bin/bash
/bin/bash=/bin/bash /bin/bash=/bin/bash
PARALLEL="--rpl 'FULLPATH chomp(\$_=\"/bin/bash=\".\`readlink -f \$_\`);' perl -e \'print \\\"@ARGV\\\n\\\"\' " parallel With script in \\\$PARALLEL FULLPATH ::: . PARALLEL="--rpl 'FULLPATH chomp(\$_=\"/bin/bash=\".\`readlink -f \$_\`);' perl -e \'print \\\"@ARGV\\\n\\\"\' " parallel With script in \\\$PARALLEL FULLPATH ::: .
With script in $PARALLEL /bin/bash=/mnt/4tb/home/tange/privat/parallel/testsuite With script in $PARALLEL /bin/bash=~/privat/parallel/testsuite
echo '### bug #42892: parallel -a nonexiting --pipepart' echo '### bug #42892: parallel -a nonexiting --pipepart'
### bug #42892: parallel -a nonexiting --pipepart ### bug #42892: parallel -a nonexiting --pipepart
parallel --pipepart -a nonexisting wc parallel --pipepart -a nonexisting wc

12
testsuite/wanted-results/parallel-local4
View file

@ -1,15 +1,15 @@
bug #46120: Suspend should suspend (at least local) children bug #46120: Suspend should suspend (at least local) children
it should burn 1.9 CPU seconds, but no more than that it should burn 1.9 CPU seconds, but no more than that
The 5 second sleep will make it be killed by timeout when it fgs The 5 second sleep will make it be killed by timeout when it fgs
1024 SHA256:SXgag2Z2L91JsrT5WjNBcARD1EpyCCj4JctVJ6Zpzm0 /mnt/4tb/home/tange/.ssh/id_dsa (DSA) 1024 SHA256:SXgag2Z2L91JsrT5WjNBcARD1EpyCCj4JctVJ6Zpzm0 ~/.ssh/id_dsa (DSA)
8192 SHA256:lYn04AefJq/5r0e4FftceviJ7JVnq9NGKY3CW9XMpO8 /mnt/4tb/home/tange/.ssh/id_rsa (RSA) 8192 SHA256:lYn04AefJq/5r0e4FftceviJ7JVnq9NGKY3CW9XMpO8 ~/.ssh/id_rsa (RSA)
4096 SHA256:jUQ9ysfprs7UOckttjjVb+j3qikUmKWDEWC+eEJkbDQ /mnt/4tb/home/tange/.ssh/id_rsa_openindiana (RSA) 4096 SHA256:jUQ9ysfprs7UOckttjjVb+j3qikUmKWDEWC+eEJkbDQ ~/.ssh/id_rsa_openindiana (RSA)
stdout /usr/bin/time -f CPUTIME=%U parallel --timeout 5 burnP6 ::: 1 | grep -q CPUTIME=1 stdout /usr/bin/time -f CPUTIME=%U parallel --timeout 5 burnP6 ::: 1 | grep -q CPUTIME=1
Zero=OK 0 Zero=OK 0
1024 SHA256:SXgag2Z2L91JsrT5WjNBcARD1EpyCCj4JctVJ6Zpzm0 /mnt/4tb/home/tange/.ssh/id_dsa (DSA) 1024 SHA256:SXgag2Z2L91JsrT5WjNBcARD1EpyCCj4JctVJ6Zpzm0 ~/.ssh/id_dsa (DSA)
8192 SHA256:lYn04AefJq/5r0e4FftceviJ7JVnq9NGKY3CW9XMpO8 /mnt/4tb/home/tange/.ssh/id_rsa (RSA) 8192 SHA256:lYn04AefJq/5r0e4FftceviJ7JVnq9NGKY3CW9XMpO8 ~/.ssh/id_rsa (RSA)
4096 SHA256:jUQ9ysfprs7UOckttjjVb+j3qikUmKWDEWC+eEJkbDQ /mnt/4tb/home/tange/.ssh/id_rsa_openindiana (RSA) 4096 SHA256:jUQ9ysfprs7UOckttjjVb+j3qikUmKWDEWC+eEJkbDQ ~/.ssh/id_rsa_openindiana (RSA)
echo 1 | stdout /usr/bin/time -f CPUTIME=%U parallel --timeout 5 burnP6 | grep -q CPUTIME=1 echo 1 | stdout /usr/bin/time -f CPUTIME=%U parallel --timeout 5 burnP6 | grep -q CPUTIME=1
Zero=OK 0 Zero=OK 0

419
testsuite/wanted-results/parallel-tutorial
View file

@ -1,4 +1,8 @@
### test parallel_tutorial ### test parallel_tutorial
fetch -o - http://pi.dk/3) | bash
/bin/bash: -c: line 0: syntax error near unexpected token `)'
/bin/bash: -c: line 0: ` fetch -o - http://pi.dk/3) | bash'
parallel -k echo ::: A B C > abc-file parallel -k echo ::: A B C > abc-file
parallel -k echo ::: D E F > def-file parallel -k echo ::: D E F > def-file
perl -e 'printf "A\0B\0C\0"' > abc0-file perl -e 'printf "A\0B\0C\0"' > abc0-file
@ -8,7 +12,9 @@
perl -e 'for(1..128){print "$_\n"}' > num128 perl -e 'for(1..128){print "$_\n"}' > num128
perl -e 'for(1..30000){print "$_\n"}' > num30000 perl -e 'for(1..30000){print "$_\n"}' > num30000
perl -e 'for(1..1000000){print "$_\n"}' > num1000000 perl -e 'for(1..1000000){print "$_\n"}' > num1000000
(echo %head1; echo %head2; perl -e 'for(1..10){print "$_\n"}') > num_%header (echo %head1; echo %head2; \
sleep .3
/bin/bash: -c: line 2: syntax error: unexpected end of file
sleep .3 sleep .3
sleep .3 sleep .3
sleep .3 sleep .3
@ -18,8 +24,10 @@ sleep .3
sleep .3 sleep .3
sleep .3 sleep .3
sleep .3 sleep .3
sleep .3 perl -e 'for(1..10){print "$_\n"}') > num_%header
/bin/bash: -c: line 0: syntax error near unexpected token `)'
/bin/bash: -c: line 0: ` perl -e 'for(1..10){print "$_\n"}') > num_%header'
perl -e 'print "HHHHAAABBBCCC"' > fixedlen
parallel echo ::: A B C parallel echo ::: A B C
A A
B B
@ -154,14 +162,14 @@ abc-file
abc0-file abc0-file
abc_-file abc_-file
def-file def-file
fixedlen
num1000000 num1000000
num128 num128
num30000 num30000
num8 num8
num_%header
tsv-file.tsv tsv-file.tsv
foo foo
/mnt/4tb/home/tange/privat/parallel/testsuite/tmp ~/privat/parallel/testsuite/tmp
my_func() { my_func() {
echo in my_func $1 echo in my_func $1
} }
@ -223,33 +231,43 @@ Job 5 of 5
9 9
9 9
9 9
parallel echo {= 'if($arg[1]==$arg[2]) { skip() }' =} ::: {1..3} ::: {1..3} parallel echo {= 'if($arg[1]==$arg[2]) { skip() }' =} \
::: {1..3} ::: {1..3}
1 2 1 2
1 3 1 3
2 1 2 1
2 3 2 3
3 1 3 1
3 2 3 2
parallel --parens ,,,, echo ',, s:\.[^.]+$::;s:\.[^.]+$::; ,,' ::: foo.tar.gz parallel --parens ,,,, echo ',, s:\.[^.]+$::;s:\.[^.]+$::; ,,' \
::: foo.tar.gz
foo foo
parallel --rpl '.. s:\.[^.]+$::;s:\.[^.]+$::;' echo '..' ::: foo.tar.gz parallel --rpl '.. s:\.[^.]+$::;s:\.[^.]+$::;' echo '..' \
foo ::: foo.tar.gz
parallel --rpl '{..} s:\.[^.]+$::;s:\.[^.]+$::;' echo '{..}' ::: foo.tar.gz
foo foo
parallel --rpl '{..} s:\.[^.]+$::;s:\.[^.]+$::;' echo '{..}'
::: foo.tar.gz
/bin/bash: line 1: :::: command not found
parallel --rpl '{%(.+?)} s/$$1$//;' echo {%.tar.gz}.zip ::: foo.tar.gz
foo.zip
parallel --rpl '{/(.+?)/(.*?)} s/$$1/$$2/;' echo {/tar.gz/zip} \
::: foo.tar.gz
foo.zip
--rpl '{} ' --rpl '{} '
--rpl '{#} $_=$job->seq()' --rpl '{#} $_=$job->seq()'
--rpl '{%} $_=$job->slot()' --rpl '{%} $_=$job->slot()'
--rpl '{/} s:.*/::' --rpl '{/} s:.*/::'
--rpl '{//} $Global::use{"File::Basename"} ||= eval "use File::Basename; 1;"; $_ = dirname($_);' --rpl '{//} $Global::use{"File::Basename"} ||=
eval "use File::Basename; 1;"; $_ = dirname($_);'
--rpl '{/.} s:.*/::; s:\.[^/.]+$::;' --rpl '{/.} s:.*/::; s:\.[^/.]+$::;'
--rpl '{.} s:\.[^/.]+$::' --rpl '{.} s:\.[^/.]+$::'
/bin/bash: --rpl: command not found /bin/bash: --rpl: command not found
/bin/bash: line 1: --rpl: command not found /bin/bash: line 1: --rpl: command not found
/bin/bash: line 2: --rpl: command not found /bin/bash: line 2: --rpl: command not found
/bin/bash: line 3: --rpl: command not found /bin/bash: line 3: --rpl: command not found
/bin/bash: line 4: --rpl: command not found
/bin/bash: line 5: --rpl: command not found /bin/bash: line 5: --rpl: command not found
/bin/bash: line 6: --rpl: command not found /bin/bash: line 6: --rpl: command not found
/bin/bash: line 7: --rpl: command not found
parallel echo {1} and {2} ::: A B ::: C D parallel echo {1} and {2} ::: A B ::: C D
A and C A and C
A and D A and D
@ -258,7 +276,8 @@ B and D
parallel echo /={1/} //={1//} /.={1/.} .={1.} ::: A/B.C D/E.F parallel echo /={1/} //={1//} /.={1/.} .={1.} ::: A/B.C D/E.F
/=B.C //=A /.=B .=A/B /=B.C //=A /.=B .=A/B
/=E.F //=D /.=E .=D/E /=E.F //=D /.=E .=D/E
parallel echo 1={1} 2={2} 3={3} -1={-1} -2={-2} -3={-3} ::: A B ::: C D ::: E F parallel echo 1={1} 2={2} 3={3} -1={-1} -2={-2} -3={-3} \
::: A B ::: C D ::: E F
1=A 2=C 3=E -1=E -2=C -3=A 1=A 2=C 3=E -1=E -2=C -3=A
1=A 2=C 3=F -1=F -2=C -3=A 1=A 2=C 3=F -1=F -2=C -3=A
1=A 2=D 3=E -1=E -2=D -3=A 1=A 2=D 3=E -1=E -2=D -3=A
@ -267,9 +286,11 @@ B and D
1=B 2=C 3=F -1=F -2=C -3=B 1=B 2=C 3=F -1=F -2=C -3=B
1=B 2=D 3=E -1=E -2=D -3=B 1=B 2=D 3=E -1=E -2=D -3=B
1=B 2=D 3=F -1=F -2=D -3=B 1=B 2=D 3=F -1=F -2=D -3=B
parallel echo '{=2 s:\.[^.]+$::;s:\.[^.]+$::; =} {1}' ::: bar ::: foo.tar.gz parallel echo '{=2 s:\.[^.]+$::;s:\.[^.]+$::; =} {1}' \
::: bar ::: foo.tar.gz
foo bar foo bar
parallel --rpl '{..} s:\.[^.]+$::;s:\.[^.]+$::;' echo '{2..} {1}' ::: bar ::: foo.tar.gz parallel --rpl '{..} s:\.[^.]+$::;s:\.[^.]+$::;' echo '{2..} {1}' \
::: bar ::: foo.tar.gz
foo bar foo bar
parallel --colsep '\t' echo 1={1} 2={2} :::: tsv-file.tsv parallel --colsep '\t' echo 1={1} 2={2} :::: tsv-file.tsv
1=f1 2=f2 1=f1 2=f2
@ -280,31 +301,83 @@ f1=A f2=C
f1=A f2=D f1=A f2=D
f1=B f2=C f1=B f2=C
f1=B f2=D f1=B f2=D
parallel --header : --colsep '\t' echo f1={f1} f2={f2} :::: tsv-file.tsv parallel --header : --colsep '\t' echo f1={f1} f2={f2} \
:::: tsv-file.tsv
f1=A f2=B f1=A f2=B
f1=C f2=D f1=C f2=D
parallel --plus echo {} ::: dir/sub/file.ext1.ext2.ext3 parallel --plus echo {} ::: dir/sub/file.ex1.ex2.ex3
parallel --plus echo {+/}/{/} ::: dir/sub/file.ext1.ext2.ext3 parallel --plus echo {+/}/{/} ::: dir/sub/file.ex1.ex2.ex3
parallel --plus echo {.}.{+.} ::: dir/sub/file.ext1.ext2.ext3 parallel --plus echo {.}.{+.} ::: dir/sub/file.ex1.ex2.ex3
parallel --plus echo {+/}/{/.}.{+.} ::: dir/sub/file.ext1.ext2.ext3 parallel --plus echo {+/}/{/.}.{+.} ::: dir/sub/file.ex1.ex2.ex3
parallel --plus echo {..}.{+..} ::: dir/sub/file.ext1.ext2.ext3 parallel --plus echo {..}.{+..} ::: dir/sub/file.ex1.ex2.ex3
parallel --plus echo {+/}/{/..}.{+..} ::: dir/sub/file.ext1.ext2.ext3 parallel --plus echo {+/}/{/..}.{+..} ::: dir/sub/file.ex1.ex2.ex3
parallel --plus echo {...}.{+...} ::: dir/sub/file.ext1.ext2.ext3 parallel --plus echo {...}.{+...} ::: dir/sub/file.ex1.ex2.ex3
parallel --plus echo {+/}/{/...}.{+...} ::: dir/sub/file.ext1.ext2.ext3 parallel --plus echo {+/}/{/...}.{+...} ::: dir/sub/file.ex1.ex2.ex3
dir/sub/file.ext1.ext2.ext3 dir/sub/file.ex1.ex2.ex3
dir/sub/file.ext1.ext2.ext3 dir/sub/file.ex1.ex2.ex3
dir/sub/file.ext1.ext2.ext3 dir/sub/file.ex1.ex2.ex3
dir/sub/file.ext1.ext2.ext3 dir/sub/file.ex1.ex2.ex3
dir/sub/file.ext1.ext2.ext3 dir/sub/file.ex1.ex2.ex3
dir/sub/file.ext1.ext2.ext3 dir/sub/file.ex1.ex2.ex3
dir/sub/file.ext1.ext2.ext3 dir/sub/file.ex1.ex2.ex3
dir/sub/file.ext1.ext2.ext3 dir/sub/file.ex1.ex2.ex3
parallel --plus echo Job {#} of {##} ::: {1..5} parallel --plus echo Job {#} of {##} ::: {1..5}
Job 1 of 5 Job 1 of 5
Job 2 of 5 Job 2 of 5
Job 3 of 5 Job 3 of 5
Job 4 of 5 Job 4 of 5
Job 5 of 5 Job 5 of 5
unset myvar
echo ${myvar:-myval}
parallel --plus echo {:-myval} ::: "$myvar"
myval
myval
myvar=abcAaAdef
echo ${myvar:2}
parallel --plus echo {:2} ::: "$myvar"
cAaAdef
cAaAdef
echo ${myvar:2:3}
parallel --plus echo {:2:3} ::: "$myvar"
echo ${myvar#bc}
parallel --plus echo {#bc} ::: "$myvar"
echo ${myvar#abc}
parallel --plus echo {#abc} ::: "$myvar"
echo ${myvar%de}
parallel --plus echo {%de} ::: "$myvar"
echo ${myvar%def}
parallel --plus echo {%def} ::: "$myvar"
echo ${myvar/def/ghi}
parallel --plus echo {/def/ghi} ::: "$myvar"
echo ${myvar^a}
parallel --plus echo {^a} ::: "$myvar"
echo ${myvar^^a}
parallel --plus echo {^^a} ::: "$myvar"
myvar=AbcAaAdef
echo ${myvar,A}
parallel --plus echo '{,A}' ::: "$myvar"
echo ${myvar,,A}
parallel --plus echo '{,,A}' ::: "$myvar"
abcAaAdef
abcAaAdef
abcaaadef
abcaaadef
cat num30000 | parallel --xargs echo | wc -l cat num30000 | parallel --xargs echo | wc -l
9 9
cat num30000 | parallel --xargs -s 10000 echo | wc -l cat num30000 | parallel --xargs -s 10000 echo | wc -l
@ -337,15 +410,15 @@ foo
A A
parallel perl -e 'print "@ARGV\n"' ::: This wont work parallel perl -e 'print "@ARGV\n"' ::: This wont work
parallel: Warning: Input is read from the terminal. You either know what you Warning: Input is read from the terminal. You either know what you
parallel: Warning: are doing (in which case: YOU ARE AWESOME!) or you forgot Warning: are doing (in which case: YOU ARE AWESOME!) or you forgot
parallel: Warning: ::: or :::: or to pipe data into parallel. If so Warning: ::: or :::: or to pipe data into parallel. If so
parallel: Warning: Press CTRL-D to exit. Warning: Press CTRL-D to exit.
perl -e 'print "@ARGV\n"' perl -e 'print "@ARGV\n"'
[CTRL-D] [CTRL-D]
/bin/bash: line 1: parallel:: command not found /bin/bash: line 1: Warning:: command not found
/bin/bash: -c: line 2: syntax error near unexpected token `(' /bin/bash: -c: line 2: syntax error near unexpected token `('
/bin/bash: -c: line 2: ` parallel: Warning: are doing (in which case: YOU ARE AWESOME!) or you forgot' /bin/bash: -c: line 2: ` Warning: are doing (in which case: YOU ARE AWESOME!) or you forgot'
parallel --trim r echo pre-{}-post ::: ' A ' parallel --trim r echo pre-{}-post ::: ' A '
pre- A-post pre- A-post
@ -367,6 +440,8 @@ pre-A-post
/bin/bash: -c: line 0: syntax error near unexpected token `)' /bin/bash: -c: line 0: syntax error near unexpected token `)'
/bin/bash: -c: line 0: `set a="def-file"; if( { test -d "$a" } ) echo "$a is a dir"' /bin/bash: -c: line 0: `set a="def-file"; if( { test -d "$a" } ) echo "$a is a dir"'
/bin/bash: -c: line 0: syntax error near unexpected token `)' /bin/bash: -c: line 0: syntax error near unexpected token `)'
/bin/bash: -c: line 0: `set a="fixedlen"; if( { test -d "$a" } ) echo "$a is a dir"'
/bin/bash: -c: line 0: syntax error near unexpected token `)'
/bin/bash: -c: line 0: `set a="num1000000"; if( { test -d "$a" } ) echo "$a is a dir"' /bin/bash: -c: line 0: `set a="num1000000"; if( { test -d "$a" } ) echo "$a is a dir"'
/bin/bash: -c: line 0: syntax error near unexpected token `)' /bin/bash: -c: line 0: syntax error near unexpected token `)'
/bin/bash: -c: line 0: `set a="num128"; if( { test -d "$a" } ) echo "$a is a dir"' /bin/bash: -c: line 0: `set a="num128"; if( { test -d "$a" } ) echo "$a is a dir"'
@ -375,8 +450,6 @@ pre-A-post
/bin/bash: -c: line 0: syntax error near unexpected token `)' /bin/bash: -c: line 0: syntax error near unexpected token `)'
/bin/bash: -c: line 0: `set a="num8"; if( { test -d "$a" } ) echo "$a is a dir"' /bin/bash: -c: line 0: `set a="num8"; if( { test -d "$a" } ) echo "$a is a dir"'
/bin/bash: -c: line 0: syntax error near unexpected token `)' /bin/bash: -c: line 0: syntax error near unexpected token `)'
/bin/bash: -c: line 0: `set a="num_%header"; if( { test -d "$a" } ) echo "$a is a dir"'
/bin/bash: -c: line 0: syntax error near unexpected token `)'
/bin/bash: -c: line 0: `set a="tsv-file.tsv"; if( { test -d "$a" } ) echo "$a is a dir"' /bin/bash: -c: line 0: `set a="tsv-file.tsv"; if( { test -d "$a" } ) echo "$a is a dir"'
parallel --tag echo foo-{} ::: A B C parallel --tag echo foo-{} ::: A B C
A foo-A A foo-A
@ -397,7 +470,8 @@ C
echo A echo A
echo B echo B
echo C echo C
parallel -j2 'printf "%s-start\n%s" {} {};sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1 parallel -j2 'printf "%s-start\n%s" {} {};
sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1
4-start 4-start
2-start 2-start
2-middle 2-middle
@ -405,7 +479,8 @@ echo C
1-start 1-start
1-middle 1-middle
1-end 1-end
parallel -j2 --ungroup 'printf "%s-start\n%s" {} {};sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1 parallel -j2 --ungroup 'printf "%s-start\n%s" {} {};
sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1
4-start 4-start
42-start 42-start
2-middle 2-middle
@ -414,7 +489,8 @@ echo C
1-middle 1-middle
1-end 1-end
-middle -middle
parallel -j2 --linebuffer 'printf "%s-start\n%s" {} {};sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1 parallel -j2 --linebuffer 'printf "%s-start\n%s" {} {};
sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1
4-start 4-start
2-start 2-start
2-middle 2-middle
@ -422,7 +498,8 @@ echo C
1-start 1-start
1-middle 1-middle
1-end 1-end
parallel -j2 -k 'printf "%s-start\n%s" {} {};sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1 parallel -j2 -k 'printf "%s-start\n%s" {} {};
sleep {};printf "%s\n" -middle;echo {}-end' ::: 4 2 1
4-start 4-start
2-start 2-start
2-middle 2-middle
@ -566,7 +643,8 @@ Computers / CPU cores / Max jobs to run
Computer:jobs running/jobs completed/%of started jobs/Average seconds to complete Computer:jobs running/jobs completed/%of started jobs/Average seconds to complete
seq 1000 | parallel -j10 --bar '(echo -n {};sleep 0.1)' 2> >(zenity --timeout=15 --progress --auto-kill --auto-close) seq 1000 | parallel -j10 --bar '(echo -n {};sleep 0.1)' \
2> >(zenity --timeout=15 --progress --auto-kill --auto-close)
BASE64-Message: GtkDialog mapped without a transient parent. This is discouraged. BASE64-Message: GtkDialog mapped without a transient parent. This is discouraged.
parallel --joblog /tmp/log exit ::: 1 2 3 0 parallel --joblog /tmp/log exit ::: 1 2 3 0
cat /tmp/log; cat /tmp/log;
@ -621,19 +699,17 @@ Seq Host Starttime JobRuntime Send Receive Exitval Signal Command
9 9
9 9
9 9
9
parallel: This job failed: parallel: This job failed:
echo X; exit X echo X; exit X
parallel: Starting no more jobs. Waiting for 1 jobs to finish. parallel: Starting no more jobs. Waiting for 1 jobs to finish.
parallel: This job failed:
echo X; exit X
parallel -j2 --halt now,fail=1 echo {}\; exit {} ::: 0 0 1 2 3 parallel -j2 --halt now,fail=1 echo {}\; exit {} ::: 0 0 1 2 3
9 9
9 9
9 9
parallel: This job failed: parallel: This job failed:
echo X; exit X echo X; exit X
parallel -j2 --halt soon,fail=20% echo {}\; exit {} ::: 0 1 2 3 4 5 6 7 8 9 parallel -j2 --halt soon,fail=20% echo {}\; exit {} \
::: 0 1 2 3 4 5 6 7 8 9
9 9
9 9
9 9
@ -652,7 +728,8 @@ echo X; exit X
9 9
parallel: This job succeeded: parallel: This job succeeded:
echo X; exit X echo X; exit X
parallel -k --retries 3 'echo tried {} >>/tmp/runs; echo completed {}; exit {}' ::: 1 2 0 parallel -k --retries 3 \
'echo tried {} >>/tmp/runs; echo completed {}; exit {}' ::: 1 2 0
cat /tmp/runs cat /tmp/runs
completed 1 completed 1
completed 2 completed 2
@ -662,11 +739,16 @@ tried 0
tried 1 tried 1
tried 1 tried 1
show_signals() { show_signals() {
perl -e 'for(keys %SIG) { $SIG{$_} = eval "sub { print \"Got $_\\n\"; }";} while(1){sleep 1}' perl -e 'for(keys %SIG) {
$SIG{$_} = eval "sub { print \"Got $_\\n\"; }";
}
while(1){sleep 1}'
} }
export -f show_signals export -f show_signals
echo | parallel --termseq TERM,200,TERM,100,TERM,50,KILL,25 -u --timeout 1 show_signals echo | parallel --termseq TERM,200,TERM,100,TERM,50,KILL,25 \
echo | parallel --termseq INT,200,TERM,100,KILL,25 -u --timeout 1 show_signals -u --timeout 1 show_signals
echo | parallel --termseq INT,200,TERM,100,KILL,25 \
-u --timeout 1 show_signals
/bin/bash: show_signals: command not found /bin/bash: show_signals: command not found
parallel --load 100% echo load is less than {} job per cpu ::: 1 parallel --load 100% echo load is less than {} job per cpu ::: 1
load is less than 1 job per cpu load is less than 1 job per cpu
@ -704,19 +786,21 @@ more
hosts hosts
parallel -S 4/$SERVER1 echo force {} cpus on server ::: 4 parallel -S 4/$SERVER1 echo force {} cpus on server ::: 4
force 4 cpus on server force 4 cpus on server
parallel --hostgroup -S @grp1/$SERVER1 -S @grp2/$SERVER2 echo {} ::: \ parallel --hostgroup -S @grp1/$SERVER1 -S @grp2/$SERVER2 echo {} \
run_on_grp1@grp1 run_on_grp2@grp2 ::: run_on_grp1@grp1 run_on_grp2@grp2
run_on_grp1 run_on_grp1
run_on_grp2 run_on_grp2
echo This is input_file > input_file echo This is input_file > input_file
parallel -S $SERVER1 --transferfile {} cat ::: input_file parallel -S $SERVER1 --transferfile {} cat ::: input_file
This is input_file This is input_file
echo This is input_file > input_file echo This is input_file > input_file
parallel -S $SERVER1 --transferfile {} --return {}.out cat {} ">"{}.out ::: input_file parallel -S $SERVER1 --transferfile {} --return {}.out \
cat {} ">"{}.out ::: input_file
cat input_file.out cat input_file.out
This is input_file This is input_file
echo This is input_file > input_file echo This is input_file > input_file
parallel -S $SERVER1 --transferfile {} --return {}.out --cleanup cat {} ">"{}.out ::: input_file parallel -S $SERVER1 --transferfile {} --return {}.out --cleanup \
cat {} ">"{}.out ::: input_file
cat input_file.out cat input_file.out
This is input_file This is input_file
echo This is input_file > input_file echo This is input_file > input_file
@ -724,7 +808,8 @@ This is input_file
cat input_file.out cat input_file.out
This is input_file This is input_file
echo common data > common_file echo common data > common_file
parallel --basefile common_file -S $SERVER1 cat common_file\; echo {} ::: foo parallel --basefile common_file -S $SERVER1 \
cat common_file\; echo {} ::: foo
common data common data
foo foo
parallel -S $SERVER1 pwd ::: "" parallel -S $SERVER1 pwd ::: ""
@ -763,42 +848,42 @@ Unknown option: :::
Unknown option: green Unknown option: green
env_parallel only works if it is a function. Do the below and restart your shell. env_parallel only works if it is a function. Do the below and restart your shell.
bash: Put this in /mnt/4tb/home/tange/.bashrc: . /usr/local/bin/env_parallel.bash bash: Put this in ~/.bashrc: . /usr/local/bin/env_parallel.bash
E.g. by doing: echo '. /usr/local/bin/env_parallel.bash' >> /mnt/4tb/home/tange/.bashrc E.g. by doing: echo '. /usr/local/bin/env_parallel.bash' >> ~/.bashrc
Supports: aliases, functions, variables, arrays Supports: aliases, functions, variables, arrays
zsh: Put this in /mnt/4tb/home/tange/.zshrc: . /usr/local/bin/env_parallel.zsh zsh: Put this in ~/.zshrc: . /usr/local/bin/env_parallel.zsh
E.g. by doing: echo '. /usr/local/bin/env_parallel.zsh' >> /mnt/4tb/home/tange/.zshenv E.g. by doing: echo '. /usr/local/bin/env_parallel.zsh' >> ~/.zshenv
Supports: functions, variables, arrays Supports: functions, variables, arrays
fish: Put this in /mnt/4tb/home/tange/.config/fish/config.fish: fish: Put this in ~/.config/fish/config.fish:
. (which env_parallel.fish) . (which env_parallel.fish)
E.g. by doing: E.g. by doing:
echo '. (which env_parallel.fish)' >> /mnt/4tb/home/tange/.config/fish/config.fish echo '. (which env_parallel.fish)' >> ~/.config/fish/config.fish
Supports: aliases, functions, variables, arrays Supports: aliases, functions, variables, arrays
ksh: Put this in /mnt/4tb/home/tange/.kshrc: source /usr/local/bin/env_parallel.ksh ksh: Put this in ~/.kshrc: source /usr/local/bin/env_parallel.ksh
E.g. by doing: echo 'source /usr/local/bin/env_parallel.ksh' >> /mnt/4tb/home/tange/.kshrc E.g. by doing: echo 'source /usr/local/bin/env_parallel.ksh' >> ~/.kshrc
Supports: aliases, functions, variables, arrays Supports: aliases, functions, variables, arrays
pdksh: Put this in /mnt/4tb/home/tange/.profile: source /usr/local/bin/env_parallel.pdksh pdksh: Put this in ~/.profile: source /usr/local/bin/env_parallel.pdksh
E.g. by doing: echo '. /usr/local/bin/env_parallel.pdksh' >> /mnt/4tb/home/tange/.profile E.g. by doing: echo '. /usr/local/bin/env_parallel.pdksh' >> ~/.profile
Supports: aliases, functions, variables, arrays Supports: aliases, functions, variables, arrays
ash: Put this in /mnt/4tb/home/tange/.profile: . /usr/local/bin/env_parallel.ash ash: Put this in ~/.profile: . /usr/local/bin/env_parallel.ash
E.g. by doing: echo '. /usr/local/bin/env_parallel.ash' >> /mnt/4tb/home/tange/.profile E.g. by doing: echo '. /usr/local/bin/env_parallel.ash' >> ~/.profile
Supports: aliases, variables Supports: aliases, variables
dash: Put this in /mnt/4tb/home/tange/.profile: . /usr/local/bin/env_parallel.dash dash: Put this in ~/.profile: . /usr/local/bin/env_parallel.dash
E.g. by doing: echo '. /usr/local/bin/env_parallel.dash' >> /mnt/4tb/home/tange/.profile E.g. by doing: echo '. /usr/local/bin/env_parallel.dash' >> ~/.profile
Supports: aliases, variables Supports: aliases, variables
csh: Put this in /mnt/4tb/home/tange/.cshrc: source /usr/local/bin/env_parallel.csh csh: Put this in ~/.cshrc: source /usr/local/bin/env_parallel.csh
E.g. by doing: echo 'source /usr/local/bin/env_parallel.csh' >> /mnt/4tb/home/tange/.cshrc E.g. by doing: echo 'source /usr/local/bin/env_parallel.csh' >> ~/.cshrc
Supports: aliases, variables, arrays with no special chars Supports: aliases, variables, arrays with no special chars
tcsh: Put this in /mnt/4tb/home/tange/.tcshrc: source /usr/local/bin/env_parallel.tcsh tcsh: Put this in ~/.tcshrc: source /usr/local/bin/env_parallel.tcsh
E.g. by doing: echo 'source /usr/local/bin/env_parallel.tcsh' >> /mnt/4tb/home/tange/.tcshrc E.g. by doing: echo 'source /usr/local/bin/env_parallel.tcsh' >> ~/.tcshrc
Supports: aliases, variables, arrays with no special chars Supports: aliases, variables, arrays with no special chars
To install in all shells run: To install in all shells run:
@ -819,6 +904,7 @@ foo bar baz
in my_func baz in my_func baz
parallel --record-env parallel --record-env
cat ~/.parallel/ignored_vars|sort cat ~/.parallel/ignored_vars|sort
BASH_FUNC_run_once%%
BASH_FUNC_run_test%% BASH_FUNC_run_test%%
COLORFGBG COLORFGBG
DBUS_SESSION_BUS_ADDRESS DBUS_SESSION_BUS_ADDRESS
@ -913,6 +999,65 @@ _
parallel --env _ -S $SERVER1 'echo $VAR; my_func2' ::: bar parallel --env _ -S $SERVER1 'echo $VAR; my_func2' ::: bar
/bin/bash: my_func2: command not found /bin/bash: my_func2: command not found
NOT='not exported var'
alias myecho=echo
not_ex() {
myecho in not_exported_func $NOT $1
}
env_parallel --env _ -S $SERVER1 'echo $NOT; not_ex' ::: bar
Unknown option: --env
Unknown option: _
Unknown option: -S
Unknown option: parallel@lo
Unknown option: echo $NOT; not_ex
Unknown option: :::
Unknown option: bar
env_parallel only works if it is a function. Do the below and restart your shell.
bash: Put this in ~/.bashrc: . /usr/local/bin/env_parallel.bash
E.g. by doing: echo '. /usr/local/bin/env_parallel.bash' >> ~/.bashrc
Supports: aliases, functions, variables, arrays
zsh: Put this in ~/.zshrc: . /usr/local/bin/env_parallel.zsh
E.g. by doing: echo '. /usr/local/bin/env_parallel.zsh' >> ~/.zshenv
Supports: functions, variables, arrays
fish: Put this in ~/.config/fish/config.fish:
. (which env_parallel.fish)
E.g. by doing:
echo '. (which env_parallel.fish)' >> ~/.config/fish/config.fish
Supports: aliases, functions, variables, arrays
ksh: Put this in ~/.kshrc: source /usr/local/bin/env_parallel.ksh
E.g. by doing: echo 'source /usr/local/bin/env_parallel.ksh' >> ~/.kshrc
Supports: aliases, functions, variables, arrays
pdksh: Put this in ~/.profile: source /usr/local/bin/env_parallel.pdksh
E.g. by doing: echo '. /usr/local/bin/env_parallel.pdksh' >> ~/.profile
Supports: aliases, functions, variables, arrays
ash: Put this in ~/.profile: . /usr/local/bin/env_parallel.ash
E.g. by doing: echo '. /usr/local/bin/env_parallel.ash' >> ~/.profile
Supports: aliases, variables
dash: Put this in ~/.profile: . /usr/local/bin/env_parallel.dash
E.g. by doing: echo '. /usr/local/bin/env_parallel.dash' >> ~/.profile
Supports: aliases, variables
csh: Put this in ~/.cshrc: source /usr/local/bin/env_parallel.csh
E.g. by doing: echo 'source /usr/local/bin/env_parallel.csh' >> ~/.cshrc
Supports: aliases, variables, arrays with no special chars
tcsh: Put this in ~/.tcshrc: source /usr/local/bin/env_parallel.tcsh
E.g. by doing: echo 'source /usr/local/bin/env_parallel.tcsh' >> ~/.tcshrc
Supports: aliases, variables, arrays with no special chars
To install in all shells run:
env_parallel --install
For details: see man env_parallel
parallel -vv --pipepart --block 1M wc :::: num30000 parallel -vv --pipepart --block 1M wc :::: num30000
<num30000 perl -e 'while(@ARGV){sysseek(STDIN,shift,0)||die;$left=shift;while($read=sysread(STDIN,$buf,$left>131072?131072:$left)){$left-=$read;syswrite(STDOUT,$buf);}}' 0 0 0 168894 | (wc) <num30000 perl -e 'while(@ARGV){sysseek(STDIN,shift,0)||die;$left=shift;while($read=sysread(STDIN,$buf,$left>131072?131072:$left)){$left-=$read;syswrite(STDOUT,$buf);}}' 0 0 0 168894 | (wc)
30000 30000 168894 30000 30000 168894
@ -920,9 +1065,11 @@ _
echo in my_func $1 > $1.out echo in my_func $1 > $1.out
} }
export -f my_func3 export -f my_func3
parallel -vv --workdir ... --nice 17 --env _ --trc {}.out -S $SERVER1 my_func3 {} ::: abc-file parallel -vv --workdir ... --nice 17 --env _ --trc {}.out \
-S $SERVER1 my_func3 {} ::: abc-file
ssh -l parallel lo -- mkdir -p ./.TMPWORKDIR && rsync --protocol 30 -rlDzR -essh\ -l\ parallel ./abc-file lo:./.TMPWORKDIR;ssh -l parallel lo -- exec perl -e @GNU_Parallel\\\=split/_/,\\\"use_IPC::Open3\\\;_use_MIME::Base64\\\"\\\;eval\\\"@GNU_Parallel\\\"\\\;\\\$chld\\\=\\\$SIG\\\{CHLD\\\}\\\;\\\$SIG\\\{CHLD\\\}\\\=\\\"IGNORE\\\"\\\;my\\\$zip\\\=\\\(grep\\\{-x\\\$_\\\}\\\"/usr/local/bin/bzip2\\\"\\\)\\\[0\\\]\\\|\\\|\\\"bzip2\\\"\\\;open3\\\(\\\$in,\\\$out,\\\"\\\>\\\&STDERR\\\",\\\$zip,\\\"-dc\\\"\\\)\\\;if\\\(my\\\$perlpid\\\=fork\\\)\\\{close\\\$in\\\;\\\$eval\\\=join\\\"\\\",\\\<\\\$out\\\>\\\;close\\\$out\\\;\\\}else\\\{close\\\$out\\\;print\\\$in\\\(decode_base64\\\(join\\\"\\\",@ARGV\\\)\\\)\\\;close\\\$in\\\;exit\\\;\\\}wait\\\;\\\$SIG\\\{CHLD\\\}\\\=\\\$chld\\\;eval\\\$eval\\\; BASE64;_EXIT_status=$?; mkdir -p ./. && rsync --protocol 30 --rsync-path=cd\ ./.TMPWORKDIR/./.\;\ rsync -rlDzR -essh\ -l\ parallel lo:./abc-file.out ./.;ssh -l parallel lo -- rm\ -f\ ./.TMPWORKDIR/abc-file\;\ sh\ -c\ rmdir\\\ ./.TMPWORKDIR/\\\ ./.parallel/tmp/\\\ ./.parallel/\\\ 2\\\>/dev/null\\\;rm\\\ -rf\\\ ./.TMPWORKDIR\\\;;ssh -l parallel lo -- rm\ -f\ ./.TMPWORKDIR/abc-file.out\;\ sh\ -c\ rmdir\\\ ./.TMPWORKDIR/\\\ ./.parallel/tmp/\\\ ./.parallel/\\\ 2\\\>/dev/null\\\;rm\\\ -rf\\\ ./.TMPWORKDIR\\\;;ssh -l parallel lo -- rm -rf .TMPWORKDIR; exit $_EXIT_status; ssh -l parallel lo -- mkdir -p ./.TMPWORKDIR && rsync --protocol 30 -rlDzR -essh\ -l\ parallel ./abc-file lo:./.TMPWORKDIR;ssh -l parallel lo -- exec perl -e @GNU_Parallel\\\=split/_/,\\\"use_IPC::Open3\\\;_use_MIME::Base64\\\"\\\;eval\\\"@GNU_Parallel\\\"\\\;\\\$chld\\\=\\\$SIG\\\{CHLD\\\}\\\;\\\$SIG\\\{CHLD\\\}\\\=\\\"IGNORE\\\"\\\;my\\\$zip\\\=\\\(grep\\\{-x\\\$_\\\}\\\"/usr/local/bin/bzip2\\\"\\\)\\\[0\\\]\\\|\\\|\\\"bzip2\\\"\\\;open3\\\(\\\$in,\\\$out,\\\"\\\>\\\&STDERR\\\",\\\$zip,\\\"-dc\\\"\\\)\\\;if\\\(my\\\$perlpid\\\=fork\\\)\\\{close\\\$in\\\;\\\$eval\\\=join\\\"\\\",\\\<\\\$out\\\>\\\;close\\\$out\\\;\\\}else\\\{close\\\$out\\\;print\\\$in\\\(decode_base64\\\(join\\\"\\\",@ARGV\\\)\\\)\\\;close\\\$in\\\;exit\\\;\\\}wait\\\;\\\$SIG\\\{CHLD\\\}\\\=\\\$chld\\\;eval\\\$eval\\\; BASE64;_EXIT_status=$?; mkdir -p ./. && rsync --protocol 30 --rsync-path=cd\ ./.TMPWORKDIR/./.\;\ rsync -rlDzR -essh\ -l\ parallel lo:./abc-file.out ./.;ssh -l parallel lo -- rm\ -f\ ./.TMPWORKDIR/abc-file\;\ sh\ -c\ rmdir\\\ ./.TMPWORKDIR/\\\ ./.parallel/tmp/\\\ ./.parallel/\\\ 2\\\>/dev/null\\\;rm\\\ -rf\\\ ./.TMPWORKDIR\\\;;ssh -l parallel lo -- rm\ -f\ ./.TMPWORKDIR/abc-file.out\;\ sh\ -c\ rmdir\\\ ./.TMPWORKDIR/\\\ ./.parallel/tmp/\\\ ./.parallel/\\\ 2\\\>/dev/null\\\;rm\\\ -rf\\\ ./.TMPWORKDIR\\\;;ssh -l parallel lo -- rm -rf .TMPWORKDIR; exit $_EXIT_status;
parallel --sqlandworker csv:////%2Ftmp%2Flog.csv seq ::: 10 ::: 12 13 14 parallel --sqlandworker csv:////%2Ftmp%2Flog.csv \
seq ::: 10 ::: 12 13 14
cat /tmp/log.csv cat /tmp/log.csv
99 99
99 99
@ -952,7 +1099,8 @@ Seq,Host,Starttime,JobRuntime,Send,Receive,Exitval,_Signal,Command,V1,V2,Stdout,
99 99
99 99
", ",
parallel --results outdir --sqlandworker csv:////%2Ftmp%2Flog2.csv seq ::: 10 ::: 12 13 14 parallel --results outdir --sqlandworker csv:////%2Ftmp%2Flog2.csv \
seq ::: 10 ::: 12 13 14
cat /tmp/log2.csv cat /tmp/log2.csv
99 99
99 99
@ -1028,7 +1176,16 @@ parallel: Error: echo is not a valid DBURL
999999 999999 9999999 999999 999999 9999999
85500 85500 598500 85500 85500 598500
25 25 176 25 25 176
echo /foo, bar/, /baz, qux/, | parallel -kN1 --recend ', ' --pipe echo JOB{#}\;cat\;echo END cat fixedlen | parallel --pipe --header .{4} --block 3 --recend '' \
'echo start; cat; echo'
start
HHHHAAA
start
HHHHBBB
start
HHHHCCC
echo /foo, bar/, /baz, qux/, | \
parallel -kN1 --recend ', ' --pipe echo JOB{#}\;cat\;echo END
JOB1 JOB1
/foo, END /foo, END
JOB2 JOB2
@ -1038,7 +1195,8 @@ JOB3
JOB4 JOB4
qux/, qux/,
END END
echo /foo, bar/, /baz, qux/, | parallel -kN1 --recstart / --pipe echo JOB{#}\;cat\;echo END echo /foo, bar/, /baz, qux/, | \
parallel -kN1 --recstart / --pipe echo JOB{#}\;cat\;echo END
JOB1 JOB1
/foo, barEND /foo, barEND
JOB2 JOB2
@ -1048,13 +1206,17 @@ JOB3
JOB4 JOB4
/, /,
END END
echo /foo, bar/, /baz, qux/, | parallel -kN1 --recend ', ' --recstart / --pipe echo JOB{#}\;cat\;echo END echo /foo, bar/, /baz, qux/, | \
parallel -kN1 --recend ', ' --recstart / --pipe \
echo JOB{#}\;cat\;echo END
JOB1 JOB1
/foo, bar/, END /foo, bar/, END
JOB2 JOB2
/baz, qux/, /baz, qux/,
END END
echo foo,bar,_baz,__qux, | parallel -kN1 --regexp --recend ,_+ --pipe echo JOB{#}\;cat\;echo END echo foo,bar,_baz,__qux, | \
parallel -kN1 --regexp --recend ,_+ --pipe \
echo JOB{#}\;cat\;echo END
JOB1 JOB1
foo,bar,_END foo,bar,_END
JOB2 JOB2
@ -1062,7 +1224,9 @@ baz,__END
JOB3 JOB3
qux, qux,
END END
echo foo,bar,_baz,__qux, | parallel -kN1 --rrs --regexp --recend ,_+ --pipe echo JOB{#}\;cat\;echo END echo foo,bar,_baz,__qux, | \
parallel -kN1 --rrs --regexp --recend ,_+ --pipe \
echo JOB{#}\;cat\;echo END
JOB1 JOB1
foo,barEND foo,barEND
JOB2 JOB2
@ -1070,52 +1234,11 @@ bazEND
JOB3 JOB3
qux, qux,
END END
cat num_%header | parallel --header '(%.*\n)*' --pipe -N3 echo JOB{#}\;cat cat num_%header | \
JOB1 parallel --header '(%.*\n)*' --pipe -N3 echo JOB{#}\;cat
%head1 cat: num_%header: No such file or directory
%head2
9
9
9
JOB2
%head1
%head2
9
9
9
JOB3
%head1
%head2
9
9
9
JOB4
%head1
%head2
99
cat num_%header | parallel --header 2 --pipe -N3 echo JOB{#}\;cat cat num_%header | parallel --header 2 --pipe -N3 echo JOB{#}\;cat
JOB1 cat: num_%header: No such file or directory
%head1
%head2
9
9
9
JOB2
%head1
%head2
9
9
9
JOB3
%head1
%head2
9
9
9
JOB4
%head1
%head2
99
parallel --pipepart -a num1000000 --block 3m wc parallel --pipepart -a num1000000 --block 3m wc
999999 999999 9999999 999999 999999 9999999
999999 999999 9999999 999999 999999 9999999
@ -1259,27 +1382,30 @@ The second is now running in the background
The first finished The first finished
The first finished running in the foreground The first finished running in the foreground
The second finished running in the foreground The second finished running in the foreground
sem --id my_id -u 'echo First started; sleep 10; echo The first finished' sem --id my_id -u 'echo First started; sleep 10; echo First done'
First started First started
The first finished First done
sem --id my_id -u 'echo Second started; sleep 10; echo The second finished' sem --id my_id -u 'echo Second started; sleep 10; echo Second done'
sem --jobs 3 --id my_id -u 'echo First started; sleep 5; echo The first finished' && Second done
sem --jobs 3 --id my_id -u 'echo Second started; sleep 6; echo The second finished' && sem --jobs 3 --id my_id -u 'echo Start 1; sleep 5; echo 1 done' &&
sem --jobs 3 --id my_id -u 'echo Third started; sleep 7; echo The third finished' && sem --jobs 3 --id my_id -u 'echo Start 2; sleep 6; echo 2 done' &&
sem --jobs 3 --id my_id -u 'echo Fourth started; sleep 8; echo The fourth finished' && sem --jobs 3 --id my_id -u 'echo Start 3; sleep 7; echo 3 done' &&
sem --jobs 3 --id my_id -u 'echo Start 4; sleep 8; echo 4 done' &&
sem --wait --id my_id sem --wait --id my_id
First started Start 1
The first finished Start 2
Third started 1 done
Fourth started Start 3
The third finished 2 done
The fourth finished Start 4
3 done
4 done
sem --id foo -u 'echo Slow started; sleep 5; echo Slow ended' && sem --id foo -u 'echo Slow started; sleep 5; echo Slow ended' &&
sem --id foo --semaphoretimeout 1 'echo Force this running after 1 sec' && sem --id foo --semaphoretimeout 1 'echo Forced running after 1 sec' &&
sem --id foo --semaphoretimeout -2 'echo Give up after 1 sec' sem --id foo --semaphoretimeout -2 'echo Give up after 2 secs'
sem --id foo --wait sem --id foo --wait
Slow started Slow started
Force this running after 1 sec Forced running after 1 sec
Slow ended Slow ended
parallel: Warning: Semaphore timed out. Stealing the semaphore. parallel: Warning: Semaphore timed out. Stealing the semaphore.
parallel: Warning: Semaphore timed out. Exiting. parallel: Warning: Semaphore timed out. Exiting.
@ -1333,7 +1459,8 @@ Web site: http://www.gnu.org/software/parallel
When using programs that use GNU Parallel to process data for publication When using programs that use GNU Parallel to process data for publication
please cite as described in 'parallel --citation'. please cite as described in 'parallel --citation'.
parallel --minversion VERSION && echo Your version is at least VERSION. parallel --minversion VERSION && \
echo Your version is at least VERSION.
VERSION VERSION
Your version is at least VERSION. Your version is at least VERSION.
parallel --citation parallel --citation

2
testsuite/wanted-results/sql03
View file

@ -46,7 +46,7 @@ Error:
sql [-hnr] [--table-size] [--db-size] [-p pass-through] [-s string] dburl [command] sql [-hnr] [--table-size] [--db-size] [-p pass-through] [-s string] dburl [command]
### Test dburl : ### Test dburl :
Error: Error:
: is not defined in /mnt/4tb/home/tange/.sql/aliases /mnt/4tb/home/tange/.dburl.aliases /etc/sql/aliases /usr/local/bin/dburl.aliases /usr/local/bin/dburl.aliases.dist : is not defined in ~/.sql/aliases ~/.dburl.aliases /etc/sql/aliases /usr/local/bin/dburl.aliases /usr/local/bin/dburl.aliases.dist
sql [-hnr] [--table-size] [--db-size] [-p pass-through] [-s string] dburl [command] sql [-hnr] [--table-size] [--db-size] [-p pass-through] [-s string] dburl [command]
### Test oracle with multiple arguments on the command line ### Test oracle with multiple arguments on the command line