parallel: --halt returns 0..100.

Passes testsuite.
This commit is contained in:
Ole Tange 2015-05-10 15:02:07 +02:00
parent 5306f0f396
commit 35cbf61ecf
8 changed files with 268 additions and 274 deletions

View file

@ -222,29 +222,30 @@ Haiku of the month:
New in this release:
* <<afventer - rykket >> Comparing the CarbonTracker and TM5-4DVar data assimilation systems for CO2 surface flux inversions http://www.atmos-chem-phys-discuss.net/15/8883/2015/acpd-15-8883-2015-discussion.html
* <<afventer opdatering>> CIDER: a pipeline for detecting waves of coordinated transcriptional regulation in gene expression time-course data http://biorxiv.org/content/biorxiv/early/2015/03/17/012518.full.pdf
* <<afventer opdatering>> GNU Parallel was used (unfortunately without citation) in: MUGBAS: a species free gene-based programme suite for post-GWAS analysis http://www.ncbi.nlm.nih.gov/pubmed/25765345
http://biorxiv.org/content/biorxiv/early/2015/05/05/018085.full.pdf
* <<Har angiveligt submittet ny version - afventer opdatering>> GNU Parallel was used (unfortunately without citation) in: MUGBAS: a species free gene-based programme suite for post-GWAS analysis http://www.ncbi.nlm.nih.gov/pubmed/25765345
taxator-tk http://algbio.cs.uni-duesseldorf.de/webapps/wa-download/ (check it)
* <<har ikke svaret og accepteret ansvar>> GNU Parallel was used in: Large Scale Author Name Disambiguation in Digital Libraries http://ieeexplore.ieee.org/xpl/abstractReferences.jsp?tp=&arnumber=7004487&url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D7004487
* << afventer svar fra Rachel >> GNU Parallel was used in: SISRS: Site Identification from Short Read Sequences https://github.com/rachelss/SISRS/
* << Update forventet juni >> GNU Parallel was used in: SISRS: Site Identification from Short Read Sequences https://github.com/rachelss/SISRS/
* <<kontaktet 2015-05-06>> GNU Parallel was used (unfortunately with wrong citation) in: TADSim: Discrete Event-Based Performance Prediction for Temperature-Accelerated Dynamics http://vruehle.de/publications/2015c.pdf
* GNU Parallel was cited in: CIDER: a pipeline for detecting waves of coordinated transcriptional regulation in gene expression time-course data http://biorxiv.org/content/biorxiv/early/2015/03/17/012518.full.pdf
* GNU Parallel was cited in: Building Genomic Analysis Pipelines in a Hackathon Setting with Bioinformatician Teams: DNA-seq, Epigenomics, Metagenomics and RNA-seq http://biorxiv.org/content/biorxiv/early/2015/05/05/018085.full.pdf
* GNU Parallel was cited in: Toward Enhanced Metadata Quality of Large-Scale Digital Libraries: Estimating Volume Time Range https://www.ideals.illinois.edu/bitstream/handle/2142/73656/186_ready.pdf
* GNU Parallel was cited in: Sequencing the cap-snatching repertoire of H1N1 influenza provides insight into the mechanism of viral transcription initiation http://nar.oxfordjournals.org/content/early/2015/04/20/nar.gkv333.full.pdf
* GNU Parallel was cited in: Genome assemblyusing Nanopore-guided long and error-free DNA reads http://www.biomedcentral.com/content/pdf/s12864-015-1519-z.pdf
* GNU Parallel was cited in: Genome assembly using Nanopore-guided long and error-free DNA reads http://www.biomedcentral.com/content/pdf/s12864-015-1519-z.pdf
* GNU Parallel was cited in: Contrasting regional architectures of schizophrenia and other complex diseases using fast variance components analysis http://biorxiv.org/content/biorxiv/early/2015/03/13/016527.full.pdf
* <<kontakt>> GNU Parallel was used (unfortunately with wrong citation) in: TADSim: Discrete Event-Based Performance Prediction for Temperature-Accelerated Dynamics http://vruehle.de/publications/2015c.pdf
* GNU Parallel was used (unfortunately with wrong citation) in: Comparing the CarbonTracker and TM5-4DVar data assimilation systems for CO2 surface flux inversions http://www.atmos-chem-phys-discuss.net/15/8883/2015/acpd-15-8883-2015-discussion.html
* GNU Parallel was used in: Gene Set Omic Analysis (GSOA) method https://bitbucket.org/srp33/gsoa

View file

@ -110,7 +110,7 @@ if($opt::nonall or $opt::onall) {
$Global::JobQueue = JobQueue->new(
\@command,\@input_source_fh,$Global::ContextReplace,$number_of_args,\@Global::ret_files);
if($opt::eta or $opt::bar or $opt::shuf) {
if($opt::eta or $opt::bar or $opt::shuf or $Global::halt_pct) {
# Count the number of jobs or shuffle all jobs
# before starting any
$Global::JobQueue->total_jobs();
@ -162,9 +162,17 @@ for(keys %Global::sshmaster) {
}
::debug("init", "Halt\n");
if($opt::halt and $Global::halt_when ne "never") {
if(not defined $Global::halt_exitstatus) {
if($Global::halt_pct) {
$Global::halt_exitstatus =
::ceil($Global::total_failed / $Global::total_started * 100);
} elsif($Global::halt_count) {
$Global::halt_exitstatus = ::min($Global::total_failed,101);
}
}
wait_and_exit($Global::halt_exitstatus);
} else {
wait_and_exit(min(undef_as_zero($Global::exitstatus),254));
wait_and_exit(min(undef_as_zero($Global::exitstatus),101));
}
sub __PIPE_MODE__ {}
@ -1093,7 +1101,6 @@ sub init_globals {
$Global::stderr_verbose = 0;
$Global::default_simultaneous_sshlogins = 9;
$Global::exitstatus = 0;
$Global::halt_exitstatus = 0;
$Global::arg_sep = ":::";
$Global::arg_file_sep = "::::";
$Global::trim = 'n';
@ -3349,16 +3356,14 @@ sub bibtex {
if(open (my $fh, ">", $ENV{'HOME'}."/.parallel/will-cite")) {
close $fh;
print "\nThank you for your support. It is much appreciated. The citation\n",
"notice is now silenced. You may also use '--will-cite'.\n",
"If you use '--will-cite' in scripts you are expected to pay\n",
"the 10000 EUR, because you are making it harder to see the\n",
"citation notice.\n\n";
"notice is now silenced. For other ways to silence the citation notice\n",
"see 'man parallel' under '--bibtex'.\n\n";
} else {
print "\nThank you for your support. It is much appreciated. The citation\n",
"cannot permanently be silenced. Use '--will-cite' instead.\n",
"If you use '--will-cite' in scripts you are expected to pay\n",
"the 10000 EUR, because you are making it harder to see the\n",
"citation notice.\n\n";
"If you use '--will-cite' in scripts you are making it harder to see the\n",
"citation notice. However, if you pay 10000 EUR, you should feel free\n",
"to use '--will-cite'.\n\n";
last;
}
}
@ -7471,11 +7476,19 @@ sub set_exitsignal {
{
my $status_printed;
my $total_jobs;
sub should_we_halt {
# Should we halt? Immediately? Gracefully?
# Returns: N/A
my $job = shift;
# --halt # => 1..100 (number of jobs failed, 101 means > 100)
# --halt % => 1..100 (pct of jobs failed)
if($Global::halt_pct and not $Global::halt_count) {
$total_jobs ||= $Global::JobQueue->total_jobs();
# From the pct compute the number of jobs that must fail/succeed
$Global::halt_count = $total_jobs * $Global::halt_pct;
}
if($job->exitstatus() or $job->exitsignal()) {
# Job failed
$Global::exitstatus++;
@ -7483,21 +7496,23 @@ sub set_exitsignal {
if($Global::halt_fail) {
::status("$Global::progname: This job failed:\n",
$job->replaced(),"\n");
if(($Global::halt_count and
$Global::halt_count <= $Global::total_failed)
or
($Global::halt_pct and
$Global::halt_pct <=
$Global::total_failed / $Global::total_started
and $Global::total_failed > 3)) {
if($Global::halt_count <= $Global::total_failed) {
# At least N jobs had failed
# or at least N% had failed and more than 3
if($Global::halt_count and $Global::halt_count == 1) {
$Global::halt_exitstatus = $job->exitstatus();
} else {
$Global::halt_exitstatus = ::min($Global::total_failed,254);
if(not defined $Global::halt_exitstatus) {
if($Global::halt_pct) {
# --halt now,fail=X% or soon,fail=X%
$Global::halt_exitstatus =
::ceil($Global::total_failed / $total_jobs * 100);
} elsif($Global::halt_count) {
# --halt now,fail=X or soon,fail=X
$Global::halt_exitstatus = ::min($Global::total_failed,101);
}
if($Global::halt_count and $Global::halt_count == 1) {
# --halt now,fail=1 or soon,fail=1
$Global::halt_exitstatus = $job->exitstatus();
}
}
::debug("halt","Pct: ",$Global::halt_pct,"<=",$Global::total_failed / $Global::total_started," count: ",$Global::halt_count,"\n");
::debug("halt","Pct: ",$Global::halt_pct," count: ",$Global::halt_count,"\n");
if($Global::halt_when eq "soon"
and scalar(keys %Global::running) > 0) {
::status
@ -7511,20 +7526,14 @@ sub set_exitsignal {
}
} else {
if($Global::halt_success) {
$Global::halt_exitstatus = $Global::total_failed;
::debug("halt","Pct: ",$Global::halt_pct,"<=",
" count: ",$Global::halt_count,"\n");
::status("$Global::progname: This job succeeded:\n",
$job->replaced(),"\n");
if(($Global::halt_count and
$Global::halt_count <=
$Global::total_completed-$Global::total_failed)
or
($Global::halt_pct and
$Global::halt_pct <=
($Global::total_completed-$Global::total_failed)
/ $Global::total_completed
and ($Global::total_completed-$Global::total_failed) > 3)) {
if($Global::halt_count <=
$Global::total_completed-$Global::total_failed) {
# At least N jobs had success
# or at least N% had success and more than 3
# or at least N% had success
$Global::halt_exitstatus = 0;
if($Global::halt_when eq "soon"
and scalar(keys %Global::running) > 0) {

View file

@ -422,9 +422,19 @@ Implies B<--semaphore>.
=item B<--bibtex>
Print the BibTeX entry for GNU B<parallel> and disable citation
Print the BibTeX entry for GNU B<parallel> and silence citation
notice.
If it is impossible for you to run B<--bibtex> you can use
B<--will-cite>.
If you use B<--will-cite> in scripts to be run by others you are
making it harder for others to see the citation notice. The
development of GNU B<parallel> is indirectly financed through
citations, so if users do not know they should cite then that makes it
harder to finance development. However, if you pay 10000 EUR, you
should feel free to use B<--will-cite>.
=item B<--block> I<size>
@ -691,42 +701,80 @@ Print a summary of the options to GNU B<parallel> and exit.
=item B<--halt> I<val>
How should GNU B<parallel> terminate?
When should GNU B<parallel> terminate? In some situations it makes no
sense to run all jobs. GNU B<parallel> should simply give up as soon
as a condition is met.
I<val> defaults to B<never>, which runs all jobs no matter what.
I<val> can also take on the form of I<when>,I<why>.
I<when> can be 'now' which means kill all running jobs and halt
immediately, or it can be 'soon' which means wait for all running jobs
to complete, but start no new jobs.
I<why> can be 'fail=X', 'fail=Y%', 'success=X', or 'success=Y%' where
X is the number of jobs that has to fail or succeed before halting,
and Y is the percentage of jobs that has to fail or succeed before
halting.
Example:
=over 23
=item Z<> --halt now,fail=1
exit when the first job fails. Kill running jobs.
=item Z<> --halt soon,fail=3
exit when 3 jobs fail, but wait for running jobs to complete.
=item Z<> --halt soon,fail=3%
exit when 3% of the jobs have failed, but wait for running jobs to complete.
=item Z<> --halt now,success=1
exit when a job succeeds. Kill running jobs.
=item Z<> --halt soon,success=3
exit when 3 jobs succeeds, but wait for running jobs to complete.
=item Z<> --halt now,success=3%
exit when 3% of the jobs have succeeded. Kill running jobs.
=back
For backwards compability these also work:
=over 7
=item Z<>0
Do not halt if a job fails. Exit status will be the number of jobs
failed. This is the default.
never
=item Z<>1
Do not start new jobs if a job fails, but complete the running jobs
including cleanup. The exit status will be the exit status from the
last failing job.
soon,fail=1
=item Z<>2
Kill off all jobs immediately and exit without cleanup. The exit
status will be the exit status from the failing job.
now,fail=1
=item Z<>-1
Do not start new jobs if a job succeeds, but complete the running jobs
including cleanup. The exit status will be the exit status from the
last failing job if any.
soon,success=1
=item Z<>-2
Kill off all jobs immediately and exit without cleanup. The exit
status will be 0.
now,success=1
=item Z<>1-99%
If I<val>% of the jobs fail and minimum 3: Do not start new jobs, but
complete the running jobs including cleanup. The exit status will be
the exit status from the last failing job.
soon,fail=1-99%
=back
@ -3482,21 +3530,25 @@ remote computers:
=head1 EXIT STATUS
If B<--halt-on-error> 0 or not specified:
Exit status depends on B<--halt-on-error> if one of these are used:
success=X, success=Y%, fail=Y%.
=over 6
=item Z<>0
All jobs ran without error.
All jobs ran without error. If success=X is used: X jobs ran without
error. If success=Y% is used: Y% of the jobs ran without error.
=item Z<>1-253
=item Z<>1-100
Some of the jobs failed. The exit status gives the number of failed jobs
Some of the jobs failed. The exit status gives the number of failed
jobs. If Y% is used the exit status is the percentage of jobs that
failed.
=item Z<>254
=item Z<>101
More than 253 jobs failed.
More than 100 jobs failed.
=item Z<>255
@ -3504,7 +3556,8 @@ Other error.
=back
If B<--halt-on-error> 1 or 2: Exit status of the failing job.
If fail=1 is used, the exit status will be the exit status of the
failing job.
=head1 DIFFERENCES BETWEEN GNU Parallel AND ALTERNATIVES
@ -3533,7 +3586,7 @@ Manipulation of input
M3. Arguments can be put anywhere in the execution line
M4. Multiple arguments can be put anywhere in the execution line
M5. Arguments can be replaced with context
M6. Input can be treated as complete execution line
M6. Input can be treated as the complete command line
Outputs
O1. Grouping output so output from different jobs do not mix

View file

@ -24,7 +24,7 @@ echo '### Test --halt-on-error 1';
(echo "sleep 1;true"; echo "sleep 2;false";echo "sleep 3;true") | parallel -j10 --halt-on-error 1;
echo $?;
(echo "sleep 1;true"; echo "sleep 2;false";echo "sleep 3;true";echo "sleep 4; non_exist") | parallel -j10 --halt 1;
(echo "sleep 1;true"; echo "sleep 2; non_exist";echo "sleep 3;true";echo "sleep 4; false") | parallel -j10 --halt 1;
echo $?
echo '**'
@ -56,20 +56,20 @@ echo '### Test --halt -2';
echo '**'
echo '### Test last dying print --halt-on-error 1';
(seq 0 8;echo 0; echo 9) | parallel -j10 -kq --halt 1 perl -e 'sleep $ARGV[0];print STDERR @ARGV,"\n"; exit shift';
echo '### Test first dying print --halt-on-error 1';
(echo 0; echo 3; seq 0 7;echo 0; echo 8) | parallel -j10 -kq --halt 1 perl -e 'sleep $ARGV[0];print STDERR @ARGV,"\n"; exit shift';
echo exit code $?
echo '### Test last dying print --halt-on-error 2';
(seq 0 8;echo 0; echo 9) | parallel -j10 -kq --halt 2 perl -e 'sleep $ARGV[0];print STDERR @ARGV,"\n"; exit shift';
(echo 0; echo 3; seq 0 7;echo 0; echo 8) | parallel -j10 -kq --halt 2 perl -e 'sleep $ARGV[0];print STDERR @ARGV,"\n"; exit shift';
echo exit code $?
echo '### Test last dying print --halt-on-error -1';
(seq 0 8;echo 0; echo 9) | parallel -j10 -kq --halt -1 perl -e 'sleep $ARGV[0];print STDERR @ARGV,"\n"; exit not shift';
(echo 0; echo 3; seq 0 7;echo 0; echo 8) | parallel -j10 -kq --halt -1 perl -e 'sleep $ARGV[0];print STDERR @ARGV,"\n"; exit not shift';
echo exit code $?
echo '### Test last dying print --halt-on-error -2';
(seq 0 8;echo 0; echo 9) | parallel -j10 -kq --halt -2 perl -e 'sleep $ARGV[0];print STDERR @ARGV,"\n"; exit not shift';
(echo 0; echo 3; seq 0 7;echo 0; echo 8) | parallel -j10 -kq --halt -2 perl -e 'sleep $ARGV[0];print STDERR @ARGV,"\n"; exit not shift';
echo exit code $?
echo '**'

View file

@ -3,7 +3,7 @@
# /tmp/parallel-local-ssh2 will by default be owned by me and should be writable by *@localhost
chmod 777 "$TMPDIR" 2>/dev/null
cat <<'EOF' | sed -e s/\$SERVER1/$SERVER1/\;s/\$SERVER2/$SERVER2/ | stdout parallel -vj5 -k --joblog /tmp/jl-`basename $0` -L1
cat <<'EOF' | sed -e s/\$SERVER1/$SERVER1/\;s/\$SERVER2/$SERVER2/ | stdout parallel -vj5 --retries 3 -k --joblog /tmp/jl-`basename $0` -L1
echo "### bug #43518: GNU Parallel doesn't obey servers' jobs capacity when an ssh login file is reloaded"
# Pre-20141106 Would reset the number of jobs run on all sshlogin if --slf changed
# Thus must take at least 25 sec to run

View file

@ -33,6 +33,8 @@ perl -ne '$/="\n\n"; /^Output/../^[^O]\S/ and next; /^ / and print;' ../../src/
s/... ... .. ..:..:.. \D+ ..../DATE OUTPUT/;
# Timestamp from --joblog
s/\d{10}.\d{3}\s+..\d+/TIMESTAMP\t9.999/g;
# Version
s/201\d{5}/VERSION/g;
# Remote script
s/(PARALLEL_PID\D+)\d+/${1}000000/g;
# /usr/bin/time -f %e

View file

@ -13,19 +13,19 @@ echo '### Test --halt-on-error 0'; (echo "sleep 1;true"; echo "sleep 2;false";
/bin/bash: non_exist: command not found
echo '**'
**
echo '### Test --halt-on-error 1'; (echo "sleep 1;true"; echo "sleep 2;false";echo "sleep 3;true") | parallel -j10 --halt-on-error 1; echo $?; (echo "sleep 1;true"; echo "sleep 2;false";echo "sleep 3;true";echo "sleep 4; non_exist") | parallel -j10 --halt 1; echo $?
echo '### Test --halt-on-error 1'; (echo "sleep 1;true"; echo "sleep 2;false";echo "sleep 3;true") | parallel -j10 --halt-on-error 1; echo $?; (echo "sleep 1;true"; echo "sleep 2; non_exist";echo "sleep 3;true";echo "sleep 4; false") | parallel -j10 --halt 1; echo $?
### Test --halt-on-error 1
1
127
parallel: This job failed:
sleep 2;false
parallel: Starting no more jobs. Waiting for 1 jobs to finish.
parallel: This job failed:
sleep 2;false
parallel: Starting no more jobs. Waiting for 2 jobs to finish.
/bin/bash: non_exist: command not found
parallel: This job failed:
sleep 4; non_exist
sleep 2; non_exist
parallel: Starting no more jobs. Waiting for 2 jobs to finish.
parallel: This job failed:
sleep 4; false
echo '**'
**
echo '### Test --halt-on-error 2'; (echo "sleep 1;true"; echo "sleep 2;false";echo "sleep 3;true") | parallel -j10 --halt-on-error 2; echo $?; (echo "sleep 1;true"; echo "sleep 2;false";echo "sleep 3;true";echo "sleep 4; non_exist") | parallel -j10 --halt 2; echo $?
@ -61,98 +61,98 @@ parallel: This job succeeded:
sleep 2;true
echo '**'
**
echo '### Test last dying print --halt-on-error 1'; (seq 0 8;echo 0; echo 9) | parallel -j10 -kq --halt 1 perl -e 'sleep $ARGV[0];print STDERR @ARGV,"\n"; exit shift'; echo exit code $?
### Test last dying print --halt-on-error 1
exit code 9
echo '### Test first dying print --halt-on-error 1'; (echo 0; echo 3; seq 0 7;echo 0; echo 8) | parallel -j10 -kq --halt 1 perl -e 'sleep $ARGV[0];print STDERR @ARGV,"\n"; exit shift'; echo exit code $?
### Test first dying print --halt-on-error 1
exit code 1
0
1
parallel: This job failed:
perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ exit\ shift 1
parallel: Starting no more jobs. Waiting for 8 jobs to finish.
2
parallel: This job failed:
perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ exit\ shift 2
parallel: Starting no more jobs. Waiting for 7 jobs to finish.
3
0
1
2
parallel: This job failed:
perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ exit\ shift 3
parallel: Starting no more jobs. Waiting for 6 jobs to finish.
3
parallel: This job failed:
perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ exit\ shift 3
parallel: Starting no more jobs. Waiting for 5 jobs to finish.
4
parallel: This job failed:
perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ exit\ shift 4
parallel: Starting no more jobs. Waiting for 5 jobs to finish.
parallel: Starting no more jobs. Waiting for 4 jobs to finish.
5
parallel: This job failed:
perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ exit\ shift 5
parallel: Starting no more jobs. Waiting for 4 jobs to finish.
parallel: Starting no more jobs. Waiting for 3 jobs to finish.
6
parallel: This job failed:
perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ exit\ shift 6
parallel: Starting no more jobs. Waiting for 3 jobs to finish.
7
parallel: This job failed:
perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ exit\ shift 7
parallel: Starting no more jobs. Waiting for 2 jobs to finish.
8
7
0
parallel: This job failed:
perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ exit\ shift 8
perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ exit\ shift 7
parallel: Starting no more jobs. Waiting for 1 jobs to finish.
9
8
parallel: This job failed:
perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ exit\ shift 9
echo '### Test last dying print --halt-on-error 2'; (seq 0 8;echo 0; echo 9) | parallel -j10 -kq --halt 2 perl -e 'sleep $ARGV[0];print STDERR @ARGV,"\n"; exit shift'; echo exit code $?
perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ exit\ shift 8
echo '### Test last dying print --halt-on-error 2'; (echo 0; echo 3; seq 0 7;echo 0; echo 8) | parallel -j10 -kq --halt 2 perl -e 'sleep $ARGV[0];print STDERR @ARGV,"\n"; exit shift'; echo exit code $?
### Test last dying print --halt-on-error 2
exit code 1
0
1
parallel: This job failed:
perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ exit\ shift 1
echo '### Test last dying print --halt-on-error -1'; (seq 0 8;echo 0; echo 9) | parallel -j10 -kq --halt -1 perl -e 'sleep $ARGV[0];print STDERR @ARGV,"\n"; exit not shift'; echo exit code $?
echo '### Test last dying print --halt-on-error -1'; (echo 0; echo 3; seq 0 7;echo 0; echo 8) | parallel -j10 -kq --halt -1 perl -e 'sleep $ARGV[0];print STDERR @ARGV,"\n"; exit not shift'; echo exit code $?
### Test last dying print --halt-on-error -1
exit code 0
0
1
parallel: This job succeeded:
perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ exit\ not\ shift 1
parallel: Starting no more jobs. Waiting for 8 jobs to finish.
2
parallel: This job succeeded:
perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ exit\ not\ shift 2
parallel: Starting no more jobs. Waiting for 7 jobs to finish.
3
0
1
2
parallel: This job succeeded:
perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ exit\ not\ shift 3
parallel: Starting no more jobs. Waiting for 6 jobs to finish.
3
parallel: This job succeeded:
perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ exit\ not\ shift 3
parallel: Starting no more jobs. Waiting for 5 jobs to finish.
4
parallel: This job succeeded:
perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ exit\ not\ shift 4
parallel: Starting no more jobs. Waiting for 5 jobs to finish.
parallel: Starting no more jobs. Waiting for 4 jobs to finish.
5
parallel: This job succeeded:
perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ exit\ not\ shift 5
parallel: Starting no more jobs. Waiting for 4 jobs to finish.
parallel: Starting no more jobs. Waiting for 3 jobs to finish.
6
parallel: This job succeeded:
perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ exit\ not\ shift 6
parallel: Starting no more jobs. Waiting for 3 jobs to finish.
7
parallel: This job succeeded:
perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ exit\ not\ shift 7
parallel: Starting no more jobs. Waiting for 2 jobs to finish.
8
7
0
parallel: This job succeeded:
perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ exit\ not\ shift 8
perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ exit\ not\ shift 7
parallel: Starting no more jobs. Waiting for 1 jobs to finish.
9
8
parallel: This job succeeded:
perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ exit\ not\ shift 9
echo '### Test last dying print --halt-on-error -2'; (seq 0 8;echo 0; echo 9) | parallel -j10 -kq --halt -2 perl -e 'sleep $ARGV[0];print STDERR @ARGV,"\n"; exit not shift'; echo exit code $?
perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ exit\ not\ shift 8
echo '### Test last dying print --halt-on-error -2'; (echo 0; echo 3; seq 0 7;echo 0; echo 8) | parallel -j10 -kq --halt -2 perl -e 'sleep $ARGV[0];print STDERR @ARGV,"\n"; exit not shift'; echo exit code $?
### Test last dying print --halt-on-error -2
exit code 0
0
1
parallel: This job succeeded:
perl -e sleep\ \$ARGV\[0\]\;print\ STDERR\ @ARGV,\"\\n\"\;\ exit\ not\ shift 1
echo '**'
@ -161,45 +161,9 @@ testhalt() { echo '### testhalt --halt '$1; (yes 0 | head -n 10; seq 10) | s
### testhalt --halt now,fail=0
parallel: This job failed:
sleep 0.3; exit 1
parallel: This job failed:
sleep 0.6; exit 2
parallel: This job failed:
sleep 0.9; exit 3
parallel: This job failed:
sleep 1.2; exit 4
parallel: This job failed:
sleep 1.5; exit 5
parallel: This job failed:
sleep 1.8; exit 6
parallel: This job failed:
sleep 2.1; exit 7
parallel: This job failed:
sleep 2.4; exit 8
parallel: This job failed:
sleep 2.7; exit 9
parallel: This job failed:
sleep 3; exit 10
0
parallel: This job failed:
sleep 0.3; exit 1
parallel: This job failed:
sleep 0.6; exit 2
parallel: This job failed:
sleep 0.9; exit 3
parallel: This job failed:
sleep 1.2; exit 4
parallel: This job failed:
sleep 1.5; exit 5
parallel: This job failed:
sleep 1.8; exit 6
parallel: This job failed:
sleep 2.1; exit 7
parallel: This job failed:
sleep 2.4; exit 8
parallel: This job failed:
sleep 2.7; exit 9
parallel: This job failed:
sleep 3; exit 10
0
### testhalt --halt now,fail=1
parallel: This job failed:
@ -232,7 +196,7 @@ parallel: This job failed:
sleep 1.5; exit 5
parallel: This job failed:
sleep 1.8; exit 6
6
30
parallel: This job failed:
sleep 0.3; exit 1
parallel: This job failed:
@ -241,7 +205,11 @@ parallel: This job failed:
sleep 0.9; exit 3
parallel: This job failed:
sleep 1.2; exit 4
4
parallel: This job failed:
sleep 1.5; exit 5
parallel: This job failed:
sleep 1.8; exit 6
30
### testhalt --halt now,fail=70%
parallel: This job failed:
sleep 0.3; exit 1
@ -263,7 +231,7 @@ parallel: This job failed:
sleep 2.7; exit 9
parallel: This job failed:
sleep 3; exit 10
0
50
parallel: This job failed:
sleep 0.3; exit 1
parallel: This job failed:
@ -278,49 +246,37 @@ parallel: This job failed:
sleep 1.8; exit 6
parallel: This job failed:
sleep 2.1; exit 7
7
parallel: This job failed:
sleep 2.4; exit 8
parallel: This job failed:
sleep 2.7; exit 9
parallel: This job failed:
sleep 3; exit 10
50
### testhalt --halt soon,fail=0
parallel: This job failed:
sleep 0.3; exit 1
parallel: Starting no more jobs. Waiting for 3 jobs to finish.
parallel: This job failed:
sleep 0.6; exit 2
parallel: Starting no more jobs. Waiting for 2 jobs to finish.
parallel: This job failed:
sleep 0.9; exit 3
parallel: Starting no more jobs. Waiting for 1 jobs to finish.
parallel: This job failed:
sleep 1.2; exit 4
parallel: This job failed:
sleep 1.5; exit 5
parallel: This job failed:
sleep 1.8; exit 6
parallel: This job failed:
sleep 2.1; exit 7
parallel: This job failed:
sleep 2.4; exit 8
parallel: This job failed:
sleep 2.7; exit 9
parallel: This job failed:
sleep 3; exit 10
0
parallel: This job failed:
sleep 0.3; exit 1
parallel: Starting no more jobs. Waiting for 3 jobs to finish.
parallel: This job failed:
sleep 0.6; exit 2
parallel: Starting no more jobs. Waiting for 2 jobs to finish.
parallel: This job failed:
sleep 0.9; exit 3
parallel: Starting no more jobs. Waiting for 1 jobs to finish.
parallel: This job failed:
sleep 1.2; exit 4
parallel: This job failed:
sleep 1.5; exit 5
parallel: This job failed:
sleep 1.8; exit 6
parallel: This job failed:
sleep 2.1; exit 7
parallel: This job failed:
sleep 2.4; exit 8
parallel: This job failed:
sleep 2.7; exit 9
parallel: This job failed:
sleep 3; exit 10
0
### testhalt --halt soon,fail=1
parallel: This job failed:
@ -334,7 +290,7 @@ sleep 0.9; exit 3
parallel: Starting no more jobs. Waiting for 1 jobs to finish.
parallel: This job failed:
sleep 1.2; exit 4
4
1
parallel: This job failed:
sleep 0.3; exit 1
parallel: Starting no more jobs. Waiting for 3 jobs to finish.
@ -346,7 +302,7 @@ sleep 0.9; exit 3
parallel: Starting no more jobs. Waiting for 1 jobs to finish.
parallel: This job failed:
sleep 1.2; exit 4
4
1
### testhalt --halt soon,fail=2
parallel: This job failed:
sleep 0.3; exit 1
@ -361,7 +317,7 @@ sleep 1.2; exit 4
parallel: Starting no more jobs. Waiting for 1 jobs to finish.
parallel: This job failed:
sleep 1.5; exit 5
5
2
parallel: This job failed:
sleep 0.3; exit 1
parallel: This job failed:
@ -375,7 +331,7 @@ sleep 1.2; exit 4
parallel: Starting no more jobs. Waiting for 1 jobs to finish.
parallel: This job failed:
sleep 1.5; exit 5
5
2
### testhalt --halt soon,fail=30%
parallel: This job failed:
sleep 0.3; exit 1
@ -398,7 +354,7 @@ sleep 2.4; exit 8
parallel: Starting no more jobs. Waiting for 1 jobs to finish.
parallel: This job failed:
sleep 2.7; exit 9
9
30
parallel: This job failed:
sleep 0.3; exit 1
parallel: This job failed:
@ -407,16 +363,20 @@ parallel: This job failed:
sleep 0.9; exit 3
parallel: This job failed:
sleep 1.2; exit 4
parallel: Starting no more jobs. Waiting for 3 jobs to finish.
parallel: This job failed:
sleep 1.5; exit 5
parallel: Starting no more jobs. Waiting for 2 jobs to finish.
parallel: This job failed:
sleep 1.8; exit 6
parallel: Starting no more jobs. Waiting for 1 jobs to finish.
parallel: Starting no more jobs. Waiting for 3 jobs to finish.
parallel: This job failed:
sleep 2.1; exit 7
7
parallel: Starting no more jobs. Waiting for 2 jobs to finish.
parallel: This job failed:
sleep 2.4; exit 8
parallel: Starting no more jobs. Waiting for 1 jobs to finish.
parallel: This job failed:
sleep 2.7; exit 9
30
### testhalt --halt soon,fail=70%
parallel: This job failed:
sleep 0.3; exit 1
@ -438,7 +398,7 @@ parallel: This job failed:
sleep 2.7; exit 9
parallel: This job failed:
sleep 3; exit 10
0
50
parallel: This job failed:
sleep 0.3; exit 1
parallel: This job failed:
@ -453,59 +413,20 @@ parallel: This job failed:
sleep 1.8; exit 6
parallel: This job failed:
sleep 2.1; exit 7
parallel: Starting no more jobs. Waiting for 3 jobs to finish.
parallel: This job failed:
sleep 2.4; exit 8
parallel: Starting no more jobs. Waiting for 2 jobs to finish.
parallel: This job failed:
sleep 2.7; exit 9
parallel: Starting no more jobs. Waiting for 1 jobs to finish.
parallel: This job failed:
sleep 3; exit 10
10
50
### testhalt --halt now,success=0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
7
0
### testhalt --halt now,success=1
parallel: This job succeeded:
sleep 0; exit 0
@ -533,6 +454,10 @@ parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
0
parallel: This job succeeded:
sleep 0; exit 0
@ -542,6 +467,10 @@ parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
0
### testhalt --halt now,success=70%
parallel: This job succeeded:
@ -552,7 +481,19 @@ parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
50
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
@ -573,50 +514,24 @@ parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
7
50
### testhalt --halt soon,success=0
parallel: This job succeeded:
sleep 0; exit 0
parallel: Starting no more jobs. Waiting for 3 jobs to finish.
parallel: This job succeeded:
sleep 0; exit 0
parallel: Starting no more jobs. Waiting for 2 jobs to finish.
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: Starting no more jobs. Waiting for 1 jobs to finish.
parallel: This job succeeded:
sleep 0; exit 0
0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
7
parallel: Starting no more jobs. Waiting for 3 jobs to finish.
0
### testhalt --halt soon,success=1
parallel: This job succeeded:
sleep 0; exit 0
@ -664,6 +579,10 @@ parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: Starting no more jobs. Waiting for 3 jobs to finish.
parallel: This job succeeded:
sleep 0; exit 0
@ -682,6 +601,10 @@ parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: Starting no more jobs. Waiting for 3 jobs to finish.
0
### testhalt --halt now,success=70%
@ -693,7 +616,19 @@ parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
50
parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
@ -714,7 +649,7 @@ parallel: This job succeeded:
sleep 0; exit 0
parallel: This job succeeded:
sleep 0; exit 0
7
50
echo '**'
**
echo '### Test slow arguments generation - https://savannah.gnu.org/bugs/?32834'; seq 1 3 | parallel -j1 "sleep 2; echo {}" | parallel -kj2 echo

View file

@ -495,19 +495,13 @@ echo 1; exit 1
1
2
3
4
5
parallel: This job failed:
echo 1; exit 1
parallel: This job failed:
echo 2; exit 2
parallel: This job failed:
echo 3; exit 3
parallel: This job failed:
echo 4; exit 4
parallel: Starting no more jobs. Waiting for 1 jobs to finish.
parallel: This job failed:
echo 5; exit 5
echo 3; exit 3
parallel -k --retries 3 'echo tried {} >>/tmp/runs; echo completed {}; exit {}' ::: 1 2 0
cat /tmp/runs
completed 1
@ -939,7 +933,7 @@ This helps funding further development; and it won't cost you a cent.
If you pay 10000 EUR you should feel free to use GNU Parallel without citing.
parallel --version
GNU parallel 20150502
GNU parallel VERSION
Copyright (C) 2007,2008,2009,2010,2011,2012,2013,2014,2015 Ole Tange
and Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
@ -950,9 +944,9 @@ Web site: http://www.gnu.org/software/parallel
When using programs that use GNU Parallel to process data for publication
please cite as described in 'parallel --bibtex'.
parallel --minversion 20130722 && echo Your version is at least 20130722.
20150502
Your version is at least 20130722.
parallel --minversion VERSION && echo Your version is at least VERSION.
VERSION
Your version is at least VERSION.
parallel --bibtex
Academic tradition requires you to cite works you base your article on.
When using programs that use GNU Parallel to process data for publication