Released as 20150622 ('Løkke')

This commit is contained in:
Ole Tange 2015-06-22 21:23:26 +02:00
parent 210e9db387
commit 2ea31ecb17
10 changed files with 172 additions and 53 deletions

144
NEWS
View file

@ -1,38 +1,148 @@
20150622
* --halt has been rewritten completely. You can now combine
percentages with success or fail. See the man page.
* Exit values 102..254 have been removed. 101 means more than 100 jobs
failed.
* Killing through --timeout, --memfree, or --halt is now done as a
process group.
* --termseq determines which signals are sent when a job is killed.
* An empty argument would previously cause no string to be
inserted. This is now changed to '' being inserted, thus prepending
a space to the output of: parallel echo {} b ::: ''
* $PARALLEL_ENV can now be set to an environment prepending the
command. Used in env_parallel as mentioned in the manpage.
* --retry-failed will retry all failed jobs in a joblog. It will
ignore any command given.
* --ssh and $PARALLEL_SSH can be used to set the command used for
ssh. The command is assumed to behave the same way as ssh.
* --fifo now works in csh, too.
* Q(...) can be used in {= =} to shell quote a string.
* GNU Parallel was cited in: Contrasting regional architectures of
schizophrenia and other complex diseases using fast variance
components analysis
http://biorxiv.org/content/biorxiv/early/2015/06/05/016527.full.pdf
* GNU Parallel was cited in: DockBench: An Integrated Informatic
Platform Bridging the Gap between the Robust Validation of Docking
Protocols and Virtual Screening Simulations
http://www.mdpi.com/1420-3049/20/6/9977
* GNU Parallel was cited in: Swedes Online: You Are More Tracked Than
You Think
http://www.diva-portal.org/smash/get/diva2:807623/FULLTEXT01.pdf
* GNU Parallel was cited in: Tutorial: Lorenz-Mie theory for 2D
scattering and resonance calculations
http://arxiv.org/pdf/1505.07691.pdf
* GNU Parallel was cited in: A quantitative assessment of the Hadoop
framework for analyzing massively parallel DNA sequencing data
http://link.springer.com/article/10.1186/s13742-015-0058-5
* GNU Parallel was cited in: A composite genome approach to identify
phylogenetically informative data from next-generation sequencing
http://www.biomedcentral.com/content/pdf/s12859-015-0632-y.pdf
* GNU Parallel was cited in: Tutorial: Lorenz-Mie theory for 2D
scattering and resonance calculations
http://arxiv.org/pdf/1505.07691.pdf
* GNU Parallel is used in: taxator-tk
http://algbio.cs.uni-duesseldorf.de/webapps/wa-download/
* GNU Parallel is used in: https://github.com/d2207197/local-mapreduce
* GNU Parallel is used in: wiki-edit-news https://github.com/albert-decatur/wiki-edit-news
* GNU Parallel is used in: A Fingerprint Identification System
https://curve.carleton.ca/system/files/theses/28733.pdf
* Job ad asking for GNU Parallel experience:
http://searchjobs.intel.com/gdansk-pol/software-validation-engineer/63A06826DAF24797AB414DC146201C2E/job/
* Using BLAT http://wangzhengyuan.blogspot.dk/2015/06/using-blat.html
* Bug fixes and man page updates.
20150522
* Security: The security issue for --sshlogin + --fifo/--cat has been fixed. Thereby all issues with http://lists.gnu.org/archive/html/parallel/2015-04/msg00045.html have been fixed.
* Security: The security issue for --sshlogin + --fifo/--cat has been
fixed. Thereby all issues with
http://lists.gnu.org/archive/html/parallel/2015-04/msg00045.html
have been fixed.
* Security: After further security analysis the issue fixed in 20150422 also fixed the problem for --tmux.
* Security: After further security analysis the issue fixed in
20150422 also fixed the problem for --tmux.
* GNU Parallel was cited in: CIDER: a pipeline for detecting waves of coordinated transcriptional regulation in gene expression time-course data http://biorxiv.org/content/biorxiv/early/2015/03/17/012518.full.pdf
* GNU Parallel was cited in: CIDER: a pipeline for detecting waves of
coordinated transcriptional regulation in gene expression
time-course data
http://biorxiv.org/content/biorxiv/early/2015/03/17/012518.full.pdf
* GNU Parallel was cited in: Building Genomic Analysis Pipelines in a Hackathon Setting with Bioinformatician Teams: DNA-seq, Epigenomics, Metagenomics and RNA-seq http://biorxiv.org/content/biorxiv/early/2015/05/05/018085.full.pdf
* GNU Parallel was cited in: Building Genomic Analysis Pipelines in a
Hackathon Setting with Bioinformatician Teams: DNA-seq, Epigenomics,
Metagenomics and RNA-seq
http://biorxiv.org/content/biorxiv/early/2015/05/05/018085.full.pdf
* GNU Parallel was cited in: Toward Enhanced Metadata Quality of Large-Scale Digital Libraries: Estimating Volume Time Range https://www.ideals.illinois.edu/bitstream/handle/2142/73656/186_ready.pdf
* GNU Parallel was cited in: Toward Enhanced Metadata Quality of
Large-Scale Digital Libraries: Estimating Volume Time Range
https://www.ideals.illinois.edu/bitstream/handle/2142/73656/186_ready.pdf
* GNU Parallel was cited in: Sequencing the cap-snatching repertoire of H1N1 influenza provides insight into the mechanism of viral transcription initiation http://nar.oxfordjournals.org/content/early/2015/04/20/nar.gkv333.full.pdf
* GNU Parallel was cited in: Sequencing the cap-snatching repertoire
of H1N1 influenza provides insight into the mechanism of viral
transcription initiation
http://nar.oxfordjournals.org/content/early/2015/04/20/nar.gkv333.full.pdf
* GNU Parallel was cited in: Genome assembly using Nanopore-guided long and error-free DNA reads http://www.biomedcentral.com/content/pdf/s12864-015-1519-z.pdf
* GNU Parallel was cited in: Genome assembly using Nanopore-guided
long and error-free DNA reads
http://www.biomedcentral.com/content/pdf/s12864-015-1519-z.pdf
* GNU Parallel was cited in: Contrasting regional architectures of schizophrenia and other complex diseases using fast variance components analysis http://biorxiv.org/content/biorxiv/early/2015/03/13/016527.full.pdf
* GNU Parallel was cited in: Contrasting regional architectures of
schizophrenia and other complex diseases using fast variance
components analysis
http://biorxiv.org/content/biorxiv/early/2015/03/13/016527.full.pdf
* GNU Parallel was used (unfortunately with wrong citation) in: Comparing the CarbonTracker and TM5-4DVar data assimilation systems for CO2 surface flux inversions http://www.atmos-chem-phys-discuss.net/15/8883/2015/acpd-15-8883-2015-discussion.html
* GNU Parallel was used (unfortunately with wrong citation) in:
Comparing the CarbonTracker and TM5-4DVar data assimilation systems
for CO2 surface flux inversions
http://www.atmos-chem-phys-discuss.net/15/8883/2015/acpd-15-8883-2015-discussion.html
* GNU Parallel was used in: Gene Set Omic Analysis (GSOA) method https://bitbucket.org/srp33/gsoa
* GNU Parallel was used in: Gene Set Omic Analysis (GSOA) method
https://bitbucket.org/srp33/gsoa
* A Quick and Neat :) Orchestrator using GNU Parallel http://www.elsotanillo.net/2015/05/a-quick-and-neat-orchestrator-using-gnu-parallel/
* A Quick and Neat :) Orchestrator using GNU Parallel
http://www.elsotanillo.net/2015/05/a-quick-and-neat-orchestrator-using-gnu-parallel/
* Execute commands on multiple computers using GNU Parallel (setting up a cluster on the cheap) https://spectraldifferences.wordpress.com/2015/04/26/execute-commands-on-multiple-computers-using-gnu-parallel-setting-up-a-cluster-on-the-cheap/
* Execute commands on multiple computers using GNU Parallel (setting
up a cluster on the cheap)
https://spectraldifferences.wordpress.com/2015/04/26/execute-commands-on-multiple-computers-using-gnu-parallel-setting-up-a-cluster-on-the-cheap/
* Functions and GNU parallel for effective cluster load management http://genomespot.blogspot.dk/2015/04/functions-and-gnu-parallel-for.html
* Functions and GNU parallel for effective cluster load management
http://genomespot.blogspot.dk/2015/04/functions-and-gnu-parallel-for.html
* Use parallel processing to save time importing databases http://drupalsun.com/node/41854
* Use parallel processing to save time importing databases
http://drupalsun.com/node/41854
* Run multiple ssh commands in parallel with GNU Parallel http://www.ameir.net/blog/archives/380-run-multiple-ssh-commands-in-parallel-with-gnu-parallel.html
* Run multiple ssh commands in parallel with GNU Parallel
http://www.ameir.net/blog/archives/380-run-multiple-ssh-commands-in-parallel-with-gnu-parallel.html
* Importing huge databases faster https://www.lullabot.com/blog/article/importing-huge-databases-faster
* Importing huge databases faster
https://www.lullabot.com/blog/article/importing-huge-databases-faster
* Run multiple ssh commands in parallel with GNU Parallel https://www.ameir.net/blog/archives/380-run-multiple-ssh-commands-in-parallel-with-gnu-parallel.html/comment-page-1
* Run multiple ssh commands in parallel with GNU Parallel
https://www.ameir.net/blog/archives/380-run-multiple-ssh-commands-in-parallel-with-gnu-parallel.html/comment-page-1
* Parallel? Gnu parallel! https://debian.pro/1834

12
README
View file

@ -40,9 +40,9 @@ document.
Full installation of GNU Parallel is as simple as:
wget http://ftpmirror.gnu.org/parallel/parallel-20150607.tar.bz2
bzip2 -dc parallel-20150607.tar.bz2 | tar xvf -
cd parallel-20150607
wget http://ftpmirror.gnu.org/parallel/parallel-20150622.tar.bz2
bzip2 -dc parallel-20150622.tar.bz2 | tar xvf -
cd parallel-20150622
./configure && make && sudo make install
@ -51,9 +51,9 @@ Full installation of GNU Parallel is as simple as:
If you are not root you can add ~/bin to your path and install in
~/bin and ~/share:
wget http://ftpmirror.gnu.org/parallel/parallel-20150607.tar.bz2
bzip2 -dc parallel-20150607.tar.bz2 | tar xvf -
cd parallel-20150607
wget http://ftpmirror.gnu.org/parallel/parallel-20150622.tar.bz2
bzip2 -dc parallel-20150622.tar.bz2 | tar xvf -
cd parallel-20150622
./configure --prefix=$HOME && make && make install
Or if your system lacks 'make' you can simply copy src/parallel

20
configure vendored
View file

@ -1,6 +1,6 @@
#! /bin/sh
# Guess values for system-dependent variables and create Makefiles.
# Generated by GNU Autoconf 2.69 for parallel 20150607.
# Generated by GNU Autoconf 2.69 for parallel 20150622.
#
# Report bugs to <bug-parallel@gnu.org>.
#
@ -579,8 +579,8 @@ MAKEFLAGS=
# Identity of this package.
PACKAGE_NAME='parallel'
PACKAGE_TARNAME='parallel'
PACKAGE_VERSION='20150607'
PACKAGE_STRING='parallel 20150607'
PACKAGE_VERSION='20150622'
PACKAGE_STRING='parallel 20150622'
PACKAGE_BUGREPORT='bug-parallel@gnu.org'
PACKAGE_URL=''
@ -1203,7 +1203,7 @@ if test "$ac_init_help" = "long"; then
# Omit some internal or obsolete options to make the list less imposing.
# This message is too long to be a string in the A/UX 3.1 sh.
cat <<_ACEOF
\`configure' configures parallel 20150607 to adapt to many kinds of systems.
\`configure' configures parallel 20150622 to adapt to many kinds of systems.
Usage: $0 [OPTION]... [VAR=VALUE]...
@ -1269,7 +1269,7 @@ fi
if test -n "$ac_init_help"; then
case $ac_init_help in
short | recursive ) echo "Configuration of parallel 20150607:";;
short | recursive ) echo "Configuration of parallel 20150622:";;
esac
cat <<\_ACEOF
@ -1345,7 +1345,7 @@ fi
test -n "$ac_init_help" && exit $ac_status
if $ac_init_version; then
cat <<\_ACEOF
parallel configure 20150607
parallel configure 20150622
generated by GNU Autoconf 2.69
Copyright (C) 2012 Free Software Foundation, Inc.
@ -1362,7 +1362,7 @@ cat >config.log <<_ACEOF
This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.
It was created by parallel $as_me 20150607, which was
It was created by parallel $as_me 20150622, which was
generated by GNU Autoconf 2.69. Invocation command line was
$ $0 $@
@ -2225,7 +2225,7 @@ fi
# Define the identity of the package.
PACKAGE='parallel'
VERSION='20150607'
VERSION='20150622'
cat >>confdefs.h <<_ACEOF
@ -2867,7 +2867,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1
# report actual input values of CONFIG_FILES etc. instead of their
# values after options handling.
ac_log="
This file was extended by parallel $as_me 20150607, which was
This file was extended by parallel $as_me 20150622, which was
generated by GNU Autoconf 2.69. Invocation command line was
CONFIG_FILES = $CONFIG_FILES
@ -2929,7 +2929,7 @@ _ACEOF
cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1
ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`"
ac_cs_version="\\
parallel config.status 20150607
parallel config.status 20150622
configured by $0, generated by GNU Autoconf 2.69,
with options \\"\$ac_cs_config\\"

View file

@ -1,4 +1,4 @@
AC_INIT([parallel], [20150607], [bug-parallel@gnu.org])
AC_INIT([parallel], [20150622], [bug-parallel@gnu.org])
AM_INIT_AUTOMAKE([-Wall -Werror foreign])
AC_CONFIG_HEADERS([config.h])
AC_CONFIG_FILES([

View file

@ -4,7 +4,7 @@
Check that documentation is updated (compare to web):
Fixet for 20150522
Fixet for 20150622
git diff last-release-commit
Unmodified beta since last version => production
Unmodified alpha since last version => beta
@ -213,6 +213,8 @@ Subject: GNU Parallel 20150622 ('Løkke') released
GNU Parallel 20150622 ('Løkke') has been released. It is available for download at: http://ftp.gnu.org/gnu/parallel/
A performance chart has been made to compare the performance of different versions: http://www.gnu.org/software/parallel/boxplot-j2-3.0ghz-3000-1000.pdf The recent jump is due to every job being wrapped in a Perl-script doing setpgrp.
Haiku of the month:
Programs very slow.
@ -222,12 +224,16 @@ Haiku of the month:
New in this release:
* An empty argument would previously cause no string to be inserted. This is now changed to '' being inserted, thus prepending a space to the output of: parallel echo {} b ::: ''
* --halt has been rewritten completely. You can now combine percentages with success or fail. See the man page.
* Exit values 102..254 have been removed. 101 means more than 100 jobs failed.
* Killing through --timeout, --memfree, or --halt is now done as a process group (whence the setpgrp wrapper).
* --termseq determines which signals are sent when a job is killed.
* An empty argument would previously cause no string to be inserted. This is now changed to '' being inserted, thus prepending a space to the output of: parallel echo {} b ::: ''
* $PARALLEL_ENV can now be set to an environment prepending the command. Used in env_parallel as mentioned in the manpage.
* --retry-failed will retry all failed jobs in a joblog. It will ignore any command given.
@ -236,7 +242,7 @@ New in this release:
* --fifo now works in csh, too.
* Killing through --timeout, --memfree, or --halt is now done as a process group.
* Q(...) can be used in {= =} to shell quote a string.
* GNU Parallel was cited in: Contrasting regional architectures of schizophrenia and other complex diseases using fast variance components analysis http://biorxiv.org/content/biorxiv/early/2015/06/05/016527.full.pdf
@ -256,12 +262,12 @@ New in this release:
* <<Afventer updateret publisering>> GNU Parallel was used in: Large Scale Author Name Disambiguation in Digital Libraries http://ieeexplore.ieee.org/xpl/abstractReferences.jsp?tp=&arnumber=7004487&url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D7004487
* <<kontaktet 2015-05-09>> GNU Parallel was used (unfortunately with wrong citation) in: TADSim: Discrete Event-Based Performance Prediction for Temperature-Accelerated Dynamics http://vruehle.de/publications/2015c.pdf
* taxator-tk http://algbio.cs.uni-duesseldorf.de/webapps/wa-download/ (check it)
* <<kontaktet 2015-06-22>> GNU Parallel was used (unfortunately with wrong citation) in: TADSim: Discrete Event-Based Performance Prediction for Temperature-Accelerated Dynamics http://vruehle.de/publications/2015c.pdf
* << Update forventet juni Rachel har lige svaret >> GNU Parallel was used in: SISRS: Site Identification from Short Read Sequences https://github.com/rachelss/SISRS/
* GNU Parallel is used in: taxator-tk http://algbio.cs.uni-duesseldorf.de/webapps/wa-download/
* GNU Parallel is used in: https://github.com/d2207197/local-mapreduce
* GNU Parallel is used in: wiki-edit-news https://github.com/albert-decatur/wiki-edit-news

View file

@ -1,6 +1,6 @@
Summary: Shell tool for executing jobs in parallel
Name: parallel
Version: 20150522
Version: 20150622
Release: 1
License: GPL
Group: Productivity/File utilities

View file

@ -24,7 +24,7 @@
use strict;
use Getopt::Long;
$Global::progname="niceload";
$Global::version = 20150607;
$Global::version = 20150622;
Getopt::Long::Configure("bundling","require_order");
get_options_from_array(\@ARGV) || die_usage();
if($opt::version) {

View file

@ -1079,7 +1079,7 @@ sub parse_options {
sub init_globals {
# Defaults:
$Global::version = 20150620;
$Global::version = 20150622;
$Global::progname = 'parallel';
$Global::infinity = 2**31;
$Global::debug = 0;
@ -6429,6 +6429,7 @@ sub wrapped {
if($ENV{'PARALLEL_ENV'}) {
# If $PARALLEL_ENV set, put that in front of the command
# Used for importing functions for fish
# Map \001 to \n to make it easer to quote \n in $PARALLEL_ENV
$ENV{'PARALLEL_ENV'} =~ s/\001/\n/g;
$command = $ENV{'PARALLEL_ENV'}."\n".$command;
}
@ -7074,8 +7075,9 @@ sub start {
local (*OUT,*ERR);
open OUT, '>&', $stdout_fh or ::die_bug("Can't dup STDOUT: $!");
open ERR, '>&', $stderr_fh or ::die_bug("Can't dup STDERR: $!");
$pid = ::open3("<&IN", ">&OUT", ">&ERR",
"exec $Global::shell -c ".::shell_quote_scalar_default($command)) ||
$pid = ::open3("<&IN", ">&OUT", ">&ERR",
"exec $Global::shell -c ".
::shell_quote_scalar_default($command)) ||
::die_bug("open3-/dev/tty");
$Global::tty_taken = $pid;
close $devtty_fh;

View file

@ -306,7 +306,8 @@ Replace with calculated I<perl expression>. B<$_> will contain the
same as B<{}>. After evaluating I<perl expression> B<$_> will be used
as the value. It is recommended to only change $_ but you have full
access to all of GNU B<parallel>'s internal functions and data
structures.
structures. You can use the function B<Q(>I<string>B<)> to shell quote
a string.
The B<{=>I<perl expression>B<=}> must be given as a single string.
@ -501,7 +502,7 @@ I<size> defaults to 1M.
See B<--pipe> and B<--pipepart> for use of this.
=item B<--cat>
=item B<--cat> (alpha testing)
Create a temporary file with content. Normally B<--pipe>/B<--pipepart>
will give data to the program on stdin (standard input). With B<--cat>
@ -694,7 +695,7 @@ Implies B<--semaphore>.
See also B<--bg>, B<man sem>.
=item B<--fifo>
=item B<--fifo> (alpha testing)
Create a temporary fifo with content. Normally B<--pipe> and
B<--pipepart> will give data to the program on stdin (standard
@ -1735,7 +1736,7 @@ Does not run the command but quotes it. Useful for making quoted
composed commands for GNU B<parallel>.
=item B<--shuf> (alpha testing)
=item B<--shuf> (beta testing)
Shuffle jobs. When having multiple input sources it is hard to
randomize jobs. --shuf will generate all jobs, and shuffle them before
@ -1919,8 +1920,8 @@ B<--tagstring> is ignored when using B<-u>, B<--onall>, and B<--nonall>.
=item B<--termseq> I<sequence> (alpha testing)
Termination sequence. When a job is killed due to B<--timeout>,
B<--memfree>, or abnormal termination of GNU B<parallel>, I<sequence>
determines how the job is killed. The default is:
B<--memfree>, B<--halt>, or abnormal termination of GNU B<parallel>,
I<sequence> determines how the job is killed. The default is:
TERM,200,TERM,100,TERM,50,KILL,25

View file

@ -566,7 +566,7 @@ $Global::Initfile && unlink $Global::Initfile;
exit ($err);
sub parse_options {
$Global::version = 20150607;
$Global::version = 20150622;
$Global::progname = 'sql';
# This must be done first as this may exec myself