Released as 20200122 ('Soleimani')

This commit is contained in:
Ole Tange 2020-01-22 18:03:29 +01:00
parent 033dd3df91
commit 0fe9be5f00
14 changed files with 144 additions and 119 deletions

19
NEWS
View file

@ -1,3 +1,22 @@
20200122
* --blocktimeout dur - Time out for reading block when using --pipe. If it takes longer than dur to read a full block, use the partial block read so far.
* Bug fixes and man page updates.
News about GNU Parallel:
* GNU Parallel course in Copenhagen https://www.prosa.dk/nc/arrangementer/arrangement/gnu-parallel-med-ole-tange/
* GNU Parallel course in Århus https://www.prosa.dk/nc/arrangementer/arrangement/gnu-parallel-og-parallelisering-i-unix-shellen/
* GNU Parallel pour accélérer vos process sous Linux https://www.yvonh.com/gnu-parallel-pour-accelerer-vos-process-sous-linux/
* How to copy a file to multiple directories in Linux https://net2.com/how-to-copy-a-file-to-multiple-directories-in-linux/
* Running linux commands in parallel https://dev.to/voyeg3r/runing-linux-commands-in-parallel-4ff8
20191222
* GNU Parallel course in Copenhagen

20
README
View file

@ -57,11 +57,11 @@ document.
Full installation of GNU Parallel is as simple as:
wget https://ftpmirror.gnu.org/parallel/parallel-20191222.tar.bz2
wget https://ftpmirror.gnu.org/parallel/parallel-20191222.tar.bz2.sig
gpg parallel-20191222.tar.bz2.sig
bzip2 -dc parallel-20191222.tar.bz2 | tar xvf -
cd parallel-20191222
wget https://ftpmirror.gnu.org/parallel/parallel-20200122.tar.bz2
wget https://ftpmirror.gnu.org/parallel/parallel-20200122.tar.bz2.sig
gpg parallel-20200122.tar.bz2.sig
bzip2 -dc parallel-20200122.tar.bz2 | tar xvf -
cd parallel-20200122
./configure && make && sudo make install
@ -70,11 +70,11 @@ Full installation of GNU Parallel is as simple as:
If you are not root you can add ~/bin to your path and install in
~/bin and ~/share:
wget https://ftpmirror.gnu.org/parallel/parallel-20191222.tar.bz2
wget https://ftpmirror.gnu.org/parallel/parallel-20191222.tar.bz2.sig
gpg parallel-20191222.tar.bz2.sig
bzip2 -dc parallel-20191222.tar.bz2 | tar xvf -
cd parallel-20191222
wget https://ftpmirror.gnu.org/parallel/parallel-20200122.tar.bz2
wget https://ftpmirror.gnu.org/parallel/parallel-20200122.tar.bz2.sig
gpg parallel-20200122.tar.bz2.sig
bzip2 -dc parallel-20200122.tar.bz2 | tar xvf -
cd parallel-20200122
./configure --prefix=$HOME && make && make install
Or if your system lacks 'make' you can simply copy src/parallel

20
configure vendored
View file

@ -1,6 +1,6 @@
#! /bin/sh
# Guess values for system-dependent variables and create Makefiles.
# Generated by GNU Autoconf 2.69 for parallel 20191222.
# Generated by GNU Autoconf 2.69 for parallel 20200122.
#
# Report bugs to <bug-parallel@gnu.org>.
#
@ -579,8 +579,8 @@ MAKEFLAGS=
# Identity of this package.
PACKAGE_NAME='parallel'
PACKAGE_TARNAME='parallel'
PACKAGE_VERSION='20191222'
PACKAGE_STRING='parallel 20191222'
PACKAGE_VERSION='20200122'
PACKAGE_STRING='parallel 20200122'
PACKAGE_BUGREPORT='bug-parallel@gnu.org'
PACKAGE_URL=''
@ -1214,7 +1214,7 @@ if test "$ac_init_help" = "long"; then
# Omit some internal or obsolete options to make the list less imposing.
# This message is too long to be a string in the A/UX 3.1 sh.
cat <<_ACEOF
\`configure' configures parallel 20191222 to adapt to many kinds of systems.
\`configure' configures parallel 20200122 to adapt to many kinds of systems.
Usage: $0 [OPTION]... [VAR=VALUE]...
@ -1281,7 +1281,7 @@ fi
if test -n "$ac_init_help"; then
case $ac_init_help in
short | recursive ) echo "Configuration of parallel 20191222:";;
short | recursive ) echo "Configuration of parallel 20200122:";;
esac
cat <<\_ACEOF
@ -1357,7 +1357,7 @@ fi
test -n "$ac_init_help" && exit $ac_status
if $ac_init_version; then
cat <<\_ACEOF
parallel configure 20191222
parallel configure 20200122
generated by GNU Autoconf 2.69
Copyright (C) 2012 Free Software Foundation, Inc.
@ -1374,7 +1374,7 @@ cat >config.log <<_ACEOF
This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.
It was created by parallel $as_me 20191222, which was
It was created by parallel $as_me 20200122, which was
generated by GNU Autoconf 2.69. Invocation command line was
$ $0 $@
@ -2237,7 +2237,7 @@ fi
# Define the identity of the package.
PACKAGE='parallel'
VERSION='20191222'
VERSION='20200122'
cat >>confdefs.h <<_ACEOF
@ -2880,7 +2880,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1
# report actual input values of CONFIG_FILES etc. instead of their
# values after options handling.
ac_log="
This file was extended by parallel $as_me 20191222, which was
This file was extended by parallel $as_me 20200122, which was
generated by GNU Autoconf 2.69. Invocation command line was
CONFIG_FILES = $CONFIG_FILES
@ -2942,7 +2942,7 @@ _ACEOF
cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1
ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`"
ac_cs_version="\\
parallel config.status 20191222
parallel config.status 20200122
configured by $0, generated by GNU Autoconf 2.69,
with options \\"\$ac_cs_config\\"

View file

@ -1,4 +1,4 @@
AC_INIT([parallel], [20191222], [bug-parallel@gnu.org])
AC_INIT([parallel], [20200122], [bug-parallel@gnu.org])
AM_INIT_AUTOMAKE([-Wall -Werror foreign])
AC_CONFIG_HEADERS([config.h])
AC_CONFIG_FILES([

View file

@ -121,6 +121,7 @@ cp src/parallel.html ../parallel-web/parallel/man.html
pushd ../parallel-web/parallel
# Bug at Savannah makes this take 30 seconds
(cd ..; cvs -z3 -d:ext:tange@cvs.savannah.gnu.org:/web/parallel co parallel)
torsocks cvs up
torsocks cvs ci -m "New release"
pushd
@ -226,16 +227,21 @@ Quote of the month:
New in this release:
* --blocktimeout dur - Time out for reading block when using --pipe. If it takes longer than dur to read a full block, use the partial block read so far.
* Bug fixes and man page updates.
News about GNU Parallel:
* GNU Parallel course in Copenhagen https://www.prosa.dk/nc/arrangementer/arrangement/gnu-parallel-med-ole-tange/
* GNU Parallel course in Århus https://www.prosa.dk/nc/arrangementer/arrangement/gnu-parallel-og-parallelisering-i-unix-shellen/
* GNU Parallel pour accélérer vos process sous Linux https://www.yvonh.com/gnu-parallel-pour-accelerer-vos-process-sous-linux/
https://net2.com/how-to-copy-a-file-to-multiple-directories-in-linux/
https://dev.to/voyeg3r/runing-linux-commands-in-parallel-4ff8
* Bug fixes and man page updates.
* How to copy a file to multiple directories in Linux https://net2.com/how-to-copy-a-file-to-multiple-directories-in-linux/
* Running linux commands in parallel https://dev.to/voyeg3r/runing-linux-commands-in-parallel-4ff8
Get the book: GNU Parallel 2018 http://www.lulu.com/shop/ole-tange/gnu-parallel-2018/paperback/product-23558902.html

View file

@ -1,6 +1,5 @@
<directory name="parallel" rev="250" srcmd5="d9d5eb1ddddf86d5e9849aaa2dded6cf" vrev="1">
<entry md5="3625e700edd0fc7d4a1aa91e33186c65" mtime="1577026224" name="parallel-20191222.tar.bz2" size="2046618" />
<entry md5="b5a350e84015ecadd04f4f128b9c294d" mtime="1577026224" name="parallel.spec" size="4701" />
<entry md5="10b9245739aac9f617a52ce72c5dcda3" mtime="1577026225" name="parallel_20191222.dsc" size="556" />
<entry md5="8ad3aa7e04af10d84e4870f04ac1a0fe" mtime="1577026225" name="parallel_20191222.tar.gz" size="2234082" />
<directory name="parallel" rev="253" srcmd5="74028fefc957c14c278b5a6b22244884" vrev="3">
<entry md5="b953ae5c5522a159dbeef61ee8dd179a" mtime="1579712470" name="parallel-20200122.tar.bz2" size="1986560" />
<entry md5="14cfc90563578337df23c6fb998960c5" mtime="1579707035" name="parallel.spec" size="4701" />
<entry md5="7ff31b7d53458710a6b324072bbb0637" mtime="1579712476" name="parallel_20200122.tar.gz" size="2226398" />
</directory>

View file

@ -1,7 +1,7 @@
Summary: Shell tool for executing jobs in parallel
Name: parallel
Version: 20191222
Version: 20200122
Release: 1.3
License: GPL-3.0-or-later
Group: Productivity/File utilities

View file

@ -1,7 +1,7 @@
Summary: Shell tool for executing jobs in parallel
Name: parallel
Version: 20191222
Version: 20200122
Release: 1.3
License: GPL-3.0-or-later
Group: Productivity/File utilities

View file

@ -23,7 +23,7 @@
use strict;
use Getopt::Long;
$Global::progname="niceload";
$Global::version = 20191222;
$Global::version = 20200122;
Getopt::Long::Configure("bundling","require_order");
get_options_from_array(\@ARGV) || die_usage();
if($opt::version) {

View file

@ -567,13 +567,13 @@ See B<--pipe> and B<--pipepart> for use of this.
=item B<--bt> I<duration>
Time out for reading block when using B<--pipe>. If it takes longer
than I<duration> to generate a full block, use the partial block read
so far.
than I<duration> to read a full block, use the partial block read so
far.
I<duration> must be in whole seconds, but can be expressed as floats
postfixed with B<s>, B<m>, B<h>, or B<d> which would multiply the
float by 1, 60, 3600, or 86400. Thus these are equivalent: B<--timeout
100000> and B<--timeout 1d3.5h16.6m4s>.
float by 1, 60, 3600, or 86400. Thus these are equivalent:
B<--blocktimeout 100000> and B<--blocktimeout 1d3.5h16.6m4s>.
@ -670,6 +670,15 @@ Even quoted newlines are parsed correctly:
When used with B<--pipe> only pass full CSV-records.
=item B<--delay> I<mytime>
Delay starting next job by I<mytime>. GNU B<parallel> will pause
I<mytime> after starting each job. I<mytime> is normally in seconds,
but can be floats postfixed with B<s>, B<m>, B<h>, or B<d> which would
multiply the float by 1, 60, 3600, or 86400. Thus these are
equivalent: B<--delay 100000> and B<--delay 1d3.5h16.6m4s>.
=item B<--delimiter> I<delim>
=item B<-d> I<delim>
@ -691,22 +700,6 @@ Use the replacement string I<replace-str> instead of B<{//}> for
dirname of input line.
=item B<-E> I<eof-str>
Set the end of file string to I<eof-str>. If the end of file string
occurs as a line of input, the rest of the input is not read. If
neither B<-E> nor B<-e> is used, no end of file string is used.
=item B<--delay> I<mytime>
Delay starting next job by I<mytime>. GNU B<parallel> will pause
I<mytime> after starting each job. I<mytime> is normally in seconds,
but can be floats postfixed with B<s>, B<m>, B<h>, or B<d> which would
multiply the float by 1, 60, 3600, or 86400. Thus these are
equivalent: B<--delay 100000> and B<--delay 1d3.5h16.6m4s>.
=item B<--dry-run>
Print the job to run on stdout (standard output), but do not run the
@ -717,6 +710,13 @@ literally, though, as the job may be scheduled on another computer or
the local computer if : is in the list.
=item B<-E> I<eof-str>
Set the end of file string to I<eof-str>. If the end of file string
occurs as a line of input, the rest of the input is not read. If
neither B<-E> nor B<-e> is used, no end of file string is used.
=item B<--eof>[=I<eof-str>]
=item B<-e>[I<eof-str>]
@ -1331,21 +1331,6 @@ run remote and are very fast to run. This is disabled for sshlogins
that specify their own ssh command.
=item B<--xargs>
Multiple arguments. Insert as many arguments as the command line
length permits.
If B<{}> is not used the arguments will be appended to the
line. If B<{}> is used multiple times each B<{}> will be replaced
with all the arguments.
Support for B<--xargs> with B<--sshlogin> is limited and may fail.
See also B<-X> for context replace. If in doubt use B<-X> as that will
most likely do what is needed.
=item B<-m>
Multiple arguments. Insert as many arguments as the command line
@ -1390,6 +1375,43 @@ This is useful for scripts that depend on features only available from
a certain version of GNU B<parallel>.
=item B<--max-args>=I<max-args>
=item B<-n> I<max-args>
Use at most I<max-args> arguments per command line. Fewer than
I<max-args> arguments will be used if the size (see the B<-s> option)
is exceeded, unless the B<-x> option is given, in which case
GNU B<parallel> will exit.
B<-n 0> means read one argument, but insert 0 arguments on the command
line.
Implies B<-X> unless B<-m> is set.
=item B<--max-replace-args>=I<max-args>
=item B<-N> I<max-args>
Use at most I<max-args> arguments per command line. Like B<-n> but
also makes replacement strings B<{1}> .. B<{>I<max-args>B<}> that
represents argument 1 .. I<max-args>. If too few args the B<{>I<n>B<}> will
be empty.
B<-N 0> means read one argument, but insert 0 arguments on the command
line.
This will set the owner of the homedir to the user:
tr ':' '\n' < /etc/passwd | parallel -N7 chown {1} {6}
Implies B<-X> unless B<-m> or B<--pipe> is set.
When used with B<--pipe> B<-N> is the number of records to read. This
is somewhat slower than B<--block>.
=item B<--nonall>
B<--onall> with no arguments. Run the command on all computers given
@ -1427,9 +1449,9 @@ of each job is saved in a file and the filename is then printed.
See also: B<--results>
=item B<--pipe>
=item B<--pipe> (alpha testing)
=item B<--spreadstdin>
=item B<--spreadstdin> (alpha testing)
Spread input to jobs on stdin (standard input). Read a block of data
from stdin (standard input) and give one block of data as input to one
@ -1539,43 +1561,6 @@ B<--progress> on a running GNU B<parallel> process.
See also B<--eta> and B<--bar>.
=item B<--max-args>=I<max-args>
=item B<-n> I<max-args>
Use at most I<max-args> arguments per command line. Fewer than
I<max-args> arguments will be used if the size (see the B<-s> option)
is exceeded, unless the B<-x> option is given, in which case
GNU B<parallel> will exit.
B<-n 0> means read one argument, but insert 0 arguments on the command
line.
Implies B<-X> unless B<-m> is set.
=item B<--max-replace-args>=I<max-args>
=item B<-N> I<max-args>
Use at most I<max-args> arguments per command line. Like B<-n> but
also makes replacement strings B<{1}> .. B<{>I<max-args>B<}> that
represents argument 1 .. I<max-args>. If too few args the B<{>I<n>B<}> will
be empty.
B<-N 0> means read one argument, but insert 0 arguments on the command
line.
This will set the owner of the homedir to the user:
tr ':' '\n' < /etc/passwd | parallel -N7 chown {1} {6}
Implies B<-X> unless B<-m> or B<--pipe> is set.
When used with B<--pipe> B<-N> is the number of records to read. This
is somewhat slower than B<--block>.
=item B<--max-line-length-allowed>
Print the maximal number of characters allowed on the command line and
@ -2860,6 +2845,21 @@ See also B<-m>.
Exit if the size (see the B<-s> option) is exceeded.
=item B<--xargs>
Multiple arguments. Insert as many arguments as the command line
length permits.
If B<{}> is not used the arguments will be appended to the
line. If B<{}> is used multiple times each B<{}> will be replaced
with all the arguments.
Support for B<--xargs> with B<--sshlogin> is limited and may fail.
See also B<-X> for context replace. If in doubt use B<-X> as that will
most likely do what is needed.
=back
@ -5112,13 +5112,13 @@ version.
A minimal, complete, and verifiable example (See description on
http://stackoverflow.com/help/mcve).
It should be a complete example that others can run that shows the problem
including all files needed to run the example. This should preferably
be small and simple, so try to remove as many options as possible. A
combination of B<yes>, B<seq>, B<cat>, B<echo>, and B<sleep> can
reproduce most errors. If your example requires large files, see if
you can make them by something like B<seq 1000000> > B<file> or B<yes
| head -n 10000000> > B<file>.
It should be a complete example that others can run that shows the
problem including all files needed to run the example. This should
preferably be small and simple, so try to remove as many options as
possible. A combination of B<yes>, B<seq>, B<cat>, B<echo>, and
B<sleep> can reproduce most errors. If your example requires large
files, see if you can make them by something like B<seq 100000000> >
B<bigfile> or B<yes | head -n 1000000000> > B<file>.
If your example requires remote execution, see if you can use
B<localhost> - maybe using another login.

View file

@ -136,13 +136,13 @@ $SERVER1 and $SERVER2 must work.
SERVER1=server.example.com
SERVER2=server2.example.net
So you must be able to do this:
So you must be able to do this without entering a password:
ssh $SERVER1 echo works
ssh $SERVER2 echo works
It can be setup by running 'ssh-keygen -t dsa; ssh-copy-id $SERVER1'
and using an empty passphrase.
and using an empty passphrase, or you can use B<ssh-agent>.
=back

View file

@ -574,7 +574,7 @@ $Global::Initfile && unlink $Global::Initfile;
exit ($err);
sub parse_options {
$Global::version = 20191222;
$Global::version = 20200122;
$Global::progname = 'sql';
# This must be done first as this may exec myself

View file

@ -52,7 +52,7 @@ short: testlocalshort
true
testlocalshort: ../src/parallel tests-to-run/*local* wanted-results/*local* prereqlocal installparallel
TRIES=1 time bash Start.sh local mem'|'100s
TRIES=1 time bash Start.sh local 'mem|100s|300s'
# sql
testdb: ../src/parallel tests-to-run/*sql* wanted-results/*sql* prereqdb

View file

@ -44,8 +44,9 @@ doit() {
echo MAXTIME=$MAXTIME RETRIES=$RETRIES MAXPROC=$MAXPROC MAXINNERPROC=$MAXINNERPROC
echo '### Filter out working servers'
# syllable often gives false positive
# pidora and syllable often gives false positive
parallel --timeout $MAXTIME -j10 ssh syllable true ::: {1..10} 2>/dev/null >/dev/null &
parallel --timeout $MAXTIME -j10 ssh pidora true ::: {1..10} 2>/dev/null >/dev/null &
POLAR_ALL="`bin/parallel --memfree 100m -j0 -k --timeout 10 echo {} ::: $P`"
POLAR="`bin/parallel --memfree 100m -j0 -k --timeout 10 $PARALLEL_SSH {} echo {} ::: $P`"
diff <(echo "$POLAR_ALL") <(echo "$POLAR")