tange/parallel
1
0
Fork 0
mirror of https://git.savannah.gnu.org/git/parallel.git synced 2024-11-28 08:57:56 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Released as 20210722 ('Blue Unity')

Browse source
This commit is contained in:
Ole Tange 2021-07-22 22:31:06 +02:00
parent 56693a99ac
commit 80544cbfc8
28 changed files with 176 additions and 117 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

17
NEWS
View file

@ -1,3 +1,20 @@
20210722
New in this release:
* --results no longer prints the result to standard output (stdout) as voted in https://lists.gnu.org/archive/html/parallel/2020-12/msg00003.html
* parset supports associative arrays in bash, ksh, zsh.
* Online HTML is now generated by Sphinx.
* Bug fixes and man page updates.
News about GNU Parallel:
* Cleaning Up Scanned Documents with Open Source Tools https://kaerumy.medium.com/cleaning-up-scanned-documents-with-open-source-tools-9d87e15305b
20210622 20210622
New in this release: New in this release:

24
README
View file

@ -57,11 +57,11 @@ document.
Full installation of GNU Parallel is as simple as: Full installation of GNU Parallel is as simple as:
wget https://ftpmirror.gnu.org/parallel/parallel-20210622.tar.bz2 wget https://ftpmirror.gnu.org/parallel/parallel-20210722.tar.bz2
wget https://ftpmirror.gnu.org/parallel/parallel-20210622.tar.bz2.sig wget https://ftpmirror.gnu.org/parallel/parallel-20210722.tar.bz2.sig
gpg parallel-20210622.tar.bz2.sig gpg parallel-20210722.tar.bz2.sig
bzip2 -dc parallel-20210622.tar.bz2 | tar xvf - bzip2 -dc parallel-20210722.tar.bz2 | tar xvf -
cd parallel-20210622 cd parallel-20210722
./configure && make && sudo make install ./configure && make && sudo make install
@ -70,11 +70,11 @@ Full installation of GNU Parallel is as simple as:
If you are not root you can add ~/bin to your path and install in If you are not root you can add ~/bin to your path and install in
~/bin and ~/share: ~/bin and ~/share:
wget https://ftpmirror.gnu.org/parallel/parallel-20210622.tar.bz2 wget https://ftpmirror.gnu.org/parallel/parallel-20210722.tar.bz2
wget https://ftpmirror.gnu.org/parallel/parallel-20210622.tar.bz2.sig wget https://ftpmirror.gnu.org/parallel/parallel-20210722.tar.bz2.sig
gpg parallel-20210622.tar.bz2.sig gpg parallel-20210722.tar.bz2.sig
bzip2 -dc parallel-20210622.tar.bz2 | tar xvf - bzip2 -dc parallel-20210722.tar.bz2 | tar xvf -
cd parallel-20210622 cd parallel-20210722
./configure --prefix=$HOME && make && make install ./configure --prefix=$HOME && make && make install
Or if your system lacks 'make' you can simply copy src/parallel Or if your system lacks 'make' you can simply copy src/parallel
@ -122,8 +122,8 @@ will love you for it.
When using programs that use GNU Parallel to process data for When using programs that use GNU Parallel to process data for
publication please cite: publication please cite:
Tange, O. (2021, June 22). GNU Parallel 20210622 ('Protasevich'). Tange, O. (2021, July 22). GNU Parallel 20210722 ('Blue Unity').
Zenodo. https://doi.org/10.5281/zenodo.5013933 Zenodo. https://doi.org/10.5281/zenodo.5123056
Copyright (C) 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, Copyright (C) 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015,
2016, 2017, 2018, 2019, 2020, 2021 Ole Tange, http://ole.tange.dk and 2016, 2017, 2018, 2019, 2020, 2021 Ole Tange, http://ole.tange.dk and

20
configure vendored
View file

@ -1,6 +1,6 @@
#! /bin/sh #! /bin/sh
# Guess values for system-dependent variables and create Makefiles. # Guess values for system-dependent variables and create Makefiles.
# Generated by GNU Autoconf 2.69 for parallel 20210622. # Generated by GNU Autoconf 2.69 for parallel 20210722.
# #
# Report bugs to <bug-parallel@gnu.org>. # Report bugs to <bug-parallel@gnu.org>.
# #
@ -579,8 +579,8 @@ MAKEFLAGS=
# Identity of this package. # Identity of this package.
PACKAGE_NAME='parallel' PACKAGE_NAME='parallel'
PACKAGE_TARNAME='parallel' PACKAGE_TARNAME='parallel'
PACKAGE_VERSION='20210622' PACKAGE_VERSION='20210722'
PACKAGE_STRING='parallel 20210622' PACKAGE_STRING='parallel 20210722'
PACKAGE_BUGREPORT='bug-parallel@gnu.org' PACKAGE_BUGREPORT='bug-parallel@gnu.org'
PACKAGE_URL='' PACKAGE_URL=''
@ -1214,7 +1214,7 @@ if test "$ac_init_help" = "long"; then
# Omit some internal or obsolete options to make the list less imposing. # Omit some internal or obsolete options to make the list less imposing.
# This message is too long to be a string in the A/UX 3.1 sh. # This message is too long to be a string in the A/UX 3.1 sh.
cat <<_ACEOF cat <<_ACEOF
\`configure' configures parallel 20210622 to adapt to many kinds of systems. \`configure' configures parallel 20210722 to adapt to many kinds of systems.
Usage: $0 [OPTION]... [VAR=VALUE]... Usage: $0 [OPTION]... [VAR=VALUE]...
@ -1281,7 +1281,7 @@ fi
if test -n "$ac_init_help"; then if test -n "$ac_init_help"; then
case $ac_init_help in case $ac_init_help in
short | recursive ) echo "Configuration of parallel 20210622:";; short | recursive ) echo "Configuration of parallel 20210722:";;
esac esac
cat <<\_ACEOF cat <<\_ACEOF
@ -1357,7 +1357,7 @@ fi
test -n "$ac_init_help" && exit $ac_status test -n "$ac_init_help" && exit $ac_status
if $ac_init_version; then if $ac_init_version; then
cat <<\_ACEOF cat <<\_ACEOF
parallel configure 20210622 parallel configure 20210722
generated by GNU Autoconf 2.69 generated by GNU Autoconf 2.69
Copyright (C) 2012 Free Software Foundation, Inc. Copyright (C) 2012 Free Software Foundation, Inc.
@ -1374,7 +1374,7 @@ cat >config.log <<_ACEOF
This file contains any messages produced by compilers while This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake. running configure, to aid debugging if configure makes a mistake.
It was created by parallel $as_me 20210622, which was It was created by parallel $as_me 20210722, which was
generated by GNU Autoconf 2.69. Invocation command line was generated by GNU Autoconf 2.69. Invocation command line was
$ $0 $@ $ $0 $@
@ -2237,7 +2237,7 @@ fi
# Define the identity of the package. # Define the identity of the package.
PACKAGE='parallel' PACKAGE='parallel'
VERSION='20210622' VERSION='20210722'
cat >>confdefs.h <<_ACEOF cat >>confdefs.h <<_ACEOF
@ -2880,7 +2880,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1
# report actual input values of CONFIG_FILES etc. instead of their # report actual input values of CONFIG_FILES etc. instead of their
# values after options handling. # values after options handling.
ac_log=" ac_log="
This file was extended by parallel $as_me 20210622, which was This file was extended by parallel $as_me 20210722, which was
generated by GNU Autoconf 2.69. Invocation command line was generated by GNU Autoconf 2.69. Invocation command line was
CONFIG_FILES = $CONFIG_FILES CONFIG_FILES = $CONFIG_FILES
@ -2942,7 +2942,7 @@ _ACEOF
cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1
ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`" ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`"
ac_cs_version="\\ ac_cs_version="\\
parallel config.status 20210622 parallel config.status 20210722
configured by $0, generated by GNU Autoconf 2.69, configured by $0, generated by GNU Autoconf 2.69,
with options \\"\$ac_cs_config\\" with options \\"\$ac_cs_config\\"

2
configure.ac
View file

@ -1,4 +1,4 @@
AC_INIT([parallel], [20210622], [bug-parallel@gnu.org]) AC_INIT([parallel], [20210722], [bug-parallel@gnu.org])
AM_INIT_AUTOMAKE([-Wall -Werror foreign]) AM_INIT_AUTOMAKE([-Wall -Werror foreign])
AC_CONFIG_HEADERS([config.h]) AC_CONFIG_HEADERS([config.h])
AC_CONFIG_FILES([ AC_CONFIG_FILES([

7
doc/haikus
View file

@ -10,7 +10,6 @@ Quote of the month:
parallel is my new kink parallel is my new kink
-- python_noob_001@reddit -- python_noob_001@reddit
@GnuParallel the best thing ever and it's not up for debate #EOchat @GnuParallel the best thing ever and it's not up for debate #EOchat
-- Nathan Thomas @DrNASApants@twitter -- Nathan Thomas @DrNASApants@twitter
@ -20,9 +19,6 @@ Quote of the month:
GNU Parallel is one of the most helpful tools I've been using recently, and it's just something like: parallel -j4 'gzip {}' ::: folder/*.csv GNU Parallel is one of the most helpful tools I've been using recently, and it's just something like: parallel -j4 'gzip {}' ::: folder/*.csv
-- Milton Pividori @miltondp@twitter -- Milton Pividori @miltondp@twitter
We use gnu parallel now - and happier for it.
-- Ben Davies @benjamindavies@twitter
This is a fantastic tool, and I wish I had upgraded from xargs years ago! This is a fantastic tool, and I wish I had upgraded from xargs years ago!
-- Stuart Anderson -- Stuart Anderson
@ -117,6 +113,9 @@ https://negfeedback.blogspot.com/2020/05/indispensable-command-line-tools.html
=== Used === === Used ===
We use gnu parallel now - and happier for it.
-- Ben Davies @benjamindavies@twitter
GNU Parallel makes my life so much easier. GNU Parallel makes my life so much easier.
I'm glad I don't have to implement multi-threaded Python scripts on the regular. I'm glad I don't have to implement multi-threaded Python scripts on the regular.
-- Fredrick Brennan @fr_brennan@twitter -- Fredrick Brennan @fr_brennan@twitter

27
doc/release_new_version
View file

@ -22,7 +22,7 @@ With the same things that goes in the announce mail
Start termux on Android Start termux on Android
Start: wssh mac ping fi.dk Start: wssh mac ping fi.dk
cd testsuite; make mem; make cd testsuite; make mem; make polar; make long; make short
== Update version == == Update version ==
@ -101,9 +101,9 @@ lbry://@GnuParallel#4/parallel-20210322.tar.bz2
# Tags: gnu parallel software # Tags: gnu parallel software
file_path="parallel-20210622.tar.bz2" file_path="`pwd`/parallel-$YYYYMMDD.tar.bz2"
title="GNU Parallel $YYYYMMDD ('$SPCTAG') [stable]" title="GNU Parallel $YYYYMMDD ('$SPCTAG')"
name="GNU-Parallel-$YYYYMMDD-$SPCTAG" name="GNU-Parallel-$YYYYMMDD-$TAG"
author="Ole Tange" author="Ole Tange"
license="GNU GPLv3 or later" license="GNU GPLv3 or later"
thumbnail_url=https://www.gnu.org/software/parallel/logo-gray+black10000.png thumbnail_url=https://www.gnu.org/software/parallel/logo-gray+black10000.png
@ -144,8 +144,8 @@ http://www.gnu.org/software/parallel/parset.html
http://www.gnu.org/software/parallel/parsort.html http://www.gnu.org/software/parallel/parsort.html
http://www.gnu.org/software/parallel/niceload.html http://www.gnu.org/software/parallel/niceload.html
cp src/*.pdf src/*.html ../parallel-web/parallel cp -r src/*.pdf src/sphinx/build/html/* ../parallel-web/parallel
cp src/parallel.html ../parallel-web/parallel/man.html cp ../parallel-web/parallel/parallel.html ../parallel-web/parallel/man.html
pushd ../parallel-web/parallel pushd ../parallel-web/parallel
# Bug at Savannah makes this take 30 seconds # Bug at Savannah makes this take 30 seconds
@ -252,33 +252,24 @@ from:tange@gnu.org
to:parallel@gnu.org, bug-parallel@gnu.org to:parallel@gnu.org, bug-parallel@gnu.org
stable-bcc: Jesse Alama <jessealama@fastmail.fm> stable-bcc: Jesse Alama <jessealama@fastmail.fm>
Subject: GNU Parallel 20210722 ('Branson+VSS Unity/Bill Cosby/Moïse/Derek Chauvin') released <<[stable]>> Subject: GNU Parallel 20210722 ('Blue Unity') released <<[stable]>>
GNU Parallel 20210722 ('') <<[stable]>> has been released. It is available for download at: lbry://@GnuParallel:4 GNU Parallel 20210722 ('Blue Unity') <<[stable]>> has been released. It is available for download at: lbry://@GnuParallel:4
<<No new functionality was introduced so this is a good candidate for a stable release.>> <<No new functionality was introduced so this is a good candidate for a stable release.>>
Please help spreading GNU Parallel by making a testimonial video like Juan Sierra Pons: http://www.elsotanillo.net/wp-content/uploads/GnuParallel_JuanSierraPons.mp4
It does not have to be as detailed as Juan's. It is perfectly fine if you just say your name, and what field you are using GNU Parallel for.
Quote of the month: Quote of the month:
<<>> <<>>
New in this release: New in this release:
* --results no longer prints the result to standard output (stdout). <<>>
* parset can save in associative arrays in bash/ksh/zsh.
* Online HTML is now generated by Sphinx.
* Bug fixes and man page updates. * Bug fixes and man page updates.
News about GNU Parallel: News about GNU Parallel:
<<>> <<>>
Get the book: GNU Parallel 2018 http://www.lulu.com/shop/ole-tange/gnu-parallel-2018/paperback/product-23558902.html Get the book: GNU Parallel 2018 http://www.lulu.com/shop/ole-tange/gnu-parallel-2018/paperback/product-23558902.html

12
packager/obs/home:tange/parallel/.osc/_files
View file

@ -1,7 +1,7 @@
<directory name="parallel" rev="310" vrev="1" srcmd5="4097379f67c84ee069ce4f44248f5d36"> <directory name="parallel" rev="311" vrev="1" srcmd5="4db07b555bac8c95b748e4b27d7b8ec2">
<entry name="PKGBUILD" md5="3da589d7ca2d32a2f3706757a9823072" size="936" mtime="1624380019" /> <entry name="PKGBUILD" md5="efe5dba5d697d74243ed766562a7ced1" size="936" mtime="1626984745" />
<entry name="parallel-20210622.tar.bz2" md5="cde9fdb71b04e150f7b2c9fb2956d9b6" size="2250984" mtime="1624380019" /> <entry name="parallel-20210722.tar.bz2" md5="bc317c08d26f81c81ef764b8e421a0e1" size="2248888" mtime="1626984745" />
<entry name="parallel.spec" md5="ede78937a7985a62235441e7d08befb9" size="5630" mtime="1624380019" /> <entry name="parallel.spec" md5="4657b2121ab380c968d795952d8405a3" size="5630" mtime="1626984745" />
<entry name="parallel_20210622.dsc" md5="734afb9792b55742b326aeff8fd546d9" size="556" mtime="1624380019" /> <entry name="parallel_20210722.dsc" md5="abe14d022d2190009c72ba95cd1c332b" size="556" mtime="1626984746" />
<entry name="parallel_20210622.tar.gz" md5="484cd5c717fabf9c81ca64754e14a618" size="2491911" mtime="1624380020" /> <entry name="parallel_20210722.tar.gz" md5="f378ff0a5e4aeb3599747bf94f08b2b7" size="2490792" mtime="1626984746" />
</directory> </directory>

2
packager/obs/home:tange/parallel/.osc/parallel.spec
View file

@ -1,7 +1,7 @@
Summary: Shell tool for executing jobs in parallel Summary: Shell tool for executing jobs in parallel
Name: parallel Name: parallel
Version: 20210622 Version: 20210722
Release: 1.3 Release: 1.3
License: GPL-3.0-or-later License: GPL-3.0-or-later
Group: Productivity/File utilities Group: Productivity/File utilities

8
src/Makefile.am
View file

@ -2,8 +2,6 @@
# #
# SPDX-License-Identifier: GPL-3.0-or-later # SPDX-License-Identifier: GPL-3.0-or-later
SUBDIRS = sphinx
bin_SCRIPTS = parallel sql niceload parcat parset parsort \ bin_SCRIPTS = parallel sql niceload parcat parset parsort \
env_parallel env_parallel.ash env_parallel.bash \ env_parallel env_parallel.ash env_parallel.bash \
env_parallel.csh env_parallel.dash env_parallel.fish \ env_parallel.csh env_parallel.dash env_parallel.fish \
@ -344,6 +342,12 @@ parallel_cheat_bw.pdf: parallel_cheat_bw.fodt
sem: parallel sem: parallel
ln -fs parallel sem ln -fs parallel sem
sphinx: parallel.rst env_parallel.rst sem.rst sql.rst \
niceload.rst parallel_tutorial.rst parallel_book.rst \
parallel_design.rst parallel_alternatives.rst parcat.rst \
parset.rst parsort.rst
cd sphinx && make
DISTCLEANFILES = parallel.1 env_parallel.1 sem.1 sql.1 niceload.1 \ DISTCLEANFILES = parallel.1 env_parallel.1 sem.1 sql.1 niceload.1 \
parallel_tutorial.7 parallel_book.7 parallel_design.7 \ parallel_tutorial.7 parallel_book.7 parallel_design.7 \
parallel_alternatives.7 parcat.1 parset.1 parsort.1 \ parallel_alternatives.7 parcat.1 parset.1 parsort.1 \

2
src/env_parallel.ash
View file

@ -385,7 +385,7 @@ _parset_main() {
return 255 return 255
fi fi
if [ "$_parset_NAME" = "--version" ] ; then if [ "$_parset_NAME" = "--version" ] ; then
echo "parset 20210623 (GNU parallel `parallel --minversion 1`)" echo "parset 20210722 (GNU parallel `parallel --minversion 1`)"
echo "Copyright (C) 2007-2021 Ole Tange, http://ole.tange.dk and Free Software" echo "Copyright (C) 2007-2021 Ole Tange, http://ole.tange.dk and Free Software"
echo "Foundation, Inc." echo "Foundation, Inc."
echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>" echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>"

2
src/env_parallel.bash
View file

@ -384,7 +384,7 @@ _parset_main() {
return 255 return 255
fi fi
if [ "$_parset_NAME" = "--version" ] ; then if [ "$_parset_NAME" = "--version" ] ; then
echo "parset 20210623 (GNU parallel `parallel --minversion 1`)" echo "parset 20210722 (GNU parallel `parallel --minversion 1`)"
echo "Copyright (C) 2007-2021 Ole Tange, http://ole.tange.dk and Free Software" echo "Copyright (C) 2007-2021 Ole Tange, http://ole.tange.dk and Free Software"
echo "Foundation, Inc." echo "Foundation, Inc."
echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>" echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>"

2
src/env_parallel.dash
View file

@ -385,7 +385,7 @@ _parset_main() {
return 255 return 255
fi fi
if [ "$_parset_NAME" = "--version" ] ; then if [ "$_parset_NAME" = "--version" ] ; then
echo "parset 20210623 (GNU parallel `parallel --minversion 1`)" echo "parset 20210722 (GNU parallel `parallel --minversion 1`)"
echo "Copyright (C) 2007-2021 Ole Tange, http://ole.tange.dk and Free Software" echo "Copyright (C) 2007-2021 Ole Tange, http://ole.tange.dk and Free Software"
echo "Foundation, Inc." echo "Foundation, Inc."
echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>" echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>"

2
src/env_parallel.ksh
View file

@ -363,7 +363,7 @@ _parset_main() {
return 255 return 255
fi fi
if [ "$_parset_NAME" = "--version" ] ; then if [ "$_parset_NAME" = "--version" ] ; then
echo "parset 20210623 (GNU parallel `parallel --minversion 1`)" echo "parset 20210722 (GNU parallel `parallel --minversion 1`)"
echo "Copyright (C) 2007-2021 Ole Tange, http://ole.tange.dk and Free Software" echo "Copyright (C) 2007-2021 Ole Tange, http://ole.tange.dk and Free Software"
echo "Foundation, Inc." echo "Foundation, Inc."
echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>" echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>"

2
src/env_parallel.mksh
View file

@ -365,7 +365,7 @@ _parset_main() {
return 255 return 255
fi fi
if [ "$_parset_NAME" = "--version" ] ; then if [ "$_parset_NAME" = "--version" ] ; then
echo "parset 20210623 (GNU parallel `parallel --minversion 1`)" echo "parset 20210722 (GNU parallel `parallel --minversion 1`)"
echo "Copyright (C) 2007-2021 Ole Tange, http://ole.tange.dk and Free Software" echo "Copyright (C) 2007-2021 Ole Tange, http://ole.tange.dk and Free Software"
echo "Foundation, Inc." echo "Foundation, Inc."
echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>" echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>"

2
src/env_parallel.sh
View file

@ -390,7 +390,7 @@ _parset_main() {
return 255 return 255
fi fi
if [ "$_parset_NAME" = "--version" ] ; then if [ "$_parset_NAME" = "--version" ] ; then
echo "parset 20210623 (GNU parallel `parallel --minversion 1`)" echo "parset 20210722 (GNU parallel `parallel --minversion 1`)"
echo "Copyright (C) 2007-2021 Ole Tange, http://ole.tange.dk and Free Software" echo "Copyright (C) 2007-2021 Ole Tange, http://ole.tange.dk and Free Software"
echo "Foundation, Inc." echo "Foundation, Inc."
echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>" echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>"

2
src/env_parallel.zsh
View file

@ -355,7 +355,7 @@ _parset_main() {
return 255 return 255
fi fi
if [ "$_parset_NAME" = "--version" ] ; then if [ "$_parset_NAME" = "--version" ] ; then
echo "parset 20210623 (GNU parallel `parallel --minversion 1`)" echo "parset 20210722 (GNU parallel `parallel --minversion 1`)"
echo "Copyright (C) 2007-2021 Ole Tange, http://ole.tange.dk and Free Software" echo "Copyright (C) 2007-2021 Ole Tange, http://ole.tange.dk and Free Software"
echo "Foundation, Inc." echo "Foundation, Inc."
echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>" echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>"

2
src/niceload
View file

@ -26,7 +26,7 @@
use strict; use strict;
use Getopt::Long; use Getopt::Long;
$Global::progname="niceload"; $Global::progname="niceload";
$Global::version = 20210623; $Global::version = 20210722;
Getopt::Long::Configure("bundling","require_order"); Getopt::Long::Configure("bundling","require_order");
get_options_from_array(\@ARGV) || die_usage(); get_options_from_array(\@ARGV) || die_usage();
if($opt::version) { if($opt::version) {

22
src/parallel
View file

@ -2223,7 +2223,7 @@ sub check_invalid_option_combinations() {
sub init_globals() { sub init_globals() {
# Defaults: # Defaults:
$Global::version = 20210623; $Global::version = 20210722;
$Global::progname = 'parallel'; $Global::progname = 'parallel';
$::name = "GNU Parallel"; $::name = "GNU Parallel";
$Global::infinity = 2**31; $Global::infinity = 2**31;
@ -5034,8 +5034,8 @@ sub usage() {
"If you use programs that use GNU Parallel to process data for an article in a", "If you use programs that use GNU Parallel to process data for an article in a",
"scientific publication, please cite:", "scientific publication, please cite:",
"", "",
" Tange, O. (2021, June 22). GNU Parallel 20210622 ('Protasevich').", " Tange, O. (2021, July 22). GNU Parallel 20210722 ('Blue Unity').",
" Zenodo. https://doi.org/10.5281/zenodo.5013933", " Zenodo. https://doi.org/10.5281/zenodo.5123056",
"", "",
# Before changing this line, please read # Before changing this line, please read
# https://www.gnu.org/software/parallel/parallel_design.html#Citation-notice # https://www.gnu.org/software/parallel/parallel_design.html#Citation-notice
@ -5065,8 +5065,8 @@ sub citation_notice() {
"If you use programs that use GNU Parallel to process data for an article in a", "If you use programs that use GNU Parallel to process data for an article in a",
"scientific publication, please cite:", "scientific publication, please cite:",
"", "",
" Tange, O. (2021, June 22). GNU Parallel 20210622 ('Protasevich').", " Tange, O. (2021, July 22). GNU Parallel 20210722 ('Blue Unity').",
" Zenodo. https://doi.org/10.5281/zenodo.5013933", " Zenodo. https://doi.org/10.5281/zenodo.5123056",
"", "",
# Before changing this line, please read # Before changing this line, please read
# https://www.gnu.org/software/parallel/parallel_design.html#Citation-notice and # https://www.gnu.org/software/parallel/parallel_design.html#Citation-notice and
@ -5189,20 +5189,20 @@ sub citation() {
"If you use programs that use GNU Parallel to process data for an article in a", "If you use programs that use GNU Parallel to process data for an article in a",
"scientific publication, please cite:", "scientific publication, please cite:",
"", "",
"\@software{tange_2021_5013933,", "\@software{tange_2021_5123056,",
" author = {Tange, Ole},", " author = {Tange, Ole},",
" title = {GNU Parallel 20210622 ('Protasevich')},", " title = {GNU Parallel 20210722 ('Blue Unity')},",
" month = Jun,", " month = Jul,",
" year = 2021,", " year = 2021,",
" note = {{GNU Parallel is a general parallelizer to run", " note = {{GNU Parallel is a general parallelizer to run",
" multiple serial command line programs in parallel", " multiple serial command line programs in parallel",
" without changing them.}},", " without changing them.}},",
" publisher = {Zenodo},", " publisher = {Zenodo},",
" doi = {10.5281/zenodo.5013933},", " doi = {10.5281/zenodo.5123056},",
" url = {https://doi.org/10.5281/zenodo.5013933}", " url = {https://doi.org/10.5281/zenodo.5123056}",
"}", "}",
"", "",
"(Feel free to use \\nocite{tange_2021_5013933})", "(Feel free to use \\nocite{tange_2021_5123056})",
"", "",
# Before changing this line, please read # Before changing this line, please read
# https://www.gnu.org/software/parallel/parallel_design.html#Citation-notice and # https://www.gnu.org/software/parallel/parallel_design.html#Citation-notice and

11
src/parallel.pod
View file

@ -1639,9 +1639,9 @@ Activate additional replacement strings: {+/} {+.} {+..} {+...} {..}
B<{##}> is the total number of jobs to be run. It is incompatible with B<{##}> is the total number of jobs to be run. It is incompatible with
B<-X>/B<-m>/B<--xargs>. B<-X>/B<-m>/B<--xargs>.
B<{0%}> zero-padded jobslot. (beta testing) B<{0%}> zero-padded jobslot.
B<{0#}> zero-padded sequence number. (beta testing) B<{0#}> zero-padded sequence number.
B<{choose_k}> is inspired by n choose k: Given a list of n elements, B<{choose_k}> is inspired by n choose k: Given a list of n elements,
choose k. k is the number of input sources and n is the number of choose k. k is the number of input sources and n is the number of
@ -1817,7 +1817,8 @@ Swap activity is computed as (swap-in)*(swap-out) which in practice is
a good value: swapping out is not a problem, swapping in is not a a good value: swapping out is not a problem, swapping in is not a
problem, but both swapping in and out usually indicates a problem. problem, but both swapping in and out usually indicates a problem.
B<--memfree> may give better results, so try using that first. B<--memfree> and B<--memsuspend> may give better results, so try using
those first.
=item B<--record-env> =item B<--record-env>
@ -1868,9 +1869,9 @@ it to the command.
Only used with B<--pipe>. Only used with B<--pipe>.
=item B<--results> I<name> =item B<--results> I<name> (alpha testing)
=item B<--res> I<name> =item B<--res> I<name> (alpha testing)
Save the output into files. Save the output into files.

29
src/parallel_alternatives.pod
View file

@ -1321,7 +1321,7 @@ Contrary to B<rush> this also works if the value is complex like:
My brother's 12" records My brother's 12" records
B<14. B<Preset variable> (`-v`), avoid repeatedly writing verbose replacement strings> B<14. Preset variable (`-v`), avoid repeatedly writing verbose replacement strings>
14$ # naive way 14$ # naive way
echo read_1.fq.gz | rush 'echo {:^_1} {:^_1}_2.fq.gz' echo read_1.fq.gz | rush 'echo {:^_1} {:^_1}_2.fq.gz'
@ -3174,6 +3174,10 @@ not, all the long jobs may end up in the same queue, so you may see:
time ./parallel-bash.bash -p 4 -c sleep {} time ./parallel-bash.bash -p 4 -c sleep {}
(12 seconds) (12 seconds)
Because it uses bash lists, the total number of jobs is limited to
167000..265000 depending on your environment. You get a segmentation
fault, when you reach the limit.
Ctrl-C does not stop spawning new jobs. Ctrl-Z does not suspend Ctrl-C does not stop spawning new jobs. Ctrl-Z does not suspend
running jobs. running jobs.
@ -3258,29 +3262,6 @@ https://vicerveza.homeunix.net/~viric/soft/ts/
https://github.com/chapmanjacobd/que https://github.com/chapmanjacobd/que
https://github.com/ExpectationMax/simple_gpu_scheduler
simple_gpu_scheduler --gpus 0 1 2 < gpu_commands.txt
parallel -j3 --shuf CUDA_VISIBLE_DEVICES='{=1 $_=slot()-1 =} {=uq;=}' < gpu_commands.txt
simple_hypersearch "python3 train_dnn.py --lr {lr} --batch_size {bs}" -p lr 0.001 0.0005 0.0001 -p bs 32 64 128 | simple_gpu_scheduler --gpus 0,1,2
parallel --header : --shuf -j3 -v CUDA_VISIBLE_DEVICES='{=1 $_=slot()-1 =}' python3 train_dnn.py --lr {lr} --batch_size {bs} ::: lr 0.001 0.0005 0.0001 ::: bs 32 64 128
simple_hypersearch "python3 train_dnn.py --lr {lr} --batch_size {bs}" --n-samples 5 -p lr 0.001 0.0005 0.0001 -p bs 32 64 128 | simple_gpu_scheduler --gpus 0,1,2
parallel --header : --shuf CUDA_VISIBLE_DEVICES='{=1 $_=slot()-1; seq() > 5 and skip() =}' python3 train_dnn.py --lr {lr} --batch_size {bs} ::: lr 0.001 0.0005 0.0001 ::: bs 32 64 128
touch gpu.queue
tail -f -n 0 gpu.queue | simple_gpu_scheduler --gpus 0,1,2 &
echo "my_command_with | and stuff > logfile" >> gpu.queue
touch gpu.queue
tail -f -n 0 gpu.queue | parallel -j3 CUDA_VISIBLE_DEVICES='{=1 $_=slot()-1 =} {=uq;=}' &
# Needed to fill job slots once
seq 3 | parallel echo true >> gpu.queue
# Add jobs
echo "my_command_with | and stuff > logfile" >> gpu.queue
# Needed to flush output from completed jobs
seq 3 | parallel echo true >> gpu.queue
https://github.com/Overv/outrun#outrun https://github.com/Overv/outrun#outrun
=head1 TESTING OTHER TOOLS =head1 TESTING OTHER TOOLS

11
src/parset.pod
View file

@ -29,12 +29,13 @@ The B<parset> and B<env_parset> functions are defined as part of
B<env_parallel>. B<env_parallel>.
If I<variablename> is a single variable name, this will be treated as If I<variablename> is a single variable name, this will be treated as
the destination variable and made into an array. the destination variable. If the variable is defined as an associative
array (using B<typeset -A myassoc>), this will be used. Otherwise the
variable will be made into a normal array.
If I<variablename> contains multiple names separated by ',' or space, If I<variablename> contains multiple names separated by ',' or space,
the names will be the destination variables. The number of names must the names will be the destination variables. The number of names must
be at least the number of jobs - otherwise some tmp files will not be be at least the number of jobs.
cleaned up.
=head1 OPTIONS =head1 OPTIONS
@ -82,8 +83,8 @@ B<parset> can read from stdin (standard input) if it is a file:
parset res echo < parallel_input_file parset res echo < parallel_input_file
but B<parset> can not be part of a pipe. In particular this means it but B<parset> can I<not> be part of a pipe. In particular this means
cannot read from a pipe or write to a pipe: it cannot read from a pipe or write to a pipe:
seq 10 | parset res echo Does not work seq 10 | parset res echo Does not work

5
src/parsort
View file

@ -37,7 +37,8 @@ Sort stdin (standard input) numerically:
=head1 PERFORMANCE =head1 PERFORMANCE
B<parsort> is faster on files, because these can be read in parallel. B<parsort> is faster on a file than on stdin (standard input), because
different parts of a file can be read in parallel.
On a 48 core machine you should see a speedup of 3x over B<sort>. On a 48 core machine you should see a speedup of 3x over B<sort>.
@ -121,7 +122,7 @@ GetOptions(
"help" => \$opt::dummy, "help" => \$opt::dummy,
) || exit(255); ) || exit(255);
$Global::progname = ($0 =~ m:(^|/)([^/]+)$:)[1]; $Global::progname = ($0 =~ m:(^|/)([^/]+)$:)[1];
$Global::version = 20210623; $Global::version = 20210722;
if($opt::version) { version(); exit 0; } if($opt::version) { version(); exit 0; }
@Global::sortoptions = @Global::sortoptions =
shell_quote(@ARGV_before[0..($#ARGV_before-$#ARGV-1)]); shell_quote(@ARGV_before[0..($#ARGV_before-$#ARGV-1)]);

3
src/sphinx/Makefile
View file

@ -13,6 +13,9 @@ BUILDDIR = build
all: copy all: copy
make html make html
install:
true dummy
copy: copy:
cp -a ../*.rst source/ cp -a ../*.rst source/

10
src/sphinx/source/conf.py
View file

@ -18,19 +18,23 @@
# #
import os import os
import sys import sys
import subprocess
sys.path.insert(0, os.path.abspath('.')) sys.path.insert(0, os.path.abspath('.'))
short_version = (subprocess.run(['parallel', '--minversion', '1'], stdout=subprocess.PIPE)).stdout.decode('utf-8')
year = short_version[0:4]
# -- Project information ----------------------------------------------------- # -- Project information -----------------------------------------------------
project = 'GNU Parallel' project = 'GNU Parallel'
copyright = '2021, Ole Tange' copyright = year+', Ole Tange'
author = 'Ole Tange' author = 'Ole Tange'
# The short X.Y version # The short X.Y version
version = '' version = short_version
# The full version, including alpha/beta/rc tags # The full version, including alpha/beta/rc tags
release = '20210122' release = short_version
# -- General configuration --------------------------------------------------- # -- General configuration ---------------------------------------------------

54
src/sphinx/source/sphinx.rst
View file

@ -11,19 +11,69 @@ Welcome to GNU Parallel's documentation!
======================================== ========================================
.. toctree:: .. toctree::
:maxdepth: 3 :maxdepth: 1
:caption: Contents: :caption: parallel
parallel parallel
.. toctree::
:maxdepth: 3
:caption: sem
sem sem
.. toctree::
:maxdepth: 3
:caption: env_parallel
env_parallel env_parallel
.. toctree::
:maxdepth: 3
:caption: parset
parset parset
.. toctree::
:maxdepth: 3
:caption: parsort
parsort parsort
.. toctree::
:maxdepth: 3
:caption: parallel_design
parallel_design parallel_design
.. toctree::
:maxdepth: 3
:caption: parallel_tutorial
parallel_tutorial parallel_tutorial
.. toctree::
:maxdepth: 3
:caption: parallel_alternatives
parallel_alternatives parallel_alternatives
.. toctree::
:maxdepth: 3
:caption: parcat
parcat parcat
.. toctree::
:maxdepth: 3
:caption: niceload
niceload niceload
.. toctree::
:maxdepth: 3
:caption: sql
sql sql

2
src/sql
View file

@ -600,7 +600,7 @@ $Global::Initfile && unlink $Global::Initfile;
exit ($err); exit ($err);
sub parse_options { sub parse_options {
$Global::version = 20210623; $Global::version = 20210722;
$Global::progname = 'sql'; $Global::progname = 'sql';
# This must be done first as this may exec myself # This must be done first as this may exec myself

9
testsuite/Makefile
View file

@ -3,7 +3,10 @@
# SPDX-License-Identifier: GPL-3.0-or-later # SPDX-License-Identifier: GPL-3.0-or-later
### Full test set (except mem) ### Full test set (except mem)
testsuite: 3 testsuite: short
true
all: mem short long polar
true true
3: ../src/parallel tests-to-run/* wanted-results/* startdb prereqlocal prereqremote 3: ../src/parallel tests-to-run/* wanted-results/* startdb prereqlocal prereqremote
@ -51,6 +54,10 @@ local: ../src/parallel tests-to-run/*local* wanted-results/*local* prereqlocal i
short: ../src/parallel tests-to-run/* wanted-results/* prereqlocal installparallel short: ../src/parallel tests-to-run/* wanted-results/* prereqlocal installparallel
TRIES=1 time bash Start.sh '' 'polar|mem|100s|300s' TRIES=1 time bash Start.sh '' 'polar|mem|100s|300s'
# long
long: ../src/parallel tests-to-run/* wanted-results/* prereqlocal installparallel
TRIES=1 time bash Start.sh '00s' NONE
# short and local # short and local
localshort: ../src/parallel tests-to-run/*local* wanted-results/*local* prereqlocal installparallel localshort: ../src/parallel tests-to-run/*local* wanted-results/*local* prereqlocal installparallel
TRIES=1 time bash Start.sh local 'mem|100s|300s' TRIES=1 time bash Start.sh local 'mem|100s|300s'

2
testsuite/tests-to-run/parallel-polarhome.sh
View file

@ -214,7 +214,7 @@ env_parallel -u -S$MASTER doit ::: 1 |
# eval 'myfunc2() { echo '$(perl -e 'print "x"x120000')'; }' # eval 'myfunc2() { echo '$(perl -e 'print "x"x120000')'; }'
# env_parallel myfunc ::: a | wc # Fail too big env # env_parallel myfunc ::: a | wc # Fail too big env
# Supported keylength: # Supported SSH keylength:
# 16300: # 16300:
# debian-ppc netbsd openbsd qnx aix centos freebsd hpux hpux-ia64 # debian-ppc netbsd openbsd qnx aix centos freebsd hpux hpux-ia64
# macosx mandriva miros raspbian redhat scosysv suse tru64 unixware # macosx mandriva miros raspbian redhat scosysv suse tru64 unixware