tange/parallel
1
0
Fork 0
mirror of https://git.savannah.gnu.org/git/parallel.git synced 2024-10-18 04:49:40 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Released as 20210822 ('Kabul')

Browse source
This commit is contained in:
Ole Tange 2021-08-22 22:05:37 +02:00
parent 5adcd33f7b
commit 4e6f4644f4
23 changed files with 160 additions and 101 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

29
NEWS
View file

@ -1,8 +1,32 @@
20210822
New in this release:
* --ctag/--ctagstring colors the tag in different colors for each job.
* You can use unit prefixes (k, m, g, etc) with -n -N -L.
* Bug fixes and man page updates.
News about GNU Parallel:
* Parallelising jobs with GNU parallel
https://blog.ronin.cloud/gnu-parallel/
* Use multiple CPU Cores with your Linux commands - awk, sed, bzip2,
grep, wc, etc. https://cdmana.com/2021/07/20210728132344693t.html
* How to execute commands in parallel in Linux
https://net2.com/how-to-execute-commands-in-parallel-in-linux/
20210722 20210722
New in this release: New in this release:
* --results no longer prints the result to standard output (stdout) as voted in https://lists.gnu.org/archive/html/parallel/2020-12/msg00003.html * --results no longer prints the result to standard output (stdout) as
voted in
https://lists.gnu.org/archive/html/parallel/2020-12/msg00003.html
* parset supports associative arrays in bash, ksh, zsh. * parset supports associative arrays in bash, ksh, zsh.
@ -12,7 +36,8 @@ New in this release:
News about GNU Parallel: News about GNU Parallel:
* Cleaning Up Scanned Documents with Open Source Tools https://kaerumy.medium.com/cleaning-up-scanned-documents-with-open-source-tools-9d87e15305b * Cleaning Up Scanned Documents with Open Source Tools
https://kaerumy.medium.com/cleaning-up-scanned-documents-with-open-source-tools-9d87e15305b
20210622 20210622

24
README
View file

@ -57,11 +57,11 @@ document.
Full installation of GNU Parallel is as simple as: Full installation of GNU Parallel is as simple as:
wget https://ftpmirror.gnu.org/parallel/parallel-20210722.tar.bz2 wget https://ftpmirror.gnu.org/parallel/parallel-20210822.tar.bz2
wget https://ftpmirror.gnu.org/parallel/parallel-20210722.tar.bz2.sig wget https://ftpmirror.gnu.org/parallel/parallel-20210822.tar.bz2.sig
gpg parallel-20210722.tar.bz2.sig gpg parallel-20210822.tar.bz2.sig
bzip2 -dc parallel-20210722.tar.bz2 | tar xvf - bzip2 -dc parallel-20210822.tar.bz2 | tar xvf -
cd parallel-20210722 cd parallel-20210822
./configure && make && sudo make install ./configure && make && sudo make install
@ -70,11 +70,11 @@ Full installation of GNU Parallel is as simple as:
If you are not root you can add ~/bin to your path and install in If you are not root you can add ~/bin to your path and install in
~/bin and ~/share: ~/bin and ~/share:
wget https://ftpmirror.gnu.org/parallel/parallel-20210722.tar.bz2 wget https://ftpmirror.gnu.org/parallel/parallel-20210822.tar.bz2
wget https://ftpmirror.gnu.org/parallel/parallel-20210722.tar.bz2.sig wget https://ftpmirror.gnu.org/parallel/parallel-20210822.tar.bz2.sig
gpg parallel-20210722.tar.bz2.sig gpg parallel-20210822.tar.bz2.sig
bzip2 -dc parallel-20210722.tar.bz2 | tar xvf - bzip2 -dc parallel-20210822.tar.bz2 | tar xvf -
cd parallel-20210722 cd parallel-20210822
./configure --prefix=$HOME && make && make install ./configure --prefix=$HOME && make && make install
Or if your system lacks 'make' you can simply copy src/parallel Or if your system lacks 'make' you can simply copy src/parallel
@ -122,8 +122,8 @@ will love you for it.
When using programs that use GNU Parallel to process data for When using programs that use GNU Parallel to process data for
publication please cite: publication please cite:
Tange, O. (2021, July 22). GNU Parallel 20210722 ('Blue Unity'). Tange, O. (2021, August 22). GNU Parallel 20210822 ('Kabul').
Zenodo. https://doi.org/10.5281/zenodo.5123056 Zenodo. https://doi.org/10.5281/zenodo.5233953
Copyright (C) 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, Copyright (C) 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015,
2016, 2017, 2018, 2019, 2020, 2021 Ole Tange, http://ole.tange.dk and 2016, 2017, 2018, 2019, 2020, 2021 Ole Tange, http://ole.tange.dk and

20
configure vendored
View file

@ -1,6 +1,6 @@
#! /bin/sh #! /bin/sh
# Guess values for system-dependent variables and create Makefiles. # Guess values for system-dependent variables and create Makefiles.
# Generated by GNU Autoconf 2.69 for parallel 20210722. # Generated by GNU Autoconf 2.69 for parallel 20210822.
# #
# Report bugs to <bug-parallel@gnu.org>. # Report bugs to <bug-parallel@gnu.org>.
# #
@ -579,8 +579,8 @@ MAKEFLAGS=
# Identity of this package. # Identity of this package.
PACKAGE_NAME='parallel' PACKAGE_NAME='parallel'
PACKAGE_TARNAME='parallel' PACKAGE_TARNAME='parallel'
PACKAGE_VERSION='20210722' PACKAGE_VERSION='20210822'
PACKAGE_STRING='parallel 20210722' PACKAGE_STRING='parallel 20210822'
PACKAGE_BUGREPORT='bug-parallel@gnu.org' PACKAGE_BUGREPORT='bug-parallel@gnu.org'
PACKAGE_URL='' PACKAGE_URL=''
@ -1214,7 +1214,7 @@ if test "$ac_init_help" = "long"; then
# Omit some internal or obsolete options to make the list less imposing. # Omit some internal or obsolete options to make the list less imposing.
# This message is too long to be a string in the A/UX 3.1 sh. # This message is too long to be a string in the A/UX 3.1 sh.
cat <<_ACEOF cat <<_ACEOF
\`configure' configures parallel 20210722 to adapt to many kinds of systems. \`configure' configures parallel 20210822 to adapt to many kinds of systems.
Usage: $0 [OPTION]... [VAR=VALUE]... Usage: $0 [OPTION]... [VAR=VALUE]...
@ -1281,7 +1281,7 @@ fi
if test -n "$ac_init_help"; then if test -n "$ac_init_help"; then
case $ac_init_help in case $ac_init_help in
short | recursive ) echo "Configuration of parallel 20210722:";; short | recursive ) echo "Configuration of parallel 20210822:";;
esac esac
cat <<\_ACEOF cat <<\_ACEOF
@ -1357,7 +1357,7 @@ fi
test -n "$ac_init_help" && exit $ac_status test -n "$ac_init_help" && exit $ac_status
if $ac_init_version; then if $ac_init_version; then
cat <<\_ACEOF cat <<\_ACEOF
parallel configure 20210722 parallel configure 20210822
generated by GNU Autoconf 2.69 generated by GNU Autoconf 2.69
Copyright (C) 2012 Free Software Foundation, Inc. Copyright (C) 2012 Free Software Foundation, Inc.
@ -1374,7 +1374,7 @@ cat >config.log <<_ACEOF
This file contains any messages produced by compilers while This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake. running configure, to aid debugging if configure makes a mistake.
It was created by parallel $as_me 20210722, which was It was created by parallel $as_me 20210822, which was
generated by GNU Autoconf 2.69. Invocation command line was generated by GNU Autoconf 2.69. Invocation command line was
$ $0 $@ $ $0 $@
@ -2237,7 +2237,7 @@ fi
# Define the identity of the package. # Define the identity of the package.
PACKAGE='parallel' PACKAGE='parallel'
VERSION='20210722' VERSION='20210822'
cat >>confdefs.h <<_ACEOF cat >>confdefs.h <<_ACEOF
@ -2880,7 +2880,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1
# report actual input values of CONFIG_FILES etc. instead of their # report actual input values of CONFIG_FILES etc. instead of their
# values after options handling. # values after options handling.
ac_log=" ac_log="
This file was extended by parallel $as_me 20210722, which was This file was extended by parallel $as_me 20210822, which was
generated by GNU Autoconf 2.69. Invocation command line was generated by GNU Autoconf 2.69. Invocation command line was
CONFIG_FILES = $CONFIG_FILES CONFIG_FILES = $CONFIG_FILES
@ -2942,7 +2942,7 @@ _ACEOF
cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1
ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`" ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`"
ac_cs_version="\\ ac_cs_version="\\
parallel config.status 20210722 parallel config.status 20210822
configured by $0, generated by GNU Autoconf 2.69, configured by $0, generated by GNU Autoconf 2.69,
with options \\"\$ac_cs_config\\" with options \\"\$ac_cs_config\\"

2
configure.ac
View file

@ -1,4 +1,4 @@
AC_INIT([parallel], [20210722], [bug-parallel@gnu.org]) AC_INIT([parallel], [20210822], [bug-parallel@gnu.org])
AM_INIT_AUTOMAKE([-Wall -Werror foreign]) AM_INIT_AUTOMAKE([-Wall -Werror foreign])
AC_CONFIG_HEADERS([config.h]) AC_CONFIG_HEADERS([config.h])
AC_CONFIG_FILES([ AC_CONFIG_FILES([

10
doc/haikus
View file

@ -4,7 +4,9 @@
Quote of the month: Quote of the month:
Have you heard of our lord and saviour GNU parallel? https://gnu.org/software/pa I really liked GNU Parallel http://gnu.org/software/parallel/
one of the best tool to execute parallel jobs in the shell
-- Luca Molteni @volothamp@twitter
Have you heard of our lord and saviour GNU parallel? Have you heard of our lord and saviour GNU parallel?
-- kxyne @Kxyne@twitter -- kxyne @Kxyne@twitter
@ -126,7 +128,11 @@ https://negfeedback.blogspot.com/2020/05/indispensable-command-line-tools.html
=== Used === === Used ===
We use gnu parallel now - and happier for it. Safe to say, @GnuParallel was a life changer during my PhD! It helped
me optimise so many of my tasks and analyses.
-- Parice Brandies @PariceBrandies@twitter
We use gnu parallel now - and happier for it.
-- Ben Davies @benjamindavies@twitter -- Ben Davies @benjamindavies@twitter
GNU Parallel makes my life so much easier. GNU Parallel makes my life so much easier.

24
doc/release_new_version
View file

@ -100,7 +100,7 @@ lbry://@GnuParallel#4/parallel-20210322.tar.bz2
# #
# Tags: gnu parallel software # Tags: gnu parallel software
. .last-doitag.txt
file_path="`pwd`/parallel-$YYYYMMDD.tar.bz2" file_path="`pwd`/parallel-$YYYYMMDD.tar.bz2"
title="GNU Parallel $YYYYMMDD ('$SPCTAG')" title="GNU Parallel $YYYYMMDD ('$SPCTAG')"
name="GNU-Parallel-$YYYYMMDD-$TAG" name="GNU-Parallel-$YYYYMMDD-$TAG"
@ -255,30 +255,32 @@ from:tange@gnu.org
to:parallel@gnu.org, bug-parallel@gnu.org to:parallel@gnu.org, bug-parallel@gnu.org
stable-bcc: Jesse Alama <jessealama@fastmail.fm> stable-bcc: Jesse Alama <jessealama@fastmail.fm>
Subject: GNU Parallel 20210822 ('South Africa/Kristina Timanovskaya/turkish fire/greek fire/tysk syndflod/Tunesia') released <<[stable]>> Subject: GNU Parallel 20210822 ('Kabul') released
GNU Parallel 20210822 ('') <<[stable]>> has been released. It is available for download at: lbry://@GnuParallel:4 GNU Parallel 20210822 ('Kabul') has been released. It is available for download at: lbry://@GnuParallel:4
<<No new functionality was introduced so this is a good candidate for a stable release.>>
Quote of the month: Quote of the month:
<<>> Safe to say, @GnuParallel was a life changer during my PhD! It helped
me optimise so many of my tasks and analyses.
-- Parice Brandies @PariceBrandies@twitter
New in this release: New in this release:
<<>> * --ctag/--ctagstring colors the tag in different colors for each job.
* You can use unit prefixes (k, m, g, etc) with -n -N -L.
* Bug fixes and man page updates. * Bug fixes and man page updates.
News about GNU Parallel: News about GNU Parallel:
<<>> * Parallelising jobs with GNU parallel https://blog.ronin.cloud/gnu-parallel/
https://cdmana.com/2021/07/20210728132344693t.html
https://github.com/gibbslab/biobash uses GNU Parallel * Use multiple CPU Cores with your Linux commands - awk, sed, bzip2, grep, wc, etc. https://cdmana.com/2021/07/20210728132344693t.html
https://net2.com/how-to-execute-commands-in-parallel-in-linux/ * How to execute commands in parallel in Linux https://net2.com/how-to-execute-commands-in-parallel-in-linux/
Get the book: GNU Parallel 2018 http://www.lulu.com/shop/ole-tange/gnu-parallel-2018/paperback/product-23558902.html Get the book: GNU Parallel 2018 http://www.lulu.com/shop/ole-tange/gnu-parallel-2018/paperback/product-23558902.html

12
packager/obs/home:tange/parallel/.osc/_files
View file

@ -1,7 +1,7 @@
<directory name="parallel" rev="311" vrev="1" srcmd5="4db07b555bac8c95b748e4b27d7b8ec2"> <directory name="parallel" rev="312" vrev="1" srcmd5="3b2d528a15b0898ec9a2649f393cb24d">
<entry name="PKGBUILD" md5="efe5dba5d697d74243ed766562a7ced1" size="936" mtime="1626984745" /> <entry name="PKGBUILD" md5="872019301c50c38abc39ba04937925d1" size="936" mtime="1629661749" />
<entry name="parallel-20210722.tar.bz2" md5="bc317c08d26f81c81ef764b8e421a0e1" size="2248888" mtime="1626984745" /> <entry name="parallel-20210822.tar.bz2" md5="1a9a3282c6287ad43936497f4c0fe61b" size="2267536" mtime="1629661749" />
<entry name="parallel.spec" md5="4657b2121ab380c968d795952d8405a3" size="5630" mtime="1626984745" /> <entry name="parallel.spec" md5="1360338a8e6c60305fb2c9a27db7a902" size="5630" mtime="1629661749" />
<entry name="parallel_20210722.dsc" md5="abe14d022d2190009c72ba95cd1c332b" size="556" mtime="1626984746" /> <entry name="parallel_20210822.dsc" md5="48d6e04ee967486209117322306aa3bd" size="556" mtime="1629661749" />
<entry name="parallel_20210722.tar.gz" md5="f378ff0a5e4aeb3599747bf94f08b2b7" size="2490792" mtime="1626984746" /> <entry name="parallel_20210822.tar.gz" md5="6f1ecb52bdf18a8ac846f3b5feb8c60f" size="2509459" mtime="1629661750" />
</directory> </directory>

2
packager/obs/home:tange/parallel/.osc/parallel.spec
View file

@ -1,7 +1,7 @@
Summary: Shell tool for executing jobs in parallel Summary: Shell tool for executing jobs in parallel
Name: parallel Name: parallel
Version: 20210722 Version: 20210822
Release: 1.3 Release: 1.3
License: GPL-3.0-or-later License: GPL-3.0-or-later
Group: Productivity/File utilities Group: Productivity/File utilities

2
src/env_parallel.ash
View file

@ -385,7 +385,7 @@ _parset_main() {
return 255 return 255
fi fi
if [ "$_parset_NAME" = "--version" ] ; then if [ "$_parset_NAME" = "--version" ] ; then
echo "parset 20210723 (GNU parallel `parallel --minversion 1`)" echo "parset 20210822 (GNU parallel `parallel --minversion 1`)"
echo "Copyright (C) 2007-2021 Ole Tange, http://ole.tange.dk and Free Software" echo "Copyright (C) 2007-2021 Ole Tange, http://ole.tange.dk and Free Software"
echo "Foundation, Inc." echo "Foundation, Inc."
echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>" echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>"

2
src/env_parallel.bash
View file

@ -384,7 +384,7 @@ _parset_main() {
return 255 return 255
fi fi
if [ "$_parset_NAME" = "--version" ] ; then if [ "$_parset_NAME" = "--version" ] ; then
echo "parset 20210723 (GNU parallel `parallel --minversion 1`)" echo "parset 20210822 (GNU parallel `parallel --minversion 1`)"
echo "Copyright (C) 2007-2021 Ole Tange, http://ole.tange.dk and Free Software" echo "Copyright (C) 2007-2021 Ole Tange, http://ole.tange.dk and Free Software"
echo "Foundation, Inc." echo "Foundation, Inc."
echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>" echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>"

2
src/env_parallel.dash
View file

@ -385,7 +385,7 @@ _parset_main() {
return 255 return 255
fi fi
if [ "$_parset_NAME" = "--version" ] ; then if [ "$_parset_NAME" = "--version" ] ; then
echo "parset 20210723 (GNU parallel `parallel --minversion 1`)" echo "parset 20210822 (GNU parallel `parallel --minversion 1`)"
echo "Copyright (C) 2007-2021 Ole Tange, http://ole.tange.dk and Free Software" echo "Copyright (C) 2007-2021 Ole Tange, http://ole.tange.dk and Free Software"
echo "Foundation, Inc." echo "Foundation, Inc."
echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>" echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>"

2
src/env_parallel.ksh
View file

@ -363,7 +363,7 @@ _parset_main() {
return 255 return 255
fi fi
if [ "$_parset_NAME" = "--version" ] ; then if [ "$_parset_NAME" = "--version" ] ; then
echo "parset 20210723 (GNU parallel `parallel --minversion 1`)" echo "parset 20210822 (GNU parallel `parallel --minversion 1`)"
echo "Copyright (C) 2007-2021 Ole Tange, http://ole.tange.dk and Free Software" echo "Copyright (C) 2007-2021 Ole Tange, http://ole.tange.dk and Free Software"
echo "Foundation, Inc." echo "Foundation, Inc."
echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>" echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>"

2
src/env_parallel.mksh
View file

@ -365,7 +365,7 @@ _parset_main() {
return 255 return 255
fi fi
if [ "$_parset_NAME" = "--version" ] ; then if [ "$_parset_NAME" = "--version" ] ; then
echo "parset 20210723 (GNU parallel `parallel --minversion 1`)" echo "parset 20210822 (GNU parallel `parallel --minversion 1`)"
echo "Copyright (C) 2007-2021 Ole Tange, http://ole.tange.dk and Free Software" echo "Copyright (C) 2007-2021 Ole Tange, http://ole.tange.dk and Free Software"
echo "Foundation, Inc." echo "Foundation, Inc."
echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>" echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>"

2
src/env_parallel.sh
View file

@ -390,7 +390,7 @@ _parset_main() {
return 255 return 255
fi fi
if [ "$_parset_NAME" = "--version" ] ; then if [ "$_parset_NAME" = "--version" ] ; then
echo "parset 20210723 (GNU parallel `parallel --minversion 1`)" echo "parset 20210822 (GNU parallel `parallel --minversion 1`)"
echo "Copyright (C) 2007-2021 Ole Tange, http://ole.tange.dk and Free Software" echo "Copyright (C) 2007-2021 Ole Tange, http://ole.tange.dk and Free Software"
echo "Foundation, Inc." echo "Foundation, Inc."
echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>" echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>"

2
src/env_parallel.zsh
View file

@ -355,7 +355,7 @@ _parset_main() {
return 255 return 255
fi fi
if [ "$_parset_NAME" = "--version" ] ; then if [ "$_parset_NAME" = "--version" ] ; then
echo "parset 20210723 (GNU parallel `parallel --minversion 1`)" echo "parset 20210822 (GNU parallel `parallel --minversion 1`)"
echo "Copyright (C) 2007-2021 Ole Tange, http://ole.tange.dk and Free Software" echo "Copyright (C) 2007-2021 Ole Tange, http://ole.tange.dk and Free Software"
echo "Foundation, Inc." echo "Foundation, Inc."
echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>" echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>"

2
src/niceload
View file

@ -26,7 +26,7 @@
use strict; use strict;
use Getopt::Long; use Getopt::Long;
$Global::progname="niceload"; $Global::progname="niceload";
$Global::version = 20210723; $Global::version = 20210822;
Getopt::Long::Configure("bundling","require_order"); Getopt::Long::Configure("bundling","require_order");
get_options_from_array(\@ARGV) || die_usage(); get_options_from_array(\@ARGV) || die_usage();
if($opt::version) { if($opt::version) {

25
src/parallel
View file

@ -562,7 +562,8 @@ sub pipe_part_files(@) {
my ($file) = @_; my ($file) = @_;
my $buf = ""; my $buf = "";
if(not -f $file and not -b $file) { if(not -f $file and not -b $file) {
::error("$file is not a seekable file."); ::error("--pipepart only works on seekable files, not streams/pipes.",
"$file is not a seekable file.");
::wait_and_exit(255); ::wait_and_exit(255);
} }
my $header = find_header(\$buf,open_or_exit($file)); my $header = find_header(\$buf,open_or_exit($file));
@ -2240,7 +2241,7 @@ sub check_invalid_option_combinations() {
sub init_globals() { sub init_globals() {
# Defaults: # Defaults:
$Global::version = 20210723; $Global::version = 20210822;
$Global::progname = 'parallel'; $Global::progname = 'parallel';
$::name = "GNU Parallel"; $::name = "GNU Parallel";
$Global::infinity = 2**31; $Global::infinity = 2**31;
@ -5051,8 +5052,8 @@ sub usage() {
"If you use programs that use GNU Parallel to process data for an article in a", "If you use programs that use GNU Parallel to process data for an article in a",
"scientific publication, please cite:", "scientific publication, please cite:",
"", "",
" Tange, O. (2021, July 22). GNU Parallel 20210722 ('Blue Unity').", " Tange, O. (2021, August 22). GNU Parallel 20210822 ('Kabul').",
" Zenodo. https://doi.org/10.5281/zenodo.5123056", " Zenodo. https://doi.org/10.5281/zenodo.5233953",
"", "",
# Before changing this line, please read # Before changing this line, please read
# https://www.gnu.org/software/parallel/parallel_design.html#Citation-notice # https://www.gnu.org/software/parallel/parallel_design.html#Citation-notice
@ -5082,8 +5083,8 @@ sub citation_notice() {
"If you use programs that use GNU Parallel to process data for an article in a", "If you use programs that use GNU Parallel to process data for an article in a",
"scientific publication, please cite:", "scientific publication, please cite:",
"", "",
" Tange, O. (2021, July 22). GNU Parallel 20210722 ('Blue Unity').", " Tange, O. (2021, August 22). GNU Parallel 20210822 ('Kabul').",
" Zenodo. https://doi.org/10.5281/zenodo.5123056", " Zenodo. https://doi.org/10.5281/zenodo.5233953",
"", "",
# Before changing this line, please read # Before changing this line, please read
# https://www.gnu.org/software/parallel/parallel_design.html#Citation-notice and # https://www.gnu.org/software/parallel/parallel_design.html#Citation-notice and
@ -5206,20 +5207,20 @@ sub citation() {
"If you use programs that use GNU Parallel to process data for an article in a", "If you use programs that use GNU Parallel to process data for an article in a",
"scientific publication, please cite:", "scientific publication, please cite:",
"", "",
"\@software{tange_2021_5123056,", "\@software{tange_2021_5233953,",
" author = {Tange, Ole},", " author = {Tange, Ole},",
" title = {GNU Parallel 20210722 ('Blue Unity')},", " title = {GNU Parallel 20210822 ('Kabul')},",
" month = Jul,", " month = Aug,",
" year = 2021,", " year = 2021,",
" note = {{GNU Parallel is a general parallelizer to run", " note = {{GNU Parallel is a general parallelizer to run",
" multiple serial command line programs in parallel", " multiple serial command line programs in parallel",
" without changing them.}},", " without changing them.}},",
" publisher = {Zenodo},", " publisher = {Zenodo},",
" doi = {10.5281/zenodo.5123056},", " doi = {10.5281/zenodo.5233953},",
" url = {https://doi.org/10.5281/zenodo.5123056}", " url = {https://doi.org/10.5281/zenodo.5233953}",
"}", "}",
"", "",
"(Feel free to use \\nocite{tange_2021_5123056})", "(Feel free to use \\nocite{tange_2021_5233953})",
"", "",
# Before changing this line, please read # Before changing this line, please read
# https://www.gnu.org/software/parallel/parallel_design.html#Citation-notice and # https://www.gnu.org/software/parallel/parallel_design.html#Citation-notice and

37
src/parallel.pod
View file

@ -1621,14 +1621,16 @@ B<--pipepart> has a few limitations:
The file must be a normal file or a block device (technically it must The file must be a normal file or a block device (technically it must
be seekable) and must be given using B<-a> or B<::::>. The file cannot be seekable) and must be given using B<-a> or B<::::>. The file cannot
be a pipe or a fifo as they are not seekable. be a pipe, a fifo, or a stream as they are not seekable.
If using a block device with lot of NUL bytes, remember to set If using a block device with lot of NUL bytes, remember to set
B<--recend ''>. B<--recend ''>.
=item * =item *
Record counting (B<-N>) and line counting (B<-L>/B<-l>) do not work. Record counting (B<-N>) and line counting (B<-L>/B<-l>) do not
work. Instead use B<--recstart> and B<--recend> to determine
where records end.
=back =back
@ -1880,9 +1882,9 @@ it to the command.
Only used with B<--pipe>. Only used with B<--pipe>.
=item B<--results> I<name> (alpha testing) =item B<--results> I<name> (beta testing)
=item B<--res> I<name> (alpha testing) =item B<--res> I<name> (beta testing)
Save the output into files. Save the output into files.
@ -4186,7 +4188,7 @@ the special short hand I<-S ..> can be used:
--trc {.}.bz2 "zcat {} | bzip2 -9 >{.}.bz2" --trc {.}.bz2 "zcat {} | bzip2 -9 >{.}.bz2"
=head2 EXAMPLE: Advanced file transfer =head2 EXAMPLE: Advanced file transfer
Assume you have files in in/*, want them processed on server, Assume you have files in in/*, want them processed on server,
and transferred back into /other/dir: and transferred back into /other/dir:
@ -4532,6 +4534,31 @@ To call B<myprog> with the sequence as argument run:
'read a; echo Name: "$a"; myprog $(tr -d "\n")' 'read a; echo Name: "$a"; myprog $(tr -d "\n")'
=head2 EXAMPLE: Call program with interleaved FASTQ records
FASTQ files have the format:
@M10991:61:000000000-A7EML:1:1101:14011:1001 1:N:0:28
CTCCTAGGTCGGCATGATGGGGGAAGGAGAGCATGGGAAGAAATGAGAGAGTAGCAAGG
+
#8BCCGGGGGFEFECFGGGGGGGGG@;FFGGGEG@FF<EE<@FFC,CEGCCGGFF<FGF
Interleaved FASTQ starts with a line like these:
@HWUSI-EAS100R:6:73:941:1973#0/1
@EAS139:136:FC706VJ:2:2104:15343:197393 1:Y:18:ATCACG
@EAS139:136:FC706VJ:2:2104:15343:197393 1:N:18:1
where '/1' and ' 1:' determines this is read 1.
This will cut big.fq into one chunk per CPU core and pass it on
stdin (standard input) to the program fastq-reader:
parallel --pipepart -a big.fq --block -1 --regexp \
--recend '\n' --recstart '@.*(/1| 1:.*)\n[A-Za-z\n\.~]' \
fastq-reader
=head2 EXAMPLE: Processing a big file using more CPUs =head2 EXAMPLE: Processing a big file using more CPUs
To process a big file or some output you can use B<--pipe> to split up To process a big file or some output you can use B<--pipe> to split up

2
src/parsort
View file

@ -122,7 +122,7 @@ GetOptions(
"help" => \$opt::dummy, "help" => \$opt::dummy,
) || exit(255); ) || exit(255);
$Global::progname = ($0 =~ m:(^|/)([^/]+)$:)[1]; $Global::progname = ($0 =~ m:(^|/)([^/]+)$:)[1];
$Global::version = 20210723; $Global::version = 20210822;
if($opt::version) { version(); exit 0; } if($opt::version) { version(); exit 0; }
@Global::sortoptions = @Global::sortoptions =
shell_quote(@ARGV_before[0..($#ARGV_before-$#ARGV-1)]); shell_quote(@ARGV_before[0..($#ARGV_before-$#ARGV-1)]);

2
src/sql
View file

@ -600,7 +600,7 @@ $Global::Initfile && unlink $Global::Initfile;
exit ($err); exit ($err);
sub parse_options { sub parse_options {
$Global::version = 20210723; $Global::version = 20210822;
$Global::progname = 'sql'; $Global::progname = 'sql';
# This must be done first as this may exec myself # This must be done first as this may exec myself

2
testsuite/tests-to-run/parallel-polarhome.sh
View file

@ -86,7 +86,7 @@ doit() {
par_nonall() { par_nonall() {
parallel -j$MAXPROC $RET_TIME_K --delay 0.1 --tag \ parallel -j$MAXPROC $RET_TIME_K --delay 0.1 --tag \
--nonall $S_POLAR -S "1/sshwithpass minix" --argsep ,:- \ --nonall $S_POLAR -S "1/sshwithpass minix" --argsep ,:- \
'source setupenv 2>/dev/null; . `pwd`/setupenv;' "$@" 'source ./setupenv 2>/dev/null; . `pwd`/setupenv;' "$@"
# setupenv contains something like this (adapted to the local path and shell) # setupenv contains something like this (adapted to the local path and shell)
# #
# PATH=$HOME/bin:$PATH:/usr/local/bin # PATH=$HOME/bin:$PATH:/usr/local/bin

34
testsuite/wanted-results/parallel-macos
View file

@ -1,17 +1,17 @@
par_big_func 1 7960 191040 par_big_func 1 7968 191232
par_big_func 1 5376 128964 par_big_func 1 5368 128772
par_big_func_name 1 4952 118848 par_big_func_name 1 4960 119040
par_big_func_name 1 48 1152 par_big_func_name 1 40 960
par_big_var_func_name 1 4960 119040 par_big_var_func_name 1 4956 118944
par_big_var_func_name 1 4960 119040 par_big_var_func_name 1 4956 118944
par_big_var_func_name 1 3416 81924 par_big_var_func_name 1 3424 82116
par_many_args 1 16376 32752 par_many_args 1 16400 32800
par_many_args 1 3624 7248 par_many_args 1 3600 7200
par_many_func 1 4376 105024 par_many_func 1 4372 104928
par_many_func 1 2292 54980 par_many_func 1 2296 55076
par_many_var 1 5116 122784 par_many_var 1 5124 122976
par_many_var 1 1552 37220 par_many_var 1 1544 37028
par_many_var_big_func 1 4388 105312 par_many_var_big_func 1 4404 105696
par_many_var_big_func 1 2280 54692 par_many_var_big_func 1 2264 54308
par_many_var_func 1 4900 117600 par_many_var_func 1 6624 158976
par_many_var_func 1 1768 42404 par_many_var_func 1 44 1028

20
testsuite/wanted-results/parallel-polarhome
View file

@ -395,7 +395,7 @@ freebsd Works on freebsd.polarhome.com
hpux Works on hpux64 hpux Works on hpux64
hpux-ia64 Works on hpux-ia64 hpux-ia64 Works on hpux-ia64
hurd Works on hurd hurd Works on hurd
macosx Works on macosx.polarhome.com macosx Works on macosx
mandriva Works on mandriva.polarhome.com mandriva Works on mandriva.polarhome.com
miros Works on miros.polarhome.com miros Works on miros.polarhome.com
netbsd Works on netbsd.polarhome.com netbsd Works on netbsd.polarhome.com
@ -1210,11 +1210,11 @@ mandriva 1 2 1 2 3 1 2 3 4
miros 1 2 1 2 3 1 2 3 4 miros 1 2 1 2 3 1 2 3 4
netbsd parset: Command not found. netbsd parset: Command not found.
netbsd arr: Undefined variable. netbsd arr: Undefined variable.
openbsd openbsd 1 2 1 2 3 1 2 3 4
openindiana 1 2 1 2 3 1 2 3 4 openindiana 1 2 1 2 3 1 2 3 4
pidora 1 2 1 2 3 1 2 3 4 pidora 1 2 1 2 3 1 2 3 4
qnx qnx
qnx parallel: Warning: Cannot figure out number of cpus. Using 1. qnx parset: Warning: Cannot figure out number of cpus. Using 1.
qnx /bin/sh: syntax error: `(' unexpected qnx /bin/sh: syntax error: `(' unexpected
raspbian 1 2 1 2 3 1 2 3 4 raspbian 1 2 1 2 3 1 2 3 4
redhat 1 2 1 2 3 1 2 3 4 redhat 1 2 1 2 3 1 2 3 4
@ -1241,14 +1241,14 @@ miros 1 2 1 2 1 2
netbsd start=2: Command not found. netbsd start=2: Command not found.
netbsd env_parset: Command not found. netbsd env_parset: Command not found.
netbsd arr: Undefined variable. netbsd arr: Undefined variable.
openbsd openbsd 2 3 3 4 4 5
openindiana 2 2 3 2 3 4 openindiana 2 2 3 2 3 4
pidora 2 2 3 2 3 4 pidora 2 2 3 2 3 4
qnx qnx
qnx /bin/sh: compgen: cannot execute - No such file or directory qnx /bin/sh: compgen: cannot execute - No such file or directory
qnx /bin/sh: compgen: cannot execute - No such file or directory qnx /bin/sh: compgen: cannot execute - No such file or directory
qnx /bin/sh: compgen: cannot execute - No such file or directory qnx /bin/sh: compgen: cannot execute - No such file or directory
qnx parallel: Warning: Cannot figure out number of cpus. Using 1. qnx parset: Warning: Cannot figure out number of cpus. Using 1.
qnx /bin/sh: syntax error: `(' unexpected qnx /bin/sh: syntax error: `(' unexpected
raspbian 2 2 3 2 3 4 raspbian 2 2 3 2 3 4
redhat 2 2 3 2 3 4 redhat 2 2 3 2 3 4
@ -1285,12 +1285,11 @@ mandriva 1 2,1 2 3,1 2 3 4
miros 1 2,1 2 3,1 2 3 4 miros 1 2,1 2 3,1 2 3 4
netbsd parset: Command not found. netbsd parset: Command not found.
netbsd var1: Undefined variable. netbsd var1: Undefined variable.
openbsd ,, openbsd 1 2,1 2 3,1 2 3 4
openindiana 1 2,1 2 3,1 2 3 4 openindiana 1 2,1 2 3,1 2 3 4
pidora 1 2,1 2 3,1 2 3 4 pidora 1 2,1 2 3,1 2 3 4
qnx ,, qnx ,,
qnx parallel: Warning: Cannot figure out number of cpus. Using 1. qnx parset: Warning: Cannot figure out number of cpus. Using 1.
qnx parallel: Warning: Cannot figure out number of cpus. Using 1.
raspbian 1 2,1 2 3,1 2 3 4 raspbian 1 2,1 2 3,1 2 3 4
redhat 1 2,1 2 3,1 2 3 4 redhat 1 2,1 2 3,1 2 3 4
scosysv 1 2,1 2 3,1 2 3 4 scosysv 1 2,1 2 3,1 2 3 4
@ -1315,15 +1314,14 @@ miros 1 2,1 2,1 2
netbsd start=2: Command not found. netbsd start=2: Command not found.
netbsd env_parset: Command not found. netbsd env_parset: Command not found.
netbsd var1: Undefined variable. netbsd var1: Undefined variable.
openbsd ,, openbsd 2 3,3 4,4 5
openindiana 2,2 3,2 3 4 openindiana 2,2 3,2 3 4
pidora 2,2 3,2 3 4 pidora 2,2 3,2 3 4
qnx ,, qnx ,,
qnx /bin/sh: compgen: cannot execute - No such file or directory qnx /bin/sh: compgen: cannot execute - No such file or directory
qnx /bin/sh: compgen: cannot execute - No such file or directory qnx /bin/sh: compgen: cannot execute - No such file or directory
qnx /bin/sh: compgen: cannot execute - No such file or directory qnx /bin/sh: compgen: cannot execute - No such file or directory
qnx parallel: Warning: Cannot figure out number of cpus. Using 1. qnx parset: Warning: Cannot figure out number of cpus. Using 1.
qnx parallel: Warning: Cannot figure out number of cpus. Using 1.
raspbian 2,2 3,2 3 4 raspbian 2,2 3,2 3 4
redhat 2,2 3,2 3 4 redhat 2,2 3,2 3 4
scosysv 2,2 3,2 3 4 scosysv 2,2 3,2 3 4