tange/parallel
1
0
Fork 0
mirror of https://git.savannah.gnu.org/git/parallel.git synced 2024-11-21 21:47:54 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Released as 20191122 ('QuantumSupremacy')

Browse source
This commit is contained in:
Ole Tange 2019-11-23 22:14:03 +01:00
parent 7bab1b45bc
commit 21fb3b6e60
25 changed files with 160 additions and 215 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
Makefile.am
View file

@ -19,8 +19,8 @@ upload:
(echo '#!/bin/bash'; \ (echo '#!/bin/bash'; \
echo; \ echo; \
echo "# To check the signature run:"; \ echo "# To check the signature run:"; \
echo "# echo | gpg"; \ echo "# echo | gpg"; \
echo "# gpg --auto-key-locate keyserver --keyserver-options auto-key-retrieve parallel-$(YYYYMMDD).tar.bz2.sig"; \ echo "# gpg --auto-key-locate keyserver --keyserver-options auto-key-retrieve parallel-$(YYYYMMDD).tar.bz2.sig"; \
echo; \ echo; \
echo "echo | gpg 2>/dev/null"; \ echo "echo | gpg 2>/dev/null"; \
echo 'gpg --auto-key-locate keyserver --keyserver-options auto-key-retrieve $$0'; \ echo 'gpg --auto-key-locate keyserver --keyserver-options auto-key-retrieve $$0'; \
@ -32,8 +32,8 @@ upload:
(echo '#!/bin/bash'; \ (echo '#!/bin/bash'; \
echo; \ echo; \
echo "# To check the signature run:"; \ echo "# To check the signature run:"; \
echo "# echo | gpg"; \ echo "# echo | gpg"; \
echo "# gpg --auto-key-locate keyserver --keyserver-options auto-key-retrieve parallel-latest.tar.bz2.sig"; \ echo "# gpg --auto-key-locate keyserver --keyserver-options auto-key-retrieve parallel-latest.tar.bz2.sig"; \
echo; \ echo; \
echo "echo | gpg 2>/dev/null"; \ echo "echo | gpg 2>/dev/null"; \
echo 'gpg --auto-key-locate keyserver --keyserver-options auto-key-retrieve $$0'; \ echo 'gpg --auto-key-locate keyserver --keyserver-options auto-key-retrieve $$0'; \

14
NEWS
View file

@ -1,3 +1,17 @@
20191122
* GNU Parallel was presented at Driving IT. Slides:
cloud.prosa.dk/s/drivingit
* Restarting supervisord processes in parallel
https://blog.naderman.de/2019/11/14/restarting-supervisord-processes-in-parallel/
* Using GNU Parallel to Create Archives Faster
https://www.reddit.com/r/DataHoarder/comments/dsgmhc/parallel_archiving_techniques/
* Bug fixes and man page updates.
20191022 20191022
* --tee will use --output-error=warn-nopipe if supported by tee. * --tee will use --output-error=warn-nopipe if supported by tee.

35
README
View file

@ -24,6 +24,9 @@ GNU Parallel makes sure output from the commands is the same output as
you would get had you run the commands sequentially. This makes it you would get had you run the commands sequentially. This makes it
possible to use output from GNU Parallel as input for other programs. possible to use output from GNU Parallel as input for other programs.
See the cheat sheet for examples that cover most of the daily usage:
www.gnu.org/s/parallel/parallel_cheat.pdf
= 10 seconds installation = = 10 seconds installation =
@ -54,11 +57,11 @@ document.
Full installation of GNU Parallel is as simple as: Full installation of GNU Parallel is as simple as:
wget https://ftpmirror.gnu.org/parallel/parallel-20191022.tar.bz2 wget https://ftpmirror.gnu.org/parallel/parallel-20191122.tar.bz2
wget https://ftpmirror.gnu.org/parallel/parallel-20191022.tar.bz2.sig wget https://ftpmirror.gnu.org/parallel/parallel-20191122.tar.bz2.sig
gpg parallel-20191022.tar.bz2.sig gpg parallel-20191122.tar.bz2.sig
bzip2 -dc parallel-20191022.tar.bz2 | tar xvf - bzip2 -dc parallel-20191122.tar.bz2 | tar xvf -
cd parallel-20191022 cd parallel-20191122
./configure && make && sudo make install ./configure && make && sudo make install
@ -67,11 +70,11 @@ Full installation of GNU Parallel is as simple as:
If you are not root you can add ~/bin to your path and install in If you are not root you can add ~/bin to your path and install in
~/bin and ~/share: ~/bin and ~/share:
wget https://ftpmirror.gnu.org/parallel/parallel-20191022.tar.bz2 wget https://ftpmirror.gnu.org/parallel/parallel-20191122.tar.bz2
wget https://ftpmirror.gnu.org/parallel/parallel-20191022.tar.bz2.sig wget https://ftpmirror.gnu.org/parallel/parallel-20191122.tar.bz2.sig
gpg parallel-20191022.tar.bz2.sig gpg parallel-20191122.tar.bz2.sig
bzip2 -dc parallel-20191022.tar.bz2 | tar xvf - bzip2 -dc parallel-20191122.tar.bz2 | tar xvf -
cd parallel-20191022 cd parallel-20191122
./configure --prefix=$HOME && make && make install ./configure --prefix=$HOME && make && make install
Or if your system lacks 'make' you can simply copy src/parallel Or if your system lacks 'make' you can simply copy src/parallel
@ -89,6 +92,18 @@ system is old or Microsoft Windows):
mv parallel sem dir-in-your-$PATH/bin/ mv parallel sem dir-in-your-$PATH/bin/
= Installation on compute servers =
If you are developing your script to run on a remote server, that does
not have GNU Parallel installed, but you have it installed on you
development machine, the use can use `parallel --embed`.
parallel --embed > newscript.sh
Just edit the last lines of newscript.sh and copy it to the compute
server.
= Test the installation = = Test the installation =
After this you should be able to do: After this you should be able to do:

20
configure vendored
View file

@ -1,6 +1,6 @@
#! /bin/sh #! /bin/sh
# Guess values for system-dependent variables and create Makefiles. # Guess values for system-dependent variables and create Makefiles.
# Generated by GNU Autoconf 2.69 for parallel 20191022. # Generated by GNU Autoconf 2.69 for parallel 20191122.
# #
# Report bugs to <bug-parallel@gnu.org>. # Report bugs to <bug-parallel@gnu.org>.
# #
@ -579,8 +579,8 @@ MAKEFLAGS=
# Identity of this package. # Identity of this package.
PACKAGE_NAME='parallel' PACKAGE_NAME='parallel'
PACKAGE_TARNAME='parallel' PACKAGE_TARNAME='parallel'
PACKAGE_VERSION='20191022' PACKAGE_VERSION='20191122'
PACKAGE_STRING='parallel 20191022' PACKAGE_STRING='parallel 20191122'
PACKAGE_BUGREPORT='bug-parallel@gnu.org' PACKAGE_BUGREPORT='bug-parallel@gnu.org'
PACKAGE_URL='' PACKAGE_URL=''
@ -1214,7 +1214,7 @@ if test "$ac_init_help" = "long"; then
# Omit some internal or obsolete options to make the list less imposing. # Omit some internal or obsolete options to make the list less imposing.
# This message is too long to be a string in the A/UX 3.1 sh. # This message is too long to be a string in the A/UX 3.1 sh.
cat <<_ACEOF cat <<_ACEOF
\`configure' configures parallel 20191022 to adapt to many kinds of systems. \`configure' configures parallel 20191122 to adapt to many kinds of systems.
Usage: $0 [OPTION]... [VAR=VALUE]... Usage: $0 [OPTION]... [VAR=VALUE]...
@ -1281,7 +1281,7 @@ fi
if test -n "$ac_init_help"; then if test -n "$ac_init_help"; then
case $ac_init_help in case $ac_init_help in
short | recursive ) echo "Configuration of parallel 20191022:";; short | recursive ) echo "Configuration of parallel 20191122:";;
esac esac
cat <<\_ACEOF cat <<\_ACEOF
@ -1357,7 +1357,7 @@ fi
test -n "$ac_init_help" && exit $ac_status test -n "$ac_init_help" && exit $ac_status
if $ac_init_version; then if $ac_init_version; then
cat <<\_ACEOF cat <<\_ACEOF
parallel configure 20191022 parallel configure 20191122
generated by GNU Autoconf 2.69 generated by GNU Autoconf 2.69
Copyright (C) 2012 Free Software Foundation, Inc. Copyright (C) 2012 Free Software Foundation, Inc.
@ -1374,7 +1374,7 @@ cat >config.log <<_ACEOF
This file contains any messages produced by compilers while This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake. running configure, to aid debugging if configure makes a mistake.
It was created by parallel $as_me 20191022, which was It was created by parallel $as_me 20191122, which was
generated by GNU Autoconf 2.69. Invocation command line was generated by GNU Autoconf 2.69. Invocation command line was
$ $0 $@ $ $0 $@
@ -2237,7 +2237,7 @@ fi
# Define the identity of the package. # Define the identity of the package.
PACKAGE='parallel' PACKAGE='parallel'
VERSION='20191022' VERSION='20191122'
cat >>confdefs.h <<_ACEOF cat >>confdefs.h <<_ACEOF
@ -2880,7 +2880,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1
# report actual input values of CONFIG_FILES etc. instead of their # report actual input values of CONFIG_FILES etc. instead of their
# values after options handling. # values after options handling.
ac_log=" ac_log="
This file was extended by parallel $as_me 20191022, which was This file was extended by parallel $as_me 20191122, which was
generated by GNU Autoconf 2.69. Invocation command line was generated by GNU Autoconf 2.69. Invocation command line was
CONFIG_FILES = $CONFIG_FILES CONFIG_FILES = $CONFIG_FILES
@ -2942,7 +2942,7 @@ _ACEOF
cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1
ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`" ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`"
ac_cs_version="\\ ac_cs_version="\\
parallel config.status 20191022 parallel config.status 20191122
configured by $0, generated by GNU Autoconf 2.69, configured by $0, generated by GNU Autoconf 2.69,
with options \\"\$ac_cs_config\\" with options \\"\$ac_cs_config\\"

2
configure.ac
View file

@ -1,4 +1,4 @@
AC_INIT([parallel], [20191022], [bug-parallel@gnu.org]) AC_INIT([parallel], [20191122], [bug-parallel@gnu.org])
AM_INIT_AUTOMAKE([-Wall -Werror foreign]) AM_INIT_AUTOMAKE([-Wall -Werror foreign])
AC_CONFIG_HEADERS([config.h]) AC_CONFIG_HEADERS([config.h])
AC_CONFIG_FILES([ AC_CONFIG_FILES([

7
doc/haikus
View file

@ -12,9 +12,6 @@ GNU parallel has helped me kill a Hadoop cluster before.
GNU Parallel is awesome GNU Parallel is awesome
-- Gaurav Jain @gauravjain49@twitter -- Gaurav Jain @gauravjain49@twitter
[L]earning about parallel was amazing for me, it gives us many beautiful solutions.
-- SergioAraujo@stackoverflow
With multicore systems everywhere GNU Parallel is a must have tool. With multicore systems everywhere GNU Parallel is a must have tool.
-- Neil H. Watson @neil_h_watson@twitter -- Neil H. Watson @neil_h_watson@twitter
@ -58,6 +55,10 @@ GNU parallel has helped me kill a Hadoop cluster before.
=== Used === === Used ===
[L]earning about parallel was amazing for me, it gives us many beautiful solutions.
-- SergioAraujo@stackoverflow
I've said it before: The command line program GNU Parallel is a godsend. I've said it before: The command line program GNU Parallel is a godsend.
-- Jo Chr. Oterhals @oterhals -- Jo Chr. Oterhals @oterhals

10
doc/release_new_version
View file

@ -209,9 +209,9 @@ from:tange@gnu.org
to:parallel@gnu.org, bug-parallel@gnu.org to:parallel@gnu.org, bug-parallel@gnu.org
stable-bcc: Jesse Alama <jessealama@fastmail.fm> stable-bcc: Jesse Alama <jessealama@fastmail.fm>
Subject: GNU Parallel 20191122 ('Berlin/Morales') released <<[stable]>> Subject: GNU Parallel 20191122 ('Quantum Supremacy') released <<[stable]>>
GNU Parallel 20191122 ('') <<[stable]>> has been released. It is available for download at: http://ftpmirror.gnu.org/parallel/ GNU Parallel 20191122 ('Quantum Supremacy') <<[stable]>> has been released. It is available for download at: http://ftpmirror.gnu.org/parallel/
<<No new functionality was introduced so this is a good candidate for a stable release.>> <<No new functionality was introduced so this is a good candidate for a stable release.>>
@ -224,11 +224,11 @@ Quote of the month:
New in this release: New in this release:
* GNU Parallel was presented at Driving IT. Slides: * GNU Parallel was presented at Driving IT. Slides: cloud.prosa.dk/s/drivingit
https://blog.naderman.de/2019/11/14/restarting-supervisord-processes-in-parallel/ * Restarting supervisord processes in parallel https://blog.naderman.de/2019/11/14/restarting-supervisord-processes-in-parallel/
* https://www.reddit.com/r/DataHoarder/comments/dsgmhc/parallel_archiving_techniques/ * Using GNU Parallel to Create Archives Faster https://www.reddit.com/r/DataHoarder/comments/dsgmhc/parallel_archiving_techniques/
* Bug fixes and man page updates. * Bug fixes and man page updates.

10
packager/obs/home:tange/parallel/.osc/_files
View file

@ -1,6 +1,6 @@
<directory name="parallel" rev="244" srcmd5="0bc3563ade8042405b36915c03015124" vrev="1"> <directory name="parallel" rev="249" srcmd5="55f61ae48d885e5fee7483aa06d5e94e" vrev="5">
<entry md5="457471b4306f1a2b356691444247a717" mtime="1571685322" name="parallel-20191022.tar.bz2" size="2080451" /> <entry md5="d1b5fdb9a5a1e8c036a075145fe99b9e" mtime="1574501414" name="parallel-20191122.tar.bz2" size="2046514" />
<entry md5="c8e6f66a12226dfda64fb14b8cdc7159" mtime="1571685322" name="parallel.spec" size="4751" /> <entry md5="e8aa083aa14c9c749148482c1c111403" mtime="1574501420" name="parallel.spec" size="4701" />
<entry md5="04f57f67c60bc39a33adbe69b11cb257" mtime="1571685322" name="parallel_20191022.dsc" size="556" /> <entry md5="8a94dce54ceb80ea0349ce06f69531a8" mtime="1574473244" name="parallel_20191122.dsc" size="556" />
<entry md5="5a61e4a88a27b355ce12f58023d8a26a" mtime="1571685323" name="parallel_20191022.tar.gz" size="2257704" /> <entry md5="8032b789c1faf4b70425aaaaf7db5995" mtime="1574473244" name="parallel_20191122.tar.gz" size="2234606" />
</directory> </directory>

8
packager/obs/home:tange/parallel/.osc/parallel.spec
View file

@ -1,7 +1,7 @@
Summary: Shell tool for executing jobs in parallel Summary: Shell tool for executing jobs in parallel
Name: parallel Name: parallel
Version: 20191022 Version: 20191122
Release: 1.3 Release: 1.3
License: GPL-3.0-or-later License: GPL-3.0-or-later
Group: Productivity/File utilities Group: Productivity/File utilities
@ -88,7 +88,7 @@ rm $RPM_BUILD_ROOT%{_docdir}/sem.pdf
rm $RPM_BUILD_ROOT%{_docdir}/sql.pdf rm $RPM_BUILD_ROOT%{_docdir}/sql.pdf
rm $RPM_BUILD_ROOT%{_docdir}/parcat.pdf rm $RPM_BUILD_ROOT%{_docdir}/parcat.pdf
rm $RPM_BUILD_ROOT%{_docdir}/parset.pdf rm $RPM_BUILD_ROOT%{_docdir}/parset.pdf
rm $RPM_BUILD_ROOT%{_docdir}/parallel_cheat.pdf rm $RPM_BUILD_ROOT%{_docdir}/parallel_cheat_bw.pdf
%clean %clean
rm -rf $RPM_BUILD_ROOT rm -rf $RPM_BUILD_ROOT
@ -98,7 +98,7 @@ rm -rf $RPM_BUILD_ROOT
/usr/bin/* /usr/bin/*
/usr/share/man/man1/* /usr/share/man/man1/*
/usr/share/man/man7/* /usr/share/man/man7/*
%doc README NEWS src/parallel.html src/env_parallel.html src/parallel_tutorial.html src/parallel_design.html src/parallel_alternatives.html src/parallel_book.html src/sem.html src/sql.html src/parcat.html src/parset.html src/niceload.html src/parallel.texi src/env_parallel.texi src/parallel_tutorial.texi src/parallel_design.texi src/parallel_alternatives.texi src/parallel_book.texi src/niceload.texi src/sem.texi src/sql.texi src/parcat.texi src/parset.texi src/parallel.pdf src/env_parallel.pdf src/parallel_tutorial.pdf src/parallel_design.pdf src/parallel_alternatives.pdf src/parallel_book.pdf src/niceload.pdf src/sem.pdf src/sql.pdf src/parcat.pdf src/parset.pdf src/parallel_cheat.pdf %doc README NEWS src/parallel.html src/env_parallel.html src/parallel_tutorial.html src/parallel_design.html src/parallel_alternatives.html src/parallel_book.html src/sem.html src/sql.html src/parcat.html src/parset.html src/niceload.html src/parallel.texi src/env_parallel.texi src/parallel_tutorial.texi src/parallel_design.texi src/parallel_alternatives.texi src/parallel_book.texi src/niceload.texi src/sem.texi src/sql.texi src/parcat.texi src/parset.texi src/parallel.pdf src/env_parallel.pdf src/parallel_tutorial.pdf src/parallel_design.pdf src/parallel_alternatives.pdf src/parallel_book.pdf src/niceload.pdf src/sem.pdf src/sql.pdf src/parcat.pdf src/parset.pdf src/parallel_cheat_bw.pdf
%changelog %changelog
* Sat Jan 22 2011 Ole Tange * Sat Jan 22 2011 Ole Tange
@ -111,6 +111,4 @@ rm -rf $RPM_BUILD_ROOT
- Upgrade to current git-version of source. Tested on build.opensuse.org - Upgrade to current git-version of source. Tested on build.opensuse.org
* Fri Aug 27 2010 Ole Tange * Fri Aug 27 2010 Ole Tange
- Untested upgrade to current git-version of source. - Untested upgrade to current git-version of source.
* Sat Aug 08 2010 Markus Ammer
- Initial package setup.

8
packager/obs/home:tange/parallel/parallel.spec
View file

@ -1,7 +1,7 @@
Summary: Shell tool for executing jobs in parallel Summary: Shell tool for executing jobs in parallel
Name: parallel Name: parallel
Version: 20191022 Version: 20191122
Release: 1.3 Release: 1.3
License: GPL-3.0-or-later License: GPL-3.0-or-later
Group: Productivity/File utilities Group: Productivity/File utilities
@ -88,7 +88,7 @@ rm $RPM_BUILD_ROOT%{_docdir}/sem.pdf
rm $RPM_BUILD_ROOT%{_docdir}/sql.pdf rm $RPM_BUILD_ROOT%{_docdir}/sql.pdf
rm $RPM_BUILD_ROOT%{_docdir}/parcat.pdf rm $RPM_BUILD_ROOT%{_docdir}/parcat.pdf
rm $RPM_BUILD_ROOT%{_docdir}/parset.pdf rm $RPM_BUILD_ROOT%{_docdir}/parset.pdf
rm $RPM_BUILD_ROOT%{_docdir}/parallel_cheat.pdf rm $RPM_BUILD_ROOT%{_docdir}/parallel_cheat_bw.pdf
%clean %clean
rm -rf $RPM_BUILD_ROOT rm -rf $RPM_BUILD_ROOT
@ -98,7 +98,7 @@ rm -rf $RPM_BUILD_ROOT
/usr/bin/* /usr/bin/*
/usr/share/man/man1/* /usr/share/man/man1/*
/usr/share/man/man7/* /usr/share/man/man7/*
%doc README NEWS src/parallel.html src/env_parallel.html src/parallel_tutorial.html src/parallel_design.html src/parallel_alternatives.html src/parallel_book.html src/sem.html src/sql.html src/parcat.html src/parset.html src/niceload.html src/parallel.texi src/env_parallel.texi src/parallel_tutorial.texi src/parallel_design.texi src/parallel_alternatives.texi src/parallel_book.texi src/niceload.texi src/sem.texi src/sql.texi src/parcat.texi src/parset.texi src/parallel.pdf src/env_parallel.pdf src/parallel_tutorial.pdf src/parallel_design.pdf src/parallel_alternatives.pdf src/parallel_book.pdf src/niceload.pdf src/sem.pdf src/sql.pdf src/parcat.pdf src/parset.pdf src/parallel_cheat.pdf %doc README NEWS src/parallel.html src/env_parallel.html src/parallel_tutorial.html src/parallel_design.html src/parallel_alternatives.html src/parallel_book.html src/sem.html src/sql.html src/parcat.html src/parset.html src/niceload.html src/parallel.texi src/env_parallel.texi src/parallel_tutorial.texi src/parallel_design.texi src/parallel_alternatives.texi src/parallel_book.texi src/niceload.texi src/sem.texi src/sql.texi src/parcat.texi src/parset.texi src/parallel.pdf src/env_parallel.pdf src/parallel_tutorial.pdf src/parallel_design.pdf src/parallel_alternatives.pdf src/parallel_book.pdf src/niceload.pdf src/sem.pdf src/sql.pdf src/parcat.pdf src/parset.pdf src/parallel_cheat_bw.pdf
%changelog %changelog
* Sat Jan 22 2011 Ole Tange * Sat Jan 22 2011 Ole Tange
@ -111,6 +111,4 @@ rm -rf $RPM_BUILD_ROOT
- Upgrade to current git-version of source. Tested on build.opensuse.org - Upgrade to current git-version of source. Tested on build.opensuse.org
* Fri Aug 27 2010 Ole Tange * Fri Aug 27 2010 Ole Tange
- Untested upgrade to current git-version of source. - Untested upgrade to current git-version of source.
* Sat Aug 08 2010 Markus Ammer
- Initial package setup.

12
src/Makefile.am
View file

@ -23,7 +23,7 @@ doc_DATA = parallel.html env_parallel.html sem.html sql.html \
parallel.pdf env_parallel.pdf sem.pdf sql.pdf niceload.pdf \ parallel.pdf env_parallel.pdf sem.pdf sql.pdf niceload.pdf \
parallel_tutorial.pdf parallel_book.pdf parallel_design.pdf \ parallel_tutorial.pdf parallel_book.pdf parallel_design.pdf \
parallel_alternatives.pdf parcat.pdf parset.pdf \ parallel_alternatives.pdf parcat.pdf parset.pdf \
parallel_cheat.pdf parallel_cheat_bw.pdf
endif endif
# Build documentation file if the tool to build exists. # Build documentation file if the tool to build exists.
@ -258,9 +258,9 @@ parset.pdf: parset.pod
pod2pdf --output-file $(srcdir)/parset.pdf $(srcdir)/parset.pod --title "GNU parset" \ pod2pdf --output-file $(srcdir)/parset.pdf $(srcdir)/parset.pod --title "GNU parset" \
|| echo "Warning: pod2pdf not found. Using old parset.pdf" || echo "Warning: pod2pdf not found. Using old parset.pdf"
parallel_cheat.pdf: parallel_cheat.fodt parallel_cheat_bw.pdf: parallel_cheat_bw.fodt
libreoffice --headless --convert-to pdf parallel_cheat.fodt \ libreoffice --headless --convert-to pdf parallel_cheat_bw.fodt \
|| echo "Warning: libreoffice failed. Using old parallel_cheat.pdf" || echo "Warning: libreoffice failed. Using old parallel_cheat_bw.pdf"
sem: parallel sem: parallel
ln -fs parallel sem ln -fs parallel sem
@ -279,7 +279,7 @@ DISTCLEANFILES = parallel.1 env_parallel.1 sem.1 sql.1 niceload.1 \
parallel.pdf env_parallel.pdf sem.pdf sql.pdf niceload.pdf \ parallel.pdf env_parallel.pdf sem.pdf sql.pdf niceload.pdf \
parallel_tutorial.pdf parallel_book.pdf parallel_design.pdf \ parallel_tutorial.pdf parallel_book.pdf parallel_design.pdf \
parallel_alternatives.pdf parcat.pdf parset.pdf \ parallel_alternatives.pdf parcat.pdf parset.pdf \
parallel_cheat.pdf parallel_cheat_bw.pdf
EXTRA_DIST = parallel sem sql niceload parcat parset env_parallel \ EXTRA_DIST = parallel sem sql niceload parcat parset env_parallel \
env_parallel.ash env_parallel.bash env_parallel.csh \ env_parallel.ash env_parallel.bash env_parallel.csh \
@ -288,4 +288,4 @@ EXTRA_DIST = parallel sem sql niceload parcat parset env_parallel \
env_parallel.tcsh env_parallel.zsh parcat.pod parset.pod \ env_parallel.tcsh env_parallel.zsh parcat.pod parset.pod \
sem.pod parallel.pod env_parallel.pod niceload.pod \ sem.pod parallel.pod env_parallel.pod niceload.pod \
parallel_tutorial.pod parallel_book.pod parallel_design.pod \ parallel_tutorial.pod parallel_book.pod parallel_design.pod \
parallel_alternatives.pod parallel_cheat.fodt $(DISTCLEANFILES) parallel_alternatives.pod parallel_cheat_bw.fodt $(DISTCLEANFILES)

12
src/Makefile.in
View file

@ -250,7 +250,7 @@ bin_SCRIPTS = parallel sql niceload parcat parset env_parallel \
@DOCUMENTATION_TRUE@ parallel.pdf env_parallel.pdf sem.pdf sql.pdf niceload.pdf \ @DOCUMENTATION_TRUE@ parallel.pdf env_parallel.pdf sem.pdf sql.pdf niceload.pdf \
@DOCUMENTATION_TRUE@ parallel_tutorial.pdf parallel_book.pdf parallel_design.pdf \ @DOCUMENTATION_TRUE@ parallel_tutorial.pdf parallel_book.pdf parallel_design.pdf \
@DOCUMENTATION_TRUE@ parallel_alternatives.pdf parcat.pdf parset.pdf \ @DOCUMENTATION_TRUE@ parallel_alternatives.pdf parcat.pdf parset.pdf \
@DOCUMENTATION_TRUE@ parallel_cheat.pdf @DOCUMENTATION_TRUE@ parallel_cheat_bw.pdf
DISTCLEANFILES = parallel.1 env_parallel.1 sem.1 sql.1 niceload.1 \ DISTCLEANFILES = parallel.1 env_parallel.1 sem.1 sql.1 niceload.1 \
parallel_tutorial.7 parallel_book.7 parallel_design.7 \ parallel_tutorial.7 parallel_book.7 parallel_design.7 \
@ -266,7 +266,7 @@ DISTCLEANFILES = parallel.1 env_parallel.1 sem.1 sql.1 niceload.1 \
parallel.pdf env_parallel.pdf sem.pdf sql.pdf niceload.pdf \ parallel.pdf env_parallel.pdf sem.pdf sql.pdf niceload.pdf \
parallel_tutorial.pdf parallel_book.pdf parallel_design.pdf \ parallel_tutorial.pdf parallel_book.pdf parallel_design.pdf \
parallel_alternatives.pdf parcat.pdf parset.pdf \ parallel_alternatives.pdf parcat.pdf parset.pdf \
parallel_cheat.pdf parallel_cheat_bw.pdf
EXTRA_DIST = parallel sem sql niceload parcat parset env_parallel \ EXTRA_DIST = parallel sem sql niceload parcat parset env_parallel \
env_parallel.ash env_parallel.bash env_parallel.csh \ env_parallel.ash env_parallel.bash env_parallel.csh \
@ -275,7 +275,7 @@ EXTRA_DIST = parallel sem sql niceload parcat parset env_parallel \
env_parallel.tcsh env_parallel.zsh parcat.pod parset.pod \ env_parallel.tcsh env_parallel.zsh parcat.pod parset.pod \
sem.pod parallel.pod env_parallel.pod niceload.pod \ sem.pod parallel.pod env_parallel.pod niceload.pod \
parallel_tutorial.pod parallel_book.pod parallel_design.pod \ parallel_tutorial.pod parallel_book.pod parallel_design.pod \
parallel_alternatives.pod parallel_cheat.fodt $(DISTCLEANFILES) parallel_alternatives.pod parallel_cheat_bw.fodt $(DISTCLEANFILES)
all: all-am all: all-am
@ -850,9 +850,9 @@ parset.pdf: parset.pod
pod2pdf --output-file $(srcdir)/parset.pdf $(srcdir)/parset.pod --title "GNU parset" \ pod2pdf --output-file $(srcdir)/parset.pdf $(srcdir)/parset.pod --title "GNU parset" \
|| echo "Warning: pod2pdf not found. Using old parset.pdf" || echo "Warning: pod2pdf not found. Using old parset.pdf"
parallel_cheat.pdf: parallel_cheat.fodt parallel_cheat_bw.pdf: parallel_cheat_bw.fodt
libreoffice --headless --convert-to pdf parallel_cheat.fodt \ libreoffice --headless --convert-to pdf parallel_cheat_bw.fodt \
|| echo "Warning: libreoffice failed. Using old parallel_cheat.pdf" || echo "Warning: libreoffice failed. Using old parallel_cheat_bw.pdf"
sem: parallel sem: parallel
ln -fs parallel sem ln -fs parallel sem

2
src/niceload
View file

@ -23,7 +23,7 @@
use strict; use strict;
use Getopt::Long; use Getopt::Long;
$Global::progname="niceload"; $Global::progname="niceload";
$Global::version = 20191023; $Global::version = 20191122;
Getopt::Long::Configure("bundling","require_order"); Getopt::Long::Configure("bundling","require_order");
get_options_from_array(\@ARGV) || die_usage(); get_options_from_array(\@ARGV) || die_usage();
if($opt::version) { if($opt::version) {

2
src/parallel
View file

@ -2071,7 +2071,7 @@ sub check_invalid_option_combinations() {
sub init_globals() { sub init_globals() {
# Defaults: # Defaults:
$Global::version = 20191023; $Global::version = 20191122;
$Global::progname = 'parallel'; $Global::progname = 'parallel';
$::name = "GNU Parallel"; $::name = "GNU Parallel";
$Global::infinity = 2**31; $Global::infinity = 2**31;

0
src/parallel_cheat.fodt → src/parallel_cheat_bw.fodt
View file

2
src/sql
View file

@ -574,7 +574,7 @@ $Global::Initfile && unlink $Global::Initfile;
exit ($err); exit ($err);
sub parse_options { sub parse_options {
$Global::version = 20191023; $Global::version = 20191122;
$Global::progname = 'sql'; $Global::progname = 'sql';
# This must be done first as this may exec myself # This must be done first as this may exec myself

17
testsuite/tests-to-run/parallel-android.sh Executable file
View file

@ -0,0 +1,17 @@
#!/bin/bash
unset TIMEOUT
. `which env_parallel.bash`
env_parallel --session
host=$(parallel -j0 --halt now,success=1 ssh {} echo {} ::: android1 android2 2>/dev/null)
echo $host >&2
doit() {
export PARALLEL_SSH='ssh -p2222'
parallel -k echo ::: Basic usage works
parallel -k -S localhost echo ::: Remote usage works
}
scp /usr/local/bin/parallel $host:/data/data/com.termux/files/usr/bin
env_parallel -S $host doit ::: a

34
testsuite/tests-to-run/parallel-local-1s.sh
View file

@ -185,7 +185,7 @@ par_too_long_line_X() {
par_test_cpu_detection() { par_test_cpu_detection() {
# Xeon 8 core server in Germany # Xeon 8 core server in Germany
cpuinfo1=" cpu1="
processor : 0 processor : 0
vendor_id : GenuineIntel vendor_id : GenuineIntel
cpu family : 6 cpu family : 6
@ -387,7 +387,7 @@ address sizes : 38 bits physical, 48 bits virtual
power management: power management:
"; ";
# Core i7-3632QM Acer laptop # Core i7-3632QM Acer laptop
cpuinfo2=" cpu2="
processor : 0 processor : 0
vendor_id : GenuineIntel vendor_id : GenuineIntel
cpu family : 6 cpu family : 6
@ -605,7 +605,7 @@ address sizes : 36 bits physical, 48 bits virtual
power management: power management:
"; ";
# Core i5-2410M laptop firewall # Core i5-2410M laptop firewall
cpuinfo3=" cpu3="
processor : 0 processor : 0
vendor_id : GenuineIntel vendor_id : GenuineIntel
cpu family : 6 cpu family : 6
@ -715,7 +715,7 @@ address sizes : 36 bits physical, 48 bits virtual
power management: power management:
"; ";
# #
cpuinfo4=" cpu4="
processor : 0 processor : 0
vendor_id : AuthenticAMD vendor_id : AuthenticAMD
cpu family : 15 cpu family : 15
@ -758,7 +758,7 @@ bogomips : 3616.94
clflush size : 64 clflush size : 64
power management: ts fid vid ttp power management: ts fid vid ttp
"; ";
cpuinfo5=" cpu5="
processor : 0 processor : 0
vendor_id : GenuineIntel vendor_id : GenuineIntel
cpu family : 6 cpu family : 6
@ -2008,7 +2008,7 @@ address sizes : 46 bits physical, 48 bits virtual
power management: power management:
"; ";
# HP Laptop Compaq 6530b # HP Laptop Compaq 6530b
cpuinfo6=" cpu6="
processor : 0 processor : 0
vendor_id : GenuineIntel vendor_id : GenuineIntel
cpu family : 6 cpu family : 6
@ -2065,7 +2065,7 @@ power management:
"; ";
# Huawei P Smart Octa-core (4x2.36 GHz Cortex-A53 & 4x1.7 GHz Cortex-A53) # Huawei P Smart Octa-core (4x2.36 GHz Cortex-A53 & 4x1.7 GHz Cortex-A53)
cpuinfo7=" cpu7="
processor : 0 processor : 0
BogoMIPS : 3.84 BogoMIPS : 3.84
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 Features : fp asimd evtstrm aes pmull sha1 sha2 crc32
@ -2138,6 +2138,22 @@ CPU variant : 0x0
CPU part : 0xd03 CPU part : 0xd03
CPU revision : 4 CPU revision : 4
";
# x96 quad-core Android
cpu8="
Processor : AArch64 Processor rev 4 (aarch64)
processor : 0
processor : 2
processor : 3
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 wp half thumb fastmult vfp edsp neon vfpv3 tlsi vfpv4 idiva idivt
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x0
CPU part : 0xd03
CPU revision : 4
Hardware : Amlogic
Serial : 210a82004906ca55455227aefac9be20
"; ";
test_one() { test_one() {
export PARALLEL_CPUINFO="$1" export PARALLEL_CPUINFO="$1"
@ -2148,8 +2164,8 @@ CPU revision : 4
} }
export -f test_one export -f test_one
parallel -j0 -0 -k --tagstring {2} test_one {1} \ parallel -j0 -0 -k --tagstring {2} test_one {1} \
::: "$cpuinfo1" "$cpuinfo2" "$cpuinfo3" "$cpuinfo4" "$cpuinfo5" "$cpuinfo6" "$cpuinfo7" \ ::: "$cpu1" "$cpu2" "$cpu3" "$cpu4" "$cpu5" "$cpu6" "$cpu7" "$cpu8" \
:::+ "2-8-8-8" "1-4-8-4" "1-2-4-2" "1-2-2-2" "2-24-48-24" "1-2-2-2" "1-8-8-8" :::+ 2-8-8-8 1-4-8-4 1-2-4-2 1-2-2-2 2-24-48-24 1-2-2-2 1-8-8-8 1-4-4-4
} }
par_null_resume() { par_null_resume() {

13
testsuite/tests-to-run/parallel-local-3s.sh
View file

@ -7,11 +7,9 @@
par_tee_with_premature_close() { par_tee_with_premature_close() {
echo '--tee --pipe should send all data to all commands' echo '--tee --pipe should send all data to all commands'
echo 'even if a command closes stdin before reading everything' echo 'even if a command closes stdin before reading everything'
seq 1000000 | echo 'tee with --output-error=warn-nopipe support'
parallel -k --tee --pipe \ seq 1000000 | parallel -k --tee --pipe ::: wc head tail 'sleep 1'
::: 'sleep 1' 'sleep 2;wc' 'sleep 2;head' 'sleep 2;tail' echo 'tee without --output-error=warn-nopipe support'
# tee without --output-error=warn-nopipe support
cat > tmp/tee <<-EOF cat > tmp/tee <<-EOF
#!/usr/bin/perl #!/usr/bin/perl
@ -25,9 +23,8 @@ par_tee_with_premature_close() {
# This gives incomplete output due to: # This gives incomplete output due to:
# * tee not supporting --output-error=warn-nopipe # * tee not supporting --output-error=warn-nopipe
# * sleep closes stdin before EOF # * sleep closes stdin before EOF
seq 1000000 | # Depending on tee it may provide partial output or no output
parallel -k --tee --pipe \ seq 1000000 | parallel -k --tee --pipe ::: wc head tail 'sleep 1'
::: 'sleep 1' 'sleep 2;wc' 'sleep 2;head' 'sleep 2;tail'
echo echo
} }

2
testsuite/tests-to-run/parallel-tutorial.sh
View file

@ -82,6 +82,8 @@ perl -ne '$/="\n\n"; /^Output/../^[^O]\S/ and next; /^ / and print;' ../../src/
s{rsync error: some files/attrs were not transferred .*\n}{}; s{rsync error: some files/attrs were not transferred .*\n}{};
s{.* GtkDialog .*\n}{}; s{.* GtkDialog .*\n}{};
s{tried 1}{}; s{tried 1}{};
s/^\s*\n//;
s/^Second done\n//;
' | ' |
perl -ne '/GTK2_RC_FILES/ and next; perl -ne '/GTK2_RC_FILES/ and next;
/GTK_RC_FILES/ and next; /GTK_RC_FILES/ and next;

6
testsuite/wanted-results/parallel-android Normal file
View file

@ -0,0 +1,6 @@
Basic
usage
works
Remote
usage
works

4
testsuite/wanted-results/parallel-local-100s
View file

@ -409,7 +409,7 @@ par_test_build_and_install rm /tmp/parallel-install/bin/sem || true
par_test_build_and_install ln -s parallel /tmp/parallel-install/bin/sem par_test_build_and_install ln -s parallel /tmp/parallel-install/bin/sem
par_test_build_and_install make[0]: Leaving directory '/tmp/parallel-00000000/src' par_test_build_and_install make[0]: Leaving directory '/tmp/parallel-00000000/src'
par_test_build_and_install /bin/mkdir -p '/tmp/parallel-install/share/doc/parallel' par_test_build_and_install /bin/mkdir -p '/tmp/parallel-install/share/doc/parallel'
par_test_build_and_install /usr/bin/install -c -m 644 parallel.html env_parallel.html sem.html sql.html niceload.html parallel_tutorial.html parallel_book.html parallel_design.html parallel_alternatives.html parcat.html parset.html parallel.texi env_parallel.texi sem.texi sql.texi niceload.texi parallel_tutorial.texi parallel_book.texi parallel_design.texi parallel_alternatives.texi parcat.texi parset.texi parallel.pdf env_parallel.pdf sem.pdf sql.pdf niceload.pdf parallel_tutorial.pdf parallel_book.pdf parallel_design.pdf parallel_alternatives.pdf parcat.pdf parset.pdf parallel_cheat.pdf '/tmp/parallel-install/share/doc/parallel' par_test_build_and_install /usr/bin/install -c -m 644 parallel.html env_parallel.html sem.html sql.html niceload.html parallel_tutorial.html parallel_book.html parallel_design.html parallel_alternatives.html parcat.html parset.html parallel.texi env_parallel.texi sem.texi sql.texi niceload.texi parallel_tutorial.texi parallel_book.texi parallel_design.texi parallel_alternatives.texi parcat.texi parset.texi parallel.pdf env_parallel.pdf sem.pdf sql.pdf niceload.pdf parallel_tutorial.pdf parallel_book.pdf parallel_design.pdf parallel_alternatives.pdf parcat.pdf parset.pdf parallel_cheat_bw.pdf '/tmp/parallel-install/share/doc/parallel'
par_test_build_and_install /bin/mkdir -p '/tmp/parallel-install/share/man/man1' par_test_build_and_install /bin/mkdir -p '/tmp/parallel-install/share/man/man1'
par_test_build_and_install /usr/bin/install -c -m 644 parallel.1 env_parallel.1 sem.1 sql.1 niceload.1 parcat.1 parset.1 '/tmp/parallel-install/share/man/man1' par_test_build_and_install /usr/bin/install -c -m 644 parallel.1 env_parallel.1 sem.1 sql.1 niceload.1 parcat.1 parset.1 '/tmp/parallel-install/share/man/man1'
par_test_build_and_install /bin/mkdir -p '/tmp/parallel-install/share/man/man7' par_test_build_and_install /bin/mkdir -p '/tmp/parallel-install/share/man/man7'
@ -823,7 +823,7 @@ par_test_build_and_install || echo "Warning: pod2pdf not found. Using old parset
par_test_build_and_install /bin/bash: pod2pdf: command not found par_test_build_and_install /bin/bash: pod2pdf: command not found
par_test_build_and_install Warning: pod2pdf not found. Using old parset.pdf par_test_build_and_install Warning: pod2pdf not found. Using old parset.pdf
par_test_build_and_install /bin/mkdir -p '/tmp/parallel-install/share/doc/parallel' par_test_build_and_install /bin/mkdir -p '/tmp/parallel-install/share/doc/parallel'
par_test_build_and_install /usr/bin/install -c -m 644 parallel.html env_parallel.html sem.html sql.html niceload.html parallel_tutorial.html parallel_book.html parallel_design.html parallel_alternatives.html parcat.html parset.html parallel.texi env_parallel.texi sem.texi sql.texi niceload.texi parallel_tutorial.texi parallel_book.texi parallel_design.texi parallel_alternatives.texi parcat.texi parset.texi parallel.pdf env_parallel.pdf sem.pdf sql.pdf niceload.pdf parallel_tutorial.pdf parallel_book.pdf parallel_design.pdf parallel_alternatives.pdf parcat.pdf parset.pdf parallel_cheat.pdf '/tmp/parallel-install/share/doc/parallel' par_test_build_and_install /usr/bin/install -c -m 644 parallel.html env_parallel.html sem.html sql.html niceload.html parallel_tutorial.html parallel_book.html parallel_design.html parallel_alternatives.html parcat.html parset.html parallel.texi env_parallel.texi sem.texi sql.texi niceload.texi parallel_tutorial.texi parallel_book.texi parallel_design.texi parallel_alternatives.texi parcat.texi parset.texi parallel.pdf env_parallel.pdf sem.pdf sql.pdf niceload.pdf parallel_tutorial.pdf parallel_book.pdf parallel_design.pdf parallel_alternatives.pdf parcat.pdf parset.pdf parallel_cheat_bw.pdf '/tmp/parallel-install/share/doc/parallel'
par_test_build_and_install pod2man --release='00000000' --center='parallel' \ par_test_build_and_install pod2man --release='00000000' --center='parallel' \
par_test_build_and_install --section=1 ./parallel.pod > ./parallel.1n \ par_test_build_and_install --section=1 ./parallel.pod > ./parallel.1n \
par_test_build_and_install && mv ./parallel.1n ./parallel.1 \ par_test_build_and_install && mv ./parallel.1n ./parallel.1 \

1
testsuite/wanted-results/parallel-local-1s
View file

@ -564,6 +564,7 @@ par_test_cpu_detection 1-2-2-2 1 2 2 2
par_test_cpu_detection 2-24-48-24 2 24 48 24 par_test_cpu_detection 2-24-48-24 2 24 48 24
par_test_cpu_detection 1-2-2-2 1 2 2 2 par_test_cpu_detection 1-2-2-2 1 2 2 2
par_test_cpu_detection 1-8-8-8 1 8 8 8 par_test_cpu_detection 1-8-8-8 1 8 8 8
par_test_cpu_detection 1-4-4-4 1 4 4 4
par_too_long_line_X bug #54869: Long lines break par_too_long_line_X bug #54869: Long lines break
par_too_long_line_X 1 29302 131012 par_too_long_line_X 1 29302 131012
par_too_long_line_X 1 12698 63490 par_too_long_line_X 1 12698 63490

24
testsuite/wanted-results/parallel-local-3s
View file

@ -1363,6 +1363,7 @@ par_sqlworker_hostname <hostname>
par_sqlworker_hostname <hostname> par_sqlworker_hostname <hostname>
par_tee_with_premature_close --tee --pipe should send all data to all commands par_tee_with_premature_close --tee --pipe should send all data to all commands
par_tee_with_premature_close even if a command closes stdin before reading everything par_tee_with_premature_close even if a command closes stdin before reading everything
par_tee_with_premature_close tee with --output-error=warn-nopipe support
par_tee_with_premature_close 1000000 1000000 6888896 par_tee_with_premature_close 1000000 1000000 6888896
par_tee_with_premature_close 1 par_tee_with_premature_close 1
par_tee_with_premature_close 2 par_tee_with_premature_close 2
@ -1384,7 +1385,8 @@ par_tee_with_premature_close 999997
par_tee_with_premature_close 999998 par_tee_with_premature_close 999998
par_tee_with_premature_close 999999 par_tee_with_premature_close 999999
par_tee_with_premature_close 1000000 par_tee_with_premature_close 1000000
par_tee_with_premature_close 12773 12774 65536 par_tee_with_premature_close tee without --output-error=warn-nopipe support
par_tee_with_premature_close 1859 1860 8192
par_tee_with_premature_close 1 par_tee_with_premature_close 1
par_tee_with_premature_close 2 par_tee_with_premature_close 2
par_tee_with_premature_close 3 par_tee_with_premature_close 3
@ -1395,16 +1397,16 @@ par_tee_with_premature_close 7
par_tee_with_premature_close 8 par_tee_with_premature_close 8
par_tee_with_premature_close 9 par_tee_with_premature_close 9
par_tee_with_premature_close 10 par_tee_with_premature_close 10
par_tee_with_premature_close 12765 par_tee_with_premature_close 1851
par_tee_with_premature_close 12766 par_tee_with_premature_close 1852
par_tee_with_premature_close 12767 par_tee_with_premature_close 1853
par_tee_with_premature_close 12768 par_tee_with_premature_close 1854
par_tee_with_premature_close 12769 par_tee_with_premature_close 1855
par_tee_with_premature_close 12770 par_tee_with_premature_close 1856
par_tee_with_premature_close 12771 par_tee_with_premature_close 1857
par_tee_with_premature_close 12772 par_tee_with_premature_close 1858
par_tee_with_premature_close 12773 par_tee_with_premature_close 1859
par_tee_with_premature_close 1277 par_tee_with_premature_close 1860
par_test_X_with_multiple_source ### Test {} multiple times in different commands par_test_X_with_multiple_source ### Test {} multiple times in different commands
par_test_X_with_multiple_source echo 1 2 3 4 5 6 7 8 9 10 ; echo 1 2 3 4 5 6 7 8 9 10 par_test_X_with_multiple_source echo 1 2 3 4 5 6 7 8 9 10 ; echo 1 2 3 4 5 6 7 8 9 10
par_test_X_with_multiple_source 1 2 3 4 5 6 7 8 9 10 par_test_X_with_multiple_source 1 2 3 4 5 6 7 8 9 10

122
testsuite/wanted-results/parallel-tutorial
View file

@ -10,7 +10,6 @@
$ bash install.sh $ bash install.sh
/bin/bash: -c: line 0: syntax error near unexpected token `)' /bin/bash: -c: line 0: syntax error near unexpected token `)'
/bin/bash: -c: line 0: ` fetch -o - http://pi.dk/3 ) > install.sh' /bin/bash: -c: line 0: ` fetch -o - http://pi.dk/3 ) > install.sh'
parallel -k echo ::: A B C > abc-file parallel -k echo ::: A B C > abc-file
parallel -k echo ::: D E F > def-file parallel -k echo ::: D E F > def-file
perl -e 'printf "A\0B\0C\0"' > abc0-file perl -e 'printf "A\0B\0C\0"' > abc0-file
@ -28,7 +27,6 @@ sleep .3
/bin/bash: -c: line 0: syntax error near unexpected token `)' /bin/bash: -c: line 0: syntax error near unexpected token `)'
/bin/bash: -c: line 0: ` perl -e 'for(1..10){print "$_\n"}') > num_%header' /bin/bash: -c: line 0: ` perl -e 'for(1..10){print "$_\n"}') > num_%header'
perl -e 'print "HHHHAAABBBCCC"' > fixedlen perl -e 'print "HHHHAAABBBCCC"' > fixedlen
parallel echo ::: A B C parallel echo ::: A B C
A A
B B
@ -319,25 +317,20 @@ myval
cAaAdef cAaAdef
echo ${myvar:2:3} echo ${myvar:2:3}
parallel --plus echo {:2:3} ::: "$myvar" parallel --plus echo {:2:3} ::: "$myvar"
echo ${myvar#bc} echo ${myvar#bc}
parallel --plus echo {#bc} ::: "$myvar" parallel --plus echo {#bc} ::: "$myvar"
echo ${myvar#abc} echo ${myvar#abc}
parallel --plus echo {#abc} ::: "$myvar" parallel --plus echo {#abc} ::: "$myvar"
echo ${myvar%de} echo ${myvar%de}
parallel --plus echo {%de} ::: "$myvar" parallel --plus echo {%de} ::: "$myvar"
echo ${myvar%def} echo ${myvar%def}
parallel --plus echo {%def} ::: "$myvar" parallel --plus echo {%def} ::: "$myvar"
echo ${myvar/def/ghi} echo ${myvar/def/ghi}
parallel --plus echo {/def/ghi} ::: "$myvar" parallel --plus echo {/def/ghi} ::: "$myvar"
echo ${myvar^a} echo ${myvar^a}
parallel --plus echo {^a} ::: "$myvar" parallel --plus echo {^a} ::: "$myvar"
echo ${myvar^^a} echo ${myvar^^a}
parallel --plus echo {^^a} ::: "$myvar" parallel --plus echo {^^a} ::: "$myvar"
myvar=AbcAaAdef myvar=AbcAaAdef
echo ${myvar,A} echo ${myvar,A}
parallel --plus echo '{,A}' ::: "$myvar" parallel --plus echo '{,A}' ::: "$myvar"
@ -374,7 +367,6 @@ foo
perl -e 'print "@ARGV\n"' A perl -e 'print "@ARGV\n"' A
A A
parallel perl -e 'print "@ARGV\n"' ::: This wont work parallel perl -e 'print "@ARGV\n"' ::: This wont work
Warning: Input is read from the terminal. You either know what you Warning: Input is read from the terminal. You either know what you
Warning: are doing (in which case: YOU ARE AWESOME!) or you forgot Warning: are doing (in which case: YOU ARE AWESOME!) or you forgot
Warning: ::: or :::: or to pipe data into parallel. If so Warning: ::: or :::: or to pipe data into parallel. If so
@ -384,7 +376,6 @@ A
/bin/bash: line 1: Warning:: command not found /bin/bash: line 1: Warning:: command not found
/bin/bash: -c: line 2: syntax error near unexpected token `(' /bin/bash: -c: line 2: syntax error near unexpected token `('
/bin/bash: -c: line 2: ` Warning: are doing (in which case: YOU ARE AWESOME!) or you forgot' /bin/bash: -c: line 2: ` Warning: are doing (in which case: YOU ARE AWESOME!) or you forgot'
parallel --trim r echo pre-{}-post ::: ' A ' parallel --trim r echo pre-{}-post ::: ' A '
pre- A-post pre- A-post
parallel --trim l echo pre-{}-post ::: ' A ' parallel --trim l echo pre-{}-post ::: ' A '
@ -544,7 +535,6 @@ B D
parallel --use-cpus-instead-of-cores -N0 sleep 1 :::: num8 parallel --use-cpus-instead-of-cores -N0 sleep 1 :::: num8
parallel --shuf echo ::: 1 2 3 ::: a b c ::: A B C parallel --shuf echo ::: 1 2 3 ::: a b c ::: A B C
123 abc ABC 123 abc ABC
seq 10 20 | parallel --tmux 'echo start {}; sleep {}; echo done {}' seq 10 20 | parallel --tmux 'echo start {}; sleep {}; echo done {}'
See output with: tmux -S /tmp/parallel-tutorial-tmpdir/tmsXXXXX attach See output with: tmux -S /tmp/parallel-tutorial-tmpdir/tmsXXXXX attach
tmux -S /tmp/tmsXXXXX attach tmux -S /tmp/tmsXXXXX attach
@ -567,19 +557,13 @@ parallel: Warning: sleep 8; echo 8
parallel: Warning: This job was killed because it timed out: parallel: Warning: This job was killed because it timed out:
parallel: Warning: sleep 7; echo 7 parallel: Warning: sleep 7; echo 7
parallel --eta sleep ::: 1 3 2 2 1 3 3 2 1 parallel --eta sleep ::: 1 3 2 2 1 3 3 2 1
Computers / CPU cores / Max jobs to run Computers / CPU cores / Max jobs to run
1:local / 8 / 8 1:local / 8 / 8
Computer:jobs running/jobs completed/%of started jobs/Average seconds to complete Computer:jobs running/jobs completed/%of started jobs/Average seconds to complete
parallel --progress sleep ::: 1 3 2 2 1 3 3 2 1 parallel --progress sleep ::: 1 3 2 2 1 3 3 2 1
Computers / CPU cores / Max jobs to run Computers / CPU cores / Max jobs to run
1:local / 8 / 8 1:local / 8 / 8
Computer:jobs running/jobs completed/%of started jobs/Average seconds to complete Computer:jobs running/jobs completed/%of started jobs/Average seconds to complete
seq 1000 | parallel -j10 --bar '(echo -n {};sleep 0.1)' \ seq 1000 | parallel -j10 --bar '(echo -n {};sleep 0.1)' \
2> >(zenity --timeout=15 --progress --auto-kill --auto-close) 2> >(zenity --timeout=15 --progress --auto-kill --auto-close)
parallel --joblog /tmp/log exit ::: 1 2 3 0 parallel --joblog /tmp/log exit ::: 1 2 3 0
@ -662,7 +646,6 @@ echo X; exit X
completed 1 completed 1
completed 2 completed 2
completed 0 completed 0
tried 0 tried 0
show_signals() { show_signals() {
perl -e 'for(keys %SIG) { perl -e 'for(keys %SIG) {
@ -770,55 +753,41 @@ Unknown option: myecho $myvar
Unknown option: ::: Unknown option: :::
Unknown option: green Unknown option: green
env_parallel only works if it is a function. Do the below and restart your shell. env_parallel only works if it is a function. Do the below and restart your shell.
bash: Put this in $HOME/.bashrc: . `which env_parallel.bash` bash: Put this in $HOME/.bashrc: . `which env_parallel.bash`
E.g. by doing: echo '. `which env_parallel.bash`' >> $HOME/.bashrc E.g. by doing: echo '. `which env_parallel.bash`' >> $HOME/.bashrc
Supports: aliases, functions, variables, arrays Supports: aliases, functions, variables, arrays
zsh: Put this in $HOME/.zshrc: . `which env_parallel.zsh` zsh: Put this in $HOME/.zshrc: . `which env_parallel.zsh`
E.g. by doing: echo '. `which env_parallel.zsh`' >> $HOME/.zshenv E.g. by doing: echo '. `which env_parallel.zsh`' >> $HOME/.zshenv
Supports: functions, variables, arrays Supports: functions, variables, arrays
fish: Put this in $HOME/.config/fish/config.fish: fish: Put this in $HOME/.config/fish/config.fish:
. (which env_parallel.fish) . (which env_parallel.fish)
E.g. by doing: E.g. by doing:
echo '. (which env_parallel.fish)' >> $HOME/.config/fish/config.fish echo '. (which env_parallel.fish)' >> $HOME/.config/fish/config.fish
Supports: aliases, functions, variables, arrays Supports: aliases, functions, variables, arrays
ksh: Put this in $HOME/.kshrc: source `which env_parallel.ksh` ksh: Put this in $HOME/.kshrc: source `which env_parallel.ksh`
E.g. by doing: echo 'source `which env_parallel.ksh`' >> $HOME/.kshrc E.g. by doing: echo 'source `which env_parallel.ksh`' >> $HOME/.kshrc
Supports: aliases, functions, variables, arrays Supports: aliases, functions, variables, arrays
mksh: Put this in $HOME/.mkshrc: source `which env_parallel.mksh` mksh: Put this in $HOME/.mkshrc: source `which env_parallel.mksh`
E.g. by doing: echo 'source `which env_parallel.mksh`' >> $HOME/.mkshrc E.g. by doing: echo 'source `which env_parallel.mksh`' >> $HOME/.mkshrc
Supports: aliases, functions, variables, arrays Supports: aliases, functions, variables, arrays
pdksh: Put this in $HOME/.profile: source `which env_parallel.pdksh` pdksh: Put this in $HOME/.profile: source `which env_parallel.pdksh`
E.g. by doing: echo '. `which env_parallel.pdksh`' >> $HOME/.profile E.g. by doing: echo '. `which env_parallel.pdksh`' >> $HOME/.profile
Supports: aliases, functions, variables, arrays Supports: aliases, functions, variables, arrays
ash: Put this in $HOME/.profile: . `which env_parallel.ash` ash: Put this in $HOME/.profile: . `which env_parallel.ash`
E.g. by doing: echo '. `which env_parallel.ash`' >> $HOME/.profile E.g. by doing: echo '. `which env_parallel.ash`' >> $HOME/.profile
Supports: aliases, variables Supports: aliases, variables
dash: Put this in $HOME/.profile: . `which env_parallel.dash` dash: Put this in $HOME/.profile: . `which env_parallel.dash`
E.g. by doing: echo '. `which env_parallel.dash`' >> $HOME/.profile E.g. by doing: echo '. `which env_parallel.dash`' >> $HOME/.profile
Supports: aliases, variables Supports: aliases, variables
csh: Put this in $HOME/.cshrc: source `which env_parallel.csh` csh: Put this in $HOME/.cshrc: source `which env_parallel.csh`
E.g. by doing: echo 'source `which env_parallel.csh`' >> $HOME/.cshrc E.g. by doing: echo 'source `which env_parallel.csh`' >> $HOME/.cshrc
Supports: aliases, variables, arrays with no special chars Supports: aliases, variables, arrays with no special chars
tcsh: Put this in $HOME/.tcshrc: source `which env_parallel.tcsh` tcsh: Put this in $HOME/.tcshrc: source `which env_parallel.tcsh`
E.g. by doing: echo 'source `which env_parallel.tcsh`' >> $HOME/.tcshrc E.g. by doing: echo 'source `which env_parallel.tcsh`' >> $HOME/.tcshrc
Supports: aliases, variables, arrays with no special chars Supports: aliases, variables, arrays with no special chars
To install in all shells run: To install in all shells run:
env_parallel --install env_parallel --install
For details: see man env_parallel For details: see man env_parallel
MYVAR='foo bar' MYVAR='foo bar'
export MYVAR export MYVAR
parallel --env MYVAR -S $SERVER1 echo '$MYVAR' ::: baz parallel --env MYVAR -S $SERVER1 echo '$MYVAR' ::: baz
@ -928,7 +897,6 @@ mysqlrootpass
VAR=foo VAR=foo
export VAR export VAR
parallel --env _ -S $SERVER1 'echo $VAR; my_func2' ::: bar parallel --env _ -S $SERVER1 'echo $VAR; my_func2' ::: bar
/bin/bash: my_func2: command not found /bin/bash: my_func2: command not found
NOT='not exported var' NOT='not exported var'
alias myecho=echo alias myecho=echo
@ -944,55 +912,41 @@ Unknown option: echo $NOT; not_ex
Unknown option: ::: Unknown option: :::
Unknown option: bar Unknown option: bar
env_parallel only works if it is a function. Do the below and restart your shell. env_parallel only works if it is a function. Do the below and restart your shell.
bash: Put this in $HOME/.bashrc: . `which env_parallel.bash` bash: Put this in $HOME/.bashrc: . `which env_parallel.bash`
E.g. by doing: echo '. `which env_parallel.bash`' >> $HOME/.bashrc E.g. by doing: echo '. `which env_parallel.bash`' >> $HOME/.bashrc
Supports: aliases, functions, variables, arrays Supports: aliases, functions, variables, arrays
zsh: Put this in $HOME/.zshrc: . `which env_parallel.zsh` zsh: Put this in $HOME/.zshrc: . `which env_parallel.zsh`
E.g. by doing: echo '. `which env_parallel.zsh`' >> $HOME/.zshenv E.g. by doing: echo '. `which env_parallel.zsh`' >> $HOME/.zshenv
Supports: functions, variables, arrays Supports: functions, variables, arrays
fish: Put this in $HOME/.config/fish/config.fish: fish: Put this in $HOME/.config/fish/config.fish:
. (which env_parallel.fish) . (which env_parallel.fish)
E.g. by doing: E.g. by doing:
echo '. (which env_parallel.fish)' >> $HOME/.config/fish/config.fish echo '. (which env_parallel.fish)' >> $HOME/.config/fish/config.fish
Supports: aliases, functions, variables, arrays Supports: aliases, functions, variables, arrays
ksh: Put this in $HOME/.kshrc: source `which env_parallel.ksh` ksh: Put this in $HOME/.kshrc: source `which env_parallel.ksh`
E.g. by doing: echo 'source `which env_parallel.ksh`' >> $HOME/.kshrc E.g. by doing: echo 'source `which env_parallel.ksh`' >> $HOME/.kshrc
Supports: aliases, functions, variables, arrays Supports: aliases, functions, variables, arrays
mksh: Put this in $HOME/.mkshrc: source `which env_parallel.mksh` mksh: Put this in $HOME/.mkshrc: source `which env_parallel.mksh`
E.g. by doing: echo 'source `which env_parallel.mksh`' >> $HOME/.mkshrc E.g. by doing: echo 'source `which env_parallel.mksh`' >> $HOME/.mkshrc
Supports: aliases, functions, variables, arrays Supports: aliases, functions, variables, arrays
pdksh: Put this in $HOME/.profile: source `which env_parallel.pdksh` pdksh: Put this in $HOME/.profile: source `which env_parallel.pdksh`
E.g. by doing: echo '. `which env_parallel.pdksh`' >> $HOME/.profile E.g. by doing: echo '. `which env_parallel.pdksh`' >> $HOME/.profile
Supports: aliases, functions, variables, arrays Supports: aliases, functions, variables, arrays
ash: Put this in $HOME/.profile: . `which env_parallel.ash` ash: Put this in $HOME/.profile: . `which env_parallel.ash`
E.g. by doing: echo '. `which env_parallel.ash`' >> $HOME/.profile E.g. by doing: echo '. `which env_parallel.ash`' >> $HOME/.profile
Supports: aliases, variables Supports: aliases, variables
dash: Put this in $HOME/.profile: . `which env_parallel.dash` dash: Put this in $HOME/.profile: . `which env_parallel.dash`
E.g. by doing: echo '. `which env_parallel.dash`' >> $HOME/.profile E.g. by doing: echo '. `which env_parallel.dash`' >> $HOME/.profile
Supports: aliases, variables Supports: aliases, variables
csh: Put this in $HOME/.cshrc: source `which env_parallel.csh` csh: Put this in $HOME/.cshrc: source `which env_parallel.csh`
E.g. by doing: echo 'source `which env_parallel.csh`' >> $HOME/.cshrc E.g. by doing: echo 'source `which env_parallel.csh`' >> $HOME/.cshrc
Supports: aliases, variables, arrays with no special chars Supports: aliases, variables, arrays with no special chars
tcsh: Put this in $HOME/.tcshrc: source `which env_parallel.tcsh` tcsh: Put this in $HOME/.tcshrc: source `which env_parallel.tcsh`
E.g. by doing: echo 'source `which env_parallel.tcsh`' >> $HOME/.tcshrc E.g. by doing: echo 'source `which env_parallel.tcsh`' >> $HOME/.tcshrc
Supports: aliases, variables, arrays with no special chars Supports: aliases, variables, arrays with no special chars
To install in all shells run: To install in all shells run:
env_parallel --install env_parallel --install
For details: see man env_parallel For details: see man env_parallel
parallel -vv --pipepart --block 1M wc :::: num30000 parallel -vv --pipepart --block 1M wc :::: num30000
<num30000 perl -e 'while(@ARGV){sysseek(STDIN,shift,0)||die;$left=shift;while($read=sysread(STDIN,$buf,$left>131072?131072:$left)){$left-=$read;syswrite(STDOUT,$buf);}}' 0 0 0 168894 |(wc) <num30000 perl -e 'while(@ARGV){sysseek(STDIN,shift,0)||die;$left=shift;while($read=sysread(STDIN,$buf,$left>131072?131072:$left)){$left-=$read;syswrite(STDOUT,$buf);}}' 0 0 0 168894 |(wc)
30000 30000 168894 30000 30000 168894
@ -1013,49 +967,34 @@ Unknown option: a
Unknown option: b Unknown option: b
parset only works if it is a function. The function is defined as part parset only works if it is a function. The function is defined as part
of env_parallel. of env_parallel.
Do the below and restart your shell. Do the below and restart your shell.
bash: Put this in $HOME/.bashrc: . `which env_parallel.bash` bash: Put this in $HOME/.bashrc: . `which env_parallel.bash`
E.g. by doing: echo '. `which env_parallel.bash`' >> $HOME/.bashrc E.g. by doing: echo '. `which env_parallel.bash`' >> $HOME/.bashrc
Supports: aliases, functions, variables, arrays Supports: aliases, functions, variables, arrays
zsh: Put this in $HOME/.zshrc: . `which env_parallel.zsh` zsh: Put this in $HOME/.zshrc: . `which env_parallel.zsh`
E.g. by doing: echo '. `which env_parallel.zsh`' >> $HOME/.zshenv E.g. by doing: echo '. `which env_parallel.zsh`' >> $HOME/.zshenv
Supports: functions, variables, arrays Supports: functions, variables, arrays
fish: Unsupported fish: Unsupported
ksh: Put this in $HOME/.kshrc: source `which env_parallel.ksh` ksh: Put this in $HOME/.kshrc: source `which env_parallel.ksh`
E.g. by doing: echo 'source `which env_parallel.ksh`' >> $HOME/.kshrc E.g. by doing: echo 'source `which env_parallel.ksh`' >> $HOME/.kshrc
Supports: aliases, functions, variables, arrays Supports: aliases, functions, variables, arrays
mksh: Put this in $HOME/.mkshrc: source `which env_parallel.mksh` mksh: Put this in $HOME/.mkshrc: source `which env_parallel.mksh`
E.g. by doing: echo 'source `which env_parallel.mksh`' >> $HOME/.mkshrc E.g. by doing: echo 'source `which env_parallel.mksh`' >> $HOME/.mkshrc
Supports: aliases, functions, variables, arrays Supports: aliases, functions, variables, arrays
pdksh: Put this in $HOME/.profile: source `which env_parallel.pdksh` pdksh: Put this in $HOME/.profile: source `which env_parallel.pdksh`
E.g. by doing: echo '. `which env_parallel.pdksh`' >> $HOME/.profile E.g. by doing: echo '. `which env_parallel.pdksh`' >> $HOME/.profile
Supports: aliases, functions, variables, arrays Supports: aliases, functions, variables, arrays
ash: Put this in $HOME/.profile: . `which env_parallel.ash` ash: Put this in $HOME/.profile: . `which env_parallel.ash`
E.g. by doing: echo '. `which env_parallel.ash`' >> $HOME/.profile E.g. by doing: echo '. `which env_parallel.ash`' >> $HOME/.profile
Supports: aliases, variables Supports: aliases, variables
dash: Put this in $HOME/.profile: . `which env_parallel.dash` dash: Put this in $HOME/.profile: . `which env_parallel.dash`
E.g. by doing: echo '. `which env_parallel.dash`' >> $HOME/.profile E.g. by doing: echo '. `which env_parallel.dash`' >> $HOME/.profile
Supports: aliases, variables Supports: aliases, variables
csh: Unsupported csh: Unsupported
tcsh: Unsupported tcsh: Unsupported
To install in all shells run: To install in all shells run:
parset --install parset --install
For details: see man parset For details: see man parset
parset myarray seq {} 5 ::: 1 2 3 parset myarray seq {} 5 ::: 1 2 3
echo "${myarray[1]}" echo "${myarray[1]}"
Unknown option: myarray Unknown option: myarray
@ -1068,49 +1007,34 @@ Unknown option: 2
Unknown option: 3 Unknown option: 3
parset only works if it is a function. The function is defined as part parset only works if it is a function. The function is defined as part
of env_parallel. of env_parallel.
Do the below and restart your shell. Do the below and restart your shell.
bash: Put this in $HOME/.bashrc: . `which env_parallel.bash` bash: Put this in $HOME/.bashrc: . `which env_parallel.bash`
E.g. by doing: echo '. `which env_parallel.bash`' >> $HOME/.bashrc E.g. by doing: echo '. `which env_parallel.bash`' >> $HOME/.bashrc
Supports: aliases, functions, variables, arrays Supports: aliases, functions, variables, arrays
zsh: Put this in $HOME/.zshrc: . `which env_parallel.zsh` zsh: Put this in $HOME/.zshrc: . `which env_parallel.zsh`
E.g. by doing: echo '. `which env_parallel.zsh`' >> $HOME/.zshenv E.g. by doing: echo '. `which env_parallel.zsh`' >> $HOME/.zshenv
Supports: functions, variables, arrays Supports: functions, variables, arrays
fish: Unsupported fish: Unsupported
ksh: Put this in $HOME/.kshrc: source `which env_parallel.ksh` ksh: Put this in $HOME/.kshrc: source `which env_parallel.ksh`
E.g. by doing: echo 'source `which env_parallel.ksh`' >> $HOME/.kshrc E.g. by doing: echo 'source `which env_parallel.ksh`' >> $HOME/.kshrc
Supports: aliases, functions, variables, arrays Supports: aliases, functions, variables, arrays
mksh: Put this in $HOME/.mkshrc: source `which env_parallel.mksh` mksh: Put this in $HOME/.mkshrc: source `which env_parallel.mksh`
E.g. by doing: echo 'source `which env_parallel.mksh`' >> $HOME/.mkshrc E.g. by doing: echo 'source `which env_parallel.mksh`' >> $HOME/.mkshrc
Supports: aliases, functions, variables, arrays Supports: aliases, functions, variables, arrays
pdksh: Put this in $HOME/.profile: source `which env_parallel.pdksh` pdksh: Put this in $HOME/.profile: source `which env_parallel.pdksh`
E.g. by doing: echo '. `which env_parallel.pdksh`' >> $HOME/.profile E.g. by doing: echo '. `which env_parallel.pdksh`' >> $HOME/.profile
Supports: aliases, functions, variables, arrays Supports: aliases, functions, variables, arrays
ash: Put this in $HOME/.profile: . `which env_parallel.ash` ash: Put this in $HOME/.profile: . `which env_parallel.ash`
E.g. by doing: echo '. `which env_parallel.ash`' >> $HOME/.profile E.g. by doing: echo '. `which env_parallel.ash`' >> $HOME/.profile
Supports: aliases, variables Supports: aliases, variables
dash: Put this in $HOME/.profile: . `which env_parallel.dash` dash: Put this in $HOME/.profile: . `which env_parallel.dash`
E.g. by doing: echo '. `which env_parallel.dash`' >> $HOME/.profile E.g. by doing: echo '. `which env_parallel.dash`' >> $HOME/.profile
Supports: aliases, variables Supports: aliases, variables
csh: Unsupported csh: Unsupported
tcsh: Unsupported tcsh: Unsupported
To install in all shells run: To install in all shells run:
parset --install parset --install
For details: see man parset For details: see man parset
cmd=("echo '<<joe \"double space\" cartoon>>'" "pwd") cmd=("echo '<<joe \"double space\" cartoon>>'" "pwd")
parset data ::: "${cmd[@]}" parset data ::: "${cmd[@]}"
echo "${data[0]}" echo "${data[0]}"
@ -1121,49 +1045,34 @@ Unknown option: echo '<<joe "double space" cartoon>>'
Unknown option: pwd Unknown option: pwd
parset only works if it is a function. The function is defined as part parset only works if it is a function. The function is defined as part
of env_parallel. of env_parallel.
Do the below and restart your shell. Do the below and restart your shell.
bash: Put this in $HOME/.bashrc: . `which env_parallel.bash` bash: Put this in $HOME/.bashrc: . `which env_parallel.bash`
E.g. by doing: echo '. `which env_parallel.bash`' >> $HOME/.bashrc E.g. by doing: echo '. `which env_parallel.bash`' >> $HOME/.bashrc
Supports: aliases, functions, variables, arrays Supports: aliases, functions, variables, arrays
zsh: Put this in $HOME/.zshrc: . `which env_parallel.zsh` zsh: Put this in $HOME/.zshrc: . `which env_parallel.zsh`
E.g. by doing: echo '. `which env_parallel.zsh`' >> $HOME/.zshenv E.g. by doing: echo '. `which env_parallel.zsh`' >> $HOME/.zshenv
Supports: functions, variables, arrays Supports: functions, variables, arrays
fish: Unsupported fish: Unsupported
ksh: Put this in $HOME/.kshrc: source `which env_parallel.ksh` ksh: Put this in $HOME/.kshrc: source `which env_parallel.ksh`
E.g. by doing: echo 'source `which env_parallel.ksh`' >> $HOME/.kshrc E.g. by doing: echo 'source `which env_parallel.ksh`' >> $HOME/.kshrc
Supports: aliases, functions, variables, arrays Supports: aliases, functions, variables, arrays
mksh: Put this in $HOME/.mkshrc: source `which env_parallel.mksh` mksh: Put this in $HOME/.mkshrc: source `which env_parallel.mksh`
E.g. by doing: echo 'source `which env_parallel.mksh`' >> $HOME/.mkshrc E.g. by doing: echo 'source `which env_parallel.mksh`' >> $HOME/.mkshrc
Supports: aliases, functions, variables, arrays Supports: aliases, functions, variables, arrays
pdksh: Put this in $HOME/.profile: source `which env_parallel.pdksh` pdksh: Put this in $HOME/.profile: source `which env_parallel.pdksh`
E.g. by doing: echo '. `which env_parallel.pdksh`' >> $HOME/.profile E.g. by doing: echo '. `which env_parallel.pdksh`' >> $HOME/.profile
Supports: aliases, functions, variables, arrays Supports: aliases, functions, variables, arrays
ash: Put this in $HOME/.profile: . `which env_parallel.ash` ash: Put this in $HOME/.profile: . `which env_parallel.ash`
E.g. by doing: echo '. `which env_parallel.ash`' >> $HOME/.profile E.g. by doing: echo '. `which env_parallel.ash`' >> $HOME/.profile
Supports: aliases, variables Supports: aliases, variables
dash: Put this in $HOME/.profile: . `which env_parallel.dash` dash: Put this in $HOME/.profile: . `which env_parallel.dash`
E.g. by doing: echo '. `which env_parallel.dash`' >> $HOME/.profile E.g. by doing: echo '. `which env_parallel.dash`' >> $HOME/.profile
Supports: aliases, variables Supports: aliases, variables
csh: Unsupported csh: Unsupported
tcsh: Unsupported tcsh: Unsupported
To install in all shells run: To install in all shells run:
parset --install parset --install
For details: see man parset For details: see man parset
parallel --sqlandworker csv:////%2Ftmp%2Flog.csv \ parallel --sqlandworker csv:////%2Ftmp%2Flog.csv \
seq ::: 10 ::: 12 13 14 seq ::: 10 ::: 12 13 14
cat /tmp/log.csv cat /tmp/log.csv
@ -1202,7 +1111,6 @@ bar quuz
sql $DBURL 'SELECT * FROM mytable ORDER BY Seq;' sql $DBURL 'SELECT * FROM mytable ORDER BY Seq;'
Error: Error:
SELECT * FROM mytable ORDER BY Seq; is not a valid DBURL SELECT * FROM mytable ORDER BY Seq; is not a valid DBURL
sql [-hnr] [--table-size] [--db-size] [-p pass-through] [-s string] dburl [command] sql [-hnr] [--table-size] [--db-size] [-p pass-through] [-s string] dburl [command]
parallel --sqlmaster $DBURLTABLE echo ::: foo bar ::: baz quuz parallel --sqlmaster $DBURLTABLE echo ::: foo bar ::: baz quuz
parallel: Error: echo is not a valid DBURL parallel: Error: echo is not a valid DBURL
@ -1319,45 +1227,36 @@ Error: no such file "@ARGV\n"
perl_echo foo bar perl_echo foo bar
/bin/bash: perl_echo: command not found /bin/bash: perl_echo: command not found
#!/usr/bin/parallel --shebang-wrap /usr/bin/perl #!/usr/bin/parallel --shebang-wrap /usr/bin/perl
print "Arguments @ARGV\n"; print "Arguments @ARGV\n";
Error: no such file "Arguments @ARGV\n" Error: no such file "Arguments @ARGV\n"
#!/usr/bin/parallel --shebang-wrap /usr/bin/python #!/usr/bin/parallel --shebang-wrap /usr/bin/python
print 'Arguments', str(sys.argv) print 'Arguments', str(sys.argv)
/bin/bash: -c: line 3: syntax error near unexpected token `(' /bin/bash: -c: line 3: syntax error near unexpected token `('
/bin/bash: -c: line 3: ` print 'Arguments', str(sys.argv)' /bin/bash: -c: line 3: ` print 'Arguments', str(sys.argv)'
#!/usr/bin/parallel --shebang-wrap /bin/bash #!/usr/bin/parallel --shebang-wrap /bin/bash
echo Arguments "$@" echo Arguments "$@"
Arguments Arguments
#!/usr/bin/parallel --shebang-wrap /bin/csh #!/usr/bin/parallel --shebang-wrap /bin/csh
echo Arguments "$argv" echo Arguments "$argv"
Arguments Arguments
#!/usr/bin/parallel --shebang-wrap /usr/bin/tclsh #!/usr/bin/parallel --shebang-wrap /usr/bin/tclsh
puts "Arguments $argv" puts "Arguments $argv"
/bin/bash: line 2: puts: command not found /bin/bash: line 2: puts: command not found
#!/usr/bin/parallel --shebang-wrap /usr/bin/Rscript --vanilla --slave #!/usr/bin/parallel --shebang-wrap /usr/bin/Rscript --vanilla --slave
args <- commandArgs(trailingOnly = TRUE) args <- commandArgs(trailingOnly = TRUE)
print(paste("Arguments ",args)) print(paste("Arguments ",args))
/bin/bash: -c: line 2: syntax error near unexpected token `(' /bin/bash: -c: line 2: syntax error near unexpected token `('
/bin/bash: -c: line 2: ` args <- commandArgs(trailingOnly = TRUE)' /bin/bash: -c: line 2: ` args <- commandArgs(trailingOnly = TRUE)'
#!/usr/bin/parallel --shebang-wrap ARG={} /usr/bin/gnuplot #!/usr/bin/parallel --shebang-wrap ARG={} /usr/bin/gnuplot
print "Arguments ", system('echo $ARG') print "Arguments ", system('echo $ARG')
/bin/bash: -c: line 2: syntax error near unexpected token `(' /bin/bash: -c: line 2: syntax error near unexpected token `('
/bin/bash: -c: line 2: ` print "Arguments ", system('echo $ARG')' /bin/bash: -c: line 2: ` print "Arguments ", system('echo $ARG')'
#!/usr/bin/parallel --shebang-wrap /usr/bin/ruby #!/usr/bin/parallel --shebang-wrap /usr/bin/ruby
print "Arguments " print "Arguments "
puts ARGV puts ARGV
Error: no such file "Arguments " Error: no such file "Arguments "
/bin/bash: line 3: puts: command not found /bin/bash: line 3: puts: command not found
#!/usr/bin/parallel --shebang-wrap /usr/bin/octave #!/usr/bin/parallel --shebang-wrap /usr/bin/octave
printf ("Arguments"); printf ("Arguments");
arg_list = argv (); arg_list = argv ();
for i = 1:nargin for i = 1:nargin
@ -1367,7 +1266,6 @@ Error: no such file "Arguments "
/bin/bash: -c: line 2: syntax error near unexpected token `"Arguments"' /bin/bash: -c: line 2: syntax error near unexpected token `"Arguments"'
/bin/bash: -c: line 2: ` printf ("Arguments");' /bin/bash: -c: line 2: ` printf ("Arguments");'
#!/usr/bin/parallel --shebang-wrap /usr/bin/clisp #!/usr/bin/parallel --shebang-wrap /usr/bin/clisp
(format t "~&~S~&" 'Arguments) (format t "~&~S~&" 'Arguments)
(format t "~&~S~&" *args*) (format t "~&~S~&" *args*)
/bin/bash: -c: line 2: unexpected EOF while looking for matching `'' /bin/bash: -c: line 2: unexpected EOF while looking for matching `''
@ -1391,7 +1289,6 @@ Arguments
/bin/bash: -c: line 0: syntax error near unexpected token `(' /bin/bash: -c: line 0: syntax error near unexpected token `('
/bin/bash: -c: line 0: ` var myArgs = process.argv.slice(2);' /bin/bash: -c: line 0: ` var myArgs = process.argv.slice(2);'
#!/usr/bin/parallel --shebang-wrap /usr/bin/lua #!/usr/bin/parallel --shebang-wrap /usr/bin/lua
io.write "Arguments" io.write "Arguments"
for a = 1, #arg do for a = 1, #arg do
io.write(" ") io.write(" ")
@ -1402,7 +1299,6 @@ Arguments
/bin/bash: -c: line 3: syntax error near unexpected token `=' /bin/bash: -c: line 3: syntax error near unexpected token `='
/bin/bash: -c: line 3: ` for a = 1, #arg do' /bin/bash: -c: line 3: ` for a = 1, #arg do'
#!/usr/bin/parallel --shebang-wrap ARGV={} /usr/bin/csharp #!/usr/bin/parallel --shebang-wrap ARGV={} /usr/bin/csharp
var argv = Environment.GetEnvironmentVariable("ARGV"); var argv = Environment.GetEnvironmentVariable("ARGV");
print("Arguments "+argv); print("Arguments "+argv);
/bin/bash: -c: line 2: syntax error near unexpected token `(' /bin/bash: -c: line 2: syntax error near unexpected token `('
@ -1427,7 +1323,6 @@ The second finished running in the foreground
First started First started
First done First done
sem --id my_id -u 'echo Second started; sleep 10; echo Second done' sem --id my_id -u 'echo Second started; sleep 10; echo Second done'
Second done
sem --jobs 3 --id my_id -u 'echo Start 1; sleep 5; echo 1 done' && sem --jobs 3 --id my_id -u 'echo Start 1; sleep 5; echo 1 done' &&
sem --jobs 3 --id my_id -u 'echo Start 2; sleep 6; echo 2 done' && sem --jobs 3 --id my_id -u 'echo Start 2; sleep 6; echo 2 done' &&
sem --jobs 3 --id my_id -u 'echo Start 3; sleep 7; echo 3 done' && sem --jobs 3 --id my_id -u 'echo Start 3; sleep 7; echo 3 done' &&
@ -1452,11 +1347,9 @@ parallel: Warning: Semaphore timed out. Stealing the semaphore.
parallel: Warning: Semaphore timed out. Exiting. parallel: Warning: Semaphore timed out. Exiting.
parallel --help parallel --help
Usage: Usage:
parallel [options] [command [arguments]] < list_of_arguments parallel [options] [command [arguments]] < list_of_arguments
parallel [options] [command [arguments]] (::: arguments|:::: argfile(s))... parallel [options] [command [arguments]] (::: arguments|:::: argfile(s))...
cat ... | parallel --pipe [options] [command [arguments]] cat ... | parallel --pipe [options] [command [arguments]]
-j n Run n jobs in parallel -j n Run n jobs in parallel
-k Keep same order -k Keep same order
-X Multiple arguments with context replace -X Multiple arguments with context replace
@ -1465,38 +1358,29 @@ cat ... | parallel --pipe [options] [command [arguments]]
{3} {3.} {3/} {3/.} {=3 perl code =} Positional replacement strings {3} {3.} {3/} {3/.} {=3 perl code =} Positional replacement strings
With --plus: {} = {+/}/{/} = {.}.{+.} = {+/}/{/.}.{+.} = {..}.{+..} = With --plus: {} = {+/}/{/} = {.}.{+.} = {+/}/{/.}.{+.} = {..}.{+..} =
{+/}/{/..}.{+..} = {...}.{+...} = {+/}/{/...}.{+...} {+/}/{/..}.{+..} = {...}.{+...} = {+/}/{/...}.{+...}
-S sshlogin Example: foo@server.example.com -S sshlogin Example: foo@server.example.com
--slf .. Use ~/.parallel/sshloginfile as the list of sshlogins --slf .. Use ~/.parallel/sshloginfile as the list of sshlogins
--trc {}.bar Shorthand for --transfer --return {}.bar --cleanup --trc {}.bar Shorthand for --transfer --return {}.bar --cleanup
--onall Run the given command with argument on all sshlogins --onall Run the given command with argument on all sshlogins
--nonall Run the given command with no arguments on all sshlogins --nonall Run the given command with no arguments on all sshlogins
--pipe Split stdin (standard input) to multiple jobs. --pipe Split stdin (standard input) to multiple jobs.
--recend str Record end separator for --pipe. --recend str Record end separator for --pipe.
--recstart str Record start separator for --pipe. --recstart str Record start separator for --pipe.
See 'man parallel' for details See 'man parallel' for details
Academic tradition requires you to cite works you base your article on. Academic tradition requires you to cite works you base your article on.
If you use programs that use GNU Parallel to process data for an article in a If you use programs that use GNU Parallel to process data for an article in a
scientific publication, please cite: scientific publication, please cite:
O. Tange (2018): GNU Parallel 2018, Mar 2018, ISBN 9781387509881, O. Tange (2018): GNU Parallel 2018, Mar 2018, ISBN 9781387509881,
DOI https://doi.org/10.5281/zenodo.1146014 DOI https://doi.org/10.5281/zenodo.1146014
This helps funding further development; AND IT WON'T COST YOU A CENT. This helps funding further development; AND IT WON'T COST YOU A CENT.
If you pay 10000 EUR you should feel free to use GNU Parallel without citing. If you pay 10000 EUR you should feel free to use GNU Parallel without citing.
parallel --version parallel --version
GNU parallel VERSION GNU parallel VERSION
Copyright (C) 2007-2019 Ole Tange and Free Software Foundation, Inc. Copyright (C) 2007-2019 Ole Tange and Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it. This is free software: you are free to change and redistribute it.
GNU parallel comes with no warranty. GNU parallel comes with no warranty.
Web site: http://www.gnu.org/software/parallel Web site: http://www.gnu.org/software/parallel
When using programs that use GNU Parallel to process data for publication When using programs that use GNU Parallel to process data for publication
please cite as described in 'parallel --citation'. please cite as described in 'parallel --citation'.
parallel --minversion VERSION && \ parallel --minversion VERSION && \
@ -1507,7 +1391,6 @@ Your version is at least VERSION.
Academic tradition requires you to cite works you base your article on. Academic tradition requires you to cite works you base your article on.
If you use programs that use GNU Parallel to process data for an article in a If you use programs that use GNU Parallel to process data for an article in a
scientific publication, please cite: scientific publication, please cite:
@book{tange_ole_2018_1146014, @book{tange_ole_2018_1146014,
author = {Tange, Ole}, author = {Tange, Ole},
title = {GNU Parallel 2018}, title = {GNU Parallel 2018},
@ -1518,20 +1401,15 @@ scientific publication, please cite:
doi = {10.5281/zenodo.1146014}, doi = {10.5281/zenodo.1146014},
url = {https://doi.org/10.5281/zenodo.1146014} url = {https://doi.org/10.5281/zenodo.1146014}
} }
(Feel free to use \nocite{tange_ole_2018_1146014}) (Feel free to use \nocite{tange_ole_2018_1146014})
This helps funding further development; AND IT WON'T COST YOU A CENT. This helps funding further development; AND IT WON'T COST YOU A CENT.
If you pay 10000 EUR you should feel free to use GNU Parallel without citing. If you pay 10000 EUR you should feel free to use GNU Parallel without citing.
More about funding GNU Parallel and the citation notice: More about funding GNU Parallel and the citation notice:
https://lists.gnu.org/archive/html/parallel/2013-11/msg00006.html https://lists.gnu.org/archive/html/parallel/2013-11/msg00006.html
https://www.gnu.org/software/parallel/parallel_design.html#Citation-notice https://www.gnu.org/software/parallel/parallel_design.html#Citation-notice
https://git.savannah.gnu.org/cgit/parallel.git/tree/doc/citation-notice-faq.txt https://git.savannah.gnu.org/cgit/parallel.git/tree/doc/citation-notice-faq.txt
If you send a copy of your published article to tange@gnu.org, it will be If you send a copy of your published article to tange@gnu.org, it will be
mentioned in the release notes of next version of GNU Parallel. mentioned in the release notes of next version of GNU Parallel.
parallel --number-of-cpus parallel --number-of-cpus
parallel --number-of-cores parallel --number-of-cores
9 9