tange/parallel
1
0
Fork 0
mirror of https://git.savannah.gnu.org/git/parallel.git synced 2024-11-22 05:57:54 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

testsuite: Cleanup after each test.

Browse source
This commit is contained in:
Ole Tange 2015-02-01 14:38:26 +01:00
parent 67c85d7658
commit 927cf13b0c
32 changed files with 173 additions and 163 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
NEWS
View file

@ -13,6 +13,8 @@
* --halt -1 and -2 now means halt if a job succeeds (so the opposite * --halt -1 and -2 now means halt if a job succeeds (so the opposite
of 1 and 2). of 1 and 2).
* --no-keep-order will reverse --keep-order.
* Bash's second fix of shellshock caused --env to break again when * Bash's second fix of shellshock caused --env to break again when
exporting functions. This has been fixed again. exporting functions. This has been fixed again.
@ -23,7 +25,7 @@
* A description of the design decisions for GNU Parallel can be found * A description of the design decisions for GNU Parallel can be found
in 'man parallel_design'. in 'man parallel_design'.
* A bug fix in replacement strings cause rewrite of the replacement * A bug fix in replacement strings caused rewrite of the replacement
function. This makes use of replacement strings alpha quality. function. This makes use of replacement strings alpha quality.
* GNU Parallel was cited in: SlideToolkit: An Assistive Toolset for * GNU Parallel was cited in: SlideToolkit: An Assistive Toolset for

87
doc/release_new_version
View file

@ -4,7 +4,7 @@
Check that documentation is updated (compare to web): Check that documentation is updated (compare to web):
Fixet for 20141122 Fixet for 20150122
git diff last-release-commit git diff last-release-commit
Unmodified beta since last version => production Unmodified beta since last version => production
Unmodified alpha since last version => beta Unmodified alpha since last version => beta
@ -172,6 +172,8 @@ http://freshmeat.net/projects/parallel/releases/new
== Update Diaspora Twitter == == Update Diaspora Twitter ==
https://joindiaspora.com/stream
New release of #GNU Parallel pi․dk/0 New in this release pi․dk/2 See the intro videos pi․dk/1 New release of #GNU Parallel pi․dk/0 New in this release pi․dk/2 See the intro videos pi․dk/1
10 secs installation: wget -O - pi․dk/3|bash 10 secs installation: wget -O - pi․dk/3|bash
@ -206,92 +208,17 @@ cc:Tim Cuthbertson <tim3d.junk@gmail.com>,
Ryoichiro Suzuki <ryoichiro.suzuki@gmail.com>, Ryoichiro Suzuki <ryoichiro.suzuki@gmail.com>,
Jesse Alama <jesse.alama@gmail.com> Jesse Alama <jesse.alama@gmail.com>
Subject: GNU Parallel 20150122 (' (((:~{> Je Suis Charlie') released Subject: GNU Parallel 20150222 ('') released
GNU Parallel 20150122 (' (((:~{> Je Suis Charlie') has been released. It is available for download at: http://ftp.gnu.org/gnu/parallel/ GNU Parallel 20150122 ('') has been released. It is available for download at: http://ftp.gnu.org/gnu/parallel/
Haiku of the month: Haiku of the month:
If not enough time: <<>>
Run jobs using Parallel.
Keeps CPUs warm.
-- Ole Tange
New in this release: New in this release:
* Remote jobs now send stderr (standard error) to stderr (standard error) instead of stdout (standard output). * Zip Folders with GNU Parallel http://fazky.github.io/Linux/2015-01-07-GNU-Parallel.html
* Remote execution command is now packed using base64 encoded bzip2. This means that big environments (app. 100 KB) can be transferred. This change makes remote execution alpha quality.
* --semaphoretimeout now takes a negative value. This means give up if the semaphore is not gotten within the timeout.
* --halt -1 and -2 now means halt if a job succeeds (so the opposite of 1 and 2).
* Bash's second fix of shellshock caused --env to break again when exporting functions. This has been fixed again.
* A semibig refactoring of big functions. All non-trivial functions are now less than 100 lines. The refactoring makes this release beta quality.
* A description of the design decisions for GNU Parallel can be found in 'man parallel_design'.
* A bug fix in replacement strings cause rewrite of the replacement function. This makes use of replacement strings alpha quality.
* GNU Parallel was cited in: SlideToolkit: An Assistive Toolset for the Histological Quantification of Whole Slide Images http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0110289
* GNU Parallel was cited in: Improving Energy and Performance with Spintronics Caches in Multicore Systems http://link.springer.com/chapter/10.1007/978-3-319-14313-2_24
* GNU Parallel was cited in: Parallel post-processing with MPI-Bash http://dl.acm.org/citation.cfm?id=2691137
* GNU Parallel was cited in: Distinguishing cause from effect using observational data: methods and benchmarks http://arxiv-web3.library.cornell.edu/pdf/1412.3773.pdf
* GNU Parallel was cited in: Data Science at the Command Line http://cdn.oreillystatic.com/oreilly/booksamplers/9781491947852_sampler.pdf
* GNU Parallel was cited in: Resolution of Genetic Map Expansion Caused by Excess Heterozygosity in Plant Recombinant Inbred Populations http://www.g3journal.org/content/4/10/1963.full.pdf
* GNU Parallel was cited in: Bayesian Inference of Protein Structure from Chemical Shift Data https://peerj.com/preprints/692v1.pdf
* GNU Parallel was cited in: Clonal interference and Mullerʼs ratchet in spatial habitats http://iopscience.iop.org/1478-3975/11/5/056003
* GNU Parallel was cited in: Web-scale content reuse detection ftp://info.isi.edu/isi-pubs/tr-692.pdf
* GNU Parallel was cited in: Computação Verde na Camada de Aplicação https://linux.ime.usp.br/~tpaiva/mac0499/files/monografia.pdf
* GNU Parallel was cited in: Frameshift alignment: statistics and post-genomic applications http://bioinformatics.oxfordjournals.org/content/30/24/3575.full.pdf+html
* GNU Parallel was cited in: Towards Recommender Engineering Tools and Experiments for Identifying Recommender Differences http://elehack.net/research/thesis/mde-thesis.pdf
* GNU Parallel was cited in: Speaker Environment Classification Using Rhythm Metrics in Levantine Arabic Dialect http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6923918
* GNU Parallel was cited in: A Formal Proof Of The Kepler Conjecture http://arxiv.org/pdf/1501.02155.pdf
* GNU Parallel was cited in: Allocation Policy Analysis for Cache Coherence Protocols for STT-MRAM-based caches http://conservancy.umn.edu/bitstream/handle/11299/169362/Nandkar_umn_0130M_15551.pdf
* GNU Parallel was cited in: Using Trusted Platform Module for securing virtual environment access in Cloud http://dar.aucegypt.edu:8080/bitstream/handle/10526/4237/Using%20Trusted%20Platform%20Module%20for%20Securing%20Virtual%20Environment%20Access%20in%20Cloud.pdf
* GNU Parallel was cited in: Weakly Supervised Learning for Compositional Sentiment Recognition http://computerlinguist.org/static/downloads/ma-thesis/Master_Thesis-Michael_Haas-Weakly_Supervised_Learning_for_Compositional_Sentiment_Recognition.pdf
* GNU Parallel was cited in: Agility and Performance in Elastic Distributed Storage http://dl.acm.org/citation.cfm?id=2668129
* GNU Parallel was used (unfortunately without citation) in: Large Scale Author Name Disambiguation in Digital Libraries http://semanticommunity.info/@api/deki/files/31523/P227_3546.pdf
* GNU Parallel is used in BEDOPS: https://media.readthedocs.org/pdf/bedops/v2p5p0/bedops.pdf
* Job ad asking for GNU Parallel skills: http://www.aplitrak.com/?adid=dXMzMTYxMjQuMzg0MTkuMTU1MEBtYW5wb3dlcmZveG5hLmFwbGl0cmFrLmNvbQ#sthash.kYCIhhty.KcoT6HnJ.dpbs
* GNU Parallel: Open Source For You (OSFY) magazine, October 2013 edition http://www.shakthimaan.com/posts/2014/11/27/gnu-parallel/news.html
* Multithreaded Encryption and Compression http://www.krazyworks.com/multithreaded-encryption-and-compression/
* GNU Parallel on OSX and Linux https://darknightelf.wordpress.com/2015/01/01/gnu-parallel-on-osx/
* How to build gnu parallel from source on Ubuntu PC? http://blog.ovidiuparvu.com/build-gnu-parallel-source-ubuntu-pc/
* GNU parallel for openaddr-process-one https://nelsonslog.wordpress.com/2015/01/09/gnu-parallel-for-openaddr-process-one/
* From SRA Project to FastQ http://danielecook.com/sra-project-fastq/
* Software Development Tooling http://www.computer.org/csdl/mags/so/2014/06/mso2014060021.pdf
* コマンドを並列に実行するGNU parallelがとても便利 http://bicycle1885.hatenablog.com/entry/2014/08/10/143612
* Bug fixes and man page updates. * Bug fixes and man page updates.

4
src/niceload.pod
View file

@ -32,9 +32,9 @@ run 1 second, suspend (3.00-1.00) seconds, run 1 second, suspend
=over 9 =over 9
=item B<-B> (alpha testing) =item B<-B> (beta testing)
=item B<--battery> (alpha testing) =item B<--battery> (beta testing)
Suspend if the system is running on battery. Short hand for: -l -1 --sensor 'cat /sys/class/power_supply/BAT0/status /proc/acpi/battery/BAT0/state 2>/dev/null |grep -i -q discharging; echo $?' Suspend if the system is running on battery. Short hand for: -l -1 --sensor 'cat /sys/class/power_supply/BAT0/status /proc/acpi/battery/BAT0/state 2>/dev/null |grep -i -q discharging; echo $?'

70
src/parallel.pod
View file

@ -58,6 +58,10 @@ parallel_tutorial>). Your command line will love you for it.
Finally you may want to look at the rest of this manual if you have Finally you may want to look at the rest of this manual if you have
special needs not already covered. special needs not already covered.
If you want to know the design decisions behind GNU B<parallel>, try:
B<man parallel_design>. This is also a good intro if you intend to
change GNU B<parallel>.
=head1 OPTIONS =head1 OPTIONS
@ -100,7 +104,7 @@ B<--env> and use B<env_parallel> instead of B<parallel>.
The command cannot contain the character \257 (macron: ¯). The command cannot contain the character \257 (macron: ¯).
=item B<{}> (alpha testing) =item B<{}> (beta testing)
Input line. This replacement string will be replaced by a full line Input line. This replacement string will be replaced by a full line
read from the input source. The input source is normally stdin read from the input source. The input source is normally stdin
@ -113,7 +117,7 @@ If the command line contains no replacement strings then B<{}> will be
appended to the command line. appended to the command line.
=item B<{.}> (alpha testing) =item B<{.}> (beta testing)
Input line without extension. This replacement string will be replaced Input line without extension. This replacement string will be replaced
by the input with the extension removed. If the input line contains by the input with the extension removed. If the input line contains
@ -129,7 +133,7 @@ The replacement string B<{.}> can be changed with B<--er>.
To understand replacement strings see B<{}>. To understand replacement strings see B<{}>.
=item B<{/}> (alpha testing) =item B<{/}> (beta testing)
Basename of input line. This replacement string will be replaced by Basename of input line. This replacement string will be replaced by
the input with the directory part removed. the input with the directory part removed.
@ -140,7 +144,7 @@ B<--basenamereplace>.
To understand replacement strings see B<{}>. To understand replacement strings see B<{}>.
=item B<{//}> (alpha testing) =item B<{//}> (beta testing)
Dirname of input line. This replacement string will be replaced by the Dirname of input line. This replacement string will be replaced by the
dir of the input line. See B<dirname>(1). dir of the input line. See B<dirname>(1).
@ -151,7 +155,7 @@ B<--dirnamereplace>.
To understand replacement strings see B<{}>. To understand replacement strings see B<{}>.
=item B<{/.}> (alpha testing) =item B<{/.}> (beta testing)
Basename of input line without extension. This replacement string will Basename of input line without extension. This replacement string will
be replaced by the input with the directory and extension part be replaced by the input with the directory and extension part
@ -163,7 +167,7 @@ B<--basenameextensionreplace>.
To understand replacement strings see B<{}>. To understand replacement strings see B<{}>.
=item B<{#}> (alpha testing) =item B<{#}> (beta testing)
Sequence number of the job to run. This replacement string will be Sequence number of the job to run. This replacement string will be
replaced by the sequence number of the job being run. It contains the replaced by the sequence number of the job being run. It contains the
@ -174,7 +178,7 @@ The replacement string B<{#}> can be changed with B<--seqreplace>.
To understand replacement strings see B<{}>. To understand replacement strings see B<{}>.
=item B<{%}> (alpha testing) =item B<{%}> (beta testing)
Job slot number. This replacement string will be replaced by the job's Job slot number. This replacement string will be replaced by the job's
slot number between 1 and number of jobs to run in parallel. There slot number between 1 and number of jobs to run in parallel. There
@ -186,7 +190,7 @@ The replacement string B<{%}> can be changed with B<--slotreplace>.
To understand replacement strings see B<{}>. To understand replacement strings see B<{}>.
=item B<{>I<n>B<}> (alpha testing) =item B<{>I<n>B<}> (beta testing)
Argument from input source I<n> or the I<n>'th argument. This Argument from input source I<n> or the I<n>'th argument. This
positional replacement string will be replaced by the input from input positional replacement string will be replaced by the input from input
@ -197,7 +201,7 @@ I<n>'th last argument.
To understand replacement strings see B<{}>. To understand replacement strings see B<{}>.
=item B<{>I<n>.B<}> (alpha testing) =item B<{>I<n>.B<}> (beta testing)
Argument from input source I<n> or the I<n>'th argument without Argument from input source I<n> or the I<n>'th argument without
extension. It is a combination of B<{>I<n>B<}> and B<{.}>. extension. It is a combination of B<{>I<n>B<}> and B<{.}>.
@ -210,7 +214,7 @@ extension removed.
To understand positional replacement strings see B<{>I<n>B<}>. To understand positional replacement strings see B<{>I<n>B<}>.
=item B<{>I<n>/B<}> (alpha testing) =item B<{>I<n>/B<}> (beta testing)
Basename of argument from input source I<n> or the I<n>'th argument. Basename of argument from input source I<n> or the I<n>'th argument.
It is a combination of B<{>I<n>B<}> and B<{/}>. It is a combination of B<{>I<n>B<}> and B<{/}>.
@ -223,7 +227,7 @@ directory (if any) removed.
To understand positional replacement strings see B<{>I<n>B<}>. To understand positional replacement strings see B<{>I<n>B<}>.
=item B<{>I<n>//B<}> (alpha testing) =item B<{>I<n>//B<}> (beta testing)
Dirname of argument from input source I<n> or the I<n>'th argument. Dirname of argument from input source I<n> or the I<n>'th argument.
It is a combination of B<{>I<n>B<}> and B<{//}>. It is a combination of B<{>I<n>B<}> and B<{//}>.
@ -235,7 +239,7 @@ the I<n>'th argument (when used with B<-N>). See B<dirname>(1).
To understand positional replacement strings see B<{>I<n>B<}>. To understand positional replacement strings see B<{>I<n>B<}>.
=item B<{>I<n>/.B<}> (alpha testing) =item B<{>I<n>/.B<}> (beta testing)
Basename of argument from input source I<n> or the I<n>'th argument Basename of argument from input source I<n> or the I<n>'th argument
without extension. It is a combination of B<{>I<n>B<}>, B<{/}>, and without extension. It is a combination of B<{>I<n>B<}>, B<{/}>, and
@ -249,7 +253,7 @@ directory (if any) and extension removed.
To understand positional replacement strings see B<{>I<n>B<}>. To understand positional replacement strings see B<{>I<n>B<}>.
=item B<{=>I<perl expression>B<=}> (alpha testing) =item B<{=>I<perl expression>B<=}> (beta testing)
Replace with calculated I<perl expression>. B<$_> will contain the Replace with calculated I<perl expression>. B<$_> will contain the
same as B<{}>. After evaluating I<perl expression> B<$_> will be used same as B<{}>. After evaluating I<perl expression> B<$_> will be used
@ -262,7 +266,7 @@ The B<{=>I<perl expression>B<=}> must be given as a single string.
See also: B<--rpl> B<--parens> See also: B<--rpl> B<--parens>
=item B<{=>I<n> I<perl expression>B<=}> (alpha testing) =item B<{=>I<n> I<perl expression>B<=}> (beta testing)
Positional equivalent to B<{= perl expression =}>. To understand Positional equivalent to B<{= perl expression =}>. To understand
positional replacement strings see B<{>I<n>B<}>. positional replacement strings see B<{>I<n>B<}>.
@ -440,7 +444,7 @@ I<size> defaults to 1M.
See B<--pipe> and B<--pipepart> for use of this. See B<--pipe> and B<--pipepart> for use of this.
=item B<--cat> (alpha testing) =item B<--cat> (beta testing)
Create a temporary file with content. Normally B<--pipe>/B<--pipepart> Create a temporary file with content. Normally B<--pipe>/B<--pipepart>
will give data to the program on stdin (standard input). With B<--cat> will give data to the program on stdin (standard input). With B<--cat>
@ -450,7 +454,7 @@ you can do: B<parallel --pipe --cat wc {}>.
See also B<--fifo>. See also B<--fifo>.
=item B<--cleanup> (alpha testing) =item B<--cleanup> (beta testing)
Remove transferred files. B<--cleanup> will remove the transferred files Remove transferred files. B<--cleanup> will remove the transferred files
on the remote computer after processing is done. on the remote computer after processing is done.
@ -564,7 +568,7 @@ If I<eof-str> is omitted, there is no end of file string. If neither
B<-E> nor B<-e> is used, no end of file string is used. B<-E> nor B<-e> is used, no end of file string is used.
=item B<--env> I<var> (alpha testing) =item B<--env> I<var> (beta testing)
Copy environment variable I<var>. This will copy I<var> to the Copy environment variable I<var>. This will copy I<var> to the
environment that the command is run in. This is especially useful for environment that the command is run in. This is especially useful for
@ -631,7 +635,7 @@ Implies B<--semaphore>.
See also B<--bg>, B<man sem>. See also B<--bg>, B<man sem>.
=item B<--fifo> (alpha testing) =item B<--fifo> (beta testing)
Create a temporary fifo with content. Normally B<--pipe> and Create a temporary fifo with content. Normally B<--pipe> and
B<--pipepart> will give data to the program on stdin (standard B<--pipepart> will give data to the program on stdin (standard
@ -683,9 +687,9 @@ See also: B<--line-buffer> B<--ungroup>
Print a summary of the options to GNU B<parallel> and exit. Print a summary of the options to GNU B<parallel> and exit.
=item B<--halt-on-error> I<val> (alpha testing) =item B<--halt-on-error> I<val> (beta testing)
=item B<--halt> I<val> (alpha testing) =item B<--halt> I<val> (beta testing)
How should GNU B<parallel> terminate? How should GNU B<parallel> terminate?
@ -741,9 +745,9 @@ B<--header :> is an alias for B<--header '.*\n'>.
If I<regexp> is a number, it is a fixed number of lines. If I<regexp> is a number, it is a fixed number of lines.
=item B<--hostgroups> (beta testing) =item B<--hostgroups>
=item B<--hgrp> (beta testing) =item B<--hgrp>
Enable hostgroups on arguments. If an argument contains '@' the string Enable hostgroups on arguments. If an argument contains '@' the string
after '@' will be removed and treated as a list of hostgroups on which after '@' will be removed and treated as a list of hostgroups on which
@ -921,7 +925,7 @@ limiting factor.
See also: B<--group> B<--ungroup> See also: B<--group> B<--ungroup>
=item B<--load> I<max-load> (alpha testing) =item B<--load> I<max-load> (beta testing)
Do not start new jobs on a given computer unless the number of running Do not start new jobs on a given computer unless the number of running
processes on the computer is less than I<max-load>. I<max-load> uses processes on the computer is less than I<max-load>. I<max-load> uses
@ -969,7 +973,7 @@ See also B<-X> for context replace. If in doubt use B<-X> as that will
most likely do what is needed. most likely do what is needed.
=item B<--memfree> I<size> (alpha testing) =item B<--memfree> I<size> (beta testing)
Minimum memory free when starting another job. The I<size> can be Minimum memory free when starting another job. The I<size> can be
postfixed with K, M, G, T, P, k, m, g, t, or p which would multiply postfixed with K, M, G, T, P, k, m, g, t, or p which would multiply
@ -1165,7 +1169,7 @@ Print the number of CPU cores and exit (used by GNU B<parallel> itself
to determine the number of CPU cores on remote computers). to determine the number of CPU cores on remote computers).
=item B<--no-keep-order> (alpha testing) =item B<--no-keep-order> (beta testing)
Overrides an earlier B<--keep-order> (e.g. if set in Overrides an earlier B<--keep-order> (e.g. if set in
B<~/.parallel/config>). B<~/.parallel/config>).
@ -1388,7 +1392,7 @@ useful if some jobs fail for no apparent reason (such as network
failure). failure).
=item B<--return> I<filename> (alpha testing) =item B<--return> I<filename> (beta testing)
Transfer files from remote computers. B<--return> is used with Transfer files from remote computers. B<--return> is used with
B<--sshlogin> when the arguments are files on the remote computers. When B<--sshlogin> when the arguments are files on the remote computers. When
@ -1489,7 +1493,7 @@ operating system and the B<-s> option. Pipe the input from /dev/null
to do anything. to do anything.
=item B<--semaphore> (alpha testing) =item B<--semaphore> (beta testing)
Work as a counting semaphore. B<--semaphore> will cause GNU Work as a counting semaphore. B<--semaphore> will cause GNU
B<parallel> to start I<command> in the background. When the number of B<parallel> to start I<command> in the background. When the number of
@ -1526,9 +1530,9 @@ Implies B<--semaphore>.
See also B<man sem>. See also B<man sem>.
=item B<--semaphoretimeout> I<secs> (alpha testing) =item B<--semaphoretimeout> I<secs> (beta testing)
=item B<--st> I<secs> (alpha testing) =item B<--st> I<secs> (beta testing)
If I<secs> > 0: If the semaphore is not released within I<secs> seconds, take it anyway. If I<secs> > 0: If the semaphore is not released within I<secs> seconds, take it anyway.
@ -1624,9 +1628,9 @@ I<secs> seconds after starting each ssh. I<secs> can be less than 1
seconds. seconds.
=item B<-S> I<[@hostgroups/][ncpu/]sshlogin[,[@hostgroups/][ncpu/]sshlogin[,...]]> (alpha testing) =item B<-S> I<[@hostgroups/][ncpu/]sshlogin[,[@hostgroups/][ncpu/]sshlogin[,...]]> (beta testing)
=item B<--sshlogin> I<[@hostgroups/][ncpu/]sshlogin[,[@hostgroups/][ncpu/]sshlogin[,...]]> (alpha testing) =item B<--sshlogin> I<[@hostgroups/][ncpu/]sshlogin[,[@hostgroups/][ncpu/]sshlogin[,...]]> (beta testing)
Distribute jobs to remote computers. The jobs will be run on a list of Distribute jobs to remote computers. The jobs will be run on a list of
remote computers. remote computers.
@ -1807,7 +1811,7 @@ Print the job to be run on stderr (standard error).
See also B<-v>, B<-p>. See also B<-v>, B<-p>.
=item B<--transfer> (alpha testing) =item B<--transfer> (beta testing)
Transfer files to remote computers. B<--transfer> is used with Transfer files to remote computers. B<--transfer> is used with
B<--sshlogin> when the arguments are files and should be transferred B<--sshlogin> when the arguments are files and should be transferred
@ -1834,7 +1838,7 @@ B<--transfer> is often used with B<--return> and B<--cleanup>.
B<--transfer> is ignored when used with B<--sshlogin :> or when not used with B<--sshlogin>. B<--transfer> is ignored when used with B<--sshlogin :> or when not used with B<--sshlogin>.
=item B<--trc> I<filename> (alpha testing) =item B<--trc> I<filename> (beta testing)
Transfer, Return, Cleanup. Short hand for: Transfer, Return, Cleanup. Short hand for:

1
testsuite/tests-to-run/niceload02.sh
View file

@ -19,3 +19,4 @@ SIZET=/tmp/parallel_sizetest
rm -f $SIZET rm -f $SIZET
tmux new-session -d -n 10 "seq 10000 | pv -qL 1000 > $SIZET" tmux new-session -d -n 10 "seq 10000 | pv -qL 1000 > $SIZET"
niceload -t .01 --sensor "stat -c %b $SIZET" -l -10 "stat -c %b $SIZET" niceload -t .01 --sensor "stat -c %b $SIZET" -l -10 "stat -c %b $SIZET"
rm $SIZET

1
testsuite/tests-to-run/niceload04.sh
View file

@ -20,3 +20,4 @@ echo '### multiple -p'
/tmp/mysleep 2 & /tmp/mysleep 2 &
echo '### --prg' echo '### --prg'
stdout /usr/bin/time -f %e niceload -l 8 -H --prg mysleep | perl -ne '$_ > 5 and print "--prg OK\n"' stdout /usr/bin/time -f %e niceload -l 8 -H --prg mysleep | perl -ne '$_ > 5 and print "--prg OK\n"'
rm /tmp/mysleep

3
testsuite/tests-to-run/parallel-local-30s.sh
View file

@ -13,7 +13,8 @@ sudo mount -t tmpfs -o size=10% none $SHM
cat <<'EOF' | sed -e 's/;$/; /;s/$SERVER1/'$SERVER1'/;s/$SERVER2/'$SERVER2'/' | stdout parallel -vj4 -k --joblog /tmp/jl-`basename $0` -L1 cat <<'EOF' | sed -e 's/;$/; /;s/$SERVER1/'$SERVER1'/;s/$SERVER2/'$SERVER2'/' | stdout parallel -vj4 -k --joblog /tmp/jl-`basename $0` -L1
echo '### Test race condition on 8 CPU (my laptop)'; echo '### Test race condition on 8 CPU (my laptop)';
seq 1 5000000 > /tmp/parallel_test; seq 1 5000000 > /tmp/parallel_test;
seq 1 10 | parallel -k "cat /tmp/parallel_test | parallel --pipe --recend '' -k gzip >/dev/null; echo {}" seq 1 10 | parallel -k "cat /tmp/parallel_test | parallel --pipe --recend '' -k gzip >/dev/null; echo {}";
rm /tmp/parallel_test
echo '**' echo '**'

4
testsuite/tests-to-run/parallel-local-ssh2.sh
View file

@ -7,7 +7,8 @@ echo "### bug #43518: GNU Parallel doesn't obey servers' jobs capacity when an s
echo -e '1/lo\n1/csh@lo\n1/tcsh@lo\n1/parallel@lo\n' > /tmp/parallel.bug43518; echo -e '1/lo\n1/csh@lo\n1/tcsh@lo\n1/parallel@lo\n' > /tmp/parallel.bug43518;
parallel --delay 0.1 -N0 echo 1/: '>>' /tmp/parallel.bug43518 ::: {1..100} & parallel --delay 0.1 -N0 echo 1/: '>>' /tmp/parallel.bug43518 ::: {1..100} &
seq 30 | stdout /usr/bin/time -f %e parallel --slf /tmp/parallel.bug43518 'sleep {=$_=$_%3?0:10=}.{%}' | seq 30 | stdout /usr/bin/time -f %e parallel --slf /tmp/parallel.bug43518 'sleep {=$_=$_%3?0:10=}.{%}' |
perl -ne '$_ > 25 and print "OK\n"' perl -ne '$_ > 25 and print "OK\n"';
rm /tmp/parallel.bug43518
echo '### --filter-hosts --slf <()' echo '### --filter-hosts --slf <()'
parallel --nonall --filter-hosts --slf <(echo localhost) echo OK parallel --nonall --filter-hosts --slf <(echo localhost) echo OK
@ -43,6 +44,7 @@ echo '### bug #42999: --pipepart with remote does not work'
parallel --sshdelay 0.3 --pipepart --block 31 -a /tmp/bug42999 -k -S parallel@lo wc | perl -pe s:/tmp/.........pip:/tmp/XXXX: ; parallel --sshdelay 0.3 --pipepart --block 31 -a /tmp/bug42999 -k -S parallel@lo wc | perl -pe s:/tmp/.........pip:/tmp/XXXX: ;
parallel --sshdelay 0.2 --pipepart --block 31 -a /tmp/bug42999 -k --fifo -S parallel@lo wc | perl -pe s:/tmp/.........pip:/tmp/XXXX: ; parallel --sshdelay 0.2 --pipepart --block 31 -a /tmp/bug42999 -k --fifo -S parallel@lo wc | perl -pe s:/tmp/.........pip:/tmp/XXXX: ;
parallel --sshdelay 0.1 --pipepart --block 31 -a /tmp/bug42999 -k --cat -S parallel@lo wc | perl -pe s:/tmp/.........pip:/tmp/XXXX: ; parallel --sshdelay 0.1 --pipepart --block 31 -a /tmp/bug42999 -k --cat -S parallel@lo wc | perl -pe s:/tmp/.........pip:/tmp/XXXX: ;
rm /tmp/bug42999
echo '### --cat gives incorrect exit value in csh' echo '### --cat gives incorrect exit value in csh'
echo false | parallel --pipe --cat -Scsh@lo 'cat {}; false' ; echo $?; echo false | parallel --pipe --cat -Scsh@lo 'cat {}; false' ; echo $?;

11
testsuite/tests-to-run/parallel-local-ssh3.sh
View file

@ -46,11 +46,18 @@ echo '### Special char file and dir transfer return and cleanup'
}; };
export -f mytouch; export -f mytouch;
parallel --env mytouch -Sparallel@lo --transfer parallel --env mytouch -Sparallel@lo --transfer
--return d"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"/g"`perl -e 'print pack("c*",1..9,11..46,48..255)'`" --return {=s:/f:/g:=}
mytouch mytouch
::: d"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"/f"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"; ::: d"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"/f"`perl -e 'print pack("c*",1..9,11..46,48..255)'`";
cat d"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"/g"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"; cat d"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"/g"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"
echo '### Uniq {=perlexpr=} in return - not used in command'
cd /tmp;
rm -f /tmp/parallel_perlexpr.2Parallel_PerlexPr;
echo local > parallel_perlexpr;
parallel -Sparallel@lo --trc {=s/pr/pr.2/=}{=s/p/P/g=} echo remote OK '>' {}.2{=s/p/P/g=} ::: parallel_perlexpr;
cat /tmp/parallel_perlexpr.2Parallel_PerlexPr;
rm -f /tmp/parallel_perlexpr.2Parallel_PerlexPr /tmp/parallel_perlexpr
# Should be changed to --return '{=s:/f:/g:=}' and tested with csh # Should be changed to --return '{=s:/f:/g:=}' and tested with csh

14
testsuite/tests-to-run/parallel-local114.sh
View file

@ -5,13 +5,17 @@ echo "### --line-buffer"
seq 10 | parallel -j20 --line-buffer 'seq {} 10 | pv -qL 10' > /tmp/parallel_l$$; seq 10 | parallel -j20 --line-buffer 'seq {} 10 | pv -qL 10' > /tmp/parallel_l$$;
seq 10 | parallel -j20 'seq {} 10 | pv -qL 10' > /tmp/parallel_$$; seq 10 | parallel -j20 'seq {} 10 | pv -qL 10' > /tmp/parallel_$$;
cat /tmp/parallel_l$$ | wc; cat /tmp/parallel_l$$ | wc;
diff /tmp/parallel_$$ /tmp/parallel_l$$ >/dev/null ; echo These must diff: $? diff /tmp/parallel_$$ /tmp/parallel_l$$ >/dev/null ;
echo These must diff: $?;
rm /tmp/parallel_l$$ /tmp/parallel_$$
echo "### --pipe --line-buffer" echo "### --pipe --line-buffer"
seq 200| parallel -N10 -L1 --pipe -j20 --line-buffer --tagstring {#} pv -qL 10 > /tmp/parallel_pl$$; seq 200| parallel -N10 -L1 --pipe -j20 --line-buffer --tagstring {#} pv -qL 10 > /tmp/parallel_pl$$;
seq 200| parallel -N10 -L1 --pipe -j20 --tagstring {#} pv -qL 10 > /tmp/parallel_p$$; seq 200| parallel -N10 -L1 --pipe -j20 --tagstring {#} pv -qL 10 > /tmp/parallel_p$$;
cat /tmp/parallel_pl$$ | wc; cat /tmp/parallel_pl$$ | wc;
diff /tmp/parallel_p$$ /tmp/parallel_pl$$ >/dev/null ; echo These must diff: $? diff /tmp/parallel_p$$ /tmp/parallel_pl$$ >/dev/null ;
echo These must diff: $?;
rm /tmp/parallel_pl$$ /tmp/parallel_p$$
echo "### --pipe --line-buffer --compress" echo "### --pipe --line-buffer --compress"
seq 200| parallel -N10 -L1 --pipe -j20 --line-buffer --compress --tagstring {#} pv -qL 10 | wc seq 200| parallel -N10 -L1 --pipe -j20 --line-buffer --compress --tagstring {#} pv -qL 10 | wc
@ -74,13 +78,13 @@ echo "### bug #36659: --sshlogin strips leading slash from ssh command";
echo "### bug #36660: --workdir mkdir does not use --sshlogin custom ssh"; echo "### bug #36660: --workdir mkdir does not use --sshlogin custom ssh";
rm -rf /tmp/foo36660; rm -rf /tmp/foo36660;
cd /tmp; echo OK > parallel_test.txt; cd /tmp; echo OK > parallel_test36660.txt;
ssh () { echo Failed; }; ssh () { echo Failed; };
export -f ssh; export -f ssh;
parallel --workdir /tmp/foo36660/bar --transfer --sshlogin '/usr/bin/ssh localhost' cat ::: parallel_test.txt; parallel --workdir /tmp/foo36660/bar --transfer --sshlogin '/usr/bin/ssh localhost' cat ::: parallel_test36660.txt;
rm -rf /tmp/foo36660 parallel_test36660.txt
echo "bug #36657: --load does not work with custom ssh"; echo "bug #36657: --load does not work with custom ssh";
cd /tmp; echo OK > parallel_test.txt;
ssh () { echo Failed; }; ssh () { echo Failed; };
export -f ssh; export -f ssh;
parallel --load=1000% -S "/usr/bin/ssh localhost" echo ::: OK parallel --load=1000% -S "/usr/bin/ssh localhost" echo ::: OK

3
testsuite/tests-to-run/parallel-local12.sh
View file

@ -10,6 +10,8 @@ EOF
chmod 755 /tmp/parallel-script-for-script chmod 755 /tmp/parallel-script-for-script
echo via pseudotty | script -q -f -c /tmp/parallel-script-for-script /dev/null echo via pseudotty | script -q -f -c /tmp/parallel-script-for-script /dev/null
sleep 2 sleep 2
rm /tmp/parallel-script-for-script
echo '### Test stdin goes to first command only ("cat" as argument)' echo '### Test stdin goes to first command only ("cat" as argument)'
cat >/tmp/parallel-script-for-script2 <<EOF cat >/tmp/parallel-script-for-script2 <<EOF
#!/bin/bash #!/bin/bash
@ -18,5 +20,6 @@ EOF
chmod 755 /tmp/parallel-script-for-script2 chmod 755 /tmp/parallel-script-for-script2
echo via pseudotty | script -q -f -c /tmp/parallel-script-for-script2 /dev/null echo via pseudotty | script -q -f -c /tmp/parallel-script-for-script2 /dev/null
sleep 2 sleep 2
rm /tmp/parallel-script-for-script2
touch ~/.parallel/will-cite touch ~/.parallel/will-cite

3
testsuite/tests-to-run/parallel-local150.sh
View file

@ -60,7 +60,8 @@ echo "### BUG-fix: bash -c 'parallel -a <(seq 1 3) echo'"
echo "### bug #35268: shell_quote doesn't treats [] brackets correctly" echo "### bug #35268: shell_quote doesn't treats [] brackets correctly"
touch /tmp/foo1; touch /tmp/foo1;
stdout parallel echo ::: '/tmp/foo[123]' stdout parallel echo ::: '/tmp/foo[123]';
rm /tmp/foo1
echo '### Test make .deb package'; echo '### Test make .deb package';
cd ~/privat/parallel/packager/debian; cd ~/privat/parallel/packager/debian;

8
testsuite/tests-to-run/parallel-local164.sh
View file

@ -15,14 +15,15 @@ echo '### test --sshdelay'
stdout /usr/bin/time -f %e parallel -j0 --sshdelay 0.5 -S localhost true ::: 1 2 3 | perl -ne 'print($_ > 1.30 ? "OK\n" : "Not OK\n")' stdout /usr/bin/time -f %e parallel -j0 --sshdelay 0.5 -S localhost true ::: 1 2 3 | perl -ne 'print($_ > 1.30 ? "OK\n" : "Not OK\n")'
echo '### bug #38299: --resume-failed -k' echo '### bug #38299: --resume-failed -k'
rm /tmp/joblog-38299; rm -f /tmp/joblog-38299;
parallel -k --resume-failed --joblog /tmp/joblog-38299 echo job{#} val {}\;exit {} ::: 0 1 2 3 0 1; parallel -k --resume-failed --joblog /tmp/joblog-38299 echo job{#} val {}\;exit {} ::: 0 1 2 3 0 1;
echo try 2. Gives failing - not 0; echo try 2. Gives failing - not 0;
parallel -k --resume-failed --joblog /tmp/joblog-38299 echo job{#} val {}\;exit {} ::: 0 1 2 3 0 1; parallel -k --resume-failed --joblog /tmp/joblog-38299 echo job{#} val {}\;exit {} ::: 0 1 2 3 0 1;
echo with exit 0; echo with exit 0;
parallel -k --resume-failed --joblog /tmp/joblog-38299 echo job{#} val {}\;exit 0 ::: 0 1 2 3 0 1; parallel -k --resume-failed --joblog /tmp/joblog-38299 echo job{#} val {}\;exit 0 ::: 0 1 2 3 0 1;
echo try 2 again. Gives empty; echo try 2 again. Gives empty;
parallel -k --resume-failed --joblog /tmp/joblog-38299 echo job{#} val {}\;exit {} ::: 0 1 2 3 0 1 parallel -k --resume-failed --joblog /tmp/joblog-38299 echo job{#} val {}\;exit {} ::: 0 1 2 3 0 1;
rm /tmp/joblog-38299
echo '### --resume -k' echo '### --resume -k'
rm -f /tmp/joblog-resume; rm -f /tmp/joblog-resume;
@ -30,7 +31,8 @@ echo '### --resume -k'
echo try 2 = nothing; echo try 2 = nothing;
parallel -k --resume --joblog /tmp/joblog-resume echo job{}id\;exit {} ::: 0 1 2 3 0 5; parallel -k --resume --joblog /tmp/joblog-resume echo job{}id\;exit {} ::: 0 1 2 3 0 5;
echo two extra; echo two extra;
parallel -k --resume --joblog /tmp/joblog-resume echo job{}id\;exit 0 ::: 0 1 2 3 0 5 6 7 parallel -k --resume --joblog /tmp/joblog-resume echo job{}id\;exit 0 ::: 0 1 2 3 0 5 6 7;
rm -f /tmp/joblog-resume
echo "bug #37694: Empty string argument skipped when using --quote" echo "bug #37694: Empty string argument skipped when using --quote"
parallel -q --nonall perl -le 'print scalar @ARGV' 'a' 'b' '' parallel -q --nonall perl -le 'print scalar @ARGV' 'a' 'b' ''

17
testsuite/tests-to-run/parallel-local22.sh
View file

@ -48,6 +48,7 @@ echo '### bug #42363: --pipepart and --fifo/--cat does not work'
seq 100 > /tmp/bug42363; seq 100 > /tmp/bug42363;
parallel --pipepart --block 31 -a /tmp/bug42363 -k --fifo wc | perl -pe s:/tmp/.........pip:/tmp/XXXX: ; parallel --pipepart --block 31 -a /tmp/bug42363 -k --fifo wc | perl -pe s:/tmp/.........pip:/tmp/XXXX: ;
parallel --pipepart --block 31 -a /tmp/bug42363 -k --cat wc | perl -pe s:/tmp/.........pip:/tmp/XXXX: ; parallel --pipepart --block 31 -a /tmp/bug42363 -k --cat wc | perl -pe s:/tmp/.........pip:/tmp/XXXX: ;
rm /tmp/bug42363
echo '### bug #42055: --pipepart -a bigfile should not require sequential reading of bigfile' echo '### bug #42055: --pipepart -a bigfile should not require sequential reading of bigfile'
parallel --pipepart -a /etc/passwd -L 1 should not be run parallel --pipepart -a /etc/passwd -L 1 should not be run
@ -71,18 +72,26 @@ echo '### bug #42892: parallel -a nonexiting --pipepart'
echo '### bug #42913: Dont use $SHELL but the shell currently running' echo '### bug #42913: Dont use $SHELL but the shell currently running'
echo '## Unknown shell => $SHELL (bash)' echo '## Unknown shell => $SHELL (bash)'
parallel -j1 "cp \`which {}\` /tmp/SHELL; /tmp/SHELL -c 'parallel -Dinit echo ::: 1' | grep which;" parallel -j1 "cp \`which {}\` /tmp/SHELL; /tmp/SHELL -c 'parallel -Dinit echo ::: 1' | grep which;"
::: ash bash csh dash fdsh fish fizsh ksh ksh93 mksh pdksh posh rbash rush rzsh sash sh static-sh tcsh yash zsh ::: ash bash csh dash fdsh fish fizsh ksh ksh93 mksh pdksh posh rbash rush rzsh sash sh static-sh tcsh yash zsh;
rm /tmp/SHELL /tmp/par*.par
echo '## Known shells -c' echo '## Known shells -c'
parallel -k "\`which {}\` -c 'parallel -Dinit echo ::: 1' | grep which;" parallel -k "\`which {}\` -c 'parallel -Dinit echo ::: 1' | grep which;"
::: ash bash csh dash fdsh fish fizsh ksh ksh93 mksh pdksh posh rbash rush rzsh sash sh static-sh tcsh yash zsh ::: ash bash csh dash fdsh fish fizsh ksh ksh93 mksh pdksh posh rbash rush rzsh sash sh static-sh tcsh yash zsh;
rm /tmp/par*.par
echo '## Known shells |' echo '## Known shells |'
parallel -k "echo 'parallel -Dinit echo ::: 1' | \`which {}\` | grep which;" parallel -k "echo 'parallel -Dinit echo ::: 1' | \`which {}\` | grep which;"
::: ash bash csh dash fdsh fish fizsh ksh ksh93 mksh pdksh posh rbash rush rzsh sash sh static-sh tcsh yash zsh ::: ash bash csh dash fdsh fish fizsh ksh ksh93 mksh pdksh posh rbash rush rzsh sash sh static-sh tcsh yash zsh;
rm /tmp/par*.par
echo '## Started directly from perl' echo '## Started directly from perl'
perl -e 'system(qw(parallel -Dinit echo ::: 1))' | grep which perl -e 'system(qw(parallel -Dinit echo ::: 1))' | grep which;
rm /tmp/par*.par
EOF EOF
# TODO This is too unstable # TODO This is too unstable
# echo '### --tmux test - check termination' # echo '### --tmux test - check termination'
# perl -e 'map {printf "$_%o%c\n",$_,$_}1..255' | stdout parallel --tmux echo {} :::: - ::: a b | perl -pe 's/\d/0/g' # perl -e 'map {printf "$_%o%c\n",$_,$_}1..255' | stdout parallel --tmux echo {} :::: - ::: a b | perl -pe 's/\d/0/g'

22
testsuite/tests-to-run/parallel-local4.sh
View file

@ -41,4 +41,26 @@ echo '### Test if length is computed correctly - first should give one line, sec
seq 4 | parallel -s 21 -m -kj1 echo {} {} seq 4 | parallel -s 21 -m -kj1 echo {} {}
seq 4 | parallel -s 20 -m -kj1 echo {} {} seq 4 | parallel -s 20 -m -kj1 echo {} {}
echo 'bug #44144: --tagstring {=s/a/b/=} broken'
# Do not be confused by {} in --rpl
parallel --rpl '{:} s/A/D/;{}' --tagstring '{1:}{-1:}{= s/A/E/=}' echo {} ::: A/B.C
# Non-standard --parens
parallel --parens ,, --rpl '{:} s/A/D/;{}' --tagstring '{1:}{-1:}, 's/A/E/, echo {} ::: A/B.C
# Non-standard --parens -i
parallel --rpl '{:} s/A/D/;{}' --tag --parens ,, -iDUMMY echo {} ::: A/B.C
echo 'env in zsh'
echo 'Normal variable export'
export B=\';
PARALLEL_SHELL=/usr/bin/zsh parallel --env B echo '$B' ::: a
echo 'Function export as variable'
export myfuncvar="() { echo myfuncvar \$*; }";
PARALLEL_SHELL=/usr/bin/zsh parallel --env myfuncvar myfuncvar ::: a
echo 'Function export as function'
myfunc() { echo myfunc $*; };
export -f myfunc;
PARALLEL_SHELL=/usr/bin/zsh parallel --env myfunc myfunc ::: a
EOF EOF

2
testsuite/tests-to-run/parallel-local5.sh
View file

@ -114,3 +114,5 @@ echo '### Test --block size=1M -j1 - more data than cpu';
echo '### Test --pipe default settings'; echo '### Test --pipe default settings';
cat /tmp/blocktest | parallel --pipe sort | sort -n | md5sum cat /tmp/blocktest | parallel --pipe sort | sort -n | md5sum
EOF EOF
rm /tmp/parallel-seq /tmp/blocktest

12
testsuite/tests-to-run/parallel-local9.sh
View file

@ -59,24 +59,28 @@ echo 'bug #41412: --timeout + --delay causes deadlock';
echo '### Test --spreadstdin - more procs than args'; echo '### Test --spreadstdin - more procs than args';
rm -f /tmp/parallel.ss.*; rm -f /tmp/parallel.ss.*;
seq 1 5 | stdout $NICEPAR -j 10 --spreadstdin 'cat >/tmp/parallel.ss.$PARALLEL_SEQ' >/dev/null; seq 1 5 | stdout $NICEPAR -j 10 --spreadstdin 'cat >/tmp/parallel.ss.$PARALLEL_SEQ' >/dev/null;
cat /tmp/parallel.ss.*; cat /tmp/parallel.ss.*;
rm -f /tmp/parallel.ss.*
echo '### Test --spreadstdin - more args than procs'; echo '### Test --spreadstdin - more args than procs';
rm -f /tmp/parallel.ss2.*; rm -f /tmp/parallel.ss2.*;
seq 1 10 | stdout $NICEPAR -j 5 --spreadstdin 'cat >/tmp/parallel.ss2.$PARALLEL_SEQ' >/dev/null; seq 1 10 | stdout $NICEPAR -j 5 --spreadstdin 'cat >/tmp/parallel.ss2.$PARALLEL_SEQ' >/dev/null;
cat /tmp/parallel.ss2.* cat /tmp/parallel.ss2.*;
rm -f /tmp/parallel.ss2.*
nice nice seq 1 1000 | $NICEPAR -j1 --spreadstdin cat "|cat "|wc -c nice nice seq 1 1000 | $NICEPAR -j1 --spreadstdin cat "|cat "|wc -c
nice nice seq 1 10000 | $NICEPAR -j10 --spreadstdin cat "|cat "|wc -c nice nice seq 1 10000 | $NICEPAR -j10 --spreadstdin cat "|cat "|wc -c
nice nice seq 1 100000 | $NICEPAR -j1 --spreadstdin cat "|cat "|wc -c nice nice seq 1 100000 | $NICEPAR -j1 --spreadstdin cat "|cat "|wc -c
nice nice seq 1 1000000 | $NICEPAR -j10 --spreadstdin cat "|cat "|wc -c nice nice seq 1 1000000 | $NICEPAR -j10 --spreadstdin cat "|cat "|wc -c
seq 1 10 | $NICEPAR --recend "\n" -j1 --spreadstdin gzip -9 >/tmp/foo.gz seq 1 10 | $NICEPAR --recend "\n" -j1 --spreadstdin gzip -9 >/tmp/foo.gz;
rm /tmp/foo.gz
echo '### Test --spreadstdin - similar to the failing below'; echo '### Test --spreadstdin - similar to the failing below';
nice seq 1 100000 | $NICEPAR --recend "\n" -j10 --spreadstdin gzip -9 >/tmp/foo2.gz; nice seq 1 100000 | $NICEPAR --recend "\n" -j10 --spreadstdin gzip -9 >/tmp/foo2.gz;
diff <(nice seq 1 100000) <(zcat /tmp/foo2.gz |sort -n); diff <(nice seq 1 100000) <(zcat /tmp/foo2.gz |sort -n);
diff <(nice seq 1 100000|wc -c) <(zcat /tmp/foo2.gz |wc -c) diff <(nice seq 1 100000|wc -c) <(zcat /tmp/foo2.gz |wc -c);
rm /tmp/foo2.gz
echo '### Test --spreadstdin - this failed during devel'; echo '### Test --spreadstdin - this failed during devel';
nice seq 1 1000000 | md5sum; nice seq 1 1000000 | md5sum;

1
testsuite/tests-to-run/parallel-remote1.sh
View file

@ -35,3 +35,4 @@ echo '### test --filter-hosts with server w/o ssh, non-existing server'
echo '### Missing: test --filter-hosts proxied through the one host' echo '### Missing: test --filter-hosts proxied through the one host'
EOF EOF
rm /tmp/myssh1 /tmp/myssh2

5
testsuite/tests-to-run/test13.sh
View file

@ -12,11 +12,12 @@ echo '### Test --keeporder'
(seq 0 2) | parallel --keeporder -j100% -S 1/:,2/parallel@parallel-server2 -q perl -e 'sleep 1;print "job{}\n";exit({})' (seq 0 2) | parallel --keeporder -j100% -S 1/:,2/parallel@parallel-server2 -q perl -e 'sleep 1;print "job{}\n";exit({})'
echo '### Test SIGTERM' echo '### Test SIGTERM'
parallel -k -j20 sleep 3';' echo ::: {1..99} >/tmp/$$ 2>&1 & parallel -k -j20 sleep 3';' echo ::: {1..99} >/tmp/parallel$$ 2>&1 &
A=$! A=$!
sleep 5; kill -TERM $A sleep 5; kill -TERM $A
wait wait
sort /tmp/$$ sort /tmp/parallel$$
rm /tmp/parallel$$
echo '### Test bug: empty line for | sh with -k' echo '### Test bug: empty line for | sh with -k'
(echo echo a ; echo ; echo echo b) | parallel -k (echo echo a ; echo ; echo echo b) | parallel -k

4
testsuite/tests-to-run/test15.sh
View file

@ -92,8 +92,8 @@ echo '### Test -x'
(seq 1 10; echo 1234; seq 12 15) | stdout xargs -s 10 -x echo (seq 1 10; echo 1234; seq 12 15) | stdout xargs -s 10 -x echo
echo '### Test -a and --arg-file: Read input from file instead of stdin' echo '### Test -a and --arg-file: Read input from file instead of stdin'
seq 1 10 >/tmp/$$-1; parallel -k -a /tmp/$$-1 echo seq 1 10 >/tmp/parallel_$$-1; parallel -k -a /tmp/parallel_$$-1 echo; rm /tmp/parallel_$$-1
seq 1 10 >/tmp/$$-2; parallel -k --arg-file /tmp/$$-2 echo seq 1 10 >/tmp/parallel_$$-2; parallel -k --arg-file /tmp/parallel_$$-2 echo; rm /tmp/parallel_$$-2
EOF EOF

2
testsuite/tests-to-run/test19.sh
View file

@ -126,3 +126,5 @@ cat /tmp/myssh1-run /tmp/myssh2-run | perl -pe 's/(PID.)\d+/${1}00000/g;s/(SEQ[
perl -pe 's/\S*parallel-server\S*/one-server/;s:[a-zA-Z0-9/+=]{500,}:base64:;' perl -pe 's/\S*parallel-server\S*/one-server/;s:[a-zA-Z0-9/+=]{500,}:base64:;'
rm /tmp/myssh1-run /tmp/myssh2-run rm /tmp/myssh1-run /tmp/myssh2-run
rm -rf /tmp/parallel.file*
rm /tmp/myssh1 /tmp/myssh2

10
testsuite/tests-to-run/test45.sh
View file

@ -29,12 +29,18 @@ echo '### Test --load read from a file - more than 3s'
echo '# This will run 10 processes in parallel for 10s'; echo '# This will run 10 processes in parallel for 10s';
seq 10 | parallel --nice 19 --timeout 10 -j0 -N0 "gzip < /dev/zero > /dev/null" & seq 10 | parallel --nice 19 --timeout 10 -j0 -N0 "gzip < /dev/zero > /dev/null" &
( echo 8 > /tmp/parallel_load_file; sleep 10; echo 1000 > /tmp/parallel_load_file ) & ( echo 8 > /tmp/parallel_load_file; sleep 10; echo 1000 > /tmp/parallel_load_file ) &
sleep 1;stdout /usr/bin/time -f %e parallel --load /tmp/parallel_load_file sleep ::: 1 | perl -ne '$_ > 9 and print "OK\n"' sleep 1;
stdout /usr/bin/time -f %e parallel --load /tmp/parallel_load_file sleep ::: 1 |
perl -ne '$_ > 9 and print "OK\n"';
rm /tmp/parallel_load_file
echo '### Test --load read from a file - less than 10s'; echo '### Test --load read from a file - less than 10s';
echo '# This will run 10 processes in parallel for 10s'; echo '# This will run 10 processes in parallel for 10s';
seq 10 | parallel --nice 19 --timeout 10 -j0 -N0 "gzip < /dev/zero > /dev/null" & seq 10 | parallel --nice 19 --timeout 10 -j0 -N0 "gzip < /dev/zero > /dev/null" &
( echo 8 > /tmp/parallel_load_file2; sleep 10; echo 1000 > /tmp/parallel_load_file2 ) & ( echo 8 > /tmp/parallel_load_file2; sleep 10; echo 1000 > /tmp/parallel_load_file2 ) &
sleep 1;stdout /usr/bin/time -f %e parallel --load /tmp/parallel_load_file2 sleep ::: 1 | perl -ne '$_ < 20 and print "OK\n"' sleep 1;
stdout /usr/bin/time -f %e parallel --load /tmp/parallel_load_file2 sleep ::: 1 |
perl -ne '$_ < 20 and print "OK\n"';
rm /tmp/parallel_load_file2
EOF EOF

6
testsuite/tests-to-run/test60.sh
View file

@ -28,11 +28,13 @@ echo '### Test read sshloginfile from STDIN';
echo '### Test --nonall --basefile'; echo '### Test --nonall --basefile';
touch /tmp/nonall--basefile; touch /tmp/nonall--basefile;
parallel --nonall --basefile /tmp/nonall--basefile -S $SSHLOGIN1,$SSHLOGIN2 ls /tmp/nonall--basefile parallel --nonall --basefile /tmp/nonall--basefile -S $SSHLOGIN1,$SSHLOGIN2 ls /tmp/nonall--basefile;
rm /tmp/nonall--basefile
echo '### Test --onall --basefile'; echo '### Test --onall --basefile';
touch /tmp/onall--basefile; touch /tmp/onall--basefile;
parallel --onall --basefile /tmp/onall--basefile -S $SSHLOGIN1,$SSHLOGIN2 ls ::: /tmp/onall--basefile parallel --onall --basefile /tmp/onall--basefile -S $SSHLOGIN1,$SSHLOGIN2 ls ::: /tmp/onall--basefile;
rm /tmp/onall--basefile
echo '### Test --workdir .'; echo '### Test --workdir .';
ssh $SSHLOGIN1 mkdir -p mydir; ssh $SSHLOGIN1 mkdir -p mydir;

2
testsuite/wanted-results/parallel-local-30s
View file

@ -1,4 +1,4 @@
echo '### Test race condition on 8 CPU (my laptop)'; seq 1 5000000 > /tmp/parallel_test; seq 1 10 | parallel -k "cat /tmp/parallel_test | parallel --pipe --recend '' -k gzip >/dev/null; echo {}" echo '### Test race condition on 8 CPU (my laptop)'; seq 1 5000000 > /tmp/parallel_test; seq 1 10 | parallel -k "cat /tmp/parallel_test | parallel --pipe --recend '' -k gzip >/dev/null; echo {}"; rm /tmp/parallel_test
### Test race condition on 8 CPU (my laptop) ### Test race condition on 8 CPU (my laptop)
1 1
2 2

2
testsuite/wanted-results/parallel-local-ssh2
View file

@ -2,7 +2,7 @@ echo "### bug #43518: GNU Parallel doesn't obey servers' jobs capacity when an s
### bug #43518: GNU Parallel doesn't obey servers' jobs capacity when an ssh login file is reloaded ### bug #43518: GNU Parallel doesn't obey servers' jobs capacity when an ssh login file is reloaded
# Pre-20141106 Would reset the number of jobs run on all sshlogin if --slf changed # Pre-20141106 Would reset the number of jobs run on all sshlogin if --slf changed
# Thus must take at least 25 sec to run # Thus must take at least 25 sec to run
echo -e '1/lo\n1/csh@lo\n1/tcsh@lo\n1/parallel@lo\n' > /tmp/parallel.bug43518; parallel --delay 0.1 -N0 echo 1/: '>>' /tmp/parallel.bug43518 ::: {1..100} & seq 30 | stdout /usr/bin/time -f %e parallel --slf /tmp/parallel.bug43518 'sleep {=$_=$_%3?0:10=}.{%}' | perl -ne '$_ > 25 and print "OK\n"' echo -e '1/lo\n1/csh@lo\n1/tcsh@lo\n1/parallel@lo\n' > /tmp/parallel.bug43518; parallel --delay 0.1 -N0 echo 1/: '>>' /tmp/parallel.bug43518 ::: {1..100} & seq 30 | stdout /usr/bin/time -f %e parallel --slf /tmp/parallel.bug43518 'sleep {=$_=$_%3?0:10=}.{%}' | perl -ne '$_ > 25 and print "OK\n"'; rm /tmp/parallel.bug43518
OK OK
echo '### --filter-hosts --slf <()' echo '### --filter-hosts --slf <()'
### --filter-hosts --slf <() ### --filter-hosts --slf <()

7
testsuite/wanted-results/parallel-local-ssh3
View file

@ -93,6 +93,11 @@ echo '### trailing space in sshlogin'
sshlogin trailing space sshlogin trailing space
echo '### Special char file and dir transfer return and cleanup' echo '### Special char file and dir transfer return and cleanup'
### Special char file and dir transfer return and cleanup ### Special char file and dir transfer return and cleanup
cd /tmp; mkdir -p d"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"; echo local > d"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"/f"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"; ssh parallel@lo rm -rf d'*'/; mytouch() { cat d"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"/f"`perl -e 'print pack("c*",1..9,11..46,48..255)'`" > d"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"/g"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"; echo remote OK >> d"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"/g"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"; }; export -f mytouch; parallel --env mytouch -Sparallel@lo --transfer --return d"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"/g"`perl -e 'print pack("c*",1..9,11..46,48..255)'`" mytouch ::: d"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"/f"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"; cat d"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"/g"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"; # Should be changed to --return '{=s:/f:/g:=}' and tested with csh cd /tmp; mkdir -p d"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"; echo local > d"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"/f"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"; ssh parallel@lo rm -rf d'*'/; mytouch() { cat d"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"/f"`perl -e 'print pack("c*",1..9,11..46,48..255)'`" > d"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"/g"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"; echo remote OK >> d"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"/g"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"; }; export -f mytouch; parallel --env mytouch -Sparallel@lo --transfer --return {=s:/f:/g:=} mytouch ::: d"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"/f"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"; cat d"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"/g"`perl -e 'print pack("c*",1..9,11..46,48..255)'`"
local local
remote OK remote OK
echo '### Uniq {=perlexpr=} in return - not used in command'
### Uniq {=perlexpr=} in return - not used in command
cd /tmp; rm -f /tmp/parallel_perlexpr.2Parallel_PerlexPr; echo local > parallel_perlexpr; parallel -Sparallel@lo --trc {=s/pr/pr.2/=}{=s/p/P/g=} echo remote OK '>' {}.2{=s/p/P/g=} ::: parallel_perlexpr; cat /tmp/parallel_perlexpr.2Parallel_PerlexPr; rm -f /tmp/parallel_perlexpr.2Parallel_PerlexPr /tmp/parallel_perlexpr
remote OK
# Should be changed to --return '{=s:/f:/g:=}' and tested with csh

5
testsuite/wanted-results/parallel-local22
View file

@ -43,7 +43,7 @@ echo '### bug #42041: Implement $PARALLEL_JOBSLOT'
2 2
echo '### bug #42363: --pipepart and --fifo/--cat does not work' echo '### bug #42363: --pipepart and --fifo/--cat does not work'
### bug #42363: --pipepart and --fifo/--cat does not work ### bug #42363: --pipepart and --fifo/--cat does not work
seq 100 > /tmp/bug42363; parallel --pipepart --block 31 -a /tmp/bug42363 -k --fifo wc | perl -pe s:/tmp/.........pip:/tmp/XXXX: ; parallel --pipepart --block 31 -a /tmp/bug42363 -k --cat wc | perl -pe s:/tmp/.........pip:/tmp/XXXX: ; echo '### bug #42055: --pipepart -a bigfile should not require sequential reading of bigfile' seq 100 > /tmp/bug42363; parallel --pipepart --block 31 -a /tmp/bug42363 -k --fifo wc | perl -pe s:/tmp/.........pip:/tmp/XXXX: ; parallel --pipepart --block 31 -a /tmp/bug42363 -k --cat wc | perl -pe s:/tmp/.........pip:/tmp/XXXX: ; rm /tmp/bug42363
13 14 32 /tmp/XXXX 13 14 32 /tmp/XXXX
11 11 33 /tmp/XXXX 11 11 33 /tmp/XXXX
11 11 33 /tmp/XXXX 11 11 33 /tmp/XXXX
@ -62,6 +62,7 @@ echo '### bug #42363: --pipepart and --fifo/--cat does not work'
11 11 33 /tmp/XXXX 11 11 33 /tmp/XXXX
11 11 33 /tmp/XXXX 11 11 33 /tmp/XXXX
10 9 29 /tmp/XXXX 10 9 29 /tmp/XXXX
echo '### bug #42055: --pipepart -a bigfile should not require sequential reading of bigfile'
### bug #42055: --pipepart -a bigfile should not require sequential reading of bigfile ### bug #42055: --pipepart -a bigfile should not require sequential reading of bigfile
parallel --pipepart -a /etc/passwd -L 1 should not be run parallel --pipepart -a /etc/passwd -L 1 should not be run
parallel: Error: --pipepart is incompatible with --max-replace-args, --max-lines, and -L. parallel: Error: --pipepart is incompatible with --max-replace-args, --max-lines, and -L.
@ -94,7 +95,7 @@ echo '### bug #42913: Dont use $SHELL but the shell currently running'
### bug #42913: Dont use $SHELL but the shell currently running ### bug #42913: Dont use $SHELL but the shell currently running
echo '## Unknown shell => $SHELL (bash)' echo '## Unknown shell => $SHELL (bash)'
## Unknown shell => $SHELL (bash) ## Unknown shell => $SHELL (bash)
parallel -j1 "cp \`which {}\` /tmp/SHELL; /tmp/SHELL -c 'parallel -Dinit echo ::: 1' | grep which;" ::: ash bash csh dash fdsh fish fizsh ksh ksh93 mksh pdksh posh rbash rush rzsh sash sh static-sh tcsh yash zsh parallel -j1 "cp \`which {}\` /tmp/SHELL; /tmp/SHELL -c 'parallel -Dinit echo ::: 1' | grep which;" ::: ash bash csh dash fdsh fish fizsh ksh ksh93 mksh pdksh posh rbash rush rzsh sash sh static-sh tcsh yash zsh; rm /tmp/SHELL
shell? /bin/bash -c cp `which ash` /tmp/SHELL; /tmp/SHELL -c 'parallel -Dinit echo ::: 1' | grep which; shell? /bin/bash -c cp `which ash` /tmp/SHELL; /tmp/SHELL -c 'parallel -Dinit echo ::: 1' | grep which;
which bash => shell path /bin/bash which bash => shell path /bin/bash
shell? /bin/bash -c cp `which bash` /tmp/SHELL; /tmp/SHELL -c 'parallel -Dinit echo ::: 1' | grep which; shell? /bin/bash -c cp `which bash` /tmp/SHELL; /tmp/SHELL -c 'parallel -Dinit echo ::: 1' | grep which;

4
testsuite/wanted-results/parallel-local9
View file

@ -64,14 +64,14 @@ bug #41412: --timeout + --delay causes deadlock
parallel -j10 --timeout 2.2 --delay 3 "sleep {}; echo {}" ::: 1 2 7 8 9 parallel -j10 --timeout 2.2 --delay 3 "sleep {}; echo {}" ::: 1 2 7 8 9
1 1
2 2
echo '### Test --spreadstdin - more procs than args'; rm -f /tmp/parallel.ss.*; seq 1 5 | stdout $NICEPAR -j 10 --spreadstdin 'cat >/tmp/parallel.ss.$PARALLEL_SEQ' >/dev/null; cat /tmp/parallel.ss.*; echo '### Test --spreadstdin - more procs than args'; rm -f /tmp/parallel.ss.*; seq 1 5 | stdout $NICEPAR -j 10 --spreadstdin 'cat >/tmp/parallel.ss.$PARALLEL_SEQ' >/dev/null; cat /tmp/parallel.ss.*; rm -f /tmp/parallel.ss.*
### Test --spreadstdin - more procs than args ### Test --spreadstdin - more procs than args
1 1
2 2
3 3
4 4
5 5
echo '### Test --spreadstdin - more args than procs'; rm -f /tmp/parallel.ss2.*; seq 1 10 | stdout $NICEPAR -j 5 --spreadstdin 'cat >/tmp/parallel.ss2.$PARALLEL_SEQ' >/dev/null; cat /tmp/parallel.ss2.* echo '### Test --spreadstdin - more args than procs'; rm -f /tmp/parallel.ss2.*; seq 1 10 | stdout $NICEPAR -j 5 --spreadstdin 'cat >/tmp/parallel.ss2.$PARALLEL_SEQ' >/dev/null; cat /tmp/parallel.ss2.*; rm -f /tmp/parallel.ss2.*
### Test --spreadstdin - more args than procs ### Test --spreadstdin - more args than procs
1 1
2 2

6
testsuite/wanted-results/test15
View file

@ -233,7 +233,7 @@ xargs: argument line too long
15 15
echo '### Test -a and --arg-file: Read input from file instead of stdin' echo '### Test -a and --arg-file: Read input from file instead of stdin'
### Test -a and --arg-file: Read input from file instead of stdin ### Test -a and --arg-file: Read input from file instead of stdin
seq 1 10 >/tmp/$$-1; parallel -k -a /tmp/$$-1 echo seq 1 10 >/tmp/parallel_$$-1; parallel -k -a /tmp/parallel_$$-1 echo; rm /tmp/parallel_$$-1
1 1
2 2
3 3
@ -244,7 +244,7 @@ seq 1 10 >/tmp/$$-1; parallel -k -a /tmp/$$-1 echo
8 8
9 9
10 10
seq 1 10 >/tmp/$$-2; parallel -k --arg-file /tmp/$$-2 echo seq 1 10 >/tmp/parallel_$$-2; parallel -k --arg-file /tmp/parallel_$$-2 echo; rm /tmp/parallel_$$-2
1 1
2 2
3 3
@ -844,7 +844,7 @@ a b
echo '### Test --nice remote' echo '### Test --nice remote'
### Test --nice remote ### Test --nice remote
stdout parallel --nice 1 -S .. -vv 'PAR=a bash -c "echo \$PAR {}"' ::: b | perl -pe 's/\S*parallel-server\S*/one-server/;s/[a-f0-9]{500,}/hex/;s/\d{5,8}/pId/g;' stdout parallel --nice 1 -S .. -vv 'PAR=a bash -c "echo \$PAR {}"' ::: b | perl -pe 's/\S*parallel-server\S*/one-server/;s/[a-f0-9]{500,}/hex/;s/\d{5,8}/pId/g;'
ssh one-server exec perl -e \\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"pId\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"1\\\"\\\;\\\$bashfunc\\\ =\\\ \\\"\\\"\\\;@ARGV=\\\"\\\\\\\\nice\\\ -n1\\\ /bin/bash\\\ -c\\\ PAR=a\\\\\\\\\\\ bash\\\\\\\\\\\ -c\\\\\\\\\\\ \\\\\\\\\\\\\\\"echo\\\\\\\\\\\ \\\\\\\\\\\ \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\$PAR\\\\\\\\\\\ b\\\\\\\\\\\\\\\"\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\); ssh one-server exec perl\ -e\ \\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"pId\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"1\\\"\\\;\\\$bashfunc\\\ =\\\ \\\"\\\"\\\;@ARGV=\\\"\\\\\\\\nice\\\ -n1\\\ /bin/bash\\\ -c\\\ PAR=a\\\\\\\\\\\ bash\\\\\\\\\\\ -c\\\\\\\\\\\ \\\\\\\\\\\\\\\"echo\\\\\\\\\\\ \\\\\\\\\\\ \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\$PAR\\\\\\\\\\\ b\\\\\\\\\\\\\\\"\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\);
a b a b
echo '### Test distribute arguments at EOF to 2 jobslots' echo '### Test distribute arguments at EOF to 2 jobslots'
### Test distribute arguments at EOF to 2 jobslots ### Test distribute arguments at EOF to 2 jobslots

4
testsuite/wanted-results/test19
View file

@ -77,7 +77,7 @@ OK
Input for ssh Input for ssh
one-server mkdir -p ./. one-server mkdir -p ./.
-l parallel one-server rsync --server -lDrRze.iLsfx . ./. -l parallel one-server rsync --server -lDrRze.iLsfx . ./.
one-server exec perl -e @GNU_Parallel=\(\"use\",\"IPC::Open3\;\",\"use\",\"MIME::Base64\"\)\;eval\"@GNU_Parallel\"\;\$SIG\{CHLD\}=\"IGNORE\"\;my\$zip=\(grep\{-x\$_\}\"/usr/local/bin/bzip2\"\)\[0\]\|\|\"bzip2\"\;my\(\$in,\$out,\$eval\)\;open3\(\$in,\$out,\"\>\&STDERR\",\$zip,\"-dc\"\)\;if\(my\$perlpid=fork\)\{close\$in\;\$eval=join\"\",\<\$out\>\;close\$out\;\}else\{close\$out\;print\$in\(decode_base64\(join\"\",@ARGV\)\)\;close\$in\;exit\;\}wait\;eval\$eval\; base64 one-server exec perl -e '@GNU_Parallel=("use","IPC::Open3;","use","MIME::Base64");eval"@GNU_Parallel";$SIG{CHLD}="IGNORE";my$zip=(grep{-x$_}"/usr/local/bin/bzip2")[0]||"bzip2";my($in,$out,$eval);open3($in,$out,">&STDERR",$zip,"-dc");if(my$perlpid=fork){close$in;$eval=join"",<$out>;close$out;}else{close$out;print$in(decode_base64(join"",@ARGV));close$in;exit;}wait;eval$eval;' base64
-l parallel one-server cd ././tmp; rsync --server --sender -lDrRze.iLsfx . ./parallel.file.' -l parallel one-server cd ././tmp; rsync --server --sender -lDrRze.iLsfx . ./parallel.file.'
'newlineX.out 'newlineX.out
-l parallel one-server cd ././tmp; rsync --server --sender -lDrRze.iLsfx . ./parallel.file.' -l parallel one-server cd ././tmp; rsync --server --sender -lDrRze.iLsfx . ./parallel.file.'
@ -90,7 +90,7 @@ one-server (rm -f ./tmp/parallel.file.'
'newlineX.out2; rmdir ./tmp/ ./ 2>/dev/null;) 'newlineX.out2; rmdir ./tmp/ ./ 2>/dev/null;)
one-server mkdir -p ./. one-server mkdir -p ./.
-l parallel one-server rsync --server -lDrRze.iLsfx . ./. -l parallel one-server rsync --server -lDrRze.iLsfx . ./.
one-server exec perl -e @GNU_Parallel=\(\"use\",\"IPC::Open3\;\",\"use\",\"MIME::Base64\"\)\;eval\"@GNU_Parallel\"\;\$SIG\{CHLD\}=\"IGNORE\"\;my\$zip=\(grep\{-x\$_\}\"/usr/local/bin/bzip2\"\)\[0\]\|\|\"bzip2\"\;my\(\$in,\$out,\$eval\)\;open3\(\$in,\$out,\"\>\&STDERR\",\$zip,\"-dc\"\)\;if\(my\$perlpid=fork\)\{close\$in\;\$eval=join\"\",\<\$out\>\;close\$out\;\}else\{close\$out\;print\$in\(decode_base64\(join\"\",@ARGV\)\)\;close\$in\;exit\;\}wait\;eval\$eval\; base64 one-server exec perl -e '@GNU_Parallel=("use","IPC::Open3;","use","MIME::Base64");eval"@GNU_Parallel";$SIG{CHLD}="IGNORE";my$zip=(grep{-x$_}"/usr/local/bin/bzip2")[0]||"bzip2";my($in,$out,$eval);open3($in,$out,">&STDERR",$zip,"-dc");if(my$perlpid=fork){close$in;$eval=join"",<$out>;close$out;}else{close$out;print$in(decode_base64(join"",@ARGV));close$in;exit;}wait;eval$eval;' base64
-l parallel one-server cd ././tmp; rsync --server --sender -lDrRze.iLsfx . ./parallel.file.' -l parallel one-server cd ././tmp; rsync --server --sender -lDrRze.iLsfx . ./parallel.file.'
'newlineX.out 'newlineX.out
-l parallel one-server cd ././tmp; rsync --server --sender -lDrRze.iLsfx . ./parallel.file.' -l parallel one-server cd ././tmp; rsync --server --sender -lDrRze.iLsfx . ./parallel.file.'

4
testsuite/wanted-results/test45
View file

@ -27,11 +27,11 @@ echo '### Test --load remote'
OK OK
echo '### Test --load read from a file - more than 3s' echo '### Test --load read from a file - more than 3s'
### Test --load read from a file - more than 3s ### Test --load read from a file - more than 3s
echo '# This will run 10 processes in parallel for 10s'; seq 10 | parallel --nice 19 --timeout 10 -j0 -N0 "gzip < /dev/zero > /dev/null" & ( echo 8 > /tmp/parallel_load_file; sleep 10; echo 1000 > /tmp/parallel_load_file ) & sleep 1;stdout /usr/bin/time -f %e parallel --load /tmp/parallel_load_file sleep ::: 1 | perl -ne '$_ > 9 and print "OK\n"' echo '# This will run 10 processes in parallel for 10s'; seq 10 | parallel --nice 19 --timeout 10 -j0 -N0 "gzip < /dev/zero > /dev/null" & ( echo 8 > /tmp/parallel_load_file; sleep 10; echo 1000 > /tmp/parallel_load_file ) & sleep 1; stdout /usr/bin/time -f %e parallel --load /tmp/parallel_load_file sleep ::: 1 | perl -ne '$_ > 9 and print "OK\n"'; rm /tmp/parallel_load_file
# This will run 10 processes in parallel for 10s # This will run 10 processes in parallel for 10s
OK OK
echo '### Test --load read from a file - less than 10s'; echo '# This will run 10 processes in parallel for 10s'; seq 10 | parallel --nice 19 --timeout 10 -j0 -N0 "gzip < /dev/zero > /dev/null" & echo '### Test --load read from a file - less than 10s'; echo '# This will run 10 processes in parallel for 10s'; seq 10 | parallel --nice 19 --timeout 10 -j0 -N0 "gzip < /dev/zero > /dev/null" &
### Test --load read from a file - less than 10s ### Test --load read from a file - less than 10s
# This will run 10 processes in parallel for 10s # This will run 10 processes in parallel for 10s
( echo 8 > /tmp/parallel_load_file2; sleep 10; echo 1000 > /tmp/parallel_load_file2 ) & sleep 1;stdout /usr/bin/time -f %e parallel --load /tmp/parallel_load_file2 sleep ::: 1 | perl -ne '$_ < 20 and print "OK\n"' ( echo 8 > /tmp/parallel_load_file2; sleep 10; echo 1000 > /tmp/parallel_load_file2 ) & sleep 1; stdout /usr/bin/time -f %e parallel --load /tmp/parallel_load_file2 sleep ::: 1 | perl -ne '$_ < 20 and print "OK\n"'; rm /tmp/parallel_load_file2
OK OK

8
testsuite/wanted-results/test61
View file

@ -1,15 +1,15 @@
echo '### Test --return of weirdly named file' echo '### Test --return of weirdly named file'
### Test --return of weirdly named file ### Test --return of weirdly named file
stdout parallel --return {} -vv -S parallel\@parallel-server3 echo '>'{} ::: 'aa<${#}" b' | perl -pe 's/\S*parallel-server\S*/one-server/;s/[a-f0-9]{500,}/hex/;s/\d{5,8}/pId/g;'; rm 'aa<${#}" b' stdout parallel --return {} -vv -S parallel\@parallel-server3 echo '>'{} ::: 'aa<${#}" b' | perl -pe 's/\S*parallel-server\S*/one-server/;s/[a-f0-9]{500,}/hex/;s/\d{5,8}/pId/g;'; rm 'aa<${#}" b'
ssh one-server exec perl -e \\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"pId\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"1\\\"\\\;\\\$bashfunc\\\ =\\\ \\\"\\\"\\\;@ARGV=\\\"echo\\\ \\\>aa\\\\\\\\\\\<\\\\\\\\\\\\\\\$\\\\\\\\\\\{\\\\\\\\\\\#\\\\\\\\\\\}\\\\\\\\\\\\\\\"\\\\\\\\\\\ b\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\);_EXIT_status=$?; mkdir -p ./.; rsync --protocol 30 --rsync-path=cd\ ././.\;\ rsync -rlDzR -essh parallel@parallel-server3:./aa\\\<\\\$\\\{\\\#\\\}\\\"\\\ b ./.; exit $_EXIT_status; ssh one-server exec perl\ -e\ \\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"pId\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"1\\\"\\\;\\\$bashfunc\\\ =\\\ \\\"\\\"\\\;@ARGV=\\\"echo\\\ \\\>aa\\\\\\\\\\\<\\\\\\\\\\\\\\\$\\\\\\\\\\\{\\\\\\\\\\\#\\\\\\\\\\\}\\\\\\\\\\\\\\\"\\\\\\\\\\\ b\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\);_EXIT_status=$?; mkdir -p ./.; rsync --protocol 30 --rsync-path=cd\ ././.\;\ rsync -rlDzR -essh parallel@parallel-server3:./aa\\\<\\\$\\\{\\\#\\\}\\\"\\\ b ./.; exit $_EXIT_status;
echo '### Test if remote login shell is csh' echo '### Test if remote login shell is csh'
### Test if remote login shell is csh ### Test if remote login shell is csh
stdout parallel -k -vv -S csh@localhost 'echo $PARALLEL_PID $PARALLEL_SEQ {}| wc -w' ::: a b c | perl -pe 's/\S*parallel-server\S*/one-server/;s/[a-f0-9]{500,}/hex/;s/\d{5,8}/pId/g;' stdout parallel -k -vv -S csh@localhost 'echo $PARALLEL_PID $PARALLEL_SEQ {}| wc -w' ::: a b c | perl -pe 's/\S*parallel-server\S*/one-server/;s/[a-f0-9]{500,}/hex/;s/\d{5,8}/pId/g;'
ssh csh@localhost exec perl -e \\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"pId\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"1\\\"\\\;\\\$bashfunc\\\ =\\\ \\\"\\\"\\\;@ARGV=\\\"echo\\\ \\\\\\\$PARALLEL_PID\\\ \\\\\\\$PARALLEL_SEQ\\\ a\\\|\\\ wc\\\ -w\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\); ssh csh@localhost exec perl\ -e\ \\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"pId\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"1\\\"\\\;\\\$bashfunc\\\ =\\\ \\\"\\\"\\\;@ARGV=\\\"echo\\\ \\\\\\\$PARALLEL_PID\\\ \\\\\\\$PARALLEL_SEQ\\\ a\\\|\\\ wc\\\ -w\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\);
3 3
ssh csh@localhost exec perl -e \\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"pId\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"2\\\"\\\;\\\$bashfunc\\\ =\\\ \\\"\\\"\\\;@ARGV=\\\"echo\\\ \\\\\\\$PARALLEL_PID\\\ \\\\\\\$PARALLEL_SEQ\\\ b\\\|\\\ wc\\\ -w\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\); ssh csh@localhost exec perl\ -e\ \\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"pId\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"2\\\"\\\;\\\$bashfunc\\\ =\\\ \\\"\\\"\\\;@ARGV=\\\"echo\\\ \\\\\\\$PARALLEL_PID\\\ \\\\\\\$PARALLEL_SEQ\\\ b\\\|\\\ wc\\\ -w\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\);
3 3
ssh csh@localhost exec perl -e \\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"pId\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"3\\\"\\\;\\\$bashfunc\\\ =\\\ \\\"\\\"\\\;@ARGV=\\\"echo\\\ \\\\\\\$PARALLEL_PID\\\ \\\\\\\$PARALLEL_SEQ\\\ c\\\|\\\ wc\\\ -w\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\); ssh csh@localhost exec perl\ -e\ \\\$ENV\\\{\\\"PARALLEL_PID\\\"\\\}=\\\"pId\\\"\\\;\\\$ENV\\\{\\\"PARALLEL_SEQ\\\"\\\}=\\\"3\\\"\\\;\\\$bashfunc\\\ =\\\ \\\"\\\"\\\;@ARGV=\\\"echo\\\ \\\\\\\$PARALLEL_PID\\\ \\\\\\\$PARALLEL_SEQ\\\ c\\\|\\\ wc\\\ -w\\\"\\\;\\\$SIG\\\{CHLD\\\}=sub\\\{\\\$done=1\\\;\\\}\\\;\\\$pid=fork\\\;unless\\\(\\\$pid\\\)\\\{setpgrp\\\;exec\\\$ENV\\\{SHELL\\\},\\\"-c\\\",\\\(\\\$bashfunc.\\\"@ARGV\\\"\\\)\\\;die\\\"exec:\\\$\\\!\\\\n\\\"\\\;\\\}do\\\{\\\$s=\\\$s\\\<1\\\?0.001+\\\$s\\\*1.03:\\\$s\\\;select\\\(undef,undef,undef,\\\$s\\\)\\\;\\\}until\\\(\\\$done\\\|\\\|getppid==1\\\)\\\;kill\\\(SIGHUP,-\\\$\\\{pid\\\}\\\)unless\\\$done\\\;wait\\\;exit\\\(\\\$\\\?\\\&127\\\?128+\\\(\\\$\\\?\\\&127\\\):1+\\\$\\\?\\\>\\\>8\\\);
3 3
echo '### Test {} multiple times in different commands' echo '### Test {} multiple times in different commands'
### Test {} multiple times in different commands ### Test {} multiple times in different commands