tange/parallel
1
0
Fork 0
mirror of https://git.savannah.gnu.org/git/parallel.git synced 2024-11-25 23:47:53 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

parallel: Make tmpname() not create a file. That can cause permission problems if you ssh to a shared filesystem as a different user.

Browse source
This commit is contained in:
Ole Tange 2015-04-23 00:15:23 +02:00
parent bbedf716c1
commit 4e3e0867b9
16 changed files with 133 additions and 170 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

12
README
View file

@ -40,9 +40,9 @@ document.
Full installation of GNU Parallel is as simple as: Full installation of GNU Parallel is as simple as:
wget http://ftpmirror.gnu.org/parallel/parallel-20150322.tar.bz2 wget http://ftpmirror.gnu.org/parallel/parallel-20150422.tar.bz2
bzip2 -dc parallel-20150322.tar.bz2 | tar xvf - bzip2 -dc parallel-20150422.tar.bz2 | tar xvf -
cd parallel-20150322 cd parallel-20150422
./configure && make && make install ./configure && make && make install
@ -51,9 +51,9 @@ Full installation of GNU Parallel is as simple as:
If you are not root you can add ~/bin to your path and install in If you are not root you can add ~/bin to your path and install in
~/bin and ~/share: ~/bin and ~/share:
wget http://ftpmirror.gnu.org/parallel/parallel-20150322.tar.bz2 wget http://ftpmirror.gnu.org/parallel/parallel-20150422.tar.bz2
bzip2 -dc parallel-20150322.tar.bz2 | tar xvf - bzip2 -dc parallel-20150422.tar.bz2 | tar xvf -
cd parallel-20150322 cd parallel-20150422
./configure --prefix=$HOME && make && make install ./configure --prefix=$HOME && make && make install
Or if your system lacks 'make' you can simply copy src/parallel Or if your system lacks 'make' you can simply copy src/parallel

2
configure.ac
View file

@ -1,4 +1,4 @@
AC_INIT([parallel], [20150322], [bug-parallel@gnu.org]) AC_INIT([parallel], [20150422], [bug-parallel@gnu.org])
AM_INIT_AUTOMAKE([-Wall -Werror foreign]) AM_INIT_AUTOMAKE([-Wall -Werror foreign])
AC_CONFIG_HEADERS([config.h]) AC_CONFIG_HEADERS([config.h])
AC_CONFIG_FILES([ AC_CONFIG_FILES([

10
doc/release_new_version
View file

@ -222,11 +222,7 @@ Haiku of the month:
New in this release: New in this release:
* Security fix. An attacker on the local system could make you overwrite one of your own files with a single byte. The problem requires: * Security fix. An attacker on the local system could make you overwrite one of your own files with a single byte. The problem exist when you use --compress or --tmux or --pipe or --cat or --fifo. The attacker must figure out the randomly chosen file name and create a symlink within a time window of 15 ms.
- you using --compress or --tmux
- the attacker must figure out the randomly chosen file name
- the attacker to create a symlink within a time window of 15 ms
* GNU Parallel now has a DOI: https://dx.doi.org/10.5281/zenodo.16303 * GNU Parallel now has a DOI: https://dx.doi.org/10.5281/zenodo.16303
@ -248,6 +244,8 @@ New in this release:
taxator-tk http://algbio.cs.uni-duesseldorf.de/webapps/wa-download/ (check it) taxator-tk http://algbio.cs.uni-duesseldorf.de/webapps/wa-download/ (check it)
* << afventer svar fra Rachel >> GNU Parallel was used in: SISRS: Site Identification from Short Read Sequences https://github.com/rachelss/SISRS/
* GNU Parallel was used in: landsat-gifworks https://github.com/KAPPS-/landsat-gifworks * GNU Parallel was used in: landsat-gifworks https://github.com/KAPPS-/landsat-gifworks
* GNU Parallel was used in: https://github.com/exascience/elprep * GNU Parallel was used in: https://github.com/exascience/elprep
@ -258,8 +256,6 @@ taxator-tk http://algbio.cs.uni-duesseldorf.de/webapps/wa-download/ (check it)
* GNU Parallel was used in: Rapid haploid SNP calling https://github.com/tseemann/snippy * GNU Parallel was used in: Rapid haploid SNP calling https://github.com/tseemann/snippy
* << afventer svar fra Rachel >> GNU Parallel was used in: SISRS: Site Identification from Short Read Sequences https://github.com/rachelss/SISRS/
* Pictures and Metadata http://www.ozzy.no/2015/02/05/pictures-and-metadata/ * Pictures and Metadata http://www.ozzy.no/2015/02/05/pictures-and-metadata/
* Task automation with bash and parallel https://biowize.wordpress.com/2015/03/23/task-automation-with-bash-and-parallel/ * Task automation with bash and parallel https://biowize.wordpress.com/2015/03/23/task-automation-with-bash-and-parallel/

2
src/niceload
View file

@ -24,7 +24,7 @@
use strict; use strict;
use Getopt::Long; use Getopt::Long;
$Global::progname="niceload"; $Global::progname="niceload";
$Global::version = 20150322; $Global::version = 20150422;
Getopt::Long::Configure("bundling","require_order"); Getopt::Long::Configure("bundling","require_order");
get_options_from_array(\@ARGV) || die_usage(); get_options_from_array(\@ARGV) || die_usage();
if($opt::version) { if($opt::version) {

55
src/parallel
View file

@ -1053,7 +1053,7 @@ sub parse_options {
sub init_globals { sub init_globals {
# Defaults: # Defaults:
$Global::version = 20150419; $Global::version = 20150422;
$Global::progname = 'parallel'; $Global::progname = 'parallel';
$Global::infinity = 2**31; $Global::infinity = 2**31;
$Global::debug = 0; $Global::debug = 0;
@ -3341,10 +3341,6 @@ sub tmpfile {
return ::tempfile(DIR=>$ENV{'TMPDIR'}, TEMPLATE => 'parXXXXX', @_); return ::tempfile(DIR=>$ENV{'TMPDIR'}, TEMPLATE => 'parXXXXX', @_);
} }
{
my($tmpdir,$fifo);
sub tmpname { sub tmpname {
# Select a name that does not exist # Select a name that does not exist
# Do not create the file as that may cause problems # Do not create the file as that may cause problems
@ -3352,34 +3348,20 @@ sub tmpfile {
my $name = shift; my $name = shift;
my($tmpname); my($tmpname);
do { do {
$tmpname = $ENV{'TMPDIR'}."/".$name.int(rand(9999)); $tmpname = $ENV{'TMPDIR'}."/".$name.
join"", map { (0..9,"a".."z","A".."Z")[rand(62)] } (1..5);
} while($Global::unlink{$tmpname}++ or -e $tmpname); } while($Global::unlink{$tmpname}++ or -e $tmpname);
# $Global::unlink{$tmpdir} ||= 1;
return $tmpname; return $tmpname;
} }
sub rmtmpname {
# Clean up mktmpname
my $tmpname = shift;
delete $Global::unlink{$tmpname};
unlink $tmpname;
rmdir $tmpdir;
}
sub tmpfifo { sub tmpfifo {
# Securely make a fifo by securely making a dir with a fifo in it # Securely make a fifo by securely making a dir with a fifo in it
use POSIX qw(mkfifo); use POSIX qw(mkfifo);
my $tmpfifo = tmpname("fifo",@_); my $tmpfifo = tmpname("fif",@_);
mkfifo($tmpfifo,0600); mkfifo($tmpfifo,0600);
return $tmpfifo; return $tmpfifo;
} }
sub rmtmpfifo {
# Clean up mktmpfifo
rmtmpname(@_);
}
}
sub uniq { sub uniq {
# Remove duplicates and return unique values # Remove duplicates and return unique values
return keys %{{ map { $_ => 1 } @_ }}; return keys %{{ map { $_ => 1 } @_ }};
@ -5554,7 +5536,7 @@ sub total_jobs {
my $start = time; my $start = time;
while($job = $self->get()) { while($job = $self->get()) {
if(time - $start > 10) { if(time - $start > 10) {
::warning("Reading all arguments takes longer than 10 seconds.\n"); ::warning("Reading ".scalar(@queue)." arguments took longer than 10 seconds.\n");
$opt::eta && ::warning("Consider removing --eta.\n"); $opt::eta && ::warning("Consider removing --eta.\n");
$opt::bar && ::warning("Consider removing --bar.\n"); $opt::bar && ::warning("Consider removing --bar.\n");
$opt::shuf && ::warning("Consider removing --shuf.\n"); $opt::shuf && ::warning("Consider removing --shuf.\n");
@ -7001,10 +6983,7 @@ sub print_dryrun_and_verbose {
my $tmux; my $tmux;
$ENV{'TMUX'} ||= "tmux"; $ENV{'TMUX'} ||= "tmux";
if(not $tmuxsocket) { if(not $tmuxsocket) {
# $tmuxsocket = ::tmpname("tmux"); $tmuxsocket = ::tmpname("tms");
(undef, $tmuxsocket) = ::tmpfile(SUFFIX => ".tms");
$Global::unlink{$tmuxsocket} = 1;
unlink $tmuxsocket;
::status("See output with: $ENV{'TMUX'} -S $tmuxsocket attach\n"); ::status("See output with: $ENV{'TMUX'} -S $tmuxsocket attach\n");
} }
# TODO sh -c wrapper for >& # TODO sh -c wrapper for >&
@ -7595,7 +7574,7 @@ sub populate {
if($opt::fifo) { if($opt::fifo) {
# Generate a tempfile name that will be used as {} # Generate a tempfile name that will be used as {}
$Global::JobQueue->{'commandlinequeue'}->{'arg_queue'}-> $Global::JobQueue->{'commandlinequeue'}->{'arg_queue'}->
unget([Arg->new(::tmpname("fifo"))]); unget([Arg->new(::tmpname("fif"))]);
} }
while (not $self->{'arg_queue'}->empty()) { while (not $self->{'arg_queue'}->empty()) {
$next_arg = $self->{'arg_queue'}->get(); $next_arg = $self->{'arg_queue'}->get();
@ -8324,22 +8303,6 @@ sub quote_args {
return $self->{'command'}; return $self->{'command'};
} }
sub size {
my $self = shift;
if(not $self->{'size'}) {
my @all_lines = ();
while(not $self->{'arg_queue'}->empty()) {
push @all_lines, CommandLine->new($self->{'command'},
$self->{'arg_queue'},
$self->{'context_replace'},
$self->{'max_number_of_args'});
}
$self->{'size'} = @all_lines;
$self->unget(@all_lines);
}
return $self->{'size'};
}
package Limits::Command; package Limits::Command;
@ -8440,9 +8403,7 @@ sub tmux_length {
} }
my @out; my @out;
for my $l (1, 2020, 16320, 100000, $len) { for my $l (1, 2020, 16320, 100000, $len) {
my ($fh, $tmpfile) = ::tmpfile(SUFFIX => ".tmb"); my $tmpfile = ::tmpname("tms");
close $fh;
unlink $tmpfile;
my $tmuxcmd = "sh -c '".$ENV{'TMUX'}." -S $tmpfile new-session -d -n echo $l". my $tmuxcmd = "sh -c '".$ENV{'TMUX'}." -S $tmpfile new-session -d -n echo $l".
("x"x$l). " 2>/dev/null' && echo $l; rm -f $tmpfile"; ("x"x$l). " 2>/dev/null' && echo $l; rm -f $tmpfile";
push @out, qx{ $tmuxcmd }; push @out, qx{ $tmuxcmd };

11
src/parallel_design.pod
View file

@ -231,10 +231,15 @@ are added to the title to force it to be outside the limits.
You can map the bad limits using: You can map the bad limits using:
perl -e 'map { $a=$_; print map { "$a,$_\n" } (1..17000) } (1..17000)' | shuf > ab;
cat ab | parallel --colsep , --tagstring '{1}{=$_="\t"=}{2}' perl -e 'sub r { int(rand(shift)).($_[0] && "\t".r(@_)) } print map { r(@ARGV)."\n" } 1..10000' 1600 1500 90 |
tmux -S /tmp/p{%}-'{=1 $_="O"x(rand(200)) =}' new-session -d -n '{=1 $_="O"x$_ =}' true'\ {=2 $_="O"x$_ =};echo $?;rm /tmp/p{%}-O*' perl -ane '$F[0]+$F[1]+$F[2] < 2037 and print ' |
parallel --colsep '\t' --tagstring '{1}\t{2}\t{3}' tmux -S /tmp/p{%}-'{=3 $_="O"x$_ =}' \
new-session -d -n '{=1 $_="O"x$_ =}' true'\ {=2 $_="O"x$_ =};echo $?;rm -f /tmp/p{%}-O*'
perl -e 'sub r { int(rand(shift)).($_[0] && "\t".r(@_)) } print map { r(@ARGV)."\n" } 1..10000' 17000 17000 90 |
parallel --colsep '\t' --tagstring '{1}\t{2}\t{3}' \
tmux -S /tmp/p{%}-'{=3 $_="O"x$_ =}' new-session -d -n '{=1 $_="O"x$_ =}' true'\ {=2 $_="O"x$_ =};echo $?;rm /tmp/p{%}-O*'
> value.csv 2>/dev/null > value.csv 2>/dev/null
R -e 'a<-read.table("value.csv");X11();plot(a[,1],a[,2],col=a[,3]+5,cex=0.1);Sys.sleep(1000)' R -e 'a<-read.table("value.csv");X11();plot(a[,1],a[,2],col=a[,3]+5,cex=0.1);Sys.sleep(1000)'

2
src/sql
View file

@ -566,7 +566,7 @@ $Global::Initfile && unlink $Global::Initfile;
exit ($err); exit ($err);
sub parse_options { sub parse_options {
$Global::version = 20150322; $Global::version = 20150422;
$Global::progname = 'sql'; $Global::progname = 'sql';
# This must be done first as this may exec myself # This must be done first as this may exec myself

4
testsuite/tests-to-run/parallel-local-ssh2.sh
View file

@ -45,8 +45,8 @@ echo '2bug #43358: shellshock breaks exporting functions using --env'
echo '### bug #42999: --pipepart with remote does not work' echo '### bug #42999: --pipepart with remote does not work'
seq 100 > /tmp/bug42999; chmod 600 /tmp/bug42999; seq 100 > /tmp/bug42999; chmod 600 /tmp/bug42999;
parallel --sshdelay 0.3 --pipepart --block 31 -a /tmp/bug42999 -k -S parallel@lo wc; parallel --sshdelay 0.3 --pipepart --block 31 -a /tmp/bug42999 -k -S parallel@lo wc;
parallel --sshdelay 0.2 --pipepart --block 31 -a /tmp/bug42999 -k --fifo -S parallel@lo wc | perl -pe 's:(/tmp\S+fifo)\d+:${1}0000:' ; parallel --sshdelay 0.2 --pipepart --block 31 -a /tmp/bug42999 -k --fifo -S parallel@lo wc | perl -pe 's:(/tmp\S+fif)\S+:${1}XXXXX:' ;
parallel --sshdelay 0.1 --pipepart --block 31 -a /tmp/bug42999 -k --cat -S parallel@lo wc | perl -pe 's:(/tmp\S+cat)\d+:${1}0000:' ; parallel --sshdelay 0.1 --pipepart --block 31 -a /tmp/bug42999 -k --cat -S parallel@lo wc | perl -pe 's:(/tmp\S+cat)\S+:${1}XXXXX:' ;
rm /tmp/bug42999 rm /tmp/bug42999
echo '### --cat gives incorrect exit value in csh' echo '### --cat gives incorrect exit value in csh'

6
testsuite/tests-to-run/parallel-local22.sh
View file

@ -46,8 +46,8 @@ echo '### bug #42041: Implement $PARALLEL_JOBSLOT'
echo '### bug #42363: --pipepart and --fifo/--cat does not work' echo '### bug #42363: --pipepart and --fifo/--cat does not work'
seq 100 > /tmp/bug42363; seq 100 > /tmp/bug42363;
parallel --pipepart --block 31 -a /tmp/bug42363 -k --fifo wc | perl -pe 's:(/tmp\S+fifo)\d+:${1}0000:'; parallel --pipepart --block 31 -a /tmp/bug42363 -k --fifo wc | perl -pe 's:(/tmp\S+fif).....:${1}XXXXX:';
parallel --pipepart --block 31 -a /tmp/bug42363 -k --cat wc | perl -pe 's:(/tmp\S+cat)\d+:${1}0000:'; parallel --pipepart --block 31 -a /tmp/bug42363 -k --cat wc | perl -pe 's:(/tmp\S+cat).....:${1}XXXXX:';
rm /tmp/bug42363 rm /tmp/bug42363
echo '### bug #42055: --pipepart -a bigfile should not require sequential reading of bigfile' echo '### bug #42055: --pipepart -a bigfile should not require sequential reading of bigfile'
@ -120,7 +120,7 @@ echo '### added transfersize/returnsize to local jobs'
echo '### --tmux test - check termination' echo '### --tmux test - check termination'
perl -e 'map {printf "$_%o%c\n",$_,$_}1..255' | perl -e 'map {printf "$_%o%c\n",$_,$_}1..255' |
stdout parallel --tmux echo {} :::: - ::: a b | stdout parallel --tmux echo {} :::: - ::: a b |
perl -pe 's:(/tmp\S+par)......tms:$1XXXXX.tms:;' perl -pe 's:(/tmp\S+/tms).....:$1XXXXX:;'
EOF EOF

7
testsuite/tests-to-run/parallel-local7.sh
View file

@ -1,7 +1,8 @@
#!/bin/bash #!/bin/bash
par_tmux_filter() { par_tmux_filter() {
perl -pe 's/par......tms/parXXXXX.tms/;s/ p\d+/pID/;' # /tmp/parallel-local7/tmsOU2Ig
perl -pe 's:(/tmp\S+/tms).....:$1XXXXX:;s/ p\d+/pID/;'
} }
export -f par_tmux_filter export -f par_tmux_filter
@ -9,7 +10,7 @@ par_tmux() {
(stdout parallel --timeout 3 --tmux --delay .3 echo '{}{=$_="\\"x$_=}'; echo $?) | par_tmux_filter (stdout parallel --timeout 3 --tmux --delay .3 echo '{}{=$_="\\"x$_=}'; echo $?) | par_tmux_filter
} }
export -f par_tmux export -f par_tmux
cat <<'EOF' | sed -e 's/;$/; /;s/$SERVER1/'$SERVER1'/;s/$SERVER2/'$SERVER2'/' | stdout parallel -vj0 --retries 2 -k --joblog /tmp/jl-`basename $0` -L1 cat <<'EOF' | sed -e 's/;$/; /;s/$SERVER1/'$SERVER1'/;s/$SERVER2/'$SERVER2'/' | stdout parallel -vj0 --timeout 60 --retries 2 -k --joblog /tmp/jl-`basename $0` -L1
echo '### tmux1.9' echo '### tmux1.9'
seq 000 100 | TMUX=tmux1.9 par_tmux seq 000 100 | TMUX=tmux1.9 par_tmux
seq 100 200 | TMUX=tmux1.9 par_tmux seq 100 200 | TMUX=tmux1.9 par_tmux
@ -72,4 +73,4 @@ echo '### Test critical lengths. Must not block'
EOF EOF
rm -f /tmp/paralocal7*; rm -f /tmp/paralocal7*

9
testsuite/tests-to-run/parallel-tutorial.sh
View file

@ -1,5 +1,6 @@
#!/bin/bash #!/bin/bash
find {$TMPDIR,/var/tmp,/tmp}/{fif,tms,par[^a]}* -mmin -10 2>/dev/null | parallel rm
cd testsuite 2>/dev/null cd testsuite 2>/dev/null
rm -rf tmp rm -rf tmp
mkdir tmp mkdir tmp
@ -23,9 +24,9 @@ perl -ne '$/="\n\n"; /^Output/../^[^O]\S/ and next; /^ / and print;' ../../src/
stdout parallel -j7 -vd'\n\n' | stdout parallel -j7 -vd'\n\n' |
perl -pe '$|=1; perl -pe '$|=1;
# --tmux # --tmux
s:(/tmp\S+par).....(/tms):$1XXXXX$2:; s:(/tmp\S+)(/tms).....:$1XXXXX$2:;
# --files # --files
s:(/tmp\S+par).....(....):$1XXXXX$2:; s:(/tmp\S+par).....(\....):$1XXXXX$2:;
# --eta --progress # --eta --progress
s/ETA.*//g; s/local:.*//g; s/ETA.*//g; s/local:.*//g;
# Sat Apr 4 11:55:40 CEST 2015 # Sat Apr 4 11:55:40 CEST 2015
@ -54,5 +55,5 @@ perl -ne '$/="\n\n"; /^Output/../^[^O]\S/ and next; /^ / and print;' ../../src/
s/tried 2\n//; s/tried 2\n//;
' '
# 3+3 .par files (from --files), 1 .tms-file from tmux attach # 3+3 .par files (from --files), 1 .tms-file from tmux attach
ls /tmp/par*.par /var/tmp/par*.par /tmp/*.tms /tmp/*.tmx 2>/dev/null | wc -l find {$TMPDIR,/var/tmp,/tmp}/{fif,tms,par[^a]}* -mmin -10 2>/dev/null | wc -l
find /tmp/par*.par /var/tmp/par*.par /tmp/*.tms /tmp/*.tmx -mmin -10 2>/dev/null | parallel rm find {$TMPDIR,/var/tmp,/tmp}/{fif,tms,par[^a]}* -mmin -10 2>/dev/null | parallel rm

38
testsuite/wanted-results/parallel-local-ssh2
View file

@ -44,7 +44,7 @@ shellshock-hardened to non-shellshock-hardened
Function non-shellshock-hardened Function non-shellshock-hardened
echo '### bug #42999: --pipepart with remote does not work' echo '### bug #42999: --pipepart with remote does not work'
### bug #42999: --pipepart with remote does not work ### bug #42999: --pipepart with remote does not work
seq 100 > /tmp/bug42999; chmod 600 /tmp/bug42999; parallel --sshdelay 0.3 --pipepart --block 31 -a /tmp/bug42999 -k -S parallel@lo wc; parallel --sshdelay 0.2 --pipepart --block 31 -a /tmp/bug42999 -k --fifo -S parallel@lo wc | perl -pe 's:(/tmp\S+fifo)\d+:${1}0000:' ; parallel --sshdelay 0.1 --pipepart --block 31 -a /tmp/bug42999 -k --cat -S parallel@lo wc | perl -pe 's:(/tmp\S+cat)\d+:${1}0000:' ; rm /tmp/bug42999 seq 100 > /tmp/bug42999; chmod 600 /tmp/bug42999; parallel --sshdelay 0.3 --pipepart --block 31 -a /tmp/bug42999 -k -S parallel@lo wc; parallel --sshdelay 0.2 --pipepart --block 31 -a /tmp/bug42999 -k --fifo -S parallel@lo wc | perl -pe 's:(/tmp\S+fif)\S+:${1}XXXXX:' ; parallel --sshdelay 0.1 --pipepart --block 31 -a /tmp/bug42999 -k --cat -S parallel@lo wc | perl -pe 's:(/tmp\S+cat)\S+:${1}XXXXX:' ; rm /tmp/bug42999
14 14 33 14 14 33
11 11 33 11 11 33
11 11 33 11 11 33
@ -54,24 +54,24 @@ echo '### bug #42999: --pipepart with remote does not work'
11 11 33 11 11 33
11 11 33 11 11 33
9 9 28 9 9 28
14 14 33 /tmp/parallel-local-ssh2/fifo0000 14 14 33 /tmp/parallel-local-ssh2/fifXXXXX
11 11 33 /tmp/parallel-local-ssh2/fifo0000 11 11 33 /tmp/parallel-local-ssh2/fifXXXXX
11 11 33 /tmp/parallel-local-ssh2/fifo0000 11 11 33 /tmp/parallel-local-ssh2/fifXXXXX
11 11 33 /tmp/parallel-local-ssh2/fifo0000 11 11 33 /tmp/parallel-local-ssh2/fifXXXXX
11 11 33 /tmp/parallel-local-ssh2/fifo0000 11 11 33 /tmp/parallel-local-ssh2/fifXXXXX
11 11 33 /tmp/parallel-local-ssh2/fifo0000 11 11 33 /tmp/parallel-local-ssh2/fifXXXXX
11 11 33 /tmp/parallel-local-ssh2/fifo0000 11 11 33 /tmp/parallel-local-ssh2/fifXXXXX
11 11 33 /tmp/parallel-local-ssh2/fifo0000 11 11 33 /tmp/parallel-local-ssh2/fifXXXXX
9 9 28 /tmp/parallel-local-ssh2/fifo0000 9 9 28 /tmp/parallel-local-ssh2/fifXXXXX
14 14 33 /tmp/parallel-local-ssh2/cat0000 14 14 33 /tmp/parallel-local-ssh2/catXXXXX
11 11 33 /tmp/parallel-local-ssh2/cat0000 11 11 33 /tmp/parallel-local-ssh2/catXXXXX
11 11 33 /tmp/parallel-local-ssh2/cat0000 11 11 33 /tmp/parallel-local-ssh2/catXXXXX
11 11 33 /tmp/parallel-local-ssh2/cat0000 11 11 33 /tmp/parallel-local-ssh2/catXXXXX
11 11 33 /tmp/parallel-local-ssh2/cat0000 11 11 33 /tmp/parallel-local-ssh2/catXXXXX
11 11 33 /tmp/parallel-local-ssh2/cat0000 11 11 33 /tmp/parallel-local-ssh2/catXXXXX
11 11 33 /tmp/parallel-local-ssh2/cat0000 11 11 33 /tmp/parallel-local-ssh2/catXXXXX
11 11 33 /tmp/parallel-local-ssh2/cat0000 11 11 33 /tmp/parallel-local-ssh2/catXXXXX
9 9 28 /tmp/parallel-local-ssh2/cat0000 9 9 28 /tmp/parallel-local-ssh2/catXXXXX
echo '### --cat gives incorrect exit value in csh' echo '### --cat gives incorrect exit value in csh'
### --cat gives incorrect exit value in csh ### --cat gives incorrect exit value in csh
echo false | parallel --pipe --cat -Scsh@lo 'cat {}; false' ; echo $?; echo false | parallel --pipe --cat -Stcsh@lo 'cat {}; false' ; echo $?; echo true | parallel --pipe --cat -Scsh@lo 'cat {}; true' ; echo $?; echo true | parallel --pipe --cat -Stcsh@lo 'cat {}; true' ; echo $?; echo '### --cat and --fifo exit value in bash' echo false | parallel --pipe --cat -Scsh@lo 'cat {}; false' ; echo $?; echo false | parallel --pipe --cat -Stcsh@lo 'cat {}; false' ; echo $?; echo true | parallel --pipe --cat -Scsh@lo 'cat {}; true' ; echo $?; echo true | parallel --pipe --cat -Stcsh@lo 'cat {}; true' ; echo $?; echo '### --cat and --fifo exit value in bash'

1
testsuite/wanted-results/parallel-local157
View file

@ -1 +0,0 @@
TEST

42
testsuite/wanted-results/parallel-local22
View file

@ -43,25 +43,25 @@ echo '### bug #42041: Implement $PARALLEL_JOBSLOT'
2 2
echo '### bug #42363: --pipepart and --fifo/--cat does not work' echo '### bug #42363: --pipepart and --fifo/--cat does not work'
### bug #42363: --pipepart and --fifo/--cat does not work ### bug #42363: --pipepart and --fifo/--cat does not work
seq 100 > /tmp/bug42363; parallel --pipepart --block 31 -a /tmp/bug42363 -k --fifo wc | perl -pe 's:(/tmp\S+fifo)\d+:${1}0000:'; parallel --pipepart --block 31 -a /tmp/bug42363 -k --cat wc | perl -pe 's:(/tmp\S+cat)\d+:${1}0000:'; rm /tmp/bug42363 seq 100 > /tmp/bug42363; parallel --pipepart --block 31 -a /tmp/bug42363 -k --fifo wc | perl -pe 's:(/tmp\S+fif).....:${1}XXXXX:'; parallel --pipepart --block 31 -a /tmp/bug42363 -k --cat wc | perl -pe 's:(/tmp\S+cat).....:${1}XXXXX:'; rm /tmp/bug42363
14 14 33 /tmp/parallel-local22/fifo0000 14 14 33 /tmp/parallel-local22/fifXXXXX
11 11 33 /tmp/parallel-local22/fifo0000 11 11 33 /tmp/parallel-local22/fifXXXXX
11 11 33 /tmp/parallel-local22/fifo0000 11 11 33 /tmp/parallel-local22/fifXXXXX
11 11 33 /tmp/parallel-local22/fifo0000 11 11 33 /tmp/parallel-local22/fifXXXXX
11 11 33 /tmp/parallel-local22/fifo0000 11 11 33 /tmp/parallel-local22/fifXXXXX
11 11 33 /tmp/parallel-local22/fifo0000 11 11 33 /tmp/parallel-local22/fifXXXXX
11 11 33 /tmp/parallel-local22/fifo0000 11 11 33 /tmp/parallel-local22/fifXXXXX
11 11 33 /tmp/parallel-local22/fifo0000 11 11 33 /tmp/parallel-local22/fifXXXXX
9 9 28 /tmp/parallel-local22/fifo0000 9 9 28 /tmp/parallel-local22/fifXXXXX
14 14 33 /tmp/parallel-local22/cat0000 14 14 33 /tmp/parallel-local22/catXXXXX
11 11 33 /tmp/parallel-local22/cat0000 11 11 33 /tmp/parallel-local22/catXXXXX
11 11 33 /tmp/parallel-local22/cat0000 11 11 33 /tmp/parallel-local22/catXXXXX
11 11 33 /tmp/parallel-local22/cat0000 11 11 33 /tmp/parallel-local22/catXXXXX
11 11 33 /tmp/parallel-local22/cat0000 11 11 33 /tmp/parallel-local22/catXXXXX
11 11 33 /tmp/parallel-local22/cat0000 11 11 33 /tmp/parallel-local22/catXXXXX
11 11 33 /tmp/parallel-local22/cat0000 11 11 33 /tmp/parallel-local22/catXXXXX
11 11 33 /tmp/parallel-local22/cat0000 11 11 33 /tmp/parallel-local22/catXXXXX
9 9 28 /tmp/parallel-local22/cat0000 9 9 28 /tmp/parallel-local22/catXXXXX
echo '### bug #42055: --pipepart -a bigfile should not require sequential reading of bigfile' echo '### bug #42055: --pipepart -a bigfile should not require sequential reading of bigfile'
### bug #42055: --pipepart -a bigfile should not require sequential reading of bigfile ### bug #42055: --pipepart -a bigfile should not require sequential reading of bigfile
parallel --pipepart -a /etc/passwd -L 1 should not be run parallel --pipepart -a /etc/passwd -L 1 should not be run
@ -353,5 +353,5 @@ Send Receive Exitval
Send Receive Exitval Send Receive Exitval
echo '### --tmux test - check termination' echo '### --tmux test - check termination'
### --tmux test - check termination ### --tmux test - check termination
perl -e 'map {printf "$_%o%c\n",$_,$_}1..255' | stdout parallel --tmux echo {} :::: - ::: a b | perl -pe 's:(/tmp\S+par)......tms:$1XXXXX.tms:;' perl -e 'map {printf "$_%o%c\n",$_,$_}1..255' | stdout parallel --tmux echo {} :::: - ::: a b | perl -pe 's:(/tmp\S+/tms).....:$1XXXXX:;'
See output with: tmux -S /tmp/parallel-local22/parXXXXX.tms attach See output with: tmux -S /tmp/parallel-local22/tmsXXXXX attach

66
testsuite/wanted-results/parallel-local7
View file

@ -1,67 +1,67 @@
echo '### tmux1.9' echo '### tmux1.9'
### tmux1.9 ### tmux1.9
seq 000 100 | TMUX=tmux1.9 par_tmux seq 000 100 | TMUX=tmux1.9 par_tmux
See output with: tmux1.9 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.9 -S /tmp/parallel-local7/tmsXXXXX attach
0 0
seq 100 200 | TMUX=tmux1.9 par_tmux seq 100 200 | TMUX=tmux1.9 par_tmux
See output with: tmux1.9 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.9 -S /tmp/parallel-local7/tmsXXXXX attach
0 0
seq 200 300 | TMUX=tmux1.9 par_tmux seq 200 300 | TMUX=tmux1.9 par_tmux
See output with: tmux1.9 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.9 -S /tmp/parallel-local7/tmsXXXXX attach
0 0
seq 300 400 | TMUX=tmux1.9 par_tmux seq 300 400 | TMUX=tmux1.9 par_tmux
See output with: tmux1.9 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.9 -S /tmp/parallel-local7/tmsXXXXX attach
0 0
seq 400 500 | TMUX=tmux1.9 par_tmux seq 400 500 | TMUX=tmux1.9 par_tmux
See output with: tmux1.9 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.9 -S /tmp/parallel-local7/tmsXXXXX attach
0 0
seq 500 600 | TMUX=tmux1.9 par_tmux seq 500 600 | TMUX=tmux1.9 par_tmux
See output with: tmux1.9 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.9 -S /tmp/parallel-local7/tmsXXXXX attach
0 0
seq 600 700 | TMUX=tmux1.9 par_tmux seq 600 700 | TMUX=tmux1.9 par_tmux
See output with: tmux1.9 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.9 -S /tmp/parallel-local7/tmsXXXXX attach
0 0
seq 700 800 | TMUX=tmux1.9 par_tmux seq 700 800 | TMUX=tmux1.9 par_tmux
See output with: tmux1.9 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.9 -S /tmp/parallel-local7/tmsXXXXX attach
0 0
seq 800 900 | TMUX=tmux1.9 par_tmux seq 800 900 | TMUX=tmux1.9 par_tmux
See output with: tmux1.9 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.9 -S /tmp/parallel-local7/tmsXXXXX attach
0 0
seq 900 1000 | TMUX=tmux1.9 par_tmux seq 900 1000 | TMUX=tmux1.9 par_tmux
See output with: tmux1.9 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.9 -S /tmp/parallel-local7/tmsXXXXX attach
0 0
seq 1000 1100 | TMUX=tmux1.9 par_tmux seq 1000 1100 | TMUX=tmux1.9 par_tmux
See output with: tmux1.9 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.9 -S /tmp/parallel-local7/tmsXXXXX attach
0 0
seq 1100 1200 | TMUX=tmux1.9 par_tmux seq 1100 1200 | TMUX=tmux1.9 par_tmux
See output with: tmux1.9 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.9 -S /tmp/parallel-local7/tmsXXXXX attach
0 0
seq 1200 1300 | TMUX=tmux1.9 par_tmux seq 1200 1300 | TMUX=tmux1.9 par_tmux
See output with: tmux1.9 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.9 -S /tmp/parallel-local7/tmsXXXXX attach
0 0
seq 1300 1400 | TMUX=tmux1.9 par_tmux seq 1300 1400 | TMUX=tmux1.9 par_tmux
See output with: tmux1.9 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.9 -S /tmp/parallel-local7/tmsXXXXX attach
0 0
seq 1400 1500 | TMUX=tmux1.9 par_tmux seq 1400 1500 | TMUX=tmux1.9 par_tmux
See output with: tmux1.9 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.9 -S /tmp/parallel-local7/tmsXXXXX attach
0 0
seq 1500 1600 | TMUX=tmux1.9 par_tmux seq 1500 1600 | TMUX=tmux1.9 par_tmux
See output with: tmux1.9 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.9 -S /tmp/parallel-local7/tmsXXXXX attach
0 0
seq 1600 1700 | TMUX=tmux1.9 par_tmux seq 1600 1700 | TMUX=tmux1.9 par_tmux
See output with: tmux1.9 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.9 -S /tmp/parallel-local7/tmsXXXXX attach
0 0
seq 1700 1800 | TMUX=tmux1.9 par_tmux seq 1700 1800 | TMUX=tmux1.9 par_tmux
See output with: tmux1.9 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.9 -S /tmp/parallel-local7/tmsXXXXX attach
0 0
seq 1800 1900 | TMUX=tmux1.9 par_tmux seq 1800 1900 | TMUX=tmux1.9 par_tmux
See output with: tmux1.9 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.9 -S /tmp/parallel-local7/tmsXXXXX attach
0 0
seq 1900 2000 | TMUX=tmux1.9 par_tmux seq 1900 2000 | TMUX=tmux1.9 par_tmux
See output with: tmux1.9 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.9 -S /tmp/parallel-local7/tmsXXXXX attach
0 0
seq 2000 2018 | TMUX=tmux1.9 par_tmux seq 2000 2018 | TMUX=tmux1.9 par_tmux
See output with: tmux1.9 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.9 -S /tmp/parallel-local7/tmsXXXXX attach
0 0
echo '### tmux1.9 fails' echo '### tmux1.9 fails'
### tmux1.9 fails ### tmux1.9 fails
@ -80,19 +80,19 @@ parallel: Error: Command line too long (4053 >= 4047) at input 0: 2022
echo '### tmux1.8' echo '### tmux1.8'
### tmux1.8 ### tmux1.8
seq 1 50 | TMUX=tmux1.8 par_tmux seq 1 50 | TMUX=tmux1.8 par_tmux
See output with: tmux1.8 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.8 -S /tmp/parallel-local7/tmsXXXXX attach
0 0
seq 51 100 | TMUX=tmux1.8 par_tmux seq 51 100 | TMUX=tmux1.8 par_tmux
See output with: tmux1.8 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.8 -S /tmp/parallel-local7/tmsXXXXX attach
0 0
seq 101 150 | TMUX=tmux1.8 par_tmux seq 101 150 | TMUX=tmux1.8 par_tmux
See output with: tmux1.8 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.8 -S /tmp/parallel-local7/tmsXXXXX attach
0 0
seq 151 200 | TMUX=tmux1.8 par_tmux seq 151 200 | TMUX=tmux1.8 par_tmux
See output with: tmux1.8 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.8 -S /tmp/parallel-local7/tmsXXXXX attach
0 0
seq 201 231 | TMUX=tmux1.8 par_tmux seq 201 231 | TMUX=tmux1.8 par_tmux
See output with: tmux1.8 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.8 -S /tmp/parallel-local7/tmsXXXXX attach
0 0
echo '### tmux1.8 fails' echo '### tmux1.8 fails'
### tmux1.8 fails ### tmux1.8 fails
@ -114,25 +114,25 @@ parallel: Error: Command line too long (480 >= 472) at input 0: 236
echo '### tmux1.8 0..255 ascii' echo '### tmux1.8 0..255 ascii'
### tmux1.8 0..255 ascii ### tmux1.8 0..255 ascii
perl -e 'print map { ($_, map { pack("c*",$_) } grep { $_>=1 && $_!=10 } 0..$_),"\n" } 0..255' | TMUX=tmux1.8 stdout parallel --tmux --timeout 3 echo | par_tmux_filter; echo $? perl -e 'print map { ($_, map { pack("c*",$_) } grep { $_>=1 && $_!=10 } 0..$_),"\n" } 0..255' | TMUX=tmux1.8 stdout parallel --tmux --timeout 3 echo | par_tmux_filter; echo $?
See output with: tmux1.8 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.8 -S /tmp/parallel-local7/tmsXXXXX attach
0 0
echo '### tmux1.9 0..255 ascii' echo '### tmux1.9 0..255 ascii'
### tmux1.9 0..255 ascii ### tmux1.9 0..255 ascii
perl -e 'print map { ($_, map { pack("c*",$_) } grep { $_>=1 && $_!=10 } 0..$_),"\n" } 0..255' | TMUX=tmux1.9 stdout parallel --tmux --timeout 3 echo | par_tmux_filter; echo $? perl -e 'print map { ($_, map { pack("c*",$_) } grep { $_>=1 && $_!=10 } 0..$_),"\n" } 0..255' | TMUX=tmux1.9 stdout parallel --tmux --timeout 3 echo | par_tmux_filter; echo $?
See output with: tmux1.9 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.9 -S /tmp/parallel-local7/tmsXXXXX attach
0 0
echo '### Test output ascii' echo '### Test output ascii'
### Test output ascii ### Test output ascii
rm -f /tmp/paralocal7*; perl -e 'print map { ($_, map { pack("c*",$_) } grep { $_!=10 } 1..$_),"\n" } 1..255' | stdout parallel --tmux echo {}'>>/tmp/paralocal7{%}' | par_tmux_filter; sort /tmp/paralocal7* | md5sum rm -f /tmp/paralocal7*; perl -e 'print map { ($_, map { pack("c*",$_) } grep { $_!=10 } 1..$_),"\n" } 1..255' | stdout parallel --tmux echo {}'>>/tmp/paralocal7{%}' | par_tmux_filter; sort /tmp/paralocal7* | md5sum
See output with: tmux -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux -S /tmp/parallel-local7/tmsXXXXX attach
a7ee232967c8aab2edf227169e8cdce0 - a7ee232967c8aab2edf227169e8cdce0 -
echo '### Test critical lengths. Must not block' echo '### Test critical lengths. Must not block'
### Test critical lengths. Must not block ### Test critical lengths. Must not block
seq 70 130 | TMUX=tmux1.8 stdout parallel --tmux echo '{}{=$_="&"x$_=}' | par_tmux_filter seq 70 130 | TMUX=tmux1.8 stdout parallel --tmux echo '{}{=$_="&"x$_=}' | par_tmux_filter
See output with: tmux1.8 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.8 -S /tmp/parallel-local7/tmsXXXXX attach
seq 70 130 | TMUX=tmux1.9 stdout parallel --tmux echo '{}{=$_="&"x$_=}' | par_tmux_filter seq 70 130 | TMUX=tmux1.9 stdout parallel --tmux echo '{}{=$_="&"x$_=}' | par_tmux_filter
See output with: tmux1.9 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.9 -S /tmp/parallel-local7/tmsXXXXX attach
seq 280 425 | TMUX=tmux1.8 stdout parallel --tmux echo '{}{=$_="a"x$_=}' | par_tmux_filter seq 280 425 | TMUX=tmux1.8 stdout parallel --tmux echo '{}{=$_="a"x$_=}' | par_tmux_filter
See output with: tmux1.8 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.8 -S /tmp/parallel-local7/tmsXXXXX attach
seq 280 425 | TMUX=tmux1.9 stdout parallel --tmux echo '{}{=$_="a"x$_=}' | par_tmux_filter seq 280 425 | TMUX=tmux1.9 stdout parallel --tmux echo '{}{=$_="a"x$_=}' | par_tmux_filter
See output with: tmux1.9 -S /tmp/parallel-local7/parXXXXX.tms attach See output with: tmux1.9 -S /tmp/parallel-local7/tmsXXXXX attach

6
testsuite/wanted-results/parallel-tutorial
View file

@ -406,7 +406,7 @@ B D
parallel --use-cpus-instead-of-cores -N0 sleep 1 :::: num8 parallel --use-cpus-instead-of-cores -N0 sleep 1 :::: num8
seq 10 20 | parallel --tmux 'echo start {}; sleep {}; echo done {}' seq 10 20 | parallel --tmux 'echo start {}; sleep {}; echo done {}'
See output with: tmux -S /tmp/parallel-tutorial/parXXXXX.tms attach See output with: tmux -S /tmp/parallel-tutorialXXXXX/tms attach
tmux -S /tmp/parXXXXX.tms attach tmux -S /tmp/parXXXXX.tms attach
no sessions no sessions
parallel --delay 2.5 echo Starting {}\;date ::: 1 2 3 parallel --delay 2.5 echo Starting {}\;date ::: 1 2 3
@ -931,7 +931,7 @@ This helps funding further development; and it won't cost you a cent.
If you pay 10000 EUR you should feel free to use GNU Parallel without citing. If you pay 10000 EUR you should feel free to use GNU Parallel without citing.
parallel --version parallel --version
GNU parallel 20150416 GNU parallel 20150419
Copyright (C) 2007,2008,2009,2010,2011,2012,2013,2014,2015 Ole Tange Copyright (C) 2007,2008,2009,2010,2011,2012,2013,2014,2015 Ole Tange
and Free Software Foundation, Inc. and Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
@ -943,7 +943,7 @@ Web site: http://www.gnu.org/software/parallel
When using programs that use GNU Parallel to process data for publication When using programs that use GNU Parallel to process data for publication
please cite as described in 'parallel --bibtex'. please cite as described in 'parallel --bibtex'.
parallel --minversion 20130722 && echo Your version is at least 20130722. parallel --minversion 20130722 && echo Your version is at least 20130722.
20150416 20150419
Your version is at least 20130722. Your version is at least 20130722.
parallel --bibtex parallel --bibtex
Academic tradition requires you to cite works you base your article on. Academic tradition requires you to cite works you base your article on.