mirror of
https://git.savannah.gnu.org/git/parallel.git
synced 2024-11-24 23:17:55 +00:00
parallel: Fixed bug #56403: --pipe block by time. Passes testsuite.
Yearly copyright update.
This commit is contained in:
Ole Tange
parent
93001714a8
commit
b804568ffd
2
README
2
README
|
|
@ -123,7 +123,7 @@ When using programs that use GNU Parallel to process data for
|
||||||
publication please cite:
|
publication please cite:
|
||||||
|
|
||||||
O. Tange (2018): GNU Parallel 2018, Mar 2018, ISBN 9781387509881,
|
O. Tange (2018): GNU Parallel 2018, Mar 2018, ISBN 9781387509881,
|
||||||
DOI https://doi.org/10.5281/zenodo.11460
|
DOI https://doi.org/10.5281/zenodo.1146014
|
||||||
|
|
||||||
|
|
||||||
= New versions =
|
= New versions =
|
||||||
|
|
|
||||||
|
|
@ -209,9 +209,9 @@ from:tange@gnu.org
|
||||||
to:parallel@gnu.org, bug-parallel@gnu.org
|
to:parallel@gnu.org, bug-parallel@gnu.org
|
||||||
stable-bcc: Jesse Alama <jessealama@fastmail.fm>
|
stable-bcc: Jesse Alama <jessealama@fastmail.fm>
|
||||||
|
|
||||||
Subject: GNU Parallel 20191222 ('Impeachment') released <<[stable]>>
|
Subject: GNU Parallel 20200122 ('Soleimani') released <<[stable]>>
|
||||||
|
|
||||||
GNU Parallel 20191222 ('') <<[stable]>> has been released. It is available for download at: http://ftpmirror.gnu.org/parallel/
|
GNU Parallel 20200122 ('Soleimani') <<[stable]>> has been released. It is available for download at: http://ftpmirror.gnu.org/parallel/
|
||||||
|
|
||||||
<<No new functionality was introduced so this is a good candidate for a stable release.>>
|
<<No new functionality was introduced so this is a good candidate for a stable release.>>
|
||||||
|
|
||||||
|
|
@ -221,6 +221,8 @@ See https://www.gnu.org/software/parallel/10-years-anniversary.html
|
||||||
|
|
||||||
Quote of the month:
|
Quote of the month:
|
||||||
|
|
||||||
|
GNU parallel is straight up incredible.
|
||||||
|
-- Ben Johnson @biobenkj@twtter
|
||||||
|
|
||||||
New in this release:
|
New in this release:
|
||||||
|
|
||||||
|
|
@ -228,7 +230,9 @@ New in this release:
|
||||||
|
|
||||||
* GNU Parallel course in Århus https://www.prosa.dk/nc/arrangementer/arrangement/gnu-parallel-og-parallelisering-i-unix-shellen/
|
* GNU Parallel course in Århus https://www.prosa.dk/nc/arrangementer/arrangement/gnu-parallel-og-parallelisering-i-unix-shellen/
|
||||||
|
|
||||||
* GNU Parallel is used in https://github.com/JeffersonLab/rfw_tsf_extractor
|
|
||||||
|
https://net2.com/how-to-copy-a-file-to-multiple-directories-in-linux/
|
||||||
|
https://dev.to/voyeg3r/runing-linux-commands-in-parallel-4ff8
|
||||||
|
|
||||||
* Bug fixes and man page updates.
|
* Bug fixes and man page updates.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,7 @@
|
||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
|
|
||||||
# Copyright (C) 2016,2017,2018
|
# Copyright (C) 2016-2020 Ole Tange, http://ole.tange.dk and Free
|
||||||
# Ole Tange and Free Software Foundation, Inc.
|
# Software Foundation, Inc.
|
||||||
#
|
#
|
||||||
# This program is free software; you can redistribute it and/or modify
|
# This program is free software; you can redistribute it and/or modify
|
||||||
# it under the terms of the GNU General Public License as published by
|
# it under the terms of the GNU General Public License as published by
|
||||||
|
|
|
||||||
|
|
@ -7,7 +7,8 @@
|
||||||
# after which 'env_parallel' works
|
# after which 'env_parallel' works
|
||||||
#
|
#
|
||||||
#
|
#
|
||||||
# Copyright (C) 2016-2019 Ole Tange and Free Software Foundation, Inc.
|
# Copyright (C) 2016-2020 Ole Tange, http://ole.tange.dk and Free
|
||||||
|
# Software Foundation, Inc.
|
||||||
#
|
#
|
||||||
# This program is free software; you can redistribute it and/or modify
|
# This program is free software; you can redistribute it and/or modify
|
||||||
# it under the terms of the GNU General Public License as published by
|
# it under the terms of the GNU General Public License as published by
|
||||||
|
|
|
||||||
|
|
@ -7,7 +7,8 @@
|
||||||
# after which 'env_parallel' works
|
# after which 'env_parallel' works
|
||||||
#
|
#
|
||||||
#
|
#
|
||||||
# Copyright (C) 2016-2019 Ole Tange and Free Software Foundation, Inc.
|
# Copyright (C) 2016-2020 Ole Tange, http://ole.tange.dk and Free
|
||||||
|
# Software Foundation, Inc.
|
||||||
#
|
#
|
||||||
# This program is free software; you can redistribute it and/or modify
|
# This program is free software; you can redistribute it and/or modify
|
||||||
# it under the terms of the GNU General Public License as published by
|
# it under the terms of the GNU General Public License as published by
|
||||||
|
|
|
||||||
|
|
@ -7,7 +7,8 @@
|
||||||
# after which 'env_parallel' works
|
# after which 'env_parallel' works
|
||||||
#
|
#
|
||||||
#
|
#
|
||||||
# Copyright (C) 2016-2019 Ole Tange and Free Software Foundation, Inc.
|
# Copyright (C) 2016-2020 Ole Tange, http://ole.tange.dk and Free
|
||||||
|
# Software Foundation, Inc.
|
||||||
#
|
#
|
||||||
# This program is free software; you can redistribute it and/or modify
|
# This program is free software; you can redistribute it and/or modify
|
||||||
# it under the terms of the GNU General Public License as published by
|
# it under the terms of the GNU General Public License as published by
|
||||||
|
|
|
||||||
|
|
@ -7,7 +7,8 @@
|
||||||
# after which 'env_parallel' works
|
# after which 'env_parallel' works
|
||||||
#
|
#
|
||||||
#
|
#
|
||||||
# Copyright (C) 2016-2019 Ole Tange and Free Software Foundation, Inc.
|
# Copyright (C) 2016-2020 Ole Tange, http://ole.tange.dk and Free
|
||||||
|
# Software Foundation, Inc.
|
||||||
#
|
#
|
||||||
# This program is free software; you can redistribute it and/or modify
|
# This program is free software; you can redistribute it and/or modify
|
||||||
# it under the terms of the GNU General Public License as published by
|
# it under the terms of the GNU General Public License as published by
|
||||||
|
|
|
||||||
|
|
@ -7,7 +7,8 @@
|
||||||
# after which 'env_parallel' works
|
# after which 'env_parallel' works
|
||||||
#
|
#
|
||||||
#
|
#
|
||||||
# Copyright (C) 2016-2019 Ole Tange and Free Software Foundation, Inc.
|
# Copyright (C) 2016-2020 Ole Tange, http://ole.tange.dk and Free
|
||||||
|
# Software Foundation, Inc.
|
||||||
#
|
#
|
||||||
# This program is free software; you can redistribute it and/or modify
|
# This program is free software; you can redistribute it and/or modify
|
||||||
# it under the terms of the GNU General Public License as published by
|
# it under the terms of the GNU General Public License as published by
|
||||||
|
|
|
||||||
|
|
@ -7,7 +7,8 @@
|
||||||
# after which 'env_parallel' works
|
# after which 'env_parallel' works
|
||||||
#
|
#
|
||||||
#
|
#
|
||||||
# Copyright (C) 2016-2019 Ole Tange and Free Software Foundation, Inc.
|
# Copyright (C) 2016-2020 Ole Tange, http://ole.tange.dk and Free
|
||||||
|
# Software Foundation, Inc.
|
||||||
#
|
#
|
||||||
# This program is free software; you can redistribute it and/or modify
|
# This program is free software; you can redistribute it and/or modify
|
||||||
# it under the terms of the GNU General Public License as published by
|
# it under the terms of the GNU General Public License as published by
|
||||||
|
|
|
||||||
|
|
@ -7,7 +7,8 @@
|
||||||
# after which 'env_parallel' works
|
# after which 'env_parallel' works
|
||||||
#
|
#
|
||||||
#
|
#
|
||||||
# Copyright (C) 2016-2019 Ole Tange and Free Software Foundation, Inc.
|
# Copyright (C) 2016-2020 Ole Tange, http://ole.tange.dk and Free
|
||||||
|
# Software Foundation, Inc.
|
||||||
#
|
#
|
||||||
# This program is free software; you can redistribute it and/or modify
|
# This program is free software; you can redistribute it and/or modify
|
||||||
# it under the terms of the GNU General Public License as published by
|
# it under the terms of the GNU General Public License as published by
|
||||||
|
|
|
||||||
|
|
@ -7,7 +7,8 @@
|
||||||
# after which 'env_parallel' works
|
# after which 'env_parallel' works
|
||||||
#
|
#
|
||||||
#
|
#
|
||||||
# Copyright (C) 2016-2019 Ole Tange and Free Software Foundation, Inc.
|
# Copyright (C) 2016-2020 Ole Tange, http://ole.tange.dk and Free
|
||||||
|
# Software Foundation, Inc.
|
||||||
#
|
#
|
||||||
# This program is free software; you can redistribute it and/or modify
|
# This program is free software; you can redistribute it and/or modify
|
||||||
# it under the terms of the GNU General Public License as published by
|
# it under the terms of the GNU General Public License as published by
|
||||||
|
|
|
||||||
|
|
@ -802,14 +802,12 @@ Copyright (C) 2007-10-18 Ole Tange, http://ole.tange.dk
|
||||||
|
|
||||||
Copyright (C) 2008-2010 Ole Tange, http://ole.tange.dk
|
Copyright (C) 2008-2010 Ole Tange, http://ole.tange.dk
|
||||||
|
|
||||||
Copyright (C) 2010-2019 Ole Tange, http://ole.tange.dk and Free
|
Copyright (C) 2010-2020 Ole Tange, http://ole.tange.dk and Free
|
||||||
Software Foundation, Inc.
|
Software Foundation, Inc.
|
||||||
|
|
||||||
|
|
||||||
=head1 LICENSE
|
=head1 LICENSE
|
||||||
|
|
||||||
Copyright (C) 2016,2017 Free Software Foundation, Inc.
|
|
||||||
|
|
||||||
This program is free software; you can redistribute it and/or modify
|
This program is free software; you can redistribute it and/or modify
|
||||||
it under the terms of the GNU General Public License as published by
|
it under the terms of the GNU General Public License as published by
|
||||||
the Free Software Foundation; either version 3 of the License, or
|
the Free Software Foundation; either version 3 of the License, or
|
||||||
|
|
|
||||||
|
|
@ -7,7 +7,8 @@
|
||||||
# after which 'env_parallel' works
|
# after which 'env_parallel' works
|
||||||
#
|
#
|
||||||
#
|
#
|
||||||
# Copyright (C) 2016-2019 Ole Tange and Free Software Foundation, Inc.
|
# Copyright (C) 2016-2020 Ole Tange, http://ole.tange.dk and Free
|
||||||
|
# Software Foundation, Inc.
|
||||||
#
|
#
|
||||||
# This program is free software; you can redistribute it and/or modify
|
# This program is free software; you can redistribute it and/or modify
|
||||||
# it under the terms of the GNU General Public License as published by
|
# it under the terms of the GNU General Public License as published by
|
||||||
|
|
|
||||||
|
|
@ -7,7 +7,8 @@
|
||||||
# after which 'env_parallel' works
|
# after which 'env_parallel' works
|
||||||
#
|
#
|
||||||
#
|
#
|
||||||
# Copyright (C) 2016-2019 Ole Tange and Free Software Foundation, Inc.
|
# Copyright (C) 2016-2020 Ole Tange, http://ole.tange.dk and Free
|
||||||
|
# Software Foundation, Inc.
|
||||||
#
|
#
|
||||||
# This program is free software; you can redistribute it and/or modify
|
# This program is free software; you can redistribute it and/or modify
|
||||||
# it under the terms of the GNU General Public License as published by
|
# it under the terms of the GNU General Public License as published by
|
||||||
|
|
|
||||||
|
|
@ -7,7 +7,8 @@
|
||||||
# after which 'env_parallel' works
|
# after which 'env_parallel' works
|
||||||
#
|
#
|
||||||
#
|
#
|
||||||
# Copyright (C) 2016-2019 Ole Tange and Free Software Foundation, Inc.
|
# Copyright (C) 2016-2020 Ole Tange, http://ole.tange.dk and Free
|
||||||
|
# Software Foundation, Inc.
|
||||||
#
|
#
|
||||||
# This program is free software; you can redistribute it and/or modify
|
# This program is free software; you can redistribute it and/or modify
|
||||||
# it under the terms of the GNU General Public License as published by
|
# it under the terms of the GNU General Public License as published by
|
||||||
|
|
|
||||||
|
|
@ -2,7 +2,7 @@
|
||||||
|
|
||||||
# Copyright (C) 2004-2010 Ole Tange, http://ole.tange.dk
|
# Copyright (C) 2004-2010 Ole Tange, http://ole.tange.dk
|
||||||
#
|
#
|
||||||
# Copyright (C) 2010-2019 Ole Tange, http://ole.tange.dk and
|
# Copyright (C) 2010-2020 Ole Tange, http://ole.tange.dk and
|
||||||
# Free Software Foundation, Inc.
|
# Free Software Foundation, Inc.
|
||||||
#
|
#
|
||||||
# This program is free software; you can redistribute it and/or modify
|
# This program is free software; you can redistribute it and/or modify
|
||||||
|
|
|
||||||
|
|
@ -305,7 +305,7 @@ Copyright (C) 2004-11-19 Ole Tange, http://ole.tange.dk
|
||||||
|
|
||||||
Copyright (C) 2005-2010 Ole Tange, http://ole.tange.dk
|
Copyright (C) 2005-2010 Ole Tange, http://ole.tange.dk
|
||||||
|
|
||||||
Copyright (C) 2010-2019 Ole Tange, http://ole.tange.dk and Free
|
Copyright (C) 2010-2020 Ole Tange, http://ole.tange.dk and Free
|
||||||
Software Foundation, Inc.
|
Software Foundation, Inc.
|
||||||
|
|
||||||
=head1 LICENSE
|
=head1 LICENSE
|
||||||
|
|
|
||||||
344
src/parallel
344
src/parallel
|
|
@ -1,6 +1,7 @@
|
||||||
#!/usr/bin/env perl
|
#!/usr/bin/env perl
|
||||||
|
|
||||||
# Copyright (C) 2007-2019 Ole Tange and Free Software Foundation, Inc.
|
# Copyright (C) 2007-2020 Ole Tange, http://ole.tange.dk and Free
|
||||||
|
# Software Foundation, Inc.
|
||||||
#
|
#
|
||||||
# This program is free software; you can redistribute it and/or modify
|
# This program is free software; you can redistribute it and/or modify
|
||||||
# it under the terms of the GNU General Public License as published by
|
# it under the terms of the GNU General Public License as published by
|
||||||
|
|
@ -504,7 +505,7 @@ sub pipe_shard_setup() {
|
||||||
} else {
|
} else {
|
||||||
$script = sharder_script();
|
$script = sharder_script();
|
||||||
}
|
}
|
||||||
|
|
||||||
# cat foo | sharder sep col fifo1 fifo2 fifo3 ... fifoN
|
# cat foo | sharder sep col fifo1 fifo2 fifo3 ... fifoN
|
||||||
|
|
||||||
if($shardbin =~ /^[a-z_][a-z_0-9]*(\s|$)/i) {
|
if($shardbin =~ /^[a-z_][a-z_0-9]*(\s|$)/i) {
|
||||||
|
|
@ -589,8 +590,7 @@ sub find_header($$) {
|
||||||
if($opt::header eq ":") { $opt::header = "(.*\n)"; }
|
if($opt::header eq ":") { $opt::header = "(.*\n)"; }
|
||||||
# Number = number of lines
|
# Number = number of lines
|
||||||
$opt::header =~ s/^(\d+)$/"(.*\n)"x$1/e;
|
$opt::header =~ s/^(\d+)$/"(.*\n)"x$1/e;
|
||||||
while(read($fh,substr($$buf_ref,length $$buf_ref,0),
|
while(sysread($fh,$$buf_ref,$Global::blocksize,length $$buf_ref)) {
|
||||||
$Global::blocksize)) {
|
|
||||||
if($$buf_ref =~ s/^($opt::header)//) {
|
if($$buf_ref =~ s/^($opt::header)//) {
|
||||||
$header = $1;
|
$header = $1;
|
||||||
last;
|
last;
|
||||||
|
|
@ -637,11 +637,11 @@ sub find_split_positions($$$) {
|
||||||
push(@pos,$pos);
|
push(@pos,$pos);
|
||||||
} else {
|
} else {
|
||||||
# Seek the the block start
|
# Seek the the block start
|
||||||
if(not seek($fh, $pos, 0)) {
|
if(not sysseek($fh, $pos, 0)) {
|
||||||
::error("Cannot seek to $pos in $file");
|
::error("Cannot seek to $pos in $file");
|
||||||
edit(255);
|
edit(255);
|
||||||
}
|
}
|
||||||
while(read($fh,substr($buf,length $buf,0),$dd_block_size)) {
|
while(sysread($fh,$buf,$dd_block_size,length $buf)) {
|
||||||
if($opt::regexp) {
|
if($opt::regexp) {
|
||||||
# If match /$recend$recstart/ => Record position
|
# If match /$recend$recstart/ => Record position
|
||||||
if($buf =~ m:^(.*$recend)$recstart:os) {
|
if($buf =~ m:^(.*$recend)$recstart:os) {
|
||||||
|
|
@ -667,7 +667,7 @@ sub find_split_positions($$$) {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if($pos[$#pos] != $size) {
|
if($pos[$#pos] != $size) {
|
||||||
# Last splitpoint was not at end of the file: add it
|
# Last splitpoint was not at end of the file: add $size as the last
|
||||||
push @pos, $size;
|
push @pos, $size;
|
||||||
}
|
}
|
||||||
close $fh;
|
close $fh;
|
||||||
|
|
@ -948,131 +948,197 @@ sub spreadstdin() {
|
||||||
my $two_gb = 2**31-1;
|
my $two_gb = 2**31-1;
|
||||||
my $blocksize = $Global::blocksize;
|
my $blocksize = $Global::blocksize;
|
||||||
my $in = *STDIN;
|
my $in = *STDIN;
|
||||||
|
my $timeout = ::multiply_time_units($opt::blocktimeout);
|
||||||
|
|
||||||
my $header = find_header(\$buf,$in);
|
my $header = find_header(\$buf,$in);
|
||||||
|
my $anything_written;
|
||||||
|
my $eof;
|
||||||
|
|
||||||
|
sub read_block() {
|
||||||
|
# Read a --blocksize from STDIN
|
||||||
|
# possibly interrupted by --blocktimeout
|
||||||
|
# Add up to the next full block
|
||||||
|
my $readsize = $blocksize - (length $buf) % $blocksize;
|
||||||
|
my ($nread,$alarm);
|
||||||
|
eval {
|
||||||
|
local $SIG{ALRM} = sub { die "alarm\n" }; # NB: \n required
|
||||||
|
# --blocktimeout (or 0 if not set)
|
||||||
|
alarm $timeout;
|
||||||
|
do {
|
||||||
|
$nread = sysread $in, $buf, $readsize, length $buf;
|
||||||
|
$readsize -= $nread;
|
||||||
|
} while($readsize and $nread);
|
||||||
|
alarm 0;
|
||||||
|
};
|
||||||
|
if ($@) {
|
||||||
|
die unless $@ eq "alarm\n"; # propagate unexpected errors
|
||||||
|
$alarm = 1;
|
||||||
|
} else {
|
||||||
|
$alarm = 0;
|
||||||
|
}
|
||||||
|
$eof = not ($nread or $alarm);
|
||||||
|
}
|
||||||
|
|
||||||
|
sub pass_n_line_records() {
|
||||||
|
# Pass records of N lines
|
||||||
|
my $n_lines = $buf =~ tr/\n/\n/;
|
||||||
|
my $last_newline_pos = rindex64(\$buf,"\n");
|
||||||
|
# Go backwards until there are full n-line records
|
||||||
|
while($n_lines % $Global::max_lines) {
|
||||||
|
$n_lines--;
|
||||||
|
$last_newline_pos = rindex64(\$buf,"\n",$last_newline_pos-1);
|
||||||
|
}
|
||||||
|
# Chop at $last_newline_pos as that is where n-line record ends
|
||||||
|
$anything_written +=
|
||||||
|
write_record_to_pipe($chunk_number++,\$header,\$buf,
|
||||||
|
$recstart,$recend,$last_newline_pos+1);
|
||||||
|
shorten(\$buf,$last_newline_pos+1);
|
||||||
|
}
|
||||||
|
|
||||||
|
sub pass_n_regexps() {
|
||||||
|
# Pass records of N regexps
|
||||||
|
# -N => (start..*?end){n}
|
||||||
|
# -L -N => (start..*?end){n*l}
|
||||||
|
my $read_n_lines = -1+
|
||||||
|
$Global::max_number_of_args * ($Global::max_lines || 1);
|
||||||
|
# (?!negative lookahead) is needed to avoid backtracking
|
||||||
|
# See: https://unix.stackexchange.com/questions/439356/
|
||||||
|
while($buf =~
|
||||||
|
/(
|
||||||
|
# Either recstart or at least one char from start
|
||||||
|
^(?: $recstart | .)
|
||||||
|
# followed something
|
||||||
|
(?:(?!$recend$recstart).)*?
|
||||||
|
# and then recend
|
||||||
|
$recend
|
||||||
|
# Then n-1 times recstart.*recend
|
||||||
|
(?:$recstart(?:(?!$recend$recstart).)*?$recend){$read_n_lines}
|
||||||
|
)
|
||||||
|
# Followed by recstart
|
||||||
|
(?=$recstart)/osx) {
|
||||||
|
$anything_written +=
|
||||||
|
write_record_to_pipe($chunk_number++,\$header,\$buf,
|
||||||
|
$recstart,$recend,length $1);
|
||||||
|
shorten(\$buf,length $1);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
sub pass_regexp() {
|
||||||
|
# Find the last recend-recstart in $buf
|
||||||
|
$eof and return;
|
||||||
|
if($buf =~ /^(.*$recend)$recstart.*?$/os) {
|
||||||
|
$anything_written +=
|
||||||
|
write_record_to_pipe($chunk_number++,\$header,\$buf,
|
||||||
|
$recstart,$recend,length $1);
|
||||||
|
shorten(\$buf,length $1);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
sub pass_csv_record() {
|
||||||
|
# Pass CVS record
|
||||||
|
# We define a CSV record as an even number of " + end of line
|
||||||
|
# This works if you use " as quoting character
|
||||||
|
my $last_newline_pos = length $buf;
|
||||||
|
# Go backwards from the last \n and search for a position
|
||||||
|
# where there is an even number of "
|
||||||
|
do {
|
||||||
|
# find last EOL
|
||||||
|
$last_newline_pos = rindex64(\$buf,"\n",$last_newline_pos-1);
|
||||||
|
# While uneven "
|
||||||
|
} while((substr($buf,0,$last_newline_pos) =~ y/"/"/)%2
|
||||||
|
and $last_newline_pos >= 0);
|
||||||
|
# Chop at $last_newline_pos as that is where CSV record ends
|
||||||
|
$anything_written +=
|
||||||
|
write_record_to_pipe($chunk_number++,\$header,\$buf,
|
||||||
|
$recstart,$recend,$last_newline_pos+1);
|
||||||
|
shorten(\$buf,$last_newline_pos+1);
|
||||||
|
}
|
||||||
|
|
||||||
|
sub pass_n() {
|
||||||
|
# Pass n records of --recend/--recstart
|
||||||
|
# -N => (start..*?end){n}
|
||||||
|
my $i = 0;
|
||||||
|
my $read_n_lines =
|
||||||
|
$Global::max_number_of_args * ($Global::max_lines || 1);
|
||||||
|
while(($i = nindex(\$buf,$recendrecstart,$read_n_lines)) != -1
|
||||||
|
and
|
||||||
|
length $buf) {
|
||||||
|
$i += length $recend; # find the actual splitting location
|
||||||
|
$anything_written +=
|
||||||
|
write_record_to_pipe($chunk_number++,\$header,\$buf,
|
||||||
|
$recstart,$recend,$i);
|
||||||
|
shorten(\$buf,$i);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
sub pass() {
|
||||||
|
# Pass records of --recend/--recstart
|
||||||
|
# Split record at fixed string
|
||||||
|
# Find the last recend+recstart in $buf
|
||||||
|
$eof and return;
|
||||||
|
my $i = rindex64(\$buf,$recendrecstart);
|
||||||
|
if($i != -1) {
|
||||||
|
$i += length $recend; # find the actual splitting location
|
||||||
|
$anything_written +=
|
||||||
|
write_record_to_pipe($chunk_number++,\$header,\$buf,
|
||||||
|
$recstart,$recend,$i);
|
||||||
|
shorten(\$buf,$i);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
sub increase_blocksize_maybe() {
|
||||||
|
if(not $anything_written
|
||||||
|
and not $opt::blocktimeout
|
||||||
|
and not $Global::no_autoexpand_block) {
|
||||||
|
# Nothing was written - maybe the block size < record size?
|
||||||
|
# Increase blocksize exponentially up to 2GB-1 (2GB causes problems)
|
||||||
|
if($blocksize < $two_gb) {
|
||||||
|
my $old_blocksize = $blocksize;
|
||||||
|
$blocksize = ::min(ceil($blocksize * 1.3 + 1), $two_gb);
|
||||||
|
::warning("A record was longer than $old_blocksize. " .
|
||||||
|
"Increasing to --blocksize $blocksize.");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
while(1) {
|
while(1) {
|
||||||
my $anything_written = 0;
|
$anything_written = 0;
|
||||||
my $buflen = length $buf;
|
read_block();
|
||||||
my $readsize = ($buflen < $blocksize) ? $blocksize-$buflen : $blocksize;
|
if($opt::r) {
|
||||||
# If $buf < $blocksize, append so it is $blocksize long after reading.
|
# Remove empty lines
|
||||||
# Otherwise append a full $blocksize
|
$buf =~ s/^\s*\n//gm;
|
||||||
if(not read($in,substr($buf,$buflen,0),$readsize)) {
|
if(length $buf == 0) {
|
||||||
# End-of-file
|
if($eof) {
|
||||||
$chunk_number != 1 and last;
|
last;
|
||||||
# Force the while-loop once if everything was read by header reading
|
} else {
|
||||||
$one_time_through++ and last;
|
next;
|
||||||
}
|
}
|
||||||
if($opt::r) {
|
}
|
||||||
# Remove empty lines
|
}
|
||||||
$buf =~ s/^\s*\n//gm;
|
if($Global::max_lines and not $Global::max_number_of_args) {
|
||||||
if(length $buf == 0) {
|
# Pass n-line records
|
||||||
next;
|
pass_n_line_records();
|
||||||
}
|
} elsif($opt::csv) {
|
||||||
}
|
# Pass a full CSV record
|
||||||
if($Global::max_lines and not $Global::max_number_of_args) {
|
pass_csv_record();
|
||||||
# Read n-line records
|
} elsif($opt::regexp) {
|
||||||
my $n_lines = $buf =~ tr/\n/\n/;
|
# Split record at regexp
|
||||||
my $last_newline_pos = rindex64(\$buf,"\n");
|
if($Global::max_number_of_args) {
|
||||||
# Go backwards until there are full n-line records
|
pass_n_regexps();
|
||||||
while($n_lines % $Global::max_lines) {
|
} else {
|
||||||
$n_lines--;
|
pass_regexp();
|
||||||
$last_newline_pos = rindex64(\$buf,"\n",$last_newline_pos-1);
|
}
|
||||||
}
|
} else {
|
||||||
# Chop at $last_newline_pos as that is where n-line record ends
|
# Pass normal --recend/--recstart record
|
||||||
$anything_written +=
|
if($Global::max_number_of_args) {
|
||||||
write_record_to_pipe($chunk_number++,\$header,\$buf,
|
pass_n();
|
||||||
$recstart,$recend,$last_newline_pos+1);
|
} else {
|
||||||
shorten(\$buf,$last_newline_pos+1);
|
pass();
|
||||||
} elsif($opt::regexp) {
|
}
|
||||||
if($Global::max_number_of_args) {
|
}
|
||||||
# -N => (start..*?end){n}
|
$eof and last;
|
||||||
# -L -N => (start..*?end){n*l}
|
increase_blocksize_maybe();
|
||||||
my $read_n_lines = -1+
|
::debug("init", "Round\n");
|
||||||
$Global::max_number_of_args * ($Global::max_lines || 1);
|
|
||||||
# (?!negative lookahead) is needed to avoid backtracking
|
|
||||||
# See: https://unix.stackexchange.com/questions/439356/
|
|
||||||
while($buf =~
|
|
||||||
/(
|
|
||||||
# Either recstart or at least one char from start
|
|
||||||
^(?: $recstart | .)
|
|
||||||
# followed something
|
|
||||||
(?:(?!$recend$recstart).)*?
|
|
||||||
# and then recend
|
|
||||||
$recend
|
|
||||||
# Then n-1 times recstart.*recend
|
|
||||||
(?:$recstart(?:(?!$recend$recstart).)*?$recend){$read_n_lines}
|
|
||||||
)
|
|
||||||
# Followed by recstart
|
|
||||||
(?=$recstart)/osx) {
|
|
||||||
$anything_written +=
|
|
||||||
write_record_to_pipe($chunk_number++,\$header,\$buf,
|
|
||||||
$recstart,$recend,length $1);
|
|
||||||
shorten(\$buf,length $1);
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
eof($in) and last;
|
|
||||||
# Find the last recend-recstart in $buf
|
|
||||||
if($buf =~ /^(.*$recend)$recstart.*?$/os) {
|
|
||||||
$anything_written +=
|
|
||||||
write_record_to_pipe($chunk_number++,\$header,\$buf,
|
|
||||||
$recstart,$recend,length $1);
|
|
||||||
shorten(\$buf,length $1);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
} elsif($opt::csv) {
|
|
||||||
# Read a full CSV record
|
|
||||||
# even number of " + end of line
|
|
||||||
my $last_newline_pos = length $buf;
|
|
||||||
do {
|
|
||||||
# find last EOL
|
|
||||||
$last_newline_pos = rindex64(\$buf,"\n",$last_newline_pos-1);
|
|
||||||
# While uneven "
|
|
||||||
} while((substr($buf,0,$last_newline_pos) =~ y/"/"/)%2
|
|
||||||
and $last_newline_pos >= 0);
|
|
||||||
# Chop at $last_newline_pos as that is where CSV record ends
|
|
||||||
$anything_written +=
|
|
||||||
write_record_to_pipe($chunk_number++,\$header,\$buf,
|
|
||||||
$recstart,$recend,$last_newline_pos+1);
|
|
||||||
shorten(\$buf,$last_newline_pos+1);
|
|
||||||
} else {
|
|
||||||
if($Global::max_number_of_args) {
|
|
||||||
# -N => (start..*?end){n}
|
|
||||||
my $i = 0;
|
|
||||||
my $read_n_lines =
|
|
||||||
$Global::max_number_of_args * ($Global::max_lines || 1);
|
|
||||||
while(($i = nindex(\$buf,$recendrecstart,$read_n_lines)) != -1
|
|
||||||
and
|
|
||||||
length $buf) {
|
|
||||||
$i += length $recend; # find the actual splitting location
|
|
||||||
$anything_written +=
|
|
||||||
write_record_to_pipe($chunk_number++,\$header,\$buf,
|
|
||||||
$recstart,$recend,$i);
|
|
||||||
shorten(\$buf,$i);
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
eof($in) and last;
|
|
||||||
# Find the last recend+recstart in $buf
|
|
||||||
my $i = rindex64(\$buf,$recendrecstart);
|
|
||||||
if($i != -1) {
|
|
||||||
$i += length $recend; # find the actual splitting location
|
|
||||||
$anything_written +=
|
|
||||||
write_record_to_pipe($chunk_number++,\$header,\$buf,
|
|
||||||
$recstart,$recend,$i);
|
|
||||||
shorten(\$buf,$i);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if(not $anything_written
|
|
||||||
and not eof($in)
|
|
||||||
and not $Global::no_autoexpand_block) {
|
|
||||||
# Nothing was written - maybe the block size < record size?
|
|
||||||
# Increase blocksize exponentially up to 2GB-1 (2GB causes problems)
|
|
||||||
if($blocksize < $two_gb) {
|
|
||||||
my $old_blocksize = $blocksize;
|
|
||||||
$blocksize = ::min(ceil($blocksize * 1.3 + 1), $two_gb);
|
|
||||||
::warning("A record was longer than $old_blocksize. " .
|
|
||||||
"Increasing to --blocksize $blocksize.");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
::debug("init", "Done reading input\n");
|
::debug("init", "Done reading input\n");
|
||||||
|
|
||||||
|
|
@ -1092,7 +1158,7 @@ sub spreadstdin() {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
# Wait for running jobs to be done
|
# Wait for running jobs to be done
|
||||||
my $sleep =1;
|
my $sleep = 1;
|
||||||
while($Global::total_running > 0) {
|
while($Global::total_running > 0) {
|
||||||
$sleep = ::reap_usleep($sleep);
|
$sleep = ::reap_usleep($sleep);
|
||||||
start_more_jobs();
|
start_more_jobs();
|
||||||
|
|
@ -1530,6 +1596,7 @@ sub options_hash() {
|
||||||
"remove-rec-sep|removerecsep|rrs" => \$opt::remove_rec_sep,
|
"remove-rec-sep|removerecsep|rrs" => \$opt::remove_rec_sep,
|
||||||
"files|output-as-files|outputasfiles" => \$opt::files,
|
"files|output-as-files|outputasfiles" => \$opt::files,
|
||||||
"block|block-size|blocksize=s" => \$opt::blocksize,
|
"block|block-size|blocksize=s" => \$opt::blocksize,
|
||||||
|
"blocktimeout|block-timeout|bt=s" => \$opt::blocktimeout,
|
||||||
"tollef" => \$opt::tollef,
|
"tollef" => \$opt::tollef,
|
||||||
"gnu" => \$opt::gnu,
|
"gnu" => \$opt::gnu,
|
||||||
"link|xapply" => \$opt::link,
|
"link|xapply" => \$opt::link,
|
||||||
|
|
@ -2071,7 +2138,7 @@ sub check_invalid_option_combinations() {
|
||||||
|
|
||||||
sub init_globals() {
|
sub init_globals() {
|
||||||
# Defaults:
|
# Defaults:
|
||||||
$Global::version = 20191222;
|
$Global::version = 20191223;
|
||||||
$Global::progname = 'parallel';
|
$Global::progname = 'parallel';
|
||||||
$::name = "GNU Parallel";
|
$::name = "GNU Parallel";
|
||||||
$Global::infinity = 2**31;
|
$Global::infinity = 2**31;
|
||||||
|
|
@ -2280,7 +2347,7 @@ sub parse_replacement_string_options() {
|
||||||
for(@opt::rpl) {
|
for(@opt::rpl) {
|
||||||
# Create $Global::rpl entries for --rpl options
|
# Create $Global::rpl entries for --rpl options
|
||||||
# E.g: "{..} s:\.[^.]+$:;s:\.[^.]+$:;"
|
# E.g: "{..} s:\.[^.]+$:;s:\.[^.]+$:;"
|
||||||
my ($shorthand,$long) = split/ /,$_,2;
|
my ($shorthand,$long) = split/\s/,$_,2;
|
||||||
$Global::rpl{$shorthand} = $long;
|
$Global::rpl{$shorthand} = $long;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -4920,7 +4987,8 @@ sub version() {
|
||||||
print join
|
print join
|
||||||
("\n",
|
("\n",
|
||||||
"GNU $Global::progname $Global::version",
|
"GNU $Global::progname $Global::version",
|
||||||
"Copyright (C) 2007-2019 Ole Tange and Free Software Foundation, Inc.",
|
"Copyright (C) 2007-2020 Ole Tange, http://ole.tange.dk and Free Software",
|
||||||
|
"Foundation, Inc.",
|
||||||
"License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>",
|
"License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>",
|
||||||
"This is free software: you are free to change and redistribute it.",
|
"This is free software: you are free to change and redistribute it.",
|
||||||
"GNU $Global::progname comes with no warranty.",
|
"GNU $Global::progname comes with no warranty.",
|
||||||
|
|
@ -5050,8 +5118,8 @@ sub embed() {
|
||||||
}
|
}
|
||||||
print "#!$Global::shell
|
print "#!$Global::shell
|
||||||
|
|
||||||
# Copyright (C) 2007-2019 $user, Ole Tange and Free Software
|
# Copyright (C) 2007-2020 $user, Ole Tange, http://ole.tange.dk
|
||||||
# Foundation, Inc.
|
# and Free Software Foundation, Inc.
|
||||||
#
|
#
|
||||||
# This program is free software; you can redistribute it and/or modify
|
# This program is free software; you can redistribute it and/or modify
|
||||||
# it under the terms of the GNU General Public License as published by
|
# it under the terms of the GNU General Public License as published by
|
||||||
|
|
@ -5263,7 +5331,7 @@ sub max(@) {
|
||||||
return $max;
|
return $max;
|
||||||
}
|
}
|
||||||
|
|
||||||
sub sum() {
|
sub sum(@) {
|
||||||
# Returns:
|
# Returns:
|
||||||
# Sum of values of array
|
# Sum of values of array
|
||||||
my @args = @_;
|
my @args = @_;
|
||||||
|
|
@ -10397,7 +10465,7 @@ sub push($) {
|
||||||
push @{$self->{'arg_list'}}, $record;
|
push @{$self->{'arg_list'}}, $record;
|
||||||
# Make @arg available for {= =}
|
# Make @arg available for {= =}
|
||||||
*Arg::arg = $self->{'arg_list_flat_orig'};
|
*Arg::arg = $self->{'arg_list_flat_orig'};
|
||||||
|
|
||||||
my $quote_arg = ($Global::quote_replace and not $Global::quoting);
|
my $quote_arg = ($Global::quote_replace and not $Global::quoting);
|
||||||
for my $perlexpr (keys %{$self->{'replacecount'}}) {
|
for my $perlexpr (keys %{$self->{'replacecount'}}) {
|
||||||
if($perlexpr =~ /^(\d+) /) {
|
if($perlexpr =~ /^(\d+) /) {
|
||||||
|
|
|
||||||
|
|
@ -562,6 +562,21 @@ jobslots process a large amount of data.
|
||||||
See B<--pipe> and B<--pipepart> for use of this.
|
See B<--pipe> and B<--pipepart> for use of this.
|
||||||
|
|
||||||
|
|
||||||
|
=item B<--blocktimeout> I<duration>
|
||||||
|
|
||||||
|
=item B<--bt> I<duration>
|
||||||
|
|
||||||
|
Time out for reading block when using B<--pipe>. If it takes longer
|
||||||
|
than I<duration> to generate a full block, use the partial block read
|
||||||
|
so far.
|
||||||
|
|
||||||
|
I<duration> must be in whole seconds, but can be expressed as floats
|
||||||
|
postfixed with B<s>, B<m>, B<h>, or B<d> which would multiply the
|
||||||
|
float by 1, 60, 3600, or 86400. Thus these are equivalent: B<--timeout
|
||||||
|
100000> and B<--timeout 1d3.5h16.6m4s>.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
=item B<--cat>
|
=item B<--cat>
|
||||||
|
|
||||||
Create a temporary file with content. Normally B<--pipe>/B<--pipepart>
|
Create a temporary file with content. Normally B<--pipe>/B<--pipepart>
|
||||||
|
|
@ -5157,8 +5172,8 @@ Copyright (C) 2007-10-18 Ole Tange, http://ole.tange.dk
|
||||||
|
|
||||||
Copyright (C) 2008-2010 Ole Tange, http://ole.tange.dk
|
Copyright (C) 2008-2010 Ole Tange, http://ole.tange.dk
|
||||||
|
|
||||||
Copyright (C) 2010-2019 Ole Tange,
|
Copyright (C) 2010-2020 Ole Tange, http://ole.tange.dk and Free
|
||||||
http://ole.tange.dk and Free Software Foundation, Inc.
|
Software Foundation, Inc.
|
||||||
|
|
||||||
Parts of the manual concerning B<xargs> compatibility is inspired by
|
Parts of the manual concerning B<xargs> compatibility is inspired by
|
||||||
the manual of B<xargs> from GNU findutils 4.4.2.
|
the manual of B<xargs> from GNU findutils 4.4.2.
|
||||||
|
|
|
||||||
|
|
@ -2290,10 +2290,69 @@ B<GNU> parallel -j 3 ::: ls df "echo hi"
|
||||||
|
|
||||||
(Last checked: 2019-08)
|
(Last checked: 2019-08)
|
||||||
|
|
||||||
|
=head2 DIFFERENCES BETWEEN rargs AND GNU Parallel
|
||||||
|
|
||||||
|
Summary table (see legend above):
|
||||||
|
I1 - - - - - I7
|
||||||
|
- - M3 M4 - -
|
||||||
|
- O2 O3 - O5 O6 - O8 -
|
||||||
|
E1 - - E4 - - -
|
||||||
|
- - - - - - - - -
|
||||||
|
- -
|
||||||
|
|
||||||
|
B<rargs> has elegant ways of doing named regexp capture and field ranges.
|
||||||
|
|
||||||
|
With GNU B<parallel> you can use B<--rpl> to get a similar
|
||||||
|
functionality as regexp capture gives, and use B<join> and B<@arg> to
|
||||||
|
get the field ranges. But the syntax is longer. This:
|
||||||
|
|
||||||
|
--rpl '{r(\d+)\.\.(\d+)} $_=join"$opt::colsep",@arg[$$1..$$2]'
|
||||||
|
|
||||||
|
would make it possible to use:
|
||||||
|
|
||||||
|
{1r3..6}
|
||||||
|
|
||||||
|
for field 3..6.
|
||||||
|
|
||||||
|
For full support of {n..m:s} including negative numbers use a dynamic
|
||||||
|
replacement string like this:
|
||||||
|
|
||||||
|
|
||||||
|
PARALLEL=--rpl\ \''{r((-?\d+)?)\.\.((-?\d+)?)((:([^}]*))?)}
|
||||||
|
$a = defined $$2 ? $$2 < 0 ? 1+$#arg+$$2 : $$2 : 1;
|
||||||
|
$b = defined $$4 ? $$4 < 0 ? 1+$#arg+$$4 : $$4 : $#arg+1;
|
||||||
|
$s = defined $$6 ? $$7 : " ";
|
||||||
|
$_ = join $s,@arg[$a..$b]'\'
|
||||||
|
export PARALLEL
|
||||||
|
|
||||||
|
You can then do:
|
||||||
|
|
||||||
|
head /etc/passwd | parallel --colsep : echo ..={1r..} ..3={1r..3} 4..={1r4..} 2..4={1r2..4} 3..3={1r3..3} ..3:-={1r..3:-} ..3:/={1r..3:/} -1={-1} -5={-5} -6={-6} -3..={1r-3..}
|
||||||
|
|
||||||
|
|
||||||
|
=head3 EXAMPLES FROM rargs MANUAL
|
||||||
|
|
||||||
|
ls *.bak | rargs -p '(.*)\.bak' mv {0} {1}
|
||||||
|
ls *.bak | parallel mv {} {.}
|
||||||
|
|
||||||
|
cat download-list.csv | rargs -p '(?P<url>.*),(?P<filename>.*)' wget {url} -O {filename}
|
||||||
|
cat download-list.csv | parallel --csv wget {1} -O {2}
|
||||||
|
# or use regexps:
|
||||||
|
cat download-list.csv |
|
||||||
|
parallel --rpl '{url} s/,.*//' --rpl '{filename} s/.*?,//' wget {url} -O {filename}
|
||||||
|
|
||||||
|
cat /etc/passwd | rargs -d: echo -e 'id: "{1}"\t name: "{5}"\t rest: "{6..::}"'
|
||||||
|
cat /etc/passwd |
|
||||||
|
parallel -q --colsep : echo -e 'id: "{1}"\t name: "{5}"\t rest: "{=6 $_=join":",@arg[6..$#arg]=}"'
|
||||||
|
|
||||||
|
https://github.com/lotabout/rargs (Last checked: 2020-01)
|
||||||
|
|
||||||
|
|
||||||
=head2 Todo
|
=head2 Todo
|
||||||
|
|
||||||
Url for spread
|
https://github.com/jreisinger/runp
|
||||||
|
|
||||||
|
https://github.com/JeiKeiLim/simple_distribute_job
|
||||||
|
|
||||||
https://github.com/reggi/pkgrun
|
https://github.com/reggi/pkgrun
|
||||||
|
|
||||||
|
|
@ -2557,7 +2616,7 @@ Copyright (C) 2007-10-18 Ole Tange, http://ole.tange.dk
|
||||||
|
|
||||||
Copyright (C) 2008-2010 Ole Tange, http://ole.tange.dk
|
Copyright (C) 2008-2010 Ole Tange, http://ole.tange.dk
|
||||||
|
|
||||||
Copyright (C) 2010-2019 Ole Tange, http://ole.tange.dk and Free
|
Copyright (C) 2010-2020 Ole Tange, http://ole.tange.dk and Free
|
||||||
Software Foundation, Inc.
|
Software Foundation, Inc.
|
||||||
|
|
||||||
Parts of the manual concerning B<xargs> compatibility is inspired by
|
Parts of the manual concerning B<xargs> compatibility is inspired by
|
||||||
|
|
|
||||||
|
|
@ -2998,9 +2998,9 @@ When asking for help, always report the full output of this:
|
||||||
|
|
||||||
Output:
|
Output:
|
||||||
|
|
||||||
GNU parallel 20180122
|
GNU parallel 20200122
|
||||||
Copyright (C) 2007-2018
|
Copyright (C) 2007-2020 Ole Tange, http://ole.tange.dk and Free Software
|
||||||
Ole Tange and Free Software Foundation, Inc.
|
Foundation, Inc.
|
||||||
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
|
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
|
||||||
This is free software: you are free to change and redistribute it.
|
This is free software: you are free to change and redistribute it.
|
||||||
GNU parallel comes with no warranty.
|
GNU parallel comes with no warranty.
|
||||||
|
|
@ -3160,7 +3160,7 @@ https://my.fsf.org/donate/
|
||||||
|
|
||||||
=back
|
=back
|
||||||
|
|
||||||
(C) 2013-2019 Ole Tange, FDLv1.3 (See fdl.txt)
|
(C) 2013-2020 Ole Tange, FDLv1.3 (See fdl.txt)
|
||||||
|
|
||||||
|
|
||||||
=cut
|
=cut
|
||||||
|
|
|
||||||
|
|
@ -60,7 +60,7 @@ GNU B<parcat> is part of GNU B<parallel>. Report bugs to
|
||||||
|
|
||||||
=head1 AUTHOR
|
=head1 AUTHOR
|
||||||
|
|
||||||
Copyright (C) 2016-2019 Ole Tange, http://ole.tange.dk and Free
|
Copyright (C) 2016-2020 Ole Tange, http://ole.tange.dk and Free
|
||||||
Software Foundation, Inc.
|
Software Foundation, Inc.
|
||||||
|
|
||||||
=head1 LICENSE
|
=head1 LICENSE
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,7 @@
|
||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
|
|
||||||
# Copyright (C) 2016-2019 Ole Tange, http://ole.tange.dk and
|
# Copyright (C) 2016-2020 Ole Tange, http://ole.tange.dk and Free
|
||||||
# Free Software Foundation, Inc.
|
# Software Foundation, Inc.
|
||||||
#
|
#
|
||||||
# This program is free software; you can redistribute it and/or modify
|
# This program is free software; you can redistribute it and/or modify
|
||||||
# it under the terms of the GNU General Public License as published by
|
# it under the terms of the GNU General Public License as published by
|
||||||
|
|
|
||||||
|
|
@ -190,7 +190,7 @@ Copyright (C) 2007-10-18 Ole Tange, http://ole.tange.dk
|
||||||
|
|
||||||
Copyright (C) 2008-2010 Ole Tange, http://ole.tange.dk
|
Copyright (C) 2008-2010 Ole Tange, http://ole.tange.dk
|
||||||
|
|
||||||
Copyright (C) 2010-2019 Ole Tange, http://ole.tange.dk and Free
|
Copyright (C) 2010-2020 Ole Tange, http://ole.tange.dk and Free
|
||||||
Software Foundation, Inc.
|
Software Foundation, Inc.
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -244,7 +244,7 @@ Report bugs to <bug-parallel@gnu.org>.
|
||||||
|
|
||||||
=head1 AUTHOR
|
=head1 AUTHOR
|
||||||
|
|
||||||
Copyright (C) 2010-2019 Ole Tange, http://ole.tange.dk and Free
|
Copyright (C) 2010-2020 Ole Tange, http://ole.tange.dk and Free
|
||||||
Software Foundation, Inc.
|
Software Foundation, Inc.
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
2
src/sql
2
src/sql
|
|
@ -341,7 +341,7 @@ Databases Using DBURLs, ;login: The USENIX Magazine, April 2011:29-32.
|
||||||
|
|
||||||
Copyright (C) 2008-2010 Ole Tange http://ole.tange.dk
|
Copyright (C) 2008-2010 Ole Tange http://ole.tange.dk
|
||||||
|
|
||||||
Copyright (C) 2010-2019 Ole Tange, http://ole.tange.dk and Free
|
Copyright (C) 2010-2020 Ole Tange, http://ole.tange.dk and Free
|
||||||
Software Foundation, Inc.
|
Software Foundation, Inc.
|
||||||
|
|
||||||
=head1 LICENSE
|
=head1 LICENSE
|
||||||
|
|
|
||||||
|
|
@ -57,13 +57,16 @@ add_server_to_hosts() {
|
||||||
fi
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
|
shellsplus() {
|
||||||
|
shells="bash sh csh ash dash tcsh zsh ksh fish fizsh mksh posh rc sash yash nopathbash nopathcsh"
|
||||||
|
shellsplus="parallel $shells"
|
||||||
|
parallel -k echo ::: $shellsplus
|
||||||
|
}
|
||||||
|
|
||||||
create_shell_logins() {
|
create_shell_logins() {
|
||||||
# SHELLS
|
# SHELLS
|
||||||
echo '# Create shell logins'
|
echo '# Create shell logins'
|
||||||
touch ~/.zshrc
|
touch ~/.zshrc
|
||||||
SSHPASS=`goodpasswd`
|
|
||||||
export SSHPASS
|
|
||||||
shells="bash sh csh ash dash tcsh zsh ksh fish fizsh mksh posh rc sash yash nopathbash nopathcsh"
|
|
||||||
del_add_user() {
|
del_add_user() {
|
||||||
shell="$1"
|
shell="$1"
|
||||||
append-if-not-exists() {
|
append-if-not-exists() {
|
||||||
|
|
@ -77,23 +80,34 @@ create_shell_logins() {
|
||||||
echo "$string" | sudo tee -a "$file"
|
echo "$string" | sudo tee -a "$file"
|
||||||
fi
|
fi
|
||||||
}
|
}
|
||||||
|
SSHPASS=`goodpasswd`
|
||||||
|
export SSHPASS
|
||||||
append-if-not-exists /etc/shells $(which $shell || which ${shell#"nopath"})
|
append-if-not-exists /etc/shells $(which $shell || which ${shell#"nopath"})
|
||||||
sudo deluser $shell && sudo mv /home/$shell /tmp/$shell.$RANDOM
|
sudo deluser $shell && sudo mv /home/$shell /tmp/$shell.$RANDOM
|
||||||
sudo groupdel $shell
|
sudo groupdel $shell
|
||||||
if echo $shell | grep -q parallel; then
|
if echo $shell | grep -q parallel; then
|
||||||
|
# User parallel should have /bin/bash
|
||||||
loginshell=/bin/bash
|
loginshell=/bin/bash
|
||||||
else
|
else
|
||||||
|
# Other users should have `which $shell` with nopath removed
|
||||||
loginshell=$(which $shell || which ${shell#"nopath"})
|
loginshell=$(which $shell || which ${shell#"nopath"})
|
||||||
fi
|
fi
|
||||||
sudo adduser --shell $loginshell --disabled-password --gecos "$shell for parallel,,," $shell &&
|
sudo adduser --shell $loginshell --disabled-password --gecos "$shell for parallel,,," $shell &&
|
||||||
echo "$shell:$SSHPASS" | sudo chpasswd
|
echo "$shell:$SSHPASS" | sudo chpasswd &&
|
||||||
|
sshpass -e ssh-copy-id $shell@lo &&
|
||||||
|
echo "$shell created"
|
||||||
}
|
}
|
||||||
export -f del_add_user
|
export -f del_add_user
|
||||||
|
|
||||||
|
echo '# (Re-)create user'
|
||||||
|
# Racecondition: if multiple adds a group it will the same group ID
|
||||||
|
shellsplus | parallel --timeout 15 --retries 5 --tag -j1 del_add_user
|
||||||
|
}
|
||||||
|
|
||||||
|
copy_ssh_keys() {
|
||||||
make_sshkey() {
|
make_sshkey() {
|
||||||
shell="$1"
|
shell="$1"
|
||||||
sshpass -e ssh-copy-id $shell@lo
|
echo Add server keys for lo and server &&
|
||||||
echo Add server keys for lo and server &&
|
|
||||||
ssh $shell@lo 'rm -f .ssh/id_rsa; ssh-keyscan lo >>.ssh/known_hosts; ssh-keyscan server >> .ssh/known_hosts' &&
|
ssh $shell@lo 'rm -f .ssh/id_rsa; ssh-keyscan lo >>.ssh/known_hosts; ssh-keyscan server >> .ssh/known_hosts' &&
|
||||||
echo Do ssh-keygen &&
|
echo Do ssh-keygen &&
|
||||||
echo | ssh -t $shell@lo ssh-keygen -b 1024 -f .ssh/id_rsa &&
|
echo | ssh -t $shell@lo ssh-keygen -b 1024 -f .ssh/id_rsa &&
|
||||||
|
|
@ -109,21 +123,17 @@ create_shell_logins() {
|
||||||
echo
|
echo
|
||||||
}
|
}
|
||||||
export -f make_sshkey
|
export -f make_sshkey
|
||||||
. `which env_parallel.bash`
|
|
||||||
shellsplus="parallel $shells"
|
|
||||||
|
|
||||||
echo '# (Re-)create user'
|
|
||||||
# Racecondition: if multiple adds a group it will the same group ID
|
|
||||||
parallel --timeout 15 --retries 5 --tag -j1 del_add_user ::: $shellsplus
|
|
||||||
echo '# Make and copy sshkey'
|
echo '# Make and copy sshkey'
|
||||||
parallel --timeout 15 --retries 5 --tag -j5 make_sshkey ::: $shellsplus
|
shellsplus | parallel --timeout 20 --retries 5 --tag -j5 make_sshkey
|
||||||
|
|
||||||
ssh_a_to_b() {
|
ssh_a_to_b() {
|
||||||
ssh $1@lo ssh $2@lo echo OK ||
|
ssh $1@lo ssh $2@lo echo OK ||
|
||||||
echo failed && false
|
echo failed && false
|
||||||
}
|
}
|
||||||
export -f ssh_a_to_b
|
export -f ssh_a_to_b
|
||||||
parallel -j8 --timeout 5 --tag ssh_a_to_b {} {} ::: $shellsplus
|
echo "# copy id from X to X"
|
||||||
|
shellsplus | parallel --timeout 10 --tag ssh_a_to_b {} {}
|
||||||
|
|
||||||
ssh_copy_id() {
|
ssh_copy_id() {
|
||||||
from="$1"
|
from="$1"
|
||||||
|
|
@ -133,9 +143,9 @@ create_shell_logins() {
|
||||||
export -f ssh_copy_id
|
export -f ssh_copy_id
|
||||||
|
|
||||||
echo '# copy id from any X to any Y'
|
echo '# copy id from any X to any Y'
|
||||||
parallel -u --bar -j3 --timeout 3 --retries 10 --tag ssh_copy_id {1}@lo {2}@lo ::: $shellsplus ::: $shellsplus
|
parallel -u --bar -j3 --timeout 3 --retries 10 --tag ssh_copy_id {1}@lo {2}@lo ::: $(shellsplus) ::: $(shellsplus)
|
||||||
echo '# Test the copying went well'
|
echo '# Test the copying went well'
|
||||||
parallel -j3 --timeout 3 --retries 10 --tag ssh_a_to_b ::: $shellsplus ::: $shellsplus
|
parallel -j2 --timeout 10 --retries 10 --tag ssh_a_to_b ::: $(shellsplus) ::: $(shellsplus)
|
||||||
|
|
||||||
echo '# change paths to no path'
|
echo '# change paths to no path'
|
||||||
(
|
(
|
||||||
|
|
@ -221,6 +231,7 @@ install_packages &&
|
||||||
setup_databases &&
|
setup_databases &&
|
||||||
add_server_to_hosts &&
|
add_server_to_hosts &&
|
||||||
create_shell_logins &&
|
create_shell_logins &&
|
||||||
|
copy_ssh_keys &&
|
||||||
lsh_setup &&
|
lsh_setup &&
|
||||||
add_freebsd &&
|
add_freebsd &&
|
||||||
misc
|
misc
|
||||||
|
|
|
||||||
|
|
@ -4,7 +4,6 @@
|
||||||
# Each should be taking 0.3-1s and be possible to run in parallel
|
# Each should be taking 0.3-1s and be possible to run in parallel
|
||||||
# I.e.: No race conditions, no logins
|
# I.e.: No race conditions, no logins
|
||||||
|
|
||||||
|
|
||||||
stdsort() {
|
stdsort() {
|
||||||
"$@" 2>&1 | LC_ALL=C sort;
|
"$@" 2>&1 | LC_ALL=C sort;
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -4,6 +4,201 @@
|
||||||
# Each should be taking 1-3s and be possible to run in parallel
|
# Each should be taking 1-3s and be possible to run in parallel
|
||||||
# I.e.: No race conditions, no logins
|
# I.e.: No race conditions, no logins
|
||||||
|
|
||||||
|
par_bug43654() {
|
||||||
|
echo "bug #43654: --bar with command not using {} - only last output line "
|
||||||
|
COLUMNS=80 stdout parallel --bar true {.} ::: 1 | perl -pe 's/.*\r/\r/'
|
||||||
|
}
|
||||||
|
|
||||||
|
par_replacement_rename() {
|
||||||
|
echo "### Test --basenamereplace"
|
||||||
|
parallel -j1 -k -X --basenamereplace FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
||||||
|
parallel -k --basenamereplace FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
||||||
|
|
||||||
|
echo "### Test --bnr"
|
||||||
|
parallel -j1 -k -X --bnr FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
||||||
|
parallel -k --bnr FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
||||||
|
|
||||||
|
echo "### Test --extensionreplace"
|
||||||
|
parallel -j1 -k -X --extensionreplace FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
||||||
|
parallel -k --extensionreplace FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
||||||
|
|
||||||
|
echo "### Test --er"
|
||||||
|
parallel -j1 -k -X --er FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
||||||
|
parallel -k --er FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
||||||
|
|
||||||
|
echo "### Test --basenameextensionreplace"
|
||||||
|
parallel -j1 -k -X --basenameextensionreplace FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
||||||
|
parallel -k --basenameextensionreplace FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
||||||
|
|
||||||
|
echo "### Test --bner"
|
||||||
|
parallel -j1 -k -X --bner FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
||||||
|
parallel -k --bner FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
||||||
|
}
|
||||||
|
|
||||||
|
par_replacement_strings() {
|
||||||
|
echo "### Test {/}"
|
||||||
|
parallel -j1 -k -X echo {/} ::: /a/b.c a/b.c b.c /a/b a/b b
|
||||||
|
|
||||||
|
echo "### Test {/.}"
|
||||||
|
parallel -j1 -k -X echo {/.} ::: /a/b.c a/b.c b.c /a/b a/b b
|
||||||
|
|
||||||
|
echo "### Test {#/.}"
|
||||||
|
parallel -j1 -k -X echo {2/.} ::: /a/number1.c a/number2.c number3.c /a/number4 a/number5 number6
|
||||||
|
|
||||||
|
echo "### Test {#/}"
|
||||||
|
parallel -j1 -k -X echo {2/} ::: /a/number1.c a/number2.c number3.c /a/number4 a/number5 number6
|
||||||
|
|
||||||
|
echo "### Test {#.}"
|
||||||
|
parallel -j1 -k -X echo {2.} ::: /a/number1.c a/number2.c number3.c /a/number4 a/number5 number6
|
||||||
|
}
|
||||||
|
|
||||||
|
par_bug34241() {
|
||||||
|
echo "### bug #34241: --pipe should not spawn unneeded processes"
|
||||||
|
echo | parallel -r -j2 -N1 --pipe md5sum -c && echo OK
|
||||||
|
}
|
||||||
|
|
||||||
|
par_test_gt_quoting() {
|
||||||
|
echo '### Test of quoting of > bug'
|
||||||
|
echo '>/dev/null' | parallel echo
|
||||||
|
|
||||||
|
echo '### Test of quoting of > bug if line continuation'
|
||||||
|
(echo '> '; echo '> '; echo '>') | parallel --max-lines 3 echo
|
||||||
|
}
|
||||||
|
|
||||||
|
par_trim_illegal_value() {
|
||||||
|
echo '### Test of --trim illegal'
|
||||||
|
stdout parallel --trim fj ::: echo
|
||||||
|
}
|
||||||
|
|
||||||
|
par_eof_on_command_line_input_source() {
|
||||||
|
echo '### Test of eof string on :::'
|
||||||
|
parallel -k -E ole echo ::: foo ole bar
|
||||||
|
}
|
||||||
|
|
||||||
|
par_empty_string_command_line() {
|
||||||
|
echo '### Test of ignore-empty string on :::'
|
||||||
|
parallel -k -r echo ::: foo '' ole bar
|
||||||
|
}
|
||||||
|
|
||||||
|
par_trailing_space_line_continuation() {
|
||||||
|
echo '### Test of trailing space continuation'
|
||||||
|
(echo foo; echo '';echo 'ole ';echo bar;echo quux) | xargs -r -L2 echo
|
||||||
|
(echo foo; echo '';echo 'ole ';echo bar;echo quux) | parallel -kr -L2 echo
|
||||||
|
parallel -kr -L2 echo ::: foo '' 'ole ' bar quux
|
||||||
|
|
||||||
|
echo '### Test of trailing space continuation with -E eof'
|
||||||
|
(echo foo; echo '';echo 'ole ';echo bar;echo quux) | xargs -r -L2 -E bar echo
|
||||||
|
(echo foo; echo '';echo 'ole ';echo bar;echo quux) | parallel -kr -L2 -E bar echo
|
||||||
|
parallel -kr -L2 -E bar echo ::: foo '' 'ole ' bar quux
|
||||||
|
}
|
||||||
|
|
||||||
|
par_colsep() {
|
||||||
|
echo '### Test of --colsep'
|
||||||
|
echo 'a%c%b' | parallel --colsep % echo {1} {3} {2}
|
||||||
|
(echo 'a%c%b'; echo a%c%b%d) | parallel -k --colsep % echo {1} {3} {2} {4}
|
||||||
|
(echo a%c%b; echo d%f%e) | parallel -k --colsep % echo {1} {3} {2}
|
||||||
|
parallel -k --colsep % echo {1} {3} {2} ::: a%c%b d%f%e
|
||||||
|
parallel -k --colsep % echo {1} {3} {2} ::: a%c%b
|
||||||
|
parallel -k --colsep % echo {1} {3} {2} {4} ::: a%c%b a%c%b%d
|
||||||
|
|
||||||
|
|
||||||
|
echo '### Test of tab as colsep'
|
||||||
|
printf 'def\tabc\njkl\tghi' | parallel -k --colsep '\t' echo {2} {1}
|
||||||
|
parallel -k -a <(printf 'def\tabc\njkl\tghi') --colsep '\t' echo {2} {1}
|
||||||
|
|
||||||
|
echo '### Test of multiple -a plus colsep'
|
||||||
|
parallel --xapply -k -a <(printf 'def\njkl\n') -a <(printf 'abc\tghi\nmno\tpqr') --colsep '\t' echo {2} {1}
|
||||||
|
|
||||||
|
echo '### Test of multiple -a no colsep'
|
||||||
|
parallel --xapply -k -a <(printf 'ghi\npqr\n') -a <(printf 'abc\tdef\njkl\tmno') echo {2} {1}
|
||||||
|
|
||||||
|
echo '### Test of quoting after colsplit'
|
||||||
|
parallel --colsep % echo {2} {1} ::: '>/dev/null%>/tmp/null'
|
||||||
|
|
||||||
|
echo '### Test of --colsep as regexp'
|
||||||
|
(echo 'a%c%%b'; echo a%c%b%d) | parallel -k --colsep %+ echo {1} {3} {2} {4}
|
||||||
|
parallel -k --colsep %+ echo {1} {3} {2} {4} ::: a%c%%b a%c%b%d
|
||||||
|
(echo 'a% c %%b'; echo a%c% b %d) | parallel -k --colsep %+ echo {1} {3} {2} {4}
|
||||||
|
(echo 'a% c %%b'; echo a%c% b %d) | parallel -k --colsep %+ echo '"{1}_{3}_{2}_{4}"'
|
||||||
|
|
||||||
|
echo '### Test of -C'
|
||||||
|
(echo 'a% c %%b'; echo a%c% b %d) | parallel -k -C %+ echo '"{1}_{3}_{2}_{4}"'
|
||||||
|
|
||||||
|
echo '### Test of --trim n'
|
||||||
|
(echo 'a% c %%b'; echo a%c% b %d) | parallel -k --trim n --colsep %+ echo '"{1}_{3}_{2}_{4}"'
|
||||||
|
parallel -k -C %+ echo '"{1}_{3}_{2}_{4}"' ::: 'a% c %%b' 'a%c% b %d'
|
||||||
|
|
||||||
|
echo '### Test of bug: If input is empty string'
|
||||||
|
(echo ; echo abcbdbebf;echo abc) | parallel -k --colsep b -v echo {1}{2}
|
||||||
|
}
|
||||||
|
|
||||||
|
par_mix_triple_colon_with_quad_colon() {
|
||||||
|
echo '### Test :::: mixed with :::'
|
||||||
|
echo '### Test :::: < ::: :::'
|
||||||
|
parallel -k echo {1} {2} {3} :::: <(seq 6 7) ::: 4 5 ::: 1 2 3
|
||||||
|
|
||||||
|
echo '### Test :::: < < :::: <'
|
||||||
|
parallel -k echo {1} {2} {3} :::: <(seq 6 7) <(seq 4 5) :::: <(seq 1 3)
|
||||||
|
|
||||||
|
echo '### Test -a :::: < :::: <'
|
||||||
|
parallel -k -a <(seq 6 7) echo {1} {2} {3} :::: <(seq 4 5) :::: <(seq 1 3)
|
||||||
|
|
||||||
|
echo '### Test -a -a :::'
|
||||||
|
parallel -k -a <(seq 6 7) -a <(seq 4 5) echo {1} {2} {3} ::: 1 2 3
|
||||||
|
|
||||||
|
echo '### Test -a - -a :::'
|
||||||
|
seq 6 7 | parallel -k -a - -a <(seq 4 5) echo {1} {2} {3} ::: 1 2 3
|
||||||
|
|
||||||
|
echo '### Test :::: < - :::'
|
||||||
|
seq 4 5 | parallel -k echo {1} {2} {3} :::: <(seq 6 7) - ::: 1 2 3
|
||||||
|
}
|
||||||
|
|
||||||
|
par_test_E() {
|
||||||
|
echo '### Test -E'
|
||||||
|
seq 1 100 | parallel -k -E 5 echo :::: - ::: 2 3 4 5 6 7 8 9 10 :::: <(seq 3 11)
|
||||||
|
|
||||||
|
echo '### Test -E one empty'
|
||||||
|
seq 1 100 | parallel -k -E 3 echo :::: - ::: 2 3 4 5 6 7 8 9 10 :::: <(seq 3 11)
|
||||||
|
|
||||||
|
echo '### Test -E 2 empty'
|
||||||
|
seq 1 100 | parallel -k -E 3 echo :::: - ::: 3 4 5 6 7 8 9 10 :::: <(seq 3 11)
|
||||||
|
|
||||||
|
echo '### Test -E all empty'
|
||||||
|
seq 3 100 | parallel -k -E 3 echo :::: - ::: 3 4 5 6 7 8 9 10 :::: <(seq 3 11)
|
||||||
|
}
|
||||||
|
|
||||||
|
par_test_job_number() {
|
||||||
|
echo '### Test {#}'
|
||||||
|
seq 1 10 | parallel -k echo {#}
|
||||||
|
}
|
||||||
|
|
||||||
|
par_seqreplace_long_line() {
|
||||||
|
echo '### Test --seqreplace and line too long'
|
||||||
|
seq 1 1000 | stdout parallel -j1 -s 210 -k --seqreplace I echo IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII \|wc | uniq -c
|
||||||
|
}
|
||||||
|
|
||||||
|
par_bug37042() {
|
||||||
|
echo '### bug #37042: -J foo is taken from the whole command line - not just the part before the command'
|
||||||
|
echo '--tagstring foo' > ~/.parallel/bug_37042_profile;
|
||||||
|
parallel -J bug_37042_profile echo ::: tag_with_foo;
|
||||||
|
parallel --tagstring a -J bug_37042_profile echo ::: tag_with_a;
|
||||||
|
parallel --tagstring a echo -J bug_37042_profile ::: print_-J_bug_37042_profile
|
||||||
|
|
||||||
|
echo '### Bug introduce by fixing bug #37042'
|
||||||
|
parallel --xapply -a <(printf 'abc') --colsep '\t' echo {1}
|
||||||
|
}
|
||||||
|
|
||||||
|
par_header() {
|
||||||
|
echo "### Test --header with -N"
|
||||||
|
(echo h1; echo h2; echo 1a;echo 1b; echo 2a;echo 2b; echo 3a)| parallel -j1 --pipe -N2 -k --header '.*\n.*\n' echo Start\;cat \; echo Stop
|
||||||
|
|
||||||
|
echo "### Test --header with --block 1k"
|
||||||
|
(echo h1; echo h2; perl -e '$a="x"x110;for(1..22){print $_,$a,"\n"}') |
|
||||||
|
parallel -j1 --pipe -k --block 1k --header '.*\n.*\n' echo Start\;cat \; echo Stop
|
||||||
|
|
||||||
|
echo "### Test --header with multiple :::"
|
||||||
|
parallel --header : echo {a} {b} {1} {2} ::: b b1 ::: a a2
|
||||||
|
}
|
||||||
|
|
||||||
par_profiles_with_space() {
|
par_profiles_with_space() {
|
||||||
echo '### bug #42902: profiles containing arguments with space'
|
echo '### bug #42902: profiles containing arguments with space'
|
||||||
|
|
|
||||||
|
|
@ -8,7 +8,8 @@ par_tee_with_premature_close() {
|
||||||
echo '--tee --pipe should send all data to all commands'
|
echo '--tee --pipe should send all data to all commands'
|
||||||
echo 'even if a command closes stdin before reading everything'
|
echo 'even if a command closes stdin before reading everything'
|
||||||
echo 'tee with --output-error=warn-nopipe support'
|
echo 'tee with --output-error=warn-nopipe support'
|
||||||
seq 1000000 | parallel -k --tee --pipe ::: wc head tail 'sleep 1'
|
correct="$(seq 1000000 | parallel -k --tee --pipe ::: wc head tail 'sleep 1')"
|
||||||
|
echo "$correct"
|
||||||
echo 'tee without --output-error=warn-nopipe support'
|
echo 'tee without --output-error=warn-nopipe support'
|
||||||
cat > tmp/tee <<-EOF
|
cat > tmp/tee <<-EOF
|
||||||
#!/usr/bin/perl
|
#!/usr/bin/perl
|
||||||
|
|
@ -24,8 +25,12 @@ par_tee_with_premature_close() {
|
||||||
# * tee not supporting --output-error=warn-nopipe
|
# * tee not supporting --output-error=warn-nopipe
|
||||||
# * sleep closes stdin before EOF
|
# * sleep closes stdin before EOF
|
||||||
# Depending on tee it may provide partial output or no output
|
# Depending on tee it may provide partial output or no output
|
||||||
seq 1000000 | parallel -k --tee --pipe ::: wc head tail 'sleep 1'
|
wrong="$(seq 1000000 | parallel -k --tee --pipe ::: wc head tail 'sleep 1')"
|
||||||
echo
|
if diff <(echo "$correct") <(echo "$wrong") >/dev/null; then
|
||||||
|
echo Wrong: They should not give the same output
|
||||||
|
else
|
||||||
|
echo OK
|
||||||
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
par_maxargs() {
|
par_maxargs() {
|
||||||
|
|
@ -170,6 +175,7 @@ par_resume_failed_k() {
|
||||||
parallel -k --resume-failed --joblog $tmp echo job{#} val {}\;exit {} ::: 0 1 2 3 0 1
|
parallel -k --resume-failed --joblog $tmp echo job{#} val {}\;exit {} ::: 0 1 2 3 0 1
|
||||||
echo with exit 0
|
echo with exit 0
|
||||||
parallel -k --resume-failed --joblog $tmp echo job{#} val {}\;exit 0 ::: 0 1 2 3 0 1
|
parallel -k --resume-failed --joblog $tmp echo job{#} val {}\;exit 0 ::: 0 1 2 3 0 1
|
||||||
|
sleep 0.5
|
||||||
echo try 2 again. Gives empty
|
echo try 2 again. Gives empty
|
||||||
parallel -k --resume-failed --joblog $tmp echo job{#} val {}\;exit {} ::: 0 1 2 3 0 1
|
parallel -k --resume-failed --joblog $tmp echo job{#} val {}\;exit {} ::: 0 1 2 3 0 1
|
||||||
rm $tmp
|
rm $tmp
|
||||||
|
|
|
||||||
|
|
@ -47,9 +47,9 @@ par_distribute_input_by_ability() {
|
||||||
echo "### Distribute input to jobs that are ready"
|
echo "### Distribute input to jobs that are ready"
|
||||||
echo "Job-slot n is 50% slower than n+1, so the order should be 1..7"
|
echo "Job-slot n is 50% slower than n+1, so the order should be 1..7"
|
||||||
seq 20000000 |
|
seq 20000000 |
|
||||||
parallel --tagstring {#} -j7 --block 300k --round-robin --pipe \
|
parallel --tagstring {#} -j7 --block 300k --round-robin --pipe \
|
||||||
'pv -qL{=$_=$job->seq()**3+9=}0000 |wc -c' |
|
'pv -qL{=$_=$job->seq()**3+9=}0000 |wc -c' |
|
||||||
sort -nk2 | field 1
|
sort -nk2 | field 1
|
||||||
}
|
}
|
||||||
|
|
||||||
par_print_before_halt_on_error() {
|
par_print_before_halt_on_error() {
|
||||||
|
|
@ -69,6 +69,29 @@ par_print_before_halt_on_error() {
|
||||||
parallel -j0 -k --tag mytest ::: -2 -1 0 1 2
|
parallel -j0 -k --tag mytest ::: -2 -1 0 1 2
|
||||||
}
|
}
|
||||||
|
|
||||||
|
par_bug56403() {
|
||||||
|
echo 'bug #56403: --pipe block by time.'
|
||||||
|
(
|
||||||
|
sleep 2; # make sure parallel is ready
|
||||||
|
echo job1a;
|
||||||
|
sleep 2;
|
||||||
|
echo job2b;
|
||||||
|
echo -n job3c;
|
||||||
|
sleep 2;
|
||||||
|
echo job3d
|
||||||
|
) | parallel --blocktimeout 1 -j1 --pipe --tagstring {#} -k cat
|
||||||
|
(
|
||||||
|
sleep 2; # make sure parallel is ready
|
||||||
|
echo job1a;
|
||||||
|
sleep 1;
|
||||||
|
echo job1b;
|
||||||
|
echo -n job2c;
|
||||||
|
sleep 4;
|
||||||
|
echo job2d
|
||||||
|
) | parallel --blocktimeout 3 -j1 --pipe --tagstring {#} -k cat
|
||||||
|
|
||||||
|
}
|
||||||
|
|
||||||
export -f $(compgen -A function | grep par_)
|
export -f $(compgen -A function | grep par_)
|
||||||
compgen -A function | grep par_ | sort |
|
compgen -A function | grep par_ | sort |
|
||||||
# parallel --joblog /tmp/jl-`basename $0` -j10 --tag -k '{} 2>&1'
|
# parallel --joblog /tmp/jl-`basename $0` -j10 --tag -k '{} 2>&1'
|
||||||
|
|
|
||||||
|
|
@ -1707,14 +1707,16 @@ _EOF
|
||||||
par_bash_environment_too_big() {
|
par_bash_environment_too_big() {
|
||||||
myscript=$(cat <<'_EOF'
|
myscript=$(cat <<'_EOF'
|
||||||
echo 'bug #50815: env_parallel should warn if the environment is too big'
|
echo 'bug #50815: env_parallel should warn if the environment is too big'
|
||||||
len_var=100+50-25-12+6-3+2-1
|
len_functions=-$(typeset -f | wc -c)/1000
|
||||||
len_var_remote=100-50+25+12-6+3
|
len_variables=-$(typeset -p | wc -c)/1000
|
||||||
len_var_quote=100-50+25-12-6+3-2
|
len_var=$len_variables+100
|
||||||
len_var_quote_remote=100-50-25+12+6-3
|
len_var_remote=$len_variables+100-50+25-12+6
|
||||||
len_fun=100+50-25-12+6-3
|
len_var_quote=$len_variables+100-50
|
||||||
len_fun_remote=100-50+25+12-6+3
|
len_var_quote_remote=$len_variables+100-50-25+12
|
||||||
len_fun_quote=100+50-25-12
|
len_fun=$len_functions+100
|
||||||
len_fun_quote_remote=100-50+25+12-6
|
len_fun_remote=$len_functions+100-50+25-12+6
|
||||||
|
len_fun_quote=$len_functions+100
|
||||||
|
len_fun_quote_remote=$len_functions+100-50
|
||||||
|
|
||||||
. `which env_parallel.bash`;
|
. `which env_parallel.bash`;
|
||||||
|
|
||||||
|
|
@ -1856,14 +1858,16 @@ par_fish_environment_too_big() {
|
||||||
par_ksh_environment_too_big() {
|
par_ksh_environment_too_big() {
|
||||||
myscript=$(cat <<'_EOF'
|
myscript=$(cat <<'_EOF'
|
||||||
echo 'bug #50815: env_parallel should warn if the environment is too big'
|
echo 'bug #50815: env_parallel should warn if the environment is too big'
|
||||||
len_var=100+50-25-12
|
len_functions=-$(functions|wc -c)/1000
|
||||||
len_var_remote=100-50+25+12-6-3
|
len_variables=-$(typeset -p | wc -c)/1000
|
||||||
len_var_quote=100+50-25-12
|
len_var=$len_variables+100
|
||||||
len_var_quote_remote=100-50+25+12-6
|
len_var_remote=$len_variables+100-50+25-12+6
|
||||||
len_fun=100+12+6-3
|
len_var_quote=$len_variables+100
|
||||||
len_fun_remote=100-50+25+12-6+3
|
len_var_quote_remote=$len_variables+100-50+25
|
||||||
len_fun_quote=100+50-25-12
|
len_fun=$len_functions+100
|
||||||
len_fun_quote_remote=100-50+25+12-6
|
len_fun_remote=$len_functions+100-50+25-12+6
|
||||||
|
len_fun_quote=$len_functions+100
|
||||||
|
len_fun_quote_remote=$len_functions+100-50
|
||||||
|
|
||||||
. `which env_parallel.ksh`;
|
. `which env_parallel.ksh`;
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -88,25 +88,34 @@ par_bar_m() {
|
||||||
uniq
|
uniq
|
||||||
}
|
}
|
||||||
|
|
||||||
|
retries() {
|
||||||
|
retries=$1
|
||||||
|
min=$2
|
||||||
|
max=$3
|
||||||
|
export PARALLEL="--retries $retries -S 12/localhost,1/:,parallel@lo -uq"
|
||||||
|
tries=$(seq 0 12 |
|
||||||
|
parallel perl -e 'print "job{}\n";exit({})' 2>/dev/null |
|
||||||
|
wc -l)
|
||||||
|
# Dont care if they are off by one
|
||||||
|
if [ $min -le $tries -o $tries -le $max ] ; then
|
||||||
|
echo OK
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
export -f retries
|
||||||
|
|
||||||
par_retries_1() {
|
par_retries_1() {
|
||||||
echo '### Test of --retries - it should run 13 jobs in total';
|
echo '### Test of --retries - it should run 13 jobs in total';
|
||||||
export PARALLEL="--retries 1 -S 12/localhost,1/:,parallel@lo -uq"
|
retries 1 11 13
|
||||||
seq 0 12 |
|
|
||||||
parallel perl -e 'print "job{}\n";exit({})' 2>/dev/null | wc -l
|
|
||||||
}
|
}
|
||||||
|
|
||||||
par_retries_2() {
|
par_retries_2() {
|
||||||
echo '### Test of --retries - it should run 25 jobs in total';
|
echo '### Test of --retries - it should run 25 jobs in total';
|
||||||
export PARALLEL="--retries 2 -S 12/localhost,1/:,parallel@lo -uq"
|
retries 2 24 25
|
||||||
seq 0 12 |
|
|
||||||
parallel perl -e 'print "job{}\n";exit({})' 2>/dev/null | wc -l
|
|
||||||
}
|
}
|
||||||
|
|
||||||
par_retries_4() {
|
par_retries_4() {
|
||||||
echo '### Test of --retries - it should run 49 jobs in total';
|
echo '### Test of --retries - it should run 49 jobs in total';
|
||||||
export PARALLEL="--retries 4 -S 12/localhost,1/:,parallel@lo -uq"
|
retries 4 48 49
|
||||||
seq 0 12 |
|
|
||||||
parallel perl -e 'print "job{}\n";exit({})' 2>/dev/null | wc -l
|
|
||||||
}
|
}
|
||||||
|
|
||||||
par_retries_bug_from_2010() {
|
par_retries_bug_from_2010() {
|
||||||
|
|
|
||||||
|
|
@ -1,167 +1,7 @@
|
||||||
#!/bin/bash
|
#!/bin/bash
|
||||||
|
|
||||||
cat <<'EOF' | sed -e 's/;$/; /;s/$SERVER1/'$SERVER1'/;s/$SERVER2/'$SERVER2'/' | stdout parallel -vj0 -k --joblog /tmp/jl-`basename $0` -L1 -r
|
cat <<'EOF' | sed -e 's/;$/; /;s/$SERVER1/'$SERVER1'/;s/$SERVER2/'$SERVER2'/' | stdout parallel -vj0 -k --joblog /tmp/jl-`basename $0` -L1 -r
|
||||||
echo "bug #43654: --bar with command not using {} - only last output line "
|
|
||||||
COLUMNS=80 stdout parallel --bar true {.} ::: 1 | perl -pe 's/.*\r/\r/'
|
|
||||||
|
|
||||||
echo "### Test --basenamereplace"
|
|
||||||
parallel -j1 -k -X --basenamereplace FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
|
||||||
parallel -k --basenamereplace FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
|
||||||
|
|
||||||
echo "### Test --bnr"
|
|
||||||
parallel -j1 -k -X --bnr FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
|
||||||
parallel -k --bnr FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
|
||||||
|
|
||||||
echo "### Test --extensionreplace"
|
|
||||||
parallel -j1 -k -X --extensionreplace FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
|
||||||
parallel -k --extensionreplace FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
|
||||||
|
|
||||||
echo "### Test --er"
|
|
||||||
parallel -j1 -k -X --er FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
|
||||||
parallel -k --er FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
|
||||||
|
|
||||||
echo "### Test --basenameextensionreplace"
|
|
||||||
parallel -j1 -k -X --basenameextensionreplace FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
|
||||||
parallel -k --basenameextensionreplace FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
|
||||||
|
|
||||||
echo "### Test --bner"
|
|
||||||
parallel -j1 -k -X --bner FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
|
||||||
parallel -k --bner FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
|
||||||
|
|
||||||
echo "### Test {/}"
|
|
||||||
parallel -j1 -k -X echo {/} ::: /a/b.c a/b.c b.c /a/b a/b b
|
|
||||||
|
|
||||||
echo "### Test {/.}"
|
|
||||||
parallel -j1 -k -X echo {/.} ::: /a/b.c a/b.c b.c /a/b a/b b
|
|
||||||
|
|
||||||
echo "### Test {#/.}"
|
|
||||||
parallel -j1 -k -X echo {2/.} ::: /a/number1.c a/number2.c number3.c /a/number4 a/number5 number6
|
|
||||||
|
|
||||||
echo "### Test {#/}"
|
|
||||||
parallel -j1 -k -X echo {2/} ::: /a/number1.c a/number2.c number3.c /a/number4 a/number5 number6
|
|
||||||
|
|
||||||
echo "### Test {#.}"
|
|
||||||
parallel -j1 -k -X echo {2.} ::: /a/number1.c a/number2.c number3.c /a/number4 a/number5 number6
|
|
||||||
|
|
||||||
echo "### bug #34241: --pipe should not spawn unneeded processes"
|
|
||||||
echo | parallel -r -j2 -N1 --pipe md5sum -c && echo OK
|
|
||||||
|
|
||||||
echo '### Test of quoting of > bug'
|
|
||||||
echo '>/dev/null' | parallel echo
|
|
||||||
|
|
||||||
echo '### Test of quoting of > bug if line continuation'
|
|
||||||
(echo '> '; echo '> '; echo '>') | parallel --max-lines 3 echo
|
|
||||||
|
|
||||||
echo '### Test of --trim illegal'
|
|
||||||
stdout parallel --trim fj ::: echo
|
|
||||||
|
|
||||||
echo '### Test of eof string on :::'
|
|
||||||
parallel -k -E ole echo ::: foo ole bar
|
|
||||||
|
|
||||||
echo '### Test of ignore-empty string on :::'
|
|
||||||
parallel -k -r echo ::: foo '' ole bar
|
|
||||||
|
|
||||||
echo '### Test of trailing space continuation'
|
|
||||||
(echo foo; echo '';echo 'ole ';echo bar;echo quux) | xargs -r -L2 echo
|
|
||||||
(echo foo; echo '';echo 'ole ';echo bar;echo quux) | parallel -kr -L2 echo
|
|
||||||
parallel -kr -L2 echo ::: foo '' 'ole ' bar quux
|
|
||||||
|
|
||||||
echo '### Test of trailing space continuation with -E eof'
|
|
||||||
(echo foo; echo '';echo 'ole ';echo bar;echo quux) | xargs -r -L2 -E bar echo
|
|
||||||
(echo foo; echo '';echo 'ole ';echo bar;echo quux) | parallel -kr -L2 -E bar echo
|
|
||||||
parallel -kr -L2 -E bar echo ::: foo '' 'ole ' bar quux
|
|
||||||
|
|
||||||
echo '### Test of --colsep'
|
|
||||||
echo 'a%c%b' | parallel --colsep % echo {1} {3} {2}
|
|
||||||
(echo 'a%c%b'; echo a%c%b%d) | parallel -k --colsep % echo {1} {3} {2} {4}
|
|
||||||
(echo a%c%b; echo d%f%e) | parallel -k --colsep % echo {1} {3} {2}
|
|
||||||
parallel -k --colsep % echo {1} {3} {2} ::: a%c%b d%f%e
|
|
||||||
parallel -k --colsep % echo {1} {3} {2} ::: a%c%b
|
|
||||||
parallel -k --colsep % echo {1} {3} {2} {4} ::: a%c%b a%c%b%d
|
|
||||||
|
|
||||||
echo '### Test of tab as colsep'
|
|
||||||
printf 'def\tabc\njkl\tghi' | parallel -k --colsep '\t' echo {2} {1}
|
|
||||||
parallel -k -a <(printf 'def\tabc\njkl\tghi') --colsep '\t' echo {2} {1}
|
|
||||||
|
|
||||||
echo '### Test of multiple -a plus colsep'
|
|
||||||
parallel --xapply -k -a <(printf 'def\njkl\n') -a <(printf 'abc\tghi\nmno\tpqr') --colsep '\t' echo {2} {1}
|
|
||||||
|
|
||||||
echo '### Test of multiple -a no colsep'
|
|
||||||
parallel --xapply -k -a <(printf 'ghi\npqr\n') -a <(printf 'abc\tdef\njkl\tmno') echo {2} {1}
|
|
||||||
|
|
||||||
echo '### Test of quoting after colsplit'
|
|
||||||
parallel --colsep % echo {2} {1} ::: '>/dev/null%>/tmp/null'
|
|
||||||
|
|
||||||
echo '### Test of --colsep as regexp'
|
|
||||||
(echo 'a%c%%b'; echo a%c%b%d) | parallel -k --colsep %+ echo {1} {3} {2} {4}
|
|
||||||
parallel -k --colsep %+ echo {1} {3} {2} {4} ::: a%c%%b a%c%b%d
|
|
||||||
(echo 'a% c %%b'; echo a%c% b %d) | parallel -k --colsep %+ echo {1} {3} {2} {4}
|
|
||||||
(echo 'a% c %%b'; echo a%c% b %d) | parallel -k --colsep %+ echo '"{1}_{3}_{2}_{4}"'
|
|
||||||
|
|
||||||
echo '### Test of -C'
|
|
||||||
(echo 'a% c %%b'; echo a%c% b %d) | parallel -k -C %+ echo '"{1}_{3}_{2}_{4}"'
|
|
||||||
|
|
||||||
echo '### Test of --trim n'
|
|
||||||
(echo 'a% c %%b'; echo a%c% b %d) | parallel -k --trim n --colsep %+ echo '"{1}_{3}_{2}_{4}"'
|
|
||||||
parallel -k -C %+ echo '"{1}_{3}_{2}_{4}"' ::: 'a% c %%b' 'a%c% b %d'
|
|
||||||
|
|
||||||
echo '### Test of bug: If input is empty string'
|
|
||||||
(echo ; echo abcbdbebf;echo abc) | parallel -k --colsep b -v echo {1}{2}
|
|
||||||
|
|
||||||
echo '### Test :::: mixed with :::'
|
|
||||||
echo '### Test :::: < ::: :::'
|
|
||||||
parallel -k echo {1} {2} {3} :::: <(seq 6 7) ::: 4 5 ::: 1 2 3
|
|
||||||
|
|
||||||
echo '### Test :::: < < :::: <'
|
|
||||||
parallel -k echo {1} {2} {3} :::: <(seq 6 7) <(seq 4 5) :::: <(seq 1 3)
|
|
||||||
|
|
||||||
echo '### Test -a :::: < :::: <'
|
|
||||||
parallel -k -a <(seq 6 7) echo {1} {2} {3} :::: <(seq 4 5) :::: <(seq 1 3)
|
|
||||||
|
|
||||||
echo '### Test -a -a :::'
|
|
||||||
parallel -k -a <(seq 6 7) -a <(seq 4 5) echo {1} {2} {3} ::: 1 2 3
|
|
||||||
|
|
||||||
echo '### Test -a - -a :::'
|
|
||||||
seq 6 7 | parallel -k -a - -a <(seq 4 5) echo {1} {2} {3} ::: 1 2 3
|
|
||||||
|
|
||||||
echo '### Test :::: < - :::'
|
|
||||||
seq 4 5 | parallel -k echo {1} {2} {3} :::: <(seq 6 7) - ::: 1 2 3
|
|
||||||
|
|
||||||
echo '### Test -E'
|
|
||||||
seq 1 100 | parallel -k -E 5 echo :::: - ::: 2 3 4 5 6 7 8 9 10 :::: <(seq 3 11)
|
|
||||||
|
|
||||||
echo '### Test -E one empty'
|
|
||||||
seq 1 100 | parallel -k -E 3 echo :::: - ::: 2 3 4 5 6 7 8 9 10 :::: <(seq 3 11)
|
|
||||||
|
|
||||||
echo '### Test -E 2 empty'
|
|
||||||
seq 1 100 | parallel -k -E 3 echo :::: - ::: 3 4 5 6 7 8 9 10 :::: <(seq 3 11)
|
|
||||||
|
|
||||||
echo '### Test -E all empty'
|
|
||||||
seq 3 100 | parallel -k -E 3 echo :::: - ::: 3 4 5 6 7 8 9 10 :::: <(seq 3 11)
|
|
||||||
|
|
||||||
echo '### Test {#}'
|
|
||||||
seq 1 10 | parallel -k echo {#}
|
|
||||||
|
|
||||||
echo '### Test --seqreplace and line too long'
|
|
||||||
seq 1 1000 | stdout parallel -j1 -s 210 -k --seqreplace I echo IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII \|wc | uniq -c
|
|
||||||
|
|
||||||
echo '### bug #37042: -J foo is taken from the whole command line - not just the part before the command'
|
|
||||||
echo '--tagstring foo' > ~/.parallel/bug_37042_profile;
|
|
||||||
parallel -J bug_37042_profile echo ::: tag_with_foo;
|
|
||||||
parallel --tagstring a -J bug_37042_profile echo ::: tag_with_a;
|
|
||||||
parallel --tagstring a echo -J bug_37042_profile ::: print_-J_bug_37042_profile
|
|
||||||
|
|
||||||
echo '### Bug introduce by fixing bug #37042'
|
|
||||||
parallel --xapply -a <(printf 'abc') --colsep '\t' echo {1}
|
|
||||||
|
|
||||||
echo "### Test --header with -N"
|
|
||||||
(echo h1; echo h2; echo 1a;echo 1b; echo 2a;echo 2b; echo 3a)| parallel -j1 --pipe -N2 -k --header '.*\n.*\n' echo Start\;cat \; echo Stop
|
|
||||||
|
|
||||||
echo "### Test --header with --block 1k"
|
|
||||||
(echo h1; echo h2; perl -e '$a="x"x110;for(1..22){print $_,$a,"\n"'})| parallel -j1 --pipe -k --block 1k --header '.*\n.*\n' echo Start\;cat \; echo Stop
|
|
||||||
|
|
||||||
echo "### Test --header with multiple :::"
|
|
||||||
parallel --header : echo {a} {b} {1} {2} ::: b b1 ::: a a2
|
|
||||||
|
|
||||||
|
echo moved to parallel-local-1s.sh
|
||||||
|
|
||||||
EOF
|
EOF
|
||||||
|
|
|
||||||
|
|
@ -206,3 +206,14 @@ env_parallel -u -S$MASTER doit ::: 1 |
|
||||||
# env_parallel myfunc ::: a | wc # OK
|
# env_parallel myfunc ::: a | wc # OK
|
||||||
# eval 'myfunc2() { echo '$(perl -e 'print "x"x120000')'; }'
|
# eval 'myfunc2() { echo '$(perl -e 'print "x"x120000')'; }'
|
||||||
# env_parallel myfunc ::: a | wc # Fail too big env
|
# env_parallel myfunc ::: a | wc # Fail too big env
|
||||||
|
|
||||||
|
# Supported keylength:
|
||||||
|
# 16300:
|
||||||
|
# debian-ppc netbsd openbsd qnx aix centos freebsd hpux hpux-ia64
|
||||||
|
# macosx mandriva miros raspbian redhat scosysv suse tru64 unixware
|
||||||
|
# 8000:
|
||||||
|
# solaris-x86
|
||||||
|
# 4000:
|
||||||
|
# openindiana solaris
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1019,7 +1019,7 @@ par_verbose_t echo bar
|
||||||
par_verbose_t echo car
|
par_verbose_t echo car
|
||||||
par_verbose_t echo far
|
par_verbose_t echo far
|
||||||
par_version ### Test --version: Version output (just check we get the same amount of lines)
|
par_version ### Test --version: Version output (just check we get the same amount of lines)
|
||||||
par_version 10
|
par_version 11
|
||||||
par_wd_3dot_local bug #45993: --wd ... should also work when run locally
|
par_wd_3dot_local bug #45993: --wd ... should also work when run locally
|
||||||
par_wd_3dot_local parallel: Error: Cannot change into non-executable dir /bi: No such file or directory
|
par_wd_3dot_local parallel: Error: Cannot change into non-executable dir /bi: No such file or directory
|
||||||
par_wd_3dot_local /bin
|
par_wd_3dot_local /bin
|
||||||
|
|
|
||||||
|
|
@ -743,9 +743,9 @@ par_shellquote tcsh \\\\\\\\ "\
|
||||||
par_shellquote tcsh "\ \ \
par_shellquote tcsh \\\\\\\\\\\\\ \ \!\"\#\$%\&\'\(\)\*+,-./0123456789:\;\<\=\>\?@ABCDEFGHIJKLMNOPQRSTUVWXYZ\[\\\]\^_\`abcdefghijklmnopqrstuvwxyz\{\|\}\~€<7F>\‚\ƒ\„\…\†\‡\ˆ\‰\Š\‹\Œ\<5C>\Ž\<5C>\<5C>\‘\’\“\”\•\–\—\˜\™\š\›\œ\<5C>\ž\Ÿ\ \¡\¢\£\¤\¥\¦\§\¨\©\ª\«\¬\\®\¯\°\±\²\³\´\µ\¶\·\¸\¹\º\»\¼\½\¾\¿\À\Á\Â\Ã\Ä\Å\Æ\Ç\È\É\Ê\Ë\Ì\Í\Î\Ï\Ð\Ñ\Ò\Ó\Ô\Õ\Ö\×\Ø\Ù\Ú\Û\Ü\Ý\Þ\ß\à\á\â\ã\ä\å\æ\ç\è\é\ê\ë\ì\í\î\ï\ð\ñ\ò\ó\ô\õ\ö\÷\ø\ù\ú\û\ü\ý\þ\ÿ
|
par_shellquote tcsh "\ \ \
par_shellquote tcsh \\\\\\\\\\\\\ \ \!\"\#\$%\&\'\(\)\*+,-./0123456789:\;\<\=\>\?@ABCDEFGHIJKLMNOPQRSTUVWXYZ\[\\\]\^_\`abcdefghijklmnopqrstuvwxyz\{\|\}\~€<7F>\‚\ƒ\„\…\†\‡\ˆ\‰\Š\‹\Œ\<5C>\Ž\<5C>\<5C>\‘\’\“\”\•\–\—\˜\™\š\›\œ\<5C>\ž\Ÿ\ \¡\¢\£\¤\¥\¦\§\¨\©\ª\«\¬\\®\¯\°\±\²\³\´\µ\¶\·\¸\¹\º\»\¼\½\¾\¿\À\Á\Â\Ã\Ä\Å\Æ\Ç\È\É\Ê\Ë\Ì\Í\Î\Ï\Ð\Ñ\Ò\Ó\Ô\Õ\Ö\×\Ø\Ù\Ú\Û\Ü\Ý\Þ\ß\à\á\â\ã\ä\å\æ\ç\è\é\ê\ë\ì\í\î\ï\ð\ñ\ò\ó\ô\õ\ö\÷\ø\ù\ú\û\ü\ý\þ\ÿ
|
||||||
par_sockets_cores_threads ### Test --number-of-sockets/cores/threads
|
par_sockets_cores_threads ### Test --number-of-sockets/cores/threads
|
||||||
par_sockets_cores_threads 1
|
par_sockets_cores_threads 1
|
||||||
|
par_sockets_cores_threads 2
|
||||||
par_sockets_cores_threads 4
|
par_sockets_cores_threads 4
|
||||||
par_sockets_cores_threads 8
|
par_sockets_cores_threads 2
|
||||||
par_sockets_cores_threads 4
|
|
||||||
par_sockets_cores_threads ### Test --use-sockets-instead-of-threads
|
par_sockets_cores_threads ### Test --use-sockets-instead-of-threads
|
||||||
par_sockets_cores_threads threads done
|
par_sockets_cores_threads threads done
|
||||||
par_sockets_cores_threads sockets done
|
par_sockets_cores_threads sockets done
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,63 @@
|
||||||
par_block_negative_prefix ### This should generate 10*2 jobs
|
par_block_negative_prefix ### This should generate 10*2 jobs
|
||||||
par_block_negative_prefix 20 40 720
|
par_block_negative_prefix 20 40 720
|
||||||
|
par_bug34241 ### bug #34241: --pipe should not spawn unneeded processes
|
||||||
|
par_bug34241 OK
|
||||||
|
par_bug37042 ### bug #37042: -J foo is taken from the whole command line - not just the part before the command
|
||||||
|
par_bug37042 foo tag_with_foo
|
||||||
|
par_bug37042 a tag_with_a
|
||||||
|
par_bug37042 a -J bug_37042_profile print_-J_bug_37042_profile
|
||||||
|
par_bug37042 ### Bug introduce by fixing bug #37042
|
||||||
|
par_bug37042 abc
|
||||||
|
par_bug43654 bug #43654: --bar with command not using {} - only last output line
|
||||||
|
par_bug43654
par_bug43654 [7m100% 1:0=0s 1 [0m[0m
|
||||||
|
par_colsep ### Test of --colsep
|
||||||
|
par_colsep a b c
|
||||||
|
par_colsep a b c
|
||||||
|
par_colsep a b c d
|
||||||
|
par_colsep a b c
|
||||||
|
par_colsep d e f
|
||||||
|
par_colsep a b c
|
||||||
|
par_colsep d e f
|
||||||
|
par_colsep a b c
|
||||||
|
par_colsep a b c
|
||||||
|
par_colsep a b c d
|
||||||
|
par_colsep ### Test of tab as colsep
|
||||||
|
par_colsep abc def
|
||||||
|
par_colsep ghi jkl
|
||||||
|
par_colsep abc def
|
||||||
|
par_colsep ghi jkl
|
||||||
|
par_colsep ### Test of multiple -a plus colsep
|
||||||
|
par_colsep abc def
|
||||||
|
par_colsep mno jkl
|
||||||
|
par_colsep ### Test of multiple -a no colsep
|
||||||
|
par_colsep abc def ghi
|
||||||
|
par_colsep jkl mno pqr
|
||||||
|
par_colsep ### Test of quoting after colsplit
|
||||||
|
par_colsep >/tmp/null >/dev/null
|
||||||
|
par_colsep ### Test of --colsep as regexp
|
||||||
|
par_colsep a b c
|
||||||
|
par_colsep a b c d
|
||||||
|
par_colsep a b c
|
||||||
|
par_colsep a b c d
|
||||||
|
par_colsep a b c
|
||||||
|
par_colsep a b c d
|
||||||
|
par_colsep a_b_c_
|
||||||
|
par_colsep a_b_c_d
|
||||||
|
par_colsep ### Test of -C
|
||||||
|
par_colsep a_b_c_
|
||||||
|
par_colsep a_b_c_d
|
||||||
|
par_colsep ### Test of --trim n
|
||||||
|
par_colsep a_b_' c '_
|
||||||
|
par_colsep a_' b '_c_d
|
||||||
|
par_colsep a_b_c_
|
||||||
|
par_colsep a_b_c_d
|
||||||
|
par_colsep ### Test of bug: If input is empty string
|
||||||
|
par_colsep echo ''
|
||||||
|
par_colsep
|
||||||
|
par_colsep echo ac
|
||||||
|
par_colsep ac
|
||||||
|
par_colsep echo ac
|
||||||
|
par_colsep ac
|
||||||
par_compress_prg_fails ### bug #44546: If --compress-program fails: fail
|
par_compress_prg_fails ### bug #44546: If --compress-program fails: fail
|
||||||
par_compress_prg_fails 1
|
par_compress_prg_fails 1
|
||||||
par_compress_prg_fails parallel: Error: false failed.
|
par_compress_prg_fails parallel: Error: false failed.
|
||||||
|
|
@ -25,6 +83,12 @@ par_compress_prg_fails --line-buffer --tag parallel: Error: false failed.
|
||||||
par_compress_prg_fails --line-buffer --tag --files 1
|
par_compress_prg_fails --line-buffer --tag --files 1
|
||||||
par_compress_prg_fails --line-buffer --tag --files parallel: Error: false failed.
|
par_compress_prg_fails --line-buffer --tag --files parallel: Error: false failed.
|
||||||
par_compress_prg_fails --line-buffer --tag --files parallel: Error: false failed.
|
par_compress_prg_fails --line-buffer --tag --files parallel: Error: false failed.
|
||||||
|
par_empty_string_command_line ### Test of ignore-empty string on :::
|
||||||
|
par_empty_string_command_line foo
|
||||||
|
par_empty_string_command_line ole
|
||||||
|
par_empty_string_command_line bar
|
||||||
|
par_eof_on_command_line_input_source ### Test of eof string on :::
|
||||||
|
par_eof_on_command_line_input_source foo
|
||||||
par_failing_compressor Compress with failing (de)compressor
|
par_failing_compressor Compress with failing (de)compressor
|
||||||
par_failing_compressor Test --tag/--line-buffer/--files in all combinations
|
par_failing_compressor Test --tag/--line-buffer/--files in all combinations
|
||||||
par_failing_compressor Test working/failing compressor/decompressor in all combinations
|
par_failing_compressor Test working/failing compressor/decompressor in all combinations
|
||||||
|
|
@ -155,6 +219,61 @@ par_fifo_under_csh 1
|
||||||
par_fifo_under_csh 868832
|
par_fifo_under_csh 868832
|
||||||
par_fifo_under_csh 1
|
par_fifo_under_csh 1
|
||||||
par_fifo_under_csh exit 22
|
par_fifo_under_csh exit 22
|
||||||
|
par_header ### Test --header with -N
|
||||||
|
par_header Start
|
||||||
|
par_header h1
|
||||||
|
par_header h2
|
||||||
|
par_header 1a
|
||||||
|
par_header 1b
|
||||||
|
par_header Stop
|
||||||
|
par_header Start
|
||||||
|
par_header h1
|
||||||
|
par_header h2
|
||||||
|
par_header 2a
|
||||||
|
par_header 2b
|
||||||
|
par_header Stop
|
||||||
|
par_header Start
|
||||||
|
par_header h1
|
||||||
|
par_header h2
|
||||||
|
par_header 3a
|
||||||
|
par_header Stop
|
||||||
|
par_header ### Test --header with --block 1k
|
||||||
|
par_header Start
|
||||||
|
par_header h1
|
||||||
|
par_header h2
|
||||||
|
par_header 1xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
||||||
|
par_header 2xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
||||||
|
par_header 3xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
||||||
|
par_header 4xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
||||||
|
par_header 5xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
||||||
|
par_header 6xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
||||||
|
par_header 7xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
||||||
|
par_header 8xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
||||||
|
par_header Stop
|
||||||
|
par_header Start
|
||||||
|
par_header h1
|
||||||
|
par_header h2
|
||||||
|
par_header 9xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
||||||
|
par_header 10xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
||||||
|
par_header 11xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
||||||
|
par_header 12xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
||||||
|
par_header 13xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
||||||
|
par_header 14xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
||||||
|
par_header 15xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
||||||
|
par_header 16xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
||||||
|
par_header Stop
|
||||||
|
par_header Start
|
||||||
|
par_header h1
|
||||||
|
par_header h2
|
||||||
|
par_header 17xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
||||||
|
par_header 18xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
||||||
|
par_header 19xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
||||||
|
par_header 20xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
||||||
|
par_header 21xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
||||||
|
par_header 22xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
||||||
|
par_header Stop
|
||||||
|
par_header ### Test --header with multiple :::
|
||||||
|
par_header a2 b1 b1 a2
|
||||||
par_header_parens bug #49538: --header and {= =}
|
par_header_parens bug #49538: --header and {= =}
|
||||||
par_header_parens OK
|
par_header_parens OK
|
||||||
par_header_parens OK
|
par_header_parens OK
|
||||||
|
|
@ -174,6 +293,85 @@ par_incomplete_linebuffer 7
|
||||||
par_incomplete_linebuffer 8
|
par_incomplete_linebuffer 8
|
||||||
par_incomplete_linebuffer 9
|
par_incomplete_linebuffer 9
|
||||||
par_incomplete_linebuffer 10
|
par_incomplete_linebuffer 10
|
||||||
|
par_mix_triple_colon_with_quad_colon ### Test :::: mixed with :::
|
||||||
|
par_mix_triple_colon_with_quad_colon ### Test :::: < ::: :::
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 4 1
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 4 2
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 4 3
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 5 1
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 5 2
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 5 3
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 4 1
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 4 2
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 4 3
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 5 1
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 5 2
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 5 3
|
||||||
|
par_mix_triple_colon_with_quad_colon ### Test :::: < < :::: <
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 4 1
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 4 2
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 4 3
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 5 1
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 5 2
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 5 3
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 4 1
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 4 2
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 4 3
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 5 1
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 5 2
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 5 3
|
||||||
|
par_mix_triple_colon_with_quad_colon ### Test -a :::: < :::: <
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 4 1
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 4 2
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 4 3
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 5 1
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 5 2
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 5 3
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 4 1
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 4 2
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 4 3
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 5 1
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 5 2
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 5 3
|
||||||
|
par_mix_triple_colon_with_quad_colon ### Test -a -a :::
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 4 1
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 4 2
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 4 3
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 5 1
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 5 2
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 5 3
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 4 1
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 4 2
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 4 3
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 5 1
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 5 2
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 5 3
|
||||||
|
par_mix_triple_colon_with_quad_colon ### Test -a - -a :::
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 4 1
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 4 2
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 4 3
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 5 1
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 5 2
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 5 3
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 4 1
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 4 2
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 4 3
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 5 1
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 5 2
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 5 3
|
||||||
|
par_mix_triple_colon_with_quad_colon ### Test :::: < - :::
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 4 1
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 4 2
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 4 3
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 5 1
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 5 2
|
||||||
|
par_mix_triple_colon_with_quad_colon 6 5 3
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 4 1
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 4 2
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 4 3
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 5 1
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 5 2
|
||||||
|
par_mix_triple_colon_with_quad_colon 7 5 3
|
||||||
par_null_resume ### --null --resume --jl
|
par_null_resume ### --null --resume --jl
|
||||||
par_null_resume a
|
par_null_resume a
|
||||||
par_null_resume b
|
par_null_resume b
|
||||||
|
|
@ -223,6 +421,64 @@ par_pxz_complains bug #44250: pxz complains File format not recognized but decom
|
||||||
par_pxz_complains ls: cannot access '/OK-if-missing-file': No such file or directory
|
par_pxz_complains ls: cannot access '/OK-if-missing-file': No such file or directory
|
||||||
par_pxz_complains can not seek in input: Illegal seek
|
par_pxz_complains can not seek in input: Illegal seek
|
||||||
par_pxz_complains ls: cannot access '/OK-if-missing-file': No such file or directory
|
par_pxz_complains ls: cannot access '/OK-if-missing-file': No such file or directory
|
||||||
|
par_replacement_rename ### Test --basenamereplace
|
||||||
|
par_replacement_rename b.c b.c b.c b b b
|
||||||
|
par_replacement_rename b.c
|
||||||
|
par_replacement_rename b.c
|
||||||
|
par_replacement_rename b.c
|
||||||
|
par_replacement_rename b
|
||||||
|
par_replacement_rename b
|
||||||
|
par_replacement_rename b
|
||||||
|
par_replacement_rename ### Test --bnr
|
||||||
|
par_replacement_rename b.c b.c b.c b b b
|
||||||
|
par_replacement_rename b.c
|
||||||
|
par_replacement_rename b.c
|
||||||
|
par_replacement_rename b.c
|
||||||
|
par_replacement_rename b
|
||||||
|
par_replacement_rename b
|
||||||
|
par_replacement_rename b
|
||||||
|
par_replacement_rename ### Test --extensionreplace
|
||||||
|
par_replacement_rename /a/b a/b b /a/b a/b b
|
||||||
|
par_replacement_rename /a/b
|
||||||
|
par_replacement_rename a/b
|
||||||
|
par_replacement_rename b
|
||||||
|
par_replacement_rename /a/b
|
||||||
|
par_replacement_rename a/b
|
||||||
|
par_replacement_rename b
|
||||||
|
par_replacement_rename ### Test --er
|
||||||
|
par_replacement_rename /a/b a/b b /a/b a/b b
|
||||||
|
par_replacement_rename /a/b
|
||||||
|
par_replacement_rename a/b
|
||||||
|
par_replacement_rename b
|
||||||
|
par_replacement_rename /a/b
|
||||||
|
par_replacement_rename a/b
|
||||||
|
par_replacement_rename b
|
||||||
|
par_replacement_rename ### Test --basenameextensionreplace
|
||||||
|
par_replacement_rename b b b b b b
|
||||||
|
par_replacement_rename b
|
||||||
|
par_replacement_rename b
|
||||||
|
par_replacement_rename b
|
||||||
|
par_replacement_rename b
|
||||||
|
par_replacement_rename b
|
||||||
|
par_replacement_rename b
|
||||||
|
par_replacement_rename ### Test --bner
|
||||||
|
par_replacement_rename b b b b b b
|
||||||
|
par_replacement_rename b
|
||||||
|
par_replacement_rename b
|
||||||
|
par_replacement_rename b
|
||||||
|
par_replacement_rename b
|
||||||
|
par_replacement_rename b
|
||||||
|
par_replacement_rename b
|
||||||
|
par_replacement_strings ### Test {/}
|
||||||
|
par_replacement_strings b.c b.c b.c b b b
|
||||||
|
par_replacement_strings ### Test {/.}
|
||||||
|
par_replacement_strings b b b b b b
|
||||||
|
par_replacement_strings ### Test {#/.}
|
||||||
|
par_replacement_strings number2
|
||||||
|
par_replacement_strings ### Test {#/}
|
||||||
|
par_replacement_strings number2.c
|
||||||
|
par_replacement_strings ### Test {#.}
|
||||||
|
par_replacement_strings a/number2
|
||||||
par_result ### Test --results
|
par_result ### Test --results
|
||||||
par_result I III
|
par_result I III
|
||||||
par_result I IIII
|
par_result I IIII
|
||||||
|
|
@ -479,6 +735,42 @@ par_result_replace /tmp/par__49983-baz C
|
||||||
par_result_replace /tmp/par__49983-baz C/seq
|
par_result_replace /tmp/par__49983-baz C/seq
|
||||||
par_result_replace /tmp/par__49983-baz C/stderr
|
par_result_replace /tmp/par__49983-baz C/stderr
|
||||||
par_result_replace /tmp/par__49983-baz C/stdout
|
par_result_replace /tmp/par__49983-baz C/stdout
|
||||||
|
par_seqreplace_long_line ### Test --seqreplace and line too long
|
||||||
|
par_seqreplace_long_line 9 1 1 101
|
||||||
|
par_seqreplace_long_line 90 1 1 201
|
||||||
|
par_seqreplace_long_line 1 parallel: Error: Command line too long (309 >= 210) at input 0: 100
|
||||||
|
par_test_E ### Test -E
|
||||||
|
par_test_E 1 2 3
|
||||||
|
par_test_E 1 2 4
|
||||||
|
par_test_E 1 3 3
|
||||||
|
par_test_E 1 3 4
|
||||||
|
par_test_E 1 4 3
|
||||||
|
par_test_E 1 4 4
|
||||||
|
par_test_E 2 2 3
|
||||||
|
par_test_E 2 2 4
|
||||||
|
par_test_E 2 3 3
|
||||||
|
par_test_E 2 3 4
|
||||||
|
par_test_E 2 4 3
|
||||||
|
par_test_E 2 4 4
|
||||||
|
par_test_E 3 2 3
|
||||||
|
par_test_E 3 2 4
|
||||||
|
par_test_E 3 3 3
|
||||||
|
par_test_E 3 3 4
|
||||||
|
par_test_E 3 4 3
|
||||||
|
par_test_E 3 4 4
|
||||||
|
par_test_E 4 2 3
|
||||||
|
par_test_E 4 2 4
|
||||||
|
par_test_E 4 3 3
|
||||||
|
par_test_E 4 3 4
|
||||||
|
par_test_E 4 4 3
|
||||||
|
par_test_E 4 4 4
|
||||||
|
par_test_E ### Test -E one empty
|
||||||
|
par_test_E 1 2
|
||||||
|
par_test_E 2 2
|
||||||
|
par_test_E ### Test -E 2 empty
|
||||||
|
par_test_E 1
|
||||||
|
par_test_E 2
|
||||||
|
par_test_E ### Test -E all empty
|
||||||
par_test_XI_mI ### Test -I
|
par_test_XI_mI ### Test -I
|
||||||
par_test_XI_mI 1 1
|
par_test_XI_mI 1 1
|
||||||
par_test_XI_mI 2 1
|
par_test_XI_mI 2 1
|
||||||
|
|
@ -565,6 +857,34 @@ par_test_cpu_detection 2-24-48-24 2 24 48 24
|
||||||
par_test_cpu_detection 1-2-2-2 1 2 2 2
|
par_test_cpu_detection 1-2-2-2 1 2 2 2
|
||||||
par_test_cpu_detection 1-8-8-8 1 8 8 8
|
par_test_cpu_detection 1-8-8-8 1 8 8 8
|
||||||
par_test_cpu_detection 1-4-4-4 1 4 4 4
|
par_test_cpu_detection 1-4-4-4 1 4 4 4
|
||||||
|
par_test_gt_quoting ### Test of quoting of > bug
|
||||||
|
par_test_gt_quoting >/dev/null
|
||||||
|
par_test_gt_quoting ### Test of quoting of > bug if line continuation
|
||||||
|
par_test_gt_quoting > > >
|
||||||
|
par_test_job_number ### Test {#}
|
||||||
|
par_test_job_number 1
|
||||||
|
par_test_job_number 2
|
||||||
|
par_test_job_number 3
|
||||||
|
par_test_job_number 4
|
||||||
|
par_test_job_number 5
|
||||||
|
par_test_job_number 6
|
||||||
|
par_test_job_number 7
|
||||||
|
par_test_job_number 8
|
||||||
|
par_test_job_number 9
|
||||||
|
par_test_job_number 10
|
||||||
par_too_long_line_X bug #54869: Long lines break
|
par_too_long_line_X bug #54869: Long lines break
|
||||||
par_too_long_line_X 1 29302 131012
|
par_too_long_line_X 1 29302 131012
|
||||||
par_too_long_line_X 1 12698 63490
|
par_too_long_line_X 1 12698 63490
|
||||||
|
par_trailing_space_line_continuation ### Test of trailing space continuation
|
||||||
|
par_trailing_space_line_continuation foo ole bar
|
||||||
|
par_trailing_space_line_continuation quux
|
||||||
|
par_trailing_space_line_continuation foo ole bar
|
||||||
|
par_trailing_space_line_continuation quux
|
||||||
|
par_trailing_space_line_continuation foo ole bar
|
||||||
|
par_trailing_space_line_continuation quux
|
||||||
|
par_trailing_space_line_continuation ### Test of trailing space continuation with -E eof
|
||||||
|
par_trailing_space_line_continuation foo ole
|
||||||
|
par_trailing_space_line_continuation foo ole
|
||||||
|
par_trailing_space_line_continuation foo ole
|
||||||
|
par_trim_illegal_value ### Test of --trim illegal
|
||||||
|
par_trim_illegal_value parallel: Error: --trim must be one of: r l rl lr.
|
||||||
|
|
|
||||||
|
|
@ -569,6 +569,8 @@ par_groupby --block 20 C1 cssv [\s,]+ 3,,2,,4,,
|
||||||
par_groupby --block 20 C1 cssv [\s,]+ NewRec
|
par_groupby --block 20 C1 cssv [\s,]+ NewRec
|
||||||
par_groupby --block 20 C1 cssv [\s,]+ a1, b1, C1,
|
par_groupby --block 20 C1 cssv [\s,]+ a1, b1, C1,
|
||||||
par_groupby --block 20 C1 cssv [\s,]+ 3 ,,, 2 ,,, 2 ,,,
|
par_groupby --block 20 C1 cssv [\s,]+ 3 ,,, 2 ,,, 2 ,,,
|
||||||
|
par_groupby --block 20 C1 cssv [\s,]+ NewRec
|
||||||
|
par_groupby --block 20 C1 cssv [\s,]+ a1, b1, C1,
|
||||||
par_groupby --block 20 C1 cssv [\s,]+ 3 2 3
|
par_groupby --block 20 C1 cssv [\s,]+ 3 2 3
|
||||||
par_groupby --block 20 C1 cssv [\s,]+ parallel: Warning: A record was longer than 20. Increasing to --blocksize 27.
|
par_groupby --block 20 C1 cssv [\s,]+ parallel: Warning: A record was longer than 20. Increasing to --blocksize 27.
|
||||||
par_groupby --block 20 C1 cssv [\s,]+ parallel: Warning: A record was longer than 27. Increasing to --blocksize 37.
|
par_groupby --block 20 C1 cssv [\s,]+ parallel: Warning: A record was longer than 27. Increasing to --blocksize 37.
|
||||||
|
|
@ -1386,27 +1388,7 @@ par_tee_with_premature_close 999998
|
||||||
par_tee_with_premature_close 999999
|
par_tee_with_premature_close 999999
|
||||||
par_tee_with_premature_close 1000000
|
par_tee_with_premature_close 1000000
|
||||||
par_tee_with_premature_close tee without --output-error=warn-nopipe support
|
par_tee_with_premature_close tee without --output-error=warn-nopipe support
|
||||||
par_tee_with_premature_close 1859 1860 8192
|
par_tee_with_premature_close OK
|
||||||
par_tee_with_premature_close 1
|
|
||||||
par_tee_with_premature_close 2
|
|
||||||
par_tee_with_premature_close 3
|
|
||||||
par_tee_with_premature_close 4
|
|
||||||
par_tee_with_premature_close 5
|
|
||||||
par_tee_with_premature_close 6
|
|
||||||
par_tee_with_premature_close 7
|
|
||||||
par_tee_with_premature_close 8
|
|
||||||
par_tee_with_premature_close 9
|
|
||||||
par_tee_with_premature_close 10
|
|
||||||
par_tee_with_premature_close 1851
|
|
||||||
par_tee_with_premature_close 1852
|
|
||||||
par_tee_with_premature_close 1853
|
|
||||||
par_tee_with_premature_close 1854
|
|
||||||
par_tee_with_premature_close 1855
|
|
||||||
par_tee_with_premature_close 1856
|
|
||||||
par_tee_with_premature_close 1857
|
|
||||||
par_tee_with_premature_close 1858
|
|
||||||
par_tee_with_premature_close 1859
|
|
||||||
par_tee_with_premature_close 1860
|
|
||||||
par_test_X_with_multiple_source ### Test {} multiple times in different commands
|
par_test_X_with_multiple_source ### Test {} multiple times in different commands
|
||||||
par_test_X_with_multiple_source echo 1 2 3 4 5 6 7 8 9 10 ; echo 1 2 3 4 5 6 7 8 9 10
|
par_test_X_with_multiple_source echo 1 2 3 4 5 6 7 8 9 10 ; echo 1 2 3 4 5 6 7 8 9 10
|
||||||
par_test_X_with_multiple_source 1 2 3 4 5 6 7 8 9 10
|
par_test_X_with_multiple_source 1 2 3 4 5 6 7 8 9 10
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,10 @@
|
||||||
|
par_bug56403 bug #56403: --pipe block by time.
|
||||||
|
par_bug56403 1 job1a
|
||||||
|
par_bug56403 2 job2b
|
||||||
|
par_bug56403 3 job3cjob3d
|
||||||
|
par_bug56403 1 job1a
|
||||||
|
par_bug56403 1 job1b
|
||||||
|
par_bug56403 2 job2cjob2d
|
||||||
par_distribute_input_by_ability ### bug #48290: round-robin does not distribute data based on business
|
par_distribute_input_by_ability ### bug #48290: round-robin does not distribute data based on business
|
||||||
par_distribute_input_by_ability ### Distribute input to jobs that are ready
|
par_distribute_input_by_ability ### Distribute input to jobs that are ready
|
||||||
par_distribute_input_by_ability Job-slot n is 50% slower than n+1, so the order should be 1..7
|
par_distribute_input_by_ability Job-slot n is 50% slower than n+1, so the order should be 1..7
|
||||||
|
|
|
||||||
|
|
@ -1229,59 +1229,14 @@ par_ksh_environment_too_big OK_bigfunc_remote
|
||||||
par_ksh_environment_too_big OK_bigfunc_quote
|
par_ksh_environment_too_big OK_bigfunc_quote
|
||||||
par_ksh_environment_too_big OK_bigfunc_quote_remote
|
par_ksh_environment_too_big OK_bigfunc_quote_remote
|
||||||
par_ksh_environment_too_big Rest should fail
|
par_ksh_environment_too_big Rest should fail
|
||||||
par_ksh_environment_too_big ksh[XXX]: perl: /usr/bin/perl: cannot execute [Argument list too long]
|
par_ksh_environment_too_big parallel: Error: Command line too long (XXX >= XXX) at input 0: fail_bigvar
|
||||||
par_ksh_environment_too_big env_parallel: Error: Your environment is too big.
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: You can try 3 different approaches:
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: 1. Run 'env_parallel --session' before you set
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: variables or define functions.
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: 2. Use --env and only mention the names to copy.
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: 3. Try running this in a clean environment once:
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: env_parallel --record-env
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: And then use '--env _'
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: For details see: man env_parallel
|
|
||||||
par_ksh_environment_too_big parallel: Error: Command line too long (XXX >= XXX) at input 0: fail_bigvar_remote
|
par_ksh_environment_too_big parallel: Error: Command line too long (XXX >= XXX) at input 0: fail_bigvar_remote
|
||||||
par_ksh_environment_too_big ksh[XXX]: perl: /usr/bin/perl: cannot execute [Argument list too long]
|
par_ksh_environment_too_big parallel: Error: Command line too long (XXX >= XXX) at input 0: fail_bigvar_quote
|
||||||
par_ksh_environment_too_big env_parallel: Error: Your environment is too big.
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: You can try 3 different approaches:
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: 1. Run 'env_parallel --session' before you set
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: variables or define functions.
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: 2. Use --env and only mention the names to copy.
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: 3. Try running this in a clean environment once:
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: env_parallel --record-env
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: And then use '--env _'
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: For details see: man env_parallel
|
|
||||||
par_ksh_environment_too_big parallel: Error: Command line too long (XXX >= XXX) at input 0: fail_bigvar_quote_remote
|
par_ksh_environment_too_big parallel: Error: Command line too long (XXX >= XXX) at input 0: fail_bigvar_quote_remote
|
||||||
par_ksh_environment_too_big ksh[XXX]: perl: /usr/bin/perl: cannot execute [Argument list too long]
|
par_ksh_environment_too_big parallel: Error: Command line too long (XXX >= XXX) at input 0: fail_bigfunc
|
||||||
par_ksh_environment_too_big env_parallel: Error: Your environment is too big.
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: You can try 3 different approaches:
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: 1. Run 'env_parallel --session' before you set
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: variables or define functions.
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: 2. Use --env and only mention the names to copy.
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: 3. Try running this in a clean environment once:
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: env_parallel --record-env
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: And then use '--env _'
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: For details see: man env_parallel
|
|
||||||
par_ksh_environment_too_big parallel: Error: Command line too long (XXX >= XXX) at input 0: fail_bigfunc_remote
|
par_ksh_environment_too_big parallel: Error: Command line too long (XXX >= XXX) at input 0: fail_bigfunc_remote
|
||||||
par_ksh_environment_too_big ksh[XXX]: perl: /usr/bin/perl: cannot execute [Argument list too long]
|
par_ksh_environment_too_big parallel: Error: Command line too long (XXX >= XXX) at input 0: fail_bigfunc_quote
|
||||||
par_ksh_environment_too_big env_parallel: Error: Your environment is too big.
|
par_ksh_environment_too_big parallel: Error: Command line too long (XXX >= XXX) at input 0: fail_bigfunc_quote_remote
|
||||||
par_ksh_environment_too_big env_parallel: Error: You can try 3 different approaches:
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: 1. Run 'env_parallel --session' before you set
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: variables or define functions.
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: 2. Use --env and only mention the names to copy.
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: 3. Try running this in a clean environment once:
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: env_parallel --record-env
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: And then use '--env _'
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: For details see: man env_parallel
|
|
||||||
par_ksh_environment_too_big ksh[XXX]: perl: /usr/bin/perl: cannot execute [Argument list too long]
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: Your environment is too big.
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: You can try 3 different approaches:
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: 1. Run 'env_parallel --session' before you set
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: variables or define functions.
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: 2. Use --env and only mention the names to copy.
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: 3. Try running this in a clean environment once:
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: env_parallel --record-env
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: And then use '--env _'
|
|
||||||
par_ksh_environment_too_big env_parallel: Error: For details see: man env_parallel
|
|
||||||
par_ksh_env_parallel_session ### Test env_parallel --session
|
par_ksh_env_parallel_session ### Test env_parallel --session
|
||||||
par_ksh_env_parallel_session ### level0 should be hidden, level1 should be transferred
|
par_ksh_env_parallel_session ### level0 should be hidden, level1 should be transferred
|
||||||
par_ksh_env_parallel_session
|
par_ksh_env_parallel_session
|
||||||
|
|
@ -2087,59 +2042,14 @@ par_bash_environment_too_big OK_bigfunc_remote
|
||||||
par_bash_environment_too_big OK_bigfunc_quote
|
par_bash_environment_too_big OK_bigfunc_quote
|
||||||
par_bash_environment_too_big OK_bigfunc_quote_remote
|
par_bash_environment_too_big OK_bigfunc_quote_remote
|
||||||
par_bash_environment_too_big Rest should fail
|
par_bash_environment_too_big Rest should fail
|
||||||
par_bash_environment_too_big /usr/local/bin/env_parallel.bash: line XXX: /usr/bin/perl: Argument list too long
|
par_bash_environment_too_big parallel: Error: Command line too long (XXX >= XXX) at input 0: fail_bigvar
|
||||||
par_bash_environment_too_big env_parallel: Error: Your environment is too big.
|
|
||||||
par_bash_environment_too_big env_parallel: Error: You can try 3 different approaches:
|
|
||||||
par_bash_environment_too_big env_parallel: Error: 1. Run 'env_parallel --session' before you set
|
|
||||||
par_bash_environment_too_big env_parallel: Error: variables or define functions.
|
|
||||||
par_bash_environment_too_big env_parallel: Error: 2. Use --env and only mention the names to copy.
|
|
||||||
par_bash_environment_too_big env_parallel: Error: 3. Try running this in a clean environment once:
|
|
||||||
par_bash_environment_too_big env_parallel: Error: env_parallel --record-env
|
|
||||||
par_bash_environment_too_big env_parallel: Error: And then use '--env _'
|
|
||||||
par_bash_environment_too_big env_parallel: Error: For details see: man env_parallel
|
|
||||||
par_bash_environment_too_big parallel: Error: Command line too long (XXX >= XXX) at input 0: fail_bigvar_remote
|
par_bash_environment_too_big parallel: Error: Command line too long (XXX >= XXX) at input 0: fail_bigvar_remote
|
||||||
par_bash_environment_too_big /usr/local/bin/env_parallel.bash: line XXX: /usr/bin/perl: Argument list too long
|
par_bash_environment_too_big parallel: Error: Command line too long (XXX >= XXX) at input 0: fail_bigvar_quote
|
||||||
par_bash_environment_too_big env_parallel: Error: Your environment is too big.
|
|
||||||
par_bash_environment_too_big env_parallel: Error: You can try 3 different approaches:
|
|
||||||
par_bash_environment_too_big env_parallel: Error: 1. Run 'env_parallel --session' before you set
|
|
||||||
par_bash_environment_too_big env_parallel: Error: variables or define functions.
|
|
||||||
par_bash_environment_too_big env_parallel: Error: 2. Use --env and only mention the names to copy.
|
|
||||||
par_bash_environment_too_big env_parallel: Error: 3. Try running this in a clean environment once:
|
|
||||||
par_bash_environment_too_big env_parallel: Error: env_parallel --record-env
|
|
||||||
par_bash_environment_too_big env_parallel: Error: And then use '--env _'
|
|
||||||
par_bash_environment_too_big env_parallel: Error: For details see: man env_parallel
|
|
||||||
par_bash_environment_too_big parallel: Error: Command line too long (XXX >= XXX) at input 0: fail_bigvar_quote_remote
|
par_bash_environment_too_big parallel: Error: Command line too long (XXX >= XXX) at input 0: fail_bigvar_quote_remote
|
||||||
par_bash_environment_too_big /usr/local/bin/env_parallel.bash: line XXX: /usr/bin/perl: Argument list too long
|
par_bash_environment_too_big parallel: Error: Command line too long (XXX >= XXX) at input 0: fail_bigfunc
|
||||||
par_bash_environment_too_big env_parallel: Error: Your environment is too big.
|
|
||||||
par_bash_environment_too_big env_parallel: Error: You can try 3 different approaches:
|
|
||||||
par_bash_environment_too_big env_parallel: Error: 1. Run 'env_parallel --session' before you set
|
|
||||||
par_bash_environment_too_big env_parallel: Error: variables or define functions.
|
|
||||||
par_bash_environment_too_big env_parallel: Error: 2. Use --env and only mention the names to copy.
|
|
||||||
par_bash_environment_too_big env_parallel: Error: 3. Try running this in a clean environment once:
|
|
||||||
par_bash_environment_too_big env_parallel: Error: env_parallel --record-env
|
|
||||||
par_bash_environment_too_big env_parallel: Error: And then use '--env _'
|
|
||||||
par_bash_environment_too_big env_parallel: Error: For details see: man env_parallel
|
|
||||||
par_bash_environment_too_big parallel: Error: Command line too long (XXX >= XXX) at input 0: fail_bigfunc_remote
|
par_bash_environment_too_big parallel: Error: Command line too long (XXX >= XXX) at input 0: fail_bigfunc_remote
|
||||||
par_bash_environment_too_big /usr/local/bin/env_parallel.bash: line XXX: /usr/bin/perl: Argument list too long
|
par_bash_environment_too_big parallel: Error: Command line too long (XXX >= XXX) at input 0: fail_bigfunc_quote
|
||||||
par_bash_environment_too_big env_parallel: Error: Your environment is too big.
|
par_bash_environment_too_big parallel: Error: Command line too long (XXX >= XXX) at input 0: fail_bigfunc_quote_remote
|
||||||
par_bash_environment_too_big env_parallel: Error: You can try 3 different approaches:
|
|
||||||
par_bash_environment_too_big env_parallel: Error: 1. Run 'env_parallel --session' before you set
|
|
||||||
par_bash_environment_too_big env_parallel: Error: variables or define functions.
|
|
||||||
par_bash_environment_too_big env_parallel: Error: 2. Use --env and only mention the names to copy.
|
|
||||||
par_bash_environment_too_big env_parallel: Error: 3. Try running this in a clean environment once:
|
|
||||||
par_bash_environment_too_big env_parallel: Error: env_parallel --record-env
|
|
||||||
par_bash_environment_too_big env_parallel: Error: And then use '--env _'
|
|
||||||
par_bash_environment_too_big env_parallel: Error: For details see: man env_parallel
|
|
||||||
par_bash_environment_too_big /usr/local/bin/env_parallel.bash: line XXX: /usr/bin/perl: Argument list too long
|
|
||||||
par_bash_environment_too_big env_parallel: Error: Your environment is too big.
|
|
||||||
par_bash_environment_too_big env_parallel: Error: You can try 3 different approaches:
|
|
||||||
par_bash_environment_too_big env_parallel: Error: 1. Run 'env_parallel --session' before you set
|
|
||||||
par_bash_environment_too_big env_parallel: Error: variables or define functions.
|
|
||||||
par_bash_environment_too_big env_parallel: Error: 2. Use --env and only mention the names to copy.
|
|
||||||
par_bash_environment_too_big env_parallel: Error: 3. Try running this in a clean environment once:
|
|
||||||
par_bash_environment_too_big env_parallel: Error: env_parallel --record-env
|
|
||||||
par_bash_environment_too_big env_parallel: Error: And then use '--env _'
|
|
||||||
par_bash_environment_too_big env_parallel: Error: For details see: man env_parallel
|
|
||||||
par_bash_env_parallel_session ### Test env_parallel --session / --end-session
|
par_bash_env_parallel_session ### Test env_parallel --session / --end-session
|
||||||
par_bash_env_parallel_session ### level0 should be hidden, level1 should be transferred
|
par_bash_env_parallel_session ### level0 should be hidden, level1 should be transferred
|
||||||
par_bash_env_parallel_session
|
par_bash_env_parallel_session
|
||||||
|
|
|
||||||
|
|
@ -29,11 +29,11 @@ par_path_remote_csh ^^^^^^^^ Not found is OK
|
||||||
par_path_remote_csh /bin:/usr/bin:/tmp OK
|
par_path_remote_csh /bin:/usr/bin:/tmp OK
|
||||||
par_path_remote_csh Done
|
par_path_remote_csh Done
|
||||||
par_retries_1 ### Test of --retries - it should run 13 jobs in total
|
par_retries_1 ### Test of --retries - it should run 13 jobs in total
|
||||||
par_retries_1 13
|
par_retries_1 OK
|
||||||
par_retries_2 ### Test of --retries - it should run 25 jobs in total
|
par_retries_2 ### Test of --retries - it should run 25 jobs in total
|
||||||
par_retries_2 25
|
par_retries_2 OK
|
||||||
par_retries_4 ### Test of --retries - it should run 49 jobs in total
|
par_retries_4 ### Test of --retries - it should run 49 jobs in total
|
||||||
par_retries_4 49
|
par_retries_4 OK
|
||||||
par_retries_bug_from_2010 ### Bug with --retries
|
par_retries_bug_from_2010 ### Bug with --retries
|
||||||
par_retries_bug_from_2010 8
|
par_retries_bug_from_2010 8
|
||||||
par_retries_bug_from_2010 8
|
par_retries_bug_from_2010 8
|
||||||
|
|
|
||||||
|
|
@ -1,433 +1,2 @@
|
||||||
echo "bug #43654: --bar with command not using {} - only last output line "
|
echo moved to parallel-local-1s.sh
|
||||||
bug #43654: --bar with command not using {} - only last output line
|
moved to parallel-local-1s.sh
|
||||||
COLUMNS=80 stdout parallel --bar true {.} ::: 1 | perl -pe 's/.*\r/\r/'
|
|
||||||
[7m100% 1:0=0s 1 [0m[0m
|
|
||||||
echo "### Test --basenamereplace"
|
|
||||||
### Test --basenamereplace
|
|
||||||
parallel -j1 -k -X --basenamereplace FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
|
||||||
b.c b.c b.c b b b
|
|
||||||
parallel -k --basenamereplace FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
|
||||||
b.c
|
|
||||||
b.c
|
|
||||||
b.c
|
|
||||||
b
|
|
||||||
b
|
|
||||||
b
|
|
||||||
echo "### Test --bnr"
|
|
||||||
### Test --bnr
|
|
||||||
parallel -j1 -k -X --bnr FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
|
||||||
b.c b.c b.c b b b
|
|
||||||
parallel -k --bnr FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
|
||||||
b.c
|
|
||||||
b.c
|
|
||||||
b.c
|
|
||||||
b
|
|
||||||
b
|
|
||||||
b
|
|
||||||
echo "### Test --extensionreplace"
|
|
||||||
### Test --extensionreplace
|
|
||||||
parallel -j1 -k -X --extensionreplace FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
|
||||||
/a/b a/b b /a/b a/b b
|
|
||||||
parallel -k --extensionreplace FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
|
||||||
/a/b
|
|
||||||
a/b
|
|
||||||
b
|
|
||||||
/a/b
|
|
||||||
a/b
|
|
||||||
b
|
|
||||||
echo "### Test --er"
|
|
||||||
### Test --er
|
|
||||||
parallel -j1 -k -X --er FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
|
||||||
/a/b a/b b /a/b a/b b
|
|
||||||
parallel -k --er FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
|
||||||
/a/b
|
|
||||||
a/b
|
|
||||||
b
|
|
||||||
/a/b
|
|
||||||
a/b
|
|
||||||
b
|
|
||||||
echo "### Test --basenameextensionreplace"
|
|
||||||
### Test --basenameextensionreplace
|
|
||||||
parallel -j1 -k -X --basenameextensionreplace FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
|
||||||
b b b b b b
|
|
||||||
parallel -k --basenameextensionreplace FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
|
||||||
b
|
|
||||||
b
|
|
||||||
b
|
|
||||||
b
|
|
||||||
b
|
|
||||||
b
|
|
||||||
echo "### Test --bner"
|
|
||||||
### Test --bner
|
|
||||||
parallel -j1 -k -X --bner FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
|
||||||
b b b b b b
|
|
||||||
parallel -k --bner FOO echo FOO ::: /a/b.c a/b.c b.c /a/b a/b b
|
|
||||||
b
|
|
||||||
b
|
|
||||||
b
|
|
||||||
b
|
|
||||||
b
|
|
||||||
b
|
|
||||||
echo "### Test {/}"
|
|
||||||
### Test {/}
|
|
||||||
parallel -j1 -k -X echo {/} ::: /a/b.c a/b.c b.c /a/b a/b b
|
|
||||||
b.c b.c b.c b b b
|
|
||||||
echo "### Test {/.}"
|
|
||||||
### Test {/.}
|
|
||||||
parallel -j1 -k -X echo {/.} ::: /a/b.c a/b.c b.c /a/b a/b b
|
|
||||||
b b b b b b
|
|
||||||
echo "### Test {#/.}"
|
|
||||||
### Test {#/.}
|
|
||||||
parallel -j1 -k -X echo {2/.} ::: /a/number1.c a/number2.c number3.c /a/number4 a/number5 number6
|
|
||||||
number2
|
|
||||||
echo "### Test {#/}"
|
|
||||||
### Test {#/}
|
|
||||||
parallel -j1 -k -X echo {2/} ::: /a/number1.c a/number2.c number3.c /a/number4 a/number5 number6
|
|
||||||
number2.c
|
|
||||||
echo "### Test {#.}"
|
|
||||||
### Test {#.}
|
|
||||||
parallel -j1 -k -X echo {2.} ::: /a/number1.c a/number2.c number3.c /a/number4 a/number5 number6
|
|
||||||
a/number2
|
|
||||||
echo "### bug #34241: --pipe should not spawn unneeded processes"
|
|
||||||
### bug #34241: --pipe should not spawn unneeded processes
|
|
||||||
echo | parallel -r -j2 -N1 --pipe md5sum -c && echo OK
|
|
||||||
OK
|
|
||||||
echo '### Test of quoting of > bug'
|
|
||||||
### Test of quoting of > bug
|
|
||||||
echo '>/dev/null' | parallel echo
|
|
||||||
>/dev/null
|
|
||||||
echo '### Test of quoting of > bug if line continuation'
|
|
||||||
### Test of quoting of > bug if line continuation
|
|
||||||
(echo '> '; echo '> '; echo '>') | parallel --max-lines 3 echo
|
|
||||||
> > >
|
|
||||||
echo '### Test of --trim illegal'
|
|
||||||
### Test of --trim illegal
|
|
||||||
stdout parallel --trim fj ::: echo
|
|
||||||
parallel: Error: --trim must be one of: r l rl lr.
|
|
||||||
echo '### Test of eof string on :::'
|
|
||||||
### Test of eof string on :::
|
|
||||||
parallel -k -E ole echo ::: foo ole bar
|
|
||||||
foo
|
|
||||||
echo '### Test of ignore-empty string on :::'
|
|
||||||
### Test of ignore-empty string on :::
|
|
||||||
parallel -k -r echo ::: foo '' ole bar
|
|
||||||
foo
|
|
||||||
ole
|
|
||||||
bar
|
|
||||||
echo '### Test of trailing space continuation'
|
|
||||||
### Test of trailing space continuation
|
|
||||||
(echo foo; echo '';echo 'ole ';echo bar;echo quux) | xargs -r -L2 echo
|
|
||||||
foo ole bar
|
|
||||||
quux
|
|
||||||
(echo foo; echo '';echo 'ole ';echo bar;echo quux) | parallel -kr -L2 echo
|
|
||||||
foo ole bar
|
|
||||||
quux
|
|
||||||
parallel -kr -L2 echo ::: foo '' 'ole ' bar quux
|
|
||||||
foo ole bar
|
|
||||||
quux
|
|
||||||
echo '### Test of trailing space continuation with -E eof'
|
|
||||||
### Test of trailing space continuation with -E eof
|
|
||||||
(echo foo; echo '';echo 'ole ';echo bar;echo quux) | xargs -r -L2 -E bar echo
|
|
||||||
foo ole
|
|
||||||
(echo foo; echo '';echo 'ole ';echo bar;echo quux) | parallel -kr -L2 -E bar echo
|
|
||||||
foo ole
|
|
||||||
parallel -kr -L2 -E bar echo ::: foo '' 'ole ' bar quux
|
|
||||||
foo ole
|
|
||||||
echo '### Test of --colsep'
|
|
||||||
### Test of --colsep
|
|
||||||
echo 'a%c%b' | parallel --colsep % echo {1} {3} {2}
|
|
||||||
a b c
|
|
||||||
(echo 'a%c%b'; echo a%c%b%d) | parallel -k --colsep % echo {1} {3} {2} {4}
|
|
||||||
a b c
|
|
||||||
a b c d
|
|
||||||
(echo a%c%b; echo d%f%e) | parallel -k --colsep % echo {1} {3} {2}
|
|
||||||
a b c
|
|
||||||
d e f
|
|
||||||
parallel -k --colsep % echo {1} {3} {2} ::: a%c%b d%f%e
|
|
||||||
a b c
|
|
||||||
d e f
|
|
||||||
parallel -k --colsep % echo {1} {3} {2} ::: a%c%b
|
|
||||||
a b c
|
|
||||||
parallel -k --colsep % echo {1} {3} {2} {4} ::: a%c%b a%c%b%d
|
|
||||||
a b c
|
|
||||||
a b c d
|
|
||||||
echo '### Test of tab as colsep'
|
|
||||||
### Test of tab as colsep
|
|
||||||
printf 'def\tabc\njkl\tghi' | parallel -k --colsep '\t' echo {2} {1}
|
|
||||||
abc def
|
|
||||||
ghi jkl
|
|
||||||
parallel -k -a <(printf 'def\tabc\njkl\tghi') --colsep '\t' echo {2} {1}
|
|
||||||
abc def
|
|
||||||
ghi jkl
|
|
||||||
echo '### Test of multiple -a plus colsep'
|
|
||||||
### Test of multiple -a plus colsep
|
|
||||||
parallel --xapply -k -a <(printf 'def\njkl\n') -a <(printf 'abc\tghi\nmno\tpqr') --colsep '\t' echo {2} {1}
|
|
||||||
abc def
|
|
||||||
mno jkl
|
|
||||||
echo '### Test of multiple -a no colsep'
|
|
||||||
### Test of multiple -a no colsep
|
|
||||||
parallel --xapply -k -a <(printf 'ghi\npqr\n') -a <(printf 'abc\tdef\njkl\tmno') echo {2} {1}
|
|
||||||
abc def ghi
|
|
||||||
jkl mno pqr
|
|
||||||
echo '### Test of quoting after colsplit'
|
|
||||||
### Test of quoting after colsplit
|
|
||||||
parallel --colsep % echo {2} {1} ::: '>/dev/null%>/tmp/null'
|
|
||||||
>/tmp/null >/dev/null
|
|
||||||
echo '### Test of --colsep as regexp'
|
|
||||||
### Test of --colsep as regexp
|
|
||||||
(echo 'a%c%%b'; echo a%c%b%d) | parallel -k --colsep %+ echo {1} {3} {2} {4}
|
|
||||||
a b c
|
|
||||||
a b c d
|
|
||||||
parallel -k --colsep %+ echo {1} {3} {2} {4} ::: a%c%%b a%c%b%d
|
|
||||||
a b c
|
|
||||||
a b c d
|
|
||||||
(echo 'a% c %%b'; echo a%c% b %d) | parallel -k --colsep %+ echo {1} {3} {2} {4}
|
|
||||||
a b c
|
|
||||||
a b c d
|
|
||||||
(echo 'a% c %%b'; echo a%c% b %d) | parallel -k --colsep %+ echo '"{1}_{3}_{2}_{4}"'
|
|
||||||
a_b_c_
|
|
||||||
a_b_c_d
|
|
||||||
echo '### Test of -C'
|
|
||||||
### Test of -C
|
|
||||||
(echo 'a% c %%b'; echo a%c% b %d) | parallel -k -C %+ echo '"{1}_{3}_{2}_{4}"'
|
|
||||||
a_b_c_
|
|
||||||
a_b_c_d
|
|
||||||
echo '### Test of --trim n'
|
|
||||||
### Test of --trim n
|
|
||||||
(echo 'a% c %%b'; echo a%c% b %d) | parallel -k --trim n --colsep %+ echo '"{1}_{3}_{2}_{4}"'
|
|
||||||
a_b_' c '_
|
|
||||||
a_' b '_c_d
|
|
||||||
parallel -k -C %+ echo '"{1}_{3}_{2}_{4}"' ::: 'a% c %%b' 'a%c% b %d'
|
|
||||||
a_b_c_
|
|
||||||
a_b_c_d
|
|
||||||
echo '### Test of bug: If input is empty string'
|
|
||||||
### Test of bug: If input is empty string
|
|
||||||
(echo ; echo abcbdbebf;echo abc) | parallel -k --colsep b -v echo {1}{2}
|
|
||||||
echo ''
|
|
||||||
|
|
||||||
echo ac
|
|
||||||
ac
|
|
||||||
echo ac
|
|
||||||
ac
|
|
||||||
echo '### Test :::: mixed with :::'
|
|
||||||
### Test :::: mixed with :::
|
|
||||||
echo '### Test :::: < ::: :::'
|
|
||||||
### Test :::: < ::: :::
|
|
||||||
parallel -k echo {1} {2} {3} :::: <(seq 6 7) ::: 4 5 ::: 1 2 3
|
|
||||||
6 4 1
|
|
||||||
6 4 2
|
|
||||||
6 4 3
|
|
||||||
6 5 1
|
|
||||||
6 5 2
|
|
||||||
6 5 3
|
|
||||||
7 4 1
|
|
||||||
7 4 2
|
|
||||||
7 4 3
|
|
||||||
7 5 1
|
|
||||||
7 5 2
|
|
||||||
7 5 3
|
|
||||||
echo '### Test :::: < < :::: <'
|
|
||||||
### Test :::: < < :::: <
|
|
||||||
parallel -k echo {1} {2} {3} :::: <(seq 6 7) <(seq 4 5) :::: <(seq 1 3)
|
|
||||||
6 4 1
|
|
||||||
6 4 2
|
|
||||||
6 4 3
|
|
||||||
6 5 1
|
|
||||||
6 5 2
|
|
||||||
6 5 3
|
|
||||||
7 4 1
|
|
||||||
7 4 2
|
|
||||||
7 4 3
|
|
||||||
7 5 1
|
|
||||||
7 5 2
|
|
||||||
7 5 3
|
|
||||||
echo '### Test -a :::: < :::: <'
|
|
||||||
### Test -a :::: < :::: <
|
|
||||||
parallel -k -a <(seq 6 7) echo {1} {2} {3} :::: <(seq 4 5) :::: <(seq 1 3)
|
|
||||||
6 4 1
|
|
||||||
6 4 2
|
|
||||||
6 4 3
|
|
||||||
6 5 1
|
|
||||||
6 5 2
|
|
||||||
6 5 3
|
|
||||||
7 4 1
|
|
||||||
7 4 2
|
|
||||||
7 4 3
|
|
||||||
7 5 1
|
|
||||||
7 5 2
|
|
||||||
7 5 3
|
|
||||||
echo '### Test -a -a :::'
|
|
||||||
### Test -a -a :::
|
|
||||||
parallel -k -a <(seq 6 7) -a <(seq 4 5) echo {1} {2} {3} ::: 1 2 3
|
|
||||||
6 4 1
|
|
||||||
6 4 2
|
|
||||||
6 4 3
|
|
||||||
6 5 1
|
|
||||||
6 5 2
|
|
||||||
6 5 3
|
|
||||||
7 4 1
|
|
||||||
7 4 2
|
|
||||||
7 4 3
|
|
||||||
7 5 1
|
|
||||||
7 5 2
|
|
||||||
7 5 3
|
|
||||||
echo '### Test -a - -a :::'
|
|
||||||
### Test -a - -a :::
|
|
||||||
seq 6 7 | parallel -k -a - -a <(seq 4 5) echo {1} {2} {3} ::: 1 2 3
|
|
||||||
6 4 1
|
|
||||||
6 4 2
|
|
||||||
6 4 3
|
|
||||||
6 5 1
|
|
||||||
6 5 2
|
|
||||||
6 5 3
|
|
||||||
7 4 1
|
|
||||||
7 4 2
|
|
||||||
7 4 3
|
|
||||||
7 5 1
|
|
||||||
7 5 2
|
|
||||||
7 5 3
|
|
||||||
echo '### Test :::: < - :::'
|
|
||||||
### Test :::: < - :::
|
|
||||||
seq 4 5 | parallel -k echo {1} {2} {3} :::: <(seq 6 7) - ::: 1 2 3
|
|
||||||
6 4 1
|
|
||||||
6 4 2
|
|
||||||
6 4 3
|
|
||||||
6 5 1
|
|
||||||
6 5 2
|
|
||||||
6 5 3
|
|
||||||
7 4 1
|
|
||||||
7 4 2
|
|
||||||
7 4 3
|
|
||||||
7 5 1
|
|
||||||
7 5 2
|
|
||||||
7 5 3
|
|
||||||
echo '### Test -E'
|
|
||||||
### Test -E
|
|
||||||
seq 1 100 | parallel -k -E 5 echo :::: - ::: 2 3 4 5 6 7 8 9 10 :::: <(seq 3 11)
|
|
||||||
1 2 3
|
|
||||||
1 2 4
|
|
||||||
1 3 3
|
|
||||||
1 3 4
|
|
||||||
1 4 3
|
|
||||||
1 4 4
|
|
||||||
2 2 3
|
|
||||||
2 2 4
|
|
||||||
2 3 3
|
|
||||||
2 3 4
|
|
||||||
2 4 3
|
|
||||||
2 4 4
|
|
||||||
3 2 3
|
|
||||||
3 2 4
|
|
||||||
3 3 3
|
|
||||||
3 3 4
|
|
||||||
3 4 3
|
|
||||||
3 4 4
|
|
||||||
4 2 3
|
|
||||||
4 2 4
|
|
||||||
4 3 3
|
|
||||||
4 3 4
|
|
||||||
4 4 3
|
|
||||||
4 4 4
|
|
||||||
echo '### Test -E one empty'
|
|
||||||
### Test -E one empty
|
|
||||||
seq 1 100 | parallel -k -E 3 echo :::: - ::: 2 3 4 5 6 7 8 9 10 :::: <(seq 3 11)
|
|
||||||
1 2
|
|
||||||
2 2
|
|
||||||
echo '### Test -E 2 empty'
|
|
||||||
### Test -E 2 empty
|
|
||||||
seq 1 100 | parallel -k -E 3 echo :::: - ::: 3 4 5 6 7 8 9 10 :::: <(seq 3 11)
|
|
||||||
1
|
|
||||||
2
|
|
||||||
echo '### Test -E all empty'
|
|
||||||
### Test -E all empty
|
|
||||||
seq 3 100 | parallel -k -E 3 echo :::: - ::: 3 4 5 6 7 8 9 10 :::: <(seq 3 11)
|
|
||||||
echo '### Test {#}'
|
|
||||||
### Test {#}
|
|
||||||
seq 1 10 | parallel -k echo {#}
|
|
||||||
1
|
|
||||||
2
|
|
||||||
3
|
|
||||||
4
|
|
||||||
5
|
|
||||||
6
|
|
||||||
7
|
|
||||||
8
|
|
||||||
9
|
|
||||||
10
|
|
||||||
echo '### Test --seqreplace and line too long'
|
|
||||||
### Test --seqreplace and line too long
|
|
||||||
seq 1 1000 | stdout parallel -j1 -s 210 -k --seqreplace I echo IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII \|wc | uniq -c
|
|
||||||
9 1 1 101
|
|
||||||
90 1 1 201
|
|
||||||
1 parallel: Error: Command line too long (309 >= 210) at input 0: 100
|
|
||||||
echo '### bug #37042: -J foo is taken from the whole command line - not just the part before the command'
|
|
||||||
### bug #37042: -J foo is taken from the whole command line - not just the part before the command
|
|
||||||
echo '--tagstring foo' > ~/.parallel/bug_37042_profile; parallel -J bug_37042_profile echo ::: tag_with_foo; parallel --tagstring a -J bug_37042_profile echo ::: tag_with_a; parallel --tagstring a echo -J bug_37042_profile ::: print_-J_bug_37042_profile
|
|
||||||
foo tag_with_foo
|
|
||||||
a tag_with_a
|
|
||||||
a -J bug_37042_profile print_-J_bug_37042_profile
|
|
||||||
echo '### Bug introduce by fixing bug #37042'
|
|
||||||
### Bug introduce by fixing bug #37042
|
|
||||||
parallel --xapply -a <(printf 'abc') --colsep '\t' echo {1}
|
|
||||||
abc
|
|
||||||
echo "### Test --header with -N"
|
|
||||||
### Test --header with -N
|
|
||||||
(echo h1; echo h2; echo 1a;echo 1b; echo 2a;echo 2b; echo 3a)| parallel -j1 --pipe -N2 -k --header '.*\n.*\n' echo Start\;cat \; echo Stop
|
|
||||||
Start
|
|
||||||
h1
|
|
||||||
h2
|
|
||||||
1a
|
|
||||||
1b
|
|
||||||
Stop
|
|
||||||
Start
|
|
||||||
h1
|
|
||||||
h2
|
|
||||||
2a
|
|
||||||
2b
|
|
||||||
Stop
|
|
||||||
Start
|
|
||||||
h1
|
|
||||||
h2
|
|
||||||
3a
|
|
||||||
Stop
|
|
||||||
echo "### Test --header with --block 1k"
|
|
||||||
### Test --header with --block 1k
|
|
||||||
(echo h1; echo h2; perl -e '$a="x"x110;for(1..22){print $_,$a,"\n"'})| parallel -j1 --pipe -k --block 1k --header '.*\n.*\n' echo Start\;cat \; echo Stop
|
|
||||||
Start
|
|
||||||
h1
|
|
||||||
h2
|
|
||||||
1xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
|
||||||
2xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
|
||||||
3xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
|
||||||
4xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
|
||||||
5xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
|
||||||
6xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
|
||||||
7xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
|
||||||
8xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
|
||||||
Stop
|
|
||||||
Start
|
|
||||||
h1
|
|
||||||
h2
|
|
||||||
9xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
|
||||||
10xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
|
||||||
11xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
|
||||||
12xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
|
||||||
13xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
|
||||||
14xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
|
||||||
15xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
|
||||||
16xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
|
||||||
Stop
|
|
||||||
Start
|
|
||||||
h1
|
|
||||||
h2
|
|
||||||
17xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
|
||||||
18xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
|
||||||
19xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
|
||||||
20xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
|
||||||
21xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
|
||||||
22xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
|
||||||
Stop
|
|
||||||
echo "### Test --header with multiple :::"
|
|
||||||
### Test --header with multiple :::
|
|
||||||
parallel --header : echo {a} {b} {1} {2} ::: b b1 ::: a a2
|
|
||||||
a2 b1 b1 a2
|
|
||||||
|
|
|
||||||
|
|
@ -39,7 +39,7 @@ https://www.gnu.org/software/parallel/parallel_design.html#Citation-notice
|
||||||
To silence this citation notice: run 'parallel --citation' once.
|
To silence this citation notice: run 'parallel --citation' once.
|
||||||
|
|
||||||
10 files to edit
|
10 files to edit
|
||||||
[?2004h[?1049h[22;0;0t[?1h=[?2004h[1;24r[?12h[?12l[27m[23m[29m[m[H[2J[?25l[24;1H"file1" [New File][2;1H½[6n[2;1H [1;1H[>c]10;?]11;?[2;1H[94m~ [3;1H~ [4;1H~ [5;1H~ [6;1H~ [7;1H~ [8;1H~ [9;1H~ [10;1H~ [11;1H~ [12;1H~ [13;1H~ [14;1H~ [15;1H~ [16;1H~ [17;1H~ [18;1H~ [19;1H~ [20;1H~ [21;1H~ [22;1H~ [23;1H~ [m[24;63H0,0-1[9CAll[1;1H[?25h[?25l[24;1H[97m[41mE173: 9 more files to edit[m[24;63H[K[?2004h[24;63H0,0-1[9CAll[1;1H[?25h[24;1H[?2004l[24;1H[K[24;1H[?2004l[?1l>[?1049l[23;0;0tAcademic tradition requires you to cite works you base your article on.
|
[?2004h[?1049h[22;0;0t[?1h=[?2004h[1;24r[?12h[?12l[27m[23m[29m[m[H[2J[?25l[24;1H"file1" [New File][2;1H[94m~ [3;1H~ [4;1H~ [5;1H~ [6;1H~ [7;1H~ [8;1H~ [9;1H~ [10;1H~ [11;1H~ [12;1H~ [13;1H~ [14;1H~ [15;1H~ [16;1H~ [17;1H~ [18;1H~ [19;1H~ [20;1H~ [21;1H~ [22;1H~ [23;1H~ [1;1H[?25h[?25l[m[24;1H[97m[41mE173: 9 more files to edit[?2004h[1;1H[?25h[24;1H[?2004l[m[24;1H[K[24;1H[?2004l[?1l>[?1049l[23;0;0tAcademic tradition requires you to cite works you base your article on.
|
||||||
If you use programs that use GNU Parallel to process data for an article in a
|
If you use programs that use GNU Parallel to process data for an article in a
|
||||||
scientific publication, please cite:
|
scientific publication, please cite:
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -61,7 +61,7 @@ echo '### Check that 4 processes are really used'
|
||||||
echo '### --version must have higher priority than retired options'
|
echo '### --version must have higher priority than retired options'
|
||||||
### --version must have higher priority than retired options
|
### --version must have higher priority than retired options
|
||||||
$NICEPAR --version -g -Y -U -W -T | tail -n9
|
$NICEPAR --version -g -Y -U -W -T | tail -n9
|
||||||
Copyright (C) 2007-2019 Ole Tange and Free Software Foundation, Inc.
|
Foundation, Inc.
|
||||||
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
|
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
|
||||||
This is free software: you are free to change and redistribute it.
|
This is free software: you are free to change and redistribute it.
|
||||||
GNU parallel comes with no warranty.
|
GNU parallel comes with no warranty.
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue