tange/parallel
1
0
Fork 0
mirror of https://git.savannah.gnu.org/git/parallel.git synced 2024-11-22 22:17:54 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

parallel: Obsoleted --tollef. To be retired 20140222.

Browse source
This commit is contained in:
Ole Tange 2013-02-18 00:59:59 +01:00
parent 21c62a2545
commit 07c5b5c704
4 changed files with 67 additions and 42 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

38
doc/release_new_version
View file

@ -171,31 +171,39 @@ cc:Sandro Cazzaniga <kharec@mandriva.org>,
Ryoichiro Suzuki <ryoichiro.suzuki@gmail.com>, Ryoichiro Suzuki <ryoichiro.suzuki@gmail.com>,
Jesse Alama <jesse.alama@gmail.com> Jesse Alama <jesse.alama@gmail.com>
Subject: GNU Parallel 20130122 ('Jyoti Singh Pandey') released Subject: GNU Parallel 20130222 ('Chelyabinsk') released
GNU Parallel 20130122 ('Jyoti Singh Pandey') has been released. It is GNU Parallel 20130222 ('Chelyabinsk') has been released. It is
available for download at: http://ftp.gnu.org/gnu/parallel/ available for download at: http://ftp.gnu.org/gnu/parallel/
New in this release: New in this release:
* --sshdelay Delay starting next ssh by secs seconds. GNU parallel * --resume works with --pipe.
will pause secs seconds after starting each ssh. secs can be less
than 1 seconds.
* Official OpenBSD port: * --resume-failed will go through --joblog, redo the failed jobs and
http://ftp.openbsd.org/ports/sysutils/parallel/ then continue like --resume.
* Official DragonFlyBSD package: * Negative positional arguments count from the end: {-1} means the
http://www.mirrorservice.org/sites/ftp.dragonflybsd.org/packages/amd64/DragonFly-3.0.0/stable/parallel/ last argument, {-2} the second to last.
* Post about niceload in Hungarian: * NetBSD CPU detection.
http://commandline.blog.hu/2013/01/02/niceload
* 自炊スキャンデータをKobo Gloに最適化 * --blocksize increases exponentially if it smaller than a full
http://interstadial.wordpress.com/2013/01/20/ record.
* GNU parallel+ssh で複数のリモートホストに複数のコマンドを実行させる * Processing n-line records (--pipe -L n) is now much faster.
http://oshiire.to/archives/1686
* GNU Parallel is the highest rated tool on:
http://www.biostars.org/show/tools/?sort=votes&since=all%20time
* GNU Parallel was loved during FSFE's #ilovefs campaign.
http://fsfe.org/news/2013/news-20130212-01.en.html
* Using GNU Parallel with s3cmd (Japanese).
http://blog.suz-lab.com/2013/02/s3cmd-gnu-paralells3.html
* Intro to GNU Parallel (Chinese).
http://guiquanz.github.com/2013/02/12/gnu-parallel-intro/
* Bug fixes and man page updates. * Bug fixes and man page updates.

35
src/parallel
View file

@ -1,6 +1,6 @@
#!/usr/bin/perl -w #!/usr/bin/perl -w
# Copyright (C) 2007,2008,2009,2010,2011,2012 Ole Tange and Free Software # Copyright (C) 2007,2008,2009,2010,2011,2012,2013 Ole Tange and Free Software
# Foundation, Inc. # Foundation, Inc.
# #
# This program is free software; you can redistribute it and/or modify # This program is free software; you can redistribute it and/or modify
@ -637,7 +637,7 @@ sub get_options_from_array {
sub parse_options { sub parse_options {
# Returns: N/A # Returns: N/A
# Defaults: # Defaults:
$Global::version = 20130212; $Global::version = 20130217;
$Global::progname = 'parallel'; $Global::progname = 'parallel';
$Global::infinity = 2**31; $Global::infinity = 2**31;
$Global::debug = 0; $Global::debug = 0;
@ -749,6 +749,8 @@ sub parse_options {
} }
if($opt::tollef and not $opt::gnu and not $opt::plain) { if($opt::tollef and not $opt::gnu and not $opt::plain) {
# Behave like tollef parallel (from moreutils) # Behave like tollef parallel (from moreutils)
::warning("YOU ARE USING --tollef. --tollef is obsolete and will be retired 20140222.\n");
::warning("See: http://lists.gnu.org/archive/html/parallel/2013-02/msg00018.html\n");
$opt::u = 1; $opt::u = 1;
$Global::grouped = 0; $Global::grouped = 0;
$Global::quoting = 1; $Global::quoting = 1;
@ -1294,19 +1296,19 @@ sub start_more_jobs {
for my $sshlogin (values %Global::host) { for my $sshlogin (values %Global::host) {
debug("Running jobs before on ".$sshlogin->string().": ".$sshlogin->jobs_running()."\n"); debug("Running jobs before on ".$sshlogin->string().": ".$sshlogin->jobs_running()."\n");
if($opt::load and $sshlogin->loadavg_too_high()) { while ($sshlogin->jobs_running() < $sshlogin->max_jobs_running()) {
# The load is too high or unknown if($opt::load and $sshlogin->loadavg_too_high()) {
next; # The load is too high or unknown
} next;
if($opt::noswap and $sshlogin->swapping()) { }
# The server is swapping if($opt::noswap and $sshlogin->swapping()) {
next; # The server is swapping
} next;
if($sshlogin->too_fast_remote_login()) { }
next; if($sshlogin->too_fast_remote_login()) {
} next;
while ($sshlogin->jobs_running() < $sshlogin->max_jobs_running()) { }
if($Global::JobQueue->empty() and not $opt::pipe) { if($Global::JobQueue->empty() and not $opt::pipe) {
last; last;
} }
debug($sshlogin->string()." has ".$sshlogin->jobs_running() debug($sshlogin->string()." has ".$sshlogin->jobs_running()
@ -2018,7 +2020,7 @@ sub version {
} }
print join("\n", print join("\n",
"GNU $Global::progname $Global::version", "GNU $Global::progname $Global::version",
"Copyright (C) 2007,2008,2009,2010,2011,2012 Ole Tange and Free Software Foundation, Inc.", "Copyright (C) 2007,2008,2009,2010,2011,2012,2013 Ole Tange and Free Software Foundation, Inc.",
"License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>", "License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>",
"This is free software: you are free to change and redistribute it.", "This is free software: you are free to change and redistribute it.",
"GNU $Global::progname comes with no warranty.", "GNU $Global::progname comes with no warranty.",
@ -2513,6 +2515,7 @@ sub loadavg {
return $self->{'loadavg'}; return $self->{'loadavg'};
} }
sub max_loadavg { sub max_loadavg {
my $self = shift; my $self = shift;
if(not defined $self->{'max_loadavg'}) { if(not defined $self->{'max_loadavg'}) {

32
src/parallel.pod
View file

@ -479,7 +479,7 @@ See also: B<--bg>, B<man sem>
Implies B<--semaphore>. Implies B<--semaphore>.
=item B<--filter-hosts> (beta testing) =item B<--filter-hosts> (alpha testing)
Remove down hosts. For each remote host: check that login through ssh Remove down hosts. For each remote host: check that login through ssh
works. If not: do not use this host. works. If not: do not use this host.
@ -1040,7 +1040,20 @@ there. As GNU B<parallel> only looks at the sequence numbers in
B<--joblog> then the input, the command, and B<--joblog> all have to B<--joblog> then the input, the command, and B<--joblog> all have to
remain unchanged; otherwise GNU B<parallel> may run wrong commands. remain unchanged; otherwise GNU B<parallel> may run wrong commands.
See also: B<--joblog>. See also: B<--joblog>, B<--resume-failed>.
=item B<--resume-failed> (alpha testing)
Retry all failed and resume from the last unfinished job. By reading
B<--joblog> GNU B<parallel> will figure out the failed jobs and run
those again. After that it will resume last unfinished job and
continue from there. As GNU B<parallel> only looks at the sequence
numbers in B<--joblog> then the input, the command, and B<--joblog>
all have to remain unchanged; otherwise GNU B<parallel> may run wrong
commands.
See also: B<--joblog>, B<--resume>.
=item B<--retries> I<n> =item B<--retries> I<n>
@ -1178,7 +1191,7 @@ Like this:
B<--shebang> must be set as the first option. B<--shebang> must be set as the first option.
=item B<--shebang-wrap> (alpha testing) =item B<--shebang-wrap> (beta testing)
GNU B<parallel> can parallelize scripts by wrapping the shebang GNU B<parallel> can parallelize scripts by wrapping the shebang
line. If the program can be run like this: line. If the program can be run like this:
@ -1497,9 +1510,9 @@ Use B<-v> B<-v> to print the wrapping ssh command when running remotely.
Print the version GNU B<parallel> and exit. Print the version GNU B<parallel> and exit.
=item B<--workdir> I<mydir> (beta testing) =item B<--workdir> I<mydir>
=item B<--wd> I<mydir> (beta testing) =item B<--wd> I<mydir>
Files transferred using B<--transfer> and B<--return> will be relative Files transferred using B<--transfer> and B<--return> will be relative
to I<mydir> on remote computers, and the command will be executed in to I<mydir> on remote computers, and the command will be executed in
@ -3432,10 +3445,11 @@ fixed in that version.
=item * =item *
A complete example that others can run that shows the problem. A A complete example that others can run that shows the problem. This
combination of B<seq>, B<cat>, B<echo>, and B<sleep> can reproduce should preferably be small and simple. A combination of B<seq>,
most errors. If your example requires large files, see if you can make B<cat>, B<echo>, and B<sleep> can reproduce most errors. If your
them by something like B<seq 1000000> > B<file>. example requires large files, see if you can make them by something
like B<seq 1000000> > B<file>.
=item * =item *

4
src/sem.pod
View file

@ -178,13 +178,13 @@ Report bugs to <bug-parallel@gnu.org>.
=head1 AUTHOR =head1 AUTHOR
Copyright (C) 2010,2011,2012 Ole Tange, http://ole.tange.dk and Free Copyright (C) 2010,2011,2012,2013 Ole Tange, http://ole.tange.dk and Free
Software Foundation, Inc. Software Foundation, Inc.
=head1 LICENSE =head1 LICENSE
Copyright (C) 2010,2011,2012 Free Software Foundation, Inc. Copyright (C) 2010,2011,2012,2013 Free Software Foundation, Inc.
This program is free software; you can redistribute it and/or modify This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by it under the terms of the GNU General Public License as published by