tange/parallel
1
0
Fork 0
mirror of https://git.savannah.gnu.org/git/parallel.git synced 2024-12-23 13:17:54 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

parallel: ignore errors from sysctl. sr #107762: get multiple sysctl: not found message when running parallel

Browse source
This commit is contained in:
Ole Tange 2011-08-02 23:18:28 +02:00
parent 82b50467ce
commit 3a7097684c
3 changed files with 55 additions and 20 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

37
doc/release_new_version
View file

@ -166,24 +166,41 @@ https://lists.gnu.org/mailman/admindb/parallel
from:tange@gnu.org from:tange@gnu.org
to:parallel@gnu.org, bug-parallel@gnu.org, info-gnu@gnu.org, bug-directory@gnu.org to:parallel@gnu.org, bug-parallel@gnu.org, info-gnu@gnu.org, bug-directory@gnu.org
cc:Peter Simons <simons@cryp.to>, Sandro Cazzaniga <kharec@mandriva.org>, cc:Peter Simons <simons@cryp.to>, Sandro Cazzaniga <kharec@mandriva.org>,
Tim Cuthbertson <tim3d.junk@gmail.com>, Ludovic Courtès <ludo@gnu.org>, Tim Cuthbertson <tim3d.junk@gmail.com>, Ludovic Courtès
Markus Ammer <mkmm@gmx-topmail.de>, Pavel Nuzhdin <pnzhdin@gmail.com>, <ludo@gnu.org>, Markus Ammer <mkmm@gmx-topmail.de>, Pavel Nuzhdin
Phil Sung <psung@alum.mit.edu>, Michael Shigorin <mike@altlinux.org>, <pnzhdin@gmail.com>, Phil Sung <psung@alum.mit.edu>, Michael
Andrew McFague <amcfague@wgen.net>, Steven M. Christensen <sunfreeware@gmail.com>, Shigorin <mike@altlinux.org>, Andrew McFague <amcfague@wgen.net>,
Chris Howey <howeyc@gmail.com>, Fethican Coşkuner <fethicanc@gmail.com>, Steven M. Christensen <sunfreeware@gmail.com>, Chris Howey
Rogério Brito <rbrito@ime.usp.br>, <howeyc@gmail.com>, Fethican Coşkuner <fethicanc@gmail.com>,
Koen Vervloesem <koen@vervloesem.eu>, R. Tyler Croy <tyler@monkeypox.org>, Rogério Brito <rbrito@ime.usp.br>, Koen Vervloesem
ryoichiro.suzuki@gmail.com,kerick@shiftedbit.net, <koen@vervloesem.eu>, R. Tyler Croy <tyler@monkeypox.org>,
Christian Faulhammer <fauli@gentoo.org>, Ryoichiro Suzuki <ryoichiro.suzuki@gmail.com>, kerick@shiftedbit.net, Christian Faulhammer <fauli@gentoo.org>,
Jesse Alama <jesse.alama@gmail.com> Ryoichiro Suzuki <ryoichiro.suzuki@gmail.com>, Jesse Alama
<jesse.alama@gmail.com>
Subject: GNU Parallel 20110822 ('Utøya') released Subject: GNU Parallel 20110822 ('Utøya') released
GNU Parallel 20110822 ('Utøya') has been released. It is GNU Parallel 20110822 ('Utøya') has been released. It is
available for download at: http://ftp.gnu.org/gnu/parallel/ available for download at: http://ftp.gnu.org/gnu/parallel/
Debian and Ubuntu people have complained GNU Parallel is not in their
official distributions.
Ubuntu people: Log into LaunchPad and on
https://bugs.launchpad.net/ubuntu/+bug/740630 click 'This bug affect
me too'.
Debian people: This is the (2 years old) bug to push:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=518696
New in this release: New in this release:
* Neat example of a parallel webcrawler in the man page.
* Blog post in Japanese.
http://dminor11th.blogspot.com/2011/08/gnu-parallel.html
* Blog post about optimizing JPEGs. Thanks to Thomas Jost. * Blog post about optimizing JPEGs. Thanks to Thomas Jost.
http://schnouki.net/2011/07/22/optimizing-jpeg-pictures/ http://schnouki.net/2011/07/22/optimizing-jpeg-pictures/

8
src/parallel
View file

@ -480,7 +480,7 @@ sub get_options_from_array {
sub parse_options { sub parse_options {
# Returns: N/A # Returns: N/A
# Defaults: # Defaults:
$Global::version = 20110722; $Global::version = 20110802;
$Global::progname = 'parallel'; $Global::progname = 'parallel';
$Global::infinity = 2**31; $Global::infinity = 2**31;
$Global::debug = 0; $Global::debug = 0;
@ -2463,7 +2463,7 @@ sub no_of_cpus_darwin {
# Number of physical CPUs on Mac Darwin # Number of physical CPUs on Mac Darwin
# undef if not Mac Darwin # undef if not Mac Darwin
my $no_of_cpus = my $no_of_cpus =
(`sysctl -n hw.physicalcpu` (`sysctl -n hw.physicalcpu 2>/dev/null`
or or
`sysctl -a hw 2>/dev/null | grep -w physicalcpu | awk '{ print \$2 }'`); `sysctl -a hw 2>/dev/null | grep -w physicalcpu | awk '{ print \$2 }'`);
return $no_of_cpus; return $no_of_cpus;
@ -2474,7 +2474,7 @@ sub no_of_cores_darwin {
# Number of CPU cores on Mac Darwin # Number of CPU cores on Mac Darwin
# undef if not Mac Darwin # undef if not Mac Darwin
my $no_of_cores = my $no_of_cores =
(`sysctl -n hw.logicalcpu` (`sysctl -n hw.logicalcpu 2>/dev/null`
or or
`sysctl -a hw 2>/dev/null | grep -w logicalcpu | awk '{ print \$2 }'`); `sysctl -a hw 2>/dev/null | grep -w logicalcpu | awk '{ print \$2 }'`);
return $no_of_cores; return $no_of_cores;
@ -2485,7 +2485,7 @@ sub no_of_cpus_freebsd {
# Number of physical CPUs on FreeBSD # Number of physical CPUs on FreeBSD
# undef if not FreeBSD # undef if not FreeBSD
my $no_of_cpus = my $no_of_cpus =
(`sysctl -a dev.cpu | grep \%parent | awk '{ print \$2 }' | uniq | wc -l | awk '{ print \$1 }'` (`sysctl -a dev.cpu 2>/dev/null | grep \%parent | awk '{ print \$2 }' | uniq | wc -l | awk '{ print \$1 }'`
or or
`sysctl hw.ncpu 2>/dev/null | awk '{ print \$2 }'`); `sysctl hw.ncpu 2>/dev/null | awk '{ print \$2 }'`);
chomp $no_of_cpus; chomp $no_of_cpus;

30
src/parallel.pod
View file

@ -51,9 +51,9 @@ B<parallel> is capable of.
You can also watch the intro video for a quick introduction: You can also watch the intro video for a quick introduction:
http://tinyogg.com/watch/TORaR/ http://tinyogg.com/watch/hfxKj/ and http://tinyogg.com/watch/TORaR/ http://tinyogg.com/watch/hfxKj/ and
http://tinyogg.com/watch/YQuXd/ or http://tinyogg.com/watch/YQuXd/ or
http://www.youtube.com/watch?v=OpaiGYxkSuQ and http://www.youtube.com/watch?v=OpaiGYxkSuQ
http://www.youtube.com/watch?v=1ntxT-47VPA http://www.youtube.com/watch?v=1ntxT-47VPA and
http://www.youtube.com/watch?v=fOX1EyHkQwc
=head1 OPTIONS =head1 OPTIONS
@ -1368,19 +1368,19 @@ B<parallel lame {} -o {.}.mp3 ::: *.wav>
=head1 EXAMPLE: Inserting multiple arguments =head1 EXAMPLE: Inserting multiple arguments
When moving a lot of files like this: B<mv * destdir> you will When moving a lot of files like this: B<mv *.log destdir> you will
sometimes get the error: sometimes get the error:
B<bash: /bin/mv: Argument list too long> B<bash: /bin/mv: Argument list too long>
because there are too many files. You can instead do: because there are too many files. You can instead do:
B<ls | parallel mv {} destdir> B<ls | grep -E '\.log$' | parallel mv {} destdir>
This will run B<mv> for each file. It can be done faster if B<mv> gets This will run B<mv> for each file. It can be done faster if B<mv> gets
as many arguments that will fit on the line: as many arguments that will fit on the line:
B<ls | parallel -m mv {} destdir> B<ls | grep -E '\.log$' | parallel -m mv {} destdir>
=head1 EXAMPLE: Context replace =head1 EXAMPLE: Context replace
@ -1860,6 +1860,24 @@ B<parallel --onall -S server1,server2 echo ::: 1 2 3>
If you have a lot of hosts use '-j0' to access more hosts in parallel. If you have a lot of hosts use '-j0' to access more hosts in parallel.
=head1 EXAMPLE: Parallelizing rsync
B<rsync> is a great tool, but sometimes it will not fill up the
available bandwidth. This is often a problem when copying several big
files over high speed connections.
The following will start one B<rsync> per big file in I<src-dir> to
I<dest-dir> on the server I<fooserver>:
B<find src-dir -type f -size +100000 | parallel -v ssh fooserver
mkdir -p /dest-dir/{//}\;rsync -Havessh {} fooserver:/dest-dir/{}>
The dirs created may end up with wrong permissions and smaller files
are not being transferred. To fix those run B<rsync> a final time:
B<rsync -Havessh src-dir/ fooserver:/dest-dir/>
=head1 EXAMPLE: Use multiple inputs in one command =head1 EXAMPLE: Use multiple inputs in one command
Copy files like foo.es.ext to foo.ext: Copy files like foo.es.ext to foo.ext: