mirror of
https://git.savannah.gnu.org/git/parallel.git
synced 2024-11-22 05:57:54 +00:00
parallel: /tmp/*.arg was not removed
This commit is contained in:
parent
8fb1966254
commit
10ac95d29a
29
README
29
README
|
@ -5,22 +5,23 @@ Please send problems and feedback to bug-parallel@gnu.org.
|
||||||
|
|
||||||
= Presentation of GNU Parallel =
|
= Presentation of GNU Parallel =
|
||||||
|
|
||||||
GNU Parallel is a shell tool for executing jobs in parallel using one
|
GNU parallel is a shell tool for executing jobs in parallel using one
|
||||||
or more computers. A job is typically a single command or a small
|
or more computers. A job is can be a single command or a small script
|
||||||
script that has to be run for each of the lines in the input. The
|
that has to be run for each of the lines in the input. The typical
|
||||||
typical input is a list of files, a list of hosts, a list of users, or
|
input is a list of files, a list of hosts, a list of users, a list of
|
||||||
a list of tables.
|
URLs, or a list of tables. A job can also be a command that reads from
|
||||||
|
a pipe. GNU parallel can then split the input and pipe it into
|
||||||
|
commands in parallel.
|
||||||
|
|
||||||
If you use xargs today you will find GNU Parallel very easy to use. If
|
If you use xargs and tee today you will find GNU parallel very easy to
|
||||||
you write loops in shell, you will find GNU Parallel may be able to
|
use as GNU parallel is written to have the same options as xargs. If
|
||||||
replace most of the loops and make them run faster by running jobs in
|
you write loops in shell, you will find GNU parallel may be able to
|
||||||
parallel. If you use ppss or pexec you will find GNU Parallel will
|
replace most of the loops and make them run faster by running several
|
||||||
often make the command easier to read.
|
jobs in parallel.
|
||||||
|
|
||||||
GNU Parallel also makes sure output from the commands is the same
|
GNU parallel makes sure output from the commands is the same output as
|
||||||
output as you would get had you run the commands sequentially. This
|
you would get had you run the commands sequentially. This makes it
|
||||||
makes it possible to use output from GNU Parallel as input for other
|
possible to use output from GNU parallel as input for other programs.
|
||||||
programs.
|
|
||||||
|
|
||||||
|
|
||||||
= Installation =
|
= Installation =
|
||||||
|
|
|
@ -1,72 +1,106 @@
|
||||||
Video
|
Video 30. 36. 41. 48
|
||||||
|
|
||||||
|
|
||||||
|
# GNU Parallel 20110522 - The Pakistan Release
|
||||||
|
|
||||||
|
I am Ole Tange. I am the author of GNU Parallel.
|
||||||
|
|
||||||
So far GNU Parallel has been focused on replacing a single
|
So far GNU Parallel has been focused on replacing a single
|
||||||
for-loop. The Pakistan release introduces a way to replace nested
|
for-loop. The Pakistan release introduces a way to replace nested
|
||||||
loops.
|
loops.
|
||||||
|
|
||||||
As example I will use the image manipulation program 'convert' from
|
# NESTED FOR LOOPS
|
||||||
ImageMagick. This will convert foo.png to jpg with a size of 800 and
|
|
||||||
JPEG-quality of 95.
|
|
||||||
|
|
||||||
convert -size 800 -quality 95 foo.png foo_800_q95.jpg
|
As example I will use the image manipulation program 'convert' from
|
||||||
|
ImageMagick. This command will convert foo.png to jpg with a size of
|
||||||
|
800 and JPEG-quality of 95.
|
||||||
|
|
||||||
|
convert -resize 800 -quality 95 foo.png foo_800_q95.jpg
|
||||||
|
|
||||||
With a for-loop it can be done on a list of files:
|
With a for-loop it can be done on a list of files:
|
||||||
|
|
||||||
time \
|
|
||||||
for file in *.png ; do
|
for file in *.png ; do
|
||||||
convert -size 800 -quality 95 $file ${file%.*}_800_q95.jpg
|
convert -resize 800 -quality 95 $file ${file%.*}_800_q95.jpg
|
||||||
done
|
done
|
||||||
|
|
||||||
Using GNU Parallel it looks like this:
|
This is the kind of loops GNU Parallel is good at replacing:
|
||||||
|
|
||||||
time parallel convert -size 800 -quality 95 {} {.}_800_q95.jpg ::: *.png
|
parallel convert -resize 800 -quality 95 {} {.}_800_q95.jpg ::: *.png
|
||||||
|
|
||||||
|
|
||||||
To get the images in 3 different JPEG-qualities you can use a nested for-loop:
|
To get the images in 3 different JPEG-qualities you can use a nested for-loop:
|
||||||
|
|
||||||
time \
|
|
||||||
for qual in 25 50 95 ; do
|
for qual in 25 50 95 ; do
|
||||||
for file in *.png ; do
|
for file in *.png ; do
|
||||||
convert -size 800 -quality $qual $file ${file%.*}_800_q${qual}.jpg
|
convert -resize 800 -quality $qual $file ${file%.*}_800_q${qual}.jpg
|
||||||
done
|
done
|
||||||
done
|
done
|
||||||
|
|
||||||
With GNU Parallel 'Pakistan' you can do this:
|
With GNU Parallel 'Pakistan' you can do this:
|
||||||
|
|
||||||
time parallel convert -size 800 -quality 95 {1} {1.}_800_q{2}.jpg ::: *.png ::: 25 50 95
|
parallel convert -resize 800 -quality {2} {1} {1.}_800_q{2}.jpg ::: *.png ::: 25 50 95
|
||||||
|
|
||||||
|
The new is that you can use the ::: multiple times. GNU Parallel will
|
||||||
|
then generate all the combinations and execute the command with these.
|
||||||
|
The {1} and {2} will be replaced by the relevant input source.
|
||||||
|
|
||||||
To get the 3 different JPEG-qualities in 2 different sizes you can
|
To get the 3 different JPEG-qualities in 2 different sizes you can
|
||||||
nest the for-loop even further:
|
nest the for-loop even further:
|
||||||
|
|
||||||
time \
|
|
||||||
for size in 800 30 ; do
|
for size in 800 30 ; do
|
||||||
for qual in 25 50 95 ; do
|
for qual in 25 50 95 ; do
|
||||||
for file in *.png ; do
|
for file in *.png ; do
|
||||||
convert -size $size -quality $qual $file ${file%.*}_${size}_q${qual}.jpg
|
convert -resize $size -quality $qual $file ${file%.*}_${size}_q${qual}.jpg
|
||||||
done
|
done
|
||||||
done
|
done
|
||||||
done
|
done
|
||||||
|
|
||||||
With GNU Parallel 'Pakistan' you can do this:
|
With GNU Parallel 'Pakistan' you can do this:
|
||||||
|
|
||||||
time parallel convert -size {3} -quality {2} {1} {1.}_{3}_q{2}.jpg ::: *.png ::: 25 50 95 ::: 800 30
|
parallel convert -resize {3} -quality {2} {1} {1.}_{3}_q{2}.jpg ::: *.png ::: 25 50 95 ::: 800 30
|
||||||
|
|
||||||
|
GNU Parallel will again generate all the combinations of the input
|
||||||
|
sources and run the jobs in parallel.
|
||||||
|
|
||||||
You can also provide the arguments in a file. This will do the same as above:
|
You can also provide the arguments in a file. This will do the same as above:
|
||||||
|
|
||||||
(echo 25; echo 50; echo 95) > qualities
|
(echo 25; echo 50; echo 95) > qualities
|
||||||
ls *.png > png-files
|
ls *.png > png-files
|
||||||
(echo 800; echo 30) > sizes
|
(echo 800; echo 30) > sizes
|
||||||
time parallel convert -size {3} -quality {2} {1} {1.}_{3}_q{2}.jpg :::: png-files qualities sizes
|
parallel convert -resize {3} -quality {2} {1} {1.}_{3}_q{2}.jpg :::: png-files qualities sizes
|
||||||
|
|
||||||
It is also possible to mix triple and quad colon. These will do the same as above:
|
But you can even mix triple and quad colon. These will do the same:
|
||||||
|
|
||||||
time parallel convert -size {3} -quality {2} {1} {1.}_{3}_q{2}.jpg :::: png-files ::: 25 50 95 ::: 800 30
|
parallel convert -resize {3} -quality {2} {1} {1.}_{3}_q{2}.jpg :::: png-files ::: 25 50 95 ::: 800 30
|
||||||
|
|
||||||
time parallel convert -size {3} -quality {2} {1} {1.}_{3}_q{2}.jpg :::: png-files ::: 25 50 95 :::: sizes
|
parallel convert -resize {3} -quality {2} {1} {1.}_{3}_q{2}.jpg :::: png-files ::: 25 50 95 :::: sizes
|
||||||
|
|
||||||
The special file '-' reads from standard input. This will do the same as above:
|
The special file '-' reads from standard input. This will do the same as above:
|
||||||
|
|
||||||
ls *.png | time parallel convert -size {3} -quality {2} {1} {1.}_{3}_q{2}.jpg :::: - ::: 25 50 95 :::: sizes
|
ls *.png | parallel convert -resize {3} -quality {2} {1} {1.}_{3}_q{2}.jpg :::: - ::: 25 50 95 :::: sizes
|
||||||
|
|
||||||
|
This is probably one of the ways you will use this feature as that can easily be combined with find:
|
||||||
|
|
||||||
|
find . -name '*.png' | \
|
||||||
|
parallel convert -resize {3} -quality {2} {1} {1.}_{3}_q{2}.jpg :::: - ::: 25 50 95 :::: sizes
|
||||||
|
|
||||||
|
# Thank you for watching
|
||||||
|
#
|
||||||
|
# If you like GNU Parallel:
|
||||||
|
# * Post this video on forums/blogs/Twitter/Facebook/Linkedin
|
||||||
|
# * Join the mailing list http://lists.gnu.org/mailman/listinfo/parallel
|
||||||
|
# * Request or write a review for your favourite blog or magazine
|
||||||
|
# * Request or build a package for your favourite distribution
|
||||||
|
# * Invite me for your next conference (Contact http://ole.tange.dk)
|
||||||
|
#
|
||||||
|
# If GNU Parallel saves you money:
|
||||||
|
# * (Have your company) donate to FSF https://my.fsf.org/donate/
|
||||||
|
#
|
||||||
|
# If you use GNU Parallel for a publication please cite:
|
||||||
|
# O. Tange (2011): GNU Parallel - The Command-Line Power Tool, ;login:
|
||||||
|
# The USENIX Magazine, February 2011:42-47.
|
||||||
|
#
|
||||||
|
# Find GNU Parallel at http://www.gnu.org/software/parallel/
|
||||||
|
|
||||||
|
|
||||||
Postkort:
|
Postkort:
|
||||||
|
|
|
@ -188,6 +188,9 @@ New in this release:
|
||||||
|
|
||||||
* Implemented {//} for the input line with the basename removed (dirname).
|
* Implemented {//} for the input line with the basename removed (dirname).
|
||||||
|
|
||||||
|
* GNU Parallel now has a logo.
|
||||||
|
http://www.gnu.org/software/parallel/logo.png
|
||||||
|
|
||||||
* Article about GNU SQL in USENIX Magazine ;login: (print)
|
* Article about GNU SQL in USENIX Magazine ;login: (print)
|
||||||
http://www.usenix.org/publications/login/2011-04/
|
http://www.usenix.org/publications/login/2011-04/
|
||||||
|
|
||||||
|
@ -209,6 +212,9 @@ New in this release:
|
||||||
* Short article about using GNU Parallel with lame:
|
* Short article about using GNU Parallel with lame:
|
||||||
http://loopkid.net/articles/2011/04/30/accelerate-lame-mp3-conversion
|
http://loopkid.net/articles/2011/04/30/accelerate-lame-mp3-conversion
|
||||||
|
|
||||||
|
* Using GNU Parallel to run tail -f in Japanese. Thanks to Clouder.
|
||||||
|
http://blog.clouder.jp/archives/001140.html
|
||||||
|
|
||||||
* BBC Research & Development uses GNU Parallel:
|
* BBC Research & Development uses GNU Parallel:
|
||||||
http://www.bbc.co.uk/blogs/researchanddevelopment/2010/11/prototyping-weeknotes-41-26112.shtml
|
http://www.bbc.co.uk/blogs/researchanddevelopment/2010/11/prototyping-weeknotes-41-26112.shtml
|
||||||
|
|
||||||
|
@ -267,6 +273,5 @@ will get that database's interactive shell.
|
||||||
When using GNU SQL for a publication please cite:
|
When using GNU SQL for a publication please cite:
|
||||||
|
|
||||||
O. Tange (2011): GNU SQL - A Command Line Tool for Accessing Different
|
O. Tange (2011): GNU SQL - A Command Line Tool for Accessing Different
|
||||||
Databases Using DBURLs, ;login: The USENIX Magazine, April
|
Databases Using DBURLs, ;login: The USENIX Magazine, April 2011:29-32.
|
||||||
2011.
|
|
||||||
>>>>>
|
>>>>>
|
||||||
|
|
|
@ -687,7 +687,8 @@ sub read_args_from_command_line {
|
||||||
# Group of arguments on the command line.
|
# Group of arguments on the command line.
|
||||||
# Put them into a file.
|
# Put them into a file.
|
||||||
# Create argfile
|
# Create argfile
|
||||||
my ($outfh,$name) = ::tempfile(SUFFIX => ".arg", DELETE => 1);
|
my ($outfh,$name) = ::tempfile(SUFFIX => ".arg");
|
||||||
|
unlink($name);
|
||||||
# Put args into argfile
|
# Put args into argfile
|
||||||
print $outfh map { $_,$/ } @group;
|
print $outfh map { $_,$/ } @group;
|
||||||
seek $outfh, 0, 0;
|
seek $outfh, 0, 0;
|
||||||
|
|
5
src/sql
5
src/sql
|
@ -997,7 +997,10 @@ sub version {
|
||||||
"This is free software: you are free to change and redistribute it.",
|
"This is free software: you are free to change and redistribute it.",
|
||||||
"GNU $Global::progname comes with no warranty.",
|
"GNU $Global::progname comes with no warranty.",
|
||||||
"",
|
"",
|
||||||
"Web site: http://www.gnu.org/software/${Global::progname}\n"
|
"Web site: http://www.gnu.org/software/${Global::progname}\n",
|
||||||
|
"When using GNU $Global::progname for a publication please cite:\n",
|
||||||
|
"O. Tange (2011): GNU SQL - A Command Line Tool for Accessing Different",
|
||||||
|
"Databases Using DBURLs, ;login: The USENIX Magazine, April 2011:29-32.\n"
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
Loading…
Reference in a new issue