mirror of
https://git.savannah.gnu.org/git/parallel.git
synced 2024-11-21 21:47:54 +00:00
parallel: /tmp/*.arg was not removed
This commit is contained in:
parent
8fb1966254
commit
10ac95d29a
29
README
29
README
|
@ -5,22 +5,23 @@ Please send problems and feedback to bug-parallel@gnu.org.
|
|||
|
||||
= Presentation of GNU Parallel =
|
||||
|
||||
GNU Parallel is a shell tool for executing jobs in parallel using one
|
||||
or more computers. A job is typically a single command or a small
|
||||
script that has to be run for each of the lines in the input. The
|
||||
typical input is a list of files, a list of hosts, a list of users, or
|
||||
a list of tables.
|
||||
GNU parallel is a shell tool for executing jobs in parallel using one
|
||||
or more computers. A job is can be a single command or a small script
|
||||
that has to be run for each of the lines in the input. The typical
|
||||
input is a list of files, a list of hosts, a list of users, a list of
|
||||
URLs, or a list of tables. A job can also be a command that reads from
|
||||
a pipe. GNU parallel can then split the input and pipe it into
|
||||
commands in parallel.
|
||||
|
||||
If you use xargs today you will find GNU Parallel very easy to use. If
|
||||
you write loops in shell, you will find GNU Parallel may be able to
|
||||
replace most of the loops and make them run faster by running jobs in
|
||||
parallel. If you use ppss or pexec you will find GNU Parallel will
|
||||
often make the command easier to read.
|
||||
If you use xargs and tee today you will find GNU parallel very easy to
|
||||
use as GNU parallel is written to have the same options as xargs. If
|
||||
you write loops in shell, you will find GNU parallel may be able to
|
||||
replace most of the loops and make them run faster by running several
|
||||
jobs in parallel.
|
||||
|
||||
GNU Parallel also makes sure output from the commands is the same
|
||||
output as you would get had you run the commands sequentially. This
|
||||
makes it possible to use output from GNU Parallel as input for other
|
||||
programs.
|
||||
GNU parallel makes sure output from the commands is the same output as
|
||||
you would get had you run the commands sequentially. This makes it
|
||||
possible to use output from GNU parallel as input for other programs.
|
||||
|
||||
|
||||
= Installation =
|
||||
|
|
|
@ -1,72 +1,106 @@
|
|||
Video
|
||||
Video 30. 36. 41. 48
|
||||
|
||||
|
||||
# GNU Parallel 20110522 - The Pakistan Release
|
||||
|
||||
I am Ole Tange. I am the author of GNU Parallel.
|
||||
|
||||
So far GNU Parallel has been focused on replacing a single
|
||||
for-loop. The Pakistan release introduces a way to replace nested
|
||||
loops.
|
||||
|
||||
As example I will use the image manipulation program 'convert' from
|
||||
ImageMagick. This will convert foo.png to jpg with a size of 800 and
|
||||
JPEG-quality of 95.
|
||||
# NESTED FOR LOOPS
|
||||
|
||||
convert -size 800 -quality 95 foo.png foo_800_q95.jpg
|
||||
As example I will use the image manipulation program 'convert' from
|
||||
ImageMagick. This command will convert foo.png to jpg with a size of
|
||||
800 and JPEG-quality of 95.
|
||||
|
||||
convert -resize 800 -quality 95 foo.png foo_800_q95.jpg
|
||||
|
||||
With a for-loop it can be done on a list of files:
|
||||
|
||||
time \
|
||||
for file in *.png ; do
|
||||
convert -size 800 -quality 95 $file ${file%.*}_800_q95.jpg
|
||||
convert -resize 800 -quality 95 $file ${file%.*}_800_q95.jpg
|
||||
done
|
||||
|
||||
Using GNU Parallel it looks like this:
|
||||
This is the kind of loops GNU Parallel is good at replacing:
|
||||
|
||||
time parallel convert -size 800 -quality 95 {} {.}_800_q95.jpg ::: *.png
|
||||
parallel convert -resize 800 -quality 95 {} {.}_800_q95.jpg ::: *.png
|
||||
|
||||
|
||||
To get the images in 3 different JPEG-qualities you can use a nested for-loop:
|
||||
|
||||
time \
|
||||
for qual in 25 50 95 ; do
|
||||
for file in *.png ; do
|
||||
convert -size 800 -quality $qual $file ${file%.*}_800_q${qual}.jpg
|
||||
convert -resize 800 -quality $qual $file ${file%.*}_800_q${qual}.jpg
|
||||
done
|
||||
done
|
||||
|
||||
With GNU Parallel 'Pakistan' you can do this:
|
||||
|
||||
time parallel convert -size 800 -quality 95 {1} {1.}_800_q{2}.jpg ::: *.png ::: 25 50 95
|
||||
parallel convert -resize 800 -quality {2} {1} {1.}_800_q{2}.jpg ::: *.png ::: 25 50 95
|
||||
|
||||
The new is that you can use the ::: multiple times. GNU Parallel will
|
||||
then generate all the combinations and execute the command with these.
|
||||
The {1} and {2} will be replaced by the relevant input source.
|
||||
|
||||
To get the 3 different JPEG-qualities in 2 different sizes you can
|
||||
nest the for-loop even further:
|
||||
|
||||
time \
|
||||
for size in 800 30 ; do
|
||||
for qual in 25 50 95 ; do
|
||||
for file in *.png ; do
|
||||
convert -size $size -quality $qual $file ${file%.*}_${size}_q${qual}.jpg
|
||||
convert -resize $size -quality $qual $file ${file%.*}_${size}_q${qual}.jpg
|
||||
done
|
||||
done
|
||||
done
|
||||
|
||||
With GNU Parallel 'Pakistan' you can do this:
|
||||
|
||||
time parallel convert -size {3} -quality {2} {1} {1.}_{3}_q{2}.jpg ::: *.png ::: 25 50 95 ::: 800 30
|
||||
parallel convert -resize {3} -quality {2} {1} {1.}_{3}_q{2}.jpg ::: *.png ::: 25 50 95 ::: 800 30
|
||||
|
||||
GNU Parallel will again generate all the combinations of the input
|
||||
sources and run the jobs in parallel.
|
||||
|
||||
You can also provide the arguments in a file. This will do the same as above:
|
||||
|
||||
(echo 25; echo 50; echo 95) > qualities
|
||||
ls *.png > png-files
|
||||
(echo 800; echo 30) > sizes
|
||||
time parallel convert -size {3} -quality {2} {1} {1.}_{3}_q{2}.jpg :::: png-files qualities sizes
|
||||
parallel convert -resize {3} -quality {2} {1} {1.}_{3}_q{2}.jpg :::: png-files qualities sizes
|
||||
|
||||
It is also possible to mix triple and quad colon. These will do the same as above:
|
||||
But you can even mix triple and quad colon. These will do the same:
|
||||
|
||||
time parallel convert -size {3} -quality {2} {1} {1.}_{3}_q{2}.jpg :::: png-files ::: 25 50 95 ::: 800 30
|
||||
parallel convert -resize {3} -quality {2} {1} {1.}_{3}_q{2}.jpg :::: png-files ::: 25 50 95 ::: 800 30
|
||||
|
||||
time parallel convert -size {3} -quality {2} {1} {1.}_{3}_q{2}.jpg :::: png-files ::: 25 50 95 :::: sizes
|
||||
parallel convert -resize {3} -quality {2} {1} {1.}_{3}_q{2}.jpg :::: png-files ::: 25 50 95 :::: sizes
|
||||
|
||||
The special file '-' reads from standard input. This will do the same as above:
|
||||
|
||||
ls *.png | time parallel convert -size {3} -quality {2} {1} {1.}_{3}_q{2}.jpg :::: - ::: 25 50 95 :::: sizes
|
||||
ls *.png | parallel convert -resize {3} -quality {2} {1} {1.}_{3}_q{2}.jpg :::: - ::: 25 50 95 :::: sizes
|
||||
|
||||
This is probably one of the ways you will use this feature as that can easily be combined with find:
|
||||
|
||||
find . -name '*.png' | \
|
||||
parallel convert -resize {3} -quality {2} {1} {1.}_{3}_q{2}.jpg :::: - ::: 25 50 95 :::: sizes
|
||||
|
||||
# Thank you for watching
|
||||
#
|
||||
# If you like GNU Parallel:
|
||||
# * Post this video on forums/blogs/Twitter/Facebook/Linkedin
|
||||
# * Join the mailing list http://lists.gnu.org/mailman/listinfo/parallel
|
||||
# * Request or write a review for your favourite blog or magazine
|
||||
# * Request or build a package for your favourite distribution
|
||||
# * Invite me for your next conference (Contact http://ole.tange.dk)
|
||||
#
|
||||
# If GNU Parallel saves you money:
|
||||
# * (Have your company) donate to FSF https://my.fsf.org/donate/
|
||||
#
|
||||
# If you use GNU Parallel for a publication please cite:
|
||||
# O. Tange (2011): GNU Parallel - The Command-Line Power Tool, ;login:
|
||||
# The USENIX Magazine, February 2011:42-47.
|
||||
#
|
||||
# Find GNU Parallel at http://www.gnu.org/software/parallel/
|
||||
|
||||
|
||||
Postkort:
|
||||
|
|
|
@ -188,6 +188,9 @@ New in this release:
|
|||
|
||||
* Implemented {//} for the input line with the basename removed (dirname).
|
||||
|
||||
* GNU Parallel now has a logo.
|
||||
http://www.gnu.org/software/parallel/logo.png
|
||||
|
||||
* Article about GNU SQL in USENIX Magazine ;login: (print)
|
||||
http://www.usenix.org/publications/login/2011-04/
|
||||
|
||||
|
@ -209,6 +212,9 @@ New in this release:
|
|||
* Short article about using GNU Parallel with lame:
|
||||
http://loopkid.net/articles/2011/04/30/accelerate-lame-mp3-conversion
|
||||
|
||||
* Using GNU Parallel to run tail -f in Japanese. Thanks to Clouder.
|
||||
http://blog.clouder.jp/archives/001140.html
|
||||
|
||||
* BBC Research & Development uses GNU Parallel:
|
||||
http://www.bbc.co.uk/blogs/researchanddevelopment/2010/11/prototyping-weeknotes-41-26112.shtml
|
||||
|
||||
|
@ -267,6 +273,5 @@ will get that database's interactive shell.
|
|||
When using GNU SQL for a publication please cite:
|
||||
|
||||
O. Tange (2011): GNU SQL - A Command Line Tool for Accessing Different
|
||||
Databases Using DBURLs, ;login: The USENIX Magazine, April
|
||||
2011.
|
||||
Databases Using DBURLs, ;login: The USENIX Magazine, April 2011:29-32.
|
||||
>>>>>
|
||||
|
|
|
@ -687,7 +687,8 @@ sub read_args_from_command_line {
|
|||
# Group of arguments on the command line.
|
||||
# Put them into a file.
|
||||
# Create argfile
|
||||
my ($outfh,$name) = ::tempfile(SUFFIX => ".arg", DELETE => 1);
|
||||
my ($outfh,$name) = ::tempfile(SUFFIX => ".arg");
|
||||
unlink($name);
|
||||
# Put args into argfile
|
||||
print $outfh map { $_,$/ } @group;
|
||||
seek $outfh, 0, 0;
|
||||
|
|
5
src/sql
5
src/sql
|
@ -997,7 +997,10 @@ sub version {
|
|||
"This is free software: you are free to change and redistribute it.",
|
||||
"GNU $Global::progname comes with no warranty.",
|
||||
"",
|
||||
"Web site: http://www.gnu.org/software/${Global::progname}\n"
|
||||
"Web site: http://www.gnu.org/software/${Global::progname}\n",
|
||||
"When using GNU $Global::progname for a publication please cite:\n",
|
||||
"O. Tange (2011): GNU SQL - A Command Line Tool for Accessing Different",
|
||||
"Databases Using DBURLs, ;login: The USENIX Magazine, April 2011:29-32.\n"
|
||||
);
|
||||
}
|
||||
|
||||
|
|
Loading…
Reference in a new issue