mirror of
https://git.savannah.gnu.org/git/parallel.git
synced 2024-11-25 23:47:53 +00:00
94 lines
3 KiB
Plaintext
94 lines
3 KiB
Plaintext
=head1 What is GNU Parallel used for
|
|
|
|
Searching for transit planets using data from the Kepler space telescope.
|
|
|
|
Searching 1700 genomes for 1000-10000 protein sequences using Amazon
|
|
EC2 compute cloud.
|
|
|
|
Processing Earth Observation data from satellites to grep for pieces
|
|
of information.
|
|
|
|
Running tons of simulations of granular materials.
|
|
|
|
Converting formats of movie frames in the film industry.
|
|
|
|
Computational fluid dynamics. Numerical simulation of the compressible
|
|
Navier-Stokes equations.
|
|
|
|
Analysing data and running simulations for searching for the Higgs
|
|
boson at the Tevatron.
|
|
|
|
|
|
=head1 search terms
|
|
|
|
run commands in parallel
|
|
|
|
Parallel shell loops
|
|
|
|
multi threading in bash xargs
|
|
|
|
# TAGS: parallel | parallel processing | multicore | multiprocessor | Clustering/Distributed Networks
|
|
# job control | multiple jobs | parallelization | text processing | cluster | filters
|
|
# Clustering Tools | Command Line Tools | Utilities | System Administration
|
|
# Bash parallel
|
|
|
|
GNU parallel execution shell bash script simultaneous concurrent linux
|
|
scripting run xargs ppss code.google.com/p/ppss/
|
|
|
|
@vvuksan @ychaker @ncb000gt
|
|
xargs can lead to nasty surprises caused by the separator problem
|
|
http://nd.gd/0t GNU Parallel http://nd.gd/0s may be better.
|
|
|
|
Comments:
|
|
|
|
http://pi.dk/0 https://www.gnu.org/software/parallel/
|
|
http://pi.dk/1 https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1
|
|
http://pi.dk/2 https://savannah.gnu.org/news/?group=parallel
|
|
|
|
http://pi.dk/5 https://en.wikipedia.org/wiki/Xargs#The_separator_problem
|
|
http://pi.dk/6 https://www.gnu.org/software/parallel/man.html#differences_between_xargs_and_gnu_parallel
|
|
http://pi.dk/7 https://www.gnu.org/software/parallel/man.html#example__distributing_work_to_local_and_remote_computers
|
|
|
|
If you like xargs you may love GNU Parallel: http://pi.dk/1
|
|
|
|
With GNU Parallel (http://pi.dk/0) you can do:
|
|
ls | grep jpeg | parallel mv {} {.}.jpg
|
|
|
|
Watch the intro video for GNU Parallel: http://pi.dk/1
|
|
|
|
If your input file names are generated by users, you need to deal with
|
|
surprising file names containing space, ', or " in the filename.
|
|
|
|
xargs can give nasty surprises due to the separator problem
|
|
http://pi.dk/5
|
|
|
|
@jaylyerly @stevenf xargs will bite you if file names contain
|
|
space http://pi.dk/5. Use GNU Parallel instead: http://pi.dk/0
|
|
|
|
Please repay by spreading the word about GNU Parallel to your
|
|
contacts/blog/facebook/linkedin/mailing lists/user group
|
|
|
|
Your use of xargs can lead to nasty surprises because of the separator
|
|
problem http://en.wikipedia.org/wiki/Xargs#The_separator_problem
|
|
|
|
GNU Parallel http://www.gnu.org/software/parallel/ does not have that
|
|
problem.
|
|
|
|
If you have GNU Parallel http://www.gnu.org/software/parallel/ installed you can do this:
|
|
|
|
|
|
You can install GNU Parallel simply by:
|
|
|
|
wget http://git.savannah.gnu.org/cgit/parallel.git/plain/src/parallel
|
|
chmod 755 parallel
|
|
cp parallel sem
|
|
|
|
Watch the intro videos for GNU Parallel to learn more:
|
|
https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1
|
|
|
|
|
|
GNU Parallel also makes it possible to run small scripts. Try this:
|
|
|
|
ls *.zip | parallel 'mkdir {.}; cd {.}; unzip ../{}'
|
|
|