parallel/doc/promo

=head1 What is GNU Parallel used for

Searching for transit planets using data from the Kepler space telescope.

Searching 1700 genomes for 1000-10000 protein sequences using Amazon
EC2 compute cloud.

Processing Earth Observation data from satellites to grep for pieces
of information.

Running tons of simulations of granular materials.

Converting formats of movie frames in the film industry.

Computational fluid dynamics. Numerical simulation of the compressible
Navier-Stokes equations.

Analysing data and running simulations for searching for the Higgs
boson at the Tevatron.


=head1 search terms

run commands in parallel

Parallel shell loops

multi threading in bash xargs

# TAGS: parallel | parallel processing | multicore | multiprocessor | Clustering/Distributed Networks
# job control | multiple jobs | parallelization | text processing | cluster | filters
# Clustering Tools | Command Line Tools | Utilities | System Administration
# Bash parallel

GNU parallel execution shell bash script simultaneous concurrent linux
scripting run xargs ppss code.google.com/p/ppss/

@vvuksan @ychaker @ncb000gt 
xargs can lead to nasty surprises caused by the separator problem
http://nd.gd/0t GNU Parallel http://nd.gd/0s may be better.

Comments:

http://pi.dk/0 https://www.gnu.org/software/parallel/
http://pi.dk/1 https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1
http://pi.dk/2 https://savannah.gnu.org/news/?group=parallel

http://pi.dk/5 https://en.wikipedia.org/wiki/Xargs#The_separator_problem
http://pi.dk/6 https://www.gnu.org/software/parallel/man.html#differences_between_xargs_and_gnu_parallel
http://pi.dk/7 https://www.gnu.org/software/parallel/man.html#example__distributing_work_to_local_and_remote_computers

If you like xargs you may love GNU Parallel: http://pi.dk/1

With GNU Parallel (http://pi.dk/0) you can do:
ls | grep jpeg | parallel mv {} {.}.jpg

Watch the intro video for GNU Parallel: http://pi.dk/1

If your input file names are generated by users, you need to deal with
surprising file names containing space, ', or " in the filename. 

xargs can give nasty surprises due to the separator problem
http://pi.dk/5

@jaylyerly @stevenf xargs will bite you if file names contain
space http://pi.dk/5. Use GNU Parallel instead: http://pi.dk/0

Please repay by spreading the word about GNU Parallel to your
contacts/blog/facebook/linkedin/mailing lists/user group

Your use of xargs can lead to nasty surprises because of the separator
problem http://en.wikipedia.org/wiki/Xargs#The_separator_problem

GNU Parallel http://www.gnu.org/software/parallel/ does not have that
problem.

If you have GNU Parallel http://www.gnu.org/software/parallel/ installed you can do this:


You can install GNU Parallel simply by:

    wget http://git.savannah.gnu.org/cgit/parallel.git/plain/src/parallel
    chmod 755 parallel
    cp parallel sem

Watch the intro videos for GNU Parallel to learn more:
https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1


GNU Parallel also makes it possible to run small scripts. Try this:

ls *.zip | parallel 'mkdir {.}; cd {.}; unzip ../{}'