mirror of
https://git.savannah.gnu.org/git/parallel.git
synced 2024-11-22 14:07:55 +00:00
82 lines
3 KiB
Plaintext
82 lines
3 KiB
Plaintext
=head1 search terms
|
||
|
||
run commands in parallel
|
||
|
||
Parallel shell loops
|
||
|
||
multi threading in bash xargs
|
||
|
||
# TAGS: parallel | parallel processing | multicore | multiprocessor | Clustering/Distributed Networks
|
||
# job control | multiple jobs | parallelization | text processing | cluster | filters
|
||
# Clustering Tools | Command Line Tools | Utilities | System Administration
|
||
# Bash parallel
|
||
|
||
GNU parallel execution shell bash script simultaneous concurrent linux
|
||
scripting run xargs ppss code.google.com/p/ppss/
|
||
|
||
@vvuksan @ychaker @ncb000gt
|
||
xargs can lead to nasty surprises caused by the separator problem
|
||
http://nd.gd/0t GNU Parallel http://nd.gd/0s may be better.
|
||
|
||
Comments:
|
||
|
||
http://nd.gd/0u http://www.gnu.org/software/parallel/
|
||
http://nd.gd/0s http://www.youtube.com/watch?v=OpaiGYxkSuQ
|
||
http://nd.gd/0t http://en.wikipedia.org/wiki/Xargs#The_separator_problem
|
||
http://nd.gd/3k http://www.gnu.org/software/parallel/man.html#differences_between_xargs_and_gnu_parallel
|
||
http://nd.gd/po http://www.gnu.org/software/parallel/man.html#example__distributing_work_to_local_and_remote_computers
|
||
|
||
If you like xargs you may love GNU Parallel: http://nd.gd/0s
|
||
|
||
|
||
With GNU Parallel (http://nd.gd/0s) you can do:
|
||
ls | grep jpeg | parallel mv {} {.}.jpg
|
||
|
||
Watch the intro video for GNU Parallel: http://nd.gd/0s
|
||
|
||
If your input file names are generated by users, you need to deal with
|
||
surprising file names containing space, ', or " in the filename.
|
||
|
||
xargs can give nasty surprises due to the separator problem
|
||
http://en.wikipedia.org/wiki/Xargs#The_separator_problem
|
||
http://nd.gd/0t
|
||
|
||
GNU Parallel http://nd.gd/0s may be better.
|
||
|
||
@jaylyerly @stevenf xargs will bite you if file names contain
|
||
space http://nd.gd/0t. Use GNU Parallel instead: http://nd.gd/0s
|
||
|
||
Please repay by spreading the word about GNU Parallel to your
|
||
contacts/blog/facebook/linkedin/mailing lists/user group
|
||
|
||
Your use of xargs can lead to nasty surprises because of the separator
|
||
problem http://en.wikipedia.org/wiki/Xargs#The_separator_problem
|
||
|
||
GNU Parallel http://www.gnu.org/software/parallel/ does not have that
|
||
problem.
|
||
|
||
Watch the intro video for GNU Parallel:
|
||
http://www.youtube.com/watch?v=OpaiGYxkSuQ
|
||
|
||
If you have GNU Parallel http://www.gnu.org/software/parallel/ installed you can do this:
|
||
|
||
Watch the intro video for GNU Parallel to learn more:
|
||
http://www.youtube.com/watch?v=OpaiGYxkSuQ
|
||
|
||
|
||
GNU Parallel also makes it possible to run small scripts. Try this:
|
||
|
||
ls *.zip | parallel ‘mkdir {.}; cd {.}; unzip ../{}’
|
||
|
||
= Other people says =
|
||
|
||
fawxtin: Using GNU Parallel to run things simultaneously, neat! ;)
|
||
cowsandmilk: if you haven't quit xargs for gnu parallel yet, you're really missing out
|
||
afader: GNU Parallel is cool
|
||
ecyrd: Okay, why hasn't anyone told me of GNU parallel before?
|
||
cashion: very cool! Did not know about GNU parallel.
|
||
toorghezi: Writing shell scripts? You should know about GNU Parallel!
|
||
Racecar564 : GNU Parallel rocks!
|
||
arclight : Very nice! I especially like the {} -> {.}.bz2 syntax.
|
||
nwp8861 : I surprised that it's so comfortable!
|