mirror of
https://git.savannah.gnu.org/git/parallel.git
synced 2024-11-22 05:57:54 +00:00
404 lines
11 KiB
Plaintext
404 lines
11 KiB
Plaintext
# SPDX-FileCopyrightText: 2021 Ole Tange, http://ole.tange.dk and Free Software and Foundation, Inc.
|
||
# SPDX-License-Identifier: GFDL-1.3-or-later
|
||
# SPDX-License-Identifier: CC-BY-SA-4.0
|
||
|
||
Quote of the month:
|
||
|
||
Parallel is amazing!
|
||
-- fatboy93@reddit
|
||
|
||
parallel is my new kink
|
||
-- python_noob_001@reddit
|
||
|
||
GNU Parallel makes my life so much easier.
|
||
I'm glad I don't have to implement multi-threaded Python scripts on the regular.
|
||
-- Fredrick Brennan @fr_brennan@twitter
|
||
|
||
|
||
@GnuParallel the best thing ever and it's not up for debate #EOchat
|
||
-- Nathan Thomas @DrNASApants@twitter
|
||
|
||
Using [GNU Parallel is] super easy if you use xargs, and it is magic for making things multi-process. Then adding in the ssh magic it can do it is an incredible tool that is completely underutilized.
|
||
-- Ancients @Ancients@twitter
|
||
|
||
GNU Parallel is one of the most helpful tools I've been using recently, and it's just something like: parallel -j4 'gzip {}' ::: folder/*.csv
|
||
-- Milton Pividori @miltondp@twitter
|
||
|
||
We use gnu parallel now - and happier for it.
|
||
-- Ben Davies @benjamindavies@twitter
|
||
|
||
This is a fantastic tool, and I wish I had upgraded from xargs years ago!
|
||
-- Stuart Anderson
|
||
|
||
GNU Parallel and ripgrep would be your friend here. Ripgrep is fast, really fast.
|
||
-- CENAPT @cenaptech@twitter
|
||
|
||
GNU Parallel is great
|
||
-- Newton's Flaming Laser Sword @swordgoesfwoosh@twitter
|
||
|
||
Try GNU parallel it's awesome, and exactly the thing you are looking for. It allows you to set number of processes running among many things. I use it a lot, and can't recommend it enough.
|
||
-- mapettheone@reddit
|
||
|
||
It's really quite amazing how powerful and flexible it is
|
||
-- schwanengesang @tensegrist@twitter
|
||
|
||
Every time I install @ubuntu, one of the first tools I install is
|
||
@gnuparallel. I love it.
|
||
-- Necati Demir @ndemir@twitter
|
||
|
||
Today I'm grateful for GNU parallel, especially with the --colsep and
|
||
--jobs parameters #GiveThanks
|
||
-- Erin Young @ErinYoun
|
||
|
||
I also prefer gnu parallel. Mainly because it makes embarrassingly
|
||
parallel tasks embarrassingly easy to run on the command line.
|
||
-- Vincent D. Warmerdam @fishnets88@twitter
|
||
|
||
It's not a data migration party until GNU Parallel is involved...
|
||
involved
|
||
involved
|
||
-- rrees @rrees@twitter
|
||
|
||
My favorite man page is that of GNU parallel.
|
||
-- Jeroen Janssens @jeroenhjanssens@twitter
|
||
|
||
I have gotten a *ton* of mileage out of jq, awk, and GNU parallel,
|
||
even at multi-GB sizes.
|
||
-- Eric Wolak @ericthewolak@twitter
|
||
|
||
GNU parallel, which works a little bit like xargs, but has a much more friendly way of handling files with spaces and automatically parallelises calls. This tool has saved me a great deal of coding because it makes it so easy to write a program which does just one part of a task and then run it in parallel with load balancing and a nice progress bar. I cannot recommend this tool enough.
|
||
|
||
https://negfeedback.blogspot.com/2020/05/indispensable-command-line-tools.html
|
||
|
||
GNU parallel all the way!
|
||
-- David Manouchehri @DaveManouchehri@twitter
|
||
|
||
I found GNU Parallel and it's awesome.
|
||
-- Teddy Choi @TeddyJChoi@twitter
|
||
|
||
Well anyway, It was blazingly fast and astonished by performance. Guess I'll never use xargs.
|
||
-- (Not) Akaming @_Akamig@twitter
|
||
|
||
Yeah, GNU parallel is a beast when used accordingly.
|
||
-- @lsde@twitter
|
||
|
||
GNU Parallel is awesome
|
||
-- Gaurav Jain @gauravjain49@twitter
|
||
|
||
My new favourite command line tool is GNU Parallel. After knowing it for a while I just recently came back to it for a refactor of a script and now I've changed like 4 different huge scripts into tiny parallel scripts that do their job literally 3 to 10 times faster and just [mind blown]
|
||
-- Izzy @angry_izzy@twitter
|
||
|
||
I love so much @GnuParallel to parallelize my SQL requests on @PostgreSQL
|
||
-- @rmaziere_85 Romain
|
||
|
||
GNU parallel really changed how I do a lot of data processing stuff
|
||
-- Brendan Dolan-Gavitt @moyix@twitter
|
||
|
||
Parallel is a life send, what an awesome piece of software.
|
||
-- Blaze9@reddit.com
|
||
|
||
It's really powerful for smallish data processing scripts
|
||
-- João Veiga @jcsvveiga@twitter
|
||
|
||
It's the MapReduce of our generation!
|
||
-- Mathias Meyer @roidrage@twitter
|
||
|
||
GNU parallel is a seriously powerful flexible and tool.
|
||
-- Greg Hurrell @wincent@twitter
|
||
|
||
[socat is] top3 on my "crazy and powerful" list (the other two being
|
||
gnu parallel and emacs).
|
||
-- Peter Kjellström @nsccap@twitter
|
||
|
||
GNU/Parallel umm... tempting.
|
||
-- k-leb k @dcatdemon@twitter
|
||
|
||
Ain't that a dirty hack ;-)
|
||
-- ./hans
|
||
|
||
Not sure if it counts as an "alt" tool but GNU parallel really took my shell scripting game to the next level.
|
||
-- @alinajaf@twitter
|
||
|
||
=== Used ===
|
||
|
||
If you work with lots of files at once
|
||
Take a good look at GNU parallel
|
||
Change your life for the better
|
||
-- French @notareverser@twitter
|
||
|
||
GNU Parallel is your friend.
|
||
Can shorten that time by X cores.
|
||
-- iRODS @irods@twitter
|
||
|
||
GNU Parallel is my new favorite thing
|
||
-- Will Tejeda @thewilltejeda
|
||
|
||
GNU Parallel is a priceless timesaver
|
||
-- June "Wakalix" @TWakalix@twitter
|
||
|
||
I think many people would be surprised to learn that GNU parallel is
|
||
"just" a single Perl script.
|
||
-- Peter Menzel @ptr_menzel@twitter
|
||
|
||
GNU Parallel is my single favourite tool for batch processing data
|
||
from the command line.
|
||
-- Jeff Wintersinger @jwintersinger
|
||
|
||
GNU parallel should be taught in class, it is one of the best tools
|
||
to run grids of experiments
|
||
-- no love deep learning @tetraduzione@twitter
|
||
|
||
I get a weird sense of satisfaction every single time I see the
|
||
lovely logo of #GNU Parallel (plus, what an underrated piece of
|
||
great software!)
|
||
-- Emre Sevinç @EmreSevinc@twitter
|
||
|
||
My favorite man page is that of GNU parallel.
|
||
-- Jeroen Janssens @jeroenhjanssens@twitter
|
||
|
||
Gnu parallel is also awesome, fwiw.
|
||
-- Rogan Dawes @RoganDawes@twitter
|
||
|
||
With multicore systems everywhere GNU Parallel is a must have tool.
|
||
-- Neil H. Watson @neil_h_watson@twitter
|
||
|
||
Who needs spark when GNU Parallel exists
|
||
-- MatthijsB @MatthijsBrs@twitter
|
||
|
||
GNU Parallel: dead simple process-level parallelization of ad hoc
|
||
tasks. Write for a chunk, let gnu manage the splitting, permutations
|
||
and pool concurrency.
|
||
-- Nick Ursa @nickursa@twitter
|
||
|
||
|
||
I wish more command line software had example pages as robust as GNU Parallel
|
||
-- Lucidbeaming @lucidbeaming
|
||
|
||
GNU parallel is straight up incredible.
|
||
-- Ben Johnson @biobenkj@twtter
|
||
|
||
GNU parallel has helped me kill a Hadoop cluster before.
|
||
-- Travis Campbell @hcoyote@twitter
|
||
|
||
Parallel is the BEST command.
|
||
-- Nick @NickInfoSec@twitter
|
||
|
||
[L]earning about parallel was amazing for me, it gives us many beautiful solutions.
|
||
-- SergioAraujo@stackoverflow
|
||
|
||
I've said it before: The command line program GNU Parallel is a godsend.
|
||
-- Jo Chr. Oterhals @oterhals
|
||
|
||
IMHO, SQLite and GNU Parallel are among the world's great software.
|
||
-- singe@reddit
|
||
|
||
It is, beyond absolutely any doubt whatsoever, the single most
|
||
important tool I use in making me a productive bioinformatician.
|
||
-- A-N-Other@reddit.com
|
||
|
||
It is SUPER easy to speed up jobs from the command line w/ GNU parallel.
|
||
-- B3n @B3njaminHimes@twitter
|
||
|
||
I want to make a shout-out for @GnuParallel, it's a work of beauty and power
|
||
-- Cristian Consonni @CristianCantoro
|
||
|
||
Amazingly useful script!
|
||
-- unxusr@reddit.com
|
||
|
||
Y'all need some GNU parallel in your lives
|
||
-- ChaKu @ChaiLovesChai@twitter
|
||
|
||
|
||
There are so many things to love about GNU parallel. You could honestly teach a whole parallel computing course with it and never have to leave it for a real language.
|
||
-- Aubrey Bailey @DNAvinci@twitter
|
||
|
||
|
||
With GNU Parallel you sure can!
|
||
I like getting things done
|
||
|
||
--Kyle Lady @kylelady@twitter
|
||
|
||
Ok! GNU Parallel is one of the best things out there. Almost as good as vanilla ice cream.
|
||
-- @coffe@mastodon.art
|
||
|
||
HOLY STUFF I LOVE GNU PARALLEL
|
||
-- Nick (but spookier) @NickInfoSec@twitter
|
||
|
||
|
||
What this country needs is a president with a strong understanding of GNU Parallel, Awk, Sed, Grep, and Diff
|
||
-- Gregg Housh @GreggHoush
|
||
|
||
I know I'm late to the party but GNU Parallel is truly amazing!
|
||
-- Sam Diaz-Munoz @sociovirology
|
||
|
||
GNU parallel is a thing of magic.
|
||
-- Josh Meyer @joshmeyerphd@twitter
|
||
|
||
I've been using GNU Parallel very much and effectively lately.
|
||
Such an easy way to get huge speed-ups with my simple bash/Perl/Python
|
||
programs -- parallelize them!
|
||
-- Ken Youens-Clark @kycl4rk@twitter
|
||
|
||
Alias and vars
|
||
export them more easily
|
||
With env_parallel
|
||
-- Ole Tange
|
||
|
||
You know what?
|
||
GNU Parallel is cool.
|
||
Concurrency, but in the Unix-philosophy style,
|
||
without the Enterprise wankeriness.
|
||
-- NickM bokkiedog@twitter
|
||
|
||
An ode to GNU parallel
|
||
An ode to GNU parallel
|
||
An ode to GNU parallel
|
||
An ode to GNU parallel
|
||
An ode to GNU parallel
|
||
An ode to GNU parallel
|
||
-- Adam Stuckert PoisonEcology@twitter
|
||
|
||
--limit can
|
||
limit jobs dynamic'ly
|
||
given a command
|
||
--ole-tange
|
||
|
||
--line-buffer
|
||
touches files on disk no more
|
||
faster than before
|
||
-- Ole Tange
|
||
|
||
Always test your script
|
||
using a --dry-run
|
||
or --dr
|
||
-- Ole Tange
|
||
|
||
--rpl
|
||
used to be a static string
|
||
Now it can take args
|
||
--ole-tange
|
||
|
||
--tee will pipe
|
||
all data to all the jobs
|
||
used with --pipe
|
||
--ole-tange
|
||
|
||
Speed is good for you
|
||
Multi speed is better still
|
||
Use GNU Parallel
|
||
-- Ole Tange
|
||
|
||
Want to monitor
|
||
many job outputs at once:
|
||
--tmuxpane
|
||
--ole-tange
|
||
|
||
Pipes are fast and good.
|
||
Use them in your programs, too.
|
||
Use GNU Parallel
|
||
-- Ole Tange
|
||
|
||
Programs using net
|
||
only spare capacity
|
||
niceload dash dash net
|
||
-- Ole Tange
|
||
|
||
xapply too strict?
|
||
:::+
|
||
is just made for you
|
||
-- Ole Tange
|
||
|
||
Save environment
|
||
Functions, vars and other stuff
|
||
Try env_parallel
|
||
-- Ole Tange
|
||
|
||
Many jobs to sort.
|
||
SQL would help a lot.
|
||
Dash dash sql
|
||
-- Ole Tange
|
||
|
||
Dash dash pipe too slow?
|
||
Dash dash pipepart is your friend.
|
||
Use GNU Parallel
|
||
-- Ole Tange
|
||
|
||
|||||
|
||
|||||||
|
||
Use GNU Parallel
|
||
-- Malcolm Cook
|
||
|
||
UNIX pipe too tight?
|
||
You should then learn dash dash pipe.
|
||
Use GNU Parallel
|
||
-- Ole Tange
|
||
|
||
In parallel land
|
||
everything is quick and fast
|
||
Use GNU Parallel.
|
||
-- Ole Tange
|
||
|
||
Jobs in parallel
|
||
faster run than single jobs
|
||
Use GNU Parallel.
|
||
-- Ole Tange
|
||
|
||
Multi core I see.
|
||
Doing nothing. I say no!
|
||
Use GNU Parallel.
|
||
-- Ole Tange
|
||
|
||
Programs very slow.
|
||
Multiple can run at once.
|
||
Use GNU Parallel.
|
||
-- Ole Tange
|
||
|
||
Many jobs to run.
|
||
Idle cores is not your thing.
|
||
Use GNU Parallel.
|
||
-- Ole Tange
|
||
|
||
SSH set up?
|
||
Instant cluster needed now?
|
||
Use GNU Parallel.
|
||
-- Ole Tange
|
||
|
||
xargs' space and quote
|
||
headache causing behaviour.
|
||
Use GNU Parallel
|
||
-- Ole Tange
|
||
|
||
If not enough time:
|
||
Run jobs using Parallel.
|
||
Keeps CPUs warm.
|
||
-- Ole Tange
|
||
|
||
Hadoop bit too much?
|
||
Want a simpler syntax now?
|
||
Use GNU Parallel.
|
||
-- Ole Tange
|
||
|
||
Shellshock pain hits us.
|
||
Dash dash env is affected.
|
||
Upgrade Parallel.
|
||
-- Ole Tange
|
||
|
||
goodbye to xargs
|
||
quoting is still a black art
|
||
welcome parallel
|
||
-- Malcolm Cook
|
||
|
||
bash for loops vanquished
|
||
jobs fan out over network
|
||
collated results!
|
||
-- Malcolm Cook
|
||
|
||
code fork headache blues?
|
||
option P is your new friend
|
||
`man parallel` now!
|
||
-- Malcolm Cook
|
||
|
||
Are you tired of
|
||
inflexible replacements?
|
||
Use Perl expressions.
|
||
-- Ole Tange |