mirror of
https://git.savannah.gnu.org/git/parallel.git
synced 2024-12-22 20:57:53 +00:00
517 lines
15 KiB
Plaintext
517 lines
15 KiB
Plaintext
# SPDX-FileCopyrightText: 2021-2022 Ole Tange, http://ole.tange.dk and Free Software and Foundation, Inc.
|
||
# SPDX-License-Identifier: GFDL-1.3-or-later
|
||
# SPDX-License-Identifier: CC-BY-SA-4.0
|
||
|
||
Quote of the month:
|
||
|
||
Gnu parallel is indeed slick. I always try to align my data to make it possible to loop over it with a nice for loop added in a call to run the jobs in parallel then becomes super easy. I love the {1..99} syntax in bash.
|
||
-- ragsofx
|
||
|
||
xargs諦めてGNU Parallel使ったら一瞬で問題が解決したので終わり
|
||
-- aria_moto @moto_aria@twitter
|
||
|
||
Such a life saver of a tool.
|
||
-- winrid@ycombinator
|
||
|
||
I used GNU Parallel to run a script locally that did what a big distributed system did, quicker and more reliably. It got to the point where people would just ask me to "run the thing" on my laptop instead of waiting for the cron.
|
||
-- winrid@ycombinator
|
||
|
||
parallel is so useful and i use it multiple times daily.
|
||
-- arrakeen@ycombinator
|
||
|
||
Parallel is Good Stuff (tm)
|
||
-- bloopernova@ycombinator
|
||
|
||
GNU Parallel is one of my favorite utilities of all time.
|
||
-- paskozdilar@ycombinator
|
||
|
||
Tool, was ich sehr ausgiebig nutze inzwischen und lieben gelernt habe: "GNU parallel"
|
||
-- nuit @nv1t@twitter
|
||
|
||
아 parallel 너무 좋지요.
|
||
-- 어엉부엉@d_ijk_stra
|
||
|
||
GNU parallel すごいな、シェルスクリプトを make -jX みたいにして並列
|
||
に走らせたいときに超便利
|
||
-- びーる @debugordie@twitter
|
||
|
||
GNU parallel いいよ、大量のデータファイルをスクリプトで処理する人とかにマジおすすめ
|
||
-- びーる @debugordie@twitter
|
||
|
||
Igual @GnuParallel te puede echar una mano. Te permite pensar y
|
||
programar en serie pero ejecutar en paralelo usando cores de una
|
||
máquina o máquinas remotas. Una vez que lo has usando te lamentas no
|
||
haberlo conocido antes :)
|
||
-- Juan Sierra Pons @juasiepo
|
||
|
||
If I could only keep 5 GNU utils, parallel would make it to the list
|
||
:)
|
||
-- 5heikki@reddit
|
||
|
||
Gnu Parallel: installed in every computer i have access to.
|
||
-- raffaele messuti @atomotic@twitter
|
||
|
||
gnu parallel is a good program
|
||
-- Pwn A. Day @pwnaday@twitter
|
||
|
||
Deus salve o gnu parallel
|
||
-- marcos @guv_Tuv@twitter
|
||
|
||
@a
|
||
4
|
||
اليوم علمت بأداة تساعد على تشغيل أوامر الصدفة بشكل متوازي حسب عدد الأنوية الموجودة مما يجعلها أداة أساسية في عدة تنفيذ المهام.
|
||
الأداة اسمها gnu parallel
|
||
-- @AbuEsra2014@twitter زايد السعيدي
|
||
|
||
GNU parallel is very easy to use and has many features for
|
||
specialized use cases. It’s a Perl script.
|
||
-- @harlekyn@twitter uʎʞǝlɹɐɥ
|
||
|
||
GNU parallelという神ツールを発見した
|
||
-- @schnell1232@twitter
|
||
|
||
How I love GNU parallel
|
||
-- @dathanvp@twitter Dathan Pattishall
|
||
|
||
GNU Parallelめっちゃ便利で偉い
|
||
-- аiгbus @airbus_P
|
||
|
||
I really liked GNU Parallel http://gnu.org/software/parallel/
|
||
one of the best tool to execute parallel jobs in the shell
|
||
-- Luca Molteni @volothamp@twitter
|
||
|
||
Have you heard of our lord and saviour GNU parallel?
|
||
-- kxyne @Kxyne@twitter
|
||
|
||
Installing GNU parallel is the second thing I do on any new Linux
|
||
machine, right after uninstalling pulseaudio.
|
||
-- Andy Cooper @integgroll@twitter
|
||
|
||
Parallel is amazing!
|
||
-- fatboy93@reddit
|
||
|
||
parallel is my new kink
|
||
-- python_noob_001@reddit
|
||
|
||
@GnuParallel the best thing ever and it's not up for debate #EOchat
|
||
-- Nathan Thomas @DrNASApants@twitter
|
||
|
||
Using [GNU Parallel is] super easy if you use xargs, and it is magic for making things multi-process. Then adding in the ssh magic it can do it is an incredible tool that is completely underutilized.
|
||
-- Ancients @Ancients@twitter
|
||
|
||
GNU Parallel is one of the most helpful tools I've been using recently, and it's just something like: parallel -j4 'gzip {}' ::: folder/*.csv
|
||
-- Milton Pividori @miltondp@twitter
|
||
|
||
This is a fantastic tool, and I wish I had upgraded from xargs years ago!
|
||
-- Stuart Anderson
|
||
|
||
GNU Parallel and ripgrep would be your friend here.
|
||
-- CENAPT @cenaptech@twitter
|
||
|
||
GNU Parallel is great
|
||
-- Newton's Flaming Laser Sword @swordgoesfwoosh@twitter
|
||
|
||
Try GNU parallel it's awesome, and exactly the thing you are looking for. It allows you to set number of processes running among many things. I use it a lot, and can't recommend it enough.
|
||
-- mapettheone@reddit
|
||
|
||
It's really quite amazing how powerful and flexible it is
|
||
-- schwanengesang @tensegrist@twitter
|
||
|
||
Every time I install @ubuntu, one of the first tools I install is
|
||
@gnuparallel. I love it.
|
||
-- Necati Demir @ndemir@twitter
|
||
|
||
Today I'm grateful for GNU parallel, especially with the --colsep and
|
||
--jobs parameters #GiveThanks
|
||
-- Erin Young @ErinYoun
|
||
|
||
It's not a data migration party until GNU Parallel is involved...
|
||
involved
|
||
involved
|
||
-- rrees @rrees@twitter
|
||
|
||
I have gotten a *ton* of mileage out of jq, awk, and GNU parallel,
|
||
even at multi-GB sizes.
|
||
-- Eric Wolak @ericthewolak@twitter
|
||
|
||
GNU parallel, which works a little bit like xargs, but has a much more friendly way of handling files with spaces and automatically parallelises calls. This tool has saved me a great deal of coding because it makes it so easy to write a program which does just one part of a task and then run it in parallel with load balancing and a nice progress bar. I cannot recommend this tool enough.
|
||
|
||
https://negfeedback.blogspot.com/2020/05/indispensable-command-line-tools.html
|
||
|
||
GNU parallel all the way!
|
||
-- David Manouchehri @DaveManouchehri@twitter
|
||
|
||
I found GNU Parallel and it's awesome.
|
||
-- Teddy Choi @TeddyJChoi@twitter
|
||
|
||
Well anyway, It was blazingly fast and astonished by performance. Guess I'll never use xargs.
|
||
-- (Not) Akaming @_Akamig@twitter
|
||
|
||
Yeah, GNU parallel is a beast when used accordingly.
|
||
-- @lsde@twitter
|
||
|
||
GNU Parallel is awesome
|
||
-- Gaurav Jain @gauravjain49@twitter
|
||
|
||
My new favourite command line tool is GNU Parallel. After knowing it for a while I just recently came back to it for a refactor of a script and now I've changed like 4 different huge scripts into tiny parallel scripts that do their job literally 3 to 10 times faster and just [mind blown]
|
||
-- Izzy @angry_izzy@twitter
|
||
|
||
I love so much @GnuParallel to parallelize my SQL requests on @PostgreSQL
|
||
-- @rmaziere_85 Romain
|
||
|
||
GNU parallel really changed how I do a lot of data processing stuff
|
||
-- Brendan Dolan-Gavitt @moyix@twitter
|
||
|
||
Parallel is a life send, what an awesome piece of software.
|
||
-- Blaze9@reddit.com
|
||
|
||
It's really powerful for smallish data processing scripts
|
||
-- João Veiga @jcsvveiga@twitter
|
||
|
||
It's the MapReduce of our generation!
|
||
-- Mathias Meyer @roidrage@twitter
|
||
|
||
GNU parallel is a seriously powerful flexible and tool.
|
||
-- Greg Hurrell @wincent@twitter
|
||
|
||
[socat is] top3 on my "crazy and powerful" list (the other two being
|
||
gnu parallel and emacs).
|
||
-- Peter Kjellström @nsccap@twitter
|
||
|
||
GNU/Parallel umm... tempting.
|
||
-- k-leb k @dcatdemon@twitter
|
||
|
||
Ain't that a dirty hack ;-)
|
||
-- ./hans
|
||
|
||
Not sure if it counts as an "alt" tool but GNU parallel really took my shell scripting game to the next level.
|
||
-- @alinajaf@twitter
|
||
|
||
=== Used ===
|
||
|
||
The syntax for GNU Parallel is so slick that I often use it just to make my script read nicer, and the parallelism is a cherry on top.
|
||
-- Epistaxis@reddit
|
||
|
||
Parallel has been (and still is) super useful and simple tool for speeding up all kinds of shell tasks during my career.
|
||
-- ValtteriL@ycombinator
|
||
|
||
It's amazing how fast you can get with bash pipelines and GNU Parallel.
|
||
-- Eric Pauley @EricPauley_
|
||
|
||
Immensely useful which I am forever grateful that it exists.
|
||
-- AlexDragusin@ycombinator
|
||
|
||
My favorite software, ever. Keep the good work.
|
||
-- Federico Alves @federicoalves@twitter
|
||
|
||
I also prefer gnu parallel. Mainly because it makes embarrassingly
|
||
parallel tasks embarrassingly easy to run on the command line.
|
||
-- Vincent D. Warmerdam @fishnets88@twitter
|
||
|
||
I'm going to make GNU parallel my new superpower this year.
|
||
-- Christian G. Warden @xn@twitter
|
||
|
||
My favorite man page is that of GNU parallel.
|
||
-- Jeroen Janssens @jeroenhjanssens@twitter
|
||
|
||
GNU parallel 便利すぎ
|
||
-- @butagannen@twitter 豚顔面
|
||
|
||
GNU parallel is a severely underrated tool for just spawning a
|
||
shitload of tasks. I use it every day and fucking love it.
|
||
-- Jane @serialexpjane@twitter
|
||
|
||
Take a look at GNU Parallel. It’s magnificent.
|
||
-- Siira@ycombinator.com
|
||
|
||
Safe to say, @GnuParallel was a life changer during my PhD! It helped
|
||
me optimise so many of my tasks and analyses.
|
||
-- Parice Brandies @PariceBrandies@twitter
|
||
|
||
We use gnu parallel now - and happier for it.
|
||
-- Ben Davies @benjamindavies@twitter
|
||
|
||
GNU Parallel makes my life so much easier.
|
||
I'm glad I don't have to implement multi-threaded Python scripts on the regular.
|
||
-- Fredrick Brennan @fr_brennan@twitter
|
||
|
||
If you work with lots of files at once
|
||
Take a good look at GNU parallel
|
||
Change your life for the better
|
||
-- French @notareverser@twitter
|
||
|
||
GNU Parallel is your friend.
|
||
Can shorten that time by X cores.
|
||
-- iRODS @irods@twitter
|
||
|
||
GNU Parallel is my new favorite thing
|
||
-- Will Tejeda @thewilltejeda
|
||
|
||
GNU Parallel is a priceless timesaver
|
||
-- June "Wakalix" @TWakalix@twitter
|
||
|
||
I think many people would be surprised to learn that GNU parallel is
|
||
"just" a single Perl script.
|
||
-- Peter Menzel @ptr_menzel@twitter
|
||
|
||
GNU Parallel is my single favourite tool for batch processing data
|
||
from the command line.
|
||
-- Jeff Wintersinger @jwintersinger
|
||
|
||
GNU parallel should be taught in class, it is one of the best tools
|
||
to run grids of experiments
|
||
-- no love deep learning @tetraduzione@twitter
|
||
|
||
I get a weird sense of satisfaction every single time I see the
|
||
lovely logo of #GNU Parallel (plus, what an underrated piece of
|
||
great software!)
|
||
-- Emre Sevinç @EmreSevinc@twitter
|
||
|
||
My favorite man page is that of GNU parallel.
|
||
-- Jeroen Janssens @jeroenhjanssens@twitter
|
||
|
||
Gnu parallel is also awesome, fwiw.
|
||
-- Rogan Dawes @RoganDawes@twitter
|
||
|
||
With multicore systems everywhere GNU Parallel is a must have tool.
|
||
-- Neil H. Watson @neil_h_watson@twitter
|
||
|
||
Who needs spark when GNU Parallel exists
|
||
-- MatthijsB @MatthijsBrs@twitter
|
||
|
||
GNU Parallel: dead simple process-level parallelization of ad hoc
|
||
tasks. Write for a chunk, let gnu manage the splitting, permutations
|
||
and pool concurrency.
|
||
-- Nick Ursa @nickursa@twitter
|
||
|
||
I wish more command line software had example pages as robust as GNU Parallel
|
||
-- Lucidbeaming @lucidbeaming
|
||
|
||
GNU parallel is straight up incredible.
|
||
-- Ben Johnson @biobenkj@twtter
|
||
|
||
GNU parallel has helped me kill a Hadoop cluster before.
|
||
-- Travis Campbell @hcoyote@twitter
|
||
|
||
Parallel is the BEST command.
|
||
-- Nick @NickInfoSec@twitter
|
||
|
||
[L]earning about parallel was amazing for me, it gives us many beautiful solutions.
|
||
-- SergioAraujo@stackoverflow
|
||
|
||
I've said it before: The command line program GNU Parallel is a godsend.
|
||
-- Jo Chr. Oterhals @oterhals
|
||
|
||
IMHO, SQLite and GNU Parallel are among the world's great software.
|
||
-- singe@reddit
|
||
|
||
It is, beyond absolutely any doubt whatsoever, the single most
|
||
important tool I use in making me a productive bioinformatician.
|
||
-- A-N-Other@reddit.com
|
||
|
||
It is SUPER easy to speed up jobs from the command line w/ GNU parallel.
|
||
-- B3n @B3njaminHimes@twitter
|
||
|
||
I want to make a shout-out for @GnuParallel, it's a work of beauty and power
|
||
-- Cristian Consonni @CristianCantoro
|
||
|
||
Amazingly useful script!
|
||
-- unxusr@reddit.com
|
||
|
||
Y'all need some GNU parallel in your lives
|
||
-- ChaKu @ChaiLovesChai@twitter
|
||
|
||
|
||
There are so many things to love about GNU parallel. You could honestly teach a whole parallel computing course with it and never have to leave it for a real language.
|
||
-- Aubrey Bailey @DNAvinci@twitter
|
||
|
||
|
||
With GNU Parallel you sure can!
|
||
I like getting things done
|
||
|
||
--Kyle Lady @kylelady@twitter
|
||
|
||
Ok! GNU Parallel is one of the best things out there. Almost as good as vanilla ice cream.
|
||
-- @coffe@mastodon.art
|
||
|
||
HOLY STUFF I LOVE GNU PARALLEL
|
||
-- Nick (but spookier) @NickInfoSec@twitter
|
||
|
||
|
||
What this country needs is a president with a strong understanding of GNU Parallel, Awk, Sed, Grep, and Diff
|
||
-- Gregg Housh @GreggHoush
|
||
|
||
I know I'm late to the party but GNU Parallel is truly amazing!
|
||
-- Sam Diaz-Munoz @sociovirology
|
||
|
||
GNU parallel is a thing of magic.
|
||
-- Josh Meyer @joshmeyerphd@twitter
|
||
|
||
I've been using GNU Parallel very much and effectively lately.
|
||
Such an easy way to get huge speed-ups with my simple bash/Perl/Python
|
||
programs -- parallelize them!
|
||
-- Ken Youens-Clark @kycl4rk@twitter
|
||
|
||
Alias and vars
|
||
export them more easily
|
||
With env_parallel
|
||
-- Ole Tange
|
||
|
||
You know what?
|
||
GNU Parallel is cool.
|
||
Concurrency, but in the Unix-philosophy style,
|
||
without the Enterprise wankeriness.
|
||
-- NickM bokkiedog@twitter
|
||
|
||
An ode to GNU parallel
|
||
An ode to GNU parallel
|
||
An ode to GNU parallel
|
||
An ode to GNU parallel
|
||
An ode to GNU parallel
|
||
An ode to GNU parallel
|
||
-- Adam Stuckert PoisonEcology@twitter
|
||
|
||
--limit can
|
||
limit jobs dynamic'ly
|
||
given a command
|
||
--ole-tange
|
||
|
||
--line-buffer
|
||
touches files on disk no more
|
||
faster than before
|
||
-- Ole Tange
|
||
|
||
Always test your script
|
||
using a --dry-run
|
||
or --dr
|
||
-- Ole Tange
|
||
|
||
--rpl
|
||
used to be a static string
|
||
Now it can take args
|
||
--ole-tange
|
||
|
||
--tee will pipe
|
||
all data to all the jobs
|
||
used with --pipe
|
||
--ole-tange
|
||
|
||
Speed is good for you
|
||
Multi speed is better still
|
||
Use GNU Parallel
|
||
-- Ole Tange
|
||
|
||
Want to monitor
|
||
many job outputs at once:
|
||
--tmuxpane
|
||
--ole-tange
|
||
|
||
Pipes are fast and good.
|
||
Use them in your programs, too.
|
||
Use GNU Parallel
|
||
-- Ole Tange
|
||
|
||
Programs using net
|
||
only spare capacity
|
||
niceload dash dash net
|
||
-- Ole Tange
|
||
|
||
xapply too strict?
|
||
:::+
|
||
is just made for you
|
||
-- Ole Tange
|
||
|
||
Save environment
|
||
Functions, vars and other stuff
|
||
Try env_parallel
|
||
-- Ole Tange
|
||
|
||
Many jobs to sort.
|
||
SQL would help a lot.
|
||
Dash dash sql
|
||
-- Ole Tange
|
||
|
||
Dash dash pipe too slow?
|
||
Dash dash pipepart is your friend.
|
||
Use GNU Parallel
|
||
-- Ole Tange
|
||
|
||
|||||
|
||
|||||||
|
||
Use GNU Parallel
|
||
-- Malcolm Cook
|
||
|
||
UNIX pipe too tight?
|
||
You should then learn dash dash pipe.
|
||
Use GNU Parallel
|
||
-- Ole Tange
|
||
|
||
In parallel land
|
||
everything is quick and fast
|
||
Use GNU Parallel.
|
||
-- Ole Tange
|
||
|
||
Jobs in parallel
|
||
faster run than single jobs
|
||
Use GNU Parallel.
|
||
-- Ole Tange
|
||
|
||
Multi core I see.
|
||
Doing nothing. I say no!
|
||
Use GNU Parallel.
|
||
-- Ole Tange
|
||
|
||
Programs very slow.
|
||
Multiple can run at once.
|
||
Use GNU Parallel.
|
||
-- Ole Tange
|
||
|
||
Many jobs to run.
|
||
Idle cores is not your thing.
|
||
Use GNU Parallel.
|
||
-- Ole Tange
|
||
|
||
SSH set up?
|
||
Instant cluster needed now?
|
||
Use GNU Parallel.
|
||
-- Ole Tange
|
||
|
||
xargs' space and quote
|
||
headache causing behaviour.
|
||
Use GNU Parallel
|
||
-- Ole Tange
|
||
|
||
If not enough time:
|
||
Run jobs using Parallel.
|
||
Keeps CPUs warm.
|
||
-- Ole Tange
|
||
|
||
Hadoop bit too much?
|
||
Want a simpler syntax now?
|
||
Use GNU Parallel.
|
||
-- Ole Tange
|
||
|
||
Shellshock pain hits us.
|
||
Dash dash env is affected.
|
||
Upgrade Parallel.
|
||
-- Ole Tange
|
||
|
||
goodbye to xargs
|
||
quoting is still a black art
|
||
welcome parallel
|
||
-- Malcolm Cook
|
||
|
||
bash for loops vanquished
|
||
jobs fan out over network
|
||
collated results!
|
||
-- Malcolm Cook
|
||
|
||
code fork headache blues?
|
||
option P is your new friend
|
||
`man parallel` now!
|
||
-- Malcolm Cook
|
||
|
||
Are you tired of
|
||
inflexible replacements?
|
||
Use Perl expressions.
|
||
-- Ole Tange |