parallel/doc/haikus

479 lines
14 KiB
Plaintext
Raw Permalink Normal View History

# SPDX-FileCopyrightText: 2021-2022 Ole Tange, http://ole.tange.dk and Free Software and Foundation, Inc.
2021-04-22 16:20:41 +00:00
# SPDX-License-Identifier: GFDL-1.3-or-later
# SPDX-License-Identifier: CC-BY-SA-4.0
Quote of the month:
GNU parallel すごいな、シェルスクリプトを make -jX みたいにして並列
に走らせたいときに超便利
-- びーる @debugordie@twitter
GNU parallel いいよ、大量のデータファイルをスクリプトで処理する人とかにマジおすすめ
-- びーる @debugordie@twitter
2022-01-22 16:00:32 +00:00
Igual @GnuParallel te puede echar una mano. Te permite pensar y
programar en serie pero ejecutar en paralelo usando cores de una
máquina o máquinas remotas. Una vez que lo has usando te lamentas no
haberlo conocido antes :)
-- Juan Sierra Pons @juasiepo
If I could only keep 5 GNU utils, parallel would make it to the list
:)
-- 5heikki@reddit
Gnu Parallel: installed in every computer i have access to.
-- raffaele messuti @atomotic@twitter
gnu parallel is a good program
-- Pwn A. Day @pwnaday@twitter
Deus salve o gnu parallel
-- marcos @guv_Tuv@twitter
2021-11-22 23:59:51 +00:00
@a201
4
اليوم علمت بأداة تساعد على تشغيل أوامر الصدفة بشكل متوازي حسب عدد الأنوية الموجودة مما يجعلها أداة أساسية في عدة تنفيذ المهام.
الأداة اسمها gnu parallel
-- @AbuEsra2014@twitter زايد السعيدي
GNU parallel is very easy to use and has many features for
specialized use cases. Its a Perl script.
-- @harlekyn@twitter uʎʞǝlɹɐɥ
GNU parallelという神ツールを発見した
-- @schnell1232@twitter
How I love GNU parallel
-- @dathanvp@twitter Dathan Pattishall
GNU Parallelめっちゃ便利で偉い
-- аiгbus @airbus_P
2021-08-22 20:05:37 +00:00
I really liked GNU Parallel http://gnu.org/software/parallel/
one of the best tool to execute parallel jobs in the shell
-- Luca Molteni @volothamp@twitter
Have you heard of our lord and saviour GNU parallel?
-- kxyne @Kxyne@twitter
Installing GNU parallel is the second thing I do on any new Linux
machine, right after uninstalling pulseaudio.
-- Andy Cooper @integgroll@twitter
2021-02-17 14:49:47 +00:00
Parallel is amazing!
-- fatboy93@reddit
parallel is my new kink
-- python_noob_001@reddit
@GnuParallel the best thing ever and it's not up for debate #EOchat
-- Nathan Thomas @DrNASApants@twitter
Using [GNU Parallel is] super easy if you use xargs, and it is magic for making things multi-process. Then adding in the ssh magic it can do it is an incredible tool that is completely underutilized.
-- Ancients @Ancients@twitter
GNU Parallel is one of the most helpful tools I've been using recently, and it's just something like: parallel -j4 'gzip {}' ::: folder/*.csv
-- Milton Pividori @miltondp@twitter
This is a fantastic tool, and I wish I had upgraded from xargs years ago!
-- Stuart Anderson
2021-12-22 21:27:34 +00:00
GNU Parallel and ripgrep would be your friend here.
2021-01-21 20:01:30 +00:00
-- CENAPT @cenaptech@twitter
2021-01-20 00:15:16 +00:00
GNU Parallel is great
-- Newton's Flaming Laser Sword @swordgoesfwoosh@twitter
Try GNU parallel it's awesome, and exactly the thing you are looking for. It allows you to set number of processes running among many things. I use it a lot, and can't recommend it enough.
-- mapettheone@reddit
It's really quite amazing how powerful and flexible it is
-- schwanengesang @tensegrist@twitter
Every time I install @ubuntu, one of the first tools I install is
@gnuparallel. I love it.
-- Necati Demir @ndemir@twitter
Today I'm grateful for GNU parallel, especially with the --colsep and
--jobs parameters #GiveThanks
-- Erin Young @ErinYoun
It's not a data migration party until GNU Parallel is involved...
involved
involved
-- rrees @rrees@twitter
I have gotten a *ton* of mileage out of jq, awk, and GNU parallel,
even at multi-GB sizes.
-- Eric Wolak @ericthewolak@twitter
GNU parallel, which works a little bit like xargs, but has a much more friendly way of handling files with spaces and automatically parallelises calls. This tool has saved me a great deal of coding because it makes it so easy to write a program which does just one part of a task and then run it in parallel with load balancing and a nice progress bar. I cannot recommend this tool enough.
https://negfeedback.blogspot.com/2020/05/indispensable-command-line-tools.html
GNU parallel all the way!
-- David Manouchehri @DaveManouchehri@twitter
I found GNU Parallel and it's awesome.
-- Teddy Choi @TeddyJChoi@twitter
2020-04-22 18:15:59 +00:00
Well anyway, It was blazingly fast and astonished by performance. Guess I'll never use xargs.
-- (Not) Akaming @_Akamig@twitter
Yeah, GNU parallel is a beast when used accordingly.
-- @lsde@twitter
2019-08-22 21:07:56 +00:00
GNU Parallel is awesome
-- Gaurav Jain @gauravjain49@twitter
2020-04-22 18:15:59 +00:00
My new favourite command line tool is GNU Parallel. After knowing it for a while I just recently came back to it for a refactor of a script and now I've changed like 4 different huge scripts into tiny parallel scripts that do their job literally 3 to 10 times faster and just [mind blown]
2019-08-22 21:07:56 +00:00
-- Izzy @angry_izzy@twitter
I love so much @GnuParallel to parallelize my SQL requests on @PostgreSQL
-- @rmaziere_85 Romain
GNU parallel really changed how I do a lot of data processing stuff
-- Brendan Dolan-Gavitt @moyix@twitter
Parallel is a life send, what an awesome piece of software.
-- Blaze9@reddit.com
It's really powerful for smallish data processing scripts
-- João Veiga @jcsvveiga@twitter
It's the MapReduce of our generation!
-- Mathias Meyer @roidrage@twitter
GNU parallel is a seriously powerful flexible and tool.
2019-08-22 21:07:56 +00:00
-- Greg Hurrell @wincent@twitter
[socat is] top3 on my "crazy and powerful" list (the other two being
gnu parallel and emacs).
-- Peter Kjellström @nsccap@twitter
GNU/Parallel umm... tempting.
-- k-leb k @dcatdemon@twitter
Ain't that a dirty hack ;-)
-- ./hans
Not sure if it counts as an "alt" tool but GNU parallel really took my shell scripting game to the next level.
-- @alinajaf@twitter
=== Used ===
2020-02-23 00:10:44 +00:00
I also prefer gnu parallel. Mainly because it makes embarrassingly
parallel tasks embarrassingly easy to run on the command line.
-- Vincent D. Warmerdam @fishnets88@twitter
2022-01-22 16:00:32 +00:00
I'm going to make GNU parallel my new superpower this year.
-- Christian G. Warden @xn@twitter
My favorite man page is that of GNU parallel.
2021-12-22 21:27:34 +00:00
-- Jeroen Janssens @jeroenhjanssens@twitter
2021-11-22 23:59:51 +00:00
GNU parallel 便利すぎ
-- @butagannen@twitter 豚顔面
2021-10-22 20:10:09 +00:00
GNU parallel is a severely underrated tool for just spawning a
shitload of tasks. I use it every day and fucking love it.
-- Jane @serialexpjane@twitter
2021-09-22 21:37:24 +00:00
Take a look at GNU Parallel. Its magnificent.
-- Siira@ycombinator.com
2021-08-22 20:05:37 +00:00
Safe to say, @GnuParallel was a life changer during my PhD! It helped
me optimise so many of my tasks and analyses.
-- Parice Brandies @PariceBrandies@twitter
We use gnu parallel now - and happier for it.
2021-07-22 20:31:06 +00:00
-- Ben Davies @benjamindavies@twitter
2021-06-22 17:40:07 +00:00
GNU Parallel makes my life so much easier.
I'm glad I don't have to implement multi-threaded Python scripts on the regular.
-- Fredrick Brennan @fr_brennan@twitter
2021-05-22 23:18:09 +00:00
If you work with lots of files at once
Take a good look at GNU parallel
Change your life for the better
-- French @notareverser@twitter
2021-04-22 16:20:41 +00:00
GNU Parallel is your friend.
Can shorten that time by X cores.
-- iRODS @irods@twitter
GNU Parallel is my new favorite thing
-- Will Tejeda @thewilltejeda
2021-04-22 16:20:41 +00:00
GNU Parallel is a priceless timesaver
2021-02-21 21:39:24 +00:00
-- June "Wakalix" @TWakalix@twitter
2021-01-21 20:01:30 +00:00
I think many people would be surprised to learn that GNU parallel is
"just" a single Perl script.
-- Peter Menzel @ptr_menzel@twitter
2020-12-21 13:30:23 +00:00
GNU Parallel is my single favourite tool for batch processing data
from the command line.
-- Jeff Wintersinger @jwintersinger
GNU parallel should be taught in class, it is one of the best tools
to run grids of experiments
-- no love deep learning @tetraduzione@twitter
2020-10-22 20:16:17 +00:00
I get a weird sense of satisfaction every single time I see the
lovely logo of #GNU Parallel (plus, what an underrated piece of
great software!)
-- Emre Sevinç @EmreSevinc@twitter
2020-09-23 15:41:03 +00:00
My favorite man page is that of GNU parallel.
-- Jeroen Janssens @jeroenhjanssens@twitter
Gnu parallel is also awesome, fwiw.
-- Rogan Dawes @RoganDawes@twitter
With multicore systems everywhere GNU Parallel is a must have tool.
-- Neil H. Watson @neil_h_watson@twitter
2020-06-22 22:34:52 +00:00
Who needs spark when GNU Parallel exists
-- MatthijsB @MatthijsBrs@twitter
GNU Parallel: dead simple process-level parallelization of ad hoc
tasks. Write for a chunk, let gnu manage the splitting, permutations
and pool concurrency.
-- Nick Ursa @nickursa@twitter
2020-04-22 18:15:59 +00:00
I wish more command line software had example pages as robust as GNU Parallel
-- Lucidbeaming @lucidbeaming
GNU parallel is straight up incredible.
-- Ben Johnson @biobenkj@twtter
2020-03-21 20:32:24 +00:00
GNU parallel has helped me kill a Hadoop cluster before.
-- Travis Campbell @hcoyote@twitter
Parallel is the BEST command.
-- Nick @NickInfoSec@twitter
2020-02-23 00:10:44 +00:00
[L]earning about parallel was amazing for me, it gives us many beautiful solutions.
-- SergioAraujo@stackoverflow
2019-10-21 19:18:32 +00:00
I've said it before: The command line program GNU Parallel is a godsend.
-- Jo Chr. Oterhals @oterhals
IMHO, SQLite and GNU Parallel are among the world's great software.
-- singe@reddit
2019-08-22 21:07:56 +00:00
It is, beyond absolutely any doubt whatsoever, the single most
important tool I use in making me a productive bioinformatician.
-- A-N-Other@reddit.com
2019-07-23 07:00:09 +00:00
It is SUPER easy to speed up jobs from the command line w/ GNU parallel.
-- B3n @B3njaminHimes@twitter
I want to make a shout-out for @GnuParallel, it's a work of beauty and power
-- Cristian Consonni @CristianCantoro
2019-05-22 20:18:04 +00:00
Amazingly useful script!
-- unxusr@reddit.com
2019-04-21 18:16:33 +00:00
Y'all need some GNU parallel in your lives
-- ChaKu @ChaiLovesChai@twitter
There are so many things to love about GNU parallel. You could honestly teach a whole parallel computing course with it and never have to leave it for a real language.
-- Aubrey Bailey @DNAvinci@twitter
With GNU Parallel you sure can!
I like getting things done
--Kyle Lady @kylelady@twitter
Ok! GNU Parallel is one of the best things out there. Almost as good as vanilla ice cream.
-- @coffe@mastodon.art
HOLY STUFF I LOVE GNU PARALLEL
-- Nick (but spookier) @NickInfoSec@twitter
What this country needs is a president with a strong understanding of GNU Parallel, Awk, Sed, Grep, and Diff
-- Gregg Housh @GreggHoush
I know I'm late to the party but GNU Parallel is truly amazing!
-- Sam Diaz-Munoz @sociovirology
GNU parallel is a thing of magic.
-- Josh Meyer @joshmeyerphd@twitter
I've been using GNU Parallel very much and effectively lately.
Such an easy way to get huge speed-ups with my simple bash/Perl/Python
programs -- parallelize them!
-- Ken Youens-Clark @kycl4rk@twitter
Alias and vars
export them more easily
With env_parallel
-- Ole Tange
You know what?
GNU Parallel is cool.
Concurrency, but in the Unix-philosophy style,
without the Enterprise wankeriness.
-- NickM bokkiedog@twitter
An ode to GNU parallel
An ode to GNU parallel
An ode to GNU parallel
An ode to GNU parallel
An ode to GNU parallel
An ode to GNU parallel
-- Adam Stuckert PoisonEcology@twitter
--limit can
limit jobs dynamic'ly
given a command
--ole-tange
--line-buffer
touches files on disk no more
faster than before
-- Ole Tange
Always test your script
using a --dry-run
or --dr
-- Ole Tange
--rpl
used to be a static string
Now it can take args
--ole-tange
--tee will pipe
all data to all the jobs
used with --pipe
--ole-tange
Speed is good for you
Multi speed is better still
Use GNU Parallel
-- Ole Tange
Want to monitor
many job outputs at once:
--tmuxpane
--ole-tange
Pipes are fast and good.
Use them in your programs, too.
Use GNU Parallel
-- Ole Tange
Programs using net
only spare capacity
niceload dash dash net
-- Ole Tange
xapply too strict?
:::+
is just made for you
-- Ole Tange
Save environment
Functions, vars and other stuff
Try env_parallel
-- Ole Tange
Many jobs to sort.
SQL would help a lot.
Dash dash sql
-- Ole Tange
Dash dash pipe too slow?
Dash dash pipepart is your friend.
Use GNU Parallel
-- Ole Tange
|||||
|||||||
Use GNU Parallel
-- Malcolm Cook
UNIX pipe too tight?
You should then learn dash dash pipe.
Use GNU Parallel
-- Ole Tange
In parallel land
everything is quick and fast
Use GNU Parallel.
-- Ole Tange
Jobs in parallel
faster run than single jobs
Use GNU Parallel.
-- Ole Tange
Multi core I see.
Doing nothing. I say no!
Use GNU Parallel.
-- Ole Tange
Programs very slow.
Multiple can run at once.
Use GNU Parallel.
-- Ole Tange
Many jobs to run.
Idle cores is not your thing.
Use GNU Parallel.
-- Ole Tange
SSH set up?
Instant cluster needed now?
Use GNU Parallel.
-- Ole Tange
xargs' space and quote
headache causing behaviour.
Use GNU Parallel
-- Ole Tange
If not enough time:
Run jobs using Parallel.
Keeps CPUs warm.
-- Ole Tange
Hadoop bit too much?
Want a simpler syntax now?
Use GNU Parallel.
-- Ole Tange
Shellshock pain hits us.
Dash dash env is affected.
Upgrade Parallel.
-- Ole Tange
goodbye to xargs
quoting is still a black art
welcome parallel
-- Malcolm Cook
bash for loops vanquished
jobs fan out over network
collated results!
-- Malcolm Cook
code fork headache blues?
option P is your new friend
`man parallel` now!
-- Malcolm Cook
Are you tired of
inflexible replacements?
Use Perl expressions.
-- Ole Tange