parallel_alternatives: Added parallelize.

This commit is contained in:
Ole Tange 2024-04-28 22:53:45 +02:00
parent 2e237c4453
commit aea0ed19bc
3 changed files with 53 additions and 7 deletions

View file

@ -29,9 +29,6 @@ write a haiku about gnu parallel and the :::+ option
Big fan of GNU parallel! Big fan of GNU parallel!
-- lovelyloafers@reddit -- lovelyloafers@reddit
Im a big fan of GNU parallel!
-- Scott Cain @scottjcain@twitter
gnu parallelが便利すぎてxargsの使い方忘れちゃった gnu parallelが便利すぎてxargsの使い方忘れちゃった
-- @mathbbQ@twitter @凍結解除手続き中 -- @mathbbQ@twitter @凍結解除手続き中
@ -226,6 +223,9 @@ https://negfeedback.blogspot.com/2020/05/indispensable-command-line-tools.html
=== Used === === Used ===
Im a big fan of GNU parallel!
-- Scott Cain @scottjcain@twitter
GNU parallel ftw GNU parallel ftw
-- hostux.social/@rmpr @_paulmairo@twitter -- hostux.social/@rmpr @_paulmairo@twitter

View file

@ -1,7 +1,8 @@
<directory name="parallel" rev="386" vrev="2" srcmd5="ab71393f2d517952d0ad97f852c9ed64"> <directory name="parallel" rev="387" vrev="3" srcmd5="d4b27f35a15d2304a8b20e96ec794edd">
<entry name="Makefile" md5="a165cdcca90c744dc5a5b17cde72ad8e" size="1182" mtime="1698101769" /> <entry name="Makefile" md5="a165cdcca90c744dc5a5b17cde72ad8e" size="1182" mtime="1698101769" />
<entry name="PKGBUILD" md5="a09cd57a567ac2a751fb3caba2e8dda2" size="714" mtime="1713819623" /> <entry name="PKGBUILD" md5="5c9ffc0c528ac07959ab642181144752" size="714" mtime="1713820602" />
<entry name="parallel-20240422.tar.bz2" md5="a2c843451c66cc5780e1e043176fc0c6" size="2564834" mtime="1713819624" /> <entry name="parallel-20240422.tar.bz2" md5="cd00451ac96da8ca88e70ca9caca88ce" size="2564733" mtime="1713820602" />
<entry name="parallel.spec" md5="e13d4c5cb25aa997ebea29d490a72551" size="6224" mtime="1713819624" /> <entry name="parallel.spec" md5="e13d4c5cb25aa997ebea29d490a72551" size="6224" mtime="1713819624" />
<entry name="parallel_20240422.dsc" md5="d2d1efe8ebfcd606261d7c13a2ad1c08" size="556" mtime="1713819624" /> <entry name="parallel_20240422.dsc" md5="d2d1efe8ebfcd606261d7c13a2ad1c08" size="556" mtime="1713819624" />
<entry name="parallel_20240422.tar.gz" md5="4a5450078b018ab51531b2b40fea239d" size="2830066" mtime="1713820603" />
</directory> </directory>

View file

@ -4323,7 +4323,7 @@ ms/job it is slow.
Output takes O(n^2) time for output of size n. 10 MB takes ~1 second, Output takes O(n^2) time for output of size n. 10 MB takes ~1 second,
30 MB takes ~7 seconds, 100 MB takes ~60 seconds, 300 MB takes ~480 30 MB takes ~7 seconds, 100 MB takes ~60 seconds, 300 MB takes ~480
seconds, 1000 GB takes seconds, 1000 MB takes ~10000 seconds.
100 MB of output takes around 1 GB of RAM. 100 MB of output takes around 1 GB of RAM.
@ -4372,6 +4372,51 @@ https://github.com/baalimago/repeater
(Last checked: 2023-12) (Last checked: 2023-12)
=head2 DIFFERENCES BETWEEN parallelize AND GNU Parallel
Summary (see legend above):
=over
=item I1 - - - - - I7
=item - - - - - M6
=item O1 - O3 O4 O5 - O7 - - -
=item E1 - - E4 - - -
=item - - - - - - - - -
=item - -
=back
B<parallelize> runs the full line as a command. If the command is not
found, there is no warning.
The output at most ~1000000 lines/s. If the lines are short this is
quite slow. The lines can at most be 2047999 bytes long. Longer lines
cause segfault.
=head3 EXAMPLES
simple.dat:
sleep 5
foo
cat alire.toml
loc src/parallelize.adb
sh loc src/*.ad?
1$ bin/parallelize -v <simple.dat
1$ parallel <simple.dat
https://github.com/simonjwright/parallelize
(Last checked: 2024-04)
=head2 Todo =head2 Todo
https://github.com/justanhduc/task-spooler https://github.com/justanhduc/task-spooler