parallel/NEWS

20110205

* --pipe splits piped data into blocks. Each block is piped to a
  program for processing. The piping and the programs will be run in
  parallel. Useful if the data you want to process is data for a
  program and not arguments.

* --blocksize sets the blocksize in bytes for --pipe. The blocksize is
  approximate. It can deviate as much as the size of one record.
  Default is 1M.

* --recstart sets the string matching the start of a
  record. Default is "".

* --recend sets the string matching the end of a
  record. Default is '\n'. To specify none use --recend "".

  If both --recstart and --recend are set, the end of a record must be
  followed immediately by a start of a record. This is useful if
  either recend or recstart can occur in the middle of a record.

* --remove-rec-sep removes the string matched by --recstart and
  --recend.

* --regexp will make GNU Parallel treat --recstart and --recend as
  regular expressions.

* --output-as-files will put the output of the programs into files and
  instead of giving the output GNU Parallel will output the name of
  these files.

* -N if used with --pipe sets the number of records to read.

* New video showing --pipe at
  http://www.youtube.com/watch?v=1ntxT-47VPA

* GNU Parallel was presented at FOSDEM.

* Article in USENIX Magazine ;login: (print)
  http://www.usenix.org/publications/login/2011-02/

* GNU Parallel is now on ohloh.net. Thanks to Wim Muskee.
  https://www.ohloh.net/p/gnu-parallel

* Advanced recursive example. Thanks to Ruarí Ødegaard.
  http://my.opera.com/ruario/blog/2011/01/24/editing-debian-packages-more-fun-with-gnu

* Small example on using GNU Parallel through 0install (German).
  http://forum.ubuntuusers.de/topic/gnu-parallel-mit-zero-install-kurzinfo-kein-wi/

* Bug fixes and man page updates.


20110122

* --joblog makes a simple log of completed jobs.

* -X now spreads arguments between job slots when reaching last
  argument. Use -j1 to avoid this.

* People on the email list have voted -j+0 to be the new default
  instead of -j9.

* First review in Polish. Thanks to Patryk Krawaczyński.
  http://nfsec.pl/root/2458

* Review in Spanish (in print).
  https://www.linux-magazine.es/issue/67/moreinfo.html

* Review in English. Thanks to Brian Gough.
  http://blogs.fsfe.org/bjg/2011/01/gnu-parallel-a-map-operator-for-the-command-line/

* Review in French. Thanks to Denis Dordoigne.
  http://linuxfr.org/2010/12/29/27715.html

* Review in Spanish.
  http://gufete.net/index.php?entry=entry110116-200022

* Article with advanced recursive example. Thanks to Ruarí Ødegaard
  http://my.opera.com/ruario/blog/2011/01/18/fun-with-gnu-parallel

* Use case for memcache.
  http://www.dctrwatson.com/2010/12/how-to-dump-memcache-keyvalue-pairs-fast/

* Bug fixes and man page updates.


20101222

* GNU niceload is now part of GNU Parallel. GNU niceload slows down a
  program if the load average is above a certain limit.

* Implemented --tmpdir to buffer standard output and standard error in
  a different place.

* Implemented --load to wait until the load is below a limit before
  starting another job on that computer.

* Implemented --nice set the niceness of jobs running both locally and
  remotely.

* Implemented --dry-run to print the job without running it.

* Implemented --tty as the old default of assigning a tty to the first
  job causes problems.

* Review with focus on clusters. Thanks to Taylor Gillespie
  http://www.unixpronews.com/unixpronews-49-20101019GNUParallelSpeedUpProcessingWithMulticoresClusters.html

* Review with focus on protein similarity. 
  http://kevinformatics.tumblr.com/post/2142473893/cluster-like-computing-using-gnu-parallel

* Review in Spanish.
  http://gr3p.com/2010/12/gnu-parallel-acelera-tus-scripts-en-linux

* Quite a few bug fixes and man page updates.


20101202

* Implemented {/} for the input line with the path removed (basename).

* Implemented {/.} for the input line with extension and path removed
  (basename).

* Output from --progress is now sent to standard error instead of
  standard output.

* --eta was broken and counted down from 0. Now fixed.

* Standard output and standard error are flushed after every job so if
  standard output does not end with a newline it will still be grouped
  with the rest of standard output.

* --command, -c, --file, and -f are now removed as options. They were
  never used in practice.

* GetOptionsFromArray rewritten to work with old Perl libraries.

* The file COPYING now contains the GNU General Public License 3

* Major rewrite to make the code more object oriented and easier to
  maintain in the future.


20101113

* Using -j myfile the number of jobs can be changed while GNU Parallel
  is running simply by changing the content of myfile.

* Implemented --profile to use different .parallel/config for
  different situations.

* Ugly newlines in $PARALLEL and .parallel/config are no longer
  needed.  Instead you have to use \ in front of special shell
  characters.

* --workdir puts the files transferred to remote machines in a
  specified directory.

* $PARALLEL_PID is set to the process id of GNU Parallel.

* $PARALLEL_SEQ is set to the sequence number of the job.

* -v now only shows the command to run. Use -v -v to show the
   ssh/rsync wrapping.

* Slow spawning error is now only a warning.

* If stdin is a tty and input is read from stdin you now get a
  warning.

* GNU sql: \n and \x0a in arguments is replaced with newline.

* Patch for Debian package and spelling mistakes. Thanks to Rogério
  Brito <rbrito at ime dot usp dot br>

* Mac OS X Homebrew package. Thanks to Jonathan Palardy <jonathan dot
  palardy at gmail dot com>

* FreeBSD port. Thanks to Chris Howey <howeyc at gmail dot com>

* Pardus package. Thanks to Fethican Coşkuner
  <fethicanc at gmail dot com>

* First review in Chinese. Thanks to 曾義峰:
  http://antbsd.twbbs.org/~ant/wordpress/?p=2876

* First review in print:
  http://www.linux-magazine.com/Issues/2010 Nov 2010

* First review in Spanish:
  http://www.muylinux.com/2010/10/18/gnu-parallel-computacion-paralela-a-golpe-de-comando

* First review in Dutch thanks to Koen Vervloesem <koen at vervloesem
  dot eu>:
  http://techworld.nl/technologie/33493/gebruik-al-je-processorkernen-met-gnu-parallel.html

* Blog review thanks to R. Tyler Croy <tyler at monkeypox dot org>:
  http://unethicalblogger.com/posts/2010/11/gnuparallel_changed_my_life

* 5000 views of the intro video:
  http://www.youtube.com/watch?v=OpaiGYxkSuQ

* As usual a bunch of bugfixes and more usage examples in the man
  page.

* GNU Parallel was presented at FSCONS 2010-11-07:
  http://www.fscons.org/fs/gnu-parallel Hopefully the
  video will be online soon.


20100922

* See GNU Parallel live at FSCONS 2010-11-07:
  http://www.fscons.org/fs/gnu-parallel

* Untested Debian and xUbuntu packages available through OpenSUSE
  build service:
  https://build.opensuse.org/package/show?package=parallel&project=home%3Atange

* Using --retries a job will be retried on another computer if it
  fails. This is useful if some jobs fail for no apparent reason (such
  as network failure).

* BSD xargs -o (open /dev/tty) is now default for the job running in
  foreground. Useful for interactive commands like:
  ls | parallel -Xuj1 vi

* GNU sql now supports SQLite.

* Renamed .dburl.aliases to .sql/aliases and /etc/sql/aliases.

* GNU sql now support --list-tables

* Alias for DBURL can contain '?query' part with %-quoting.


20100906

* Using --shebang GNU Parallel can be used as the parser for a script.
  E.g: #!/usr/bin/parallel --shebang traceroute (followed by lines of
  hosts)

* First community generated bugfixes

* Alt Linux package of GNU Parallel. Thanks to Michael Shigorin <mike
  at altlinux dot org>

* Sunfreeware package of GNU Parallel. Thanks to Steven M. Christensen
  <sunfreeware at gmail.com>

* Untested CentOS, Fedora, Mandriva, RedHat, and SUSE packages
  available through OpenSUSE build service:
  https://build.opensuse.org/package/show?package=parallel&project=home%3Atange

* Review of GNU Parallel. Thanks to Andrew McFague amcfague at wgen dot net
  http://www.andrew-mcfague.com/linux/utilities-linux/
  commands-every-serious-nix-user-should-know/#parallel

* First 1000 views of the intro video

* sql - a small script to access sql bases from the command line which
  is a handy companion to parallel --colsep


20100822

* Counting semaphore functionality: start a job in the background. If
  N jobs are already running, wait for one to complete. Examples:
  sem 'sleep 2; echo foo'; sem 'sleep 1; echo bar'; sem --wait
  sem -j2 'sleep 2; echo foo'; sem -j2 'sleep 1; echo bar'; sem --wait

* With --colsep a table can be used as input. Example:
  cat tab_sep_table | parallel --colsep '\t' echo col1 {1} col2 {2}

* --trim can remove white space around arguments.

* --sshloginfile '..' means use ~/.parallel/sshloginfile

* Zero install package. Thanks to Tim Cuthbertson <tim3d dot junk at
  gmail dot com>

* OpenSUSE package. Thanks to Markus Ammer <mkmm at gmx-topmail dot
  de>

* NixOS package. Thanks to Ludovic Courtès <ludo at gnu dot org>

* Web review http://oentend.blogspot.com/2010/08/gnu-parallel.html
  Thanks to Pavel Nuzhdin <pnzhdin at gmail dot com>

* Web review http://psung.blogspot.com/2010/08/gnu-parallel.html
  Thanks to Phil Sung <psung at alum dot mit dot edu>


20100722

* Arguments can now be given on the command line. Example:
  parallel bzip2 ::: *.txt

* xapply like functionality. Example:
  parallel diff {1} {2} :::: filelist1 filelist2
  parallel diff {1} {2} ">"{1.}-{2.}.diff :::: filelist1 filelist2

* Arch Linux package. Thanks to Peter Simons <simons at cryp dot to>

* Mandriva package. Thanks to Sandro Cazzaniga <kharec at mandriva dot
  org>

* -L -l -n -s now implies -X unless -m is set


20100620

* New video showing the new options.
  http://www.youtube.com/watch?v=OpaiGYxkSuQ or at
  http://tinyogg.com/watch/TORaR/ and http://tinyogg.com/watch/hfxKj/

* 100% options complete with xargs. All options for xargs can now be
  used in GNU Parallel - even the more exotic.

* --basefile for transferring basedata. When running jobs on remote
  computers --basefile will transfer files before the first jobs is
  run. It can be used to transfer data that remains the same for each
  job such as scripts or lookup tables.

* --progress shows progress. To see how many jobs is running on each
  server use --progress. It can be turned on even after GNU Parallel
  is started.

* --eta shows estimated time left in seconds.

* --halt-on-error stops if an error occurs. GNU Parallel will default
  to run all jobs - even if some of them fail. With --halt-on-error
  GNU Parallel can ignore errors, wait for the currently running jobs
  to finish, or stop immediately when an error occurs.


20100601

* GNU Parallel now has support for running jobs on remote machines

* Intro video http://www.youtube.com/watch?v=LlXDtd_pRaY
  http://openvideo.dailymotion.com/video/xdmqwz_gnu-parallel-script-processing-and_tech 


20100424

* GNU Parallel is now official GNU software