tange/parallel
1
0
Fork 0
mirror of https://git.savannah.gnu.org/git/parallel.git synced 2024-11-22 14:07:55 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Updated docs.

Browse source
This commit is contained in:
Ole Tange 2013-11-22 23:31:46 +01:00
parent b1d3013f50
commit ff41d4360f
5 changed files with 160 additions and 56 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

43
NEWS
View file

@ -1,3 +1,46 @@
20131122
* A citation notice is printed on stderr only if stderr is a terminal,
the user has not specified --no-notice and the user has not run
--bibtex once. This makes the release alpha quality.
* --compress will compress temporary files. If the output is big and
very compressible this will take up less disk space in $TMPDIR and
possibly be faster due to less disk I/O.
* --compress-program comtrols which program to use for compressing
temporary files.
* --bar show progress as a progress bar compatible with zenity.
* --resume can now be used with --result: Jobs already run will be
skipped.
* --transfer and --basefile support paths relative to the --workdir by
inserting /./ into the path.
* GNU Parallel was used (unfortunately with improper citation) in:
'fastphylo: Fast tools for phylogenetics'
http://www.biomedcentral.com/1471-2105/14/334/abstract
* Using GNU parallel
http://davetang.org/muse/2013/11/18/using-gnu-parallel/
* Techlux - GNU - Parallel (German)
https://techlux.de/blog/2013/11/07/gnu-parallel/
* awk, sed, bzip2, grep, wc на всех ядрах
http://vk.com/page-30666517_45528467
* 如何利用多核CPU來加速你的Linux命令 — awk, sed, bzip2, grep, wc等
http://www.hksilicon.com/kb/articles/290543/CPULinuxawk-sed-bzip2-grep-wc
* GNU Parallel (Japanese)
http://jarp.does.notwork.org/diary/201311b.html#20131117
* Bug fixes and man page updates.
20131022 20131022
* --transfer files with /./ in the path will copy the files relative * --transfer files with /./ in the path will copy the files relative

19
doc/release_new_version
View file

@ -208,6 +208,25 @@ available for download at: http://ftp.gnu.org/gnu/parallel/
New in this release: New in this release:
* A citation notice is printed on stderr only if stderr is a terminal,
the user has not specified --no-notice and the user has not run
--bibtex once. This makes the release alpha quality.
* --compress will compress temporary files. If the output is big and
very compressible this will take up less disk space in $TMPDIR and
possibly be faster due to less disk I/O.
* --compress-program comtrols which program to use for compressing
temporary files.
* --bar show progress as a progress bar compatible with zenity.
* --resume can now be used with --result: Jobs already run will be
skipped.
* --transfer and --basefile support paths relative to the --workdir by
inserting /./ into the path.
* GNU Parallel was used (unfortunately with improper citation) in: * GNU Parallel was used (unfortunately with improper citation) in:
'fastphylo: Fast tools for phylogenetics' 'fastphylo: Fast tools for phylogenetics'
http://www.biomedcentral.com/1471-2105/14/334/abstract http://www.biomedcentral.com/1471-2105/14/334/abstract

BIN
src/parallel.pdf
View file

Binary file not shown.

65
src/parallel.pod
View file

@ -301,6 +301,16 @@ string that is not in the command line.
See also: B<:::>. See also: B<:::>.
=item B<--bar> (alpha testing)
Show progress as a progress bar. In the bar is shown: % of jobs
completed, estimated seconds left, and number of jobs started.
It is compatible with B<zenity>:
seq 1000 | parallel -j30 --bar '(echo {};sleep 0.1)' 2> >(zenity --progress --auto-kill) | wc
=item B<--basefile> I<file> (alpha testing) =item B<--basefile> I<file> (alpha testing)
=item B<--bf> I<file> (alpha testing) =item B<--bf> I<file> (alpha testing)
@ -338,9 +348,10 @@ See also: B<--fg>, B<man sem>.
Implies B<--semaphore>. Implies B<--semaphore>.
=item B<--bibtex> =item B<--bibtex> (alpha testing)
Print the BibTeX entry for GNU B<parallel>. Print the BibTeX entry for GNU B<parallel> and disable citation
notice.
=item B<--block> I<size> =item B<--block> I<size>
@ -395,7 +406,7 @@ I<regexp> is a Perl Regular Expression:
http://perldoc.perl.org/perlre.html http://perldoc.perl.org/perlre.html
=item B<--compress> (pre-alpha testing) =item B<--compress> (alpha testing)
Compress temporary files. If the output is big and very compressible Compress temporary files. If the output is big and very compressible
this will take up less disk space in $TMPDIR and possibly be faster due to less this will take up less disk space in $TMPDIR and possibly be faster due to less
@ -406,7 +417,7 @@ B<plzip>, B<bzip2>, B<lzma>, B<lzip>, B<xz> in that order, and use the
first available. first available.
=item B<--compress-program> I<prg> (pre-alpha testing) =item B<--compress-program> I<prg> (alpha testing)
Use I<prg> for compressing temporary files. It is assumed that I<prg Use I<prg> for compressing temporary files. It is assumed that I<prg
-dc> will decompress stdin (standard input) to stdout (standard -dc> will decompress stdin (standard input) to stdout (standard
@ -597,7 +608,7 @@ specified, and for B<-I>{} otherwise. This option is deprecated;
use B<-I> instead. use B<-I> instead.
=item B<--joblog> I<logfile> (beta testing) =item B<--joblog> I<logfile> (alpha testing)
Logfile for executed jobs. Save a list of the executed jobs to Logfile for executed jobs. Save a list of the executed jobs to
I<logfile> in the following TAB separated format: sequence number, I<logfile> in the following TAB separated format: sequence number,
@ -792,7 +803,7 @@ This is useful for scripts that depend on features only available from
a certain version of GNU B<parallel>. a certain version of GNU B<parallel>.
=item B<--nonall> (beta testing) =item B<--nonall>
B<--onall> with no arguments. Run the command on all computers given B<--onall> with no arguments. Run the command on all computers given
with B<--sshlogin> but take no arguments. GNU B<parallel> will log with B<--sshlogin> but take no arguments. GNU B<parallel> will log
@ -803,7 +814,7 @@ This is useful for running the same command (e.g. uptime) on a list of
servers. servers.
=item B<--onall> (beta testing) =item B<--onall>
Run all the jobs on all computers given with B<--sshlogin>. GNU Run all the jobs on all computers given with B<--sshlogin>. GNU
B<parallel> will log into B<--jobs> number of computers in parallel B<parallel> will log into B<--jobs> number of computers in parallel
@ -815,11 +826,11 @@ When using B<--group> the output will be grouped by each server, so
all the output from one server will be grouped together. all the output from one server will be grouped together.
=item B<--output-as-files> (beta testing) =item B<--output-as-files> (alpha testing)
=item B<--outputasfiles> (beta testing) =item B<--outputasfiles> (alpha testing)
=item B<--files> (beta testing) =item B<--files> (alpha testing)
Instead of printing the output to stdout (standard output) the output Instead of printing the output to stdout (standard output) the output
of each job is saved in a file and the filename is then printed. of each job is saved in a file and the filename is then printed.
@ -932,6 +943,14 @@ Print the number of CPU cores and exit (used by GNU B<parallel> itself
to determine the number of CPU cores on remote computers). to determine the number of CPU cores on remote computers).
=item B<--no-notice> (alpha testing)
Do not display citation notice. A citation notice is printed on stderr
(standard error) only if stderr (standard error) is a terminal, the
user has not specified B<--no-notice>, and the user has not run
B<--bibtex> once.
=item B<--nice> I<niceness> =item B<--nice> I<niceness>
Run the command at this niceness. For simple commands you can just add Run the command at this niceness. For simple commands you can just add
@ -1077,15 +1096,16 @@ will generate the files:
See also B<--files>, B<--header>, B<--joblog>. See also B<--files>, B<--header>, B<--joblog>.
=item B<--resume> =item B<--resume> (alpha testing)
Resumes from the last unfinished job. By reading B<--joblog> GNU Resumes from the last unfinished job. By reading B<--joblog> or the
B<parallel> will figure out the last unfinished job and continue from B<--results> dir GNU B<parallel> will figure out the last unfinished
there. As GNU B<parallel> only looks at the sequence numbers in job and continue from there. As GNU B<parallel> only looks at the
B<--joblog> then the input, the command, and B<--joblog> all have to sequence numbers in B<--joblog> then the input, the command, and
remain unchanged; otherwise GNU B<parallel> may run wrong commands. B<--joblog> all have to remain unchanged; otherwise GNU B<parallel>
may run wrong commands.
See also B<--joblog>, B<--resume-failed>. See also B<--joblog>, B<--results>, B<--resume-failed>.
=item B<--resume-failed> =item B<--resume-failed>
@ -1466,9 +1486,10 @@ See also B<-v>, B<-p>.
=item B<--transfer> (alpha testing) =item B<--transfer> (alpha testing)
Transfer files to remote computers. B<--transfer> is used with Transfer files to remote computers. B<--transfer> is used with
B<--sshlogin> when the arguments are files and should be transferred to B<--sshlogin> when the arguments are files and should be transferred
the remote computers. The files will be transferred using B<rsync> and to the remote computers. The files will be transferred using B<rsync>
will be put relative to the default login dir. E.g. and will be put relative to the default work dir. If the path contains
/./ the remaining path will be relative to the work dir. E.g.
echo foo/bar.txt | parallel \ echo foo/bar.txt | parallel \
--sshlogin server.example.com --transfer wc --sshlogin server.example.com --transfer wc
@ -1575,9 +1596,9 @@ Use B<-v> B<-v> to print the wrapping ssh command when running remotely.
Print the version GNU B<parallel> and exit. Print the version GNU B<parallel> and exit.
=item B<--workdir> I<mydir> =item B<--workdir> I<mydir> (alpha testing)
=item B<--wd> I<mydir> =item B<--wd> I<mydir> (alpha testing)
Files transferred using B<--transfer> and B<--return> will be relative Files transferred using B<--transfer> and B<--return> will be relative
to I<mydir> on remote computers, and the command will be executed in to I<mydir> on remote computers, and the command will be executed in

89
src/parallel.texi
View file

@ -317,6 +317,16 @@ string that is not in the command line.
See also: @strong{:::}. See also: @strong{:::}.
@item @strong{--bar} (alpha testing)
@anchor{@strong{--bar} (alpha testing)}
Show progress as a progress bar. In the bar is shown: % of jobs
completed, estimated seconds left, and number of jobs started.
It is compatible with @strong{zenity}:
seq 1000 | parallel -j30 --bar '(echo @{@};sleep 0.1)' 2> >(zenity --progress --auto-kill) | wc
@item @strong{--basefile} @emph{file} (alpha testing) @item @strong{--basefile} @emph{file} (alpha testing)
@anchor{@strong{--basefile} @emph{file} (alpha testing)} @anchor{@strong{--basefile} @emph{file} (alpha testing)}
@ -357,10 +367,11 @@ See also: @strong{--fg}, @strong{man sem}.
Implies @strong{--semaphore}. Implies @strong{--semaphore}.
@item @strong{--bibtex} @item @strong{--bibtex} (alpha testing)
@anchor{@strong{--bibtex}} @anchor{@strong{--bibtex} (alpha testing)}
Print the BibTeX entry for GNU @strong{parallel}. Print the BibTeX entry for GNU @strong{parallel} and disable citation
notice.
@item @strong{--block} @emph{size} @item @strong{--block} @emph{size}
@anchor{@strong{--block} @emph{size}} @anchor{@strong{--block} @emph{size}}
@ -418,8 +429,8 @@ separating the columns. The n'th column can be access using
@emph{regexp} is a Perl Regular Expression: @emph{regexp} is a Perl Regular Expression:
http://perldoc.perl.org/perlre.html http://perldoc.perl.org/perlre.html
@item @strong{--compress} (pre-alpha testing) @item @strong{--compress} (alpha testing)
@anchor{@strong{--compress} (pre-alpha testing)} @anchor{@strong{--compress} (alpha testing)}
Compress temporary files. If the output is big and very compressible Compress temporary files. If the output is big and very compressible
this will take up less disk space in $TMPDIR and possibly be faster due to less this will take up less disk space in $TMPDIR and possibly be faster due to less
@ -429,8 +440,8 @@ GNU @strong{parallel} will try @strong{lzop}, @strong{pigz}, @strong{gzip}, @str
@strong{plzip}, @strong{bzip2}, @strong{lzma}, @strong{lzip}, @strong{xz} in that order, and use the @strong{plzip}, @strong{bzip2}, @strong{lzma}, @strong{lzip}, @strong{xz} in that order, and use the
first available. first available.
@item @strong{--compress-program} @emph{prg} (pre-alpha testing) @item @strong{--compress-program} @emph{prg} (alpha testing)
@anchor{@strong{--compress-program} @emph{prg} (pre-alpha testing)} @anchor{@strong{--compress-program} @emph{prg} (alpha testing)}
Use @emph{prg} for compressing temporary files. It is assumed that @emph{prg Use @emph{prg} for compressing temporary files. It is assumed that @emph{prg
-dc} will decompress stdin (standard input) to stdout (standard -dc} will decompress stdin (standard input) to stdout (standard
@ -629,8 +640,8 @@ This option is a synonym for @strong{-I}@emph{replace-str} if @emph{replace-str}
specified, and for @strong{-I}@{@} otherwise. This option is deprecated; specified, and for @strong{-I}@{@} otherwise. This option is deprecated;
use @strong{-I} instead. use @strong{-I} instead.
@item @strong{--joblog} @emph{logfile} (beta testing) @item @strong{--joblog} @emph{logfile} (alpha testing)
@anchor{@strong{--joblog} @emph{logfile} (beta testing)} @anchor{@strong{--joblog} @emph{logfile} (alpha testing)}
Logfile for executed jobs. Save a list of the executed jobs to Logfile for executed jobs. Save a list of the executed jobs to
@emph{logfile} in the following TAB separated format: sequence number, @emph{logfile} in the following TAB separated format: sequence number,
@ -845,8 +856,8 @@ GNU @strong{parallel} is less than @emph{version} the exit code is
This is useful for scripts that depend on features only available from This is useful for scripts that depend on features only available from
a certain version of GNU @strong{parallel}. a certain version of GNU @strong{parallel}.
@item @strong{--nonall} (beta testing) @item @strong{--nonall}
@anchor{@strong{--nonall} (beta testing)} @anchor{@strong{--nonall}}
@strong{--onall} with no arguments. Run the command on all computers given @strong{--onall} with no arguments. Run the command on all computers given
with @strong{--sshlogin} but take no arguments. GNU @strong{parallel} will log with @strong{--sshlogin} but take no arguments. GNU @strong{parallel} will log
@ -856,8 +867,8 @@ computer. @strong{-j} adjusts how many computers to log into in parallel.
This is useful for running the same command (e.g. uptime) on a list of This is useful for running the same command (e.g. uptime) on a list of
servers. servers.
@item @strong{--onall} (beta testing) @item @strong{--onall}
@anchor{@strong{--onall} (beta testing)} @anchor{@strong{--onall}}
Run all the jobs on all computers given with @strong{--sshlogin}. GNU Run all the jobs on all computers given with @strong{--sshlogin}. GNU
@strong{parallel} will log into @strong{--jobs} number of computers in parallel @strong{parallel} will log into @strong{--jobs} number of computers in parallel
@ -868,14 +879,14 @@ adjusts how many computers to log into in parallel.
When using @strong{--group} the output will be grouped by each server, so When using @strong{--group} the output will be grouped by each server, so
all the output from one server will be grouped together. all the output from one server will be grouped together.
@item @strong{--output-as-files} (beta testing) @item @strong{--output-as-files} (alpha testing)
@anchor{@strong{--output-as-files} (beta testing)} @anchor{@strong{--output-as-files} (alpha testing)}
@item @strong{--outputasfiles} (beta testing) @item @strong{--outputasfiles} (alpha testing)
@anchor{@strong{--outputasfiles} (beta testing)} @anchor{@strong{--outputasfiles} (alpha testing)}
@item @strong{--files} (beta testing) @item @strong{--files} (alpha testing)
@anchor{@strong{--files} (beta testing)} @anchor{@strong{--files} (alpha testing)}
Instead of printing the output to stdout (standard output) the output Instead of printing the output to stdout (standard output) the output
of each job is saved in a file and the filename is then printed. of each job is saved in a file and the filename is then printed.
@ -990,6 +1001,14 @@ itself to determine the number of physical CPUs on remote computers).
Print the number of CPU cores and exit (used by GNU @strong{parallel} itself Print the number of CPU cores and exit (used by GNU @strong{parallel} itself
to determine the number of CPU cores on remote computers). to determine the number of CPU cores on remote computers).
@item @strong{--no-notice} (alpha testing)
@anchor{@strong{--no-notice} (alpha testing)}
Do not display citing notice. A citation notice is printed on stderr
(standard error) only if stderr (standard error) is a terminal, the
user has not specified @strong{--no-notice} and the user has not run --bibtex
once.
@item @strong{--nice} @emph{niceness} @item @strong{--nice} @emph{niceness}
@anchor{@strong{--nice} @emph{niceness}} @anchor{@strong{--nice} @emph{niceness}}
@ -1151,16 +1170,17 @@ will generate the files:
See also @strong{--files}, @strong{--header}, @strong{--joblog}. See also @strong{--files}, @strong{--header}, @strong{--joblog}.
@item @strong{--resume} @item @strong{--resume} (alpha testing)
@anchor{@strong{--resume}} @anchor{@strong{--resume} (alpha testing)}
Resumes from the last unfinished job. By reading @strong{--joblog} GNU Resumes from the last unfinished job. By reading @strong{--joblog} or the
@strong{parallel} will figure out the last unfinished job and continue from @strong{--results} dir GNU @strong{parallel} will figure out the last unfinished
there. As GNU @strong{parallel} only looks at the sequence numbers in job and continue from there. As GNU @strong{parallel} only looks at the
@strong{--joblog} then the input, the command, and @strong{--joblog} all have to sequence numbers in @strong{--joblog} then the input, the command, and
remain unchanged; otherwise GNU @strong{parallel} may run wrong commands. @strong{--joblog} all have to remain unchanged; otherwise GNU @strong{parallel}
may run wrong commands.
See also @strong{--joblog}, @strong{--resume-failed}. See also @strong{--joblog}, @strong{--results}, @strong{--resume-failed}.
@item @strong{--resume-failed} @item @strong{--resume-failed}
@anchor{@strong{--resume-failed}} @anchor{@strong{--resume-failed}}
@ -1572,9 +1592,10 @@ See also @strong{-v}, @strong{-p}.
@anchor{@strong{--transfer} (alpha testing)} @anchor{@strong{--transfer} (alpha testing)}
Transfer files to remote computers. @strong{--transfer} is used with Transfer files to remote computers. @strong{--transfer} is used with
@strong{--sshlogin} when the arguments are files and should be transferred to @strong{--sshlogin} when the arguments are files and should be transferred
the remote computers. The files will be transferred using @strong{rsync} and to the remote computers. The files will be transferred using @strong{rsync}
will be put relative to the default login dir. E.g. and will be put relative to the default work dir. If the path contains
/./ the remaining path will be relative to the work dir. E.g.
@verbatim @verbatim
echo foo/bar.txt | parallel \ echo foo/bar.txt | parallel \
@ -1691,11 +1712,11 @@ Use @strong{-v} @strong{-v} to print the wrapping ssh command when running remot
Print the version GNU @strong{parallel} and exit. Print the version GNU @strong{parallel} and exit.
@item @strong{--workdir} @emph{mydir} @item @strong{--workdir} @emph{mydir} (alpha testing)
@anchor{@strong{--workdir} @emph{mydir}} @anchor{@strong{--workdir} @emph{mydir} (alpha testing)}
@item @strong{--wd} @emph{mydir} @item @strong{--wd} @emph{mydir} (alpha testing)
@anchor{@strong{--wd} @emph{mydir}} @anchor{@strong{--wd} @emph{mydir} (alpha testing)}
Files transferred using @strong{--transfer} and @strong{--return} will be relative Files transferred using @strong{--transfer} and @strong{--return} will be relative
to @emph{mydir} on remote computers, and the command will be executed in to @emph{mydir} on remote computers, and the command will be executed in