diff --git a/NEWS b/NEWS index d1fd9efb..04df30a1 100644 --- a/NEWS +++ b/NEWS @@ -1,3 +1,46 @@ +20131122 + +* A citation notice is printed on stderr only if stderr is a terminal, + the user has not specified --no-notice and the user has not run + --bibtex once. This makes the release alpha quality. + +* --compress will compress temporary files. If the output is big and + very compressible this will take up less disk space in $TMPDIR and + possibly be faster due to less disk I/O. + +* --compress-program comtrols which program to use for compressing + temporary files. + +* --bar show progress as a progress bar compatible with zenity. + +* --resume can now be used with --result: Jobs already run will be + skipped. + +* --transfer and --basefile support paths relative to the --workdir by + inserting /./ into the path. + +* GNU Parallel was used (unfortunately with improper citation) in: + 'fastphylo: Fast tools for phylogenetics' + http://www.biomedcentral.com/1471-2105/14/334/abstract + +* Using GNU parallel + http://davetang.org/muse/2013/11/18/using-gnu-parallel/ + +* Techlux - GNU - Parallel (German) + https://techlux.de/blog/2013/11/07/gnu-parallel/ + +* awk, sed, bzip2, grep, wc на всех ядрах + http://vk.com/page-30666517_45528467 + +* 如何利用多核CPU來加速你的Linux命令 — awk, sed, bzip2, grep, wc等 + http://www.hksilicon.com/kb/articles/290543/CPULinuxawk-sed-bzip2-grep-wc + +* GNU Parallel (Japanese) + http://jarp.does.notwork.org/diary/201311b.html#20131117 + +* Bug fixes and man page updates. + + 20131022 * --transfer files with /./ in the path will copy the files relative diff --git a/doc/release_new_version b/doc/release_new_version index 387e6547..7ffa8786 100644 --- a/doc/release_new_version +++ b/doc/release_new_version @@ -208,6 +208,25 @@ available for download at: http://ftp.gnu.org/gnu/parallel/ New in this release: +* A citation notice is printed on stderr only if stderr is a terminal, + the user has not specified --no-notice and the user has not run + --bibtex once. This makes the release alpha quality. + +* --compress will compress temporary files. If the output is big and + very compressible this will take up less disk space in $TMPDIR and + possibly be faster due to less disk I/O. + +* --compress-program comtrols which program to use for compressing + temporary files. + +* --bar show progress as a progress bar compatible with zenity. + +* --resume can now be used with --result: Jobs already run will be + skipped. + +* --transfer and --basefile support paths relative to the --workdir by + inserting /./ into the path. + * GNU Parallel was used (unfortunately with improper citation) in: 'fastphylo: Fast tools for phylogenetics' http://www.biomedcentral.com/1471-2105/14/334/abstract diff --git a/src/parallel.pdf b/src/parallel.pdf index 4122d3a1..21052a9f 100644 Binary files a/src/parallel.pdf and b/src/parallel.pdf differ diff --git a/src/parallel.pod b/src/parallel.pod index 61aae6d4..7a3ff917 100644 --- a/src/parallel.pod +++ b/src/parallel.pod @@ -301,6 +301,16 @@ string that is not in the command line. See also: B<:::>. +=item B<--bar> (alpha testing) + +Show progress as a progress bar. In the bar is shown: % of jobs +completed, estimated seconds left, and number of jobs started. + +It is compatible with B: + +seq 1000 | parallel -j30 --bar '(echo {};sleep 0.1)' 2> >(zenity --progress --auto-kill) | wc + + =item B<--basefile> I (alpha testing) =item B<--bf> I (alpha testing) @@ -338,9 +348,10 @@ See also: B<--fg>, B. Implies B<--semaphore>. -=item B<--bibtex> +=item B<--bibtex> (alpha testing) -Print the BibTeX entry for GNU B. +Print the BibTeX entry for GNU B and disable citation +notice. =item B<--block> I @@ -395,7 +406,7 @@ I is a Perl Regular Expression: http://perldoc.perl.org/perlre.html -=item B<--compress> (pre-alpha testing) +=item B<--compress> (alpha testing) Compress temporary files. If the output is big and very compressible this will take up less disk space in $TMPDIR and possibly be faster due to less @@ -406,7 +417,7 @@ B, B, B, B, B in that order, and use the first available. -=item B<--compress-program> I (pre-alpha testing) +=item B<--compress-program> I (alpha testing) Use I for compressing temporary files. It is assumed that I will decompress stdin (standard input) to stdout (standard @@ -597,7 +608,7 @@ specified, and for B<-I>{} otherwise. This option is deprecated; use B<-I> instead. -=item B<--joblog> I (beta testing) +=item B<--joblog> I (alpha testing) Logfile for executed jobs. Save a list of the executed jobs to I in the following TAB separated format: sequence number, @@ -792,7 +803,7 @@ This is useful for scripts that depend on features only available from a certain version of GNU B. -=item B<--nonall> (beta testing) +=item B<--nonall> B<--onall> with no arguments. Run the command on all computers given with B<--sshlogin> but take no arguments. GNU B will log @@ -803,7 +814,7 @@ This is useful for running the same command (e.g. uptime) on a list of servers. -=item B<--onall> (beta testing) +=item B<--onall> Run all the jobs on all computers given with B<--sshlogin>. GNU B will log into B<--jobs> number of computers in parallel @@ -815,11 +826,11 @@ When using B<--group> the output will be grouped by each server, so all the output from one server will be grouped together. -=item B<--output-as-files> (beta testing) +=item B<--output-as-files> (alpha testing) -=item B<--outputasfiles> (beta testing) +=item B<--outputasfiles> (alpha testing) -=item B<--files> (beta testing) +=item B<--files> (alpha testing) Instead of printing the output to stdout (standard output) the output of each job is saved in a file and the filename is then printed. @@ -932,6 +943,14 @@ Print the number of CPU cores and exit (used by GNU B itself to determine the number of CPU cores on remote computers). +=item B<--no-notice> (alpha testing) + +Do not display citation notice. A citation notice is printed on stderr +(standard error) only if stderr (standard error) is a terminal, the +user has not specified B<--no-notice>, and the user has not run +B<--bibtex> once. + + =item B<--nice> I Run the command at this niceness. For simple commands you can just add @@ -1077,15 +1096,16 @@ will generate the files: See also B<--files>, B<--header>, B<--joblog>. -=item B<--resume> +=item B<--resume> (alpha testing) -Resumes from the last unfinished job. By reading B<--joblog> GNU -B will figure out the last unfinished job and continue from -there. As GNU B only looks at the sequence numbers in -B<--joblog> then the input, the command, and B<--joblog> all have to -remain unchanged; otherwise GNU B may run wrong commands. +Resumes from the last unfinished job. By reading B<--joblog> or the +B<--results> dir GNU B will figure out the last unfinished +job and continue from there. As GNU B only looks at the +sequence numbers in B<--joblog> then the input, the command, and +B<--joblog> all have to remain unchanged; otherwise GNU B +may run wrong commands. -See also B<--joblog>, B<--resume-failed>. +See also B<--joblog>, B<--results>, B<--resume-failed>. =item B<--resume-failed> @@ -1466,9 +1486,10 @@ See also B<-v>, B<-p>. =item B<--transfer> (alpha testing) Transfer files to remote computers. B<--transfer> is used with -B<--sshlogin> when the arguments are files and should be transferred to -the remote computers. The files will be transferred using B and -will be put relative to the default login dir. E.g. +B<--sshlogin> when the arguments are files and should be transferred +to the remote computers. The files will be transferred using B +and will be put relative to the default work dir. If the path contains +/./ the remaining path will be relative to the work dir. E.g. echo foo/bar.txt | parallel \ --sshlogin server.example.com --transfer wc @@ -1575,9 +1596,9 @@ Use B<-v> B<-v> to print the wrapping ssh command when running remotely. Print the version GNU B and exit. -=item B<--workdir> I +=item B<--workdir> I (alpha testing) -=item B<--wd> I +=item B<--wd> I (alpha testing) Files transferred using B<--transfer> and B<--return> will be relative to I on remote computers, and the command will be executed in diff --git a/src/parallel.texi b/src/parallel.texi index c696995c..8814faf5 100644 --- a/src/parallel.texi +++ b/src/parallel.texi @@ -317,6 +317,16 @@ string that is not in the command line. See also: @strong{:::}. +@item @strong{--bar} (alpha testing) +@anchor{@strong{--bar} (alpha testing)} + +Show progress as a progress bar. In the bar is shown: % of jobs +completed, estimated seconds left, and number of jobs started. + +It is compatible with @strong{zenity}: + +seq 1000 | parallel -j30 --bar '(echo @{@};sleep 0.1)' 2> >(zenity --progress --auto-kill) | wc + @item @strong{--basefile} @emph{file} (alpha testing) @anchor{@strong{--basefile} @emph{file} (alpha testing)} @@ -357,10 +367,11 @@ See also: @strong{--fg}, @strong{man sem}. Implies @strong{--semaphore}. -@item @strong{--bibtex} -@anchor{@strong{--bibtex}} +@item @strong{--bibtex} (alpha testing) +@anchor{@strong{--bibtex} (alpha testing)} -Print the BibTeX entry for GNU @strong{parallel}. +Print the BibTeX entry for GNU @strong{parallel} and disable citation +notice. @item @strong{--block} @emph{size} @anchor{@strong{--block} @emph{size}} @@ -418,8 +429,8 @@ separating the columns. The n'th column can be access using @emph{regexp} is a Perl Regular Expression: http://perldoc.perl.org/perlre.html -@item @strong{--compress} (pre-alpha testing) -@anchor{@strong{--compress} (pre-alpha testing)} +@item @strong{--compress} (alpha testing) +@anchor{@strong{--compress} (alpha testing)} Compress temporary files. If the output is big and very compressible this will take up less disk space in $TMPDIR and possibly be faster due to less @@ -429,8 +440,8 @@ GNU @strong{parallel} will try @strong{lzop}, @strong{pigz}, @strong{gzip}, @str @strong{plzip}, @strong{bzip2}, @strong{lzma}, @strong{lzip}, @strong{xz} in that order, and use the first available. -@item @strong{--compress-program} @emph{prg} (pre-alpha testing) -@anchor{@strong{--compress-program} @emph{prg} (pre-alpha testing)} +@item @strong{--compress-program} @emph{prg} (alpha testing) +@anchor{@strong{--compress-program} @emph{prg} (alpha testing)} Use @emph{prg} for compressing temporary files. It is assumed that @emph{prg -dc} will decompress stdin (standard input) to stdout (standard @@ -629,8 +640,8 @@ This option is a synonym for @strong{-I}@emph{replace-str} if @emph{replace-str} specified, and for @strong{-I}@{@} otherwise. This option is deprecated; use @strong{-I} instead. -@item @strong{--joblog} @emph{logfile} (beta testing) -@anchor{@strong{--joblog} @emph{logfile} (beta testing)} +@item @strong{--joblog} @emph{logfile} (alpha testing) +@anchor{@strong{--joblog} @emph{logfile} (alpha testing)} Logfile for executed jobs. Save a list of the executed jobs to @emph{logfile} in the following TAB separated format: sequence number, @@ -845,8 +856,8 @@ GNU @strong{parallel} is less than @emph{version} the exit code is This is useful for scripts that depend on features only available from a certain version of GNU @strong{parallel}. -@item @strong{--nonall} (beta testing) -@anchor{@strong{--nonall} (beta testing)} +@item @strong{--nonall} +@anchor{@strong{--nonall}} @strong{--onall} with no arguments. Run the command on all computers given with @strong{--sshlogin} but take no arguments. GNU @strong{parallel} will log @@ -856,8 +867,8 @@ computer. @strong{-j} adjusts how many computers to log into in parallel. This is useful for running the same command (e.g. uptime) on a list of servers. -@item @strong{--onall} (beta testing) -@anchor{@strong{--onall} (beta testing)} +@item @strong{--onall} +@anchor{@strong{--onall}} Run all the jobs on all computers given with @strong{--sshlogin}. GNU @strong{parallel} will log into @strong{--jobs} number of computers in parallel @@ -868,14 +879,14 @@ adjusts how many computers to log into in parallel. When using @strong{--group} the output will be grouped by each server, so all the output from one server will be grouped together. -@item @strong{--output-as-files} (beta testing) -@anchor{@strong{--output-as-files} (beta testing)} +@item @strong{--output-as-files} (alpha testing) +@anchor{@strong{--output-as-files} (alpha testing)} -@item @strong{--outputasfiles} (beta testing) -@anchor{@strong{--outputasfiles} (beta testing)} +@item @strong{--outputasfiles} (alpha testing) +@anchor{@strong{--outputasfiles} (alpha testing)} -@item @strong{--files} (beta testing) -@anchor{@strong{--files} (beta testing)} +@item @strong{--files} (alpha testing) +@anchor{@strong{--files} (alpha testing)} Instead of printing the output to stdout (standard output) the output of each job is saved in a file and the filename is then printed. @@ -990,6 +1001,14 @@ itself to determine the number of physical CPUs on remote computers). Print the number of CPU cores and exit (used by GNU @strong{parallel} itself to determine the number of CPU cores on remote computers). +@item @strong{--no-notice} (alpha testing) +@anchor{@strong{--no-notice} (alpha testing)} + +Do not display citing notice. A citation notice is printed on stderr +(standard error) only if stderr (standard error) is a terminal, the +user has not specified @strong{--no-notice} and the user has not run --bibtex +once. + @item @strong{--nice} @emph{niceness} @anchor{@strong{--nice} @emph{niceness}} @@ -1151,16 +1170,17 @@ will generate the files: See also @strong{--files}, @strong{--header}, @strong{--joblog}. -@item @strong{--resume} -@anchor{@strong{--resume}} +@item @strong{--resume} (alpha testing) +@anchor{@strong{--resume} (alpha testing)} -Resumes from the last unfinished job. By reading @strong{--joblog} GNU -@strong{parallel} will figure out the last unfinished job and continue from -there. As GNU @strong{parallel} only looks at the sequence numbers in -@strong{--joblog} then the input, the command, and @strong{--joblog} all have to -remain unchanged; otherwise GNU @strong{parallel} may run wrong commands. +Resumes from the last unfinished job. By reading @strong{--joblog} or the +@strong{--results} dir GNU @strong{parallel} will figure out the last unfinished +job and continue from there. As GNU @strong{parallel} only looks at the +sequence numbers in @strong{--joblog} then the input, the command, and +@strong{--joblog} all have to remain unchanged; otherwise GNU @strong{parallel} +may run wrong commands. -See also @strong{--joblog}, @strong{--resume-failed}. +See also @strong{--joblog}, @strong{--results}, @strong{--resume-failed}. @item @strong{--resume-failed} @anchor{@strong{--resume-failed}} @@ -1572,9 +1592,10 @@ See also @strong{-v}, @strong{-p}. @anchor{@strong{--transfer} (alpha testing)} Transfer files to remote computers. @strong{--transfer} is used with -@strong{--sshlogin} when the arguments are files and should be transferred to -the remote computers. The files will be transferred using @strong{rsync} and -will be put relative to the default login dir. E.g. +@strong{--sshlogin} when the arguments are files and should be transferred +to the remote computers. The files will be transferred using @strong{rsync} +and will be put relative to the default work dir. If the path contains +/./ the remaining path will be relative to the work dir. E.g. @verbatim echo foo/bar.txt | parallel \ @@ -1691,11 +1712,11 @@ Use @strong{-v} @strong{-v} to print the wrapping ssh command when running remot Print the version GNU @strong{parallel} and exit. -@item @strong{--workdir} @emph{mydir} -@anchor{@strong{--workdir} @emph{mydir}} +@item @strong{--workdir} @emph{mydir} (alpha testing) +@anchor{@strong{--workdir} @emph{mydir} (alpha testing)} -@item @strong{--wd} @emph{mydir} -@anchor{@strong{--wd} @emph{mydir}} +@item @strong{--wd} @emph{mydir} (alpha testing) +@anchor{@strong{--wd} @emph{mydir} (alpha testing)} Files transferred using @strong{--transfer} and @strong{--return} will be relative to @emph{mydir} on remote computers, and the command will be executed in