Makefile: If pod2pdf is installed: Make .pdf files.

This commit is contained in:
Ole Tange 2013-09-18 02:00:10 +02:00
parent aaac155ae2
commit 42ea8b4880
9 changed files with 166 additions and 149 deletions

12
README
View file

@ -30,11 +30,7 @@ The 10 seconds installation will try do to a full installation; if
that fails, a personal installation; if that fails, a minimal that fails, a personal installation; if that fails, a minimal
installation. installation.
wget -O - pi.dk/3 | bash (wget -O - pi.dk/3 || curl pi.dk/3/) | bash
or:
curl pi.dk/3/ | bash
This will literally install faster than reading the rest of this This will literally install faster than reading the rest of this
document. document.
@ -85,8 +81,10 @@ This will send 3 ping packets to 3 different hosts in parallel and print
the output when they complete. the output when they complete.
Watch the intro video for a quick introduction: Watch the intro video for a quick introduction:
https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1 or at https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1
http://tinyogg.com/watch/TORaR/ and http://tinyogg.com/watch/hfxKj/
Walk through the tutorial (man parallel_tutorial). You command line
with love you for it.
= New versions = = New versions =

View file

@ -106,6 +106,7 @@ echo put parallel-latest.tar.bz2{,.sig,*asc} | ncftp ftp://ftp-upload.gnu.org/in
cd ~/privat/parallel/packager/obs cd ~/privat/parallel/packager/obs
# Update version number # Update version number
em home:tange/parallel/parallel.spec em home:tange/parallel/parallel.spec
find home:tange/parallel/* -type f | grep -v parallel.spec | parallel -Xj1 osc rm {} find home:tange/parallel/* -type f | grep -v parallel.spec | parallel -Xj1 osc rm {}
# This should not create new files # This should not create new files
osc up home:tange/parallel/ osc up home:tange/parallel/
@ -150,6 +151,7 @@ pushd
== Commit released version == == Commit released version ==
git diff git diff
git commit -a git commit -a
== Update Savannah == == Update Savannah ==
@ -166,8 +168,8 @@ http://freshmeat.net/projects/parallel/releases/new
== Update Diaspora Twitter == == Update Diaspora Twitter ==
New release of #GNU Parallel pi.dk/0 New in this release pi.dk/2 See the intro videos pi.dk/1 New release of #GNU Parallel pidk/0 New in this release pidk/2 See the intro videos pidk/1
10 secs installation: wget -O - pi.dk/3|sh 10 secs installation: wget -O - pidk/3|bash
[x] Twitter [x] Twitter
Aspect: Public Aspect: Public
@ -199,74 +201,24 @@ cc:Sandro Cazzaniga <kharec@mandriva.org>,
Ryoichiro Suzuki <ryoichiro.suzuki@gmail.com>, Ryoichiro Suzuki <ryoichiro.suzuki@gmail.com>,
Jesse Alama <jesse.alama@gmail.com> Jesse Alama <jesse.alama@gmail.com>
Subject: GNU Parallel 20130822 ('Compostela') released Subject: GNU Parallel 20130922 ('') released
GNU Parallel 20130822 ('Compostela') has been released. It is GNU Parallel 20130922 ('') has been released. It is
available for download at: http://ftp.gnu.org/gnu/parallel/ available for download at: http://ftp.gnu.org/gnu/parallel/
It includes a big code reorganization making this release beta
quality.
New in this release: New in this release:
* A tutorial showing much of GNU Parallel's functionality. Spend an * Cloning at warp speed 100 VMs in 1 minute using one command line
hour walking through the tutorial. Your commandline will love you http://www.beebotech.com.au/tag/command-line/
for it. man parallel_tutorial
* --line-buffer will buffer output on line basis. --group keeps the * Calcul Quebec's introduction to GNU Parallel
output together for a whole job. --ungroup allows output to mixup https://wiki.calculquebec.ca/w/GNU_parallel
with half a line coming from one job and half a line coming from
another job. --line-buffer fits between these two; it prints a full
line, but will allow for mixing lines of different jobs.
* --record-env records all variables and functions set. Useful to * Brincando com o GNU parallel
record the normal environment before using --env _. http://blog.tadeucruz.com/2013/09/brincando-com-o-gnu-parallel/
* --env _ exports all functions and variables to the remote server * GNU Parallelで並列処理を行う
except for the ones recorded. http://qiita.com/PENGUINANA_/items/af27a833e835fe17f09b
* New signing key. Due to recommendations from NIST
http://www.keylength.com/en/4/ the signing key was changed from
1024D/ID:FFFFFFF1 to 9888R/ID:88888888.
* Agalma: an automated phylogenomics workflow
http://arxiv.org/pdf/1307.6432
* Job ad asking for GNU Parallel expertise
http://seattle.craigslist.org/est/sof/4006079352.html
* Transient Beowulf Clustering with GNU Parallel and SSHfs
http://www.reddit.com/r/linux/comments/1ka8mn/transient_beowulf_clustering_with_gnu_parallel/
* Aligning to unique regions
http://davetang.org/muse/2013/07/22/aligning-to-unique-regions/
* Top 10 scripting tricks for basic bioinformatics
http://students.washington.edu/bowmanjs/wordpress/?p=873
* Using GNU Parallel to speed up and simplify data analyzes
http://www.sergeymarkov.com/blog/2013/07/using-gnu-parallel-to-speed-up-and-simplify-data-analyzes/
* Compression of files in parallel using GNU parallel
http://codextechnicanum.blogspot.dk/2013/07/compression-of-files-in-parallel-using.html
* Using GNU Parallel to roll-your-own Map Reduce!
http://www.rankfocus.com/?p=1
* Using GNU Parallel with Amazon
media.amazonwebservices.com/AWS_Amazon_EMR_Best_Practices.pdf
* Some useful comments on GNU Parallel
https://news.ycombinator.com/item?id=6209767
* Using GNU Parallel to count reads mapping to genes for multiple BAMs
http://drduanehassane.com/blog/sequencing-resources
* TimeMachineっぽいバックアップスクリプト
http://rio.tc/2013/07/timemachine-1.html
* GNU ParallelでAndroid NDKの全バージョンを一括ダウンロード
http://qiita.com/mazgi/items/b10bf0ff3da2045d19ab
* Bug fixes and man page updates. * Bug fixes and man page updates.
@ -298,8 +250,10 @@ You can install GNU Parallel in just 10 seconds with:
wget -O - pi.dk/3 | sh wget -O - pi.dk/3 | sh
Watch the intro video on Watch the intro video on
http://www.youtube.com/playlist?list=PL284C9FF2488BC6D1 or at http://www.youtube.com/playlist?list=PL284C9FF2488BC6D1
http://tinyogg.com/watch/TORaR/ and http://tinyogg.com/watch/hfxKj/
Walk through the tutorial (man parallel_tutorial). Your commandline
will love you for it.
When using GNU Parallel for a publication please cite: When using GNU Parallel for a publication please cite:

View file

@ -56,13 +56,20 @@ make install prefix=$RPM_BUILD_ROOT%{_prefix} exec_prefix=$RPM_BUILD_ROOT%{_pref
mandir=$RPM_BUILD_ROOT%{_mandir} mandir=$RPM_BUILD_ROOT%{_mandir}
rm $RPM_BUILD_ROOT%{_docdir}/parallel.html rm $RPM_BUILD_ROOT%{_docdir}/parallel.html
rm $RPM_BUILD_ROOT%{_docdir}/parallel_tutorial.html
rm $RPM_BUILD_ROOT%{_docdir}/niceload.html rm $RPM_BUILD_ROOT%{_docdir}/niceload.html
rm $RPM_BUILD_ROOT%{_docdir}/sem.html rm $RPM_BUILD_ROOT%{_docdir}/sem.html
rm $RPM_BUILD_ROOT%{_docdir}/sql.html rm $RPM_BUILD_ROOT%{_docdir}/sql.html
rm $RPM_BUILD_ROOT%{_docdir}/parallel.texi rm $RPM_BUILD_ROOT%{_docdir}/parallel.texi
rm $RPM_BUILD_ROOT%{_docdir}/parallel_tutorial.texi
rm $RPM_BUILD_ROOT%{_docdir}/niceload.texi rm $RPM_BUILD_ROOT%{_docdir}/niceload.texi
rm $RPM_BUILD_ROOT%{_docdir}/sem.texi rm $RPM_BUILD_ROOT%{_docdir}/sem.texi
rm $RPM_BUILD_ROOT%{_docdir}/sql.texi rm $RPM_BUILD_ROOT%{_docdir}/sql.texi
rm $RPM_BUILD_ROOT%{_docdir}/parallel.pdf
rm $RPM_BUILD_ROOT%{_docdir}/parallel_tutorial.pdf
rm $RPM_BUILD_ROOT%{_docdir}/niceload.pdf
rm $RPM_BUILD_ROOT%{_docdir}/sem.pdf
rm $RPM_BUILD_ROOT%{_docdir}/sql.pdf
%clean %clean
rm -rf $RPM_BUILD_ROOT rm -rf $RPM_BUILD_ROOT
@ -71,7 +78,7 @@ rm -rf $RPM_BUILD_ROOT
%defattr(-,root,root,-) %defattr(-,root,root,-)
/usr/bin/* /usr/bin/*
/usr/share/man/man1/* /usr/share/man/man1/*
%doc README NEWS src/parallel.html src/sem.html src/sql.html src/niceload.html %doc README NEWS src/parallel.html src/parallel_tutorial.html src/sem.html src/sql.html src/niceload.html
%changelog %changelog
* Sat Jan 22 2011 Ole Tange * Sat Jan 22 2011 Ole Tange

View file

@ -1,6 +1,6 @@
bin_SCRIPTS = parallel sem sql niceload bin_SCRIPTS = parallel sem sql niceload
man_MANS = parallel.1 sem.1 sql.1 niceload.1 parallel_tutorial.1 man_MANS = parallel.1 sem.1 sql.1 niceload.1 parallel_tutorial.1
doc_DATA = parallel.html sem.html sql.html niceload.html parallel_tutorial.html parallel.texi sem.texi sql.texi niceload.texi parallel_tutorial.texi doc_DATA = parallel.html sem.html sql.html niceload.html parallel_tutorial.html parallel.texi sem.texi sql.texi niceload.texi parallel_tutorial.texi parallel.pdf sem.pdf sql.pdf niceload.pdf parallel_tutorial.pdf
parallel.1: parallel.pod Makefile parallel.1: parallel.pod Makefile
pod2man --release='$(PACKAGE_VERSION)' --center='$(PACKAGE_NAME)' \ pod2man --release='$(PACKAGE_VERSION)' --center='$(PACKAGE_NAME)' \
@ -66,12 +66,33 @@ niceload.texi: niceload.pod
# If pod2texi is not installed: Forget about it # If pod2texi is not installed: Forget about it
pod2texi --output=$(srcdir)/niceload.texi $(srcdir)/niceload.pod || true pod2texi --output=$(srcdir)/niceload.texi $(srcdir)/niceload.pod || true
parallel.pdf: parallel.pod
# If pod2pdf is not installed: Forget about it
pod2pdf --output-file $(srcdir)/parallel.pdf $(srcdir)/parallel.pod || true
parallel_tutorial.pdf: parallel_tutorial.pod
# If pod2pdf is not installed: Forget about it
pod2pdf --output-file $(srcdir)/parallel_tutorial.pdf $(srcdir)/parallel_tutorial.pod || true
sem.pdf: sem.pod
# If pod2pdf is not installed: Forget about it
pod2pdf --output-file $(srcdir)/sem.pdf $(srcdir)/sem.pod || true
sql.pdf: sql
# If pod2pdf is not installed: Forget about it
pod2pdf --output-file $(srcdir)/sql.pdf $(srcdir)/sql || true
niceload.pdf: niceload.pod
# If pod2pdf is not installed: Forget about it
pod2pdf --output-file $(srcdir)/niceload.pdf $(srcdir)/niceload.pod || true
sem: parallel sem: parallel
ln -fs parallel sem ln -fs parallel sem
DISTCLEANFILES = parallel.html sem.html sql.html niceload.html parallel_tutorial.html DISTCLEANFILES = parallel.html sem.html sql.html niceload.html parallel_tutorial.html
EXTRA_DIST = parallel sem sql niceload \ EXTRA_DIST = parallel sem sql niceload \
parallel.1 sem.1 sql.1 niceload.1 parallel_tutorial.1 \ parallel.1 sem.1 sql.1 niceload.1 parallel_tutorial.1 \
parallel.html sem.html sql.html niceload.html parallel_tutorial.1 \ parallel.html sem.html sql.html niceload.html parallel_tutorial.html \
sem.pod parallel.pod niceload.pod parallel_tutorial.pod \ sem.pod parallel.pod niceload.pod parallel_tutorial.pod \
parallel.texi sem.texi sql.texi niceload.texi parallel_tutorial.texi parallel.texi sem.texi sql.texi niceload.texi parallel_tutorial.texi \
parallel.pdf sem.pdf sql.pdf niceload.pdf parallel_tutorial.pdf

View file

@ -176,13 +176,14 @@ top_builddir = @top_builddir@
top_srcdir = @top_srcdir@ top_srcdir = @top_srcdir@
bin_SCRIPTS = parallel sem sql niceload bin_SCRIPTS = parallel sem sql niceload
man_MANS = parallel.1 sem.1 sql.1 niceload.1 parallel_tutorial.1 man_MANS = parallel.1 sem.1 sql.1 niceload.1 parallel_tutorial.1
doc_DATA = parallel.html sem.html sql.html niceload.html parallel_tutorial.html parallel.texi sem.texi sql.texi niceload.texi parallel_tutorial.texi doc_DATA = parallel.html sem.html sql.html niceload.html parallel_tutorial.html parallel.texi sem.texi sql.texi niceload.texi parallel_tutorial.texi parallel.pdf sem.pdf sql.pdf niceload.pdf parallel_tutorial.pdf
DISTCLEANFILES = parallel.html sem.html sql.html niceload.html parallel_tutorial.html DISTCLEANFILES = parallel.html sem.html sql.html niceload.html parallel_tutorial.html
EXTRA_DIST = parallel sem sql niceload \ EXTRA_DIST = parallel sem sql niceload \
parallel.1 sem.1 sql.1 niceload.1 parallel_tutorial.1 \ parallel.1 sem.1 sql.1 niceload.1 parallel_tutorial.1 \
parallel.html sem.html sql.html niceload.html parallel_tutorial.1 \ parallel.html sem.html sql.html niceload.html parallel_tutorial.html \
sem.pod parallel.pod niceload.pod parallel_tutorial.pod \ sem.pod parallel.pod niceload.pod parallel_tutorial.pod \
parallel.texi sem.texi sql.texi niceload.texi parallel_tutorial.texi parallel.texi sem.texi sql.texi niceload.texi parallel_tutorial.texi \
parallel.pdf sem.pdf sql.pdf niceload.pdf parallel_tutorial.pdf
all: all-am all: all-am
@ -552,6 +553,26 @@ niceload.texi: niceload.pod
# If pod2texi is not installed: Forget about it # If pod2texi is not installed: Forget about it
pod2texi --output=$(srcdir)/niceload.texi $(srcdir)/niceload.pod || true pod2texi --output=$(srcdir)/niceload.texi $(srcdir)/niceload.pod || true
parallel.pdf: parallel.pod
# If pod2pdf is not installed: Forget about it
pod2pdf --output-file $(srcdir)/parallel.pdf $(srcdir)/parallel.pod || true
parallel_tutorial.pdf: parallel_tutorial.pod
# If pod2pdf is not installed: Forget about it
pod2pdf --output-file $(srcdir)/parallel_tutorial.pdf $(srcdir)/parallel_tutorial.pod || true
sem.pdf: sem.pod
# If pod2pdf is not installed: Forget about it
pod2pdf --output-file $(srcdir)/sem.pdf $(srcdir)/sem.pod || true
sql.pdf: sql
# If pod2pdf is not installed: Forget about it
pod2pdf --output-file $(srcdir)/sql.pdf $(srcdir)/sql || true
niceload.pdf: niceload.pod
# If pod2pdf is not installed: Forget about it
pod2pdf --output-file $(srcdir)/niceload.pdf $(srcdir)/niceload.pod || true
sem: parallel sem: parallel
ln -fs parallel sem ln -fs parallel sem

View file

@ -765,7 +765,7 @@ sub get_options_from_array {
sub parse_options { sub parse_options {
# Returns: N/A # Returns: N/A
# Defaults: # Defaults:
$Global::version = 20130822; $Global::version = 20130918;
$Global::progname = 'parallel'; $Global::progname = 'parallel';
$Global::infinity = 2**31; $Global::infinity = 2**31;
$Global::debug = 0; $Global::debug = 0;

View file

@ -124,7 +124,7 @@
.\" ======================================================================== .\" ========================================================================
.\" .\"
.IX Title "PARALLEL_TUTORIAL 1" .IX Title "PARALLEL_TUTORIAL 1"
.TH PARALLEL_TUTORIAL 1 "2013-08-22" "20130822" "parallel" .TH PARALLEL_TUTORIAL 1 "2013-09-18" "20130822" "parallel"
.\" For nroff, turn off justification. Always turn off hyphenation; it makes .\" For nroff, turn off justification. Always turn off hyphenation; it makes
.\" way too many mistakes in technical documents. .\" way too many mistakes in technical documents.
.if n .ad l .if n .ad l
@ -135,70 +135,80 @@ This tutorial shows off much of \s-1GNU\s0 Parallel's functionality. The
tutorial is meant to learn the options in \s-1GNU\s0 Parallel. The tutorial tutorial is meant to learn the options in \s-1GNU\s0 Parallel. The tutorial
is not to show realistic examples from the real world. is not to show realistic examples from the real world.
.PP .PP
Spend an hour walking through the tutorial. Your commandline will Spend an hour walking through the tutorial. Your command line will
love you for it. love you for it.
.SH "Prerequisites" .SH "Prerequisites"
.IX Header "Prerequisites" .IX Header "Prerequisites"
To run this tutorial you must have the following: To run this tutorial you must have the following:
.IP "parallel >= version 20130814" 9 .IP "parallel >= version 20130814" 9
.IX Item "parallel >= version 20130814" .IX Item "parallel >= version 20130814"
Install the newest version with:
.Sp
.Vb 1 .Vb 1
\& Most of the tutorial will work on older versions, too. \& (wget \-O \- pi.dk/3 || curl pi.dk/3/) | bash
.Ve .Ve
.Sp
This will also install the newest version of the tutorial:
.Sp
.Vb 1
\& man parallel_tutorial
.Ve
.Sp
Most of the tutorial will work on older versions, too.
.IP "abc-file:" 9 .IP "abc-file:" 9
.IX Item "abc-file:" .IX Item "abc-file:"
The file can be generated by:
.Sp
.Vb 1 .Vb 1
\& The file can be generated by:
\&
\& parallel \-k echo ::: A B C > abc\-file \& parallel \-k echo ::: A B C > abc\-file
.Ve .Ve
.IP "def-file:" 9 .IP "def-file:" 9
.IX Item "def-file:" .IX Item "def-file:"
The file can be generated by:
.Sp
.Vb 1 .Vb 1
\& The file can be generated by:
\&
\& parallel \-k echo ::: D E F > def\-file \& parallel \-k echo ::: D E F > def\-file
.Ve .Ve
.IP "abc0\-file:" 9 .IP "abc0\-file:" 9
.IX Item "abc0-file:" .IX Item "abc0-file:"
The file can be generated by:
.Sp
.Vb 1 .Vb 1
\& The file can be generated by:
\&
\& perl \-e \*(Aqprintf "A\e0B\e0C\e0"\*(Aq > abc0\-file \& perl \-e \*(Aqprintf "A\e0B\e0C\e0"\*(Aq > abc0\-file
.Ve .Ve
.IP "abc_\-file:" 9 .IP "abc_\-file:" 9
.IX Item "abc_-file:" .IX Item "abc_-file:"
The file can be generated by:
.Sp
.Vb 1 .Vb 1
\& The file can be generated by: \& perl \-e \*(Aqprintf "A_B_C_"\*(Aq > abc_\-file
\&
\& perl \-e \*(Aqprintf "A_B_C_%s"\*(Aq > abc_\-file
.Ve .Ve
.IP "tsv\-file.tsv" 9 .IP "tsv\-file.tsv" 9
.IX Item "tsv-file.tsv" .IX Item "tsv-file.tsv"
The file can be generated by:
.Sp
.Vb 1 .Vb 1
\& The file can be generated by:
\&
\& perl \-e \*(Aqprintf "f1\etf2\enA\etB\enC\etD\en"\*(Aq > tsv\-file.tsv \& perl \-e \*(Aqprintf "f1\etf2\enA\etB\enC\etD\en"\*(Aq > tsv\-file.tsv
.Ve .Ve
.IP "num30000" 9 .IP "num30000" 9
.IX Item "num30000" .IX Item "num30000"
The file can be generated by:
.Sp
.Vb 1 .Vb 1
\& The file can be generated by:
\&
\& perl \-e \*(Aqfor(1..30000){print "$_\en"}\*(Aq > num30000 \& perl \-e \*(Aqfor(1..30000){print "$_\en"}\*(Aq > num30000
.Ve .Ve
.IP "num1000000" 9 .IP "num1000000" 9
.IX Item "num1000000" .IX Item "num1000000"
The file can be generated by:
.Sp
.Vb 1 .Vb 1
\& The file can be generated by:
\&
\& perl \-e \*(Aqfor(1..1000000){print "$_\en"}\*(Aq > num1000000 \& perl \-e \*(Aqfor(1..1000000){print "$_\en"}\*(Aq > num1000000
.Ve .Ve
.IP "num_%header" 9 .IP "num_%header" 9
.IX Item "num_%header" .IX Item "num_%header"
The file can be generated by:
.Sp
.Vb 1 .Vb 1
\& The file can be generated by:
\&
\& (echo %head1; echo %head2; perl \-e \*(Aqfor(1..10){print "$_\en"}\*(Aq) > num_%header \& (echo %head1; echo %head2; perl \-e \*(Aqfor(1..10){print "$_\en"}\*(Aq) > num_%header
.Ve .Ve
.ie n .IP "For remote running: ssh login on 2 servers with no password in $SERVER1 and $SERVER2" 9 .ie n .IP "For remote running: ssh login on 2 servers with no password in $SERVER1 and $SERVER2" 9
@ -207,15 +217,17 @@ To run this tutorial you must have the following:
.Vb 2 .Vb 2
\& SERVER1=server.example.com \& SERVER1=server.example.com
\& SERVER2=server2.example.net \& SERVER2=server2.example.net
\& .Ve
\& You must be able to: .Sp
\& You must be able to:
.Sp
.Vb 2
\& ssh $SERVER1 echo works \& ssh $SERVER1 echo works
\& ssh $SERVER2 echo works \& ssh $SERVER2 echo works
\&
\& It can be setup by running \*(Aqssh\-keygen \-t dsa; ssh\-copy\-id $SERVER1\*(Aq
\& and using an empty pass phrase.
.Ve .Ve
.Sp
It can be setup by running 'ssh\-keygen \-t dsa; ssh-copy-id \f(CW$SERVER1\fR'
and using an empty pass phrase.
.SH "Input sources" .SH "Input sources"
.IX Header "Input sources" .IX Header "Input sources"
\&\s-1GNU\s0 Parallel reads input from input sources. These can be files, the \&\s-1GNU\s0 Parallel reads input from input sources. These can be files, the
@ -764,8 +776,8 @@ Output:
For better parallelism \s-1GNU\s0 Parallel can distribute the arguments For better parallelism \s-1GNU\s0 Parallel can distribute the arguments
between all the parallel jobs when end of file is met. between all the parallel jobs when end of file is met.
.PP .PP
Running 4 jobs in parallel will split the last line of arguments will Running 4 jobs in parallel will split the last line of arguments into
be split into 4 jobs resulting in a total of 5 jobs: 4 jobs resulting in a total of 5 jobs:
.PP .PP
.Vb 1 .Vb 1
\& cat num30000 | parallel \-\-jobs 4 \-m echo | wc \-l \& cat num30000 | parallel \-\-jobs 4 \-m echo | wc \-l

View file

@ -120,7 +120,7 @@
<p>This tutorial shows off much of GNU Parallel's functionality. The <p>This tutorial shows off much of GNU Parallel's functionality. The
tutorial is meant to learn the options in GNU Parallel. The tutorial tutorial is meant to learn the options in GNU Parallel. The tutorial
is not to show realistic examples from the real world.</p> is not to show realistic examples from the real world.</p>
<p>Spend an hour walking through the tutorial. Your commandline will <p>Spend an hour walking through the tutorial. Your command line will
love you for it.</p> love you for it.</p>
<p> <p>
</p> </p>
@ -131,70 +131,67 @@ love you for it.</p>
<dt><strong><a name="parallel_version_20130814" class="item">parallel &gt;= version 20130814</a></strong></dt> <dt><strong><a name="parallel_version_20130814" class="item">parallel &gt;= version 20130814</a></strong></dt>
<dd> <dd>
<p>Install the newest version with:</p>
<pre> <pre>
Most of the tutorial will work on older versions, too.</pre> (wget -O - pi.dk/3 || curl pi.dk/3/) | bash</pre>
<p>This will also install the newest version of the tutorial:</p>
<pre>
man parallel_tutorial</pre>
<p>Most of the tutorial will work on older versions, too.</p>
</dd> </dd>
<dt><strong><a name="abc_file" class="item">abc-file:</a></strong></dt> <dt><strong><a name="abc_file" class="item">abc-file:</a></strong></dt>
<dd> <dd>
<pre> <p>The file can be generated by:</p>
The file can be generated by:</pre>
<pre> <pre>
parallel -k echo ::: A B C &gt; abc-file</pre> parallel -k echo ::: A B C &gt; abc-file</pre>
</dd> </dd>
<dt><strong><a name="def_file" class="item">def-file:</a></strong></dt> <dt><strong><a name="def_file" class="item">def-file:</a></strong></dt>
<dd> <dd>
<pre> <p>The file can be generated by:</p>
The file can be generated by:</pre>
<pre> <pre>
parallel -k echo ::: D E F &gt; def-file</pre> parallel -k echo ::: D E F &gt; def-file</pre>
</dd> </dd>
<dt><strong><a name="abc0_file" class="item">abc0-file:</a></strong></dt> <dt><strong><a name="abc0_file" class="item">abc0-file:</a></strong></dt>
<dd> <dd>
<pre> <p>The file can be generated by:</p>
The file can be generated by:</pre>
<pre> <pre>
perl -e 'printf &quot;A\0B\0C\0&quot;' &gt; abc0-file</pre> perl -e 'printf &quot;A\0B\0C\0&quot;' &gt; abc0-file</pre>
</dd> </dd>
<dt><strong><a name="abc_file2" class="item">abc_-file:</a></strong></dt> <dt><strong><a name="abc_file2" class="item">abc_-file:</a></strong></dt>
<dd> <dd>
<p>The file can be generated by:</p>
<pre> <pre>
The file can be generated by:</pre> perl -e 'printf &quot;A_B_C_&quot;' &gt; abc_-file</pre>
<pre>
perl -e 'printf &quot;A_B_C_%s&quot;' &gt; abc_-file</pre>
</dd> </dd>
<dt><strong><a name="tsv_file_tsv" class="item">tsv-file.tsv</a></strong></dt> <dt><strong><a name="tsv_file_tsv" class="item">tsv-file.tsv</a></strong></dt>
<dd> <dd>
<pre> <p>The file can be generated by:</p>
The file can be generated by:</pre>
<pre> <pre>
perl -e 'printf &quot;f1\tf2\nA\tB\nC\tD\n&quot;' &gt; tsv-file.tsv</pre> perl -e 'printf &quot;f1\tf2\nA\tB\nC\tD\n&quot;' &gt; tsv-file.tsv</pre>
</dd> </dd>
<dt><strong><a name="num30000" class="item">num30000</a></strong></dt> <dt><strong><a name="num30000" class="item">num30000</a></strong></dt>
<dd> <dd>
<pre> <p>The file can be generated by:</p>
The file can be generated by:</pre>
<pre> <pre>
perl -e 'for(1..30000){print &quot;$_\n&quot;}' &gt; num30000</pre> perl -e 'for(1..30000){print &quot;$_\n&quot;}' &gt; num30000</pre>
</dd> </dd>
<dt><strong><a name="num1000000" class="item">num1000000</a></strong></dt> <dt><strong><a name="num1000000" class="item">num1000000</a></strong></dt>
<dd> <dd>
<pre> <p>The file can be generated by:</p>
The file can be generated by:</pre>
<pre> <pre>
perl -e 'for(1..1000000){print &quot;$_\n&quot;}' &gt; num1000000</pre> perl -e 'for(1..1000000){print &quot;$_\n&quot;}' &gt; num1000000</pre>
</dd> </dd>
<dt><strong><a name="num_header" class="item">num_%header</a></strong></dt> <dt><strong><a name="num_header" class="item">num_%header</a></strong></dt>
<dd> <dd>
<pre> <p>The file can be generated by:</p>
The file can be generated by:</pre>
<pre> <pre>
(echo %head1; echo %head2; perl -e 'for(1..10){print &quot;$_\n&quot;}') &gt; num_%header</pre> (echo %head1; echo %head2; perl -e 'for(1..10){print &quot;$_\n&quot;}') &gt; num_%header</pre>
</dd> </dd>
@ -205,14 +202,12 @@ $SERVER1 and $SERVER2</a></strong></dt>
<pre> <pre>
SERVER1=server.example.com SERVER1=server.example.com
SERVER2=server2.example.net</pre> SERVER2=server2.example.net</pre>
<pre> <p>You must be able to:</p>
You must be able to:</pre>
<pre> <pre>
ssh $SERVER1 echo works ssh $SERVER1 echo works
ssh $SERVER2 echo works</pre> ssh $SERVER2 echo works</pre>
<pre> <p>It can be setup by running 'ssh-keygen -t dsa; ssh-copy-id $SERVER1'
It can be setup by running 'ssh-keygen -t dsa; ssh-copy-id $SERVER1' and using an empty pass phrase.</p>
and using an empty pass phrase.</pre>
</dd> </dd>
</dl> </dl>
<p> <p>
@ -568,8 +563,8 @@ line length of 10000 chars 17 commands will be run:</p>
17</pre> 17</pre>
<p>For better parallelism GNU Parallel can distribute the arguments <p>For better parallelism GNU Parallel can distribute the arguments
between all the parallel jobs when end of file is met.</p> between all the parallel jobs when end of file is met.</p>
<p>Running 4 jobs in parallel will split the last line of arguments will <p>Running 4 jobs in parallel will split the last line of arguments into
be split into 4 jobs resulting in a total of 5 jobs:</p> 4 jobs resulting in a total of 5 jobs:</p>
<pre> <pre>
cat num30000 | parallel --jobs 4 -m echo | wc -l</pre> cat num30000 | parallel --jobs 4 -m echo | wc -l</pre>
<p>Output:</p> <p>Output:</p>

View file

@ -6,7 +6,7 @@ This tutorial shows off much of GNU Parallel's functionality. The
tutorial is meant to learn the options in GNU Parallel. The tutorial tutorial is meant to learn the options in GNU Parallel. The tutorial
is not to show realistic examples from the real world. is not to show realistic examples from the real world.
Spend an hour walking through the tutorial. Your commandline will Spend an hour walking through the tutorial. Your command line will
love you for it. love you for it.
=head1 Prerequisites =head1 Prerequisites
@ -17,53 +17,62 @@ To run this tutorial you must have the following:
=item parallel >= version 20130814 =item parallel >= version 20130814
Most of the tutorial will work on older versions, too. Install the newest version with:
(wget -O - pi.dk/3 || curl pi.dk/3/) | bash
This will also install the newest version of the tutorial:
man parallel_tutorial
Most of the tutorial will work on older versions, too.
=item abc-file: =item abc-file:
The file can be generated by: The file can be generated by:
parallel -k echo ::: A B C > abc-file parallel -k echo ::: A B C > abc-file
=item def-file: =item def-file:
The file can be generated by: The file can be generated by:
parallel -k echo ::: D E F > def-file parallel -k echo ::: D E F > def-file
=item abc0-file: =item abc0-file:
The file can be generated by: The file can be generated by:
perl -e 'printf "A\0B\0C\0"' > abc0-file perl -e 'printf "A\0B\0C\0"' > abc0-file
=item abc_-file: =item abc_-file:
The file can be generated by: The file can be generated by:
perl -e 'printf "A_B_C_%s"' > abc_-file perl -e 'printf "A_B_C_"' > abc_-file
=item tsv-file.tsv =item tsv-file.tsv
The file can be generated by: The file can be generated by:
perl -e 'printf "f1\tf2\nA\tB\nC\tD\n"' > tsv-file.tsv perl -e 'printf "f1\tf2\nA\tB\nC\tD\n"' > tsv-file.tsv
=item num30000 =item num30000
The file can be generated by: The file can be generated by:
perl -e 'for(1..30000){print "$_\n"}' > num30000 perl -e 'for(1..30000){print "$_\n"}' > num30000
=item num1000000 =item num1000000
The file can be generated by: The file can be generated by:
perl -e 'for(1..1000000){print "$_\n"}' > num1000000 perl -e 'for(1..1000000){print "$_\n"}' > num1000000
=item num_%header =item num_%header
The file can be generated by: The file can be generated by:
(echo %head1; echo %head2; perl -e 'for(1..10){print "$_\n"}') > num_%header (echo %head1; echo %head2; perl -e 'for(1..10){print "$_\n"}') > num_%header
@ -73,13 +82,13 @@ $SERVER1 and $SERVER2
SERVER1=server.example.com SERVER1=server.example.com
SERVER2=server2.example.net SERVER2=server2.example.net
You must be able to: You must be able to:
ssh $SERVER1 echo works ssh $SERVER1 echo works
ssh $SERVER2 echo works ssh $SERVER2 echo works
It can be setup by running 'ssh-keygen -t dsa; ssh-copy-id $SERVER1' It can be setup by running 'ssh-keygen -t dsa; ssh-copy-id $SERVER1'
and using an empty pass phrase. and using an empty pass phrase.
=back =back
@ -504,8 +513,8 @@ Output:
For better parallelism GNU Parallel can distribute the arguments For better parallelism GNU Parallel can distribute the arguments
between all the parallel jobs when end of file is met. between all the parallel jobs when end of file is met.
Running 4 jobs in parallel will split the last line of arguments will Running 4 jobs in parallel will split the last line of arguments into
be split into 4 jobs resulting in a total of 5 jobs: 4 jobs resulting in a total of 5 jobs:
cat num30000 | parallel --jobs 4 -m echo | wc -l cat num30000 | parallel --jobs 4 -m echo | wc -l