diff --git a/doc/release_new_version b/doc/release_new_version index fb58a344..7613e114 100644 --- a/doc/release_new_version +++ b/doc/release_new_version @@ -204,9 +204,9 @@ cc:Sandro Cazzaniga , Ryoichiro Suzuki , Jesse Alama -Subject: GNU Parallel 20120122 ('') released +Subject: GNU Parallel 20140122 ('Opportunity') released -GNU Parallel 20140122 ('') has been released. It is +GNU Parallel 20140122 ('Opportunity') has been released. It is available for download at: http://ftp.gnu.org/gnu/parallel/ New in this release: @@ -218,6 +218,9 @@ New in this release: * HaploClique uses GNU Parallel https://github.com/armintoepfer/haploclique +* Scraping NSScreencast + https://blog.nicolai86.eu/posts/2014-01-12/scraping-nsscreencast/ + * 30 Cool Open Source Software I Discovered in 2013 http://www.cyberciti.biz/open-source/30-cool-best-open-source-softwares-of-2013/ @@ -227,9 +230,15 @@ New in this release: * Parallel the execution of a job that read from stdin http://www.linuxask.com/questions/parallel-the-execution-of-a-job-that-read-from-stdin +* Mon Make à moi (6:38-11:50) + http://videos.rennes.inria.fr/ReNaBI-GO2013/indexPierreLindenbaum.html + * Shell-Abarbeitung beschleunigen: Wie Sie mit parallelen Prozesse effizienter in der Shell arbeiten https://www.hosteurope.ch/blog/shell-abarbeitung-beschleunigen-wie-sie-mit-parallelen-prozesse-effizienter-in-der-shell-arbeiten/ +* Summary of GNU Parallel tutorial + http://hacktracking.blogspot.dk/2014/01/gnu-parallel-tutorial.html + * Bug fixes and man page updates. diff --git a/src/load-parallel-results.r b/src/optional/R/load-parallel-results.r similarity index 99% rename from src/load-parallel-results.r rename to src/optional/R/load-parallel-results.r index 67aceacb..2cd7e8a1 100644 --- a/src/load-parallel-results.r +++ b/src/optional/R/load-parallel-results.r @@ -91,8 +91,6 @@ raw_to_data.table <- function(raw, ...) { require(data.table) ## Keep all columns except stdout and stderr varnames = setdiff(colnames(raw), c("stdout","stderr")) - ## Remove rownames - rownames(raw) = NULL ## after data.table feature request the as.data.frame can be skipped ## and will thus be much faster ddt = as.data.table(as.data.frame(raw,stringsAsFactors=FALSE)) diff --git a/src/parallel.pdf b/src/parallel.pdf index d2f2187c..5f539465 100644 Binary files a/src/parallel.pdf and b/src/parallel.pdf differ diff --git a/src/parallel.pod b/src/parallel.pod index 1494f44c..fac4fff0 100644 --- a/src/parallel.pod +++ b/src/parallel.pod @@ -710,8 +710,8 @@ to see the difference: parallel -j4 sleep {}\; echo {} ::: 2 1 4 3 parallel -j4 -k sleep {}\; echo {} ::: 2 1 4 3 -If used with B<--onall> or B<--nonall> output will be sorted according -to sshlogin. +If used with B<--onall> or B<--nonall> the output will grouped by +sshlogin in sorted order. =item B<-L> I diff --git a/src/parallel.texi b/src/parallel.texi index 7479167d..4da9931d 100644 --- a/src/parallel.texi +++ b/src/parallel.texi @@ -760,8 +760,8 @@ to see the difference: parallel -j4 -k sleep {}\; echo {} ::: 2 1 4 3 @end verbatim -If used with @strong{--onall} or @strong{--nonall} output will be sorted according -to sshlogin. +If used with @strong{--onall} or @strong{--nonall} the output will grouped by +sshlogin in sorted order. @item @strong{-L} @emph{max-lines} @anchor{@strong{-L} @emph{max-lines}} diff --git a/src/parallel_tutorial.1 b/src/parallel_tutorial.1 index 5cd00680..f3503728 100644 --- a/src/parallel_tutorial.1 +++ b/src/parallel_tutorial.1 @@ -124,7 +124,7 @@ .\" ======================================================================== .\" .IX Title "PARALLEL_TUTORIAL 1" -.TH PARALLEL_TUTORIAL 1 "2013-12-03" "20131222" "parallel" +.TH PARALLEL_TUTORIAL 1 "2014-01-14" "20131222" "parallel" .\" For nroff, turn off justification. Always turn off hyphenation; it makes .\" way too many mistakes in technical documents. .if n .ad l @@ -777,8 +777,14 @@ Output: For better parallelism \s-1GNU\s0 Parallel can distribute the arguments between all the parallel jobs when end of file is met. .PP -Running 4 jobs in parallel will split the last line of arguments into -4 jobs resulting in a total of 5 jobs: +Below \s-1GNU\s0 Parallel reads the last argument when generating the second +job. When \s-1GNU\s0 Parallel reads the last argument, it spreads all the +arguments for the second job over 4 jobs instead, as 4 parallel jobs +are requested. +.PP +The first job will be the same as the \-\-xargs example above, but the +second job will be split into 4 evenly sized jobs, resulting in a +total of 5 jobs: .PP .Vb 1 \& cat num30000 | parallel \-\-jobs 4 \-m echo | wc \-l @@ -790,6 +796,22 @@ Output: \& 5 .Ve .PP +This is even more visible when running 4 jobs with 10 arguments. The +10 arguments are being spread over 4 jobs: +.PP +.Vb 1 +\& parallel \-\-jobs 4 \-m echo ::: {1..10} +.Ve +.PP +Output: +.PP +.Vb 4 +\& 1 2 3 +\& 4 5 6 +\& 7 8 9 +\& 10 +.Ve +.PP A replacement string can be part of a word. \-m will not repeat the context: .PP .Vb 1 diff --git a/src/parallel_tutorial.html b/src/parallel_tutorial.html index e7b42553..6af2c89c 100644 --- a/src/parallel_tutorial.html +++ b/src/parallel_tutorial.html @@ -564,13 +564,28 @@ line length of 10000 chars 17 commands will be run:

17

For better parallelism GNU Parallel can distribute the arguments between all the parallel jobs when end of file is met.

-

Running 4 jobs in parallel will split the last line of arguments into -4 jobs resulting in a total of 5 jobs:

+

Below GNU Parallel reads the last argument when generating the second +job. When GNU Parallel reads the last argument, it spreads all the +arguments for the second job over 4 jobs instead, as 4 parallel jobs +are requested.

+

The first job will be the same as the --xargs example above, but the +second job will be split into 4 evenly sized jobs, resulting in a +total of 5 jobs:

   cat num30000 | parallel --jobs 4 -m echo | wc -l

Output:

   5
+

This is even more visible when running 4 jobs with 10 arguments. The +10 arguments are being spread over 4 jobs:

+
+  parallel --jobs 4 -m echo ::: {1..10}
+

Output:

+
+  1 2 3
+  4 5 6
+  7 8 9
+  10

A replacement string can be part of a word. -m will not repeat the context:

   parallel --jobs 4 -m echo pre-{}-post ::: A B C D E F G
diff --git a/src/parallel_tutorial.pdf b/src/parallel_tutorial.pdf index ca22a4ce..d9dc5a6c 100644 Binary files a/src/parallel_tutorial.pdf and b/src/parallel_tutorial.pdf differ diff --git a/src/parallel_tutorial.pod b/src/parallel_tutorial.pod index b4c0bcf7..52c1d5f4 100644 --- a/src/parallel_tutorial.pod +++ b/src/parallel_tutorial.pod @@ -514,8 +514,14 @@ Output: For better parallelism GNU Parallel can distribute the arguments between all the parallel jobs when end of file is met. -Running 4 jobs in parallel will split the last line of arguments into -4 jobs resulting in a total of 5 jobs: +Below GNU Parallel reads the last argument when generating the second +job. When GNU Parallel reads the last argument, it spreads all the +arguments for the second job over 4 jobs instead, as 4 parallel jobs +are requested. + +The first job will be the same as the --xargs example above, but the +second job will be split into 4 evenly sized jobs, resulting in a +total of 5 jobs: cat num30000 | parallel --jobs 4 -m echo | wc -l @@ -523,6 +529,18 @@ Output: 5 +This is even more visible when running 4 jobs with 10 arguments. The +10 arguments are being spread over 4 jobs: + + parallel --jobs 4 -m echo ::: {1..10} + +Output: + + 1 2 3 + 4 5 6 + 7 8 9 + 10 + A replacement string can be part of a word. -m will not repeat the context: parallel --jobs 4 -m echo pre-{}-post ::: A B C D E F G diff --git a/testsuite/tests-to-run/parallel-polarhome.sh b/testsuite/tests-to-run/parallel-polarhome.sh index 1ebb1f29..99824722 100644 --- a/testsuite/tests-to-run/parallel-polarhome.sh +++ b/testsuite/tests-to-run/parallel-polarhome.sh @@ -1,10 +1,10 @@ #!/bin/bash -P_ALL="vax freebsd solaris openbsd netbsd debian alpha aix redhat hpux ultrix qnx irix tru64 openindiana suse openstep mandriva ubuntu scosysv unixware dragonfly centos miros hurd raspberrypi" +P_ALL="vax freebsd solaris openbsd netbsd debian alpha aix redhat hpux ultrix qnx irix tru64 openindiana suse solaris-x86 mandriva ubuntu scosysv unixware dragonfly centos miros hurd raspberrypi" P_NOTWORKING="vax alpha openstep" -P_NOTWORKING_YET="ultrix irix minix" +P_NOTWORKING_YET="ultrix irix" -P_WORKING="freebsd solaris openbsd netbsd debian aix redhat hpux qnx tru64 openindiana suse mandriva ubuntu scosysv unixware dragonfly centos miros hurd raspberrypi" +P_WORKING="minix freebsd solaris openbsd netbsd debian aix redhat hpux qnx tru64 openindiana suse solaris-x86 mandriva ubuntu scosysv unixware dragonfly centos miros hurd raspberrypi" P="$P_WORKING" POLAR=`parallel -k echo {}.polarhome.com ::: $P` diff --git a/testsuite/wanted-results/parallel-polarhome b/testsuite/wanted-results/parallel-polarhome index d6ee48bd..6617da2d 100644 --- a/testsuite/wanted-results/parallel-polarhome +++ b/testsuite/wanted-results/parallel-polarhome @@ -1,5 +1,8 @@ ### Tests on polarhome machines Setup on polarhome machines +copy_and_test minix.polarhome.com +minix.polarhome.com ### Run the test on minix.polarhome.com +minix.polarhome.com Works on minix.polarhome.com copy_and_test freebsd.polarhome.com freebsd.polarhome.com ### Run the test on freebsd.polarhome.com freebsd.polarhome.com Works on freebsd.polarhome.com @@ -38,6 +41,9 @@ openindiana.polarhome.com parallel: Warning: Cannot figure out number of CPU cor copy_and_test suse.polarhome.com suse.polarhome.com ### Run the test on suse.polarhome.com suse.polarhome.com Works on suse.polarhome.com +copy_and_test solaris-x86.polarhome.com +solaris-x86.polarhome.com ### Run the test on solaris-x86.polarhome.com +solaris-x86.polarhome.com Works on solaris-x86.polarhome.com copy_and_test mandriva.polarhome.com mandriva.polarhome.com ### Run the test on mandriva.polarhome.com mandriva.polarhome.com Works on mandriva.polarhome.com