tange/parallel
1
0
Fork 0
mirror of https://git.savannah.gnu.org/git/parallel.git synced 2024-11-26 07:57:58 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Cleanup for next release.

Browse source
This commit is contained in:
Ole Tange 2015-03-22 14:14:50 +01:00
parent 1eafde239f
commit 585a29c062
3 changed files with 26 additions and 86 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

68
doc/release_new_version
View file

@ -4,7 +4,7 @@
Check that documentation is updated (compare to web): Check that documentation is updated (compare to web):
Fixet for 20150122 Fixet for 20150422
git diff last-release-commit git diff last-release-commit
Unmodified beta since last version => production Unmodified beta since last version => production
Unmodified alpha since last version => beta Unmodified alpha since last version => beta
@ -208,75 +208,21 @@ cc:Tim Cuthbertson <tim3d.junk@gmail.com>,
Ryoichiro Suzuki <ryoichiro.suzuki@gmail.com>, Ryoichiro Suzuki <ryoichiro.suzuki@gmail.com>,
Jesse Alama <jesse.alama@gmail.com> Jesse Alama <jesse.alama@gmail.com>
Subject: GNU Parallel 20150322 ('Hellwig') released Subject: GNU Parallel 20150422 ('') released
GNU Parallel 20150322 ('Hellwig') has been released. It is available for download at: http://ftp.gnu.org/gnu/parallel/ GNU Parallel 20150422 ('') has been released. It is available for download at: http://ftp.gnu.org/gnu/parallel/
Haiku of the month: Haiku of the month:
Stand up for your rights. <<>>
VMware does not comply.
Hellwig goes to court.
(https://sfconservancy.org/linux-compliance/vmware-lawsuit-faq.html)
New in this release: New in this release:
* --number-of-cores respects 'taskset' on GNU/Linux. * <<afventer opdatering>> CIDER: a pipeline for detecting waves of coordinated transcriptional regulation in gene expression time-course data http://biorxiv.org/content/biorxiv/early/2015/03/17/012518.full.pdf
* --joblog --pipe gives the data send and received in the log. * <<afventer opdatering>> GNU Parallel was used (unfortunately without citation) in: MUGBAS: a species free gene-based programme suite for post-GWAS analysis http://www.ncbi.nlm.nih.gov/pubmed/25765345
* GNU Parallel was tested to support 100 GB sized records in --pipe. A few bugs was fixed to support >2 GB records. It works, but is rather slow. taxator-tk http://algbio.cs.uni-duesseldorf.de/webapps/wa-download/ (check it)
* GNU Parallel was cited in: RIG: Recalibration and Interrelation of genomic sequence data with the GATK http://www.g3journal.org/content/early/2015/02/13/g3.115.017012.full.pdf+html
* GNU Parallel was cited in: MPI-blastn and NCBI-TaxCollector: Improving metagenomic analysis with high performance classification and wide taxonomic attachment http://www.worldscientific.com/doi/abs/10.1142/S0219720014500139?af=R&
* GNU Parallel was cited in: Recent evolution in Rattus norvegicus is shaped by declining effective population size http://biorxiv.org/content/biorxiv/early/2015/03/01/015818.full.pdf
* GNU Parallel was cited in: Evidence for DCO+ as a probe of ionization in the warm disk surface http://arxiv.org/pdf/1503.02659.pdf
* GNU Parallel was cited in: De novo assembly and annotation of the Asian tiger mosquito (Aedes albopictus) repeatome with dnaPipeTE from raw genomic reads and comparative analysis with the yellow fever mosquito (Aedes aegypti) http://gbe.oxfordjournals.org/content/early/2015/03/11/gbe.evv050.full.pdf
* GNU Parallel was cited in: A General Approach to Network Configuration Analysis http://research.microsoft.com/en-us/um/people/ratul/papers/nsdi2015-batfish.pdf
* GNU Parallel was cited in: Scrimer: designing primers from transcriptome data http://onlinelibrary.wiley.com/doi/10.1111/1755-0998.12403/pdf
* GNU Parallel was cited in: Efficient Retrieval of Key Material for Inspecting Potentially Malicious Traffic in the Cloud http://sacko.uk/pdf/2015.1.pdf
* GNU Parallel will be presented at Strataconf: Poor Man's Parallel Pipelines http://strataconf.com/big-data-conference-uk-2015/public/schedule/detail/40031
* GNU Parallel was used in: https://github.com/alexbyrnes/FCC-Political-Ads_The-Code
* GNU Parallel was used in: https://github.com/martymac/fpart
* GNU Parallel was used in: https://github.com/hoytak/diabetic-retinopathy-code
* GNU Parallel was used in: https://github.com/mehmattski/HybSeqPipeline
* GNU Parallel was used in: http://search.cpan.org/~ajpage/Bio-Roary-2.0.0/lib/Bio/Roary/JobRunner/Parallel.pm
* Using GNU Parallel on a Raspberry Pi cluster: http://www.dcglug.org.uk/cluster-progress/
* An introduction to vector tiling and map reduce in postgis: http://dimensionaledge.com/intro-vector-tiling-map-reduce-postgis/
* Running scripts in parallel with GNU Parallel: http://code.jasonbhill.com/2015/03/
* Mahout: Parallelising the creation of DecisionTrees: http://www.markhneedham.com/blog/2012/12/27/mahout-parallelising-the-creation-of-decisiontrees/
* High-Performance Scientific Computing: Running serial jobs in parallel https://support.scinet.utoronto.ca/education/staticpublic/course178/serial.pdf
* Parallel cardinality on your laptop https://highonscience.wordpress.com/2015/03/15/parallel-cardinality-on-your-laptop/
* Experiments about a better locate using grep http://a3nm.net/blog/better_locate.html
* Homework with GNU Parallel: https://support.scinet.utoronto.ca/education/staticpublic/course178content349.html
* Parallel - Jobs in Skripten parallelisieren: https://slzm.de/blog/linux-tool-des-tages-parallel-jobs-in-skripten-parallelisieren/#more-612
* GNU Parallel, czyli 100% użycia procesora http://matmatyk.blogspot.dk/2015/03/gnu-parallel-czyli-100-uzycia-procesora.html
* Bug fixes and man page updates.
GNU Parallel - For people who live life in the parallel lane. GNU Parallel - For people who live life in the parallel lane.

14
src/parallel
View file

@ -1052,7 +1052,7 @@ sub parse_options {
sub init_globals { sub init_globals {
# Defaults: # Defaults:
$Global::version = 20150322; $Global::version = 20150323;
$Global::progname = 'parallel'; $Global::progname = 'parallel';
$Global::infinity = 2**31; $Global::infinity = 2**31;
$Global::debug = 0; $Global::debug = 0;
@ -3524,6 +3524,7 @@ sub tmpfile {
# N/A # N/A
if(not $disk_full_fh) { if(not $disk_full_fh) {
($disk_full_fh, $name) = ::tmpfile(SUFFIX => ".df"); ($disk_full_fh, $name) = ::tmpfile(SUFFIX => ".df");
# Separate unlink due to NFS dealing badly with File::Temp
unlink $name; unlink $name;
$b8193 = "x"x8193; $b8193 = "x"x8193;
} }
@ -3787,14 +3788,7 @@ sub memfree {
# Returns: # Returns:
# $memfree in bytes # $memfree in bytes
my $self = shift; my $self = shift;
if(not $self->{'memfree'}) { $self->memfree_recompute();
$self->memfree_recompute();
}
# if(time - $self->{'last_memfree'} >= 1) {
# More than 10 seconds old: Recompute
$self->{'last_memfree'} = time;
$self->memfree_recompute();
# }
return (not defined $self->{'memfree'} or $self->{'memfree'}) return (not defined $self->{'memfree'} or $self->{'memfree'})
} }
@ -5640,7 +5634,7 @@ sub max_file_name_length {
rmdir($testdir."/".$dir); rmdir($testdir."/".$dir);
$len *= 16; $len *= 16;
$dir = "x"x$len; $dir = "x"x$len;
} while (mkdir $testdir."/".$dir); } while ($len < $upper and mkdir $testdir."/".$dir);
# Then search for the actual max length between $len/16 and $len # Then search for the actual max length between $len/16 and $len
my $min = $len/16; my $min = $len/16;
my $max = $len; my $max = $len;

30
src/parallel.pod
View file

@ -488,7 +488,7 @@ I<regexp> is a Perl Regular Expression:
http://perldoc.perl.org/perlre.html http://perldoc.perl.org/perlre.html
=item B<--compress> (alpha testing) =item B<--compress> (beta testing)
Compress temporary files. If the output is big and very compressible Compress temporary files. If the output is big and very compressible
this will take up less disk space in $TMPDIR and possibly be faster this will take up less disk space in $TMPDIR and possibly be faster
@ -499,9 +499,9 @@ B<plzip>, B<bzip2>, B<lzma>, B<lzip>, B<xz> in that order, and use the
first available. first available.
=item B<--compress-program> I<prg> (alpha testing) =item B<--compress-program> I<prg> (beta testing)
=item B<--decompress-program> I<prg> (alpha testing) =item B<--decompress-program> I<prg> (beta testing)
Use I<prg> for (de)compressing temporary files. It is assumed that I<prg Use I<prg> for (de)compressing temporary files. It is assumed that I<prg
-dc> will decompress stdin (standard input) to stdout (standard -dc> will decompress stdin (standard input) to stdout (standard
@ -514,9 +514,9 @@ If receiving SIGNING, GNU B<parallel> will send SIGINT to tasks
running on remote computers thus killing them. running on remote computers thus killing them.
=item B<--delimiter> I<delim> (alpha testing) =item B<--delimiter> I<delim> (beta testing)
=item B<-d> I<delim> (alpha testing) =item B<-d> I<delim> (beta testing)
Input items are terminated by I<delim>. Quotes and backslash are not Input items are terminated by I<delim>. Quotes and backslash are not
special; every character in the input is taken literally. Disables special; every character in the input is taken literally. Disables
@ -780,7 +780,7 @@ specified, and for B<-I>{} otherwise. This option is deprecated;
use B<-I> instead. use B<-I> instead.
=item B<--joblog> I<logfile> (alpha testing) =item B<--joblog> I<logfile> (beta testing)
Logfile for executed jobs. Save a list of the executed jobs to Logfile for executed jobs. Save a list of the executed jobs to
I<logfile> in the following TAB separated format: sequence number, I<logfile> in the following TAB separated format: sequence number,
@ -1037,9 +1037,9 @@ of each job is saved in a file and the filename is then printed.
See also: B<--results> See also: B<--results>
=item B<--pipe> (alpha testing) =item B<--pipe> (beta testing)
=item B<--spreadstdin> (alpha testing) =item B<--spreadstdin> (beta testing)
Spread input to jobs on stdin (standard input). Read a block of data Spread input to jobs on stdin (standard input). Read a block of data
from stdin (standard input) and give one block of data as input to one from stdin (standard input) and give one block of data as input to one
@ -1160,13 +1160,13 @@ exit (used by GNU B<parallel> itself to determine the line length
on remote computers). on remote computers).
=item B<--number-of-cpus> (alpha testing) =item B<--number-of-cpus> (beta testing)
Print the number of physical CPUs and exit (used by GNU B<parallel> Print the number of physical CPUs and exit (used by GNU B<parallel>
itself to determine the number of physical CPUs on remote computers). itself to determine the number of physical CPUs on remote computers).
=item B<--number-of-cores> (alpha testing) =item B<--number-of-cores> (beta testing)
Print the number of CPU cores and exit (used by GNU B<parallel> itself Print the number of CPU cores and exit (used by GNU B<parallel> itself
to determine the number of CPU cores on remote computers). to determine the number of CPU cores on remote computers).
@ -1178,7 +1178,7 @@ Overrides an earlier B<--keep-order> (e.g. if set in
B<~/.parallel/config>). B<~/.parallel/config>).
=item B<--nice> I<niceness> (alpha testing) =item B<--nice> I<niceness> (beta testing)
Run the command at this niceness. For simple commands you can just add Run the command at this niceness. For simple commands you can just add
B<nice> in front of the command. But if the command consists of more B<nice> in front of the command. But if the command consists of more
@ -1387,7 +1387,7 @@ useful if some jobs fail for no apparent reason (such as network
failure). failure).
=item B<--return> I<filename> (beta testing) =item B<--return> I<filename>
Transfer files from remote computers. B<--return> is used with Transfer files from remote computers. B<--return> is used with
B<--sshlogin> when the arguments are files on the remote computers. When B<--sshlogin> when the arguments are files on the remote computers. When
@ -1435,7 +1435,7 @@ B<--keep-order> will not work with B<--round-robin> as it is
impossible to track which input block corresponds to which output. impossible to track which input block corresponds to which output.
=item B<--rpl> 'I<tag> I<perl expression>' (alpha testing) =item B<--rpl> 'I<tag> I<perl expression>' (beta testing)
Use I<tag> as a replacement string for I<perl expression>. This makes Use I<tag> as a replacement string for I<perl expression>. This makes
it possible to define your own replacement strings. GNU B<parallel>'s it possible to define your own replacement strings. GNU B<parallel>'s
@ -1763,7 +1763,7 @@ the lines will be prepended with the sshlogin instead.
B<--tag> is ignored when using B<-u>. B<--tag> is ignored when using B<-u>.
=item B<--tagstring> I<str> (alpha testing) =item B<--tagstring> I<str> (beta testing)
Tag lines with a string. Each output line will be prepended with Tag lines with a string. Each output line will be prepended with
I<str> and TAB (\t). I<str> can contain replacement strings such as I<str> and TAB (\t). I<str> can contain replacement strings such as
@ -1780,7 +1780,7 @@ different dir for the files. Setting B<--tmpdir> is equivalent to
setting $TMPDIR. setting $TMPDIR.
=item B<--tmux> (alpha testing) =item B<--tmux> (beta testing)
Use B<tmux> for output. Start a B<tmux> session and run each job in a Use B<tmux> for output. Start a B<tmux> session and run each job in a
window in that session. No other output will be produced. window in that session. No other output will be produced.