tange/parallel
1
0
Fork 0
mirror of https://git.savannah.gnu.org/git/parallel.git synced 2024-11-25 23:47:53 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

parallel: Fixed bug #41613: --compress --line-buffer. With testsuite.

Browse source
This commit is contained in:
Ole Tange 2014-03-02 14:36:43 +01:00
parent 7f955d2427
commit faff0ebb1e
4 changed files with 45 additions and 43 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

46
doc/release_new_version
View file

@ -207,54 +207,20 @@ cc:Sandro Cazzaniga <kharec@mandriva.org>,
Ryoichiro Suzuki <ryoichiro.suzuki@gmail.com>, Ryoichiro Suzuki <ryoichiro.suzuki@gmail.com>,
Jesse Alama <jesse.alama@gmail.com> Jesse Alama <jesse.alama@gmail.com>
Subject: GNU Parallel 20140222 ('Euromaidan') released Subject: GNU Parallel 20140322 ('') released
GNU Parallel 20140222 ('Euromaidan') has been released. It is available for download at: http://ftp.gnu.org/gnu/parallel/ GNU Parallel 20140322 ('') has been released. It is available for download at: http://ftp.gnu.org/gnu/parallel/
Packagers that default to --tollef should take note that --tollef is now retired after a full year of being obsolete.
The --compress change is so big that this release is of alpha quality.
New in this release: New in this release:
* --tollef has been retired. * Batch convert mp3 CBR to VBR http://blogs.zercle.com/bouroo/2013/03/batch-convert-mp3-cbr-to-vbr/
* --compress has be redesigned due to bugs. * Running Behat in parallel with Selenium grid http://zalas.eu/running-behat-in-parallel-with-selenium-grid/
* Format of --eta and --joblog has changed slightly. * Git-Mirror per cronjob aktuell halten https://www.schtobia.de/2013/git-mirror-per-cronjob-aktuell-halten/
* GNU Parallel was cited in: Speaker Verification and Keyword Spotting Systems for Forensic Applications http://lantana.tenet.res.in/website_files/thesis/Phd/srikanth.pdf * Пишем многопоточный скрипт на bash http://dafter.ru/duf/howto/130.html
* GNU Parallel was cited in: Scaleable Code Clone Detection http://scg.unibe.ch/archive/phd/schwarz-phd.pdf
* GNU Parallel was cited in: Clone Detection that scales http://scg.unibe.ch/archive/masters/Vogt14a.pdf
* GNU Parallel was cited in: Scalable Positional Analysis for Studying Evolution of Nodes in Networks http://arxiv-web3.library.cornell.edu/pdf/1402.3797
* GNU Parallel was cited in: Aggregate Characterization of User Behavior in Twitter and Analysis of the Retweet Graph http://arxiv.org/pdf/1402.2671v1
* GNU Parallel was used (unfortunately without citation) in: Causal Network Models for Predicting Compound Targets and Driving Pathways in Cancer http://jbx.sagepub.com/content/early/2014/02/10/1087057114522690.full
* GNU Parallel was used (unfortunately without citation) in: Performance and Energy Efficiency of Common Compression / Decompression Utilities http://www.researchgate.net/publication/243962643_Performance_and_Energy_Efficiency_of_Common_CompressionDecompression_Utilities_An_Experimental_Study_in_Mobile_and_Workstation_Computer_Platforms/file/3deec51d1dbc0474f9.pdf
* GNU Parallel was used (unfortunately without citation) in: SpringFS: Bridging Agility and Performance in Elastic Distributed Storage http://www.istc-cc.cmu.edu/publications/papers/2014/SpringFS-FAST14.pdf
* GNU Parallel was recommended (without citation) in: Redesigning the Specificity of Protein-DNA Interactions with Rosetta http://link.springer.com/protocol/10.1007/978-1-62703-968-0_17
* GNU Parallel is co-distributed with RepeatExplorer http://www.vcru.wisc.edu/simonlab/bioinformatics/programs/repeatexplorer/README.txt
* Convert MP3 to a scrolling spectrum waterfall plot video http://jdesbonnet.blogspot.dk/2014/02/convert-mp3-to-scrolling-spectrum.html
* PHP wrapper class for the GNU Parallel tool https://github.com/geopal-solutions/gnu-parallel-wrapper/tree/master
* Exploratory Data Analysis http://www.slideshare.net/thinrhino/gnunify
* Copias de archivos con rsync y parallel http://linuxmanr4.com/2014/02/20/copias-de-archivos-con-rsync-y-parallel
* Single-Thread-Programme auf Multicore-Rechnern parallelisieren http://www.adlerweb.info/blog/2014/02/08/linux-single-thread-programme-auf-multicore-rechnern-parallelisieren
* Распараллеливаем процессы для ускорения вычислений и выполнения заданий в Linux http://savepearlharbor.com/?p=210480
* Bug fixes and man page updates. * Bug fixes and man page updates.

5
src/parallel
View file

@ -5025,6 +5025,11 @@ sub print {
# Remove the printed part # Remove the printed part
substr($$partial,0,$i+1)=""; substr($$partial,0,$i+1)="";
} }
}
if(defined $self->{'exitstatus'}) {
# The job is finished, there might be line with no \n left
print $out_fd $$partial;
} }
if($self->fh($fdno,"rpid") and CORE::kill 0, $self->fh($fdno,"rpid")) { if($self->fh($fdno,"rpid") and CORE::kill 0, $self->fh($fdno,"rpid")) {

31
testsuite/tests-to-run/parallel-local9.sh
View file

@ -6,10 +6,35 @@ XAP="nice nice parallel --xapply"
export XAP export XAP
cat <<'EOF' | sed -e 's/$SERVER1/'$SERVER1'/;s/$SERVER2/'$SERVER2'/' | stdout parallel -j0 -k -L1 cat <<'EOF' | sed -e 's/$SERVER1/'$SERVER1'/;s/$SERVER2/'$SERVER2'/' | stdout parallel -j0 -k -L1
echo 'bug #41613: --compress --line-buffer no newline';
perl -e 'print "It worked"'| parallel --pipe --compress --line-buffer cat; echo
echo 'bug #41613: --compress --line-buffer no --tagstring';
diff
<(perl -e 'for("x011".."x110"){print "$_\t", (map { rand } (1..100000)),"\n"}'|
parallel -N10 -L1 --pipe -j6 --block 20M --compress
pv -qL 3000000 | perl -pe 's/(....).*/$1/')
<(perl -e 'for("x011".."x110"){print "$_\t", (map { rand } (1..100000)),"\n"}'|
parallel -N10 -L1 --pipe -j6 --block 20M --compress --line-buffer
pv -qL 3000000 | perl -pe 's/(....).*/$1/')
>/dev/null
|| echo 'Good: --line-buffer matters'
echo 'bug #41613: --compress --line-buffer with --tagstring';
diff
<(perl -e 'for("x011".."x110"){print "$_\t", (map { rand } (1..100000)),"\n"}'|
parallel -N10 -L1 --pipe -j6 --block 20M --compress --tagstring {#}
pv -qL 3000000 | perl -pe 's/(....).*/$1/')
<(perl -e 'for("x011".."x110"){print "$_\t", (map { rand } (1..100000)),"\n"}'|
parallel -N10 -L1 --pipe -j6 --block 20M --compress --tagstring {#} --line-buffer
pv -qL 3000000 | perl -pe 's/(....).*/$1/')
>/dev/null
|| echo 'Good: --line-buffer matters'
echo 'bug #41412: --timeout + --delay causes deadlock'; echo 'bug #41412: --timeout + --delay causes deadlock';
seq 10 | parallel -j10 --timeout 1 --delay .3 echo seq 10 | parallel -j10 --timeout 1 --delay .3 echo;
parallel -j3 --timeout 1 --delay 2 echo ::: 1 2 3 parallel -j3 --timeout 1 --delay 2 echo ::: 1 2 3;
parallel -j10 --timeout 2.2 --delay 3 "sleep {}; echo {}" ::: 1 2 4 5 6 parallel -j10 --timeout 2.2 --delay 3 "sleep {}; echo {}" ::: 1 2 4 5 6
echo '### Test --spreadstdin - more procs than args'; echo '### Test --spreadstdin - more procs than args';
rm -f /tmp/parallel.ss.*; rm -f /tmp/parallel.ss.*;

6
testsuite/wanted-results/parallel-local9
View file

@ -1,3 +1,9 @@
bug #41613: --compress --line-buffer no newline
It worked
bug #41613: --compress --line-buffer no --tagstring
Good: --line-buffer matters
bug #41613: --compress --line-buffer with --tagstring
Good: --line-buffer matters
bug #41412: --timeout + --delay causes deadlock bug #41412: --timeout + --delay causes deadlock
1 1
2 2