mirror of
https://git.savannah.gnu.org/git/parallel.git
synced 2024-11-24 15:07:55 +00:00
env_parallel.bash tested: Works better as variable than as tempfile.
This commit is contained in:
parent
e663fe2636
commit
9892e7deac
2
README
2
README
|
@ -6,7 +6,7 @@ Please send problems and feedback to bug-parallel@gnu.org.
|
||||||
= Presentation of GNU Parallel =
|
= Presentation of GNU Parallel =
|
||||||
|
|
||||||
GNU Parallel is a shell tool for executing jobs in parallel using one
|
GNU Parallel is a shell tool for executing jobs in parallel using one
|
||||||
or more computers. A job is can be a single command or a small script
|
or more computers. A job can be a single command or a small script
|
||||||
that has to be run for each of the lines in the input. The typical
|
that has to be run for each of the lines in the input. The typical
|
||||||
input is a list of files, a list of hosts, a list of users, a list of
|
input is a list of files, a list of hosts, a list of users, a list of
|
||||||
URLs, or a list of tables. A job can also be a command that reads from
|
URLs, or a list of tables. A job can also be a command that reads from
|
||||||
|
|
|
@ -219,9 +219,9 @@ cc:Tim Cuthbertson <tim3d.junk@gmail.com>,
|
||||||
Ryoichiro Suzuki <ryoichiro.suzuki@gmail.com>,
|
Ryoichiro Suzuki <ryoichiro.suzuki@gmail.com>,
|
||||||
Jesse Alama <jesse.alama@gmail.com>
|
Jesse Alama <jesse.alama@gmail.com>
|
||||||
|
|
||||||
Subject: GNU Parallel 20160422 ('') released <<[stable]>>
|
Subject: GNU Parallel 20160422 ('Gulshan-i-Iqbal') released <<[stable]>>
|
||||||
|
|
||||||
GNU Parallel 20160422 ('') <<[stable]>> has been released. It is available for download at: http://ftp.gnu.org/gnu/parallel/
|
GNU Parallel 20160422 ('Gulshan-i-Iqbal') <<[stable]>> has been released. It is available for download at: http://ftp.gnu.org/gnu/parallel/
|
||||||
|
|
||||||
<<No new functionality was introduced so this is a good candidate for a stable release.>>
|
<<No new functionality was introduced so this is a good candidate for a stable release.>>
|
||||||
|
|
||||||
|
@ -248,8 +248,18 @@ for Big Data Applications https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumb
|
||||||
|
|
||||||
* <<Citation needed: Introspecting for RSA Key Material to Assist Intrusion Detection http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=7331177&url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D7331177>>
|
* <<Citation needed: Introspecting for RSA Key Material to Assist Intrusion Detection http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=7331177&url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D7331177>>
|
||||||
|
|
||||||
|
* GNU Parallel was cited in: How Can We Measure the Similarity Between Resumes of Selected Candidates for a Job? https://www.researchgate.net/publication/275954089_How_can_we_measure_the_similarity_between_resumes_of_selected_candidates_for_a_job
|
||||||
|
|
||||||
|
* GNU Parallel was cited in: Automatic Methods for Assisted Recruitment https://www.researchgate.net/publication/297738658_Automatic_Methods_for_Assisted_Recruitment
|
||||||
|
|
||||||
* GNU Parallel was cited in: Tools and techniques for computational reproducibility http://biorxiv.org/content/biorxiv/early/2016/03/17/022707.full.pdf
|
* GNU Parallel was cited in: Tools and techniques for computational reproducibility http://biorxiv.org/content/biorxiv/early/2016/03/17/022707.full.pdf
|
||||||
|
|
||||||
|
* GNU Parallel was cited in: Reinterpretation of ATLAS 8 TeV searches for Natural SUSY with a R-Sneutrino LSP http://arxiv.org/pdf/1603.06130.pdf
|
||||||
|
|
||||||
|
* GNU Parallel was cited in: An Operational Radiometric Landsat Preprocessing Framework for Large-Area Time Series Applications https://www.uni-trier.de/fileadmin/fb6/prof/FER/Publikationen/frantz_et_al_ieee-tgrs-2016-post-print.pdf
|
||||||
|
|
||||||
|
* Downloading a list of URLs http://blog.gypsydave5.com/2016/02/04/xargs-and-curl/
|
||||||
|
|
||||||
* qbatch uses GNU Parallel: https://pypi.python.org/pypi/qbatch/1.0rc2
|
* qbatch uses GNU Parallel: https://pypi.python.org/pypi/qbatch/1.0rc2
|
||||||
|
|
||||||
* FaceCrop uses GNU Parallel: https://github.com/EderSantana/FaceCrop
|
* FaceCrop uses GNU Parallel: https://github.com/EderSantana/FaceCrop
|
||||||
|
|
|
@ -25,6 +25,7 @@
|
||||||
# or write to the Free Software Foundation, Inc., 51 Franklin St,
|
# or write to the Free Software Foundation, Inc., 51 Franklin St,
|
||||||
# Fifth Floor, Boston, MA 02110-1301 USA
|
# Fifth Floor, Boston, MA 02110-1301 USA
|
||||||
|
|
||||||
|
# Supports env of 127426 bytes
|
||||||
env_parallel() {
|
env_parallel() {
|
||||||
export PARALLEL_ENV="$(echo "shopt -s expand_aliases 2>/dev/null"; alias;typeset -p |
|
export PARALLEL_ENV="$(echo "shopt -s expand_aliases 2>/dev/null"; alias;typeset -p |
|
||||||
grep -vFf <(readonly) |
|
grep -vFf <(readonly) |
|
||||||
|
@ -33,3 +34,17 @@ env_parallel() {
|
||||||
`which parallel` "$@";
|
`which parallel` "$@";
|
||||||
unset PARALLEL_ENV;
|
unset PARALLEL_ENV;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# Supports env of 127375 bytes
|
||||||
|
#
|
||||||
|
# env_parallel() {
|
||||||
|
# # Saving to a tempfile
|
||||||
|
# export PARALLEL_ENV=`tempfile`;
|
||||||
|
# (echo "shopt -s expand_aliases 2>/dev/null"; alias;typeset -p |
|
||||||
|
# grep -vFf <(readonly) |
|
||||||
|
# grep -v 'declare .. (GROUPS|FUNCNAME|DIRSTACK|_|PIPESTATUS|USERNAME|BASH_[A-Z_]+) ';
|
||||||
|
# typeset -f) > $PARALLEL_ENV
|
||||||
|
# `which parallel` "$@";
|
||||||
|
# rm "$PARALLEL_ENV"
|
||||||
|
# unset PARALLEL_ENV;
|
||||||
|
# }
|
||||||
|
|
|
@ -20,10 +20,17 @@ environment to GNU Parallel.
|
||||||
If the shell function is not loaded, a dummy script will be run
|
If the shell function is not loaded, a dummy script will be run
|
||||||
instead that explains how to install the function.
|
instead that explains how to install the function.
|
||||||
|
|
||||||
|
=head2 Environment space
|
||||||
|
|
||||||
B<env_parallel> only works if the size of the current environment is
|
B<env_parallel> only works if the size of the current environment is
|
||||||
smaller than half of the max size and smaller than 25% of the max if
|
smaller than the maximal length of a command and smaller than half of
|
||||||
running remotely. E.g. The max size of Bash's environment is 256 KB,
|
the max if running remotely. E.g. The max size of Bash's command is
|
||||||
so B<env_parallel> will fail if B<set | wc -c> is bigger than 128 KB.
|
128 KB, so B<env_parallel> will fail if B<set | wc -c> is bigger than
|
||||||
|
128 KB. Technically the limit is in execve(1) which IPC::open3 uses.
|
||||||
|
|
||||||
|
Bash completion functions are well-known for taking up well over 128
|
||||||
|
KB of environment space and the primary reason for causing
|
||||||
|
B<env_parallel> to fail.
|
||||||
|
|
||||||
|
|
||||||
=head1 OPTIONS
|
=head1 OPTIONS
|
||||||
|
|
Loading…
Reference in a new issue