tange/parallel
1
0
Fork 0
mirror of https://git.savannah.gnu.org/git/parallel.git synced 2024-11-21 21:47:54 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

parallel: Fixed bug #64237: --filter-hosts and VisualHostKey=yes.

Browse source
This commit is contained in:
Ole Tange 2023-06-04 09:46:04 +02:00
parent 7aa3453bdd
commit aa78d2d42f
14 changed files with 49 additions and 34 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

27
doc/release_new_version
View file

@ -105,6 +105,7 @@ lbry://@GnuParallel#4/parallel-20210322.tar.bz2
. .last-doitag.txt . .last-doitag.txt
file_path="`pwd`/parallel-$YYYYMMDD.tar.bz2" file_path="`pwd`/parallel-$YYYYMMDD.tar.bz2"
title="GNU Parallel $YYYYMMDD ('$SPCTAG') [stable]" title="GNU Parallel $YYYYMMDD ('$SPCTAG') [stable]"
title="GNU Parallel $YYYYMMDD ('$SPCTAG')" title="GNU Parallel $YYYYMMDD ('$SPCTAG')"
name="GNU-Parallel-$YYYYMMDD-$TAG" name="GNU-Parallel-$YYYYMMDD-$TAG"
author="Ole Tange" author="Ole Tange"
@ -263,38 +264,24 @@ from:tange@gnu.org
to:parallel@gnu.org, bug-parallel@gnu.org to:parallel@gnu.org, bug-parallel@gnu.org
stable-bcc: Jesse Alama <jessealama@fastmail.fm> stable-bcc: Jesse Alama <jessealama@fastmail.fm>
Subject: GNU Parallel 20230522 ('Charles') released [stable] Subject: GNU Parallel 20230622 ('Tina Turner/Odisha<<>>') released <<[stable]>>
GNU Parallel 20230522 ('Charles') has been released. It is available for download at: lbry://@GnuParallel:4 GNU Parallel 20230622 ('<<>>') has been released. It is available for download at: lbry://@GnuParallel:4
Quote of the month: Quote of the month:
Recently learned how to use GNU parallel (from ChatGPT, no less!) and I've gone mad with power <<>>
-- Mark, Anthropomorphic Anuran @reject_resubmit@twitter
New in this release: New in this release:
* No new features. This is a candidate for a stable release. <<>>
* Bug fixes and man page updates. * Bug fixes and man page updates.
News about GNU Parallel: News about GNU Parallel:
http://blog.faucher.net/2023/05/building-evil-android-phone-bot-army.html
* Introduction to parallel computing https://youtu.be/fvrHXV8yqU4?t=2874 <<>>
* Script for summarizing GNU parallel joblog file https://github.com/gavinmdouglas/parallel_joblog_summary
* How to copy a single file to multiple directories in Linux or Unix https://www.cyberciti.biz/faq/linux-unix-copy-a-file-to-multiple-directories-using-cp-command/
* Search in your Jupyter notebooks from the CLI, fast. https://dev.to/attilavm/search-in-your-jupyter-notebooks-from-the-cli-fast-1408
* Parallel Job Orchestration with GNU Parallel https://www.youtube.com/watch?v=2tVpUfND3LI
* Optional Individual Submission 4 Job Handling 20266001 - GNU Parallel https://www.youtube.com/watch?v=eC_RPuFCcU8
* DOE Cross-facility Workflows Training - April 12, 2023 https://www.youtube.com/watch?v=Ke3sirM-aQQ
* How to run MiXCR 4.x on multiple patient samples using GNU Parallel https://www.youtube.com/watch?v=OXg-WHlB_dk
GNU Parallel - For people who live life in the parallel lane. GNU Parallel - For people who live life in the parallel lane.

2
src/env_parallel.ash
View file

@ -393,7 +393,7 @@ _parset_main() {
fi fi
if [ "$_parset_NAME" = "--version" ] ; then if [ "$_parset_NAME" = "--version" ] ; then
# shellcheck disable=SC2006 # shellcheck disable=SC2006
echo "parset 20230522 (GNU parallel `parallel --minversion 1`)" echo "parset 20230523 (GNU parallel `parallel --minversion 1`)"
echo "Copyright (C) 2007-2023 Ole Tange, http://ole.tange.dk and Free Software" echo "Copyright (C) 2007-2023 Ole Tange, http://ole.tange.dk and Free Software"
echo "Foundation, Inc." echo "Foundation, Inc."
echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>" echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>"

2
src/env_parallel.bash
View file

@ -395,7 +395,7 @@ _parset_main() {
fi fi
if [ "$_parset_NAME" = "--version" ] ; then if [ "$_parset_NAME" = "--version" ] ; then
# shellcheck disable=SC2006 # shellcheck disable=SC2006
echo "parset 20230522 (GNU parallel `parallel --minversion 1`)" echo "parset 20230523 (GNU parallel `parallel --minversion 1`)"
echo "Copyright (C) 2007-2023 Ole Tange, http://ole.tange.dk and Free Software" echo "Copyright (C) 2007-2023 Ole Tange, http://ole.tange.dk and Free Software"
echo "Foundation, Inc." echo "Foundation, Inc."
echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>" echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>"

2
src/env_parallel.dash
View file

@ -393,7 +393,7 @@ _parset_main() {
fi fi
if [ "$_parset_NAME" = "--version" ] ; then if [ "$_parset_NAME" = "--version" ] ; then
# shellcheck disable=SC2006 # shellcheck disable=SC2006
echo "parset 20230522 (GNU parallel `parallel --minversion 1`)" echo "parset 20230523 (GNU parallel `parallel --minversion 1`)"
echo "Copyright (C) 2007-2023 Ole Tange, http://ole.tange.dk and Free Software" echo "Copyright (C) 2007-2023 Ole Tange, http://ole.tange.dk and Free Software"
echo "Foundation, Inc." echo "Foundation, Inc."
echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>" echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>"

2
src/env_parallel.ksh
View file

@ -376,7 +376,7 @@ _parset_main() {
fi fi
if [ "$_parset_NAME" = "--version" ] ; then if [ "$_parset_NAME" = "--version" ] ; then
# shellcheck disable=SC2006 # shellcheck disable=SC2006
echo "parset 20230522 (GNU parallel `parallel --minversion 1`)" echo "parset 20230523 (GNU parallel `parallel --minversion 1`)"
echo "Copyright (C) 2007-2023 Ole Tange, http://ole.tange.dk and Free Software" echo "Copyright (C) 2007-2023 Ole Tange, http://ole.tange.dk and Free Software"
echo "Foundation, Inc." echo "Foundation, Inc."
echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>" echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>"

2
src/env_parallel.mksh
View file

@ -378,7 +378,7 @@ _parset_main() {
fi fi
if [ "$_parset_NAME" = "--version" ] ; then if [ "$_parset_NAME" = "--version" ] ; then
# shellcheck disable=SC2006 # shellcheck disable=SC2006
echo "parset 20230522 (GNU parallel `parallel --minversion 1`)" echo "parset 20230523 (GNU parallel `parallel --minversion 1`)"
echo "Copyright (C) 2007-2023 Ole Tange, http://ole.tange.dk and Free Software" echo "Copyright (C) 2007-2023 Ole Tange, http://ole.tange.dk and Free Software"
echo "Foundation, Inc." echo "Foundation, Inc."
echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>" echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>"

2
src/env_parallel.sh
View file

@ -393,7 +393,7 @@ _parset_main() {
fi fi
if [ "$_parset_NAME" = "--version" ] ; then if [ "$_parset_NAME" = "--version" ] ; then
# shellcheck disable=SC2006 # shellcheck disable=SC2006
echo "parset 20230522 (GNU parallel `parallel --minversion 1`)" echo "parset 20230523 (GNU parallel `parallel --minversion 1`)"
echo "Copyright (C) 2007-2023 Ole Tange, http://ole.tange.dk and Free Software" echo "Copyright (C) 2007-2023 Ole Tange, http://ole.tange.dk and Free Software"
echo "Foundation, Inc." echo "Foundation, Inc."
echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>" echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>"

2
src/env_parallel.zsh
View file

@ -368,7 +368,7 @@ _parset_main() {
fi fi
if [ "$_parset_NAME" = "--version" ] ; then if [ "$_parset_NAME" = "--version" ] ; then
# shellcheck disable=SC2006 # shellcheck disable=SC2006
echo "parset 20230522 (GNU parallel `parallel --minversion 1`)" echo "parset 20230523 (GNU parallel `parallel --minversion 1`)"
echo "Copyright (C) 2007-2023 Ole Tange, http://ole.tange.dk and Free Software" echo "Copyright (C) 2007-2023 Ole Tange, http://ole.tange.dk and Free Software"
echo "Foundation, Inc." echo "Foundation, Inc."
echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>" echo "License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>"

2
src/niceload
View file

@ -26,7 +26,7 @@
use strict; use strict;
use Getopt::Long; use Getopt::Long;
$Global::progname="niceload"; $Global::progname="niceload";
$Global::version = 20230522; $Global::version = 20230523;
Getopt::Long::Configure("bundling","require_order"); Getopt::Long::Configure("bundling","require_order");
get_options_from_array(\@ARGV) || die_usage(); get_options_from_array(\@ARGV) || die_usage();
if($opt::version) { if($opt::version) {

13
src/parallel
View file

@ -2799,7 +2799,7 @@ sub check_invalid_option_combinations() {
sub init_globals() { sub init_globals() {
# Defaults: # Defaults:
$Global::version = 20230522; $Global::version = 20230523;
$Global::progname = 'parallel'; $Global::progname = 'parallel';
$::name = "GNU Parallel"; $::name = "GNU Parallel";
$Global::infinity = 2**31; $Global::infinity = 2**31;
@ -5121,8 +5121,10 @@ sub parse_host_filtering() {
next; next;
} }
if(/\t(perl: warning:|LANGUAGE =|LC_ALL =|LANG =|are supported and installed|Disconnected from|Received disconnect from)/ if(/\t(perl: warning:|LANGUAGE =|LC_ALL =|LANG =|are supported and installed|Disconnected from|Received disconnect from)/
or or
/\tWarning: / /\tWarning: /
or
/\t(Host key fingerprint is|\+-.*-\+|\|.*\|)/
) { ) {
# Skip these (from perl): # Skip these (from perl):
# perl: warning: Setting locale failed. # perl: warning: Setting locale failed.
@ -5137,6 +5139,11 @@ sub parse_host_filtering() {
# Skip these (from ssh): # Skip these (from ssh):
# Warning: Permanently added * to the list of known hosts. # Warning: Permanently added * to the list of known hosts.
# Warning: Identity file * not accessible: * # Warning: Identity file * not accessible: *
# (VisualHostKey=yes)
# Host key fingerprint is SHA256:...
# +--[ED25519 256]--+
# | o |
# +----[SHA256]-----+
} elsif(not defined $nsockets{$col[0]}) { } elsif(not defined $nsockets{$col[0]}) {
$nsockets{$col[0]} = $col[1]; $nsockets{$col[0]} = $col[1];
} elsif(not defined $ncores{$col[0]}) { } elsif(not defined $ncores{$col[0]}) {

2
src/parsort
View file

@ -137,7 +137,7 @@ GetOptions(
"help" => \$opt::dummy, "help" => \$opt::dummy,
) || exit(255); ) || exit(255);
$Global::progname = ($0 =~ m:(^|/)([^/]+)$:)[1]; $Global::progname = ($0 =~ m:(^|/)([^/]+)$:)[1];
$Global::version = 20230522; $Global::version = 20230523;
if($opt::version) { version(); exit 0; } if($opt::version) { version(); exit 0; }
# Remove -D and --parallel=N # Remove -D and --parallel=N
my @s = (grep { ! /^-D$|^--parallel=\S+$/ } my @s = (grep { ! /^-D$|^--parallel=\S+$/ }

2
src/sql
View file

@ -670,7 +670,7 @@ $Global::Initfile && unlink $Global::Initfile;
exit ($err); exit ($err);
sub parse_options { sub parse_options {
$Global::version = 20230522; $Global::version = 20230523;
$Global::progname = 'sql'; $Global::progname = 'sql';
# This must be done first as this may exec myself # This must be done first as this may exec myself

7
testsuite/tests-to-run/parallel-local-ssh8.sh
View file

@ -133,7 +133,12 @@ par_z_csh_environment_variables_set() {
par_filter_host_noise() { par_filter_host_noise() {
echo '### bug #63296: --filter-hosts option gets confused by output from SSH command' echo '### bug #63296: --filter-hosts option gets confused by output from SSH command'
parallel --ssh "ssh -i id_rsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o GlobalKnownHostsFile=/dev/null " -S localhost --nonall --tag --filter-hosts echo OK parallel --ssh "ssh -i id_rsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o GlobalKnownHostsFile=/dev/null " -S localhost --nonall --tag --filter-hosts echo OK
parallel --ssh "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o GlobalKnownHostsFile=/dev/null " -S localhost --nonall --tag --filter-hosts echo OK parallel --ssh "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o GlobalKnownHostsFile=/dev/null " -S localhost --nonall --tag --filter-hosts echo OK
echo '### bug #64237: --filter-hosts and VisualHostKey=yes'
stdout parallel --ssh "ssh -o VisualHostKey=yes" -S localhost --nonall --tag --filter-hosts echo OK |
perl -pe 's/(Host key fingerprint is).*/$1/; s/\t(\+|\|).*\1/\t$1,,,,,,,,,,,,,,,,,$1/;'
parallel --ssh "ssh -o VisualHostKey=yes -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o GlobalKnownHostsFile=/dev/null " -S localhost --nonall --tag --filter-hosts echo OK
} }
export -f $(compgen -A function | grep par_) export -f $(compgen -A function | grep par_)

16
testsuite/wanted-results/parallel-local-ssh8
View file

@ -4,6 +4,22 @@ par_filter_host_noise localhost Warning: Identity file id_rsa not accessible: No
par_filter_host_noise localhost Warning: Permanently added 'localhost' (ED25519) to the list of known hosts. par_filter_host_noise localhost par_filter_host_noise localhost Warning: Permanently added 'localhost' (ED25519) to the list of known hosts. par_filter_host_noise localhost
par_filter_host_noise localhost OK par_filter_host_noise localhost OK
par_filter_host_noise localhost Warning: Permanently added 'localhost' (ED25519) to the list of known hosts. par_filter_host_noise localhost par_filter_host_noise localhost Warning: Permanently added 'localhost' (ED25519) to the list of known hosts. par_filter_host_noise localhost
par_filter_host_noise ### bug #64237: --filter-hosts and VisualHostKey=yes
par_filter_host_noise localhost OK
par_filter_host_noise localhost Host key fingerprint is
par_filter_host_noise localhost +,,,,,,,,,,,,,,,,,+
par_filter_host_noise localhost |,,,,,,,,,,,,,,,,,|
par_filter_host_noise localhost |,,,,,,,,,,,,,,,,,|
par_filter_host_noise localhost |,,,,,,,,,,,,,,,,,|
par_filter_host_noise localhost |,,,,,,,,,,,,,,,,,|
par_filter_host_noise localhost |,,,,,,,,,,,,,,,,,|
par_filter_host_noise localhost |,,,,,,,,,,,,,,,,,|
par_filter_host_noise localhost |,,,,,,,,,,,,,,,,,|
par_filter_host_noise localhost |,,,,,,,,,,,,,,,,,|
par_filter_host_noise localhost |,,,,,,,,,,,,,,,,,|
par_filter_host_noise localhost +,,,,,,,,,,,,,,,,,+ par_filter_host_noise localhost
par_filter_host_noise localhost OK
par_filter_host_noise localhost Warning: Permanently added 'localhost' (ED25519) to the list of known hosts. par_filter_host_noise localhost
par_load_csh ### Gave Word too long. par_load_csh ### Gave Word too long.
par_load_csh a par_load_csh a
par_path_remote_bash bug #47695: How to set $PATH on remote? Bash par_path_remote_bash bug #47695: How to set $PATH on remote? Bash