sql: sql:sql::alias now works (with testsuite)

This commit is contained in:
Ole Tange 2011-03-09 16:23:53 +01:00
parent 66c0168f4d
commit a5a4d67686
5 changed files with 33 additions and 24 deletions

View file

@ -1307,11 +1307,11 @@ Let us assume a website stores images like:
where YYYYMMDD is the date and ## is the number 01-10. This will where YYYYMMDD is the date and ## is the number 01-10. This will
generate the past 30 days as YYYYMMDD: generate the past 30 days as YYYYMMDD:
B<seq 1 30 | parallel date -d '"today -{} days"' +%Y%m%d> B<seq 30 | parallel date -d '"today -{} days"' +%Y%m%d>
Based on this we can let GNU B<parallel> generate 10 B<wget>s per day: Based on this we can let GNU B<parallel> generate 10 B<wget>s per day:
I<the above> B<| parallel -I {o} seq -w 1 10 "|" parallel wget I<the above> B<| parallel -I {o} seq -w 10 "|" parallel wget
http://www.example.com/path/to/{o}_{}.jpg> http://www.example.com/path/to/{o}_{}.jpg>
=head1 EXAMPLE: Rewriting a for-loop and a while-read-loop =head1 EXAMPLE: Rewriting a for-loop and a while-read-loop
@ -1435,20 +1435,20 @@ handy).
To run B<echo> on B<server.example.com>: To run B<echo> on B<server.example.com>:
seq 1 10 | parallel --sshlogin server.example.com echo seq 10 | parallel --sshlogin server.example.com echo
To run commands on more than one remote computer run: To run commands on more than one remote computer run:
seq 1 10 | parallel --sshlogin server.example.com,server2.example.net echo seq 10 | parallel --sshlogin server.example.com,server2.example.net echo
Or: Or:
seq 1 10 | parallel --sshlogin server.example.com \ seq 10 | parallel --sshlogin server.example.com \
--sshlogin server2.example.net echo --sshlogin server2.example.net echo
If the login username is I<foo> on I<server2.example.net> use: If the login username is I<foo> on I<server2.example.net> use:
seq 1 10 | parallel --sshlogin server.example.com \ seq 10 | parallel --sshlogin server.example.com \
--sshlogin foo@server2.example.net echo --sshlogin foo@server2.example.net echo
To distribute the commands to a list of computers, make a file To distribute the commands to a list of computers, make a file
@ -1460,7 +1460,7 @@ I<mycomputers> with all the computers:
Then run: Then run:
seq 1 10 | parallel --sshloginfile mycomputers echo seq 10 | parallel --sshloginfile mycomputers echo
To include the local computer add the special sshlogin ':' to the list: To include the local computer add the special sshlogin ':' to the list:
@ -1477,7 +1477,7 @@ If the number of CPU cores on the remote computers is not identified
correctly the number of CPU cores can be added in front. Here the correctly the number of CPU cores can be added in front. Here the
computer has 8 CPU cores. computer has 8 CPU cores.
seq 1 10 | parallel --sshlogin 8/server.example.com echo seq 10 | parallel --sshlogin 8/server.example.com echo
=head1 EXAMPLE: Transferring of files =head1 EXAMPLE: Transferring of files
@ -1573,7 +1573,7 @@ B<parallel -a <(seq 6) -a <(seq 6 -1 1) echo>
Convert files from all subdirs to PNG-files with consecutive numbers Convert files from all subdirs to PNG-files with consecutive numbers
(useful for making input PNG's for B<ffmpeg>): (useful for making input PNG's for B<ffmpeg>):
B<parallel -a <(find . -type f | sort) -a <(seq 1 $(find . -type f|wc -l)) convert {1} {2}.png> B<parallel -a <(find . -type f | sort) -a <(seq $(find . -type f|wc -l)) convert {1} {2}.png>
Alternative version: Alternative version:
@ -1606,7 +1606,7 @@ B<parallel -a table_file.tsv --trim n --colsep '\t' cmd -o {2} -i {1}>
If you want to run the same command with the same arguments 10 times If you want to run the same command with the same arguments 10 times
in parallel you can do: in parallel you can do:
B<seq 1 10 | parallel -n0 my_command my_args> B<seq 10 | parallel -n0 my_command my_args>
=head1 EXAMPLE: Working as cat | sh. Resource inexpensive jobs and evaluation =head1 EXAMPLE: Working as cat | sh. Resource inexpensive jobs and evaluation
@ -1684,7 +1684,7 @@ A mutex is a counting semaphore allowing only one job to run. This
will edit the file I<myfile> and prepends the file with lines with the will edit the file I<myfile> and prepends the file with lines with the
numbers 1 to 3. numbers 1 to 3.
seq 1 3 | parallel sem sed -i -e 'i{}' myfile seq 3 | parallel sem sed -i -e 'i{}' myfile
As I<myfile> can be very big it is important only one process edits As I<myfile> can be very big it is important only one process edits
the file at the same time. the file at the same time.
@ -1692,7 +1692,7 @@ the file at the same time.
Name the semaphore to have multiple different semaphores active at the Name the semaphore to have multiple different semaphores active at the
same time: same time:
seq 1 3 | parallel sem --id mymutex sed -i -e 'i{}' myfile seq 3 | parallel sem --id mymutex sed -i -e 'i{}' myfile
=head1 EXAMPLE: Start editor with filenames from stdin (standard input) =head1 EXAMPLE: Start editor with filenames from stdin (standard input)
@ -1787,7 +1787,7 @@ $-sign. Here is an example using $PARALLEL_SEQ. This variable is set
by GNU B<parallel> itself, so the evaluation of the $ must be done by by GNU B<parallel> itself, so the evaluation of the $ must be done by
the sub shell started by GNU B<parallel>: the sub shell started by GNU B<parallel>:
B<seq 1 10 | parallel -N2 echo seq:\$PARALLEL_SEQ arg1:{1} arg2:{2}> B<seq 10 | parallel -N2 echo seq:\$PARALLEL_SEQ arg1:{1} arg2:{2}>
If the variable is set before GNU B<parallel> starts you can do this: If the variable is set before GNU B<parallel> starts you can do this:
@ -1822,12 +1822,12 @@ Prints: B<test two spaces between each word>
$$ is the shell variable containing the process id of the shell. This $$ is the shell variable containing the process id of the shell. This
will print the process id of the shell running GNU B<parallel>: will print the process id of the shell running GNU B<parallel>:
B<seq 1 10 | parallel echo $$> B<seq 10 | parallel echo $$>
And this will print the process ids of the sub shells started by GNU And this will print the process ids of the sub shells started by GNU
B<parallel>. B<parallel>.
B<seq 1 10 | parallel echo \$\$> B<seq 10 | parallel echo \$\$>
If the special characters should not be evaluated by the sub shell If the special characters should not be evaluated by the sub shell
then you need to protect it against evaluation from both the shell then you need to protect it against evaluation from both the shell
@ -1936,7 +1936,7 @@ shell.
B<Example:> B<Example:>
B<seq 1 10 | parallel -N2 echo seq:'$'PARALLEL_SEQ arg1:{1} arg2:{2}> B<seq 10 | parallel -N2 echo seq:'$'PARALLEL_SEQ arg1:{1} arg2:{2}>
=item $TMPDIR =item $TMPDIR
@ -2479,7 +2479,7 @@ middleman(mdm) is also a tool for running jobs in parallel.
Here are the shellscripts of http://mdm.berlios.de/usage.html ported Here are the shellscripts of http://mdm.berlios.de/usage.html ported
to GNU B<parallel>: to GNU B<parallel>:
B<seq 1 19 | parallel buffon -o - | sort -n >>B< result> B<seq 19 | parallel buffon -o - | sort -n >>B< result>
B<cat files | parallel cmd> B<cat files | parallel cmd>

15
src/sql
View file

@ -55,7 +55,9 @@ Example: echo 'SELECT * FROM foo;' | sql mysql:///
=item B<--dbsize> =item B<--dbsize>
Size of database. Show the size of the database on disk. Size of database. Show the size of the database on disk. For Oracle
this requires access to read the table I<dba_data_files> - the user
I<system> has that.
=item B<--help> =item B<--help>
@ -102,7 +104,7 @@ still prints number of rows found.
The string following -p will be given to the database connection The string following -p will be given to the database connection
program as arguments. Multiple -p's will be joined with program as arguments. Multiple -p's will be joined with
space. Example: space. Example: pass '-U' and the user name to the program:
I<-p "-U scott"> can also be written I<-p -U -p scott>. I<-p "-U scott"> can also be written I<-p -U -p scott>.
@ -162,7 +164,7 @@ For this to work B<--shebang> or B<-Y> must be set as the first option.
=head1 DBURL =head1 DBURL
A DBURL has the following syntax: A DBURL has the following syntax:
[sql:]vendor://[user[:password]@][host][:port]/[database][?sqlquery] [sql:]vendor://[[user][:password]@][host][:port]/[database][?sqlquery]
To quote special characters use %-encoding specified in To quote special characters use %-encoding specified in
http://tools.ietf.org/html/rfc3986#section-2.1 (E.g. a password http://tools.ietf.org/html/rfc3986#section-2.1 (E.g. a password
@ -267,14 +269,14 @@ B<SELECT * FROM foo;>
Then do: Then do:
B<chmod 755 demosql; ./demosql> B<chmod +x demosql; ./demosql>
=head2 Use --colsep to process multiple columns =head2 Use --colsep to process multiple columns
Use GNU B<parallel>'s B<--colsep> to separate columns: Use GNU B<parallel>'s B<--colsep> to separate columns:
B<sql -s '\t' :mydburl 'SELECT * FROM foo;' | parallel --colsep '\t' do_stuff {4} {1}> B<sql -s '\t' :myalias 'SELECT * FROM foo;' | parallel --colsep '\t' do_stuff {4} {1}>
=head2 Retry if the connection fails =head2 Retry if the connection fails
@ -282,7 +284,7 @@ B<sql -s '\t' :mydburl 'SELECT * FROM foo;' | parallel --colsep '\t' do_stuff {4
If the access to the database fails occationally B<--retries> can help If the access to the database fails occationally B<--retries> can help
make sure the query succeeds: make sure the query succeeds:
B<sql --retries 5 :myalias 'SELECT * FROM really_big_table;'> B<sql --retries 5 :myalias 'SELECT * FROM really_big_foo;'>
=head2 Get info about the running database system =head2 Get info about the running database system
@ -834,6 +836,7 @@ sub Usage {
sub get_alias { sub get_alias {
my $alias = shift; my $alias = shift;
$alias =~ s/^(sql:)*//; # Accept aliases prepended with sql:
if ($alias !~ /^:/) { if ($alias !~ /^:/) {
return $alias; return $alias;
} }

View file

@ -49,6 +49,9 @@ echo :sqlunittest mysql://sqlunittest:CB5A1FFFA5A@localhost:3306/sqlunittest >>
perl -i -ne '$seen{$_}++ || print' ~/.sql/aliases perl -i -ne '$seen{$_}++ || print' ~/.sql/aliases
sql :sqlunittest "SELECT 'Yes it does' as 'Test if .sql/aliases works';" sql :sqlunittest "SELECT 'Yes it does' as 'Test if .sql/aliases works';"
echo "### Test sql:sql::alias"
sql sql:sql::sqlunittest "SELECT 'Yes it works' as 'Test sql:sql::alias';"
echo "### Test --noheaders --no-headers -n" echo "### Test --noheaders --no-headers -n"
sql -n :sqlunittest 'select * from unittest order by id' \ sql -n :sqlunittest 'select * from unittest order by id' \
| parallel -k --colsep '\t' echo {2} {1} | parallel -k --colsep '\t' echo {2} {1}

View file

@ -10,7 +10,7 @@ echo '### Test --gnu'
parallel -k --gnu echo ::: 1 2 3 -- a b c parallel -k --gnu echo ::: 1 2 3 -- a b c
echo "### test global config" echo "### test global config"
echo /etc/parallel/config | sudo parallel "echo --tollef > " echo /etc/parallel/config | sudo parallel "mkdir -p /etc/parallel; echo --tollef > "
parallel -k echo -- 1 2 3 ::: a b c parallel -k echo -- 1 2 3 ::: a b c
parallel -k --gnu echo ::: 1 2 3 -- a b c parallel -k --gnu echo ::: 1 2 3 -- a b c
echo --gnu > ~/.parallel/config echo --gnu > ~/.parallel/config

View file

@ -16,6 +16,9 @@ Yes it does
### Test .sql/aliases ### Test .sql/aliases
Test if .sql/aliases works Test if .sql/aliases works
Yes it does Yes it does
### Test sql:sql::alias
Test sql:sql::alias
Yes it works
### Test --noheaders --no-headers -n ### Test --noheaders --no-headers -n
abc 1 abc 1
def 3 def 3