如何在Perl中提高MySQL查询的性能，在MySQL Workbench中直接执行的查询速度要快1600倍

Question

My MySQL query in Perl takes much longer than the same query in MySQL Workbench. I am trying to improve the performance of the Perl Query to be about the same as the Workbench query.
I am running on Microsoft Windows 10 pro 64-bit, ActivePerl 5.24.0 Build 2400 64-bit, using DBI V1.636, DBD-MySQL 4.033 and MySQL V5.7 64-bit.
There is not any resource constraint that I can find. So InnoDB Buffer Pool – 40% utilized. Table Open Cache Efficiency of 99%. 64GB of RAM, 16 processors running at 3.4Ghz. Most of the queries run through this environment are acceptably fast. There is one class of queries that runs very slowly in Perl, but reasonably quickly in Workbench. Here is an example of the query.

SELECT t1.CSI_ID, t2.Signal_Date, t2.NextGain   
    FROM        
        equity as t1    
    INNER JOIN      
        (equity_signal AS t2, market_boundary AS t3) 
ON 
(t2.equity_CSI_ID = t1.CSI_ID AND 
Yearweek(t3.Signal_Date)= Yearweek(t2.Signal_Date)) 
    WHERE       
        YearWeek(t2.Signal_Date) = 201643   
        AND t2.currency_idCurrency = 'USD'
        AND t1.UseEquity = 1
        AND t1.NoData = 0
        AND t2.Spike = 0
        AND t1.Exchange IN ('NYSE','NASDAQ')
        AND t2.Liquidity>t3.Value 
        AND t3.Currency='USD'
        AND t3.Market='US'
        AND t3.Type='LQ'
        AND t3.Threshold=500
        AND t2.StdDev < 1
        AND (t2.CalcBool & 63) = 63
    ORDER BY (t2.Buy) 
DESC LIMIT 50

Here is the explain diagram for the Query, when it successfully executes. This query in MySQL workbench took 4.047 seconds.

T1 in the query has 29000 rows, 14 columns and is 3.5MB. T2 in the query has 8500000 rows, 34 columns and 1.7GB with 1.2GB Index. T3 in the query has 54000 rows, 7 columns and is 3.5 MB

Explain Statement Results from WorkBench

The explain statement results when executed through Perl are as follows, which is the same tabular result I just received from Workbench:
1 SIMPLE t3 ref PRIMARY PRIMARY 38 const 27354 0.10 Using where; Using temporary; Using filesort
1 SIMPLE t1 ALL PRIMARY 29304 0.20 Using where; Using join buffer (Block Nested Loop)
1 SIMPLE t2 ref PRIMARY,Liquidity,Buy,StdDev PRIMARY 4 investing2.t1.CSI_ID 92 0.56 Using where

During execution of the Query, there are no concurrent or competing processes of my creation accessing or using the tables in question. When I run the same query in Perl, I have MySQL using 13% of the available CPU (across 16 CPU's) for over 100 minutes.

Case 1, Portfolio = 11 YearWeek = 201644
PortDatah Start Sat Nov 5 14:47:00 2016 End Sat Nov 5 16:33:23 2016
Portfolio = 11 YearWeek = 201644
Case 1, Portfolio = 11 YearWeek = 201643
PortDatah Start Sat Nov 5 16:33:23 2016 End Sat Nov 5 18:19:47 2016

So running times of 106 minutes and 23 seconds the first time, and 106 minutes and 24 seconds for the second run. On an entirely different platform, ( MySQL 5.5, 32 GB RAM, Perl 5.16.3) the same code works with comparable performance to the Workbench observed results.
Here are what I think are the relevant the code snippets.

#!D:\perl64\bin
use strict;
use warnings;
use DBI;
use List::Util qw(first);
use threads;
use Date::Simple (':all');
use Switch::Plain;
my @Portfolio;
$Portfolio[11]="SELECT t1.CSI_ID, t2.Signal_Date, t2.NextGain   
    FROM        
        equity as t1    
    INNER JOIN      
        (equity_signal AS t2, market_boundary AS t3) 
ON (t2.equity_CSI_ID = t1.CSI_ID 
AND Yearweek(t3.Signal_Date)=Yearweek(t2.Signal_Date))  
    WHERE   
        YearWeek(t2.Signal_Date) = 201643   
        AND t2.currency_idCurrency = 'USD'
        AND t1.UseEquity = 1
        AND t1.NoData = 0
        AND t2.Spike = 0
        AND t1.Exchange IN ('NYSE','NASDAQ')
        AND t2.Liquidity>t3.Value 
        AND t3.Currency='USD'
        AND t3.Market='US'
        AND t3.Type='LQ'
        AND t3.Threshold=500
        AND t2.StdDev < 1
        AND (t2.CalcBool & 63) = 63
    ORDER BY (t2.Buy) 
DESC LIMIT 50";
my $thread11 = threads->create(\&PortfolioA,11,$Portfolio[11]);
my $return = $thread11->join();

sub PortfolioA {
my $PortID = shift;
    my $PortSQL = shift;
    my $PortDatah = $dbh->prepare($PortSQL)
        or die "Couldn't prepare statement: " . $dbh->errstr;
$Starttime=localtime();
    $PortDatah->execute($YearWeek)
        or die "Couldn't execute statement: " . $dbh->errstr;
    $Endtime=localtime();
    print "PortDatah Start $Starttime  End $Endtime \n";    

I have looked for errors in the logs, I have tried to tune the MySQL Server. So InnoDB Buffer Pool is 12 GB, the Join Buffer Size is 4GB, Sort Buffer Size is 4GB, there are 8 InnoDB page cleaners, Query Cache Size is 2GB. I added an Index on Signal_Date for Equity_Signal which is the largest table in the database. I am considering trying to install previous versions of Perl and MySQL to see if that resolves things. I just don't see what I am missing.

Here is the results from a single query using trace level 4.

    DBI::st=HASH(0x3081000) trace level set to 0x0/4 (DBI @ 0x0/0) in DBI 1.636-ithread (pid 10776)
-> execute for DBD::mysql::st (DBI::st=HASH(0x3081318)~0x3081000 '201644') thr#26278d8
Called: dbd_bind_ph
-> dbd_st_execute for 03081360
>- dbd_st_free_result_sets
<- dbd_st_free_result_sets RC -1
<- dbd_st_free_result_sets
mysql_st_internal_execute MYSQL_VERSION_ID 50172
>parse_params statement SELECT t1.CSI_ID, t2.Signal_Date, t2.NextGain   
                        FROM        
                            equity as t1    
                        INNER JOIN      
                            (equity_signal AS t2, market_boundary AS t3) ON (t2.equity_CSI_ID = t1.CSI_ID AND Yearweek(t3.Signal_Date)=Yearweek(t2.Signal_Date))    
                        WHERE       
                            YearWeek(t2.Signal_Date) = ?    
                            AND t2.currency_idCurrency = 'USD'
                            AND t1.UseEquity = 1
                            AND t1.NoData = 0
                            AND t2.Spike = 0
                            AND t1.Exchange IN ('NYSE','NASDAQ')
                            AND t2.Liquidity>t3.Value 
                            AND t3.Currency='USD'
                            AND t3.Market='US'
                            AND t3.Type='LQ'
                            AND t3.Threshold=500
                            AND t2.StdDev < 1
                            AND (t2.CalcBool & 63) = 63
                        ORDER BY t2.Buy DESC        
                        LIMIT 50
Binding parameters: SELECT t1.CSI_ID, t2.Signal_Date, t2.NextGain   
                        FROM        
                            equity as t1    
                        INNER JOIN      
                            (equity_signal AS t2, market_boundary AS t3) ON (t2.equity_CSI_ID = t1.CSI_ID AND Yearweek(t3.Signal_Date)=Yearweek(t2.Signal_Date))    
                        WHERE       
                            YearWeek(t2.Signal_Date) = '201644' 
                            AND t2.currency_idCurrency = 'USD'
                            AND t1.UseEquity = 1
                            AND t1.NoData = 0
                            AND t2.Spike = 0
                            AND t1.Exchange IN ('NYSE','NASDAQ')
                            AND t2.Liquidity>t3.Value 
                            AND t3.Currency='USD'
                            AND t3.Market='US'
                            AND t3.Type='LQ'
                            AND t3.Threshold=500
                            AND t2.StdDev < 1
                            AND (t2.CalcBool & 63) = 63
                        ORDER BY t2.Buy DESC        
                        LIMIT 50
DBI::st=HASH(0x2f71530) trace level set to 0x0/4 (DBI @ 0x0/0) in DBI 1.636-ithread (pid 11064)
-> execute for DBD::mysql::st (DBI::st=HASH(0x2f71848)~0x2f71530 '201644') thr#25178d8
Called: dbd_bind_ph
-> dbd_st_execute for 02f71890
>- dbd_st_free_result_sets
<- dbd_st_free_result_sets RC -1
<- dbd_st_free_result_sets
mysql_st_internal_execute MYSQL_VERSION_ID 50172
>parse_params statement SELECT t1.CSI_ID, t2.Signal_Date, t2.NextGain   
                        FROM        
                            equity as t1    
                        INNER JOIN      
                            (equity_signal AS t2, market_boundary AS t3) ON (t2.equity_CSI_ID = t1.CSI_ID AND Yearweek(t3.Signal_Date)=Yearweek(t2.Signal_Date))    
                        WHERE       
                            YearWeek(t2.Signal_Date) = ?    
                            AND t2.currency_idCurrency = 'USD'
                            AND t1.UseEquity = 1
                            AND t1.NoData = 0
                            AND t2.Spike = 0
                            AND t1.Exchange IN ('NYSE','NASDAQ')
                            AND t2.Liquidity>t3.Value 
                            AND t3.Currency='USD'
                            AND t3.Market='US'
                            AND t3.Type='LQ'
                            AND t3.Threshold=500
                            AND t2.StdDev < 1
                            AND (t2.CalcBool & 63) = 63
                        ORDER BY t2.Buy DESC        
                        LIMIT 50
Binding parameters: SELECT t1.CSI_ID, t2.Signal_Date, t2.NextGain   
                        FROM        
                            equity as t1    
                        INNER JOIN      
                            (equity_signal AS t2, market_boundary AS t3) ON (t2.equity_CSI_ID = t1.CSI_ID AND Yearweek(t3.Signal_Date)=Yearweek(t2.Signal_Date))    
                        WHERE       
                            YearWeek(t2.Signal_Date) = '201644' 
                            AND t2.currency_idCurrency = 'USD'
                            AND t1.UseEquity = 1
                            AND t1.NoData = 0
                            AND t2.Spike = 0
                            AND t1.Exchange IN ('NYSE','NASDAQ')
                            AND t2.Liquidity>t3.Value 
                            AND t3.Currency='USD'
                            AND t3.Market='US'
                            AND t3.Type='LQ'
                            AND t3.Threshold=500
                            AND t2.StdDev < 1
                            AND (t2.CalcBool & 63) = 63
                        ORDER BY t2.Buy DESC        
                        LIMIT 50
 <- dbd_st_execute returning imp_sth->row_num 19
    <- execute= ( 19 ) [1 items] at C:/workspace/DIY Investing/CSI/I2Portfolio1debug.pl line 873 via  at C:/workspace/DIY Investing/CSI/I2Portfolio1debug.pl line 700
    -> fetchrow_array for DBD::mysql::st (DBI::st=HASH(0x2f71848)~0x2f71530) thr#25178d8
    -> dbd_st_fetch
        dbd_st_fetch for 02f71890, chopblanks 0
    dbd_st_fetch result set details
    imp_sth->result=03a71110
    mysql_num_fields=3
    mysql_num_rows=19
    mysql_affected_rows=19
    dbd_st_fetch for 02f71890, currow= 1
    <- dbd_st_fetch, 3 cols
    <- fetchrow_array= ( '9825' '2016-11-04' '0' ) [3 items] row1 at C:/workspace/DIY Investing/CSI/I2Portfolio1debug.pl line 878 via  at C:/workspace/DIY Investing/CSI/I2Portfolio1debug.pl line 700
    -> fetchrow_array for DBD::mysql::st (DBI::st=HASH(0x2f71848)~0x2f71530) thr#25178d8
    -> dbd_st_fetch
        dbd_st_fetch for 02f71890, chopblanks 0
    dbd_st_fetch result set details
    imp_sth->result=03a71110
    mysql_num_fields=3
    mysql_num_rows=19
    mysql_affected_rows=1
    dbd_st_fetch for 02f71890, currow= 2
    <- dbd_st_fetch, 3 cols
    -> fetchrow_array for DBD::mysql::st (DBI::st=HASH(0x2f71848)~0x2f71530) thr#25178d8
    -> dbd_st_fetch
        dbd_st_fetch for 02f71890, chopblanks 0
    dbd_st_fetch result set details
    imp_sth->result=03a71110
    mysql_num_fields=3
    mysql_num_rows=19
    mysql_affected_rows=1
    dbd_st_fetch for 02f71890, currow= 3
    <- dbd_st_fetch, 3 cols
    -> fetchrow_array for DBD::mysql::st (DBI::st=HASH(0x2f71848)~0x2f71530) thr#25178d8
    -> dbd_st_fetch
        dbd_st_fetch for 02f71890, chopblanks 0
    dbd_st_fetch result set details
    imp_sth->result=03a71110
    mysql_num_fields=3
    mysql_num_rows=19
    mysql_affected_rows=1
    dbd_st_fetch for 02f71890, currow= 4
    <- dbd_st_fetch, 3 cols
    -> fetchrow_array for DBD::mysql::st (DBI::st=HASH(0x2f71848)~0x2f71530) thr#25178d8
    -> dbd_st_fetch
        dbd_st_fetch for 02f71890, chopblanks 0
    dbd_st_fetch result set details
    imp_sth->result=03a71110
    mysql_num_fields=3
    mysql_num_rows=19
    mysql_affected_rows=1
    dbd_st_fetch for 02f71890, currow= 5
    <- dbd_st_fetch, 3 cols
    -> fetchrow_array for DBD::mysql::st (DBI::st=HASH(0x2f71848)~0x2f71530) thr#25178d8
    -> dbd_st_fetch
        dbd_st_fetch for 02f71890, chopblanks 0
    dbd_st_fetch result set details
    imp_sth->result=03a71110
    mysql_num_fields=3
    mysql_num_rows=19
    mysql_affected_rows=1
    dbd_st_fetch for 02f71890, currow= 6
    <- dbd_st_fetch, 3 cols
    -> fetchrow_array for DBD::mysql::st (DBI::st=HASH(0x2f71848)~0x2f71530) thr#25178d8
    -> dbd_st_fetch
        dbd_st_fetch for 02f71890, chopblanks 0
    dbd_st_fetch result set details
    imp_sth->result=03a71110
    mysql_num_fields=3
    mysql_num_rows=19
    mysql_affected_rows=1
    dbd_st_fetch for 02f71890, currow= 7
    <- dbd_st_fetch, 3 cols
    -> fetchrow_array for DBD::mysql::st (DBI::st=HASH(0x2f71848)~0x2f71530) thr#25178d8
    -> dbd_st_fetch
        dbd_st_fetch for 02f71890, chopblanks 0
    dbd_st_fetch result set details
    imp_sth->result=03a71110
    mysql_num_fields=3
    mysql_num_rows=19
    mysql_affected_rows=1
    dbd_st_fetch for 02f71890, currow= 8
    <- dbd_st_fetch, 3 cols
    -> fetchrow_array for DBD::mysql::st (DBI::st=HASH(0x2f71848)~0x2f71530) thr#25178d8
    -> dbd_st_fetch
        dbd_st_fetch for 02f71890, chopblanks 0
    dbd_st_fetch result set details
    imp_sth->result=03a71110
    mysql_num_fields=3
    mysql_num_rows=19
    mysql_affected_rows=1
    dbd_st_fetch for 02f71890, currow= 9
    <- dbd_st_fetch, 3 cols
    -> fetchrow_array for DBD::mysql::st (DBI::st=HASH(0x2f71848)~0x2f71530) thr#25178d8
    -> dbd_st_fetch
        dbd_st_fetch for 02f71890, chopblanks 0
    dbd_st_fetch result set details
    imp_sth->result=03a71110
    mysql_num_fields=3
    mysql_num_rows=19
    mysql_affected_rows=1
    dbd_st_fetch for 02f71890, currow= 10
    <- dbd_st_fetch, 3 cols
    -> fetchrow_array for DBD::mysql::st (DBI::st=HASH(0x2f71848)~0x2f71530) thr#25178d8
    -> dbd_st_fetch
        dbd_st_fetch for 02f71890, chopblanks 0
    dbd_st_fetch result set details
    imp_sth->result=03a71110
    mysql_num_fields=3
    mysql_num_rows=19
    mysql_affected_rows=1
    dbd_st_fetch for 02f71890, currow= 11
    <- dbd_st_fetch, 3 cols
    -> fetchrow_array for DBD::mysql::st (DBI::st=HASH(0x2f71848)~0x2f71530) thr#25178d8
    -> dbd_st_fetch
        dbd_st_fetch for 02f71890, chopblanks 0
    dbd_st_fetch result set details
    imp_sth->result=03a71110
    mysql_num_fields=3
    mysql_num_rows=19
    mysql_affected_rows=1
    dbd_st_fetch for 02f71890, currow= 12
    <- dbd_st_fetch, 3 cols
    -> fetchrow_array for DBD::mysql::st (DBI::st=HASH(0x2f71848)~0x2f71530) thr#25178d8
    -> dbd_st_fetch
        dbd_st_fetch for 02f71890, chopblanks 0
    dbd_st_fetch result set details
    imp_sth->result=03a71110
    mysql_num_fields=3
    mysql_num_rows=19
    mysql_affected_rows=1
    dbd_st_fetch for 02f71890, currow= 13
    <- dbd_st_fetch, 3 cols
    -> fetchrow_array for DBD::mysql::st (DBI::st=HASH(0x2f71848)~0x2f71530) thr#25178d8
    -> dbd_st_fetch
        dbd_st_fetch for 02f71890, chopblanks 0
    dbd_st_fetch result set details
    imp_sth->result=03a71110
    mysql_num_fields=3
    mysql_num_rows=19
    mysql_affected_rows=1
    dbd_st_fetch for 02f71890, currow= 14
    <- dbd_st_fetch, 3 cols
    -> fetchrow_array for DBD::mysql::st (DBI::st=HASH(0x2f71848)~0x2f71530) thr#25178d8
    -> dbd_st_fetch
        dbd_st_fetch for 02f71890, chopblanks 0
    dbd_st_fetch result set details
    imp_sth->result=03a71110
    mysql_num_fields=3
    mysql_num_rows=19
    mysql_affected_rows=1
    dbd_st_fetch for 02f71890, currow= 15
    <- dbd_st_fetch, 3 cols
    -> fetchrow_array for DBD::mysql::st (DBI::st=HASH(0x2f71848)~0x2f71530) thr#25178d8
    -> dbd_st_fetch
        dbd_st_fetch for 02f71890, chopblanks 0
    dbd_st_fetch result set details
    imp_sth->result=03a71110
    mysql_num_fields=3
    mysql_num_rows=19
    mysql_affected_rows=1
    dbd_st_fetch for 02f71890, currow= 16
    <- dbd_st_fetch, 3 cols
    -> fetchrow_array for DBD::mysql::st (DBI::st=HASH(0x2f71848)~0x2f71530) thr#25178d8
    -> dbd_st_fetch
        dbd_st_fetch for 02f71890, chopblanks 0
    dbd_st_fetch result set details
    imp_sth->result=03a71110
    mysql_num_fields=3
    mysql_num_rows=19
    mysql_affected_rows=1
    dbd_st_fetch for 02f71890, currow= 17
    <- dbd_st_fetch, 3 cols
    -> fetchrow_array for DBD::mysql::st (DBI::st=HASH(0x2f71848)~0x2f71530) thr#25178d8
    -> dbd_st_fetch
        dbd_st_fetch for 02f71890, chopblanks 0
    dbd_st_fetch result set details
    imp_sth->result=03a71110
    mysql_num_fields=3
    mysql_num_rows=19
    mysql_affected_rows=1
    dbd_st_fetch for 02f71890, currow= 18
    <- dbd_st_fetch, 3 cols
    -> fetchrow_array for DBD::mysql::st (DBI::st=HASH(0x2f71848)~0x2f71530) thr#25178d8
    -> dbd_st_fetch
        dbd_st_fetch for 02f71890, chopblanks 0
    dbd_st_fetch result set details
    imp_sth->result=03a71110
    mysql_num_fields=3
    mysql_num_rows=19
    mysql_affected_rows=1
    dbd_st_fetch for 02f71890, currow= 19
    <- dbd_st_fetch, 3 cols
    -> fetchrow_array for DBD::mysql::st (DBI::st=HASH(0x2f71848)~0x2f71530) thr#25178d8
    -> dbd_st_fetch
        dbd_st_fetch for 02f71890, chopblanks 0
    dbd_st_fetch result set details
    imp_sth->result=03a71110
    mysql_num_fields=3
    mysql_num_rows=19
    mysql_affected_rows=1
    dbd_st_fetch for 02f71890, currow= 20
    dbd_st_fetch, no more rows to fetch
--> dbd_st_finish
    >- dbd_st_free_result_sets
    <- dbd_st_free_result_sets RC -1
    <- dbd_st_free_result_sets

I'll try to re-work the queries to avoid Yearweek (I was using Yearweek, because the data is weekly, but sometimes with Friday as the last day and sometimes as Thursday, and I want to compare weeks between markets - just not in this query example), but I should be able to use Interval as well. Also, I'm thinking of trying this whole thing in Python or Ruby, as opposed to installing back-level products.

Here are the Create Table Entries for each table:

t1 - equity

CREATE TABLE `equity` (
  `CSI_ID` int(11) NOT NULL,
  `Symbol` varchar(45) NOT NULL,
  `EquityName` varchar(45) NOT NULL,
  `Exchange` varchar(45) NOT NULL,
  `currency_idCurrency` varchar(6) NOT NULL,
  `PriceForm` int(11) DEFAULT NULL,
  `StartDate` date DEFAULT NULL,
  `LastUpdate` date DEFAULT NULL,
  `industry_idSector` varchar(10) NOT NULL,
  `industry_idIndustry` varchar(10) NOT NULL,
  `NoData` tinyint(1) NOT NULL DEFAULT '0',
  `UseEquity` tinyint(1) NOT NULL DEFAULT '1',
  `Category` varchar(10) DEFAULT NULL,
  `OutShares` float DEFAULT NULL,
  PRIMARY KEY (`CSI_ID`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

t2 equity_signal (I'm not showing the unused columns in this table)

CREATE TABLE `equity_signal` (
  `equity_CSI_ID` int(11) NOT NULL,
  `Signal_Date` date NOT NULL,
  `currency_idCurrency` varchar(6) NOT NULL,
  `Signal_Sequence` int(11) NOT NULL,
  `Adjusted_Close` float NOT NULL,
  `Liquidity` float NOT NULL,
  `Gain` float NOT NULL,
  `NextGain` float DEFAULT NULL,
  `StdDev` double NOT NULL,
  `Spike` tinyint(1) NOT NULL,
  `Buy` float NOT NULL,
  `CalcBool` int(11) NOT NULL,
  PRIMARY KEY (`equity_CSI_ID`,`Signal_Date`,`currency_idCurrency`),
  KEY `Liquid` (`currency_idCurrency`,`Signal_Date`,`Liquidity`),
  KEY `Buy` (`currency_idCurrency`,`Signal_Date`,`Buy`),
  KEY `STD` (`currency_idCurrency`,`Signal_Date`,`StdDev`),
  KEY `Signal_Date` (`Signal_Date`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

And t3 as market_boundary

CREATE TABLE `market_boundary` (
  `Market` varchar(12) NOT NULL,
  `Signal_Date` date NOT NULL,
  `Currency` varchar(6) NOT NULL,
  `Type` varchar(6) NOT NULL,
  `Threshold` int(11) NOT NULL,
  `Percent` float NOT NULL,
  `Value` float NOT NULL,
  PRIMARY KEY (`Market`,`Signal_Date`,`Currency`,`Type`,`Threshold`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

Answer 1

Change the yearweek tests to be

WHERE Signal_date >= '2016-10-22'
  AND Signal_date  < '2016-10-22' + INTERVAL 1 WEEK

(or whatever the correct date is.)

And separately check test t3.Signal_date and t2.Signal_date that way.

Then build a suitable composite index on each table -- ending with Signal_date .

After seeing SHOW CREATE TABLE , I may have more advice.

Back to your question... I don't have a good answer of why one client would run soooo much faster than the other. (My caching comment is not sufficient to explain 1600x.) But, fixing the index should help both clients, perhaps more than 1600x.

Answer 2

I converted the SQL into a stored procedure, and I am now just using Perl to call the stored procedure. The stored procedure doesn't just retrieve the results, it also writes the results to the next table. I'm about to try this threaded, but the current response time is about 8 seconds for the query, and the insert to be completed. I'm not sure what the original problem was, it may have been a cursor issue associated with fact it was threaded, or a cursor issue generally as my tests with an un-threaded procedure didn't work either.