Excel ADODB 连接到 csv 不返回所有记录

Question

I'm using Excel VBA code to connect to a CSV file (24,179,689 rows) via ADODB connection. The macro runs and gets data from the excel with a specific filter on one column. With this filter that I'm trying now, it should return about 1500 rows of data.

I have checked this by loading the CSV manually somewhere else, and the data is actually there. But when I use the ADODB connection, my recordset remains empty.

I did some extra test: count(*) on the complete CSV file, and there I see the error: it only returns 155,535 rows. So probably the specific filter that I'm applying is not in that data and therefore it returns 0 rows.

This is my code:

Public adoConn As ADODB.Connection
Public adoRS As ADODB.Recordset

Sub getdata()

Set adoConn = New ADODB.Connection
Set adoRS = New ADODB.Recordset

Dim rawFile As String
Dim strSQL As String

'The xlsx file to treat as a database
rawFile = "myPathName"

'Open a connection to the workbook
sconnect = "Provider=Microsoft.ACE.OLEDB.16.0;Data Source=" & rawFile & ";Extended Properties='text;HDR=YES;FMT=Delimited'"

'Write the SQL necessary to get the data you want 
sql2 = "SELECT count(*) from [MyFileName.csv]"

'Now we open up our recordset using the connection and the sql statement
adoRS.Open sql2, adoConn, adOpenStatic

Debug.Print (adoRS.EOF)

'Last, we dump the results in this viz sheet
Blad1.Range("A1").CopyFromRecordset adoRS

adoRS.Close
adoConn.Close

End Sub

So then it returns the 155,535.

I also tried by creating a ADODB command and not using the connection as above. Or with connection timeouts. No results.

Is this a memory issue or something else? How can it be solved?

Answer 1

Please try this and see if it does what you want. Also, set a Reference to Microsoft ActiveX Data Objects 2.8 Library.

Sub sbADO()
Dim sSQLQry As String
Dim ReturnArray

Dim Conn As New ADODB.Connection
Dim mrs As New ADODB.Recordset

Dim DBPath As String, sconnect As String

DBPath = "C:\your_path_here\"

sconnect = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & DBPath & ";Extended Properties='text;HDR=YES;FMT=Delimited'"

Conn.Open sconnect
    sSQLSting = "SELECT * From CSV1.csv WHERE ID = 2"
    mrs.Open sSQLSting, Conn
        ActiveSheet.Range("A2").CopyFromRecordset mrs
    'Close Recordset
    mrs.Close

Conn.Close

My CSV looks like this.

I have a little over a million rows in my CSV; I can't put in 23 million rows, Anyway, in my test, with just over a million rows, I got the exact results that I expected, in about 1 second, and my computer is super-super-super-slow!!