importing csv file in powershell

Question

I have an issue with importing csv files in powershell. I am using following commands for read csv files.

$CsvFilePath=“D:\MyShareOuts\*.csv”

$DataImport=Import-Csv -Path (Get-ChildItem $CsvFilePath)|Out-DataTable

Write-DataTable -ServerInstance $server -Database $Database -TableName $Table -Username $Username -Password $Password -Data $DataImport

Since the CsvFilePath contain thousands of files with large size, importing taking long time and powershell stop working or hang for long time on import-csv code line. Finally importing not happen. Can any one suggest valuable solution for make it happen successfully.

Answer 1

What about trying to process files one by one instead of 1000s in on go?

$CsvFilePath=“D:\MyShareOuts\*.csv”

Get-ChildItem $CsvFilePath | Foreach-Object {

    $DataImport = Import-Csv $_.FullName | Out-DataTable

    Write-DataTable -ServerInstance $server -Database $Database -TableName $Table -Username $Username -Password $Password -Data $DataImport

}

I haven't used Out-/Write-Datable , but it might also support something like this(very untested):

$CsvFilePath=“D:\MyShareOuts\*.csv”

Get-ChildItem $CsvFilePath |
Import-Csv |
Out-DataTable | Foreach-Object {

    Write-DataTable -ServerInstance $server -Database $Database -TableName $Table -Username $Username -Password $Password -Data $_

}