PowerShell script to replace commas within double quotes with nothing

Question

I have a comma separated CSV file, where I intend to replace the commas in double quotes to nothing and also replace double quotes with nothing:

^{Editor's note : The original form of this question asked to "change [the] delimiter to pipe" ( | ), which is no longer a requirement; gms0ulman's answer was written when it still was.}

$inform = Get-Content C:\test.csv
$inform | % { 
$info = $_.ToString().Replace(",","") 
$var = $info 
$var | Out-file C:\test1.csv -Append 
}

Any help would be much appreciated.

In:

1,2,"Test,ABC"

Out:

1,2,TestABC

Answer 1

Import the CSV. Convert it to a CSV with a different delimiter. Replace the commas. Convert the delimiter back. Replace the double quotes. Write out the resulting file.

Import-Csv -Path C:\MyFile.csv |
    ConvertTo-Csv -Delimiter '|' | 
    ForEach-Object { $_ -replace ',',[String]::Empty } | 
    ConvertFrom-Csv -Delimiter '|' | 
    ConvertTo-Csv |
    ForEach-Object { $_ -replace -replace '"',[String]::Empty } |
    Set-Content -Path C:\MyFile_fixed.csv

Answer 2

I would break this down into two steps. Another StackOverflow user may be able to give you a one-liner.

Import-Csv C:\test.csv | Export-Csv tempfile.csv -Delimiter "|"
(Get-Content tempfile.csv).Replace(",","").Replace('"',"") | Out-File test1.csv

Answer 3

The following should do what you want (tested in PSv5.1):

Import-Csv C:\test.csv | ForEach-Object -Begin { $writeHeader = $True } {
  if ($writeHeader) { $writeHeader = $False; $_.psobject.properties.Name -join ',' }
  $_.psobject.properties.Value -replace ',', '' -join ','
} | Set-Content -Encoding UTF8 test1.csv

• Import-Csv reads your CSV file into custom objects ( [pscustomobject] instances) whose properties contain the column values with double quotes removed.

  • Since the column values are then stored in distinct properties, column- internal , instances can therefore blindly replaced without worrying about column- separating , instances.
  • That the enclosing double quotes were automatically stripped is a beneficial side effect, though care must be taken not to reintroduce them on output - read on.

• The problem is that you can not use Export-Csv after modifying the objects, because it invariably adds double quotes (back) around all output values.

• Therefore, a custom mini-script must be executed for each custom object, using ForEach-Object :

  • -Begin { $writeHeader = $True } is executed once at the beginning to signal the need to output a header row before the first data row.

  • $_.psobject.properties is the collection of all properties defined on the input object, named for the header columns, and containing a given data row's values.

  • $_.psobject.properties.Name -join ',' outputs the header row, simply by joining the property names - which are the column headers - with , to yield a single output string.

  • $_.psobject.properties.Value -replace ',', '' removes any value-internal , instances (replaces them with the empty string), and -join ',' again joins the resulting values as-is with , , outputting a data row.

• Set-Content - which is preferable to Out-File here, because the output objects are already strings - is used to write to the output file.

  • Note the -Encoding parameter to control the output character encoding -adjust as needed.

  • In Windows PowerShell (versions up to v5.1), not using -Encoding would default to your system's "ANSI" code page (even though the help topic claims ASCII), whereas Out-File would default to UTF-16LE ("Unicode").

Answer 4

Does your csv have headers? Are the values to be changed in the same column?

If it looks something like this :

h1,h2,h3
1,2,"Test,ABC"
3,4,"Test,DEF"

This should work:

$Csv = Import-Csv -path C:\MyFile.csv
$Csv.H3 | foreach {$_.Replace('"',"").Replace(",","")}

Edit: Made it work. But basically the same as mklement0's solution

$Csv = Import-Csv -path C:\MyFile.csv
$Csv | Foreach {$_.H3 = $_.H3.Replace(",","")}
$CsvObject = $Csv | Convertto-Csv -NoTypeInformation
$CsvObject.replace('"','') | 
Set-Content C:\OutFile.Csv