PowerShell Delete or Skip Columns in CSV

Question

First of all: I'm a PowerShell rookie. I have little experience using PowerShell to modify or change CSV files.

Our system gives out a uncommon CSV format, which looks like this:

Example1;Example2;Name;Lastname;ContentOfExample1;ContentOfExample2;John;Doe

The header is on every row infront of the information. I want to get rid of some Columns like Example1 and Example2.

As a second step I need to assign a new header

-Header Name,Lastname,Adress,Phone,.. and so on.

I'm thankful for any tipps :-)

Answer 1

By definition this pattern results in an even number of ";" delimited elements. You can use that to your advantage, by arithmetically assigning properties to objects then re-emitting them to a new CSV file.

Might look something like:

Get-Content C:\Temp\InitialCSVFile.csv |
ForEach-Object{
    $TempArr  = $_.Split( ';' )
    $TempHash = [Ordered]@{}
    For($i = 0; $i -lt ($TempArr.Count / 2); ++$i)
    {
        $TempHash[ $TempArr[ $i ] ] = $TempArr[ $i+4 ]
    }
    [PSCustomObject]$TempHash
} |
Export-CSV -path C:\Temp\TestCSV.csv -NoTypeInformation -Append -Delimiter ';'

The code is reading the file contents as plain strings, not a semi-structured format like CSV. As each line is piped to ForEach-Object the .Split() string method is creating an array ( $_ -split ';' would work too). The we instantiate a Hash/Dictionary object to hold some key value pairs. Once that's done a traditional For loop is used to reference the kay names & values. The name is element 0 and therefore it's value should be 0+4. Note: the loop is coded to stop at the halfway point in the array. That's why the even number of elements I mentioned earlier is important!

Once the hash table is complete the code casts it to a [PSCustomObject] and sends it down the pipeline to Export-CSV which of course deals in objects. This should result in a new CSV file that looks something like:

Example1          Example2          Name Lastname
--------          --------          ---- --------
ContentOfExample1 ContentOfExample2 John Doe
ContentOfExample1 ContentOfExample2 John Doe

Note: Obviously the data is redundant because I just repeated your sample in the input file. That shouldn't be a problem with your live data.

Note: May not need to repeatedly recreate $TempHash , since we'll reassign each key's value on each loop internal iteration. For now I'll let this example stand as is.

Update: To Exclude Properties:

$ExcludeProperties = @( 'Example1', 'Example2' )

Get-Content C:\Temp\InitialCSVFile.csv |
ForEach-Object{
    $TempArr  = $_.Split( ';' )
    $TempHash = [Ordered]@{}
    For($i = 0; $i -lt ($TempArr.Count / 2); ++$i)
    {
        $TempHash[ $TempArr[ $i ] ] = $TempArr[ $i+4 ]
    }
    [PSCustomObject]$TempHash
} |
Select-Object -Property * -ExcludeProperty $ExcludeProperties |
Export-CSV -path C:\Temp\TestCSV.csv -NoTypeInformation -Append -Delimiter ';'

Answer 2

A strange way to output a CSV indeed..

What you could do is to split the first line by the delimiter character ; in order to get the headers for each column.

Once you have that, the rest should not be too hard to do:

$csv = Get-Content -Path 'D:\Test\blah.csv' | Where-Object {$_ -match '\S'}

$parts = $csv[0] -split ';'
# calculate the number of parts that make up the headers
[int]$numberOfHeaders = $parts.Count / 2
# join the headers into a string
$header = $parts[0..($numberOfHeaders - 1)] -join ';'
# cut off the headers from every line
$rows = foreach ($line in $csv) { $line.Substring($header.Length + 1) }

# convert to an array of objects, skip the first two columns and export to a new file
$header, $rows | ConvertFrom-Csv -Delimiter ';' | 
    Select-Object * -ExcludeProperty $parts[0..1] | 
    Export-Csv -Path 'D:\Test\blah2.csv' -Delimiter ';' -NoTypeInformation

Answer 3

Assuming the number of columns could be random and the properties to exclude are known, you can do the following to parse your data as custom objects:

Get-Content file.csv | Foreach-Object {
    $count = 0 # Tracks column counts to split the row evenly
    $cols = $_ -split ';'
    # $headers gets the first half of the columns. $data gets the remainder.
    $headers,$data = $cols.where({$count++ -lt $cols.count/2},'Split')
    # Uses calculated properties to add your new properties. You will need to fill in your own logic since you provided none here.
    ($headers -join ';'),($data -join ';') | ConvertFrom-Csv -Delimiter ';' |
        Select-Object *,@{n='Address';e={'Electric Avenue'}},@{n='Phone';e={'867-5309'}} -exclude example1,example2
}

---

If all data in a csv file contains the same headers, you could just use Export-Csv to create a proper CSV from the data:

Get-Content file.csv | Foreach-Object {
    $count = 0 # Tracks column counts to split the row evenly
    $cols = $_ -split ';'
    $headers,$data = $cols.where({$count++ -lt $cols.count/2},'Split')
    ($headers -join ';'),($data -join ';') | ConvertFrom-Csv -Delimiter ';' |
        Select-Object *,@{n='Address';e={'Electric Avenue'}},@{n='Phone';e={'867-5309'}} -exclude example1,example2
} | Export-Csv output.csv -NoType

---

If each individual row could have a varied number of columns, you will likely need a CSV file per row unless you parse all of the data and determine all the possible column names. If you want to keep the same format as the source but just want to manipulate columns and data, you can do the following, which will work with a varied number of columns:

Get-Content file.csv | Foreach-Object {
    $count = 0 # Tracks column counts to split the row evenly
    $cols = $_ -split ';'
    $headers,$data = $cols.where({$count++ -lt $cols.count/2},'Split')
    $newObj = ($headers -join ';'),($data -join ';') | ConvertFrom-Csv -Delimiter ';' |
        Select-Object *,@{n='Address';e={'Electric Avenue'}},@{n='Phone';e={'867-5309'}} -exclude example1,example2
    "{0};{1}" -f ($newObj.psobject.properties.name -join ';'),($newObj.psobject.properties.value -join ';')
}