Keep Same Encoding With Set-Content Multiple Files in PowerShell

Question

I'm attempting to write a script to be used to migrate an application from server to server and/or from one drive letter to another drive letter. My goal is to copy the directory from one location, move it to another, and then run a script to edit all instances of the old hostname, IP address, and drive letter to reflect the new hostname, IP address, and drive letter on the new server. This appears to do exactly that:

ForEach($File in (Get-ChildItem $path\* -Include *.xml,*.config -Recurse)){
    (Get-Content $File.FullName -Raw) -replace [RegEx]::Escape($oldhost),$newhost `
                                 -replace [RegEx]::Escape($oldip),$newip `
                                 -replace "$olddriveletter(?=:\Application)",$newDriveLetter | 
     Set-Content $File.FullName -NoNewLine
}

The one problem I am having is that the files all have different types of encoding. Some ANSI, some UTF-8, some Unicode, etc. When I run the script, it saves everything as ANSI and then my application fails to work. I know how to add the encoding parameter, but is there any way to keep the same encoding on each individual file, without writing out a script specifying each individual file in the directory and the encoding that each individual file has?

Answer 1

That would be difficult. It's too bad that get-content doesn't pass an encoding property. Here's a script that tries to get the encoding if there's a signature. Maybe you can just run it first and check them all. But some windows files are unicode no bom. At least xml files can say the encoding. get-childitem *.xml | select-string encoding get-childitem *.xml | select-string encoding There might be a better way to load xml files, see the bottom answer: Powershell: Setting Encoding for Get-Content Pipeline

# encoding.ps1
# https://stackoverflow.com/questions/3825390/effective-way-to-find-any-files-encoding
param([Parameter(ValueFromPipeline=$True)] $filename)
process {
  $reader = [IO.StreamReader]::new($filename, [Text.Encoding]::default,$true)
  $peek = $reader.Peek()
  $encoding = $reader.currentencoding
  $reader.close()
  [pscustomobject]@{Name=split-path $filename -leaf
                BodyName=$encoding.BodyName
            EncodingName=$encoding.EncodingName}
}
# end encoding.ps1


PS C:\users\me> get-childitem chinese16.txt | encoding

Name          BodyName EncodingName
----          -------- ------------
chinese16.txt utf-16   Unicode

Something like this will use the encoding indicated in the xml file, even if it didn't truly match beforehand. (This also makes the xml pretty.)

PS C:\users\me> [xml]$xml = get-content file.xml
PS C:\users\me> $xml.save("$pwd\file.xml")

Answer 2

Use the file.exe from the git binaries to find out the encoding. Then, add the encoding parameter to the set-content line with if else statements to meet your needs.

ForEach($File in (Get-ChildItem $path\*)){
    $Content = Get-Content $File.FullName -Raw -replace [RegEx]::Escape($oldhost),$newhost `
                                 -replace [RegEx]::Escape($oldip),$newip `
                                 -replace "$olddriveletter(?=:\Application)",$newDriveLetter 
    $Encoding = file --mime-encoding $File
    $FullName = $File.FullName
    Write-Host "$FullName - $Encoding"
    if(-NOT ($Encoding -like "UTF")){
        Set-Content $Content -NoNewLine -Encoding UTF8
    }
    else {
        Set-Content $Content -NoNewLine 
    }
}

Reference: https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.management/set-content http://gnuwin32.sourceforge.net/packages/file.htm