Powershell Why the difference between the two json contents?

Question

I have two variables of Byte[] type (I chose this type for a specific need, so it need to be retained.) which are declared as:

$first = New-Object Byte[] 32
$second = New-Object Byte[] 32

and, I initalized each index of both the variables.

Now, I created a hashtable $List1 as:

$List1=@{"First" = $first; "Second" = $second}

I am using the syntax below for creating the json file:

$List1 | ConvertTo-Json | Set-Content  -Path $jsonFile1

This is the first json file content:

{
"First":  {
             "value":  [
                           210,
                           195,
                           131,
                           176,
                           88,
                           154,
                           57,
                           37,
                           2,
                           75,
                           182,
                           190,
                           156,
                           43,
                           113,
                           199,
                           63,
                           25,
                           109,
                           92,
                           220,
                           91,
                           219,
                           252,
                           113,
                           68,
                           202,
                           12,
                           147,
                           194,
                           36,
                           177
                       ],
             "Count":  32
         },
"Second":  {
           "value":  [
                         238,
                         225,
                         12,
                         172,
                         134,
                         94,
                         42,
                         204,
                         27,
                         78,
                         39,
                         166,
                         229,
                         111,
                         143,
                         254
                     ],
           "Count":  16
       }
}

Then I read the first json file into a temporary variable as below:

$tempHash = Get-Content -Path $jsonFile1 -Raw| ConvertFrom-Json

Since $tempHash is a PSCustomObject , I create a new hashtable $List2 as below:

$List2 = @{"First" = $tempHash.First.value; "Second"= $tempHash.Second.value}

which I use to create the second json file as below:

$List2 | ConvertTo-Json | Set-Content  -Path $jsonFile2

This is the second json file content:

    {
"First":  [
             133,
             231,
             19,
             173,
             60,
             50,
             105,
             68,
             38,
             109,
             99,
             155,
             2,
             188,
             216,
             9,
             8,
             225,
             203,
             15,
             167,
             8,
             188,
             76,
             192,
             154,
             183,
             194,
             1,
             122,
             143,
             137
         ],
"Second":  [
           27,
           3,
           57,
           67,
           155,
           145,
           181,
           194,
           250,
           10,
           65,
           90,
           41,
           230,
           243,
           196
       ]
}

I am using the same syntax to create both the json files. So, why are their structures different?

[Edit]

I suspect, the difference is because of this very syntax:

$List1=@{"First" = $first; "Second" = $second}

because Byte[] type variable does not work as a simple integer[] type variable. Correct me.

[Edit]

So, it turns out Byte[] type variable has two different keys. "value" which holds the actual array of byte values, and "Count" which holds the number of elements in the Byte[] variable. However, when we invoke the Byte[] type variable like:

$first

which is of Byte[] type, we get only the values listed under the "value" key. The value under the "count" key is never displayed in console, yet it is passed to the hashtable somehow.

And, One more point to be noted. If I use:

 $List2 = @{"First" = $tempHash.First; "Second"= $tempHash.Second}

then, I'll have to use:

$List2.First.Value #to get the value of the "First" key

and that makes me feel uncomfortable because for the $List1 hashtable, I only needed to use:

 $List1.First #to get the value of the "First" key.

[Workaround]

I created a hastable $List as the original hashtable as below for strictly one time use only :

$List | ConvertTo-Json | Set-Content  -Path $jsonFile

Then, I created two hastables $List1 and $List2 as below from the original $jsonFile above.

 $tempHash = Get-Content -Path $jsonFile -Raw| ConvertFrom-Json
 $List1 = @{"First" = $tempHash.First; "Second" = tempHash.Second}
 $List2 = @{"First" = $tempHash.First; "Second" = tempHash.Second}

It helped me keep consistency while referring to their keys and values.

Now, I use

#to fetch the values of the "First" keys of both hashtables.
$List1.First.value #and
$List2.First.value

Similarly, I do the same for "Second" key for both hashtables $List1 and $List2 .

#to fetch the values of the "Second" keys of both hashtables.
$List1.Second.value #and
$List2.Second.value

[Edit]

It turned out to be a bug in my version of Powershell as ststed by @mklement0 below. The perfect solution will be to instead use the syntax below as instructed by @mklement0 :

# Ensure that the input array is constructed without the extra [psobject] wrapper.
$First = [Byte[]]::new(32)
$Second = [Byte[]]::new(32)

Answer 1

• The result of the first ConvertTo-Json call is a quirk in Windows PowerShell as of v5.1: the resulting JSON should have First and Second contain an array directly rather than an object with value and Count properties, with value containing the array.

  • This behavior has been fixed in PowerShell Core ; it may or may not get fixed in Windows PowerShell (still present as of v5.1).
    $a = New-Object Byte[] 2; @{ a = $a } | ConvertTo-Json -Compress $a = New-Object Byte[] 2; @{ a = $a } | ConvertTo-Json -Compress yields:
    • {"a":[0,0]} in PowerShell Core v6.0.1 - OK.
    • {"a":{"value":[0,0],"Count":2}} in Windows PowerShell v5.1 - BROKEN.

• In your case it is the use of New-Object that triggers the quirk .

  • Most likely it is related to this issue ; note, however, that said issue isn't resolved in PowerShell Core either, but the fix linked to above resolves the problems in this context.

---

Workaround :

At the start of your script/session, run:

Remove-TypeData System.Array

This removes the obsolete ETS-supplied .Count property from all array objects, which makes the problem go away for [psobject] -wrapped objects (such as returned by New-Object ) - for an explanation, see this answer of mine.

---

More cumbersome workarounds :

The problem goes away if you ensure that -is [psobject] no longer reports true for the input arrays , which can be done in one of the following ways:

• [PSv5+]: $First = [Byte[]]::new(32) - use of an expression rather than a command makes the problem go away, because it doesn't create an extra, invisible [psobject] wrapper.

• [PSv4-]: $First = (New-Object Byte[] 32).psobject.BaseObject - explicitly bypassing the extra [psobject] wrapper makes the problem go away.

Simplified example (PSv5+, but easily adapted to earlier versions; file operations omitted, because they are incidental to the problem):

# Ensure that the input array is constructed without the extra [psobject] wrapper.
$First = [Byte[]]::new(2)

# Construct a hashtable...
$List1 = @{ First = $first }
# ... and convert it to JSON:
($json = $List1 | ConvertTo-Json)

The above now correctly yields ( no extraneous object with value and count properties):

{
    "First":  [
                  0,
                  0
              ]
}

Reconverting this JSON string to an object now works as expected:

# Re-convert: $tempObj.First then *directly* contains the input array
#             (there is no .Value property anymore).
$tempObj = $json | ConvertFrom-Json

# Construct a new hashtable...
$List2 = @{ First = $tempObj.First }
# ... and convert it to JSON.
$List2 | ConvertTo-Json

The result is the same JSON string as above.

Answer 2

Change this:

$List2 = @{"First" = $tempHash.First.value; "Second"= $tempHash.Second.value}

To this:

$List2 = @{"First" = $tempHash.First; "Second"= $tempHash.Second}

This will maintain the value and count structures. On testing, it's also kept the order on the values in the value section.

---

Edit
To enhance your workaround, you can put in this step so that you can access the values without having to use .value :

$firstValues = [byte[]]($tempHash.First.value)
$secondValues = [byte[]]($tempHash.Second.value)

$List1 = @{"First" = $firstValues; "Second" = $secondValues}
$List2 = @{"First" = $firstValues; "Second" = $secondValues}

---

Self-Contained Test Script

$jsonFile1 = "jsonFile1.json"
$jsonFile2 = "jsonFile2.json"

$first = New-Object Byte[] 32
$second = New-Object Byte[] 16

foreach($i in 0..($first.COunt -1)){
    $first[$i] = Get-random -minimum 1 -maximum 254
}

foreach($i in 0..($second.COunt -1)){
    $second[$i] = Get-random -minimum 1 -maximum 254
}

$List = @{"First" = $first; "Second" = $second}
$List | ConvertTo-Json | Set-Content -Path $jsonFile1

$tempHash = Get-Content -Path $jsonFile1 -Raw | ConvertFrom-Json

$firstValues = [byte[]]($tempHash.First.value)
$secondValues = [byte[]]($tempHash.Second.value)

$List1 = @{"First" = $firstValues; "Second" = $secondValues}
$List2 = @{"First" = $firstValues; "Second" = $secondValues}