how can I pass a json file into a command line variable using the set command

Question

I want to read the content of a json file into a cmd variable using set. I tried this:

set /p TEXT=<C:\myjson.json

but when I print the values, I only see the bracket as there is a new line.

C:\Windows\system32>echo %TEXT%
{

C:\Windows\system32>

JSON looks like this:

{
  "type": "service_account",
  "project_id": "myproject",
  "private_key_id": "sfackq0355ggeskvcPAFKCADACsdfa",
  "private_key": "-----BEGIN PRIVATE KEY-----  AAAB3NzaC1yc2EAAAABIwAAAQEAklOUpkDHrfHY17SbrmTIpNLTGK9Tjom/BWDSUGPl+nafzlHDTYW7hdI4yZ5ew18JH4JW9jbhUFrviQzM7xlELEVf4h9lFX5QVkbPppSwg0cda3Pbv7kOdJ/MTyBlWXFCR+HAo3FXRitBqxiX1nKhXpHAZsMciLq8V6RjsNAQwdsdMFvSlVK/7XAt3FaoJoAsncM1Q9x5+3V0Ww68/eIFmb1zuUFljQJKprrX88XypNDvjYNby6vw/Pb0rwert/EnmZ+AW4OZPnTPI89ZPmVMLuayrD2cE86Z/il8b+gw3r3+1nKatmIkjn2so1d01QraTlMqVSsbxNrRFi9wrf+M7Q==",
  "client_email": "myemail@mydomain.com",
  "client_id": "45844444558569745",
  "auth_uri": "url goes here",
  "token_uri": "another url here"
}

Answer 1

There are lots of issues that make your goal difficult, if not impossible to achieve with batch.

• A JSON file is likely to be encoded as UTF-8, which is awkward to deal with in batch.
• SET /P can only read a single line, or 1021 characters, whichever comes first.
• FOR /F can be used to read multiple lines, and the values can be concatenated. But the concatenation requires one of two options, each with its own issues
  • Normal percent expansion can be used, but then poison characters like & , | , < etc. can cause problems if the data also includes " literals. Also, percent expansion will require at least one CALL , which can lead to doubling of quoted ^ characters.
  • Delayed expansion can be used, but then FOR /F expansion will corrupt ! and possibly ^ within the data unless they are escaped first.
    • Using batch to escape ! requires percent expansion, which again can cause problems with poison characters.
• Batch (cmd.exe) can only work with environment variables with maximum length of 8191 characters. The OS can handle much larger, but batch can't make use of such large values. There is simply no workaround for this batch limitation.

Below is your sample JSON with some additional characters added to introduce all the complications above (except for the length limit).

test.json (utf-8 encoded)

{
  "type": "service_account\nline2",
  "project_id": "myproject €! \"<^>\"",
  "private_key_id": "sfackq0355ggeskvcPAFKCADACsdfa",
  "private_key": "-----BEGIN PRIVATE KEY-----  AAAB3NzaC1yc2EAAAABIwAAAQEAklOUpkDHrfHY17SbrmTIpNLTGK9Tjom/BWDSUGPl+nafzlHDTYW7hdI4yZ5ew18JH4JW9jbhUFrviQzM7xlELEVf4h9lFX5QVkbPppSwg0cda3Pbv7kOdJ/MTyBlWXFCR+HAo3FXRitBqxiX1nKhXpHAZsMciLq8V6RjsNAQwdsdMFvSlVK/7XAt3FaoJoAsncM1Q9x5+3V0Ww68/eIFmb1zuUFljQJKprrX88XypNDvjYNby6vw/Pb0rwert/EnmZ+AW4OZPnTPI89ZPmVMLuayrD2cE86Z/il8b+gw3r3+1nKatmIkjn2so1d01QraTlMqVSsbxNrRFi9wrf+M7Q==",
  "client_email": "myemail@mydomain.com",
  "client_id": "45844444558569745",
  "auth_uri": "url goes here ^",
  "token_uri": "another url here ^"
}

Assuming your JSON is < 8191 characters and is encoded as UTF-8 (or ASCII), then the following pure batch code will work.

@echo off
setlocal enableDelayedExpansion

:: Create a linefeed variable
set ^"LF=^
%= Generates a linefeed character =%
^"

:: Save active code page
for /f "tokens=2 delims=:." %%P in ('chcp') do set CHCP=%%P

:: Set active code page to UTF-8
chcp 65001 >nul

:: Read the file into str
set "str="
for /f usebackq^ delims^=^ eol^= %%A in ("test.json") do call :StoreLine

:: Restore active code page
chcp %CHCP% >nul

:: Show the result
set str

exit /b


:StoreLine
:: Safely escapes all ^ as ^^ and ! as ^! before appending %ln%!LF! to !str!
setlocal disableDelayedExpansion
for %%. in (.) do set "ln=%%A"
set "ln=%ln:"=""%"
set "ln=%ln:^=^^%"
set "ln=%ln:!=^!%"
endlocal & set "ln=%ln%" !
set "str=!str!!ln:""="!!LF!"
exit /b

If you don't care about preserving formatting whitespace, then the following simplified code will work (I also removed the comments). Note that newlines and leading spaces and tabs are removed, but other whitepace on a line is preserved.

@echo off
setlocal enableDelayedExpansion

for /f "tokens=2 delims=:." %%P in ('chcp') do set CHCP=%%P
chcp 65001 >nul
set "str="
for /f "usebackq tokens=*" %%A in ("test.json") do call :StoreLine
chcp %CHCP% >nul

set str

exit /b


:StoreLine
:: Safely escapes all ^ as ^^ and ! as ^! before appending %ln%!LF! to !str!
setlocal disableDelayedExpansion
for %%. in (.) do set "ln=%%A"
set "ln=%ln:"=""%"
set "ln=%ln:^=^^%"
set "ln=%ln:!=^!%"
endlocal & set "ln=%ln%" !
set "str=!str!!ln:""="!"
exit /b

-- OUTPUT --

str={"type": "service_account\nline2","project_id": "myproject €! \"<^>\"","private_key_id": "sfackq0355ggeskvcPAFKCADACsdfa","private_key": "-----BEGIN PRIVATE KEY-----  AAAB3NzaC1yc2EAAAABIwAAAQEAklOUpkDHrfHY17SbrmTIpNLTGK9Tjom/BWDSUGPl+nafzlHDTYW7hdI4yZ5ew18JH4JW9jbhUFrviQzM7xlELEVf4h9lFX5QVkbPppSwg0cda3Pbv7kOdJ/MTyBlWXFCR+HAo3FXRitBqxiX1nKhXpHAZsMciLq8V6RjsNAQwdsdMFvSlVK/7XAt3FaoJoAsncM1Q9x5+3V0Ww68/eIFmb1zuUFljQJKprrX88XypNDvjYNby6vw/Pb0rwert/EnmZ+AW4OZPnTPI89ZPmVMLuayrD2cE86Z/il8b+gw3r3+1nKatmIkjn2so1d01QraTlMqVSsbxNrRFi9wrf+M7Q==","client_email": "myemail@mydomain.com","client_id": "45844444558569745","auth_uri": "url goes here ^","token_uri": "another url here ^"}

Either pure batch solution above requires that delayed expansion is enabled. It is possible to write additional code to transfer the variable across the ENDLOCAL barrier. But I wouldn't bother.

You can use my JREPL.BAT text processing utility to store the text in a variable, and that works whether delayed expansion is enabled or not. JREPL.BAT is a hybrid JScript/batch script that uses the Windows CSCRIPT engine. It is pure script that runs natively on any Windows machine from XP onward.

call jrepl.bat "^" "" /f "test.json|utf-8" /rtn str

You can use the find/replace capability to completely remove all formatting whitespace.

call jrepl "\q[\c\q]*\q|\s" "$&|" /m /xseq /t "|" /f "test.json|utf-8" /rtn str

-- RESULT --

str={"type":"service_account\nline2","project_id":"myproject €! \"<^>\"","private_key_id":"sfackq0355ggeskvcPAFKCADACsdfa","private_key":"-----BEGIN PRIVATE KEY-----  AAAB3NzaC1yc2EAAAABIwAAAQEAklOUpkDHrfHY17SbrmTIpNLTGK9Tjom/BWDSUGPl+nafzlHDTYW7hdI4yZ5ew18JH4JW9jbhUFrviQzM7xlELEVf4h9lFX5QVkbPppSwg0cda3Pbv7kOdJ/MTyBlWXFCR+HAo3FXRitBqxiX1nKhXpHAZsMciLq8V6RjsNAQwdsdMFvSlVK/7XAt3FaoJoAsncM1Q9x5+3V0Ww68/eIFmb1zuUFljQJKprrX88XypNDvjYNby6vw/Pb0rwert/EnmZ+AW4OZPnTPI89ZPmVMLuayrD2cE86Z/il8b+gw3r3+1nKatmIkjn2so1d01QraTlMqVSsbxNrRFi9wrf+M7Q==","client_email":"myemail@mydomain.com","client_id":"45844444558569745","auth_uri":"url goes here ^","token_uri":"another url here ^"}

The /F parameter can be modified to support nearly any encoding. On my machine JREPL (CSCRIPT) defaults to windows-1252 if no encoding is specified.

Now I would not recommend getting JREPL.BAT just for this purpose. But JREPL is an extremely powerful text processing tool. If you frequently find yourself wanting to manipulate text or files with batch, then JREPL can be an invaluable tool.