Read both strings and raw bytes from standard input in java

Question

I have a program which will be receiving information from an external source via System.in. There are two input modes: line mode and raw mode. During line mode, input is simply a series of UTF-8 strings, each terminated with a line feed character. At some point while in line mode, I will receive notification that I am about to receive N bytes of raw data. At that point the input switches to raw mode and I receive exactly N bytes of raw binary data, which are not valid UTF-8 characters. After this point, it returns to line mode.

Is there a way to easily switch between reading strings and reading raw data? My only thought is to read an InputStream byte by byte and translate to characters as I go. Are there any ways to wrap System.in with multiple types of input streams? I feel like reading from two different wrappers would cause problems.

(FIXED) Update:

I tried parsifal's suggestion, but am running into a problem. To simulate the switching input modes, I modified my test harness. (I realized that another process I have will eventually need to output this way as well.) I don't know if the problem is caused by the send or receive end. When I switch between output modes, it doesn't seem to be reading in the bytes properly. Also, it's always the same byte values that appear. Here are some code excerpts:

FIX: The problem was that apparently you can't switch from the OutputStreamWriter to OutputStream too quickly. I added a 1ms sleep command before sending the raw bytes, and the problem is solved!

Test Harness:

Process p = processList.get(pubName); //Stored list of started Processes
OutputStream o = p.getOutputStream(); //Returns OutputStream which feeds into stdin
out = new OutputStreamWriter(runPublisher.getOutputStream());

byte[] payload = new byte[25];
out.write("\nPAYLOAD\nRAW\n"); // "RAW\n" signals raw mode
out.write(String.valueOf(payload.length) + "\n");
out.flush();
Thread.sleep(1); //This fixed the problem I was having.
System.out.println(Arrays.toString(payload));
o.write(payload);
o.flush();

Client:

InputStreamReader inReader = new InputStreamReader(System.in);

while(true){
    try{
        if((chIn = inReader.read())!= -1){
            if(chIn == (int)'\n'){
                if(rawMode){
                    if(strIn.equals("ENDRAW"))
                        rawMode = false;
                    else{
                        System.out.println(strIn);
                        //Exception on next line
                        int rawSize = Integer.parseInt(strIn);
                        payload = new byte[rawSize];
                        int t = System.in.read(payload);
                        System.out.println("Read " + t + " bytes");
                        System.out.print(Arrays.toString(payload));
                    }
                }else if(strIn.startsWith("RAW")){
                    rawMode = true;
                }else {
                    // Do other things
                }
                strIn = "";
            }else
                strIn += (char)chIn;
        }else
            break;
    }catch(IOException e){break;}
}

And the outputs (prior to adding Sleep statement) look like this:

Test Harness:
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]

Client:
25
Read 9 bytes
[83, 72, 85, 84, 68, 79, 87, 78, 10, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]

Exception in thread "main" java.lang.NumberFormatException: For input string: "
    at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
    at java.lang.Integer.parseInt(Integer.java:470)
    at java.lang.Integer.parseInt(Integer.java:514)
    at myClass.handleCommand(myClass.java:249)

Answer 1

You can wrap System.in with an InputStreamReader that specifies "utf-8" encoding, and then read character-by-character. Accumulate characters into a StringBuilder and dispatch whenever appropriate (nomially when you see '\\n' , but possibly based on a test of the builder).

When you want to read binary data, just read from the underlying InputStream ( System.in ). The InputStreamReader performs translation as-needed, and does not buffer data.

You do not want to use any sort of buffered stream or reader in the stack. This will eliminate any opportunity to use a readLine() method, at least if you confine yourself to the JDK classes.

---

Edit based on your latest updates:

I think that your switching between raw and cooked mode is a bit suspicious. If I were to implement this, I'd create two primitive operations, String readLine() and byte[] readData(length) . The first accumulates characters up to a newline, the second reads a fixed buffer. Then your main loop looks something like this:

InputStream in = // ...
Reader rd = new InputStreamReader(in, "USASCII");  // or whatever encoding you use

while (true) {
    String command = readLine(rd );
    if (command .equals("RAW")) {
        int length = Integer.parseInt(readLine(rd ));
        byte[] data = readData(in , length);
        if (! readLine(rd ).equals("ENDRAW")) {
            throw // an exception that indicates protocol violation
        }
    }
    else // process other commands
}

I would also wrap the whole thing up in an object, which is constructed around the stream, and perhaps uses callbacks to dispatch the data packets.

Answer 2

the best bet is probably to just read byte-by-byte (using System.in.read() )into a buffer until you hit the UTF-8 line feed byte 0x0A, then translate that byte buffer into a string (using new String(byte[] bytes, "UTF-8") ).

note that read() called on a InputStream will return an int with a value from 0 to 255, you'll need to cast it into a byte. You can accumulate bytes in a Collection of some sort, then use standard Collection framework tools to convert it to an array for consumption by the String constructor.

When you see the indicator that its going to switch over (presumably some sort of in-stream signalling, certain specific bytes), then switch to your raw byte reading code.