使用tcp从PHP发送到JAVA后，数据已损坏

Question

I am trying to send data from PHP TCP server to JAVA TCP client. I am comparing my results by comparing hex values of the data.

PHP script reads STDIN, sends it through socket one byte at a time and java reads it using DataInputStream.read(), converts to hex and displays.

If I manually type data into script - it works ok. If I use file with data - it works OK But when I assign /dev/urandom(even few bytes) - the data on the java side is coming corrupted. There is always a hex of value efbfbd in random places instead of correct data. Please help me with this issue. PHP code:

$f = fopen( 'php://stdin', 'rb' );
while($line = fread($f, 1)){

    $length = 1;
    echo bin2hex($line)."\n";
    echo socket_write($client, $line, 1)."\n";

     $sent = socket_write($client, $line, $length);

if ($sent === false) {

    break;
}

// Check if the entire message has been sented
if ($sent < $length) {

    // If not sent the entire message.
    // Get the part of the message that has not yet been sented as message
    $line = substr($line, $sent);

    // Get the length of the not sented part
    $length -= $sent;

}

Java code:

in = new DataInputStream(clientSocket.getInputStream());

            byte[] data = new byte[1];

            int count = 0;
            while(in.available() > 0){
                //System.out.println(in.available());
                     in.read(data);
                String message = new String(data);

                System.out.println(message);
                //System.out.flush();

                System.out.println( toHex(message) );
                //in.flush();
                message = "";



            }

Answer 1

You're stumbling upon encoding. By calling new String(data) the byte array is converted using your default encoding to a string, whatever this encoding may is (you can set the encoding by java -Dfile.encoding=UTF-8 to UTF-8 for example).

The Java code you want would most likely look the following:

    in = new DataInputStream(clientSocket.getInputStream());

    byte[] data = new byte[1];

    int count = 0;
    while (in.available() > 0) {
        // System.out.println(in.available());
        in.read(data);
        String hexMessage = Integer.toHexString(data[0] & 0xFF);
        String stringMessage = new String(data, "UTF-8"); // US-ASCII, ISO-8859-1, ...
        System.out.println(hexMessage);
    }

Update: I missed the 32bit issue. The 8-bit byte , which is signed in Java, is sign-extended to a 32-bit int . To effectively undo this sign extension, one can mask the byte with 0xFF .

Answer 2

There are two main issues with your Java program.

First - the use of in.available() . It does not tell you how many bytes there are still in the message. It merely says how many bytes are ready in the stream and for available reading without being blocked. For example, if the server sends two packets of data over the socket, one has arrived, but one is still being sent over the Internet, and each packet has 200 bytes (this is just an example), then in the first call you'll get the answer 200 . If you read 200 bytes, you're sure not to be blocked. But if the second packet has not arrived yet, your next check of in.available() will return 0 . If you stop at this point, you only have half the data. Not what you wanted.

Typically you either have to read until you reach end-of-stream ( InputStream.read() returns -1), and then you can't use the same stream anymore and you close the socket, or you have a specific protocol that tells you how many bytes to expect and you read that number of bytes.

---

But that's not the reason for the strange values you see in output from your program. The reason is that Java and PHP represent strings completely differently. In PHP, a string can contain any bytes at all, and the interpretation of them as characters is up to the prorgrammer.

This basically means that a PHP string is the equivalent of a byte[] in Java.

But Java Strings are completely different. It consists internally of an array of char , and char is always two bytes in UTF-16 encoding. When you convert bytes you read into a Java String , it's always done by encoding the bytes using some character encoding so that the appropriate characters are stored in the string.

For example, if your bytes are 44 4F 4C 4C , and the character encoding is ISO-8859-1, this will be interpreted as the characters \D , \O , \L , \L . It will be a string of four characters - "DOLL" . But if your character encoding is UTF-16 , the bytes will be interpreted as \䑏 and \䱌 . A string of only two characters, "䑏䱌" .

When you were reading from the console or from a file, the data was probably in the encoding that Java expects by default. This is usually the case when the file is written in pure English, with just English letters, spaces and punctuation. These are all 7-bit characters which are the same in ISO-8859-1 and UTF-8, which are the common defaults. But in /dev/urandom you'd have some bytes in the range 80 through FF , which may be treated differently when interpreted into a UTF-16 Java string.

Furthermore, you didn't show your toHex() method in Java. It probably reads bytes back from the string again, but using which encoding? If you read the bytes into the String using ISO-8859-1 , and got them out in UTF-8 , you'd get completely different bytes.

If you want to see exactly what PHP sent you, don't put the bytes in a String . Write a toHex method that works on byte arrays, and use the byte[] you read directly.

---

Also, always remember to check the number of bytes returned by read() and only interpret that number of bytes! read() does not always fill the entire array. So in your new toHex() method, you need to also pass the number of bytes read as a parameter, so that it doesn't display the parts of the array after them. In your case you just have a one-byte array - which is not recommended - but even in this case, read() can return 0, and it's a perfectly legal value indicating that in this particular call to read() there were no bytes available although there may be some available in the next read() .

Answer 3

As the comment above says you might be having troubles with the string representation of the bytes String message = new String(data); To be certain, you should get the data bytes and encode them in Base64 for example. You can use a library such as Apache Commons or Java 8 to do that. You should be able to do something similar in PHP to compare.