How do I encrypt and send DES key with RSA private key and then public key in Java?

Question

EDIT: The second question is no longer needed. The problem was a result of padding issues; changing the parameters of the Cipher to "RSA/ECB/PKCS1Padding" fixed it, which was already implemented in the other Ciphers used.

For a school assignment I have to create a two programs in Java (a client and server) that use RSA asymmetric encryption to create and agree on a session key that uses DES encryption. The final message in the exchange contains a key that has been generated by the client, which is encrypted by the client's private key, and then encrypted again using the server's public key. The server can then decrypt the message using its private key and then decrypt it again with the client's public key to obtain the DES key. However, the first encryption results in a byte array of size 256, and the second encryption requires a byte array that is smaller than that. Is there a way to manipulate the data such that I can encrypt the key twice, as specified in the assignment? Note that this is a requirement, as is using the DES algorithm for the session key.

Additionally, disregarding the second encryption such that the next part of the assignment, which allows the client and server to send encrypted messages to one another, produces another problem. Currently, the client wraps the key using the server's public key, and then the server unwraps it using its private key. However, unwrapping the session key on the server side produces an InvalidKeyException; the unwrapped key length is 256 bytes, which is completely wrong.

On the server side, I have:

        byte[] m4 = new byte[256];
        datIn.read(m4);
        cUwp.init(Cipher.UNWRAP_MODE, myKey);
        ks = (SecretKey)cUwp.unwrap(m4, "DES", Cipher.SECRET_KEY);
        System.out.println("Recieved key:\n" + ks);
        try{
            Cipher desCipher = Cipher.getInstance("DES");
            desCipher.init(Cipher.DECRYPT_MODE, ks);
        }
        catch(NoSuchPaddingException|InvalidKeyException e){
            System.out.println("Error: " + e);
        }

On the client side:

        KeyGenerator keygen = KeyGenerator.getInstance("DES");
        SecretKey key = keygen.generateKey();
        cWrp.init(Cipher.WRAP_MODE, theirKey);
        byte[] m4 = cWrp.wrap(key);
        datOut.write(m4);
        ks = key;
        try{
            Cipher desCipher = Cipher.getInstance("DES");
            desCipher.init(Cipher.DECRYPT_MODE, ks);
        }
        catch(NoSuchPaddingException|InvalidKeyException e){
            System.out.println("Error: " + e);
        }

I am not getting any size discrepancies in any other parts of my code; any other decrypted messages are the proper size, but none of them use the wrap() method, as they are Strings and not SecretKeys. Is there something that I'm missing?

Answer 1

You're doing this completely wrong. The encryption must be done with the public key, and the decryption by the private key. Otherwise anybody can decrypt it.

Throw it all away and use TLS.