How to decrypt data in PHP encrypted with JSEncrypt

Question

I'm trying to secure communication between a JS front-end and a PHP backend by using symmetric and asymmetric encryption. I'm creating a symmetric key on the client and encrypting it with the server's public key with JSEncrypt and sending it to the server for future use. However, I'm getting stuck when I get the data on the server side. openssl_open requires an envelope to decrypt the symmetric key and I'm not even positive what data is supposed to be in the envelope. I was under the impression that the envelope is the symmetric key that was encrypted with the public key, but using that has not worked. I've also tried different combinations of decoding as I've read that JSEncrypt encodes the message in base 64 and the key in hex, but those attempts are fruitless as well.

let pub = "-----BEGIN PUBLIC KEY-----...-----END PUBLIC KEY-----";

//I have a function that just creates a random string of characters
let key = generateKey(32);
let aesData = CryptoJS.AES.encrypt( "test", key );
let symKey = aesData.key + ":::" + aesData.iv;
let msg = aesData.toString();

let rsa = new JSEncrypt();
rsa.setPublicKey( pub );
let cryptKey = rsa.encrypt( symKey );

//I'm passing the data through a hidden form field
$("#key").val(cryptKey + ":::" + msg);

$key = openssl_get_privatekey( file_get_contents( $_SERVER["PRIV_KEY"]) );
$encryptedKey = explode( ":::", $msg )[0];
$realMsg = base64_decode(explode( ":::", $msg )[1]);

openssl_open($realMsg, $decrypted, $encryptedKey, $key);
return $decrypted;

The code above outputs nothing because the openssl_open call fails (returns false). When I base 64 decode the $encryptedKey variable, I get:

�vEi���pΕ��d_���@����욲JE��

but the symmetric key changes every time, so the output changes every time as well. Like I said, I've tried different encoding combinations, but they all return similar nonsense. As the JS code shows, I've encrypted the message "test".

I've never implemented encryption before, so I might be way off the mark here, but after staring at this code for days, any insight would be appreciated.

Edit: I'm having problems decrypting with my private key in PHP, not with the symmetric key

Answer 1

Figured it out!!! So, I found out that PHP has a function to decrypt without needing an envelope called openssl_private_decrypt that uses a private key to decrypt a message. By using that function and base 64 decoding the encrypted key, I am able to decrypt the symmetric key on the server side and will hopefully be able to decrypt the message symmetrically now. For those interested, my code on the server side is:

$key = openssl_get_privatekey( file_get_contents( $_SERVER['PRIV_KEY'] ) );
$encryptedKey = base64_decode(explode( ":::", $msg )[0]);

if( openssl_private_decrypt($encryptedKey, $decrypted, $key) )
{
    return $decrypted;
}
return $encryptedKey;

And on the client side, my code is the same as it was above. Hope this helps someone!