RSA Java encryption and Node.js decryption is not working

Question

I have a system that requires a RSA keypair to be generated in javascript, have the public key then stored in a database at the server side (as a string), then the server side which is in Java will encrypt a string with the stored public key and send it to the client side which will decrypt the string with the private key.

I'm using a browsified version of node-rsa on my client browser.

First at the client i generate a keypair and export the keys, storing them as strings

var NodeRSA = require('node-rsa');
var key = new NodeRSA({b: 1024});
key.exportKey("pkcs8-private");
key.exportKey("pkcs8-public-pem");

The exported private key is stored at the client and the public at the server

Next i used java to encrypt a string with the public key received, so i parse the pkcs8 public key into a Java PublicKey object.

String pubKey = "<Retrieved pkcs8 public key>";
pubKey = pubKey.replaceAll("(-+BEGIN PUBLIC KEY-+\\r?\\n|-+END PUBLIC KEY-+\\r?\\n?)", "");
byte[] keyBytes = Base64.decodeBase64(pubKey);
X509EncodedKeySpec spec = new X509EncodedKeySpec(keyBytes);
KeyFactory kf = KeyFactory.getInstance("RSA");
PublicKey pk = kf.generatePublic(spec);

And encrypt a text with it

byte[] cipherText;
Cipher cipher = Cipher.getInstance("RSA");
cipher.init(Cipher.ENCRYPT_MODE, pk);
cipherText = cipher.doFinal("Hello World!".getBytes());
return Base64.encodeBase64String(cipherText);

Which works nicely and returns me a Base64 encoded encrypted string like this

WTS1J2f4w5icsUOCtulyHDaBmB5lN7D8mnj0QWMDBkUGiPHkM8nHVx9pd0MtbQAQNasQS2X8kisLMYyEMPasFZtDH0zX1e8lNYaW0xMKsg++ge87f+95nl+TmxDy6S1m7Ce/n0wXno+0MbSv8YsJtsUcAleyyfQX2bxqX8u7Gjs=

Then i try to decrypt it the string at the client side

First i reimport the stored keys in node-rsa

var NodeRSA = require('node-rsa');
var key = new NodeRSA();
key.importKey("<exported private key string>","pkcs8-private");
key.importKey("<exported public key string>","pkcs8-public-pem");

Then i try to decrypt the Base64 encoded encrypted string

key.decrypt("<Base64 Encoded Encrypted>", 'utf-8');

This is where the problem happens, javascript throws this error

Uncaught Error: Error during decryption (probably incorrect key). Original error: Error: Error decoding message, the lHash calculated from the label provided and the lHash in the encrypted data do not match.(…) However i have tested that if i encrypt and decrypt the text just within javascript, it works just fine. This makes me think that it's some difference between the way i encrypted it at java and how it's done at javascript

Could anyone point out the mistake that I've made here please?

Answer 1

Oh i found the solution. It was a difference in the encryption method.

I just had to initialize Cipher with

Cipher.getInstance("RSA/ECB/OAEPWithSHA1AndMGF1Padding");

instead of

Cipher.getInstance("RSA");

to match node-rsa

Answer 2

Instead of change the encryption mode on my backend, I change it on front.

const rsa = new NodeRSA({ b: 2048 });
rsa.setOptions({ encryptionScheme: 'pkcs1' });
rsa.importKey(
    'key',
    'pkcs8-public',
);

and keep using Cipher.getInstance("RSA"); to encrypt and decrypt on backend.