RSA decryption using modulus and exponent

Question

My task: I have encrypted (RSA) data and public key as modulus and exponent. I have to write decryption code.
My problem with it: My implementation doesn't work ;) As far as I know philosophy is simple "open text" == rsa(public_key, rsa(private_key, "open text")) Edit: Exactly my assumption was wrong (Assumption is mother of all fu..ups ;) ). It should be "open text" == rsa(private_key, rsa(public_key, "open text")) because in RSA, public key is used for encryption and private for decryption.

I assumed that I can have public key which doesn't correspond to private key using during encryption so for tests I created own keys in such way:

openssl genrsa -des3 -out server.key 1024
openssl req -new -key server.key -out server.csr
openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt

I got public key modulus and exponent using command:

openssl x509 -in server.crt -text

For encryption testing I'm using code

//Reads private key from file
//StringPasswordFinder is my tmp implementation of PasswordFinder
PEMReader pemReader = new PEMReader(new FileReader("/path/to/server.key"), new StringPasswordFinder());
KeyPair keyPair = (KeyPair) pemReader.readObject();
PrivateKey pk = keyPair.getPrivate();
//text for encryption
String openText = "openText";
//encryption
Cipher rsaCipher = Cipher.getInstance("RSA", "BC");
rsaCipher.init(Cipher.ENCRYPT_MODE, pk);
byte[] encrypted = rsaCipher.doFinal(openText.getBytes("utf-8"));

And for decryption of encrypted text I use code

//modulus hex got using openssl
byte[] modulus = Hex.decodeHex("very long hex".toCharArray());
//exponent hex got using openssl
byte[] exponent = Hex.decodeHex("010001".toCharArray());
//initialization of rsa decryption engine
RSAEngine rsaEngine = new RSAEngine();
rsaEngine.init(false, new RSAKeyParameters(false, new BigInteger(modulus), new BigInteger(exponent)));
//input - encrypted stream
ByteArrayInputStream bais = new ByteArrayInputStream(encrypted);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
//reading blocks from the input stream and decrypting them
int bytesRead = 0;
byte[] block = new byte[rsaEngine.getInputBlockSize()];
while ((bytesRead = bais.read(block)) > -1) {
    baos.write(rsaEngine.processBlock(block, 0, bytesRead));
}
//dispalying decrypted text
System.out.println(new String(baos.toByteArray(), "utf-8"));

And after all displayed text is not. Can anybody show me where I'm wrong?

Edit: Summing up this problem has no solution. Because it's not possible encrypt message using private key and later decrypt it using public one. At general I mixed up encryption with signing message and decryption with verification. Because during making signature private key is used and public is used during verification. Btw, MByD thx for important clue.

Answer 1

I am not so familiar with java libraries for RSA, the times I tried to implement RSA in java was to build all calculations by myself, but if I understood you correct , I see 2 problems:

1. the data should be encrypted with the public key and decrypted with private key, not the other way around (since everyone with public key will be able to decrypt it...)
2. the public key should match the private key, otherwise, anyone with any private key will be able to decrypt data encrypted with any public key...

Also, for very long data, you should not use public key encryption. Instead, encrypt the data in some other algorithm (RC4, AES, etc.) and encrypt the key in RSA (similar to PGP approach)