java - RSA BadPaddingException when trying to decrypt an int

Question

I know similar questions have been asked before, but I can't find an answer specific to my problem. I am trying to use RSA encryption to take an image with encrypted pixels and decrypt it using the same private key. I keep on running into a problem though when I try to decrypt the int I get from the pixels of the image using getRGB(). Here is my Code:

int pixel = img.getRGB(1, 1);
System.out.println(pixel);

byte[] bytes = ByteBuffer.allocate(4).putInt(pixel).array();
System.out.println(fromByteArray(bytes));
bytes = rsa.RSADecryptB(privk, bytes);
int nom = ByteBuffer.wrap(bytes).getInt(); // big-endian by default
System.out.println(nom);

This is in my main class which calls method RSADecryptB from another class, here is that method:

public static []byte RSADecryptB(PrivateKey privatekey, byte[] cipherText) {
byte[] ll = null;
  try {
     /* Create cipher for decryption. */
     Cipher decrypt_cipher = Cipher.getInstance("RSA");
     decrypt_cipher.init(Cipher.DECRYPT_MODE, privatekey);


     ll = decrypt_cipher.doFinal(cipherText);

  } catch (IllegalBlockSizeException e) {
     System.out.println("1");
     e.printStackTrace();
  }
  catch (InvalidKeyException et) {
     System.out.println("2");
     et.printStackTrace();
  }
  catch (NoSuchAlgorithmException ev) {
     System.out.println("3");
     ev.printStackTrace();
  }
  catch (BadPaddingException ea) {
     System.out.println("4");
     ea.printStackTrace();
  }
  catch (NoSuchPaddingException eo) {
     System.out.println("5");
     eo.printStackTrace();
  }
  return ll;

}

When I run the program the output is this:

-1606258341
5
4
javax.crypto.BadPaddingException: Decryption error
at sun.security.rsa.RSAPadding.unpadV15(RSAPadding.java:380)
at sun.security.rsa.RSAPadding.unpad(RSAPadding.java:291)
at com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:363)
at com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:389)
at javax.crypto.Cipher.doFinal(Cipher.java:2165)
at Encryption5.RSADecryptB(Encryption5.java:126)
at ImageEncryptor.main(ImageEncryptor.java:405)
Exception in thread "main" java.lang.NullPointerException
at java.nio.ByteBuffer.wrap(ByteBuffer.java:396)
at ImageEncryptor.main(ImageEncryptor.java:406)

I don't understand why this is going on. -1606258341 is the value of the pixel, the output of 5 and 4 are because of the NoSuchPaddingException and the BadPaddingException. If anyone can help explain why this is happening and what I'm doing wrong that would be great. Thanks!

Edit: I know I probably shouldn't be using RSA directly, but I am also doing the encryption myself, pixel by pixel. I'm doing this more to practice using bufferedimages/ byte arrays / rsa rather than to create the most efficient/secure program. Also, when I decrypt the integer in my encryption method it works fine. Its when I decrypt it from a different method that I get the error. Also, after encrypting the image is saved to a file, then extracted to decrypt. Here is my Encryption method also (it doesn't use any padding):

 int w = img.getWidth();
     int h = img.getHeight();
     byte[] bytes = null;
     ByteBuffer wrappedo = null;
     int nom = 0;
     ByteBuffer wrapped = null;
     int num = 0;
     int pixel = 0;
System.out.println("width, height: " + w + ", " + h);

for (int i = 0; i < h; i++) {
  for (int j = 0; j < w; j++) {

     pixel = img.getRGB(j, i);

bytes = ByteBuffer.allocate(4).putInt(pixel).array();
bytes = rsa.RSAEncrypt(bytes);
wrappedo = ByteBuffer.wrap(bytes); // big-endian by default
nom = wrappedo.getInt();

img.setRGB(j, i, nom);

}

  } 

Then here's the RSAEncrypt method:

public static byte[] RSAEncrypt(byte[] data) {
byte[] cipherData = null;
try {
PublicKey pubKey = readKeysFromFile();
Cipher cipher = Cipher.getInstance("RSA");
cipher.init(Cipher.ENCRYPT_MODE, pubKey);
cipherData = cipher.doFinal(data);
}
catch (Exception e) {}

return cipherData;
}

Answer 1

You need the method that was used to encrypt the image. It almost certainly cannot be encrypted the way you currently expect it to be. It may be plain / raw / textbook / toy RSA (just modular exponentiation) but in that case you need to create the decryption routine yourself.

It is very unlikely that each and every pixel was encrypted separately. As JB Nizet already mentioned, that would be very inefficient. RSA output is the size of the key pair (which in turn is the same as the size in bytes of the modulus). The key size of an RSA key pair should at the very minimum be 512 bits to make it hard to decrypt, and about 2048 bits to be secure. Usually a hybrid scheme is used to encrypt larger text using RSA (first encrypt using AES, then encrypt the AES key using RSA, for instance).

Furthermore, you are currently specifying "RSA" as algorithm. For the SUNRSA provider (and therefore most others) this translates into "RSA/ECB/PKCS1Padding" as "ECB" and "PKCS1Padding" are the defaults. PKCS#1 padding has an overhead of 11 bytes. So if your ciphertext is smaller than the modulus or the ciphertext is smaller than 11 bytes than the decryption must fail - and it does.