Base64 Upload Image Issue

Question

Sorry for the long question, but I wanted to express it as well as I could, and ask it in a way that is understood easily. I have a program that allows a user to crop an image using croppie.js in JavaScript, and send the image to a Hunchentoot server on the backend running a Lisp program. I am having an issue saving the Base64 image to a .png file once the user uploads it. Once the post request is sent to the server I am getting the Base64 image as a string, removing invalid characters from the Base64 request by creating a subsequence of the string without the heading sent by the post request and also substituting the "%" character for the "+" character to make the Base64 valid. Next I remove the substring +3D+3D at the end of my string, because the s-base64 library that I am using in Common Lisp complains +3D+3D is invalid padding, and I replace it with "==" which is considered valid padding. Next I create a byte array by using the s-base64 library to translate the Base64 string to a byte array, and store it in a variable. Then I loop through the byte array and write each byte to the output file. When that is finished I decided to print the end of the byte array to the console so that I could see if the output and ending padding is valid, which it appears to be. Here is that part of the code, with comments to make it clearer:

(define-easy-handler (handle-image :uri "/handle-image.html") ()
    (let ((data-source (hunchentoot:raw-post-data :force-text t))) ;get Base64 string
      (let ((new-string (subseq data-source 36))) ;create a subsequence of Base64 string
         (let ((final-string (substitute #\+ #\% new-string))) ;substitute % for +
            (let ((end (search "+3D+3D" final-string))) ;find the invalid padding
                (setf final-string (concatenate 'string (subseq final-string 0 end) "==")) ;add valid padding
                (let ((byte-array (with-input-from-string (in final-string) ;create byte array (or simple-vector) out of Base64 string
                                     (decode-base64-bytes in))))
                   (with-open-file (out "path/path/path/path/profile-image.png" ;create output stream to store file
                                         :direction :output
                                         :if-exists :supersede
                                         :element-type 'unsigned-byte)
                      (dotimes (i (length byte-array)) ;write each byte from the byte array to output stream
                        (write-byte (aref byte-array i) out)))) ;close stream
                (format t "!!!!!!!!: ~a" (subseq final-string (- (length final-string) 30))))))) ;print ending to console to ensure proper padding
       "Upload Successful") ;send response to client

And here is some of my JavaScript code:

$(document).ready(function(){

     $image_crop = $('#image_demo').croppie({
        enableExif: true,
        viewport: {
          width:200,
          height:200,
          type:'square' //circle
        },
        boundary:{
          width:300,
          height:300
        }
   });

As you can see, I first create the cropper. I allow the user to have a 200 x 200 square to crop, and the total size of the cropping space is 300 x 300. There are no issues with that part of the code:

$('#upload_image').on('change', function(){
var reader = new FileReader();
reader.onload = function (event) {
  $image_crop.croppie('bind', {
    url: event.target.result
  }).then(function(){
    console.log('jQuery bind complete');
  });
}

Above I bind the image they've uploaded to the cropper (like if you upload a Facebook image), when they select a file. Again, no issues:

 reader.readAsDataURL(this.files[0]);
$('#uploadImageModal').modal('show');

Above I read the file that they've selected and then the modal "pops up" as if you're cropping a Facebook or Instagram photo:

$('.crop_image').click(function(event){
       $image_crop.croppie('result', {
             type: 'canvas',
             size: 'viewport'
        }).then(function(response){
               $.ajax({
                    url:"handle-image.html",
                    type: "POST",
                    data:{"image": response},
                    success:function(data){
                            $('#uploadImageModal').modal('hide');
                            $('#uploaded_image').html(data);
                     }
              });
          })
       });

Above I upload the ajax request, and if the upload was successful they will get a message from the server that it was success, and I hide the modal for image cropping as well.

Now the issue is that the image is simply blank. I know that the Base64 is valid because I used a Base64 conversion tool to see if the string was valid. I also went back and did my research in regards to Bits, Bytes, Pixels, and dimensions of images to see how the computer interacts with them, so i'm not sure why my image is simply displaying blank. Here is a look at what the cropper looks like on my website:

The bind is working, and then I get the message that the upload was successful. BUT after writing the image and viewing it in the file system it is either a blank image, or will sometimes say that the image type is not supported.

The reason why I am tagging PHP in this post is because I am sure some people have had similar issues in PHP with uploading a cropped image via ajax, and some of those solutions might be applicable in this case, but obviously will require me translating the solution to the lisp syntax. My assumption is that something is wrong with my code when I translate the string to a byte array and write it to a file, but I thought it'd be good to post other sections of my code if I am overlooking something.

Answer 1

As Brad commented, you should first try to use binary uploads directly.

That aside: if you encounter a % in a base64-encoded string, it most likely means that the entire thing is additionally URL-encoded. A quick apropos search gave do-urlencode as a library to decode that. Replacing % with + makes valid base64, but the result does not necessarily represent valid jpg.

Also: use let* instead of nested let forms. Maybe use write-sequence instead of byte-wise output.

Answer 2

Thanks to the answer from @Brad and @Svante I was able to solve the problem. I decided to put the image that I want to upload within a form element, add the blob image from the canvas as FormData for the form, and then send the FormData via an ajax post request:

 $('.crop_image').on('click mousedown touchstart', function(event){ //When the crop image button is pressed event.
    $image_crop.croppie('result', { //Get the result of the cropper

        size: 'viewport', //Set the size to the viewport size (180 x 120)
        format: 'png', //.png format
        type: 'blob'
    }).then(function (blob){
        $form = $('#uploadForm');
        var fd = new FormData($form);
        fd.append('upload_image', blob);
        $.ajax({
        url: "handle-image.html",
        type: "POST",
        data: fd,
        processData: false,
        contentType: false,
        success: function (data){
            $('#uploadImageModal').modal('hide');
            $('#uploaded_image').html(data);
        }
        });
    })
 });

Here I just decided to change the croppie result from the type "canvas"to the type "form", and specify that it will be a .png file. From there I added the form data from my newly created form to the .ajax request and went from there. Thanks to @Brad and @Svante for the help here.