如何在flask网络服务器上检索数据作为文件对象？

Question

I am currently developing a web interface where you can draw a digit and upload it. The image goes through a machine learning model of digit recognition and return the digit that has been transcribed by the user.

HTML Code :

<canvas id="myCanvas">
        Sorry, your browser does not support HTML5 canvas technology.
</canvas>
<button onclick="clickonbutton()">Upload</button>

Canvas is used to draw digit and button to send it to a flask web server.

Javascript Code :

<script type="text/javascript">
function clickonbutton() {
        var canvas = document.getElementById('myCanvas');

        //Convert HTML5 Canvas to bytes-like object then File object
        canvas.toBlob(function(blob) {
                var file = new File([blob], "image.png", { type: "image/png", lastModified : new Date()});

                //Prepare a XMLHttpRequest to send the data with POST method to the server
                var request = new XMLHttpRequest();
                request.open("POST", "/");
                request.setRequestHeader('Content-Type','application/x-www-form-urlencoded');
                //Sending file...
                request.send(file);
         });
}               
</script>

When the user click on the button "Upload" after editing the HTML5 canvas, it run this function which send the file to the server.

Python script (server-side) :

@app.route('/', methods=['GET', 'POST'])
def upload_file():
    if request.method == 'POST':
        #Convert MultiDict to Dict object then string
        data = str(request.form.to_dict())

        #Format string to remove Dict elements
        data = data.replace("{","")
        data = data.replace("}","")
        data = data.replace("'","")
        data = data.replace(": ","")

        #Convert string to bytes-like object
        img_bytes = data.encode()

        #Predict the digit of the user
        _, predicted_class, probabilities = get_prediction(image_bytes=img_bytes)
        predicted_class = int(predicted_class)
        classify(img=_, ps=probabilities)

        #Return the result of the digit prediction
        return render_template('result.html',class_name=predicted_class)
    return render_template('index.html')

On server-side, i get the data from request.form but these are in a ImmutableMultiDict object so i converted the data to string then bytes-like object. Unfortunately, conversion using the .replace() method corrupts the PNG file and there's no tool to do that conversion properly...

Is there a solution to get the data directly in a File object without using an ImmutableMultiDict object ?

Answer 1

So with help from this thread on using the Fetch API and this answer on HTML5 canvas drawing I figured this could be done with something like:

<div id='sketch'>
  <canvas id="myCanvas">
  </canvas>
</div>

<button id='btn'>Upload</button>

<div id='result' style='font-size:2em;'></div>

Notice I've added a div to output the results (the class name) on the same page.

Then in the Javascript use a FormData object: (I've missed out the drawing function for clarity):

// Define the dom elements
const canvas = document.getElementById('myCanvas');
const btn = document.getElementById('btn');
const result = document.getElementById('result');

// This will handle the upload
const upload = (file) => {
  fetch('/', { // Your POST endpoint
    method: 'POST',
    body: file // This is your file object
  }).then(
    response => response.json() // if the response is a JSON object
  ).then(
    success => { result.innerText = success['class_name']; } // Write class_name into results div.
  ).catch(
    error => console.log(error) // Handle the error response object
  );
};

// Listener which handles the button click:
btn.addEventListener('click', function() {

  canvas.toBlob(function(blob) {
    var data = new FormData();
    data.append('file', blob);
    upload(data);
  });

}, false);

Now on the server side, this can be handled like a regular Flask file upload :

@app.route('/', methods=['GET', 'POST'])
def upload_file():
    if request.method == 'POST':
        file = request.files['file']

        #Convert string to bytes-like object
        img_bytes = file.read()

        #Predict the digit of the user
        _, predicted_class, probabilities = get_prediction(image_bytes=img_bytes)
        predicted_class = int(predicted_class)
        classify(img=_, ps=probabilities)

        #Return the result of the digit prediction
        return {'class_name': predicted_class}
    return render_template('index.html')

Any post request now returns a jsonified dictionary, which is in turn written to the results div on the page.

There's probably room for improvment here, like some validation on the server-side aswell as rate limiting and error handling, but the basic concept works.