How to interact with an Emscripten Web Worker directly from a native JavaScript front

Question

How can I efficiently send a list of arguments into an Emscripten Web Worker from native JavaScript? My data (set of arguments) is an inhomogeneous combination of large arrays of Float32Array , UInt16Array , int , etc. The example below works, but I cannot use a list, dictionary, or any other way of making a tuple of TypedArrays mixed with int. Currently, unless the input is one single TypedArray, it is converted into a few bytes. For example, the following line will send three single bytes, one for each argument.

        test_data_worker([b8verts, 150000, b8faces]);  // doesn't work

My question is in fact, how to encode a tuple into a C struct. I can use libraries such as struct.js but they will be inefficient (needs conversion, and the arguments will not be transferable). What solution Emscripten has for such a case. Note that my special requirement is that I want my JavaScript to postMessage() directly into an Emscripten-compiled C++ function, without being mediated by a sender C++ program on the front side, or a receiver JavaScript code on the Worker side.

<html>
<script>
    'use strict';
    var worker = new Worker('./emworker.compiled.js');

    var api_info = {
        test: {
            funcName: "worker_function",
            callbackId: 3,
        },
    };

    function test_data_worker(arguments_data_buffer) {
        // The protocol used by Emscripten
        worker.postMessage( {
                funcName: api_info.test.funcName,
                callbackId: api_info.test.callbackId,
                data: arguments_data_buffer,
                // 'finalResponse': false,
        }
        // , [arguments_data_buffer]  //uncommet for transferable
       );
    }

    function demo_request() {
        var verts = new Float32Array([3.141592651234567890123456780,1,2, 3,4,5, 6,7,8, 9,0.5,1.5]);
        var faces = new Uint16Array([0,1,2, 1,2,3, 0,1,3, 0,2,3]);
        var b8verts=new Uint8Array(verts.buffer);
        var b8faces=new Uint8Array(faces.buffer);

        test_data_worker(b8verts);  // b8verts.buffer to make it transferrable
// Desired:
        //test_data_worker([b8verts, b8faces]);  // Doesnt work. sends two bytes instead
        //test_data_worker([b8verts, 60, b8faces]);  // doesnt work
        //test_data_worker({verts:b8verts, d:60, faces:b8faces});  // doesnt work
    }


    worker.addEventListener('message', function(event) {
        // event.data is {callbackId: -1, finalResponse: true, data: 0}
        switch (event.data.callbackId) {
            case api_info.test.callbackId: //api_revlookup.:
                // console.log("Result sent back from web worker", event.data);
                // Reinterpret data
                var uint8Arr = event.data.data;
                var returned_message = String.fromCharCode.apply(null, uint8Arr)
                console.log(returned_message);
                break;
            default:
                console.error("Unrecognised message sent back.");
        }
    }, false);

    demo_request();
    console.log("request() sent.");
</script>
</html>

and the Worker's C++ code is like

#include <iostream>
#include <emscripten/emscripten.h>

extern "C" {
    int worker_function(void* data, int size);
}

std::string hexcode(unsigned char byte) {
    const static char codes[] = "0123456789abcdef_";
    std::string result = codes[(byte/16) % 16] + ( codes[byte % 16] + std::string());
    return std::string(result = codes[(byte/16) % 16] + ( codes[byte % 16] + std::string()));
}


int worker_function(void* data, int size) {
    {
    std::cout << "As bytes: ";
    size_t i = 0 ;
    char* char_data = (char*)data;
    for (; i < size; ++i) {
        std::cout << hexcode((char_data)[i]) << " ";
    }
    std::cout << std::endl;
    }

    {
    std::cout << "As float: ";
    float* typed_data = (float*)data;
    size_t typed_size = (size + sizeof(float)-1) / sizeof(float);
    for (size_t i = 0; i < typed_size; ++i) {
        std::cout << typed_data[i] << " ";
    }
    std::cout << std::endl;
    }

    std::string result = std::string("I received ") + std::to_string(size) + " bytes.";
    char* resstr = (char*)result.c_str();
    emscripten_worker_respond(resstr, result.size()); // not needed really
    return 314;  // ignored
}

void worker_function2(float*verts, int numverts, int*faces, int numfaces) {
    //
}

int main(){return 0;}

which is compiled with Emscripten in the following way:

em++  -s EXPORTED_FUNCTIONS="['_main', '_worker_function' , '_worker_function2' ]" \
  -s NO_EXIT_RUNTIME=1 \
  -s DEMANGLE_SUPPORT=1 \
  -s BUILD_AS_WORKER=1 -DBUILD_AS_WORKER  \
  -pedantic -std=c++14 \
  emworker.cpp \
  -o ./emworker.compiled.js

My API on the web worker needs to send and receive tuples of multiple types, ie it will have input and output like the following:

typedef struct vf_pair {
    std::vector<float> verts,  // or a pair<float*,int>
    std::vector<int> faces
} mesh_geometry;
int query_shape(char* reduce_operator, char* shape_spec_json, float* points, int point_count);
struct vf_pair get_latest_shape(int obj_id);
struct vf_pair meshify(char* implicit_object);

Answer 1

You can find the patterns I used in the following repo. It is efficient. It exchanges large typed arrays and primitive-typed values back and forth with Web Workers:

https://github.com/sohale/implisolid/blob/master/js_iteration_2/js/implisolid_worker.js

How to interact with an Emscripten Web Worker directly from a native JavaScript front

Question

1 answers

solution1
0 ACCPTED 2020-02-07 11:19:47

How to interact with an Emscripten Web Worker directly from a native JavaScript front

Question

1 answers

solution1 0 ACCPTED 2020-02-07 11:19:47

solution1
0 ACCPTED 2020-02-07 11:19:47