How to run cross-domain XHR request with correct permissions in a Webextension?

Question

Goal : I want to develope a Firefox Webextension (similar to Chrome extensions), which detects HTML and JavaScript files, before they are loaded. If there is specific content in those files, they will be blocked, otherwise they are allowed to pass.

Problem : The file content which have a different domain can not be collected, because they throw a "Cross-Origin" error, because the Access-Control-Allow-Origin header is missing.

I read lots of stuff about this problem, and the documentation says, that the Access-Control-Allow-Origin header will not be needed, if the permissions are set in the Webextension manifest. Here the quotes of the Mozilla Doc :

Use the permissions key to request special powers for your extension. [...] The key can contain three kinds of permissions: [...] host permissions [...] Host permissions are specified as match patterns, and each pattern identifies a group of URLs for which the extension is requesting extra privileges. The extra privileges include: XHR access to those origins [...]

My manifest.json:

{
  [...],    
  "permissions": [
    "tabs",
    "*://*/*",
    "webRequest",
    "webRequestBlocking",
    "<all_urls>"
  ],    
  "background": {
    "scripts": ["backgroundscript.js"]
  },    
  "content_scripts": [
    {
      "matches": ["*://*/*"],
      "js": ["/lib/jquery-2.2.4.min.js", "/contentscript.js"],
      "run_at": "document_start"
    }
  ]
}

Here, i have "*://*/*" in the permissions key, what means, each web ressource should have permissions and the Cross-Origin error should not occur? Or i am wrong? Can anyone tell my, why i get the error or how i can avoid it?

My backgroundscript.js:

chrome.webRequest.onBeforeRequest.addListener(
    logURL,
    {urls: ["<all_urls>"], types: ["main_frame", "script"]},
    ["blocking"]
);

function logURL(requestDetails) {
    chrome.tabs.sendMessage(
        requestDetails.tabId,
        {action: "getContentByURL", url: requestDetails.url, type: requestDetails.type},
        function(response) {
            console.log(response);
        }
    );
    if(requestDetails.type == 'script') {
        // here will be the conditions, based on the content of the files,
        // if they will be canceled or allowed to pass
        // actually, there is just a dummy "false"
        return {cancel: false};
    }
}

My contentscript.js:

chrome.runtime.onMessage.addListener(
    function(message, sender, sendResponse) {
        var contentAll = [];
        if(message.action == 'getContentByURL') {
            var pageContent = getContentByURL(message.url);
            contentAll.push(pageContent);
            sendResponse({"content" : contentAll});
        }
    }
);
function getContentByURL(url) {
    $(document).ready(function() {
        $.get(url, function(data) {
            console.log(data);
        });
    });
}

In the contentscript.js, i use the jQuery $.get method, to access the website content. I also tried $.ajax with dataType jsonp , but in this case, i get a infinity access chain, and the script tries to load the ressources unlimited times. I don't understand it at all, why this happens, maybe because i use the chrome.webRequest.onBeforeRequest Listener, which will be accessed if a new connection occurs, and in this case it runs into the endless loop?

In the Mozilla Doc i read, that chrome.webRequest.onBeforeRequest has a parameter, the requestBody:

Contains the HTTP request body data. [...] 1. Firefox does not support the "requestBody" option.

1. This solution would be the best => but it is not available
2. I tried $.get with permission pattern => i get Cross-Origin error
3. I tried $.ajax with jsonp and same permission pattern => i get endless loop

So the question again: How can i access the content of files of a different domain without a Cross-Origin error, where the domain name is open (pattern like "*://*/*")?

Answer 1

Finally i could fix my problem with the following code at contentscript.js:

chrome.runtime.onMessage.addListener(
    function(message, sender, sendResponse) {
        if(message.action == 'getContentByURL') {
            getContentByURL(message.url, function(result) {
                sendResponse({"content" : result});
            });
        } else {
            sendResponse('error');
        }
        return true;
    }
);

function getContentByURL(url, callback) {
    var req = new XMLHttpRequest();
    if(req) {
        req.open('GET', url, true);
        req.onreadystatechange =  function() {
            if (req.readyState == 4) {
                callback(req.responseText);
            }
        };
        req.send();
    }
}

One important change is to use the XMLHttpRequest() object instead of the jQuery methods. This was the solution for my asked question in my case. I tried this before, but missed the check at req.onreadystatechange, so i did it wrong. I tried it also with req.onload, this works for me too!

To get the example run, there are two another important points.

First, i have to throw the content (req.responseText) back with a callback, so i can send the response back from the content script to the background script.

Second, because the response message is asynchronous, i have to set return on true. This is the notification for the background script message listener, to wait for the response. If this is missing, the response message with the website content will never deliver at the background script.

Finally, this leads to a "general" problem which is not directly part of the question. chrome.webRequest.onBeforeRequest in the backgroundscript.js requires a synchron handling for the "cancel" return values (true/false) for the decision making to block the loaded urls. But to load the content, there will always be an asynchron request needed, so this problem is not fixable? If i find a solution, i will update this answer.

I hope this answer will help others too.