javascript : Async/await in .replace

Question

I am using the async/await function the following way

async function(){
  let output = await string.replace(regex, async (match)=>{
    let data = await someFunction(match)
    console.log(data); //gives correct data
    return data
  })
  return output;
}

But the returned data is an promise object. Just confused about the way it should be implemented in such functions with callback.

Answer 1

An easy function to use and understand for some async replace :

async function replaceAsync(str, regex, asyncFn) {
    const promises = [];
    str.replace(regex, (match, ...args) => {
        const promise = asyncFn(match, ...args);
        promises.push(promise);
    });
    const data = await Promise.all(promises);
    return str.replace(regex, () => data.shift());
}

It does the replace function twice so watch out if you do something heavy to process. For most usages though, it's pretty handy.

Use it like this:

replaceAsync(myString, /someregex/g, myAsyncFn)
    .then(replacedString => console.log(replacedString))

Or this:

const replacedString = await replaceAsync(myString, /someregex/g, myAsyncFn);

Don't forget that your myAsyncFn has to return a promise.

An example of asyncFunction :

async function myAsyncFn(match) {
    // match is an url for example.
    const fetchedJson = await fetch(match).then(r => r.json());
    return fetchedJson['date'];
}

function myAsyncFn(match) {
    // match is a file
    return new Promise((resolve, reject) => {
        fs.readFile(match, (err, data) => {
            if (err) return reject(err);
            resolve(data.toString())
        });
    });
}

Answer 2

The native replace method does not deal with asynchronous callbacks, you cannot use it with a replacer that returns a promise.

We can however write our own replace function that deals with promises:

async function(){
  return string.replace(regex, async (match)=>{
    let data = await someFunction(match)
    console.log(data); //gives correct data
    return data;
  })
}

function replaceAsync(str, re, callback) {
    // http://es5.github.io/#x15.5.4.11
    str = String(str);
    var parts = [],
        i = 0;
    if (Object.prototype.toString.call(re) == "[object RegExp]") {
        if (re.global)
            re.lastIndex = i;
        var m;
        while (m = re.exec(str)) {
            var args = m.concat([m.index, m.input]);
            parts.push(str.slice(i, m.index), callback.apply(null, args));
            i = re.lastIndex;
            if (!re.global)
                break; // for non-global regexes only take the first match
            if (m[0].length == 0)
                re.lastIndex++;
        }
    } else {
        re = String(re);
        i = str.indexOf(re);
        parts.push(str.slice(0, i), callback.apply(null, [re, i, str]));
        i += re.length;
    }
    parts.push(str.slice(i));
    return Promise.all(parts).then(function(strings) {
        return strings.join("");
    });
}

Answer 3

So, there's no overload of replace that takes a promise. So simply restate your code:

async function(){
  let data = await someFunction();
  let output = string.replace(regex, data)
  return output;
}

of course, if you need to use the match value to pass to the asynchronous function, things get a bit more complicated:

var sourceString = "sheepfoohelloworldgoocat";
var rx = /.o+/g;

var matches = [];
var mtch;
rx.lastIndex = 0; //play it safe... this regex might have state if it's reused
while((mtch = rx.exec(sourceString)) != null)
{
    //gather all of the matches up-front
    matches.push(mtch);
}
//now apply async function someFunction to each match
var promises = matches.map(m => someFunction(m));
//so we have an array of promises to wait for...
//you might prefer a loop with await in it so that
//you don't hit up your async resource with all
//these values in one big thrash...
var values = await Promise.all(promises);
//split the source string by the regex,
//so we have an array of the parts that weren't matched
var parts = sourceString.split(rx);
//now let's weave all the parts back together...
var outputArray = [];
outputArray.push(parts[0]);
values.forEach((v, i) => {
    outputArray.push(v);
    outputArray.push(parts[i + 1]);
});
//then join them back to a string... voila!
var result = outputArray.join("");

Answer 4

Here's an improved and more modern version of Overcl9ck's answer :

async function replaceAsync(string, regexp, replacerFunction) {
    const replacements = await Promise.all(
        Array.from(string.matchAll(regexp),
            match => replacerFunction(...match)));
    let i = 0;
    return string.replace(regexp, () => replacements[i++]);
}

This requires a newer browser baseline due to String.prototype.matchAll , which landed across the board in 2019 (except Edge which got it in early 2020 with the Chromium-based Edge). But it's at least as simple while also being more efficient, only matching the first time through, rather than creating a useless string, and not mutating the array of replacements in an expensive way.

Answer 5

This replaceAsync function iterates through all occurrences of a substring in a string by a regex and enable you to use an asynchronous exampleReplaceFunc function to replace them one by one (eg based on the match group as parameter).

 const replaceAsync = async (str, regex, getNewSubstr) => { while (str.match(regex)) { const result = str.match(regex); const { index } = result; const [match, group1] = result; const newSubstr = await getNewSubstr(match, group1); str = `${str.substr(0, index)}${newSubstr}${str.substr( index + match.length )}`; } return str; }; const exampleReplaceFunc = async (match, group) => { return new Promise(resolve => { setTimeout(() => { console.log(`'${match}' has been changed to 'new${group}'`); resolve(`new${group}`); }, 1500); }); }; const app = async () => { const str = "aaaaold1 aaold2aa aold3aa old4 aold5aa"; console.log('original string:', str) const newStr = await replaceAsync(str, /old([\\d])/, exampleReplaceFunc); console.log('new string:', newStr); }; app(); 

Answer 6

Here is a pretty alternative method using a recursive function:

async function replaceAsync(str, regex, asyncFn) {
    const matches = str.match(regex);
    if (matches) {
        const replacement = await asyncFn(...matches);
        str = str.replace(matches[0], replacement);
        str = await replaceAsync(str, regex, asyncFn);
    }
    return str;
}

Answer 7

And yet another solution, this time in TypeScript. Similar to Maxime's solution , it avoids the "semantically unusual" initial replace() -call in many of the other solutions by using match() instead.

async function replaceAsync(str: string, regex: RegExp, asyncFn: (match: string) => Promise<string>): Promise<string> {
  const promises = (str.match(regex) ?? []).map((match: string) => asyncFn(match));
  const data = await Promise.all(promises);
  return str.replace(regex, () => data.shift()!);
}

Answer 8

This is Overcl9ck's solution implemented in TS:

const replaceAsync = async (str: string, regex: RegExp, asyncFn: (match: any, ...args: any) => Promise<any>) => {
    const promises: Promise<any>[] = []
    str.replace(regex, (match, ...args) => {
        promises.push(asyncFn(match, args))
        return match
    })
    const data = await Promise.all(promises)
    return str.replace(regex, () => data.shift())
}