带有特殊字符的坏词过滤器

Question

I am using https://www.npmjs.com/package/bad-words and i created regex for filter special characters.

const Filter = require('bad-words');
const badWordsFilter = new Filter({replaceRegex:  /[A-Za-z0-9öÖÇ窺ĞğİıÜü_]/g});
badWordsFilter.addWords(['badword', 'şğ'])

If word doesn't contain turkish character, it works. But if i write turkish character like ş or ğ it is not filtering.

Is my regex wrong?

I found this code in documentation:

var filter = new Filter({ regex: /\*|\.|$/gi });
var filter = new Filter({ replaceRegex:  /[A-Za-z0-9가-힣_]/g }); 
//multilingual support for word filtering

Answer 1

You obviously have an encoding problem since your regex works out of your app, see here: https://regex101.com/r/VpItfH/3/ .

So I think encoding your characters in your regex in your app may help:

See the encoded regex result here: https://regex101.com/r/VpItfH/4/

---

More details

Trying the following encoded regex in a PCRE regex engine will work ( https://regex101.com/r/VpItfH/5 ):

/[A-Za-z0-9\x{f6}\x{d6}\x{c7}\x{e7}\x{15e}\x{15f}\x{11e}\x{11f}\x{130}\x{131}\x{dc}\x{fc}_]/g

but when selecting a javascript regex engine the { , } will break the unicode so you need to remove them and if the character is not recognized then replace \\x with \\u0\u003c/code> . Eg \\x{15e} becomes \Ş

Then you can do the same match as when you use /[A-Za-z0-9öÖÇ窺ĞğİıÜü_]/g .

Note : to get the unicode form of a character, you can do "Ğ".charCodeAt(0).toString(16); and prefix it with \\x or \\u0\u003c/code> .

Hope this can help, and at least acknowledge that you can encode characters inside a regex and still match the same. :)

Answer 2

Can you please try with:

var filter = new Filter({ replaceRegex: /(\\w+)/gi });

For sure you have to use replaceRegex option.

---

The pattern matches everything case insentively.

Here's what /(\\w+)/gi does descriptively (thanks to regex101 ):

1. 1st Capturing Group (\\w+).
   1. \\w+ matches any word character (equal to [a-zA-Z0-9_])
   2. + Quantifier — Matches between one and unlimited times, as many times as possible, giving back as needed (greedy)
2. Global pattern flags
   1. i modifier : insensitive. Case insensitive match (ignores case of [a-zA-Z])
   2. g modifier : global. All matches (don't return after first match)

Answer 3

You need to make that regular expression Unicode-aware by adding the u flag to it. More precisely, change /[A-Za-z0-9öÖÇ窺ĞğİıÜü_]/g into /[A-Za-z0-9öÖÇ窺ĞğİıÜü_]/gu (added a u at the end). This will work only in modern browsers (basically, all but Internet Explorer) though. There are other options as well, that you may want to consider if you want to support older browsers.

Answer 4

Encode your javascript file into utf-8 and update your meta tag to:

<meta http-equiv="content-type" content="text/html;charset=utf-8" />

Hoping this will help you.