拆分字符串并确保结果数组中没有重复项的最有效方法是什么？

Question

I am splitting a javscript string into an array whose elements just contain sequences of cyrillic characters.

    var text = "где по его проекту был реализован первый в мире компьютер с хранимой в памяти программой — ACE."
    text=text.toLowerCase();
    var re = /[^йцукенгшщзхъёэждлорпавыфячсмитьбю]+/;
    words = text.split(re);

In the above snippet words will contain the following

["где", "по", "его", "проекту", "был", "реализован", "первый", "в", "мире", "компьютер", "с", "хранимой", "в", "памяти", "программой", ""] 

I need to remove the duplicate from the array. Namely I should only see the occurence of "в" once. I know I can after the split and go through the array doing this but not sure what is the best way. Is it possible to do this with the split regex?

Jonathan

Answer 1

Not the most efficient, but it's clean and simple.

text.split(re).filter(function(str, idx, txtArray) {
    return txtArray.indexOf(str) === idx; 
});

Basically, if the first index found doesn't match the current index in the iteration, it's a duplicate.

Answer 2

You have to go through the array. You can remember whether you've seen instances of the string before using an object as a map, eg:

var a = /* ...get the array... */;
var unique = [];
var n, len;
var str;
var seen = {};
for (n = 0, len = a.length; n < len; ++n) {
    str = a[n];
    if (!seen[str]) {
        seen[str] = true;
        unique.push(str);
    }
}

If there's any chance one of the string values may be a name that already exists on objects (so, "toString" , "valueOf" , "hasOwnProperty" , and such), you have to modify the if (!seen[str]) check to use hasOwnProperty instead:

if (!seen.hasOwnProperty(str)) {

...but if the strings are as you've shown, you don't need that. Another alternative is to use a prefix like "xx":

var keystr = "xx" + str;
if (!seen[keystr]) {
    seen[keystr] = true;
    // ...
}

Since there are no object properties on raw objects that start with "xx" , and almost certainly never will be.

---

In a comment you've said:

I guess by efficient I mean the most elegant of idiomatic javascript way to do this.

Interesting, that's not a definition I'd've used. :-) Okay, here's another approach using ES5's filter , which is definitely more JavaScript-y:

var a = /* ...get the array... */;
var seen = {};
a = a.filter(function(str) {
    if (!seen[str]) {
        seen[str] = true;
        return true;
    }
    return false;
});

Answer 3

If you are willing to use a third party library, then I would recommend to have a look at Underscore . This Library provides a uniq method, that you would apply in the following way:

words = _.uniq(text.split(re));

Answer 4

You can get the "prettiness" of the .indexOf solution using some other built-in functions:

var uniq = Object.keys(text.split(re).reduce(function(words, word) {
  words[word] = null;
  return words;
}, {}));

This'll only work in newer versions of JavaScript (that is, not old versions of IE). This has the advantage, like Mr. Crowder's version, of not being an O(n ² ) algorithm. On fairly large strings without many duplicates (say, a page full of text), those .indexOf() calls will start to warm up the client CPU.

Note that this will give you the unique words in no particular order.

Answer 5

如何在正则表达式中使用负前瞻并使用.match方法返回匹配数组。

([йцукенгшщзхъёэждлорпавыфячсмитьбю]+)(?!.*\1)

Answer 6

You could do this (splitter : " " ) :

var m = 'azerty rty aze rty aze'
    .replace(/(^| )([^ ]+)(?= |$)(?=.* \2( |$))/g, '') // removes duplicates
    .match(/[^ ]+/g) 
m; // ["azerty", "rty", "aze"]

Surely not the most efficient way though.