I want to get the most used word from an array. The only problem is that the Swedish characters (Å, Ä, and Ö) will only show as .
$string = 'This is just a test post with the Swedish characters Å, Ä, and Ö. Also as lower cased characters: å, ä, and ö.';
echo '<pre>';
print_r(array_count_values(str_word_count($string, 1, 'àáãâçêéíîóõôúÀÁÃÂÇÊÉÍÎÓÕÔÚ')));
echo '</pre>';
That code will output the following:
Array
(
[This] => 1
[is] => 1
[just] => 1
[a] => 1
[test] => 1
[post] => 1
[with] => 1
[the] => 1
[Swedish] => 1
[characters] => 2
[�] => 1
[�] => 1
[and] => 2
[�] => 1
[Also] => 1
[as] => 1
[lower] => 1
[cased] => 1
[�] => 1
[�] => 1
[�] => 1
)
How can I make it to "see" the Swedish characters and other special characters?
All of this is running under the assumption that you're using UTF-8.
You can take a naive approach using preg_split()
to split your string on any separator, punctuation, or control character.
preg_split
example: $split = preg_split('/[\pZ\pP\pC]/u', $string, -1, PREG_SPLIT_NO_EMPTY);
print_r(array_count_values($split));
Array
(
[This] => 1
[is] => 1
[just] => 1
[a] => 1
[test] => 1
[post] => 1
[with] => 1
[the] => 1
[Swedish] => 1
[characters] => 2
[Å] => 1
[Ä] => 1
[and] => 2
[Ö] => 1
[Also] => 1
[as] => 1
[lower] => 1
[cased] => 1
[å] => 1
[ä] => 1
[ö] => 1
)
This works fine for your given string, but does not necessarily split words in a way that is locale-aware. For example contractions such as "isn't" would be broken up into "isn" and "t" by this.
Thankfully the Intl extension adds a great deal of functionality for dealing with things like this in PHP 7.
The plan would be to:
* Normalize the input with Normalizer::normalize()
to make sure graphemes are all encoded in a consistent manner. For example ä
might be encoded, and hence counted, in a couple ways:
Get an IntlBreakIterator
that breaks on words in a locale-dependent way via IntlBreakIterator::createWordInstance()
. This understands what makes up a "word" for the given locale, including handling contractions like "isn't".
Get its IntlPartsIterator
viaIntlBreakIterator::getPartsIterator()
for ease of iterating over the text fragments.
Skip things you don't care about via IntlChar::ispunct()
and IntlChar::isspace()
(*Note that you'll likely want to perform normalization regardless of what method you use to break up the string - it'd be appropriate to do before the preg_split
above or whatever you decide to go with.)
$string = Normalizer::normalize($string);
$iter = IntlBreakIterator::createWordInstance("sv_SE");
$iter->setText($string);
$words = $iter->getPartsIterator();
$split = [];
foreach ($words as $word) {
// skip text fragments consisting only of a space or punctuation character
if (IntlChar::isspace($word) || IntlChar::ispunct($word)) {
continue;
}
$split[] = $word;
}
print_r(array_count_values($split));
Array
(
[This] => 1
[is] => 1
[just] => 1
[a] => 1
[test] => 1
[post] => 1
[with] => 1
[the] => 1
[Swedish] => 1
[characters] => 2
[Å] => 1
[Ä] => 1
[and] => 2
[Ö] => 1
[Also] => 1
[as] => 1
[lower] => 1
[cased] => 1
[å] => 1
[ä] => 1
[ö] => 1
)
This is more verbose but may be worthwhile if you'd prefer ICU (the library backing the Intl extension) to do the heavy lifting when it comes to understanding what makes up a word.
Here is a solution with regex using Unicode punctuation to split the "words" then just a regular array occurrence count.
array_count_values(preg_split('/[[:punct:]\s]+/u', $string, -1, PREG_SPLIT_NO_EMPTY));
Produces:
Array
(
[This] => 1
[is] => 1
[just] => 1
[a] => 1
[test] => 1
[post] => 1
[with] => 1
[the] => 1
[Swedish] => 1
[characters] => 2
[Å] => 1
[Ä] => 1
[and] => 2
[Ö] => 1
[Also] => 1
[as] => 1
[lower] => 1
[cased] => 1
[å] => 1
[ä] => 1
[ö] => 1
)
This was tested in a unicode console, you might want to empose a encoding if you are using a browser. Either make a <meta>
tag or set encoding within your browser, or send PHP headers.
我设法通过将ÅåÄäÖö
添加到àáãâçêéíîóõôúÀÁÃÂÇÊÉÍÎÓÕÔÚ
来删除 符号。
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.