I have a ConcurrentDictionary<string, int> word = new ConcurrentDictionary<string, int>();
And i have List for files: static List<string> file = new List<string>(60);
Before, I used to use this path to write file information string line;
string[] lines;
string[] linesUnos;
string[] sp = { " " };
int i = (int)obj;
StreamReader reder = new StreamReader(file[i]);
line = " ";
while (!reder.EndOfStream)
{
line = reder.ReadLine().ToLower();
lines = line.Split(sp, StringSplitOptions.RemoveEmptyEntries);
foreach (var w in lines )
{
word.AddOrUpdate(w, 1, (key, oldValue) => oldValue + 1);
}
}
In this case the right to use a LINQ? I try this way,but it't not right:
IEnumerable<string> f, w;
line = reder.ReadLine().ToLower();
f = from lin in line.Split(sp, StringSplitOptions.RemoveEmptyEntries) select lin;
word = line.ToDictionary(s => s.Key, s => s.Count());
Let me try doing some reverse engineering . It seems, that you have several files (in the file
list) organized like this:
some words
several new words
nothing new
and you want some kind of a dictionary as a result
{"some", 1} // 1: "some' appears just once
{"words", 2} // 2: 'words' appears twice
{"several", 1}
{"new", 2}
{"nothing", 1}
...
where Key
is a word itself, and Value
is its frequency. If it's your case and if you want to implement it with a help of Linq while using parallel processes , try using PLinq ; something like this:
Dictionary<string, int> dict = file
.AsParallel() // Doing in parallel
.SelectMany(file => File.ReadLines(files)) // Read all lines of all files
.SelectMany(line => line.Split( // Split each line into words
new char[] {' '},
StringSplitOptions.RemoveEmptyEntries))
// You may want to process words here, e.g. get rid of "stop words"
.GroupBy(w => w) // Group by each word
.ToDictionary(chunk => chunk.Key, chunk => chunk.Count());
Please, notice that you'll get just Dictionary
, not ConcurrentDictionary
.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.