repeated words from text file in c++

Question

This program finds words and erases all of them that are repeated in the text file. Here, I wrote this code to do so by inputting a specific word, but I want the program to find the kind of words by itself and in the result will show only the unrepeated words. I have tried by best but have failed. Could you advise me to find a better way?

int main()
{
   ifstream fin("name.txt");
   ofstream fout("result.txt");
   string word;
   string line;

   int pos;
   int ccount=0;;

   cout<<"Please input word:";
   cin>>word;

   while(getline(fin,line))
   {
       pos= 0;
       while(line.find(word,pos)!=string::npos)
       {
               pos = line.find(word,pos);
               line.erase(pos,word.length());
       }

        fout<<line<<endl;
   }
    fin.close();
    fout.close();
}

Answer 1

You can use an std::vector to keep track of words as you read them and a std::set to make sure you only add it once to the std::vector . The reason you want the std::vector is because the std::set won't keep the std::string s in the order of insertion. This should do the job:

#include <algorithm>
#include <fstream>
#include <unordered_set>
#include <sstream>
#include <string>
#include <vector>

int main()
{
    std::vector<std::string> words;
    std::unordered_set<std::string> uniqueWords;

    std::ifstream fin("in.txt");
    if (fin.is_open())
    {
        std::string line;
        while (getline(fin, line))
        {
            std::istringstream iss(line);
            for (std::string word; iss >> word;)
            {
                std::string lowercaseWord = word;
                std::transform(word.begin(), word.end(), lowercaseWord.begin(), ::tolower);
                if (uniqueWords.find(lowercaseWord) == uniqueWords.end())
                {
                    uniqueWords.insert(lowercaseWord);
                    words.push_back(word);
                }
            }
        }

        fin.close();
    }

    std::ofstream fout("out.txt");
    if (fout.is_open())
    {
        for each (std::string word in words)
        {
            fout << word << " ";
        }

        fout.close();
    }

    return 0;
}

Answer 2

Ok it can be done with only reasonably complex tools: only the locale is required to convert all words to lower case and process punctuations.

Specifications:

• a word is a sequence of non space characters neither starting nor ending with a punctuation
• words are separated with each others by at least one space character
• in order to identify unique words they are first converted to lower case
• uniq word will be stored in a set: the insertion order is lost and only alphabetic order remains
• spaces and punctuations are those of the default locale

Here is a possible code:

#include <iostream>
#include <fstream>
#include <string>
#include <set>
#include <locale>

int main() {
    std::ifstream in("name.txt");      // io streams
    std::ofstream out("result.txt")
    std::set<std::string> uniq_words;  // a set to store unique words

    // ctyp facet of default locale to identify punctuations
    std::locale loc;
    const std::ctype<char>& ctyp = std::use_facet<std::ctype<char> >(loc);

    std::string word;
    while (in >> word) {          // split text in words
        // ignore any punctuation at the beginning or the end of the word
        auto beg = word.begin();
        while ((beg != word.end()) && ctyp.is(ctyp.punct, *beg)) beg++;
        auto end = word.end();
        while ((end > beg) && ctyp.is(std::ctype<char>::punct, end[-1])) end--;

        for (auto it=beg; it != end; it++) {     // convert what is kept to lower case
            *it = ctyp.tolower(*it);
        }
        uniq_words.insert(std::string(beg, end));   // insert the lower case word in the set
    }
    // Ok, we have all unique words: write them to output file
    for (auto w: uniq_words) {
        out << w << " ";
    }
    out << std::endl;
    return 0;
}