Removing duplicates from a list collection

Question

Hope someone can help me. I am using c# and I am somewhat new to it. I am loading a text file into my app and splitting the data on "," I am reading part of string into a <list> when the data is read there is a lot of duplicates that vary depending on the txt file that I load. Can someone tell me how to check the and remove any and all duplicates that come up. There is no way of knowing what duplicates will show up as there is infinite possibilities that it could be.

Thanks for the help in advance

Answer 1

如果您的目标是.NET 3.5，请使用Distinct扩展方法：

var deduplicated = list.Distinct();

Answer 2

如果将字符串加载到Set而不是List则会自动丢弃重复项。

Answer 3

A simple/dirty example follows:

public List<string> RemoveDuplicates(List<string> listWithDups)
{
   cleanList = new List<string>();
   foreach (string s in listWithDups)
   {
      if (!cleanList.Contains(s))
         cleanList.Add(s);
   }
   return cleanList;
}

As a warning: String.Split on very large strings can cause consume huge amounts of memory and cause exceptions.

Answer 4

Here's an article with some examples and explanations in C#. Basically, you just keep track of the uniques, and check each element.

Alex