在列表/文件中查找重复项。 [Groovy/Java]

Question

I have an input file where each line is a special record. I would gladly work on the file level but might be a more convenient way to transfer the file into a list. (each object in the list = each row in the file) In the input file, there can be several duplicate rows. The goal: Split the given file/list into unique records and duplicate records, ie, Records which are present multiple times, keep one occurrence and other duplicate parts store in a new list I found an easy way how to remove duplicates but never found a way how to store them

File inputFile = new File("....")
inputFile.eachLine {    inputList.add(it)   } //fill the list
List inputList = [1,1,3,3,1,2,2,3,4,1,5,6,7,7,8,9,8,10]
inputList = inputList.unique() // remove duplicates
println inputList
// inputList = [1, 3, 2, 4, 5, 6, 7, 8, 9, 10]

The output should look like: Two lists/files with removed duplicates and duplicates itself

inputList = [1,3,2,4,5,6,7,8,9,10] //only one ocurance of each line
listOfDuplicates = [1,1,1,3,3,2,7,8] //duplicates removed from original list

The output does not need to correspond with the initial order of items. Thank you for help, Matt

Answer 1

You could simply iterate over the list yourself:

def inputList = [1,1,3,3,1,2,2,3,4,1,5,6,7,7,8,9,8,10]

def uniques = []
def duplicates = []

inputList.each { uniques.contains(it) ? duplicates << it : uniques << it }

assert inputList.size() == uniques.size() + duplicates.size()
assert uniques == [1,3,2,4,5,6,7,8,9,10] //only one ocurance of each line
assert duplicates == [1,3,1,2,3,1,7,8] //duplicates removed from original list

inputList = uniques // if desired

Answer 2

There are many ways to do this,following is the simplest way

def list = [1,1,3,3,1,2,2,3,4,1,5,6,7,7,8,9,8,10]
        def unique=[]
        def duplicates=[]
        list.each {
            if(unique.contains(it))
                duplicates.add(it)
            else
                unique.add(it)

        }
        println list //[1, 1, 3, 3, 1, 2, 2, 3, 4, 1, 5, 6, 7, 7, 8, 9, 8, 10]
        println unique //[1, 3, 2, 4, 5, 6, 7, 8, 9, 10]
        println duplicates //[1, 3, 1, 2, 3, 1, 7, 8]

Hope this will helps you

Answer 3

Something very straight-forward:

List inputList = [1,1,3,3,1,2,2,3,4,1,5,6,7,7,8,9,8,10] 
def uniques = [], duplicates = []

Iterator iter = inputList.iterator()
iter.each{
  iter.remove()
  inputList.contains( it ) ? ( duplicates << it ) : ( uniques << it )
}

assert [2, 3, 4, 1, 5, 6, 7, 9, 8, 10] == uniques
assert [1,1,3,3,1,2,7,8] == duplicates

Answer 4

This code should solve the problem

 List listOfDuplicates = inputList.clone()
 listOfDuplicates.removeAll{
    listOfDuplicates.count(it) == 1
 }

Answer 5

If order of duplicates isn't important:

def list = [1,1,3,3,1,2,2,3,4,1,5,6,7,7,8,9,8,10]
def (unique, dups) = list.groupBy().values()*.with{ [it[0..0], tail()] }.transpose()*.sum()
assert unique == [1,3,2,4,5,6,7,8,9,10]
assert dups == [1,1,1,3,3,2,7,8]

Answer 6

The more the merrier:

groovy:000> list.groupBy().values()*.tail().flatten()
===> [1, 1, 1, 3, 3, 2, 7, 8]

1. Group by identity (this is basically a "frequencies" function).
2. Take just the values
3. Clip the first element
4. Combine the lists