如何在python中打印唯一行

Question

I am pulling data from oracle db and need print the unique values in standard output:

My data is like this:

server1.CRITICAL_INCIDENTS 1418223897 0.000000 host=server1 type=oracle_database source=Oracle dc=DC1
server1.ResponseTimepertransaction 1418223577 2.467900 host=server1 type=oracle_database source=Oracle dc=DC1
server1.DataDictionaryHitPercent 1418223577 100.000000 host=server1 type=oracle_database source=Oracle dc=DC1
server1.FullIndexScanspersecond 1418223577 0.000000 host=server1 type=oracle_database source=Oracle dc=DC1
server1.ExecutesPerformedwithoutParsesPercent 1418223577 66.666667 host=server1 type=oracle_database source=Oracle dc=DC1
server1.SortsinMemoryPercent 1418223577 100.000000 host=server1 type=oracle_database source=Oracle dc=DC1
server1.BufferCacheHitPercent 1418223577 100.000000 host=server1 type=oracle_database source=Oracle dc=DC1
server1.DatabaseCPUTimePercent 1418223577 81.048665 host=server1 type=oracle_database source=Oracle dc=DC1
server1.CRITICAL_INCIDENTS 1418223897 0.000000 host=server1 type=oracle_database source=Oracle dc=DC1
server1.CRITICAL_INCIDENTS 1418223897 0.2000000 host=server1 type=oracle_database source=Oracle dc=DC1
server1.ResponseTimepertransaction 1418223577 2.467900 host=server1 type=oracle_database source=Oracle dc=DC1

When I am printing this values, I only need to print the unique values. The data that comes from Oracle is Date format and when I convert this to epoch, I may get duplicate values for the same time for the same metric. If the values for time stamp and metric is the same, I only need to print one of them.

For example, I only need to print one of these lines. They have the same time (1418223897) and the same metric (server1.CRITICAL_INCIDENTS). The values are diffent, one is 0.0000000 and 0.2000000, but it is ok to have different values.

server1.CRITICAL_INCIDENTS 1418223897 0.000000 host=server1 type=oracle_database source=Oracle dc=DC1
    server1.CRITICAL_INCIDENTS 1418223897 0.2000000 host=server1 type=oracle_database source=Oracle dc=DC1

I tried this:

import pyodbc
import re
sql="DateTime, Server, Server_Type, Metric, Value from oracle_table"

cnxn = pyodbc.connect("DSN=dsn1;UID=userid;PWD=passwd123")

cursor = cnxn.cursor()


cursor.execute(sql)
row = cursor.fetchall()

    seenAlready = set()
    for line in row:
        if line[4]:
            if float(line[4])>=0:
                outputLine = line[0],line[1],line[2],line[3],line[4]
                outputLine1=line[0],line[3] #DateTime and Metric

                if outputLine1 in seenAlready:
                            continue
                    else:
                            print ' '.join([str(i) for i in outputLine])
                            seenAlready.add(outputLine1)

This is not quite working because even tought DateTime and Metric are the same, each row may be unique because Value my be different.

How could I fix it so that I only print one line for the same DateTime and Metric row?

Answer 1

If you collect all the data and put it into a set (or put each column in its own set) then you will never have duplicates because sets can only have unique items. If a value is an exact duplicate of another value then it should just ignore that if you try and add it to the set. After that you can just loop through the set with a print function.

seenAlready = set()
for line in row:
    if line[4]:
        if float(line[4])>=0:
            outputLine = line[0],line[1],line[2],line[3],line[4]
            outputLine1=line[0],line[3] #DateTime and Metric

            seenAlready.add(outputLine1)

for line in seenAlready:
    print(line) #or whatever formatted value you need to print

Something like this, or in the same vein of thinking. Printing from the set would make the most sense because you are guaranteed to have a set of unique items.