[英]python remove repeated values in set
我有一个看起来像这样的集合:
my_set = {
[
{
"sample_id": "read1",
"seg_1": None,
"lukM-F": "D",
"23s_SA": None,
"see": None,
"sed": "ND"
},
{
"sample_id": "read2",
"seg_1": None,
"lukM-F": "ND",
"23s_SA": None,
"see": "D",
"sed": "ND"
},
{
"sample_id": "read3",
"seg_1": None,
"lukM-F": "D",
"23s_SA": None,
"see": "ND",
"sed": "None"
}
]
}
我想删除整个字符串中值为“ None”的键。 例如,例如:如果“ none”是每个sample_id(read1 AND read2 AND read3)中键“ seg_1”的值,则将其全部删除。 如果在“ seg_1”中有一个“无”,例如在read1中说,而另外两个sample_id不是“无”,则保留“ seg_1”及其值。 因此,我想得出以下结论:
my_set = {
[
{
"sample_id": "read1",
"lukM-F": "D",
"see": None,
"sed": "ND"
},
{
"sample_id": "read2",
"lukM-F": "ND",
"see": "D",
"sed": "ND"
},
{
"sample_id": "read3",
"lukM-F": "D",
"see": "ND",
"sed": "None"
}
]
}
请注意,seg_1和23s_SA现在已被删除,因为它们在所有sample_id中的值为“ None”。
我花了很长时间尝试这样做,但没有成功。 我终于将集合转换为dict,然后列出,然后遍历所有列表,并删除所有列表中始终不包含任何内容的所有项目。
number_of_samples = len(my_set)
each_sample_list = [[] for i in range(0, number_of_samples)]
n = 0
for data_in_dict in my_set:
for k,val in data_in_dict.items():
each_sample_list[n].append([k,val])
if n == number_of_samples:
break
else:
print each_sample_list[n]
n += 1
我曾考虑过使用itertools izip遍历多个列表,但不确定这是否行得通。 任何帮助将不胜感激。
谢谢
您可以创建计数器,然后删除所有需要的密钥:
import collections
import itertools
source = [
{
"sample_id": "read1",
"seg_1": None,
"lukM-F": "D",
"23s_SA": None,
"see": None,
"sed": "ND"
},
{
"sample_id": "read2",
"seg_1": None,
"lukM-F": "ND",
"23s_SA": None,
"see": "D",
"sed": "ND"
},
{
"sample_id": "read3",
"seg_1": None,
"lukM-F": "D",
"23s_SA": None,
"see": "ND",
"sed": "None"
}
]
size = len(source)
# for python2 you should use iteritems() method
iterators_chain = itertools.chain(*[x.items() for x in source])
counter = collections.Counter(iterators_chain)
for (key, val), count in counter.items():
if count == size and val is None:
for x in source:
x.pop(key)
您的my_set
不是有效的集合,因为集合项必须是可哈希的,列表也不是哈希的。 但无论如何...
这是一种不需要任何导入的方法。 它使用集合来确定要保留哪些密钥。
my_stuff = [
{
"sample_id": "read1",
"seg_1": None,
"lukM-F": "D",
"23s_SA": None,
"see": None,
"sed": "ND"
},
{
"sample_id": "read2",
"seg_1": None,
"lukM-F": "ND",
"23s_SA": None,
"see": "D",
"sed": "ND"
},
{
"sample_id": "read3",
"seg_1": None,
"lukM-F": "D",
"23s_SA": None,
"see": "ND",
"sed": None
}
]
allkeys = set(k for d in my_stuff for k in d)
goodkeys = set(k for k in allkeys if any(d.get(k) for d in my_stuff))
badkeys = allkeys - goodkeys
for d in my_stuff:
for k in badkeys:
del d[k]
for d in my_stuff:
print(d)
输出
{'lukM-F': 'D', 'see': None, 'sed': 'ND', 'sample_id': 'read1'}
{'lukM-F': 'ND', 'see': 'D', 'sed': 'ND', 'sample_id': 'read2'}
{'lukM-F': 'D', 'see': 'ND', 'sed': None, 'sample_id': 'read3'}
allkeys
和goodkeys
那些set(...)
构造可以用现代Python版本中的set comprehension替换,但是我在这台古老的机器上使用Python 2.6.6。
构建allkeys
集的另一种方法是
allkeys = set()
for d in my_stuff:
allkeys.update(d.keys())
尽管代码更多,但运行速度更快,因为.update
以C速度处理dict
的整个键集合,而另一种方法则必须以Python速度遍历键。 当然,如果可以保证列表的每个dict
中的键集始终相同,则可以进一步优化它。
利用键在list
中的所有dict
必须为None
bkeys = [k for k, v in next(iter(my_stuff), {}).items() if v is None]
bkeys = [k for k in bkeys if all(d[k] is None for d in my_stuff)]
my_stuff = [{k: v for k, v in d.items() if k not in bkeys} for d in my_stuff]
新的my_stuff
打印输出:
{'see': None, 'sed': 'ND', 'lukM-F': 'D', 'sample_id': 'read1'}
{'see': 'D', 'sed': 'ND', 'lukM-F': 'ND', 'sample_id': 'read2'}
{'see': 'ND', 'sed': None, 'lukM-F': 'D', 'sample_id': 'read3'}
没有dict
理解,只需将最后一行更改为:
my_stuff = [dict(((k, v) for k, v in d.items() if k not in bkeys)) for d in my_stuff]
编辑为仅使用第一个项目的None
键(如果存在)。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.