[英]Processing multiple files and write a csv file for each
我編寫了一個適用於單個文件的代碼,但我必須更改每個文件的名稱。 它讀取一個pickle文件,將其寫入一個txt文件,然后對txt文件的上下文進行一些處理並生成一個數字列表,最后將列表存儲在dataframe中,並將dataframe寫入Z6287CB5675FFE88BAFEF.
def get_value_of_list(bit_list):
p_number = 0
for i in bit_list:
if i == 1:
p_number = p_number + 1
return p_number
def cross_entropy(p, q):
return -sum([p[i] * log2(q[i]) for i in range(len(p))])
if __name__ == "__main__":
file_name = 'pickleData_AIMchat2.txt'
pickle_file = 'AIMchat2.pickle'
pk = PickleToFile(file_name, pickle_file)
pk.create_pickle_file()
h = HexToBinary(file_name)
hex_list = h.read_file()
num_of_bits = 8
scale = 16
bin_data = []
for i in hex_list:
bin_data.append(bin(int(i, scale))[2:].zfill(num_of_bits))
my_bit_list = []
for byte in bin_data:
bit_list = []
for bit in byte:
bit_list.append(int(bit))
num_of_one_divided_by_eight = get_value_of_list(bit_list) / 8
my_bit_list.append(num_of_one_divided_by_eight)
cross_entropy_list = []
i = 0
while i < len(my_bit_list):
cross = cross_entropy([my_bit_list[i]], [my_bit_list[i + 1]])
cross_entropy_list.append(cross)
i = i + 2
df = pd.DataFrame(cross_entropy_list)
df.to_csv(r'AIMchat2.csv', index=False, index_label=False, chunksize=1000000, header=False)
我已將create_pickle_file()
更改為以下代碼以讀取目錄中的文件:
class PickleToFile:
def __init__(self, name, pickle_file):
self.name = name
self.pickle_file = pickle_file
def create_pickle_file(self):
basepath = Path()
files_in_basepath = basepath.iterdir('pickle/')
for item in files_in_basepath:
if item.is_file():
checkThePickle = open(self.pickle_file, "rb")
with open(self.name, 'w') as filehandler:
for listItem in checkThePickle:
filehandler.write('%s\n' % listItem)
但是由於在讀取文件后它將它寫入一個文本文件,然后是一個 csv 文件,我不知道該怎么做。 感謝任何建議。
如果您正在尋找目錄中的文件列表並處理它們,這應該可以滿足您的需求:
獲得此文件列表后,請執行循環:
for each in list_of_files:
process_function(each)
然后,您正在路上,其中“process_function”是 function,參數是文件名。
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.