[英]Python text extraction
我正在使用 python 進行文本提取。 輸出並不像我想要的那樣理想!
我有一個包含如下信息的文本文件:
FN Clarivate Analytics Web of Science
VR 1.0
PT J
AU Chen, G
Gully, SM
Whiteman, JA
Kilcullen, RN
AF Chen, G
Gully, SM
Whiteman, JA
Kilcullen, RN
TI Examination of relationships among trait-like individual differences,
state-like individual differences, and learning performance
SO JOURNAL OF APPLIED PSYCHOLOGY
CT 13th Annual Conference of the
Society-for-Industrial-and-Organizational-Psychology
CY APR 24-26, 1998
CL DALLAS, TEXAS
SP Soc Ind & Org Psychol
RI Gully, Stanley/D-1302-2012
OI Gully, Stanley/0000-0003-4037-3883
SN 0021-9010
PD DEC
PY 2000
VL 85
IS 6
BP 835
EP 847
DI 10.1037//0021-9010.85.6.835
UT WOS:000165745400001
PM 11125649
ER
當我像這樣使用我的代碼時
import random
import sys
filepath = "data\jap_2000-2001-plain.txt"
with open(filepath) as f:
articles = f.read().strip().split("\n")
articles_list = []
author = ""
title = ""
year = ""
doi = ""
for article in articles:
if "AU" in article:
author = article.split("#")[-1]
if "TI" in article:
title = article.split("#")[-1]
if "PY" in article:
year = article.split("#")[-1]
if "DI" in article:
doi = article.split("#")[-1]
if article == "ER#":
articles_list.append("{}, {}, {}, https://doi.org/{}".format(author, title, year, doi))
print("Oh hello sir, how many articles do you like to get?")
amount = input()
random_articles = random.sample(articles_list, k = int(amount))
for i in random_articles:
print(i)
print("\n")
exit = input('Please enter exit to exit: \n')
if exit in ['exit','Exit']:
print("Goodbye sir!")
sys.exit()
提取不包括換行后輸入的數據,如果我運行此代碼,輸出看起來像“AU Chen,G”並且不包括其他名稱,與標題等相同。
我的輸出看起來像:
Chen, G. 特質間關系的檢驗, 2000, doi.dx.10.1037//0021-9010.85.6.835
所需的輸出應該是:
Chen, G., Gully, SM., Whiteman, JA., Kilcullen, RN., 2000,特質樣個體差異、狀態樣個體差異和學習表現之間關系的檢驗,doi.dx.10.1037//0021 -9010.85.6.835
但提取只包括每一行的第一行——
有什么建議?
您需要在解析文件時跟蹤您所在的部分。 有更簡潔的方法來編寫狀態機,但作為一個快速而簡單的示例,您可以執行以下操作。
基本上,將每個部分的所有行添加到該部分的列表中,然后組合列表並在最后執行任何操作。 請注意,我沒有對此進行測試,只是通過偽編碼向您展示總體思路。
authors = []
title = []
section = None
for line in articles:
line = line.strip()
# Check for start of new section, select the right list to add to
if line.startswith("AU"):
line = line[3:]
section = authors
elif line.startswith("TI"):
line = line[3:]
section = title
# Other sections..
...
# Add line to the current section
if line and section is not None:
section.append(line)
authors_str = ', '.join(authors)
title_str = ' '.join(title)
print authors_str, title_str
初步了解
根據你的例子,我相信:
ER
分隔的人工部分標志着記錄結束。ER
部分不包含可用的文本。也可能是這樣:
FN
標簽開始。FN / ER
對之外遇到的任何文本都可以忽略。建議設計
如果這是真的,我建議您使用該邏輯編寫文本處理器:
ER
。ER
狀態中的文本,直到遇到FN
行。ER
狀態時,將累積記錄添加到累積記錄列表中。在此過程結束時,您將獲得一個記錄列表,其中包含各種累積的標簽。 然后,您可以以各種方式處理標簽。
像這樣的東西:
from warnings import warn
Debug = True
def read_lines_from(file):
"""Read and split lines from file. This is a separate function, instead
of just using file.readlines(), in case extra work is needed like
dos-to-unix conversion inside a unix environment.
"""
with open(file) as f:
text = f.read()
lines = text.split('\n')
return lines
def parse_file(file):
"""Parse file in format given by
https://stackoverflow.com/questions/54520331
"""
lines = read_lines_from(file)
state = 'ER'
records = []
current = None
for line_no, line in enumerate(lines):
tag, rest = line[:2], line[3:]
if Debug:
print(F"State: {state}, Tag: {tag}, Rest: {rest}")
# Skip empty lines
if tag == '':
if Debug:
print(F"Skip empty line at {line_no}")
continue
if tag == ' ':
# Append text, except in ER state.
if state != 'ER':
if Debug:
print(F"Append text to {state}: {rest}")
current[state].append(rest)
continue
# Found a tag. Process it.
if tag == 'ER':
if Debug:
print("Tag 'ER'. Completed record:")
print(current)
records.append(current)
current = None
state = tag
continue
if tag == 'FN':
if state != 'ER':
warn(F"Found 'FN' tag without previous 'ER' at line {line_no}")
if len(current.keys()):
warn(F"Previous record (FN:{current['FN']}) discarded.")
if Debug:
print("Tag 'FN'. Create empty record.")
current = {}
# All tags except ER get this:
if Debug:
print(F"Tag '{tag}'. Create list with rest: {rest}")
current[tag] = [rest]
state = tag
return records
if __name__ == '__main__':
records = parse_file('input.txt')
print('Records =', records)
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.