[英]How to create paragraphs from markov chain output?
我想修改下面的腳本,以便它從腳本生成的隨機數量的句子中創建段落。 換句話說,在添加換行符之前連接一個隨機數(如1-5)的句子。
腳本工作正常,但輸出是由換行符分隔的短句。 我想把一些句子收集成段落。
關於最佳實踐的任何想法? 謝謝。
"""
from: http://code.activestate.com/recipes/194364-the-markov-chain-algorithm/?in=lang-python
"""
import random;
import sys;
stopword = "\n" # Since we split on whitespace, this can never be a word
stopsentence = (".", "!", "?",) # Cause a "new sentence" if found at the end of a word
sentencesep = "\n" #String used to seperate sentences
# GENERATE TABLE
w1 = stopword
w2 = stopword
table = {}
for line in sys.stdin:
for word in line.split():
if word[-1] in stopsentence:
table.setdefault( (w1, w2), [] ).append(word[0:-1])
w1, w2 = w2, word[0:-1]
word = word[-1]
table.setdefault( (w1, w2), [] ).append(word)
w1, w2 = w2, word
# Mark the end of the file
table.setdefault( (w1, w2), [] ).append(stopword)
# GENERATE SENTENCE OUTPUT
maxsentences = 20
w1 = stopword
w2 = stopword
sentencecount = 0
sentence = []
while sentencecount < maxsentences:
newword = random.choice(table[(w1, w2)])
if newword == stopword: sys.exit()
if newword in stopsentence:
print ("%s%s%s" % (" ".join(sentence), newword, sentencesep))
sentence = []
sentencecount += 1
else:
sentence.append(newword)
w1, w2 = w2, newword
編輯01:
好吧,我拼湊了一個簡單的“段落包裝器”,它可以很好地將句子收集到段落中,但它與句子生成器的輸出相混淆 - 我對第一個單詞的重復性過高,例如,其他的問題。
但前提是聲音; 我只需要弄清楚為什么句子循環的功能受到段落循環的添加的影響。 如果您能看到問題,請告知:
###
# usage: $ python markov_sentences.py < input.txt > output.txt
# from: http://code.activestate.com/recipes/194364-the-markov-chain-algorithm/?in=lang-python
###
import random;
import sys;
stopword = "\n" # Since we split on whitespace, this can never be a word
stopsentence = (".", "!", "?",) # Cause a "new sentence" if found at the end of a word
paragraphsep = "\n\n" #String used to seperate sentences
# GENERATE TABLE
w1 = stopword
w2 = stopword
table = {}
for line in sys.stdin:
for word in line.split():
if word[-1] in stopsentence:
table.setdefault( (w1, w2), [] ).append(word[0:-1])
w1, w2 = w2, word[0:-1]
word = word[-1]
table.setdefault( (w1, w2), [] ).append(word)
w1, w2 = w2, word
# Mark the end of the file
table.setdefault( (w1, w2), [] ).append(stopword)
# GENERATE PARAGRAPH OUTPUT
maxparagraphs = 10
paragraphs = 0 # reset the outer 'while' loop counter to zero
while paragraphs < maxparagraphs: # start outer loop, until maxparagraphs is reached
w1 = stopword
w2 = stopword
stopsentence = (".", "!", "?",)
sentence = []
sentencecount = 0 # reset the inner 'while' loop counter to zero
maxsentences = random.randrange(1,5) # random sentences per paragraph
while sentencecount < maxsentences: # start inner loop, until maxsentences is reached
newword = random.choice(table[(w1, w2)]) # random word from word table
if newword == stopword: sys.exit()
elif newword in stopsentence:
print ("%s%s" % (" ".join(sentence), newword), end=" ")
sentencecount += 1 # increment the sentence counter
else:
sentence.append(newword)
w1, w2 = w2, newword
print (paragraphsep) # newline space
paragraphs = paragraphs + 1 # increment the paragraph counter
# EOF
編輯02:
將以下sentence = []
添加到elif
語句中。 以機智;
elif newword in stopsentence:
print ("%s%s" % (" ".join(sentence), newword), end=" ")
sentence = [] # I have to be here to make the new sentence start as an empty list!!!
sentencecount += 1 # increment the sentence counter
編輯03:
這是此腳本的最后一次迭代。 感謝悲傷幫助整理出來。 我希望其他人可以玩得開心,我知道我會的。 ;)
僅供參考:有一個小工件 - 如果你使用這個腳本,你可能想要清理一個額外的段末空間。 但是,除此之外,馬爾可夫鏈文本生成的完美實現。
###
# usage: python markov_sentences.py < input.txt > output.txt
# from: http://code.activestate.com/recipes/194364-the-markov-chain-algorithm/?in=lang-python
###
import random;
import sys;
stopword = "\n" # Since we split on whitespace, this can never be a word
stopsentence = (".", "!", "?",) # Cause a "new sentence" if found at the end of a word
sentencesep = "\n" #String used to seperate sentences
# GENERATE TABLE
w1 = stopword
w2 = stopword
table = {}
for line in sys.stdin:
for word in line.split():
if word[-1] in stopsentence:
table.setdefault( (w1, w2), [] ).append(word[0:-1])
w1, w2 = w2, word[0:-1]
word = word[-1]
table.setdefault( (w1, w2), [] ).append(word)
w1, w2 = w2, word
# Mark the end of the file
table.setdefault( (w1, w2), [] ).append(stopword)
# GENERATE SENTENCE OUTPUT
maxsentences = 20
w1 = stopword
w2 = stopword
sentencecount = 0
sentence = []
paragraphsep = "\n"
count = random.randrange(1,5)
while sentencecount < maxsentences:
newword = random.choice(table[(w1, w2)]) # random word from word table
if newword == stopword: sys.exit()
if newword in stopsentence:
print ("%s%s" % (" ".join(sentence), newword), end=" ")
sentence = []
sentencecount += 1 # increment the sentence counter
count -= 1
if count == 0:
count = random.randrange(1,5)
print (paragraphsep) # newline space
else:
sentence.append(newword)
w1, w2 = w2, newword
# EOF
你需要復制
sentence = []
回到了
elif newword in stopsentence:
條款。
所以
while paragraphs < maxparagraphs: # start outer loop, until maxparagraphs is reached
w1 = stopword
w2 = stopword
stopsentence = (".", "!", "?",)
sentence = []
sentencecount = 0 # reset the inner 'while' loop counter to zero
maxsentences = random.randrange(1,5) # random sentences per paragraph
while sentencecount < maxsentences: # start inner loop, until maxsentences is reached
newword = random.choice(table[(w1, w2)]) # random word from word table
if newword == stopword: sys.exit()
elif newword in stopsentence:
print ("%s%s" % (" ".join(sentence), newword), end=" ")
sentence = [] # I have to be here to make the new sentence start as an empty list!!!
sentencecount += 1 # increment the sentence counter
else:
sentence.append(newword)
w1, w2 = w2, newword
print (paragraphsep) # newline space
paragraphs = paragraphs + 1 # increment the paragraph counter
編輯
這是一個不使用外循環的解決方案。
"""
from: http://code.activestate.com/recipes/194364-the-markov-chain-algorithm/?in=lang-python
"""
import random;
import sys;
stopword = "\n" # Since we split on whitespace, this can never be a word
stopsentence = (".", "!", "?",) # Cause a "new sentence" if found at the end of a word
sentencesep = "\n" #String used to seperate sentences
# GENERATE TABLE
w1 = stopword
w2 = stopword
table = {}
for line in sys.stdin:
for word in line.split():
if word[-1] in stopsentence:
table.setdefault( (w1, w2), [] ).append(word[0:-1])
w1, w2 = w2, word[0:-1]
word = word[-1]
table.setdefault( (w1, w2), [] ).append(word)
w1, w2 = w2, word
# Mark the end of the file
table.setdefault( (w1, w2), [] ).append(stopword)
# GENERATE SENTENCE OUTPUT
maxsentences = 20
w1 = stopword
w2 = stopword
sentencecount = 0
sentence = []
paragraphsep == "\n\n"
count = random.randrange(1,5)
while sentencecount < maxsentences:
newword = random.choice(table[(w1, w2)])
if newword == stopword: sys.exit()
if newword in stopsentence:
print ("%s%s" % (" ".join(sentence), newword), end=" ")
sentence = []
sentencecount += 1
count -= 1
if count == 0:
count = random.randrange(1,5)
print (paragraphsep)
else:
sentence.append(newword)
w1, w2 = w2, newword
你了解這段代碼嗎? 我打賭你可以找到打印句子的位,並將其改為一起打印幾個句子而不返回。 您可以在句子位周圍添加另一個while循環以獲得多個段落。
語法提示:
print 'hello'
print 'there'
hello
there
print 'hello',
print 'there'
hello there
print 'hello',
print
print 'there'
關鍵是print語句末尾的逗號會阻止行末尾的返回,而空白的print語句會打印一個返回值。
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.