[英]difference between similar() and concordance in nltk
我已阅读从text1.similar(“怪物”)和text1.concordance(“怪物”)此。
对于 python 中自然语言处理工具包的text1.concordance('monstrous')
和text1.similar('monstrous')
之间的区别,我无法得到满意的答案。
那么请您举例详细说明一下好吗?
使用concordance(token)
为您提供围绕参数token
的上下文。 它将向您显示token
出现的句子。
使用similar(token)
返回出现在相同的上下文中的单词列表token
。 在这种情况下,上下文只是直接位于token
两侧的单词。
因此,查看 Moby Dick 文本 ( text1
)。 我们可以检查'monstrous'
的一致性
text1.concordance('monstrous')
# returns:
Displaying 11 of 11 matches:
ong the former , one was of a most monstrous size . ... This came towards us ,
ON OF THE PSALMS . " Touching that monstrous bulk of the whale or ork we have r
ll over with a heathenish array of monstrous clubs and spears . Some were thick
d as you gazed , and wondered what monstrous cannibal and savage could ever hav
that has survived the flood ; most monstrous and most mountainous ! That Himmal
they might scout at Moby Dick as a monstrous fable , or still worse and more de
th of Radney .'" CHAPTER 55 Of the Monstrous Pictures of Whales . I shall ere l
ing Scenes . In connexion with the monstrous pictures of whales , I am strongly
ere to enter upon those still more monstrous stories of them which are to be fo
ght have been rummaged out of this monstrous cabinet there is no telling . But
of Whale - Bones ; for Whales of a monstrous size are oftentimes cast up dead u
然后我们可以得到一个出现在与'monstrous'
相似的上下文中的单词列表。 返回的第一行的上下文是'most _____ size'
。
text1.similar('monstrous')
# returns:
determined maddens contemptible modifies abundant tyrannical puzzled
trustworthy impalpable gamesome curious mean pitiable untoward
christian subtly passing domineering uncommon true
如果我们使用单词'true'
并检查它与text.concordance('true')
的一致性,我们将返回单词“ true
”的 87 次使用中的前 25 次。 这并不是非常有用,但是 NLTK 确实提供了一种称为common_contexts
的附加方法,该方法显示单词列表的使用何时共享相同的周围单词。
text1.common_contexts(['monstrous', 'true'])
# returns:
the_pictures
这个结果告诉我们,在"the monstrous pictures"
鲸记"the monstrous pictures"
, "the true pictures"
"the monstrous pictures"
和"the true pictures"
这两个词都出现了。
我将举例说明:
text1.similar("monstrous")
将输出具有相似上下文的单词,例如word1 ______ word2
。 例如,它输出单词doleful 。 如果你运行:
text1.concordance("monstrous")
您将在匹配项中看到以下行:
在洪水中幸存下来的; 最可怕,最山区! 那个喜马拉雅
如果你运行:
text1.concordance("doleful")
您将在匹配项中看到以下行:
网站观点。 有一场最悲哀、最嘲讽的葬礼! 海
和
text1.common_contexts(["monstrous", "doleful"])
将输出monstrous和doleful 的常用周边词,分别是“most”和“and”
大多数_和
Concordance(token)
为您提供使用Concordance(token)
的上下文。 Similar(token)
为您提供出现在相似上下文中的其他词。
为了说明,这里有一个更一般的描述来近似它们的功能。
1) Concordance(token)
:这会在您的标记的左侧和右侧返回预定义数量的单词(我们将此单词集合称为“Z”)。 它为您的令牌出现在文本中的每个实例执行此操作。
2) similar(token)
:如果某个词出现在集合“Z”的词中的可能性很大,则会在此处列出该词。
我将在简·奥斯汀 (Jane Austen) 1811 年 (text2) 所著的《理智与情感》(Sense and Sensibility) 一书中用“怪异”一词及其相似词的详尽示例来解释:
common_contexts shared by "monstrous" and "very": ['a_pretty', 'am_glad', 'a_lucky', 'is_pretty', 'be_glad']
context: a_pretty
"a very pretty"
ct . She was short and plump , had a very pretty face , and the finest express
some delightful stew - ponds , and a very pretty canal ; and every thing , in
from Norland , Elinor had painted a very pretty pair of screens for her siste
"a monstrous pretty"
. " Now , Palmer , you shall see a monstrous pretty girl ." He immediately went
context: am_glad
"am very glad"
ouble you with my affairs . But I am very glad to find it was only my own fanc
be ordained in readiness ; and I am very glad to find things are so forward b
"am monstrous glad"
your sister is to marry him . I am monstrous glad of it , for then I shall have
usual noisy cheerfulness , " I am monstrous glad to see you -- sorry I could n
t however , as it turns out , I am monstrous glad there was never any thing in
possible that she should ." " I am monstrous glad of it . Good gracious ! I hav
context: a_lucky
"a very lucky"
Colonel Brandon may be , Edward is a very lucky man .-- You will not mention t
"a monstrous lucky"
ou may tell your sister . She is a monstrous lucky girl to get him , upon my ho
context: is_pretty
"is very pretty"
t is only the Palmers . Charlotte is very pretty , I can tell you . You may se
" All this ," thought Elinor , " is very pretty ; but it can impose upon neit
eau of mine ." " Aye , aye , that is very pretty talking -- but it won ' t do
"is monstrous pretty"
k how you will like them . Lucy is monstrous pretty , and so good humoured and
context: be_glad
"be very glad"
trusting YOU ; indeed , I should be very glad of your advice how to manage in
o ever he might have , she should be very glad to have it all , you know , or
ant company , I am sure we should be very glad to come and stay with her for a
y ' s sister , my dear . I should be very glad to get her so good a mistress .
patience !-- And though one would be very glad to do a kindness by poor Mr . F
d , assuring her that Fanny would be very glad to see her , invited her to com
"be monstrous glad"
Jennings , " I am sure I shall be monstrous glad of Miss Marianne ' s company
common_contexts shared by "monstrous" and "so": ['am_glad', 'is_pretty']
context: am_glad
"am so glad"
delight in seeing them again . " I am so glad to see you !" said she , seating
eems very agreeable ." " Well -- I am so glad you do . I thought you would , h
small , was pleasing to her . " I am so glad we are got acquainted at last ,"
an be like it , to be sure ! And I am so glad your sister is going to be well
d otherwise have been learnt . " I am so glad to meet you ;" said Miss Steele
"am monstrous glad"
your sister is to marry him . I am monstrous glad of it , for then I shall have
usual noisy cheerfulness , " I am monstrous glad to see you -- sorry I could n
t however , as it turns out , I am monstrous glad there was never any thing in
possible that she should ." " I am monstrous glad of it . Good gracious ! I hav
context: is_pretty
"is so pretty"
n it must be some other place that is so pretty I suppose ." When they were se
"is monstrous pretty"
k how you will like them . Lucy is monstrous pretty , and so good humoured and
common_contexts shared by "monstrous" and "exceedingly": ['be_glad', 'is_fond']
context: be_glad
"be exceedingly glad"
but , my dear Elinor , I shall be exceedingly glad to know more of it ; and I t
with her ; otherwise I should be exceedingly glad to do it . You know I am alw
"be monstrous glad"
Jennings , " I am sure I shall be monstrous glad of Miss Marianne ' s company
context: is_fond
"is exceedingly fond"
han you did ; not but what she is exceedingly fond of YOU , but so it happened
"is monstrous fond"
so scornfully ! for they say he is monstrous fond of her , as well he may . I s
common_contexts shared by "monstrous" and "heartily": ['am_glad', 'be_glad']
context: am_glad
"am heartily glad"
ney enough to attempt it ." " I am heartily glad of it ," he cried . " May she
"am monstrous glad"
your sister is to marry him . I am monstrous glad of it , for then I shall have
usual noisy cheerfulness , " I am monstrous glad to see you -- sorry I could n
t however , as it turns out , I am monstrous glad there was never any thing in
possible that she should ." " I am monstrous glad of it . Good gracious ! I hav
context: be_glad
"be heartily glad"
saying now and then , ' I shall be heartily glad to hear she is well married .
"be monstrous glad"
Jennings , " I am sure I shall be monstrous glad of Miss Marianne ' s company
common_contexts shared by "monstrous" and "a": ['was_happy']
context: was_happy
"was a happy"
ne ' s behaviour as they travelled was a happy specimen of what future complais
"was monstrous happy"
thing of the kind . So then he was monstrous happy , and talked on some time ab
common_contexts shared by "monstrous" and "as": ['is_pretty']
context: is_pretty
"is as pretty"
, my dear , for shewing it me . It is as pretty a letter as ever I saw , and d
wing it me . It is as pretty a letter as ever I saw , and does Lucy ' s head a
"is monstrous pretty"
k how you will like them . Lucy is monstrous pretty , and so good humoured and
common_contexts shared by "monstrous" and "good": ['a_deal']
context: a_deal
"a good deal"
l ; but as she had already imbibed a good deal of Marianne ' s romance , witho
He and I have been at times thrown a good deal together , while you have been
th Mrs . Dashwood , so we shall be a good deal in Harley Street , I dare say ,
alone ," he replied , " for I have a good deal to say to you . This living of
"a monstrous deal"
e very genteel people . He makes a monstrous deal of money , and they keep thei
common_contexts shared by "monstrous" and "great": ['a_deal']
context: a_deal
"a great deal"
wn way , many cunning tricks , and a great deal of noise , as to outweigh all t
what you are doing . I have known a great deal of the trouble of annuities ; f
me as what belongs to this house . A great deal too handsome , in my opinion ,
inciple by my mother . I have seen a great deal of him , have studied his senti
, fat , elderly woman , who talked a great deal , seemed very happy , and rathe
een thirty and forty . He has seen a great deal of the world ; has been abroad
in all she does -- sometimes talks a great deal and always with animation -- bu
ey recommended the army . That was a great deal too smart for me . The law was
r , I dare say we should have seen a great deal of him in Somersetshire , if it
well married ! I hope you will be a great deal at Combe Magna . It is a sweet
old them it is all very true , and a great deal more . You will be delighted wi
ment at all ; indeed , she does me a great deal more harm than good , for I am
ings received the information with a great deal of joy , and many assurances of
ut it did not signify , for it was a great deal too far off to visit ; she hate
e in time . A thousand a - year is a great deal for a mother to give away , to
ied ; " but however there is still a great deal to be done . There is not a sto
not be forgotten .-- She must have a great deal to leave ." " Nothing at all ,
ious that everybody else should do a great deal ; and an offer from Colonel Bra
hinks just the same . He and I had a great deal of talk about it ; and the best
; so she told him directly ( with a great deal about sweet and love , you know
nd of whom therefore she must have a great deal to say , her mind was so much m
nfidant of himself , talked to her a great deal of the parsonage at Delaford ,
in the course of a twelve - month a great deal of instruction which I now feel
you , for as Colonel Brandon seems a great deal at home , nobody can tell what
"a monstrous deal"
e very genteel people . He makes a monstrous deal of money , and they keep thei
common_contexts shared by "monstrous" and "extremely": ['am_glad']
context: am_glad
"am extremely glad"
more than I can express ." " I am extremely glad to hear it , upon my word ; e
"am monstrous glad"
your sister is to marry him . I am monstrous glad of it , for then I shall have
usual noisy cheerfulness , " I am monstrous glad to see you -- sorry I could n
t however , as it turns out , I am monstrous glad there was never any thing in
possible that she should ." " I am monstrous glad of it . Good gracious ! I hav
common_contexts shared by "monstrous" and "remarkably": ['a_pretty']
context: a_pretty
"a remarkably pretty"
lexion , regular features , and a remarkably pretty figure . Marianne was stil
"a monstrous pretty"
. " Now , Palmer , you shall see a monstrous pretty girl ." He immediately went
common_contexts shared by "monstrous" and "sweet": ['a_pretty']
context: a_pretty
"a sweet pretty"
at his house ; but they say it is a sweet pretty place ." " As vile a spot as
"a monstrous pretty"
. " Now , Palmer , you shall see a monstrous pretty girl ." He immediately went
common_contexts shared by "monstrous" and "vast": ['a_deal']
context: a_deal
"a vast deal"
s convenience ; and it HAS cost me a vast deal of money ." " More than you thi
othing but only civility ?-- I saw a vast deal more . Such kindness as fell to
here come the Richardsons . I had a vast deal more to say to you , but I must
ell , I am convinced that there is a vast deal of inconsistency in almost ever
"a monstrous deal"
e very genteel people . He makes a monstrous deal of money , and they keep thei
common_contexts shared by "monstrous" and "amazingly": ['am_glad']
context: am_glad
"am amazingly glad"
l be in no hurry to be gone . I am amazingly glad you did not keep to YOUR WORD
"am monstrous glad"
your sister is to marry him . I am monstrous glad of it , for then I shall have
usual noisy cheerfulness , " I am monstrous glad to see you -- sorry I could n
t however , as it turns out , I am monstrous glad there was never any thing in
possible that she should ." " I am monstrous glad of it . Good gracious ! I hav
该输出是由以下代码生成的。
from nltk.book import *
from contextlib import redirect_stdout
import io
def get_similar(text, word):
f = io.StringIO()
with redirect_stdout(f):
text.similar(word)
return f.getvalue().split()
def get_concordance(text, word):
f = io.StringIO()
with redirect_stdout(f):
text.concordance(word, lines=1000)
return f.getvalue().split(sep='\n')[1:-1]
def get_common_contexts(text, word_list):
f = io.StringIO()
with redirect_stdout(f):
text.common_contexts(word_list)
return f.getvalue().split()
def print_full_similars(text, word):
word_concordances = get_concordance(text, word)
similars = get_similar(text, word)
for similar in similars:
similar_concordances = get_concordance(text, similar)
common_contexts = get_common_contexts(text, [word, similar])
print('common_contexts shared by "' + word + '" and "' + similar + '":', common_contexts)
for context in common_contexts:
print('\tcontext:', context)
before, after = context.split('_')
print('\t\t"' + before, similar, after + '"')
for concordance in similar_concordances:
if (before + ' ' + similar + ' ' + after).lower() in concordance.lower():
print('\t\t\t'+ concordance)
print('\t\t"' + before, word, after + '"')
for concordance in word_concordances:
if (before + ' ' + word + ' ' + after).lower() in concordance.lower():
print('\t\t\t'+ concordance)
print()
print_full_similars(text2, 'monstrous')
只需更改最后一行即可获得其他示例,如 Herman Melville 1851 年撰写的 Moby Dick:
print_full_similars(text1, 'monstrous')
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.