nltk中similar()和concordance之間的區別

Question

我已閱讀從text1.similar（“怪物”）和text1.concordance（“怪物”）此。

對於 python 中自然語言處理工具包的text1.concordance('monstrous')和text1.similar('monstrous')之間的區別，我無法得到滿意的答案。

那么請您舉例詳細說明一下好嗎？

Answer 1

使用concordance(token)為您提供圍繞參數token的上下文。 它將向您顯示token出現的句子。

使用similar(token)返回出現在相同的上下文中的單詞列表token 。 在這種情況下，上下文只是直接位於token兩側的單詞。

因此，查看 Moby Dick 文本 ( text1 )。 我們可以檢查'monstrous'的一致性

text1.concordance('monstrous')

# returns:
Displaying 11 of 11 matches:
ong the former , one was of a most monstrous size . ... This came towards us ,
ON OF THE PSALMS . " Touching that monstrous bulk of the whale or ork we have r
ll over with a heathenish array of monstrous clubs and spears . Some were thick
d as you gazed , and wondered what monstrous cannibal and savage could ever hav
that has survived the flood ; most monstrous and most mountainous ! That Himmal
they might scout at Moby Dick as a monstrous fable , or still worse and more de
th of Radney .'" CHAPTER 55 Of the Monstrous Pictures of Whales . I shall ere l
ing Scenes . In connexion with the monstrous pictures of whales , I am strongly
ere to enter upon those still more monstrous stories of them which are to be fo
ght have been rummaged out of this monstrous cabinet there is no telling . But
of Whale - Bones ; for Whales of a monstrous size are oftentimes cast up dead u

然后我們可以得到一個出現在與'monstrous'相似的上下文中的單詞列表。 返回的第一行的上下文是'most _____ size' 。

text1.similar('monstrous')

# returns:
determined maddens contemptible modifies abundant tyrannical puzzled
trustworthy impalpable gamesome curious mean pitiable untoward
christian subtly passing domineering uncommon true

如果我們使用單詞'true'並檢查它與text.concordance('true')的一致性，我們將返回單詞“ true ”的 87 次使用中的前 25 次。 這並不是非常有用，但是 NLTK 確實提供了一種稱為common_contexts的附加方法，該方法顯示單詞列表的使用何時共享相同的周圍單詞。

text1.common_contexts(['monstrous', 'true'])

# returns:
the_pictures

這個結果告訴我們，在"the monstrous pictures"鯨記"the monstrous pictures" ， "the true pictures" "the monstrous pictures"和"the true pictures"這兩個詞都出現了。

Answer 2

我將舉例說明：

text1.similar("monstrous")

將輸出具有相似上下文的單詞，例如word1 ______ word2 。 例如，它輸出單詞doleful 。 如果你運行：

text1.concordance("monstrous")

您將在匹配項中看到以下行：

在洪水中幸存下來的； 最可怕，最山區！ 那個喜馬拉雅

如果你運行：

text1.concordance("doleful")

您將在匹配項中看到以下行：

網站觀點。 有一場最悲哀、最嘲諷的葬禮！ 海

和

text1.common_contexts(["monstrous", "doleful"])

將輸出monstrous和doleful 的常用周邊詞，分別是“most”和“and”

大多數_和

Answer 3

Concordance(token)為您提供使用Concordance(token)的上下文。 Similar(token)為您提供出現在相似上下文中的其他詞。

為了說明，這里有一個更一般的描述來近似它們的功能。

1) Concordance(token) ：這會在您的標記的左側和右側返回預定義數量的單詞（我們將此單詞集合稱為“Z”）。 它為您的令牌出現在文本中的每個實例執行此操作。

2） similar(token) ：如果某個詞出現在集合“Z”的詞中的可能性很大，則會在此處列出該詞。

Answer 4

根據NLTK 文檔。 索引視圖向我們顯示給定單詞的每次出現以及一些上下文。 例如：

類似用於查找出現在類似上下文范圍內的其他單詞。 例如：

Answer 5

我將在簡·奧斯汀 (Jane Austen) 1811 年 (text2) 所著的《理智與情感》(Sense and Sensibility) 一書中用“怪異”一詞及其相似詞的詳盡示例來解釋：

common_contexts shared by "monstrous" and "very": ['a_pretty', 'am_glad', 'a_lucky', 'is_pretty', 'be_glad']
  context: a_pretty
    "a very pretty"
      ct . She was short and plump , had a very pretty face , and the finest express
      some delightful stew - ponds , and a very pretty canal ; and every thing , in 
       from Norland , Elinor had painted a very pretty pair of screens for her siste
    "a monstrous pretty"
      . " Now , Palmer , you shall see a monstrous pretty girl ." He immediately went

  context: am_glad
    "am very glad"
      ouble you with my affairs . But I am very glad to find it was only my own fanc
       be ordained in readiness ; and I am very glad to find things are so forward b
    "am monstrous glad"
      your sister is to marry him . I am monstrous glad of it , for then I shall have
       usual noisy cheerfulness , " I am monstrous glad to see you -- sorry I could n
      t however , as it turns out , I am monstrous glad there was never any thing in 
      possible that she should ." " I am monstrous glad of it . Good gracious ! I hav

  context: a_lucky
    "a very lucky"
      Colonel Brandon may be , Edward is a very lucky man .-- You will not mention t
    "a monstrous lucky"
      ou may tell your sister . She is a monstrous lucky girl to get him , upon my ho

  context: is_pretty
    "is very pretty"
      t is only the Palmers . Charlotte is very pretty , I can tell you . You may se
       " All this ," thought Elinor , " is very pretty ; but it can impose upon neit
      eau of mine ." " Aye , aye , that is very pretty talking -- but it won ' t do 
    "is monstrous pretty"
      k how you will like them . Lucy is monstrous pretty , and so good humoured and 

  context: be_glad
    "be very glad"
       trusting YOU ; indeed , I should be very glad of your advice how to manage in
      o ever he might have , she should be very glad to have it all , you know , or 
      ant company , I am sure we should be very glad to come and stay with her for a
      y ' s sister , my dear . I should be very glad to get her so good a mistress .
      patience !-- And though one would be very glad to do a kindness by poor Mr . F
      d , assuring her that Fanny would be very glad to see her , invited her to com
    "be monstrous glad"
       Jennings , " I am sure I shall be monstrous glad of Miss Marianne ' s company 

common_contexts shared by "monstrous" and "so": ['am_glad', 'is_pretty']
  context: am_glad
    "am so glad"
      delight in seeing them again . " I am so glad to see you !" said she , seating
      eems very agreeable ." " Well -- I am so glad you do . I thought you would , h
       small , was pleasing to her . " I am so glad we are got acquainted at last ,"
      an be like it , to be sure ! And I am so glad your sister is going to be well 
      d otherwise have been learnt . " I am so glad to meet you ;" said Miss Steele 
    "am monstrous glad"
      your sister is to marry him . I am monstrous glad of it , for then I shall have
       usual noisy cheerfulness , " I am monstrous glad to see you -- sorry I could n
      t however , as it turns out , I am monstrous glad there was never any thing in 
      possible that she should ." " I am monstrous glad of it . Good gracious ! I hav

  context: is_pretty
    "is so pretty"
      n it must be some other place that is so pretty I suppose ." When they were se
    "is monstrous pretty"
      k how you will like them . Lucy is monstrous pretty , and so good humoured and 

common_contexts shared by "monstrous" and "exceedingly": ['be_glad', 'is_fond']
  context: be_glad
    "be exceedingly glad"
      but , my dear Elinor , I shall be exceedingly glad to know more of it ; and I t
       with her ; otherwise I should be exceedingly glad to do it . You know I am alw
    "be monstrous glad"
       Jennings , " I am sure I shall be monstrous glad of Miss Marianne ' s company 

  context: is_fond
    "is exceedingly fond"
      han you did ; not but what she is exceedingly fond of YOU , but so it happened 
    "is monstrous fond"
      so scornfully ! for they say he is monstrous fond of her , as well he may . I s

common_contexts shared by "monstrous" and "heartily": ['am_glad', 'be_glad']
  context: am_glad
    "am heartily glad"
      ney enough to attempt it ." " I am heartily glad of it ," he cried . " May she
    "am monstrous glad"
      your sister is to marry him . I am monstrous glad of it , for then I shall have
       usual noisy cheerfulness , " I am monstrous glad to see you -- sorry I could n
      t however , as it turns out , I am monstrous glad there was never any thing in 
      possible that she should ." " I am monstrous glad of it . Good gracious ! I hav

  context: be_glad
    "be heartily glad"
      saying now and then , ' I shall be heartily glad to hear she is well married .
    "be monstrous glad"
       Jennings , " I am sure I shall be monstrous glad of Miss Marianne ' s company 

common_contexts shared by "monstrous" and "a": ['was_happy']
  context: was_happy
    "was a happy"
      ne ' s behaviour as they travelled was a happy specimen of what future complais
    "was monstrous happy"
      thing of the kind . So then he was monstrous happy , and talked on some time ab

common_contexts shared by "monstrous" and "as": ['is_pretty']
  context: is_pretty
    "is as pretty"
      , my dear , for shewing it me . It is as pretty a letter as ever I saw , and d
      wing it me . It is as pretty a letter as ever I saw , and does Lucy ' s head a
    "is monstrous pretty"
      k how you will like them . Lucy is monstrous pretty , and so good humoured and 

common_contexts shared by "monstrous" and "good": ['a_deal']
  context: a_deal
    "a good deal"
      l ; but as she had already imbibed a good deal of Marianne ' s romance , witho
      He and I have been at times thrown a good deal together , while you have been 
      th Mrs . Dashwood , so we shall be a good deal in Harley Street , I dare say ,
      alone ," he replied , " for I have a good deal to say to you . This living of 
    "a monstrous deal"
      e very genteel people . He makes a monstrous deal of money , and they keep thei

common_contexts shared by "monstrous" and "great": ['a_deal']
  context: a_deal
    "a great deal"
      wn way , many cunning tricks , and a great deal of noise , as to outweigh all t
       what you are doing . I have known a great deal of the trouble of annuities ; f
      me as what belongs to this house . A great deal too handsome , in my opinion , 
      inciple by my mother . I have seen a great deal of him , have studied his senti
      , fat , elderly woman , who talked a great deal , seemed very happy , and rathe
      een thirty and forty . He has seen a great deal of the world ; has been abroad 
      in all she does -- sometimes talks a great deal and always with animation -- bu
      ey recommended the army . That was a great deal too smart for me . The law was 
      r , I dare say we should have seen a great deal of him in Somersetshire , if it
       well married ! I hope you will be a great deal at Combe Magna . It is a sweet 
      old them it is all very true , and a great deal more . You will be delighted wi
      ment at all ; indeed , she does me a great deal more harm than good , for I am 
      ings received the information with a great deal of joy , and many assurances of
      ut it did not signify , for it was a great deal too far off to visit ; she hate
      e in time . A thousand a - year is a great deal for a mother to give away , to 
      ied ; " but however there is still a great deal to be done . There is not a sto
      not be forgotten .-- She must have a great deal to leave ." " Nothing at all , 
      ious that everybody else should do a great deal ; and an offer from Colonel Bra
      hinks just the same . He and I had a great deal of talk about it ; and the best
       ; so she told him directly ( with a great deal about sweet and love , you know
      nd of whom therefore she must have a great deal to say , her mind was so much m
      nfidant of himself , talked to her a great deal of the parsonage at Delaford , 
       in the course of a twelve - month a great deal of instruction which I now feel
      you , for as Colonel Brandon seems a great deal at home , nobody can tell what 
    "a monstrous deal"
      e very genteel people . He makes a monstrous deal of money , and they keep thei

common_contexts shared by "monstrous" and "extremely": ['am_glad']
  context: am_glad
    "am extremely glad"
       more than I can express ." " I am extremely glad to hear it , upon my word ; e
    "am monstrous glad"
      your sister is to marry him . I am monstrous glad of it , for then I shall have
       usual noisy cheerfulness , " I am monstrous glad to see you -- sorry I could n
      t however , as it turns out , I am monstrous glad there was never any thing in 
      possible that she should ." " I am monstrous glad of it . Good gracious ! I hav

common_contexts shared by "monstrous" and "remarkably": ['a_pretty']
  context: a_pretty
    "a remarkably pretty"
      lexion , regular features , and a remarkably pretty figure . Marianne was stil
    "a monstrous pretty"
      . " Now , Palmer , you shall see a monstrous pretty girl ." He immediately went

common_contexts shared by "monstrous" and "sweet": ['a_pretty']
  context: a_pretty
    "a sweet pretty"
       at his house ; but they say it is a sweet pretty place ." " As vile a spot as 
    "a monstrous pretty"
      . " Now , Palmer , you shall see a monstrous pretty girl ." He immediately went

common_contexts shared by "monstrous" and "vast": ['a_deal']
  context: a_deal
    "a vast deal"
      s convenience ; and it HAS cost me a vast deal of money ." " More than you thi
      othing but only civility ?-- I saw a vast deal more . Such kindness as fell to
       here come the Richardsons . I had a vast deal more to say to you , but I must
      ell , I am convinced that there is a vast deal of inconsistency in almost ever
    "a monstrous deal"
      e very genteel people . He makes a monstrous deal of money , and they keep thei

common_contexts shared by "monstrous" and "amazingly": ['am_glad']
  context: am_glad
    "am amazingly glad"
      l be in no hurry to be gone . I am amazingly glad you did not keep to YOUR WORD
    "am monstrous glad"
      your sister is to marry him . I am monstrous glad of it , for then I shall have
       usual noisy cheerfulness , " I am monstrous glad to see you -- sorry I could n
      t however , as it turns out , I am monstrous glad there was never any thing in 
      possible that she should ." " I am monstrous glad of it . Good gracious ! I hav

該輸出是由以下代碼生成的。

from nltk.book import *
from contextlib import redirect_stdout
import io

def get_similar(text, word):
    f = io.StringIO()
    with redirect_stdout(f):
        text.similar(word)
    return f.getvalue().split()

def get_concordance(text, word):
    f = io.StringIO()
    with redirect_stdout(f):
        text.concordance(word, lines=1000)
    return f.getvalue().split(sep='\n')[1:-1]

def get_common_contexts(text, word_list):
    f = io.StringIO()
    with redirect_stdout(f):
        text.common_contexts(word_list)
    return f.getvalue().split()

def print_full_similars(text, word):
    word_concordances = get_concordance(text, word)
    similars = get_similar(text, word)
    for similar in similars:
        similar_concordances = get_concordance(text, similar)
        common_contexts = get_common_contexts(text, [word, similar])
        print('common_contexts shared by "' + word + '" and "' + similar + '":', common_contexts)
        for context in common_contexts:
            print('\tcontext:', context)
            before, after = context.split('_')
            print('\t\t"' + before, similar, after + '"')
            for concordance in similar_concordances: 
                if (before + ' ' + similar + ' ' + after).lower() in concordance.lower():
                    print('\t\t\t'+ concordance)
            print('\t\t"' + before, word, after + '"')
            for concordance in word_concordances: 
                if (before + ' ' + word + ' ' + after).lower() in concordance.lower():
                    print('\t\t\t'+ concordance)
            print()

print_full_similars(text2, 'monstrous')

只需更改最后一行即可獲得其他示例，如 Herman Melville 1851 年撰寫的 Moby Dick：

print_full_similars(text1, 'monstrous')