語料庫中每個文本的平均句子長度（python3和nltk）

Question

我正在分析NLTK軟件包中的首個地址語料庫，作為python編程課程簡介的一部分。 我想找出語料庫中每個文本的平均句子長度（以便以后進行比較），但是我似乎被困在這里。

我創建了這個函數：

def averageSentence(text):
    sents = inaugural.sents(fileids=['fileid_here.txt']  
    avg = sum(len(word) for word in sents) / len(sents)  
    print(avg)

哪一個（如果我是對的）應該給我單個文本的平均句子長度。 現在，我知道我需要一個for循環 。 我不應該能夠通過我剛剛定義的函數相對簡單地實現循環嗎？ 這非常令人沮喪。

編輯：這是我走了多遠：

for fileid in inaugural.fileids():
    avg_sents = averageSentence(fileid)
    print = sum(avg_sents) / avg_sents

Answer 1

嘗試：

>>> from __future__ import division
>>> from nltk.corpus import inaugural
>>> total_lens = 0
>>> for i, sent in enumerate(inaugural.sents()):
...     total_lens += len(sent)
... 
>>> total_lens 
145735
>>> i
4867
>>> avg_sent_len = total_lens / i
>>> avg_sent_len
29.943497020752
>>> avg_sent_len = total_lens / (i+1)
>>> avg_sent_len
29.9373459326212

注意，當分母足夠大時+1並沒有多大關系。

---

所有文本的Mirco平均句子長度

以下代碼是單行代碼，但不建議使用，因為您可能兩次實現了一個生成器：

>>> sum(len(sent) for sent in inaugural.sents()) / len(inaugural.sents())
29.9373459326212

---

Marco在所有文本中的平均句子長度 ：

>>> sum(sum(len(sent) for sent in inaugural.sents(fileids=[fileid])) / len(inaugural.sents(fileids=[fileid])) for fileid in inaugural.fileids()) / len(inaugural.fileids())
32.84054349411484

---

每個文本的平均句子長度 ：

>>> from __future__ import division
>>> from nltk.corpus import inaugural
>>> inaugural.fileids()
[u'1789-Washington.txt', u'1793-Washington.txt', u'1797-Adams.txt', u'1801-Jefferson.txt', u'1805-Jefferson.txt', u'1809-Madison.txt', u'1813-Madison.txt', u'1817-Monroe.txt', u'1821-Monroe.txt', u'1825-Adams.txt', u'1829-Jackson.txt', u'1833-Jackson.txt', u'1837-VanBuren.txt', u'1841-Harrison.txt', u'1845-Polk.txt', u'1849-Taylor.txt', u'1853-Pierce.txt', u'1857-Buchanan.txt', u'1861-Lincoln.txt', u'1865-Lincoln.txt', u'1869-Grant.txt', u'1873-Grant.txt', u'1877-Hayes.txt', u'1881-Garfield.txt', u'1885-Cleveland.txt', u'1889-Harrison.txt', u'1893-Cleveland.txt', u'1897-McKinley.txt', u'1901-McKinley.txt', u'1905-Roosevelt.txt', u'1909-Taft.txt', u'1913-Wilson.txt', u'1917-Wilson.txt', u'1921-Harding.txt', u'1925-Coolidge.txt', u'1929-Hoover.txt', u'1933-Roosevelt.txt', u'1937-Roosevelt.txt', u'1941-Roosevelt.txt', u'1945-Roosevelt.txt', u'1949-Truman.txt', u'1953-Eisenhower.txt', u'1957-Eisenhower.txt', u'1961-Kennedy.txt', u'1965-Johnson.txt', u'1969-Nixon.txt', u'1973-Nixon.txt', u'1977-Carter.txt', u'1981-Reagan.txt', u'1985-Reagan.txt', u'1989-Bush.txt', u'1993-Clinton.txt', u'1997-Clinton.txt', u'2001-Bush.txt', u'2005-Bush.txt', u'2009-Obama.txt']
>>> for fileid in inaugural.fileids():
...     avg = sum(len(sent) for sent in inaugural.sents(fileids=[fileid])) / len(inaugural.sents(fileids=[fileid]))
...     print fileid, avg
... 
1789-Washington.txt 64.0833333333
1793-Washington.txt 36.75
1797-Adams.txt 69.8648648649
1801-Jefferson.txt 46.0714285714
1805-Jefferson.txt 52.9777777778
1809-Madison.txt 60.2380952381
1813-Madison.txt 39.5151515152
1817-Monroe.txt 30.2704918033
1821-Monroe.txt 38.0542635659
1825-Adams.txt 42.5675675676
1829-Jackson.txt 48.32
1833-Jackson.txt 42.2333333333
1837-VanBuren.txt 43.9052631579
1841-Harrison.txt 43.6428571429
1845-Polk.txt 33.9607843137
1849-Taylor.txt 53.7272727273
1853-Pierce.txt 35.1634615385
1857-Buchanan.txt 34.808988764
1861-Lincoln.txt 29.0217391304
1865-Lincoln.txt 29.0740740741
1869-Grant.txt 30.2195121951
1873-Grant.txt 33.5909090909
1877-Hayes.txt 46.1694915254
1881-Garfield.txt 28.9196428571
1885-Cleveland.txt 41.5454545455
1889-Harrison.txt 30.2547770701
1893-Cleveland.txt 37.1206896552
1897-McKinley.txt 33.6230769231
1901-McKinley.txt 24.5
1905-Roosevelt.txt 33.0606060606
1909-Taft.txt 36.7672955975
1913-Wilson.txt 28.0147058824
1917-Wilson.txt 27.6
1921-Harding.txt 25.2080536913
1925-Coolidge.txt 22.5482233503
1929-Hoover.txt 24.6202531646
1933-Roosevelt.txt 24.2705882353
1937-Roosevelt.txt 21.03125
1941-Roosevelt.txt 22.5882352941
1945-Roosevelt.txt 24.5
1949-Truman.txt 21.7931034483
1953-Eisenhower.txt 22.5609756098
1957-Eisenhower.txt 20.8369565217
1961-Kennedy.txt 29.7307692308
1965-Johnson.txt 18.2446808511
1969-Nixon.txt 22.8773584906
1973-Nixon.txt 29.3913043478
1977-Carter.txt 26.0377358491
1981-Reagan.txt 22.0551181102
1985-Reagan.txt 23.380952381
1989-Bush.txt 18.7103448276
1993-Clinton.txt 22.9012345679
1997-Clinton.txt 21.9821428571
2001-Bush.txt 18.8144329897
2005-Bush.txt 25.0105263158
2009-Obama.txt 24.3392857143

---

宏平均字長在所有文本中的平均值 ：

>>> sum([sum(len(sent) for sent in inaugural.sents(fileids=[fileid])) for fileid in inaugural.fileids()]) / len(inaugural.fileids())
2602.410714285714