在J中创建频率矩阵（计算数组中的项目数）

Question

>text
┌───────────┬──────────┬───────────┬──────────┬──────────┬─────────┬──────────┬─────────────┬─────────────┬──────────┬───────────────┬──────────┬──────────┬────────────┬─────────────────┬──────────┬──────────┬──────────────┬─────────────┬─────────────┬────...
│speak      │conceal   │terribl    │option    │write     │book     │come      │tuesdai      │matter       │act       │conceal        │catastroph│integr    │depart      │justic           │put       │wai       │choic         │realli       │bad          │opti...
├───────────┼──────────┼───────────┼──────────┼──────────┼─────────┼──────────┼─────────────┼─────────────┼──────────┼───────────────┼──────────┼──────────┼────────────┼─────────────────┼──────────┼──────────┼──────────────┼─────────────┼─────────────┼────...
│trump      │logu      │talk       │entir     │time      │talk     │entir     │time         │discov       │someth    │frequent       │doe       │logu      │thi         │direct           │logu      │direct    │logu          │differ       │direct       │cons...
├───────────┼──────────┼───────────┼──────────┼──────────┼─────────┼──────────┼─────────────┼─────────────┼──────────┼───────────────┼──────────┼──────────┼────────────┼─────────────────┼──────────┼──────────┼──────────────┼─────────────┼─────────────┼────...
│cohen      │lawyer    │object     │taint     │team      │anoth    │unusu     │move         │lawyer       │trump     │file           │emerg     │motion    │court       │sundai           │night     │sai       │presid        │object       │extraordinari│meas...
├───────────┼──────────┼───────────┼──────────┼──────────┼─────────┼──────────┼─────────────┼─────────────┼──────────┼───────────────┼──────────┼──────────┼────────────┼─────────────────┼──────────┼──────────┼──────────────┼─────────────┼─────────────┼────...
│photo      │presid    │trump      │fire      │jame      │comei    │director  │mai          │did          │mean      │end            │comei     │time      │public      │memoir           │higher    │loyalti   │releas        │comei        │featur       │wide...
├───────────┼──────────┼───────────┼──────────┼──────────┼─────────┼──────────┼─────────────┼─────────────┼──────────┼───────────────┼──────────┼──────────┼────────────┼─────────────────┼──────────┼──────────┼──────────────┼─────────────┼─────────────┼────...
│british    │deleg     │organ      │wrote     │twitter   │russia   │syria     │allow        │access       │douma     │unfett         │access    │essenti   │russia      │syria            │cooper    │western   │diplomat      │confirm      │syria        │russ...
├───────────┼──────────┼───────────┼──────────┼──────────┼─────────┼

cleaned_text

    ┌─────┬───────┬───────┬──────┬─────┬────┬────┬───────┬──────┬───┬───────┬──────────┬──────┬──────┬──────┬───┬───┬─────┬──────┬───┬──────┬──────────┬──────┬────┬────┬────┬────────┬─────┬─────┬───────┬───────┬───────┬───────┬───┬─────┬───────┬────┬───────┬──...
    │speak│conceal│terribl│option│write│book│come│tuesdai│matter│act│conceal│catastroph│integr│depart│justic│put│wai│choic│realli│bad│option│catastroph│option│hard│call│tell│congress│thing│chang│clinton│fervent│support│disagre│sai│least│philipp│rein│longtim│tr...
    └─────┴───────┴───────┴──────┴─────┴────┴────┴───────┴──────┴───┴───────┴──────────┴──────┴──────┴──────┴───┴───┴─────┴──────┴───┴──────┴──────────┴──────┴────┴────┴────┴────────┴─────┴─────┴───────┴───────┴───────┴───────┴───┴─────┴───────┴────┴───────┴──...

each row of "text" is a news article, and I am trying to figure out the number of each vocab from cleaned_text in each article so that I can create a frequency matrix like this:

    art1 art2 art3 ...
mai 4    5    4 
sai 1    0    0
...

I am looking e. and E. verbs to count the number of each vocab in each article, but I am having a hard time to use them in this case.

Can anyone help me on this issue??? Thank you!

Answer 1

I would use a slightly different approach. To keep things simple, I will use the example of p

   p
┌─────┬─────┬─────┬─────┬─────┐
│pants│shirt│shirt│hat  │pants│
├─────┼─────┼─────┼─────┼─────┤
│shoes│shoes│socks│pants│shirt│
├─────┼─────┼─────┼─────┼─────┤
│shirt│hat  │pants│shoes│shoes│
├─────┼─────┼─────┼─────┼─────┤
│socks│pants│shirt│shirt│hat  │
├─────┼─────┼─────┼─────┼─────┤
│pants│shoes│shoes│socks│pants│
├─────┼─────┼─────┼─────┼─────┤
│shirt│shirt│hat  │pants│shoes│
└─────┴─────┴─────┴─────┴─────┘

To get a count of each article of clothing I need to compare each row to the whole vocabulary. I get the whole vocabulary by ravelling ( , ) p and getting the nub ( ~. ) This ensures that every possible word in p is accounted for.

   ~.@:,p
┌─────┬─────┬───┬─────┬─────┐
│pants│shirt│hat│shoes│socks│
└─────┴─────┴───┴─────┴─────┘

Now I will transpose ( |: ) p so that the I can compare each row to the nub using =/ and finish off with totalling up the sum across each item. +/@:

   +/@:(|: =/ ~.@,)p
2 2 1 0 0
1 1 0 2 1
1 1 1 2 0
1 2 1 0 1
2 0 0 2 1
1 2 1 1 0

Reading these numbers against the nub I see the first row has 2-pants 2-shirts 1-hat 0-shoes and 0-socks and by inspection this is correct. The second row has 1-pant 1-shirt 0-hats 2-shoes and 1-sock and so on...

Hope this helps.