Topic Modelling by Category in Data

Question

I wanted to see if there's a way we can get the topics by topic modelling using LDA by category instead of whole dataset.

My data looks like this.

Comment                                                                                  Division
Smooth execution of Regional Administration in my absence. Well done.                    Finance
Job well done in completing CPs and making the facility available well in time.          Finance
Good Job performed on the successful implementation of Cash IVR.                         Commercial

I want to get topics by division

My Current Code gives me overall topics.

library(udpipe)
library(topicmodels)
x <- udpipe(x= pnotips$Feedback.Comments, object= ud_model)
x$topic_level_id <- unique_identifier(x, fields = c("doc_id", "paragraph_id", "sentence_id"))

dtf <- subset(x, upos %in% c("NOUN", "ADJ"))


dtf <- document_term_frequencies(dtf, document = "topic_level_id", term = "lemma")
dtm <- document_term_matrix(x = dtf)
dtm_clean <- dtm_remove_lowfreq(dtm, minfreq = 3)
## Build topic model + get topic terminology
m <- LDA(dtm_clean, k = 4, method = "Gibbs", 
         control = list(nstart = 5, burnin = 2000, best = TRUE, seed = 1:5))
topicterminology <- predict(m, type = "terms", min_posterior = 0.025, min_terms = 5)

I want to get topics by each Division_Name

Sample Data

  structure(list(Feedback.Comments = c("Excellent kick start of p", 
"Nauman is very collaborative when it comes to team deliverable. He takes ownership and ensure to support whenever needed or asked for. ", 
"Thank you for being very collaborative and designing and planning the whole workshop that deemed success today for R", 
"Amazing knowledge sharing session conducted by you. Truly innovative.", 
"Thanks a lot for your collaboration during my training dates", 
"During Prepaid Consolidation Step 1, you have done excellent job in handling the Mediation stream resulting in a smooth delivery.  The highlights of this delivery was the collaboration which was executed excellently.", 
"He handles all the organization customers in a very collaborative manner.", 
"Noor ul Amin is very supportive and initiative hungry person, always take very quick/bold step when ever any issue happened. ", 
"Keeping check on timely rectification of observations by HSSE with good speed.", 
"Smooth execution of Regional Administration in my absence. Well done.", 
"Good Job performed on the successful implementation of Jazz Cash IVR. the 1st selfservice IVR for financial transactions in industry.", 
"Despite challenges on the resource side you have done exceptionally well in managing the UATs, Prepaid Consolidation and assigned tasks.\n\nWe need focus more on FCR & NPS related areas so we are able to meet our KPIs, looking forward for stats and feedback on time. It would be better if we dedicate one resources on this side and not deploy all resources on prepaid consolidation (it will not give us any benefit)", 
"Job well done in reorganizing all the investments to fixed portfolios.\n Keep it up.", 
"Well done in reorganizing PF process and resolving legacy issues.", 
"Job well done in completing CPs and making the facility available well in time.", 
"You always seems supportive on these requests.sometimes you also submitted input in late hours of the day. Keep ot up.", 
"Well done on completing Hiperos screening for almost 30 profiles. Please pass the feedback to Khurram and Babar.", 
"You always make your concerns clear at a judgment. It always good to have a critical view on things, helps avoiding mistakes. Keep it up.", 
"Both FLT in Lahore and Karachi were planned, managed and executed to the perfection under your lead. Wonderful collaboration with P&O and cross functional teams. Good job and good management. ", 
"Very good resource. Always up to the expectations.\nDid good job in back office evaluations"
), division_name = c("People & Organization", "People & Organization", 
"People & Organization", "People & Organization", "People & Organization", 
"Technology", "Finance", "Finance", "People & Organization", 
"People & Organization", "Commercial", "Commercial", "Finance", 
"Finance", "Finance", "Finance", "Finance", "Finance", "People & Organization", 
"Commercial")), row.names = c(1L, 3L, 4L, 5L, 6L, 7L, 8L, 9L, 
10L, 11L, 17L, 21L, 23L, 24L, 25L, 29L, 31L, 32L, 35L, 37L), class = "data.frame")

Answer 1

You can run a separate topic model for each division - this is easily done by subsetting the data by department (using something like department1data <- subset(mydata, division == "Finance") ) and then running topic models. This gives you the topics each department discusses, and should work reasonably well.

Alternatively, you could use the ````posterior``` method or something similar to try looking at the topic loadings per document that comes from each division, and, for example, averaging over them. I find the tidy package to be useful when working with LDA in R (see section 6.2.2 in the linked tutorial if this is what you want to do).