Alternatives to using `DT:replaceData()` when `server=FALSE` on shiny application deployed via kubernetes

Question

. 。 Answers to this question are eligible for a +100 reputation bounty. TJGorrie wants to to this question: 这个问题：

Currently I am exploring configuring the kubernetes pods with shinyproxy to sort this out. I am otherwise still interested in a alternative to replaceData that can be used when using clientside processing.

For various reasons I want to be able to use a proxied data table and replaceData while client side processing is being used ie DT::renderDataTable(..., server = FALSE) .

Context

I have a shiny application/dashboard that communicates to a database and presents information to a user. The user is able to fill out a form in the application which will be added to the database and then the shiny app updates the data by making a query to the database to fetch the new information.

The application is currently being deployed via kubernetes using a LoadBalancer with the intention to use multiple replicas to scale up the application as needed. The application is not being run through shinyproxy.

Caveats

Currently, when the application is being run by a single replica (process) the application will behave perfectly fine and is able to use server=TRUE . However when I increase the number of processes/replicas to run, the data is not able to be presented to users unless server=FALSE is specified in renderDataTable . For a currently unknown reason but I suspect it might be due to the sessions not being sticky to IPs

While the code is able to function fine when server = TRUE if I want to allow multiple users to application they all cannot share a single process as the application will become very slow once multiple connections are made. As a result I likely need to use server=FALSE so each user is able to see the data at the cost of a very important functional detail ( replaceData stops working). The product owner of the application is insistent that this behaviour remains intact as the data present is often large and requires some column sorting and paging to find a piece of information you want to look at. And when submitting a form, if I do not use replaceData and reconstruct the table from scratch the users previous table state is lost.

So while I could tear down the datatable and regenerate it within an observeEvent

observeEvent(input$button, {
    ...
    output$table = renderDataTable({DT::datatable(df(), selection = 'single', callback = 
    JS("$.fn.dataTable.ext.errMode = 'none';"))}, server = FALSE)
    ...
})

this would provide a solution that would yield unfavourable behaviour even though it will update the table accordingly.

Repoducible Example

This will create an application with a button and a table. Select a row on the table and then click the button. The expected behaviour would be that the table updates with 'new_content' on the row that is selected. This will only work when server=TRUE , nothing will happen when server=FALSE .

library(shiny)
library(DT)
data(iris)

server <- function(input, output, session) {
  iris$new_col = ''
  df = reactive({iris})
  output$table = renderDataTable({
      DT::datatable(df(), selection = 'single', 
        callback = JS("$.fn.dataTable.ext.errMode = 'none';"))}, server = FALSE) # When TRUE code works fine,,,
  proxy = dataTableProxy('table')

  observeEvent(input$button, {
    # This line would be replacing the write to a db
    iris[input$table_rows_selected, 'new_col'] <- 'changed'
    # This line would be replacing the query to the db to reflect changes the user (and potentially other users have made between loading the data previously.
    df <- reactive({iris})
    proxy %>% replaceData(df(), rownames = TRUE, resetPaging = FALSE)
  })
}
    
ui <- fluidPage(
  actionButton('button', 'Press Me'),
  DT::DTOutput('table') 
)

shinyApp(ui, server)

I have done a fairly extensive search on SO and this was the closest question I could find: DT Editing in Shiny application with client-side processing (server = F) throws JSON Error however this isn't actually answered and provides an answer of "it just does not work".

kubernetes.yaml (only look if you are a wizard)

I am including the yaml file incase there are some kubernetes boffins that know how to specifically solve the above issue with some clever trickery. The described problem might stem from sessions being swapped between replicas thus the data gets miscommunicated but I am honestly not the best at kubernetes... If that is the case and I would then be able to use server=TRUE within the shiny application then this would also solve the problem.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-appname
spec:
  replicas: 5
  selector:
    matchLabels:
      app: appname
  template:
    metadata:
      labels:
        app: appname
    spec:
      containers:
      - name: appname 
        securityContext:
            privileged: false
        image: appname:latest
        ports: 
        - name: http
          containerPort: 3838
---
apiVersion: v1
kind: Service
metadata:
  name: servive-appname
spec:
  ports:
  - name: http
    port: 3838
    protocol: TCP
    targetPort: 3838
  selector:
    app: appname
  type: LoadBalancer
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: ingress-appname
  annotations:
    nginx.org/websocket-services: "service-appname"
spec:
  tls:
  - hosts:
    - appname.url.com
  rules:
  - host: appname.url.com
    http:
      paths:
      - path: /
        backend:
          serviceName: service-appname
          servicePort: 3838

Answer 1

We can try to use reactiveValues combined with the information of input$table_rows_selected . The server argument is equal to FALSE as requested.

library(shiny)
library(DT)
data(iris)

server <- function(input, output, session) {
  iris$new_col = ''
  df = reactiveValues(iris = iris)
  
  
  
  output$table = renderDataTable({
    DT::datatable(df$iris, selection = 'single', 
                  callback = JS("$.fn.dataTable.ext.errMode = 'none';"))}, server = FALSE) # When TRUE code works fine,,,
  
  
  
  observeEvent(input$button, {
    
    # This line would be replacing the write to a db
    df$iris[input$table_rows_selected, c('new_col')] <- 'changed!'
    
  })
}

ui <- fluidPage(
  actionButton('button', 'Press Me'),
  DT::DTOutput('table') 
)

shinyApp(ui, server)

Answer 2

Here is a client-side approach, building up on @jpdugo17's answer and @TJGorrie's initial example, using the stateSave option to maintain the table state on re-rendering. selectPage and updateSearch can be used along with dataTableProxy - the state of input$table_state$order needs to be passed as an option:

library(shiny)
library(DT)
data(iris)

iris$new_col <- ''

server <- function(input, output, session) {
  
  DF = reactiveValues(iris = iris)
  
  output$table <- DT::renderDataTable(expr = {
    if (is.null(isolate(input$table_state))) {
      DT::datatable(
        DF$iris,
        selection = 'single',
        callback = JS("$.fn.dataTable.ext.errMode = 'none';"),
        options = list(stateSave = TRUE)
      )
    } else {
      # print(isolate(input$table_state$order))
      DT::datatable(
        DF$iris,
        selection = 'single',
        callback = JS("$.fn.dataTable.ext.errMode = 'none';"),
        options = list(
          stateSave = TRUE,
          order = isolate(input$table_state$order),
          paging = TRUE,
          pageLength = isolate(input$table_state$length)
        )
      )
    }
  }, server = FALSE)
  
  proxy <- dataTableProxy('table')
  
  observeEvent(input$button, {
    DF$iris[input$table_rows_selected, c('new_col')] <- 'changed!'
  })

  observeEvent(DF$iris, {
    selectPage(proxy, page = input$table_state$start/input$table_state$length+1)
    updateSearch(proxy, keywords = list(global = input$table_state$search$search, columns = NULL)) # see input$table_state$columns if needed
  }, ignoreInit = TRUE, priority = -1)
}

ui <- fluidPage(
  actionButton('button', 'Press Me'),
  DT::DTOutput('table') 
)

shinyApp(ui, server)

Here is a related article .

Answer 3

You can achieve session affinity using cookies if you are using kubernetes/ingress-nginx.

https://kubernetes.github.io/ingress-nginx/examples/affinity/cookie/

but from your yaml, you are using nginx.org's kubernetes-ingress, then you can read

https://github.com/nginxinc/kubernetes-ingress/blob/master/examples/session-persistence/README.md

but it's supported only in NGINX Plus.