Return a List from multiple WS calls in a Scala foreach

Question

I am making a WS call to a service that returns a list of a users connections. After I have received the response I do a foreach on the list and in the foreach I make a WS call to another service to get more details for each connection.

Currently I am trying to use a ListBuffer but due to the async nature of the calls it is being returned empty before the details have been gathered.

My code is as below which returns an empty List to my controller:

def getAllConnections(username: String) = {
    connectionsConnector.getAllConnections(username).map {
      connections =>
        val connectionsList: ListBuffer[ConnectionsResponse] = ListBuffer()
        connections.map {
          connection =>
            usersService.getUser(connection.connectionUsername).foreach {
              case Some(user) =>
                val blah = ConnectionsResponse(user, connection)
                connectionsList.+=(blah)
            }
        }
        connectionsList.toList
    }
  }

Any suggestions on how I can return a Future[List] to my controller would be great, thanks.

Answer 1

for {
  connections <- connectionsConnector.getAllConnections(username)
  usersWithConnection <- Future.traverse(connections){ c =>  userService.getUser(c.connectionUsername).map(u => (u,c))  }
} yield usersWithConnection.collect{ case (Some(user), conn) => ConnectionsResponse(user, conn)}

Should give you some ideas at least. We can use a for comprehension in the context of a future. Future.traverse will turn a list of futures into a future of a list. Needing to return the connection along with the user adds an extra complication but we can just map over the individual future to include the connection with the user.

Answer 2

Use the monadic for loop:

def getAllConnections(username: String) = connectionsConnector.getAllConnections(username) map { connections ->
    for {
      connection  <- connections
      user        <- usersService.getUser(connection.connectionUsername)
    }
    yield ConnectionsResponse(user, connection)
}

I had to guess the exact types you're using so this may need to be adapted, but something very similar to the above should solve your problem.

The outer map maps the original future, and since the first generator of the for comprehension is a list, the result will also be a list.