Google home to send card to the Google Home app

Question

I made a small Google Home App and my service returns a response with a SimpleMessage + Card.

It works perfectly when running the app in the console.actions.google.com simulator. I get the card all good.

But when I test talking to the Google Home, it only sends the text, no trace of the Cards anywhere.

However If i talk to the Google home app on my phone, it does send the card correctly.

Is there something to enable to be able to receive cards sent by Google Home? Is it possible at all?

Answer 1

There is no way to make cards that were sent while the user is talking via Google Home visible, but there are several techniques that you, as a developer, can use if cards are necessary.

First of all - good design suggests that cards should be use to supplement the conversation, not be the focus of the conversation. Make sure the voice conversation itself is important and use the visual elements only when necessary. If your action is overly visual - it may be better suited as a mobile or web app, rather than an Action.

If your device requires a screen, then you can set this in the Action Console when you configure your question. This will, however, prevent it from being used on a Google Home device.

If you don't want to go this route, and want to allow it to be used on a smart speaker, but still take advantage of a screen where it is available, you have a few options.

First is that you can just send the cards. As you've discovered, they won't show up, but they won't cause any problems.

If you want to act slightly differently if a screen is available, you can check for the surface capabilities that the user's Assistant is capable of at that moment. If you're using the node.js library, you can have a command such as

let hasScreen = app.hasSurfaceCapability(app.SurfaceCapabilities.SCREEN_OUTPUT)

to determine if a screen is available and take action based on the variable hasScreen . If you're using JSON, you need to check the array at surface.capabilities or data.google.surface.capabilities to see if "actions.capability.SCREEN_OUTPUT" is one of the available surfaces.

If not, and you get to a point in the conversation where you feel you need to send a visual result, you can also request to continue the conversation on a device that does support screen output.

First, you'll need to make sure that they have a screen available. You'll do this with the node.js library with something like

const screenAvailable = app.hasAvailableSurfaceCapabilities(app.SurfaceCapabilities.SCREEN_OUTPUT);

or by checking the availableSurfaces.capabilities or data.google.availableSurfaces.capabilities parameters in JSON.

If one is available, you can request to continue the conversation there with something like

app.askForNewSurface(context, notif, [app.SurfaceCapabilities.SCREEN_OUTPUT]);

where context is the message that will be said on the Google Home, and notif is the notification that will appear on their mobile device (for example) to let them continue the conversation. If using JSON, you'll need to use a actions.intent.NEW_SURFACE next intent.

Either way, the user will get a notification on their mobile device. Selecting the notification will start up the Assistant on that device and your Action will be called again with parameters that let you check if they are on the new surface. If so - you can send the card.