Test Suite for Actions on Google Smart Home Error

Question

I'm implementing a Smart Home action, and while running tests on the Test Suite , I got the following error message during a brightness trait test:

Starting test...
  Utterance (1/1): Ok Google. Set the Dimmer brightness to 75 percent . . . . . . . . . . . . . . . . . . . . FAIL

AssertionError: Expected state to include: {"brightness":{"xRange":[55,95]}}, actual state: {"on":true}: expected false to be true

All the onoff requests are working fine and the Test Suite doesn't complain, just say "PASS" as supposed to be.

But the BrightnessAbsolute request was received and processed successfully by my back-end application (and I can even see the lights been changed as the Test Suite runs), returning the JSON response back in the format described in the documentation.

Request:

{
   "inputs":{
      "0":{
         "context":{
            "locale_country":"CA",
            "locale_language":"en"
         },
         "intent":"action.devices.EXECUTE",
         "payload":{
            "commands":{
               "0":{
                  "devices":{
                     "0":{
                        "id":"xxxxxxxx"
                     }
                  },
                  "execution":{
                     "0":{
                        "command":"action.devices.commands.BrightnessAbsolute",
                        "params":{
                           "brightness":75
                        }
                     }
                  }
               }
            }
         }
      }
   },
   "requestId":"12608597522424949295"
}

Response:

{
   "requestId":"12608597522424949295",
   "payload":{
      "commands":{
         "0":{
            "ids":{
               "0":"xxxxxxxx"
            },
            "status":"SUCCESS",
            "states":{
               "on":true,
               "brightness":75,
               "online":true
            }
         }
      }
   }
}

Does anyone have an idea on how to make this test pass on the Test Suite ? Thank you.

Answer 1

The test suite verifies commands based on the state reported in Home Graph for each device under test. It's important to note that the EXECUTE response is only used to provide a direct result back to the user (ie voice through the Assistant or UI in the Home app) and these response do not update Home Graph state . Currently, only the Report State API updates Home Graph for the given device.

Since Report State is required to update Home Graph, it's beneficial to call the API following SYNC to provide Home Graph with the initial state of the device. This is especially important if you only call Report State on changes (ie not when a device is commanded to the state it's already in). This may be what you're experiencing since the test suite is reporting there is no value in Home Graph for brightness.

If you do not follow this pattern, then calling Report State after every EXECUTE would also ensure Home Graph is up to date, but it would require you to make more API calls.

Answer 2

Could you solve this in the end? I'm having the same issue with the test suit and I'm pretty sure I'm reporting a state exactly equivalent to the one returning in Query