is there a way to speak and play audio file simultaneously on google assistant

Question

I am using dialogflow fulfillment, In my welcome intent first the audio file plays and then bot speaks.

is there a way to make both the things work simultaneously, means audio file should play in background in low volume and bot speak the welcome text over audio.

app.intent('welcome', (conv) => {
    conv.ask(`<speak><audio src="https://actions.google.com/sounds/v1/cartoon/cartoon_cowbell.ogg"></audio><speak>Welcome! What would you like to check?</speak></speak>`);
});

Answer 1

Yes, you can use the <par> and <media> tags in Google's SSML to create parallel "tracks" of audio that are played at the same time. Note that these aren't standard SSML, but are Google extensions.

The <par> tag indicates that the contents are all to be played in parallel. Inside it, you typically have multiple <media> blocks (although you can also have other <par> blocks, as well as <seq> blocks for sequential portions). Each <media> block can have attributes indicating some adjustments for that block (such as volume or start offsets) and contains either <speak> or <audio> blocks.

So to do what you describe, you could have something like this:

<speak>
  <par>
    <media>
      <audio src="https://actions.google.com/sounds/v1/cartoon/cartoon_cowbell.ogg"></audio>
    </media>
    <media>
      <speak>Welcome! What would you like to check?</speak>
    </media>
  </par>
</speak>

Google also provides a tool to help you design this: https://actions-on-google-labs.github.io/nightingale-ssml-editor/