我可以使用 Javascript 到 stream 使用 ALAudioDevice（setClientPreferences 然后订阅）来自 Pepper 的麦克风的音频吗？

Question

I was successfully streaming audio sound from Pepper's microphone to my notebook using python:我使用 python 成功地将 Pepper 麦克风的音频声音传输到我的笔记本电脑：

 class SoundReceiverModule(naoqi.ALModule):
    .......
    def start( self ):
        audio = naoqi.ALProxy( "ALAudioDevice", self.strNaoIp, 9559 );
        nNbrChannelFlag = 3; 
        nDeinterleave = 0;
        nSampleRate = 48000;
        audio.setClientPreferences( self.getName(),  nSampleRate, nNbrChannelFlag, nDeinterleave ); 
        audio.subscribe( self.getName() );  

    def processRemote( self, nbOfChannels, nbrOfSamplesByChannel, aTimeStamp, buffer ):
        aSoundDataInterlaced = np.fromstring( str(buffer), dtype=np.int16 );
        aSoundData = np.reshape( aSoundDataInterlaced, (nbOfChannels, nbrOfSamplesByChannel), 'F' );
    .......

However, when I do it using Javascript, no signal was generated after subscribing the ALAudioDevice.但是，当我使用 Javascript 执行此操作时，订阅 ALAudioDevice 后未生成任何信号。 Can anyone help me to solve this problem?谁能帮我解决这个问题？ Thanks!谢谢！

Answer 1

I can't test this at the moment sorry, but I hope this serves as a starting point.抱歉，我目前无法对此进行测试，但我希望这是一个起点。 Please let me know if this works, or if not and you get any errors.请让我知道这是否有效，或者如果没有，您会收到任何错误。

class SoundReceiverModule {
    constructor(){
    }
    processRemote(nbOfChannels, nbrOfSamplesByChannel, buffer, timeStamp){
        console.log(buffer)
    }
}

const channels = 3;
const deinterleaved = 0;
const sampleRate = 48000;
const moduleName = SoundReceiverModule.name;

// with /libs/qimessaging/2/qimessaging.js included
QiSession.connect(function (session) {
    session.service("ALAudioDevice").then(function (ALAudioDevice) {
        ALAudioDevice.setClientPreferences(moduleName, sampleRate, channels, deinterleaved);
        ALAudioDevice.subscribe(moduleName);
    });
});

Answer 2

Here's a different approach I'll add as an alternate answer because it's more of a workaround.这是我将添加的另一种方法作为替代答案，因为它更像是一种解决方法。 The idea is to use the SoundReceiverModule in Python like you have, then publish the sound data in an event which can be received by the javascript APIs.这个想法是像您一样使用 Python 中的 SoundReceiverModule，然后在 javascript API 可以接收的事件中发布声音数据。 We will pass this data through the new ALMemory event, sound-data-processed .我们将通过新的 ALMemory 事件sound-data-processed传递这些数据。

You'll need to make a Choregraphe app that looks like this.您需要制作一个看起来像这样的 Choregraphe 应用程序。 Note that the web page must be stored as html/index.html .请注意，web 页面必须存储为html/index.html 。

In index.html put the following.在 index.html 中放入以下内容。 Of course, you can move this code to a separate.js file and include it in the html.当然，您可以将此代码移动到单独的.js 文件中，并将其包含在 html 中。

<!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no">
    <title>SoundReceiverModuleTest</title>
</head>

<body style="background: black;">
    <script src="/libs/qimessaging/2/qimessaging.js"></script>
    <script type="text/javascript">
        QiSession.connect(function (session) {
            session.service("ALMemory").then(function (ALMemory) {
                ALMemory.subscriber("sound-data-processed").then(function (subscriber){
                    subscriber.signal.connect(function (data){
                        // here is the sound data from the Python event
                        // alert will show it on Pepper's tablet screen
                        alert(data);
                    });
                });
            });
        });
    </script>
</body>

</html>

I've slightly modified the example Python audio module from the docs here .我稍微修改了此处文档中的示例 Python 音频模块。 You can run it from an external computer (if you have the python qi sdk installed) like so:您可以从外部计算机运行它（如果您安装了 python qi sdk），如下所示：

python audio_module.py --ip <nao-ip>

Where audio_module.py is how I've saved the below file, and <nao-ip> is the IP address of your Pepper on the same network as your computer. audio_module.py是我保存以下文件的方式，而<nao-ip>是 Pepper 在与计算机相同的网络上的 IP 地址。

#! /usr/bin/env python
# -*- encoding: UTF-8 -*-

"""Example: Get Signal from Front Microphone & Calculate its rms Power"""


import qi
import argparse
import sys
import time
import numpy as np


class SoundProcessingModule(object):
    """
    A simple get signal from the front microphone of Nao & calculate its rms power.
    It requires numpy.
    """

    def __init__( self, app):
        """
        Initialise services and variables.
        """
        super(SoundProcessingModule, self).__init__()
        app.start()
        session = app.session

        # Get the service ALAudioDevice.
        self.audio_service = session.service("ALAudioDevice")
        self.memory_service = session.service('ALMemory')
        self.isProcessingDone = False
        self.nbOfFramesToProcess = 20
        self.framesCount=0
        self.micFront = []
        self.module_name = "SoundProcessingModule"

    def startProcessing(self):
        """
        Start processing
        """
        # ask for the front microphone signal sampled at 16kHz
        # if you want the 4 channels call setClientPreferences(self.module_name, 48000, 0, 0)
        self.audio_service.setClientPreferences(self.module_name, 16000, 3, 0)
        self.audio_service.subscribe(self.module_name)

        while self.isProcessingDone == False:
            time.sleep(1)

        self.audio_service.unsubscribe(self.module_name)

    def processRemote(self, nbOfChannels, nbOfSamplesByChannel, timeStamp, inputBuffer):
        """
        Publish sound data to ALMemory event.
        """
        self.framesCount = self.framesCount + 1

        if (self.framesCount <= self.nbOfFramesToProcess):
            # convert inputBuffer to signed integer as it is interpreted as a string by python
            soundData = self.convertStr2SignedInt(inputBuffer)
            # send the data to an ALMemory event that can be read in Javascript
            self.memory_service.raise_event('sound-data-processed', soundData)
        else :
            self.isProcessingDone=True

    def convertStr2SignedInt(self, data) :
        """
        This function takes a string containing 16 bits little endian sound
        samples as input and returns a vector containing the 16 bits sound
        samples values converted between -1 and 1.
        """
        signedData=[]
        ind=0;
        for i in range (0,len(data)/2) :
            signedData.append(data[ind]+data[ind+1]*256)
            ind=ind+2

        for i in range (0,len(signedData)) :
            if signedData[i]>=32768 :
                signedData[i]=signedData[i]-65536

        for i in range (0,len(signedData)) :
            signedData[i]=signedData[i]/32768.0

        return signedData


if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("--ip", type=str, default="127.0.0.1",
                        help="Robot IP address. On robot or Local Naoqi: use '127.0.0.1'.")
    parser.add_argument("--port", type=int, default=9559,
                        help="Naoqi port number")

    args = parser.parse_args()
    try:
        # Initialize qi framework.
        connection_url = "tcp://" + args.ip + ":" + str(args.port)
        app = qi.Application(["SoundProcessingModule", "--qi-url=" + connection_url])
    except RuntimeError:
        print ("Can't connect to Naoqi at ip \"" + args.ip + "\" on port " + str(args.port) +".\n"
               "Please check your script arguments. Run with -h option for help.")
        sys.exit(1)
    MySoundProcessingModule = SoundProcessingModule(app)
    app.session.registerService("SoundProcessingModule", MySoundProcessingModule)
    MySoundProcessingModule.startProcessing()

Answer 3

Error generated when the choregraphe was run:运行 choregraphe 时产生的错误：

[ERROR] behavior.box:_safeCallOfUserMethod:125 _Behavior__lastUploadedChoregrapheBehaviorbehavior_11662506592:/ALSoundDevice_1: Traceback (most recent call last): File "/opt/aldebaran/lib/python2.7/site-packages/albehavior.py", line 115, in _safeCallOfUserMethod func() File "<string>", line 31, in onUnload File "/opt/aldebaran/lib/python2.7/site-packages/ialbehavior.py", line 108, in <lambda> __getattr__ = lambda self, name: _swig_getattr(self, behavior, name) File "/opt/aldebaran/lib/python2.7/site-packages/ialbehavior.py", line 57, in _swig_getattr raise AttributeError(name) AttributeError: soundReceiver [ERROR] behavior.box:_safeCallOfUserMethod:125 _Behavior__lastUploadedChoregrapheBehaviorbehavior_11662506592:/ALSoundDevice_1: Traceback (most recent call last): File "/opt/aldebaran/lib/python2.7/site-packages/albehavior.py", line 115, in _safeCallOfUserMethod func() File "<string>", line 34, in onInput_onStart TypeError: __init__() takes exactly 2 arguments (1 given)

However, when I added an input name to self.soundReceiver = SoundReceiverModule( 'SoundReceiver' ), then pepper's shoulder leds turned yellow (then red) after running the choregraphe and no signal was received on browsing the webpage http://pepper-ip/apps/....但是，当我在 self.soundReceiver = SoundReceiverModule( 'SoundReceiver' ) 中添加一个输入名称时，运行 choregraphe 后，pepper 的肩部 LED 变为黄色（然后是红色），并且在浏览网页 http://pepper-ip 时未收到任何信号/应用/....

Answer 4

The problem was partly solved:问题部分解决了：

import numpy as np
class SoundReceiverModule(ALModule):
    def __init__( self, strModuleName, strNaoIp ):
        try:
            ALModule.__init__(self, strModuleName )
            self.BIND_PYTHON( self.getName(), "callback" )
            self.strNaoIp = strNaoIp
        except BaseException, err:
            print( "loading error: %s" % str(err) )
    
        def __del__( self ):
            print( "SoundReceiverModule.__del__: cleaning everything" );
            self.stop();        

    def start(self):
        audio = ALProxy("ALAudioDevice", self.strNaoIp, 9559)
        nNbrChannelFlag = 3
        nDeinterleave = 0
        nSampleRate = 48000
        audio.setClientPreferences(self.getName(),  nSampleRate, nNbrChannelFlag, nDeinterleave)
        audio.subscribe(self.getName())

    def stop(self):
        audio = ALProxy("ALAudioDevice", self.strNaoIp, 9559)
        audio.unsubscribe(self.getName())

    def processRemote(self, nbOfChannels, nbrOfSamplesByChannel, aTimeStamp, buffer):
        self.memory = ALProxy("ALMemory") 
        aSoundDataInterlaced = np.fromstring(str(buffer), dtype=np.int16)
        aSoundData = np.reshape(aSoundDataInterlaced, (nbOfChannels, nbrOfSamplesByChannel), 'F')
        self.memory.raiseEvent('sound-data-processed', aSoundData)

class MyClass(GeneratedClass):
    def __init__(self):
        GeneratedClass.__init__(self)
        self.myBroker = ALBroker("myBroker", "0.0.0.0", 0, "192.168.24.201", 9559)

    def onLoad(self):   
        #put initialization code here
        pass

    def onUnload(self):
        #put clean-up code here
        SoundReceiver.stop()

    def onInput_onStart(self):
        global SoundReceiver
        SoundReceiver = SoundReceiverModule("SoundReceiver", "192.168.24.201")
        SoundReceiver.start()

    def onInput_onStop(self):
        self.myBroker.shutdown()
        self.onUnload() 
        
    def onStopped():    
        self.onStopped()

However, the unsolved problem is 'no sound data streamed from subscribing the SoundReceiver'.但是，未解决的问题是“订阅 SoundReceiver 时没有声音数据流”。

我可以使用 Javascript 到 stream 使用 ALAudioDevice（setClientPreferences 然后订阅）来自 Pepper 的麦克风的音频吗？

问题描述

4 个解决方案

解决方案1
0 2020-06-14 10:58:14

解决方案2
0 2020-06-20 04:49:01

解决方案3
0 2020-06-22 01:29:25

解决方案4
0 2020-06-26 03:33:30

我可以使用 Javascript 到 stream 使用 ALAudioDevice（setClientPreferences 然后订阅）来自 Pepper 的麦克风的音频吗？

问题描述

4 个解决方案

解决方案1 0 2020-06-14 10:58:14

解决方案2 0 2020-06-20 04:49:01

解决方案3 0 2020-06-22 01:29:25

解决方案4 0 2020-06-26 03:33:30

解决方案1
0 2020-06-14 10:58:14

解决方案2
0 2020-06-20 04:49:01

解决方案3
0 2020-06-22 01:29:25

解决方案4
0 2020-06-26 03:33:30