在 Python 中捕获输出音频频谱

Question

The challenge

To capture the spectrum of the output audio.

1st attempt

import gtk, gst

def playerbin_message(bus, message):
    if message.type == gst.MESSAGE_ELEMENT:
        struct = message.structure
        if struct.get_name() == 'spectrum':
            print struct['magnitude']
pipeline = gst.parse_launch(
    'pulsesrc device="alsa_output.pci-0000_00_1b.0.analog-stereo.monitor" ! spectrum ! fakesink')
bus = pipeline.get_bus()
bus.add_signal_watch()
bus.connect('message', playerbin_message)
pipeline.set_state(gst.STATE_PLAYING)
gtk.main()

Where does it fail?

From some reason, the script work only sometimes. Usually it won't print anything.

@otopolsky found in verbose mode log the following line:

INFO spectrum gstspectrum.c:1051:gst_spectrum_transform_ip:<spectrum0> interval 0:00:00.100000000, fpi 4410, error 0:00:00.000000000

2nd attempt

As @otopolsky adviced, I tried running:

GST_DEBUG=4,spectrum:7 gst-launch-0.10 pulsesrc device="alsa_output.pci-0000_00_1b.0.analog-stereo.monitor" ! spectrum ! fakesink > out.log

And got this output .

This get stuck :

gst-launch-0.10 -v -m pulsesrc device="alsa_output.pci-0000_00_1b.0.analog-stereo.monitor" ! spectrum ! fakesink

While this works :

gst-launch-1.0 -v -m pulsesrc device="alsa_output.pci-0000_00_1b.0.analog-stereo.monitor" ! spectrum ! fakesink

(only the version was changed).

3rd attempt

Moved to python gi:

import gi
gi.require_version('Gst', '1.0')
from gi.repository import GObject, Gst

GObject.threads_init()
Gst.init(None)

def handler(bus, msg):
    if msg.type == Gst.MessageType.ELEMENT:
        struct = msg.get_structure()
        print struct.get_value('magnitude')

p = Gst.parse_launch(
    'pulsesrc device="alsa_output.pci-0000_00_1b.0.analog-stereo.monitor" ! spectrum ! fakesink')

bus = p.get_bus()
bus.add_signal_watch()
bus.connect('message', handler)
p.set_state(Gst.State.PLAYING)

ctx = GObject.main_context_default()
while ctx:
    ctx.iteration()

Where does it fail?

TypeError: unknown type GstValueList

The questions

• How can the scripts be fixed to work on every execution?
• Is there another way to capture output spectrum using Python?

Answer 1

You are using 0.10.36 .. quite old.. can you not use newer gstreamer, say 1.2?

you are running Ubuntu so it should not be a problem.. the 0.10.x and 1.2 can easily coexist beside each other.. I dont know if its ok with your python bindings..

This may be the problematic line (from your pastebin, please add that part into your question..)

INFO spectrum gstspectrum.c:1051:gst_spectrum_transform_ip:<spectrum0> interval 0:00:00.100000000, fpi 4410, error 0:00:00.000000000

This is the source code .. It says something about rounding error and uninitialized FFT:

/* If we don't have a FFT context yet (or it was reset due to parameter * changes) get one and allocate memory for everything
*/

/* number of sample frames we process before posting a message * interval is in ns */

/* rounding error for frames_per_interval in ns, * aggregated it in accumulated_error */

you can run again to get more details with:

export GST_DEBUG=4,spectrum:7

Can you run the pipe in gst-launch?

GST_DEBUG=4,spectrum:7 gst-launch-0.10 pulsesrc device="alsa_output.pci-0000_00_1b.0.analog-stereo.monitor" ! spectrum ! fakesink

UPDATE:

Try with alsasrc if you hear it.. (autoaudiosink or alsasink):

gst-launch-1.0 alsasrc device=hw:0,2 ! spectrum ! autoaudiosink

hw:X,Y format explanation: X = card number, Y = device name

Getting it from arecord -l for input devices like microphone etc..

and aplay -l for output devices..

> arecord -l | grep card
card 0: PCH [HDA Intel PCH], device 0: ALC887-VD Analog [ALC887-VD Analog]
card 0: PCH [HDA Intel PCH], device 2: ALC887-VD Alt Analog [ALC887-VD Alt Analog]

Answer 2

It seems that GstValueList is not supported by gst-python.

There is an inelegant way to get data from the GstValueList . We can convert the message structure into a string, then convert the string to a float array.

import gi
gi.require_version('Gst', '1.0')
from gi.repository import GObject, Gst
import re

GObject.threads_init()
Gst.init(None)


def on_message(bus, msg):
    struct = msg.get_structure()
    if struct.get_name() == 'spectrum':
        struct_str = struct.to_string()
        magnitude_str = re.match(r'.*magnitude=\(float\){(.*)}.*', struct_str)
        if magnitude_str:
            magnitude = map(float, magnitude_str.group(1).split(','))
            print magnitude

pipeline = Gst.parse_launch(
    'audiotestsrc ! spectrum interval=1000000000 bands=16 post-messages=true message-magnitude=1 ! directsoundsink'
)

bus = pipeline.get_bus()
bus.add_signal_watch()
bus.connect('message::element', on_message)

pipeline.set_state(Gst.State.PLAYING)

loop = GObject.MainLoop()
loop.run()

Answer 3

Realising this question is very old (but helpful!), but...

I had the same problem and figured out it was just a matter of using the get_list() binding not get_value() . This gives you a boolean and and a GObject.ValueArray which seems usable:

struct = msg.get_structure()
success, values = struct.get_list("magnitude")
if struct.get_name() == "spectrum":
    mags = [values.get_nth(i) for i in range(values.n_values)]