How to do FFT on input from microphone/line in with Python

Question

I'm trying to compute the FFT of some sound which is coming from the microphone input. The FFT is computed using Python's Numpy. It is already working when I use a saved wave file. Now I want to do this with data coming from the microphone input.

File fft.py

import numpy as np
from struct import unpack

power = []

# Return power array index corresponding to a particular frequency
def piff(val, sample_rate, chunk):
   return int(4 * chunk * val/sample_rate)


def calculate_levels(matrix, weighting, data, chunk, sample_rate):
   # Convert raw data (ASCII string) to numpy array
   data = unpack("%dh"%(len(data)/2),data)
   data = np.array(data, dtype='h')

   # Apply FFT - real data
   fourier = np.fft.rfft(data)

   # Remove last element in array to make it the same size as chunk
   fourier = np.delete(fourier, len(fourier) - 1)

   # Find average 'amplitude' for specific frequency ranges in Hz
   power = np.abs(fourier)

   lower_bound = 0
   upper_bound = 32
   for i in range(len(matrix)):
       mean = np.mean(power[piff(lower_bound, sample_rate, chunk) : piff(upper_bound, sample_rate, chunk):1])
       matrix[i] = int(mean) if np.isnan(mean) == False else 0

       lower_bound = upper_bound
       upper_bound = upper_bound << 1

   # Tidy up column values for the LED matrix
   matrix = np.multiply(matrix, weighting)

   # Set all values smaller than first argument to 0, all greater than second argument to 4095
   matrix = matrix.clip(0, 4095) 
   return matrix

The output of my calculate_levels function is a vector with the level of certain frequencies.

File myprogram.py:

import argparse
import socket
import time
import alsaaudio
import wave
import decoder
import sys
import os
import numpy
from fft import calculate_levels

# Initialize the FFT matrix
matrix    = [0, 0, 0, 0, 0, 0, 0, 0]
weighting = [512, 256, 1024, 1024, 2048, 4096, 4096, 4096]

# Precalculate weighting matrix
weighting = numpy.true_divide(weighting, 1000000)

# Set up audio
# args.chunk is 512
# args.sample_rate is 44100 
inp = alsaaudio.PCM(alsaaudio.PCM_CAPTURE, alsaaudio.PCM_NORMAL)
inp.setchannels(2)
inp.setrate(args.sample_rate)
inp.setformat(alsaaudio.PCM_FORMAT_S16_LE)
inp.setperiodsize(args.chunk)

out = alsaaudio.PCM(alsaaudio.PCM_PLAYBACK, alsaaudio.PCM_NORMAL)
out.setchannels(2)
out.setrate(args.sample_rate)
out.setformat(alsaaudio.PCM_FORMAT_S16_LE)
out.setperiodsize(args.chunk)

# Store the recorded audio in a file
w = wave.open('test.wav', 'w')
w.setnchannels(2)
w.setsampwidth(2)
w.setframerate(args.sample_rate)

# Process audio file   
print "Listening..."

while True:
    l, data = inp.read()
    matrix = calculate_levels(matrix, weighting, data, args.chunk, args.sample_rate)

    # Do something with the spectrum ...

    # Go on reading the music file
    out.write(data)

    # Save
    w.writeframes(data)

Unfortunately, I get the following error:

File "fft.py", line 13, in calculate_levels
    data = unpack("%dh"%(len(data)/2),data)
struct.error: unpack requires a string argument of length 2048

How can this error be avoided?

Update: This is the output when I put some print commands in the calculate_levels method:

   print type(data)
   print data
   data = unpack("%dh"%(len(data)/2),data)
   print type(data)
   print data
   data = np.array(data, dtype='h')

Output

<type 'str'>
\F9\FFDi\84\80:\CE\D2\EE}`&\89  \9B\9E  \E6\B3\CEi
// and some more unreadable values


<type 'tuple'>
(-7, 3908, 2153, 2948, 1920, 1338, 462, 2258, 494, 3965, 1632, 4134, 2441, 3227, 2462, 2022, 947, 1230, -1175, 1942, -528, 2771, 1296, 894, -243, -1598, -2106, -1121, -856, -18, -507, -1664, -2082, -3105, -2045, -1497, -1101, -146, -721, -1675, -1027, -3448, -2358, -2581, -1954, -779, 293, -955, -199, -1962, -2096, -1719, -1497, -1170, -719, -980, -879, -503, -265, 65, 311, 611, 101, 825, -335, 493, -482, 1092, 574, 1927, 1006, 1044, -688, 521, -1455, 1179, -589, 559, -873, -331, -1506, 11, -1134, 639, -1190, 1307, -1239, 1581, -707, 1556, 195, 2611, 1751, 3189, 2523, 2508, 1693, 2991, 1555, 4211, 2555, 4153, 2669, 3332, 2111, 2187, 1426, 1120, 501, 773, 352, 836, 476, 459, -775, -1248, -2499, -2067, -2338, -588, -1038, -1271, -1423, -3376, -2563, -1127, -1387, 796, -513, -1740, -1918, -1997, -1538, -450, -112, -1896, -1712, -1518, -2252, 1556, 636, 1574, 1365, 17, -468, 336, -606, 2116, 701, 3832, 1914, 3565, 2623, 2620, 2080, 3725, 1456, 4205, 1560, 2905, 1057, 2416, 193, 1880, 121, 1223, 358, 1834, -253, 1592, -1279, 499, -1207, 830, -675, 558, -1843, -400, -2675, 1257, -616, 2825, 639, 808, -1402, -487, -2721, 807, -1708, 703, -1120, -230, -1089, 830, -817, 1058, -1263, -829, -2162, -1325, -1992, -104, -739, 524, 481, 49, 239, -1104, -1179, -1233, -1210, -266, 177, -626, 265, -1584, -866, -1148, -1086, -626, -612, -621, -507, -419, -498, -510, -335, -144, -200, 866, 164, 1237, 516, 1569, 526, 2606, 1009, 2924, 1696, 2476, 1401, 3153, 1339, 4358, 2067, 3798, 1466, 2466, 227, 2649, 518, 2940, 751, 2358, -99, 2548, -48, 2172, 324, 326, -450, 230, -722, 1865, 411, 1834, 485, 346, -758, -277, -537, 446, 1002, 619, 527, -790, -1503, -1401, -1608, -988, -478, -2039, -587, -3092, -1511, -2225, -1516, -1332, -643, -777, 580, -765, 605, -1777, -845, -659, -759, 1550, 908, 148, 221, -1697, -1383, -155, -300, 596, 616, -459, -1002, -145, -1579, 877, -263, 1257, 230, 1613, 166, 1340, 648, 1207, 1061, 2685, 1292, 4016, 1880, 4375, 2573, 5205, 2973, 5332, 2842, 3979, 2320, 4156, 1911, 5216, 1519, 3989, 1147, 2849, 1358, 3308, 1145, 1860, -354, -90, -1164, 767, -734, 945, -1206, -1201, -2302, -1662, -2264, -577, -1992, -1073, -1869, -2345, -2065, -2178, -2923, -1219, -2819, -1362, -1499, -1976, -1311, -2017, -1982, -2464, -1834, -2060, -883, -258, 79, -754, -372, -2562, -1399, -1170, -210, 395, 1048, -507, -241, -443, -851, 356, 246, 123, 195, 183, -607, 337, -903, 150, -1327, 441, -1231, 572, -499, 674, -423, 1551, -529, 2183, 133, 2516, 884, 2731, 906, 2331, 937, 2644, 1644, 4033, 2440, 4428, 2367, 3390, 1729, 2417, 1231, 2223, 1011, 1937, 484, 833, -494, 617, -650, 1476, 59, 769, -417, -778, -1411, -781, -912, -480, -712, -975, -1637, -1033, -1523, -1097, -601, -1588, -328, -1538, -601, -1795, -1523, -3066, -2779, -3254, -2178, -2106, -605, -2174, -810, -2857, -1665, -1680, -1182, -449, -630, -749, -825, -642, -627, 9, -464, 636, -195, 1932, 902, 2333, 1033, 1535, -35, 2300, 411, 3107, 1353, 2289, 987, 2584, 1496, 3974, 2716, 4104, 2275, 3897, 1689, 4442, 2549, 5094, 3206, 5257, 2981, 4773, 2414, 4837, 2278, 5167, 2544, 3814, 1760, 2408, 280, 2810, 147, 2558, 251, 606, -984, -732, -1954, -816, -1769, -915, -1615, -1429, -2093, -2005, -2778, -2630, -2809, -3413, -2374, -3565, -2589, -2903, -3151, -2586, -2852, -2967, -2414, -3221, -2722, -3115, -2872, -2553, -2606, -2052, -2442, -1730, -1908, -1136, -1098, -444, -930, -40, -1171, 381, -928, 904, -440, 1359, 94, 1888, 572, 2441, 1008, 2942, 1270, 2845, 1021, 2705, 814, 3665, 1272, 4378, 1739, 4148, 1783, 4456, 1900, 4824, 1817, 4273, 1601, 4245, 2154, 5118, 2913, 5252, 2778, 3910, 1981, 2359, 1672, 2515, 2066, 3134, 2586, 1955, 1907, 69, 27, -581, -914, -299, 240, -723, 407, -2513, -2092, -3197, -3232, -1843, -1812, -1800, -1545, -2818, -2252, -2573, -1905, -2648, -1957, -3196, -2083, -2637, -1456, -2684, -1795, -3012, -2071, -2503, -1044, -2809, -1145, -3131, -2067, -2137, -1361, -1347, -242, -589, 0, 283, 77, 239, -24, 714, 509, 1792, 1859, 2001, 1993, 2407, 995, 3220, 1271, 3698, 2584, 4424, 3153, 4830, 2989, 4504, 2616, 4761, 2324, 5039, 2379, 4840, 2616, 5108, 2793, 5526, 3092, 5287, 2673, 4604, 1520, 3841, 1024, 3127, 950, 2299, 167, 1802, -453, 1637, -834, 534, -1987, -763, -2817, -895, -2820, -1194, -3118, -2027, -3514, -2400, -3645, -2975, -3705, -3058, -3004, -2114, -1850, -2460, -2179, -3865, -3417, -4033, -3303, -3425, -2323, -3101, -1990, -2887, -2073, -2729, -2246, -2435, -2633, -2194, -2490, -2099, -1757, -1358, -1348, -373, -1032, 77, -534, 480, -372, 657, -287, 804, -39, 1687, 351, 2263, 922, 1908, 853, 2099, 200, 3010, 621, 3591, 1314, 3832, 847, 3980, 744, 4071, 1294, 4480, 1412, 4892, 1510, 4662, 1504, 4194, 1566, 4246, 2331, 3926, 2134, 2694, 1042, 1698, 902, 923, 559, -140, -558, -374, -484, -479, -473, -2060, -1858, -3301, -2478, -2781, -2294, -2574, -2611, -3138, -2509, -2994, -1950, -2509, -1773, -2460, -1740, -2396, -1440, -1900, -951, -1573, -658, -1791, -903, -1656, -1068, -999, -769, -974, -773, -1078, -714, -210, -272, 408, 123, 130, 263, 161, 289, 1073, 680, 2151, 1698, 2437, 2237, 2542, 2074, 3248, 2244, 3371, 2328, 3137, 2230, 4142, 2708, 4850, 2778, 4192, 1999, 4154, 2484, 4748, 3758, 4808, 3579, 4698, 2679, 4311, 2577, 3714, 2762, 3260, 2352, 2629, 1576, 2116, 731, 1693, -129, 736, -717, 26, -834, -774, -1712, -2432, -3376, -2623, -3491, -1766, -2376, -2759, -2540, -3966, -3388, -3420, -3343, -3268, -3151, -3685, -2913, -3044, -1930, -2222, -1083, -1913, -1264, -1927, -1590, -2126, -1210, -1826, -885, -922, -867, -10, -224, 535, 559, 701, 558, 984, 523, 1107, 316, 1373, -11, 2734, 947, 3870, 2294, 3280, 1795, 3026, 798, 3888, 786, 4238, 1193, 4537, 2008, 4763, 2556, 3650, 1613, 2549, 682, 2764, 1188, 3077, 1904, 2780, 1528, 2262, 931, 1603, 937, 1066, 1165, 410, 997, -129, 440, -107, -450, -517, -1525, -1825, -2009, -2739, -2216, -3139, -3149, -3852, -3677, -4249, -3248, -4134, -3446, -4136, -3956, -3730, -3020, -2834, -1707, -2691, -1511, -2708, -1377, -1478, -450, -160, 326, -152, 512, -201, 701, 527, 773, 1354, 756, 1733, 904, 1802, 711, 2034, 30, 2391, -106, 2543, 332, 3023, 508, 3721, 654, 3515, 566, 2633, -74, 2492, -122, 2920, 559, 3028, 706, 2597, 279, 1878, -28, 1429, -252, 1694, -70, 2172, 526, 2054, 601, 1758, 331, 1983, 546, 2001)


<type 'str'>
\86a\EAO    (\CF
// and some more unreadable values

<type 'tuple'>
(902, 1889, 1002, 2383, 1320, 2767, 1674, 2298, 1553, 1815, 1237, 1434, 834, 839, 153, 199, -387, -526, -419, -1433, -784, -2398, -2047, -3160, -2961, -2947, -2812, -2373, -2321, -2718, -1907, -2965, -1642, -2377, -1601, -2156, -1632, -1940, -1735, -1260, -1348, -1038, -448, -749, -255, 32, -557, 501, -318, 989, -189, 1678, -43, 1849, 456, 1927, 337, 2421, 77, 2654, 870, 2304, 845, 2132, 66, 2681, 739, 2964, 1319, 1927, 396, 1043, 194, 1280, 858, 1467, 906, 1561, 841, 1802, 710, 1261, 267, 551, 194, 878, 191, 1602, 26, 1842, 265, 1453, 430, 692, 127, 64, -464, -250, -1231, -484, -1379, -777, -868, -1332, -1316, -1746, -2051, -2099, -2036, -2851, -2387, -3324, -2989, -2957, -2930, -2508, -3092, -2369, -3130, -2065, -2100, -1138, -1350, -311, -1497, -15, -922, 802, 417, 1890, 1341, 2348, 1924, 2945, 2464, 3668, 2737, 3369, 2969, 3364, 3162, 3895, 2806, 3379, 2427, 3025, 2474, 3499, 2190, 3027, 1812, 2228, 1602, 2284, 859, 1976, 56, 1299, 83, 754, 146, 81, -475, -525, -1112, -945, -1366, -1146, -1409, -1004, -1404, -1231, -1374, -1625, -1144, -1170, -695, -655, -232, -504, 155, 75, 557, 639, 976, 535, 997, 220, 511, -65, 366, 120, 579, 686, 433, 278, -237, -591, -1160, -804, -1728, -1072, -1896, -1448, -2328, -991, -2671, -210, -2114, 64, -1671, -87, -1848, 74, -1748, 701, -1264, 1089, -598, 1158, 96, 1727, 517, 2286, 777, 2296, 1115, 2453, 1522, 2842, 2213, 2997, 2826, 2989, 2951, 2898, 3105, 2785, 3151, 2593, 2658, 1807, 2356, 1493, 2531, 1569, 1930, 510, 455, -408, -77, -155, -50, -826, -1286, -1735, -2532, -1259, -2232, -1061, -2052, -1915, -2880, -2013, -3146, -1328, -2713, -792, -2428, -489, -2069, -66, -1240, 365, -322, 381, 55, 7, -426, -195, -921, -194, -723, -261, -354, -592, -269, -969, -569, -1077, -927, -1239, -907, -1384, -1000, -937, -1329, -625, -1422, -570, -1293, 168, -810, 1250, 270, 2027, 965, 2850, 944, 3355, 1173, 3834, 1743, 4796, 2293, 5440, 2734, 5296, 2861, 4907, 2555, 4573, 2121, 4211, 1733, 3618, 1199, 2904, 749, 2411, 393, 1521, -335, 369, -1250, -161, -1611, -597, -1748, -1392, -2189, -1844, -2479, -2205, -2638, -2914, -3052, -3164, -2915, -2785, -2222, -2470, -1957, -2308, -1724, -1962, -1137, -1414, -827, -1138, -535, -620, 262, 627, 1151, 1495, 1738, 1204, 1795, 946, 1595, 1277, 1768, 1737, 1980, 1913, 1742, 1587, 1462, 1000, 802, 584, -191, 457, -443, 613, -232, 741, -791, 1021, -1352, 1524, -1310, 1510, -1363, 1381, -1400, 2064, -828, 2617, -203, 2465, -11, 2519, -6, 2557, 104, 2303, 578, 2267, 1172, 2009, 1641, 1456, 1889, 962, 1485, 402, 757, -124, 428, -488, 395, -1039, -41, -1577, -678, -1913, -1324, -2513, -2129, -3391, -2800, -3902, -3159, -3861, -3415, -3463, -3397, -2896, -2819, -2450, -2162, -2269, -1905, -1948, -1767, -1363, -1397, -1004, -867, -578, -152, 460, 761, 1394, 1666, 1741, 2027, 1988, 1952, 2067, 1861, 2087, 1811, 2347, 1610, 2367, 1386, 2211, 1377, 2278, 1344, 2258, 800, 2026, 332, 1980, 195, 2066, 36, 2134, 98, 2385, 685, 2820, 1129, 2898, 1095, 2511, 1075, 2590, 1283, 3015, 1465, 2589, 1456, 1802, 1029, 1573, 432, 1291, 238, 805, 298, 623, 66, 356, -408, -478, -852, -1465, -1479, -2088, -2090, -2141, -2022, -1989, -1751, -2343, -2153, -3078, -2973, -3457, -3331, -3282, -3359, -3001, -3084, -2937, -2826, -2651, -2905, -2042, -2543, -1580, -1551, -1283, -883, -575, -628, 428, -4, 1491, 1093, 2368, 2074, 2859, 2600, 3173, 2864, 3532, 3194, 4036, 3281, 4541, 3493, 4583, 3844, 4097, 3315, 3497, 2216, 3048, 1952, 3122, 2063, 3312, 1710, 2772, 1244, 2054, 889, 1781, 421, 1706, 169, 1796, 375, 1764, 432, 1273, 208, 790, -87, 394, -624, -179, -1451, -283, -1601, -150, -1022, -754, -1073, -1616, -1945, -2346, -2648, -3020, -2867, -3256, -2853, -3346, -2843, -3734, -2892, -3932, -2900, -3985, -3083, -4205, -3230, -4072, -3082, -3693, -2981, -3175, -2670, -2571, -1801, -1777, -1019, -744, -484, 400, 295, 1122, 1095, 1640, 1772, 2532, 2435, 3610, 2863, 4126, 3171, 4141, 3432, 4350, 3333, 4523, 2866, 4372, 2566, 4365, 2378, 4098, 1805, 3432, 1146, 3256, 1029, 2963, 674, 2154, -295, 1889, -635, 1911, -409, 1499, -757, 1246, -1285, 1064, -1354, 852, -1096, 963, -909, 1043, -874, 1114, -490, 1345, 193, 952, 98, 215, -263, -90, 86, -158, 324, -341, 155, -945, 55, -1621, -304, -1754, -700, -1942, -631, -2445, -852, -2382, -1361, -2021, -1422, -2106, -1551, -2294, -2046, -2314, -2325, -1843, -2247, -1203, -2055, -1128, -1942, -889, -2252, -312, -2569, 42, -1966, 804, -961, 1802, -424, 2164, -148, 2491, 242, 3170, 718, 3543, 1335, 3927, 2068, 4565, 2873, 4801, 3471, 4667, 3519, 4329, 3176, 3708, 3001, 3219, 2899, 2880, 2474, 2174, 1879, 1313, 1377, 591, 742, -119, -94, -773, -605, -1139, -876, -1522, -1268, -1857, -1425, -2164, -1537, -2498, -1936, -2531, -2186, -2406, -2192, -2474, -2122, -2288, -1661, -1922, -1425, -2211, -1921, -2598, -2279, -2330, -2207, -2070, -1951, -2145, -1655, -2165, -1597, -2001, -1481, -1745, -1213, -1503, -1120, -1346, -840, -945, -505, -487, -553, -249, -359, 358, 275, 1330, 817, 2232, 1420, 2760, 1975, 3297, 2081, 4161, 2379, 4854, 2803, 4822, 3033, 4755, 3299, 4892, 3128, 4562, 2501, 4015, 2360, 3615, 2112, 2964, 1308, 2060, 499, 1003, -525, -19, -1526, -631, -1833, -1008, -2028, -1533, -2556, -2049, -2951, -2617, -3028, -2917, -2784, -2354, -2245, -1828, -1936, -2041, -1860, -2094, -1503, -1685, -808, -1276, -475, -926, -566, -550, 61, -209, 747, 177, 567, 258, 374, 247, 509, 396, 381, 373, 465, -68, 474, -102, -78, 316, -302, 308, -225, 7, -514, 273, -438, 842, 19, 1371, 305, 1963, 510, 2195, 288, 2390, 195, 3105, 936, 3766, 1559, 4095, 1685, 4272, 1882, 4012, 1962, 3877, 1942, 4287, 2417, 4271, 2917, 3709, 2738, 3200, 2220, 2547, 2045, 1502, 1602, 597, 705, 247, 327, -199, 103, -1328, -833, -2438, -1645, -2839, -2035, -2838, -2516, -3063, -2761, -3146, -2835, -2549, -2688, -1712, -1897, -1780, -1644, -1963, -2030, -1416, -1627, -866, -1038, -543, -1030, 20, -592, 367, 206, 517, 420, 649, 100, 565, -82, 549, 308, 734, 460, 982, 179, 1184, 390, 1021, 661, 610, 264, 422, -66, 622, -123, 1004, -169, 1367, 126, 1500, 518, 1724, 724, 2165, 784, 2485, 803, 2481, 851, 2507, 809, 2778, 933, 2802, 1202, 2422, 1096, 2283, 646, 2023, 279, 1383, -12, 974, -337, 388, -691, -525, -1114, -855, -1335, -1047, -1611, -1727, -2092, -2184, -2478, -2586)
<type 'str'>
\86a\EAO    (\CF
\8A\FA\D5\9ABG\99

Traceback (most recent call last):
  File "myprogram.py", line 409, in listen
    matrix = calculate_levels(matrix, weighting, data, args.chunk, args.sample_rate)
  File "fft.py", line 17, in calculate_levels
    data = unpack("%dh"%(len(data)/2),data)
struct.error: unpack requires a string argument of length 28

Furthermore, the sound from the microphone is recorded and written to test.wav, but there is no output while it is recording. After all, I'd be satisfied without the sound output. (There I'd split the sound coming from the microphone before it enters the sound card)

Answer 1

Seems the problem is in data parsing. By given output the data is not exactly "string". If it contains non-parseable symbols the exception may happen. I suggest to experiment with data types: parse it as repr, convert to unicode or other format before unpack, eliminate non-parseable symbols... To find non-parseable symbols you may use loop, giving to unpack every single symbol from data till failure. If I got it correctly, symbols are divided by slash / HTH.

Answer 2

I've made microphone FFT. If you need it yet just ask