I'm studying tensorflow speech command sample. The Android codebase I use is the same on tensorflow GitHub android sample and mainly focus on SpeechActivity.java and RecognizeCommands.java . I didn't change anything except logging messages.
As far as I know,
(1) SpeechActivity.java will pass model referred result ( outputScores
) and currentTime
to recognizeCommands.processLatestResults
to do posterior smooth.
// Run the model.
inferenceInterface.feed(SAMPLE_RATE_NAME, sampleRateList);
inferenceInterface.feed(INPUT_DATA_NAME, floatInputBuffer, RECORDING_LENGTH, 1);
inferenceInterface.run(outputScoresNames);
inferenceInterface.fetch(OUTPUT_SCORES_NAME, outputScores);
// Use the smoother to figure out if we've had a real recognition event.
long currentTime = System.currentTimeMillis();
final RecognizeCommands.RecognitionResult result =
recognizeCommands.processLatestResults(outputScores, currentTime);
(2) in ProcessLatestResults(), previousResults
is used to store output score which was inferred in recent 500 ms ( averageWindowDurationMs
== 500) and averageScores
would be the final score we want/use in the future.
// Add the latest results to the head of the queue.
previousResults.addLast(new Pair<Long, float[]>(currentTimeMS, currentResults));
// Prune any earlier results that are too old for the averaging window.
final long timeLimit = currentTimeMS - averageWindowDurationMs;
while (previousResults.getFirst().first < timeLimit) {
previousResults.removeFirst();
}
...
// Calculate the average score across all the results in the window.
float[] averageScores = new float[labelsCount];
for (Pair<Long, float[]> previousResult : previousResults) {
final float[] scoresTensor = previousResult.second;
int i = 0;
while (i < scoresTensor.length) {
averageScores[i] += scoresTensor[i] / howManyResults;
++i;
}
}
My problems/questions are
(1) While the for loop is calculating the average, previousResult.second
value read from each item is identical. However, it is not possible. My question is did I miss something in logging information so that print out the wrong previousResult.second
value? or those scores array is truly identical? Here is how I log:
Log.d("tmp", "start average");
// Calculate the average score across all the results in the window.
float[] averageScores = new float[labelsCount];
for (Pair<Long, float[]> previousResult : previousResults) {
final float[] scoresTensor = previousResult.second;
Log.d("tmp", "previousResult("+previousResult.first+"): ["+Arrays.toString(previousResult.second)+"]" );
int i = 0;
while (i < scoresTensor.length) {
averageScores[i] += scoresTensor[i] / howManyResults;
++i;
}
}
and here is the log message during average loop process twice. In the first time, the scores array in previousResults
are the same [0.16993265, 0.15456167, 0.027866788, 0.107177936, 0.12646474, 0.053816866, 0.082612425, 0.059116375, 0.038425073, 0.06992877, 0.033074524, 0.07702225]
and this is not possible.
start avarage
previousResult(1520998400247): [[0.16993265, 0.15456167, 0.027866788, 0.107177936, 0.12646474, 0.053816866, 0.082612425, 0.059116375, 0.038425073, 0.06992877, 0.033074524, 0.07702225]]
previousResult(1520998400301): [[0.16993265, 0.15456167, 0.027866788, 0.107177936, 0.12646474, 0.053816866, 0.082612425, 0.059116375, 0.038425073, 0.06992877, 0.033074524, 0.07702225]]
previousResult(1520998400354): [[0.16993265, 0.15456167, 0.027866788, 0.107177936, 0.12646474, 0.053816866, 0.082612425, 0.059116375, 0.038425073, 0.06992877, 0.033074524, 0.07702225]]
previousResult(1520998400408): [[0.16993265, 0.15456167, 0.027866788, 0.107177936, 0.12646474, 0.053816866, 0.082612425, 0.059116375, 0.038425073, 0.06992877, 0.033074524, 0.07702225]]
previousResult(1520998400466): [[0.16993265, 0.15456167, 0.027866788, 0.107177936, 0.12646474, 0.053816866, 0.082612425, 0.059116375, 0.038425073, 0.06992877, 0.033074524, 0.07702225]]
previousResult(1520998400520): [[0.16993265, 0.15456167, 0.027866788, 0.107177936, 0.12646474, 0.053816866, 0.082612425, 0.059116375, 0.038425073, 0.06992877, 0.033074524, 0.07702225]]
previousResult(1520998400574): [[0.16993265, 0.15456167, 0.027866788, 0.107177936, 0.12646474, 0.053816866, 0.082612425, 0.059116375, 0.038425073, 0.06992877, 0.033074524, 0.07702225]]
previousResult(1520998400629): [[0.16993265, 0.15456167, 0.027866788, 0.107177936, 0.12646474, 0.053816866, 0.082612425, 0.059116375, 0.038425073, 0.06992877, 0.033074524, 0.07702225]]
previousResult(1520998400683): [[0.16993265, 0.15456167, 0.027866788, 0.107177936, 0.12646474, 0.053816866, 0.082612425, 0.059116375, 0.038425073, 0.06992877, 0.033074524, 0.07702225]]
previousResult(1520998400737): [[0.16993265, 0.15456167, 0.027866788, 0.107177936, 0.12646474, 0.053816866, 0.082612425, 0.059116375, 0.038425073, 0.06992877, 0.033074524, 0.07702225]]
....
....
start average
previousResult(1520998400301): [[0.14775836, 0.18298364, 0.026629224, 0.12195902, 0.111195154, 0.058891248, 0.07295453, 0.05453651, 0.04063993, 0.06559348, 0.032576166, 0.084282786]]
previousResult(1520998400354): [[0.14775836, 0.18298364, 0.026629224, 0.12195902, 0.111195154, 0.058891248, 0.07295453, 0.05453651, 0.04063993, 0.06559348, 0.032576166, 0.084282786]]
previousResult(1520998400408): [[0.14775836, 0.18298364, 0.026629224, 0.12195902, 0.111195154, 0.058891248, 0.07295453, 0.05453651, 0.04063993, 0.06559348, 0.032576166, 0.084282786]]
previousResult(1520998400466): [[0.14775836, 0.18298364, 0.026629224, 0.12195902, 0.111195154, 0.058891248, 0.07295453, 0.05453651, 0.04063993, 0.06559348, 0.032576166, 0.084282786]]
previousResult(1520998400520): [[0.14775836, 0.18298364, 0.026629224, 0.12195902, 0.111195154, 0.058891248, 0.07295453, 0.05453651, 0.04063993, 0.06559348, 0.032576166, 0.084282786]]
previousResult(1520998400574): [[0.14775836, 0.18298364, 0.026629224, 0.12195902, 0.111195154, 0.058891248, 0.07295453, 0.05453651, 0.04063993, 0.06559348, 0.032576166, 0.084282786]]
previousResult(1520998400629): [[0.14775836, 0.18298364, 0.026629224, 0.12195902, 0.111195154, 0.058891248, 0.07295453, 0.05453651, 0.04063993, 0.06559348, 0.032576166, 0.084282786]]
previousResult(1520998400683): [[0.14775836, 0.18298364, 0.026629224, 0.12195902, 0.111195154, 0.058891248, 0.07295453, 0.05453651, 0.04063993, 0.06559348, 0.032576166, 0.084282786]]
previousResult(1520998400737): [[0.14775836, 0.18298364, 0.026629224, 0.12195902, 0.111195154, 0.058891248, 0.07295453, 0.05453651, 0.04063993, 0.06559348, 0.032576166, 0.084282786]]
previousResult(1520998400791): [[0.14775836, 0.18298364, 0.026629224, 0.12195902, 0.111195154, 0.058891248, 0.07295453, 0.05453651, 0.04063993, 0.06559348, 0.032576166, 0.084282786]]
...
...
(2) According to the message log, you can see that the scores array corresponding to 1520998400301
at first time is [0.16993265, 0.15456167, 0.027866788, 0.107177936, 0.12646474, 0.053816866, 0.082612425, 0.059116375, 0.038425073, 0.06992877, 0.033074524, 0.07702225]
, but in the next time, the scores array become [0.14775836, 0.18298364, 0.026629224, 0.12195902, 0.111195154, 0.058891248, 0.07295453, 0.05453651, 0.04063993, 0.06559348, 0.032576166, 0.084282786]
My second problem is that I don't know how could this happened. My code is identical with RecognizeCommands.java . Any clues or advice would be very helpful, thank you.
Try to change this code:
// Add the latest results to the head of the queue.
previousResults.addLast(new Pair<Long, float[]>(currentTimeMS, currentResults));
As following:
// Add the latest results to the head of the queue.
previousResults.addLast(new Pair<Long, float[]>(currentTimeMS,
Arrays.copyOf(currentResults, currentResults.length)));
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.