使用 java 和 twilio 将 Google 语音转为文本

Question

I am having problem while converting audio file to text using google speech to text. I am able to download the file from Twilio but when I supply that audio file to google speech then it gives me 0 length response. But if I convert this downloaded file using vlc media player and then supply it to google speech then it gives me right output. Please help me on this I am stuck for about a week now.

After getting response from Twilio I save it in a file with.wav extension

InputStream in = new URL(jsonObject.get("redirect_to").toString()).openStream();
Files.copy(in, Paths.get("src/main/resources/mp.wav"), StandardCopyOption.REPLACE_EXISTING);

Below is the google speech to text code.

Path path = Paths.get("src/main/resources/mp.wav");
        byte[] content = Files.readAllBytes(path);
        ByteString audioBytes = ByteString.copyFrom(content);

        try (SpeechClient speech = SpeechClient.create()) {
            RecognitionConfig recConfig =
                    RecognitionConfig.newBuilder()
                            .setEncoding(RecognitionConfig.AudioEncoding.LINEAR16)
                            .setLanguageCode("en-US")
                            .setSampleRateHertz(44100)
                            .setModel("default")
                            .setAudioChannelCount(2)
                            .build();


            RecognitionAudio recognitionAudio = RecognitionAudio.newBuilder().setContent(audioBytes).build();

            OperationFuture<LongRunningRecognizeResponse, LongRunningRecognizeMetadata> response =
                    speech.longRunningRecognizeAsync(recConfig, recognitionAudio);

            while (!response.isDone()) {
                System.out.println("Waiting for response...");
                Thread.sleep(10000);
            }

            List<SpeechRecognitionResult> results = response.get().getResultsList();

            for (SpeechRecognitionResult result : results) {
                // There can be several alternative transcripts for a given chunk of speech. Just use the
                // first (most likely) one here.
                SpeechRecognitionAlternative alternative = result.getAlternativesList().get(0);
                System.out.printf("Transcription: %s%n", alternative.getTranscript());
            }

        } catch (InterruptedException | ExecutionException e) {
            e.printStackTrace();
        }

Answer 1

As @philnash has suggested, by appending a .mp3 extension to the recording URL, the MP3 version of the recording can be downloaded from Twilio. The same applies to the '.wav' extension as well.

InputStream in = new URL(jsonObject.get(“redirect_to”).toString()+”.mp3”).openStream(); // or “.wav”
Files.copy(in, Paths.get(“src/main/resources/mp.wav”), StandardCopyOption.REPLACE_EXISTING);

I tested this out with a sample Twilio recording and the ffprobe results are below.

Downloaded .wav file

Input #0, **wav**, from 'from-twilio-change-extension.wav':
  Duration: 00:00:14.60, bitrate: 128 kb/s
    Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 8000 Hz, 1 channels, s16, 128 kb/s

Downloaded .mp3 file

Input #0, **mp3**, from 'from-twilio-change-extension.mp3':
  Duration: 00:00:14.68, start: 0.000000, bitrate: 32 kb/s
    Stream #0:0: Audio: mp3, 22050 Hz, mono, fltp, 32 kb/s

As for audio encodings supported by the Speech-to-Text API, both WAV and MP3 are supported but MP3 is a Beta feature available only in the version v1p1beta1 . So, the client library imports will look like com.google.cloud.speech.v1p1beta1.Packages... . The audio encoding in RecognitionConfig has to be changed according to the encoding of the audio file used. For a .wav file, RecognitionConfig.AudioEncoding.LINEAR16 has to be used, and for a .mp3 file, RecognitionConfig.AudioEncoding.MP3 has to be used.

---

An alternative would be to use the FFMPEG tool to convert audio files into one of the recognized codecs by Speech-to-Text. More information about usage of the tool can be found here . In your scenario, the .mka to .wav / .mp3 conversion can be done from the Java code as shown below.

String[] ffmpegCommand = {"ffmpeg", "-i", "/full/path/to/inputFile.mka", "/full/path/to/outputFile.wav"};

ProcessBuilder pb = new ProcessBuilder(ffmpegCommand);
pb.inheritIO();
pb.start();