如何将 Speech-to-Text 添加到使用 React Native 编写的聊天应用程序中

Question

My friends and I are working on a project that is a chat application that will implement Speech-To-Text, Text-to-Speech, and Translation APIs of Google. Gifted Chat and Firebase on the chat application. Chat App is working well with Firebase. We added TTS on it and it is also working well but we can't add the STT. We aim that users can use a microphone and the app can convert that speech into text. This text will automatically appear on the text box of the user. We believe that we must manually add STT to Gifted Chats modules but we don't know how to do it. There is also no source on the Internet about that. We will be so happy if anyone can help us. Thank you!

Answer 1

You can use react-native-voice library

Here's the example usage:

import Voice from '@react-native-community/voice';
import React, {Component} from 'react';

class VoiceTest extends Component {
  constructor(props) {
    Voice.onSpeechStart = this.onSpeechStartHandler.bind(this);
    Voice.onSpeechEnd = this.onSpeechEndHandler.bind(this);
    Voice.onSpeechResults = this.onSpeechResultsHandler.bind(this);
  }
  onStartButtonPress(e){
    Voice.start('en-US');
  }
  ...
}

Answer 2

Use react-native-voice library for the speech recognition

There is a text and onInputTextChanged prop in Gifted Chat that can help you handle the speech result appearing in the text box.

import Voice from '@react-native-community/voice';
import React, { useState, useEffect } from 'react';
import { GiftedChat, Composer } from 'react-native-gifted-chat';

const VoiceTest = () => {
  const [speech, setSpeech] = useState('');

  useEffect(() => {
    Voice.onSpeechStart = onSpeechStart;
    Voice.onSpeechEnd = onSpeechEnd;
    Voice.onSpeechError = onSpeechError;
    Voice.onSpeechResults = onSpeechResults;
    Voice.onSpeechPartialResults = onSpeechPartialResults;
  
  return () => Voice.destroy().then(Voice.removeAllListeners)
}, [])

  const onSpeechStart = () => {
  ...
  }

  const onSpeechEnd = () => {
  ...
  }

  const onSpeechError = () => {
  ...
  }

  const onSpeechResults = (e) => {
  ...
  setSpeech(e.value[0])
  }

  const onSpeechPartialResults = (e) => {
  ...
  setSpeech(e.value[0])
  }

  const startListening = () => {
  ...
  // You can set the locale to any language you want it to recognize, I am using Nigerian English.
  Voice.start('en-NG')
  }

  const renderComposer = (props) => {
  // Adds a Mic Button in the text box, you can style it as you want
  return (
    <View style={{ flexDirection: 'row' }}>
     <Composer {...props} />
     <MicrophoneButton onPress={startListening} />
    </View>
   )
  }

  return (
    ...
    <GiftedChat 
      renderComposer={renderComposer}
      text={speech}
      onInputTextChanged={setSpeech} 
     />
    ...
  )
 ...