SpeechRecognition to TextToSpeech Ionic2 Android application

Ionic Framework ionic-v3
1

Hello,

I’m trying to experiment with:

import { TextToSpeech } from '@ionic-native/text-to-speech';

and

import { SpeechRecognition, SpeechRecognitionListeningOptionsAndroid, SpeechRecognitionListeningOptionsIOS } from '@ionic-native/speech-recognition';

but I can figure out, how to pronounce recognition result with tts in Android device with Ionic2 application after recognition. This way I have text-to-speech result only with second request to listening, before I start to speech, if I use this.sayText(); after this.speech.startListening :

  listenForSpeech(): void {
    this.androidOptions = {
      prompt: 'Speak into your phone' 
    }
    if (this.platform.is('android')) {
      this.speech.startListening(this.androidOptions).subscribe(data => this.speechList = data, error => console.log(error));

      this.sayText();
    } 
  }

for example with random one from array:

  async sayText(){
   var rand = this.speechList[Math.floor(Math.random() * this.speechList.length)];
    this.tts.speak(rand)
    .then(() => console.log('Success'))
    .catch((reason: any) => console.log(reason));
  }
2

Hi Lado,

Correct me if I’m wrong, but I think you’re trying to convert speech into a text array and then have your device say one of the results at random. If that’s the case, then you need to put your sayText method call into the success callback for the speech.startListening method. It should look something like this:

listenForSpeech(): void {
    this.androidOptions = {
      prompt: 'Speak into your phone'
    }
    if (this.platform.is('android')) {
      this.speech.startListening(this.androidOptions).subscribe(
        (data) => {
          this.speechList = data;
          this.sayText();
        }, error => console.log(error));


    }
  }

The speech.startListening method is asynchronous, so the sayText method needs to wait for the success callback method to be executed before it can evaluate the contents of the speechList array.

I’ve posted a working sample project with this feature in github:

1 Like
3

@davidplummer

Hello

Thank you for your answer, very helpful

Also I’m trying to figure out, first with SpeechRecognition, what variants I have to setup popup window, for example if I can avoid popup window, or somehow make recognition permanently on?

With TextToSpeech I’m trying to find, what is available for developer. For example, if I can use something like viseme event, I’m understand, it can’t be event in this case, but maybe some different method, if it is exists with google tts at all, as well as, speech rate etc.

4

I know the SpeechRecognition plugin allows you to specify the title for the popup by passing a string argument to the prompt parameter when calling the startListening method. You can also hide the popup by setting showPopup to false when you call startListening (this is hidden by default for iOS).

This github repo fleshes out the documentation for the SpeechRecognition plugin.

Here’s the repo for the TextToSpeech plugin shown in the Ionic Framework API documentation. It explains the different methods and arguments you can utilize for the plugin, like changing the talking speed and accent of the API.

1 Like
5

@davidplummer

Thank you, very useful links

but still I can’t figure out, how to display result simultaneously with recognition, for example if this.sayText(); contains rand: string; variable:

  async sayText() {
   this.rand = this.speechList[Math.floor(Math.random() * this.speechList.length)];
    this.tts.speak(this.rand)
    .then(() => console.log('Success'))
    .catch((reason: any) => console.log(reason));
  }

from:

 listenForSpeech(): void {
    this.androidOptions = {
      showPopup: false
    }
    if (this.platform.is('android')) {
      this.speech.startListening(this.androidOptions).subscribe(
        (data) => {
          this.speechList = data;
          this.sayText();
        }, error => console.log(error));
    }
  }

how to display this output with html, which shows result with second startListening request by previous data, same way as it was with tts:

<ion-header>
  <ion-navbar>
    <ion-title>
      Test
    </ion-title>
  </ion-navbar>
</ion-header>
<ion-content padding>
  <ion-card text-center>
    <ion-card-content>
      {{ rand }}
    </ion-card-content>
  </ion-card>
  <button full ion-button (click)="listenForSpeech()">Listen</button>
</ion-content>
6

Hi Lado,

I can’t reproduce your error. Can you show me the code for the entire controller and template you’re working with?

7

@davidplummer

I’ve edited last reply, this is all the code. I’m just trying to show one of recognition results with html page. Now this.sayText(); located into the success callback for the speech.startListening method and assigns variable with this.rand = this.speechList[Math.floor(Math.random() * this.speechList.length)];, but html <ion-card-content> {{ rand }} </ion-card-content> applies value only with second request with previous data

8

Hi Lado,

I was able to resolve your problem. Because sayText is being called asynchronously, the UI update for the rand text doesn’t occur. You can trigger this manually by using NgZone. This works for me.

  1. Import the NgZone module.
import { NgZone } from "@angular/core"
  1. Make a class instance of NgZone.
constructor(public ngz: NgZone){ }
  1. Call the statement block in your sayText method from inside the NgZone callback.
async sayText() {
this.ngz.run(()=>{
   this.rand = this.speechList[Math.floor(Math.random() * this.speechList.length)];
    this.tts.speak(this.rand)
    .then(() => console.log('Success'))
    .catch((reason: any) => console.log(reason));
  }
})
1 Like