Add Speech-To-Text commands for (mostly) hands-free play

[blamarche]

After trying this great app (thanks for making it open source) I was thinking this would be an amazing activity for long commutes or other activities where looking at/tapping on your device would be unwise. Is this something on the roadmap or has had any work been done to this end? If not, I may look into adding it if it would be merged in.

[onyxbits]

Am 26.12.2018 20:26, schrieb Brendan LaMarche:

After trying this great app (thanks for making it open source) I was thinking this would be an amazing activity for long commutes or other activities where looking at/tapping on your device would be unwise. Is this something on the roadmap or has had any work been done to this end? If not, I may look into adding it if it would be merged in. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#30>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAbpgRI4ccFC-3VRtvjBkr5cNvSoinTDks5u881UgaJpZM4ZiBqo>.

There are no plans for adding this as voice recognition can't be done (well) on the device itself (at least not on older devices). I also don't feel well about requesting microphone and network permission in order to ship your audio data over to Google or Amazon. Have you tried installing a voice enabled keyboard?

[ralphtrickey]

Out of curiosity, is that because of privacy concerns or battery/network usage? From my viewpoint, the privacy ship has sailed years ago.

I like the idea, but you'd have to flag a large area to the main screen to turn recognition on or off for a commute. I suspect that leaving it on constantly would drain the battery quickly as well as using up network bandwidth. Actually, you might be able to capture the button on a bluetooth headset to toggle the mic on and off with an inactivity timer.

My 2c,
Ralph

[onyxbits]

Am 26.12.2018 21:17, schrieb Ralph Trickey:

Out of curiosity, is that because of privacy concerns or battery/network usage? From my viewpoint, the privacy ship has sailed years ago. I like the idea, but you'd have to flag a large area to the main screen to turn recognition on or off for a commute. I suspect that leaving it on constantly would drain the battery quickly as well as using up network bandwidth. Actually, you might be able to capture the button on a bluetooth headset to toggle the mic on and off with an inactivity timer. My 2c, Ralph

Mostly privacy concerns, but also battery and bandwidth usage. Not to mention that this opens a can of worms for negative reviews on Play. Also: even if privacy was dead, I'm not one to stab the corpse with needles for good meassure.

[blamarche]

I can absolutely understand the concern with privacy!

For such a limited set of commands, I am fairly confident it could be done using a local STT engine and some common string comparison algos instead of uploading to a server first.

For example you'd rarely need to detect words other than the initial command followed by something likely existing within the past few text descriptions. With modern android permissions you don't need to ask for microphone permission until its enabled by the user. I'd imagine this setting would be off by default and just another checkbox next to the TTS option.

Since the primary use case example I proposed is a commute, I don't think battery life would be a major concern since you could have it charging and even potentially have the screen off entirely.

[onyxbits]

Am 26.12.2018 21:27, schrieb Brendan LaMarche:

I can absolutely understand the concern with privacy! For such a limited set of commands, I am fairly confident it could be done using a local STT engine and some common string comparison algos instead of uploading to a server first.

Sorry, no. People will compare the results of the local engine with what they are used to from Google Now. If the local engine underperforms (and it will), then I'll get poor reviews.

For example you'd rarely need to detect words other than the initial command followed by something likely existing within the past few text descriptions. With modern android permissions you don't need to ask for microphone permission until its enabled by the user. I'd imagine this setting would be off by default and just another checkbox next to the TTS option. Since the primary use case example I proposed is a commute, I don't think battery life would be a major concern since you could have it charging and even potentially have the screen off entirely.

Playing text adventures while driving a car? Sounds awfully distracting. I wouldn't really want to support that use case.

[blamarche]

It seems you are determined to shut this down, as you keep coming up with new reasons. Its your project, so do as you will, but can you really not think of other use cases?

How about for the vision or mobility impaired? The TTS is a good first step, but that doesn't help someone type easily and those voice keyboards don't have ease of use the advantage of app integration would provide (like I said, to be able to correct commands based on contents of previous descriptions, auto-listen for next command, etc).

I guess the only option then is to fork and make a separate app? That'd be a real shame.

PS: Voice command is one of the safest machine-interaction methods for drivers on the road. Far better than those touch-screen controls they are slapping into every vehicle console these days. With voice command, this app wouldn't be significantly more distracting than an audio book or podcast or even just a regular phone call.

[onyxbits]

Am 27.12.2018 18:56, schrieb Brendan LaMarche:

It seems you are determined to shut this down, as you keep coming up with new reasons. Its your project, so do as you will, but can you really not think of other use cases?

Let me put it this way, there are two things that are not going to happen: a) Making TF dependant on any external service (especially not one that requires API keys or is well known for privacy invasion). Since TF is free, I don't earn the money to pay for an external service and I won't let users pay with their privacy either. b) Using a local, low quality VR engine that fails half of the time and/or doesn't work out of the box (has to be trained by the user). Since TF is freem there isn't any money for marketing beyond begging users for reviews on Play. Having ratings tank because a feature doesn't work as advertised is a no-go.

I guess the only option then is to fork and make a separate app? That'd be a real shame.

Threatening a fork is not really a convincing argument. Showing a proof of concept would be: make a (standalone) demo app, one activity screen only (doesn't have to be fancy) that just listens to the user and prints what it understood to a ListView. If that works reliably (under the conditions mentioned above), we can talk.

[blamarche]

I mean no ill will towards you or this project, thats for sure. As I said before, I am thankful this project is open source!

To do the proof of concept you state adequately and have the local text recognition be accurate enough it would be just as simple for me to add it into to the actual TextFiction source, since it'll need the context of the fiction text itself. On its own, a local recognizer won't be good enough.

When I get some time, I will likely add this STT feature. If you won't merge it in, it'll be a fork with a separate listing in the app store, simple as that. Its not a threat, just a statement of fact.