- "Hand Talk" is already a commonly-used name for Plains Indian Sign Language, and so search results for the Hand Talk app end up burying or displacing search results for this endangered indigenous sign language.
- Hand Talk's translations are according to multiple reviews absolutely fucking dogshit, so dogshit that it would in fact seem like no proficient signers were actually involved in its development.
When the app boasts about being powered by "artificial intelligence", and its website has a whole page titled "for your company", I can't exactly say that I'm surprised. It really seems like this is just some tech startup trying to make big promises about accessibility and AI in order to win investors, reminding me of much of the technology shown in this video — while Hand Talk's main purpose in practice is probably just to allow corpos to cut costs on accessibility by "letting the AI do it".
It becomes quite striking, then, that on the "for you" page of their official website, that all five reviews that they choose to highlight from regular everyday people come from hearing people with no prior knowledge of Deaf culture or SL, rather than actual Deaf people talking about how the app has helped them; that none of the awards they highlight on the "about us" page come from Deaf organizations, and only one of the awards is actually related to accessibility; and that none of the three founders of the startup have any apparent background in Deaf culture whatsoever... Yes, I am sure that "advertising graduate and strategic communication specialist elected by Forbes as one of the most promising young people in Brazil" decided to found a sign language translation app Out Of The Goodness Of His Heart.
But hey, it's not like I'm proficient in any sign languages myself, and it's not like I've actually given the app a try, so maybe I'm just being presumptive from just a few bad reviews and a general "gross tech startup" vibe that this app is shittier than it really is. If any Deaf people have any positive experiences with this app, or with any apps like it, then I'd quite like to know your thoughts.
What stood out to me about the Hand Talk app was how it uses animated characters, because this reminded me of my own idea for SLiki, a sign language wiki made up of collaborative character animations. The difference is that Hand Talk presents itself as a translation tool, whereas SLiki would assume its users to already be proficient in the sign language in question; and while Hand Talk has a closed set of signs that it knows, and fingerspells everything else, SLiki would allow users to freely modify and crowdsource signs. And also SLiki would probably use Reimu and Marisa or some shit as its default models instead of these generic corporate art style characters that Hand Talk uses, and SLiki would be FOSS that would actually, y'know, try to incorporate its target demographic into the development process.
Once again, If Only I Knew How To Code.
I don't know any sign languages but I'm interested in linguistics and I can assure you that sign languages aren't "speaking in your dialect except with your hands".
Sign languages are independent languages and the grammar is not the same as its spoken counterpart. It's not a 1:1 for the same local language and anyone who speaks a second language knows that you can't just transpose your native language into another directly.
Not knowing a thing about this app, I can almost certainly guess that the techbros who designed this have no familiarity with linguistics and so they've taken a selection of the most commonly used words in English and copied them into ASL signs, thus the syntax is all janky and borderline incomprehensible.
For a rough approximation of what I'm talking about with janky syntax like this, here's a 1:1 Japanese to English direct transposition:
"ASL is doesn't speak however language immense is. Bothersome."
You can probably get a sense of what I'm saying but you're not going to understand what I'm actually saying (I guess unless you can figure out the Japanese words I'm referring to). A direct translation is going to give you impressions but it's really insufficient for clarity of communication and it's not a tool for conversation.
The only situation that I can imagine this being of some use would be where you are trying to communicate with a deaf person and one of the two of you has low/no literacy.
I don't know that much of any sign language, but I do know that "What's your name?" in ASL is "YOUR NAME WHAT", and it seemed in one of the videos I saw of this app that this was indeed what the machine spat out. On Hand Talk's official website's front page, there's also a video of the one character signing "HELLO ME fs-H-U-G-O", but the fingerspelling looked a little questionable, as the O was clearly turned. Turning your O's and C's as far as I understand is considered to be a beginner mistake for ASL learners, and is taken as a sign (no pun intended) that you learned fingerspelling from looking at a chart of the manual alphabet — where the O's and C's are commonly turned to the side to more clearly show the handshape — rather than from actually using the language around L1 signers who would immediately notice your mistake and tell you, "You're gonna get carpal tunnel if you keep doing that.".
The specific reviews I read and saw about the bad translation of the Hand Talk app mentioned specifically that the machine inappropriately used the sign KILL (sorry, "UNALIVE") in a sentence relating to picking someone up from the mall; and the machine would fingerspell many content words even when there are established signs for those things. Another promotional video showed the character using the directional verb HELP in a way that seemed appropriate, but I can't speak for the other signs in that sentence.
So I don't know if it really is as simple as just taking ASL signs and putting them in English word order, or if the syntax is a little fixed up "for the cameras" but otherwise is unreliable. Machine translation of sign languages is in any case very complex. It is difficult to imagine that a mobile app could know how to properly navigate all the different articulators which simultaneously contribute to meaning in sign language, or the cultural nuances and the ways of phrasing things that L1 signers would actually use.