• Concept
• Motivation
• Competition
• Tech Specs
• Requirements
• Architecture
• Use Cases
• System States
• Risks
• Error Handling
• Implementation
• Test Cases
• Evaluation
• Lessons
• Fun Stuff
• References
• 18-549 Home

MEMBERS

• Bhargav Bhat
• Hemant Sikaria
• Jorge L. Meza
• Wesley Jin

PROJECT CONCEPT

HandTalk is a wireless glove designed to recognize basic American Sign Language (ASL) hand signs and voice them by interfacing with a Java enabled cellphone/PDA.

MOTIVATION

We feel that HandTalk could greatly aid the deaf more comfortably communicate with those unfamiliar with ASL.

HANDTALK V0.1 DEMO

Checkout our first end to end test for our glove. It shows the result for an input to our system from beginning to end. More features are yet to come.

• Demo1

COMPETITIVE ANALYSIS

KMI Embedded RF Finger System Similar to our system Uses RF to communicate Not very portable Demo video here
Metamotion Data Glove Very precise Very expensive Not portable Website
Berkeley Acceleration Sensing Glove Not portable Uses 3D Accelerometers Website

TECHNICAL SPECIFICATIONS

• Flex Sensor x 10 ($10 each): Jameco Electronics - Odeg:1OK
• Nokia N95
• BlueSentry Bluetooth Data Acquisition Module from Roving Networks

• Nuance RealSpeak Text to Speech Engine ($10)
• TALKS - for interim use

• Wireless: Bluetooth for glove to communicate with the cellphone/PDA

REQUIREMENTS

• Glove needs to recognize the ASL alphabet
• Glove needs to be portable and have low power consumption
• Glove needs to have a simple mechanism to switch between the ready and standby state
• System needs to recognize and voice sign within 2 seconds
• The support application on the cellphone/PDA needs to be compatible with most cellphones/PDAs currently on the market

ARCHITECTURE

USE CASES (INTERACTION DIAGRAMS)

SYSTEM STATES & TRANSITIONS

To be completed.

RISKS & MITIGATION STRATEGIES

Risks	Mitigation
Bluetooth is tough to implement	Module has complete implementation on-board, so there's no real coding required
Possible low resolution from the sensors + A2D	Use our own alphabet/gestures - possibly change application Expand the voltage range by looking at more complicated analog circuit setups - we're using a simple voltage divider
Fewer than ideal number of I/O pins (8 vs. 10+)	2 flex sensors for each of 3 fingers + 1 sensor for 2 fingers - we can only decide which through user testing Use 3D accelerometer + 1 flex sensor per finger - involves rethinking of application

ERROR HANDLING

To be completed.

IMPLEMENTATION DETAILS

To be completed.

TEST CASES

• Bluesentry was connected and we were streaming data when the connection was suddenly lost.
  - User should be notified and we should go back to the connections page.
• Bluesentry was connected but suddenly dies.
  - Notify user and stop program. User can press exit or start BlueSentry again and try reconnecting. Go back to Discoverer page.
• SD Card testing -
  • Tried to load a file that was in an invalid format/wrong file.
    Action - Die gracefully.
  • User forgets to save a file/overwriting a file? (currently throws exception)
  • Different dictionaries - same signature in two different libraries might mean separate things. The data outputted
    should not be corrupted.
  • Specifying file path - check the path. (have SD card path hardcoded and then just append user.
• User Input fields - trying to screw up all input fields. Check if bluetooth address is not the BlueSentry device.
• Learning interface - check if they are adding new words/conflicts and notify appropriately.
• START-STOP-START cycle of the connection handler with our bluetooth device should work properly.
• Red button of death - deal with finding the function that would be called in that scenario.
• Check for Memory Leaks

EXPERIMENTAL EVALUATION

To be completed.

LESSONS LEARNED

To be completed.

FUN STUFF

To be completed.

REFERENCES

• Project Proposal and Requirements, Team Project Presentation, February 1, 2008
• Design & Architecture, Team Project Presentation, February 15, 2008
• Mid-semester Project Status, Team Project Presentation, Date TBD
• Test Plan & Experimental Validation Team Project Presentation, March 28, 2008
• Final Project Presentation, Team Project Presentation, Date TBD
• Project Poster, May 2, 2008