Problem: Modern games lack an immersive feel due to unrealistic interfaces. Controllers with joysticks and buttons do not produce a life-like environment. This makes it harder to transport oneself into the game world, reducing realism.
Solution: Utilizing the tracking capabilities of a WiiMote and the Xbox Kinect, design a game controller interface that relies on the human body as much as possible to increase realism in a First Person Shooter (FPS). Specifically, the WiiMote will be used to aim, the WiiMote Nunchuck's joystick will move the camera and the WiiMote's D-Pad will move the game character. The Xbox Kinect will track motions such as reloading, switching weapons, throwing a grenade, jumping, using a melee weapon and possibly headtracking. Finally, a wearable tactile feedback suit will vibrate in specific locations where the player has been shot. Ideally, the vest will consist of 26 vibrational (or other feedback, should we find a better solution) pads: A pair on each shin/calf, quad/hamstring, left lower back/left lower ab, right lower back/right lower ab, left upper back/left chest, right upper back/right chest, each tricep/bicep, the front and back of the forearm and a final pair on the back/front of the head. We reserve the possibility of reducing the pad count. This should provide a game interface that immerses the player into the virtual world, helping him/her believe they are part of the game. For the purpose of this project, we will create our own first person shooter game which incorporates the aforementioned interface. Towards the end of the semester we will survey the game experience by inviting people from within the CMU community to play our game. Possible extensions of the project, if time permits: Porting parts of our entire interface to existing games or game API's, using biofeedback to further enhance the gaming experience (reference: http://valvesoftware.com/publications/2011/ValveBiofeedback-Ambinder.pdf), coupling the project with research done on the field of omnidirectional treadmills.
Tentative Timeline: 1/27 Choose a game engine to code our FPS Choose a vibration motor for the vest Find drivers for WiiMote and Kinect to operate on a PC 2/29 Playable FPS with keyboard controls to move, shoot, crouch and lay prone 3 vibration motors hooked up to the Arduino that can be made to vibrate through hardcoded #define. Recognize crouching and grenade throwing with the Kinect Use WiiMote as mouse on computer 3/31 Have the WiiMote and Kinect work with the game individually, but not necessarily in parallel Add gesture recognition for reloading, switching weapons and knifing/weapon bashing. Intermediate Report 4/30 Playable FPS with aiming by WiiMote and gesture control from the Kinect. FPS communicates with Arduino by telling it the number of the vibration motor to power 5/4 Polish existing capabilities Final report
Budget: Nunchuck for WiiMote 12$ WiiGun for plugging in the WiiMote 7$ Any materials necessary for the vest 60$ That would make a humble budget of 80$ for the main project. Let's round it up to 100$ for unexpected expenses.