Pianissimo

Team 12

Maung Aung

mza

Kevin Dou

khd

Peter Wei

pwei

Diana Zhu

dianaz

Concept

Pianissimo is a device to be placed placed on top of the piano, with no additional hardware needed and out of the way of the pianist. The device is preloaded with MIDI files and will use a tutor mode to teach songs. In tutor mode, the device runs through the song with the player by listening to the input of the notes currently being played and highlighting the next sequence of notes to be played via laser projection directly onto the piano keys.

Motivation

Learning to play the piano has a very expensive barrier of entry, with the cost of private lessons totaling an upwards of a hundred dollars per hour. As a result, self-taught musicians have been on the rise, supported by the prevalence of video tutorials on YouTube and other online resources. We want to create a product that aids the musician directly—through their instrument—in ways videos cannot. This device will be specifically designed for the piano and will attempt to teach any chosen piece of music to the fledgling musician.

Competitive Analysis

Projected Instrument Augmentation system (PIANO)

The PIANO system uses a large screen that is attached to a keyboard that shows upcoming notes and gives feedback by lighting up corresponding keys. It's designed to work with a specific keyboard, which means it isn't immediately usable on classical acoustic pianos, and it isn't very light or portable.

Pianissimo, on the other hand, is compatible with pianos and keyboards of many different shapes and sizes. It's low-impact; there's no need for a large screen to cover almost all of the piano or keyboard. And, it's compact and easy to set up.

Yamaha EZ-200 Lighted Keyboard

Lighted keyboards, such as the one above, are one option for the self-taught pianist. However, they are limited in the number of pre-programmed songs that they can teach and they require the additional investment of buying yet another keyboard that might not be used after all the songs are exhausted.

Pianissimo is a smaller, non-redundant investment that keeps giving back. It is an enhancement to any piano or keyboard that is capable of reading in any MIDI file, so the number of songs it can demonstrate and its potential to teach is virtually limitless.

Use Case: Play Song

Participating Actors
The use case is initiated by a player.

Brief Description
The use case allows the player to set up the device and play a basic song.

Pre-Condition
The player must have access to a piano.

Flow of Events:

Basic Flow

The use case starts with the player placing the device onto a piano and turning the system on.
The system asks the player to calibrate the device by projecting a laser beam onto four keys starting at the middle C key.
The player adjusts the laser beam so the projection is directly over the keys and confirms placement.
The system asks the player to select a song to practice.
The player chooses the song by pressing the select button.
The system then projects the first note of the song onto the piano and listens for input.
The player presses the projected note and the system proceeds to project the next sequence of notes.
When the song finishes, the system returns to song selection.

Alternate Flows

In step 1, if the player is using a keyboard, then the device is mounted onto the keyboard using a tripod. The use case resumes with step 2.
In step 5, if the player does not want to play the current song, they can select another by using the arrow keys. The use case resumes with step 6.
In step 6, if the player presses the wrong note, nothing happens and the system continues to listen for input.

Architecture

Functional Architecture

Our project is composed of two main systems that communicate with the user as well as with some preloaded MIDI files of music that the user wishes to learn or perform. The two main systems are the audio processor (acting as the main sensor module) and the laser projection system (acting as the main actuator module). The systems will act together to tutor a user in learning a piece of music.

For more information, please take a look at our Final Report.

Requirements

Functional

must be able to convert a MIDI file to a usable format
must be able to accurately identify the current notes being pressed on the piano
must be able to highlight keys on a piano or keyboard

Timing

must be able to detect the current notes being played within 350 milliseconds
must project the next sequence of keys within 250 milliseconds

Usability

must not interfere with the actual playing of the song or the pianist's ability
must not obstruct the view of the keys from the player
must not interfere with nor modify the sound from the piano or keyboard

Interaction Diagram

Hardware

Audio Input

Audio-Technica AT2005USB Cardioid Dynamic USB/XLR Microphone for recording notes played on the keyboard

Laser Projection

4-Pin PWM Fan to rotate the mirror that will reflect the laser bean onto the piano keys

Lightweight mirror for reflecting laser beam onto the piano keys

Laser module to illuminate piano keys and provide feedback

Miscellaneous

Raspberry Pi for central processing and laser projection control

700 MHz ARM1176JZF-S core CPU
512 MB RAM
Storage: MicroSD
Vendor: Amazon

Software

Note Detection

As the piano player plays a song, the system will recognize the notes being played and compare those notes to a MIDI file. The FFTW C subroutine library will give use the necessary tools for transforming audio signals to the frequency domain in order to perform this comparison.

Key Lighting

We will have exact control over the on or off state of the laser pointer in order to achieve precise lighting of the piano keys. As the motor rotates, the laser pointer will likewise be able to illuminate different keys in (pseudo) real-time.

Tutor Mode

This mode is useful for the piano player who has just picked up a new sonata or prelude. The set of keys to be played will remain illuminated until the correct keys have been pressed; once they have been pressed, the system will move on to the next set of keys to be played.

Protocols

Inputs

The microphone will give audio input to the processor via USB.

Internal

The microphone input will be compared with the MIDI file and the system will output a key map that corresponds to the physical piano keys and be given to the laser projection system.

Outputs

The laser pointer will be controlled by the processor.

Interface

In order to support quick development and design with modularity in mind, there will be a common interface between note detection and the laser projection output.

PIANISSIMO

THE PIANO ASSISTANT