An Android application about multi-pitch sound analysis using the fast fourier transformation.
Abstract: Mobile phones are spread around the world. The number of smart phone users increases massively. Statistical projections predict a number of 1.75 billion smart phones possessed by seven billion people on earth by the end of the year 2014, a way to reach many people is to program applications for the smart phone. A common way to interact to smart phones is to touch the screen. We tested another way of interaction, using environmental sound analysis. We developed a music game for Android phones to support students during the process of learning to play the piano. We assume the user to know the notation of notes and to play an instrument that complies with the theory of harmonics. Our application solves the frequency detection problem using the short time fourier transform. In a controlled comparison two prototypes based on differing game ideas were compared. A final prototype was implemented with the favoured functional elements of the controlled comparison experiment. In a cognitive walkthrough we tested the final prototype for usability issues. The results pointed out some issues with the memorability of the game concept. We achieved an average score of 71% for the memorability. High values were measured for the user satisfaction with an average value of 81%. In another experiment we measured the volume of noise in three environments (a silent room, outside, and close to a street). A mechanism to filter noisy signals was implemented based on the measurements. The application works best (with a hit rate of 100%) for instruments with a range of notes between D3 and E7. We reach a hit rate of 70% for the piano, whose pitch range starts at A1 and ends at G7. The main difference between our application and related research topics is, that our application works on low quality microphones and runs only on the limited resources provided by the phone. Our results demonstrate that the hardware of today’s mobile phones is already fast enough to perform time-consuming operations almost real-time. Even fast enough to give the user the impression of an instantaneous feed-back.