eleven

For this activity, we add another dimension to the image processing and add some audio into the mix.  The main goal of this activity is, well, to play music from a music sheet – a task that I can surprisingly do when equipped with a laptop and the right software. 

Before any tunes can be heard though, a music sheet had to be obtained.  I chose Rain, Rain Go Away because, although I love the soothing sound of rain, the recent string of typhoons actually made me miss school.

Figure 1. Music sheet obtained from [1].

I cropped out one bar and the selection process wasn’t that hard since there;s only one bar to the whole song.  In the same way, a musical note was selected.  Shown below are the binarized and log-scale Fourier Transforms of these two (Figures 2 and 3).

Figure 2. Cropped musical sheet shown in its (top) original and (middle) binarized form along with its(bottom) log-scale Fourier Transform.

Figure 3. Musical note shown in its (top) original and (middle) binarized form along with its(bottom) log-scale Fourier Transform.

I was naïve enough to think that simply correlating the two images would be enough for me to isolate the individual notes in Figure 2.  Life isn’t like that though, as you can see from the first image in Figure 4, it looks like something I would see without my glasses (background info: I have really bad eyesight).  It’s out of focus, migraine-inducing and, I thought so at the time, depressing.

But we must forge on!  I used the binarized correlation image as a mask and multiplied this with the correlation matrix.  Using edge detection, the peak location of each note was found but for better visualization, a convolution with a circle was done.

Figure 4. Summary of image processing. (top) correlation, (middle) filtering and (bottom) peak identification

To know what note should be played when, the x- and y-values relative to the first note are obtained.  The x-values can tell when the note should be played and the y-value, the specific note to be played. 

Finally, here’s the finished product.  I saved the file twice for two reasons.  First, because I’m not sure the first go lets you hear enough of the song.  Second, because in my head the lyrics are “Rain, rain, go away, come again another day.  Rain, rain, go away, little children want to play.”  My mother or my kindergarten teacher must not have taught me right – or I must not have remembered the right words.  Or, those could actually be the right words.  None the less, enjoy this ten-second computer-generated rendition of Rain, Rain Go Away.

Figure 5. Scilab generated music file of Rain, Rain Go Away

I shouldn’t have been surprised that Scilab can play audio music.  It’s slightly infuriating that I had piano lessons and came out of them, not fully knowing how to read notes or play music from a music sheet but that with technology, I can.  I should feel empowered by this added modern musical appendage, but I don’t.  I can’t help but mourn for the money that went into all those lessons when I could have just taken up a computer class to learn Scilab since the end results are pretty much the same.

With a smile, I have to say that I’ve only had to press the Upload button on my YouTube account homepage for this class.  Doesn’t that make you feel all giddy inside? Another 9/10 for me for finishing the activity but not doing so on time.

REFERENCES:
1. “Rain Rain Go Away Sheet Music”, http://www.gmajormusictheory.org/Freebies/FirstPieces/0Rainrai.pdf