twelve

Allow me to prepare you for disappointment.  I didn’t get far with this activity.  In my opinion, I didn’t go anywhere at all. 

This is all so ironic since this was actually an activity I was excited about.  When I learned we would be processing text from images, my mind immediately jumped to the infinite number of things I could do – create a typewritten of the notes I’d accumulated over the years, decipher the chicken scratch of most doctors, etc. etc.  That was a happier, simpler time, before I knew that the actual  implementation would make me want to throw dreams out the window.

Shown below is a document that an unidentified person with semi-illegible handwriting prepared.  The white spaces at the corners are present because of the rotation that the original image underwent to make the horizontal lines appear horizontal. 

Figure 1. Rotated image of a form that was filled out with handwriting. 

The handwritten text that I wanted to extract are shown below along with, what I can say, is the best I can do.  I’m not sure if it’s what I’m supposed to see or if I’m half there or nowhere near there. 

Figure 2.  Handwritten text to be processed.

Figure 3.  Results of the processing on the handwritten text.

So, I’ll just say what I did.  I binarized the image and removed the horizontal line using a mask.  Maybe the horizontal lines are too thick or maybe the mask I used was wrong.  Either way, I’ve spent too long on this activity.  A real scientist must know when something is beyond their capabilities.

I am aware that I gave up on this activity and that it’s late so I’m just giving myself a measly 7/10.

Sigh.

REFERENCES:
1. A12 – Preprocessing Text – Applied Physics 186 Manual.

eleven

For this activity, we add another dimension to the image processing and add some audio into the mix.  The main goal of this activity is, well, to play music from a music sheet – a task that I can surprisingly do when equipped with a laptop and the right software. 

Before any tunes can be heard though, a music sheet had to be obtained.  I chose Rain, Rain Go Away because, although I love the soothing sound of rain, the recent string of typhoons actually made me miss school.

Figure 1. Music sheet obtained from [1].

I cropped out one bar and the selection process wasn’t that hard since there;s only one bar to the whole song.  In the same way, a musical note was selected.  Shown below are the binarized and log-scale Fourier Transforms of these two (Figures 2 and 3).

Figure 2. Cropped musical sheet shown in its (top) original and (middle) binarized form along with its(bottom) log-scale Fourier Transform.

Figure 3. Musical note shown in its (top) original and (middle) binarized form along with its(bottom) log-scale Fourier Transform.

I was naïve enough to think that simply correlating the two images would be enough for me to isolate the individual notes in Figure 2.  Life isn’t like that though, as you can see from the first image in Figure 4, it looks like something I would see without my glasses (background info: I have really bad eyesight).  It’s out of focus, migraine-inducing and, I thought so at the time, depressing.

But we must forge on!  I used the binarized correlation image as a mask and multiplied this with the correlation matrix.  Using edge detection, the peak location of each note was found but for better visualization, a convolution with a circle was done.

Figure 4. Summary of image processing. (top) correlation, (middle) filtering and (bottom) peak identification

To know what note should be played when, the x- and y-values relative to the first note are obtained.  The x-values can tell when the note should be played and the y-value, the specific note to be played. 

Finally, here’s the finished product.  I saved the file twice for two reasons.  First, because I’m not sure the first go lets you hear enough of the song.  Second, because in my head the lyrics are “Rain, rain, go away, come again another day.  Rain, rain, go away, little children want to play.”  My mother or my kindergarten teacher must not have taught me right – or I must not have remembered the right words.  Or, those could actually be the right words.  None the less, enjoy this ten-second computer-generated rendition of Rain, Rain Go Away.

Figure 5. Scilab generated music file of Rain, Rain Go Away

I shouldn’t have been surprised that Scilab can play audio music.  It’s slightly infuriating that I had piano lessons and came out of them, not fully knowing how to read notes or play music from a music sheet but that with technology, I can.  I should feel empowered by this added modern musical appendage, but I don’t.  I can’t help but mourn for the money that went into all those lessons when I could have just taken up a computer class to learn Scilab since the end results are pretty much the same.

With a smile, I have to say that I’ve only had to press the Upload button on my YouTube account homepage for this class.  Doesn’t that make you feel all giddy inside? Another 9/10 for me for finishing the activity but not doing so on time.

REFERENCES:
1. “Rain Rain Go Away Sheet Music”, http://www.gmajormusictheory.org/Freebies/FirstPieces/0Rainrai.pdf

ten

In certain situations, it is desirable to separate certain features from the rest of an image.  Normally, these features are called regions of interest (ROI).  Here, the closing and opening morphological operations that were previously used (see here) were utilized to differentiate cancer cells from normal cells.

Note that the closing morphological operation is actually an erosion operation followed by a dilation while opening is a dilation followed by an erosion [1].  If you can remember, the erosion and dilation operations require a structuring element.  In this case, I simply weaved a snippet of code into the while chunk that was the SCILAB program needed for the activity so I could generate a circle which I then took as the structuring element.

Since cancer cells are larger than normal cells (probably because they destroy – a.k.a. eat healthy cells), the normal cell size had to be determined so that you could actually tell which is which.  Call it a calibration of sorts but an image (Fig. 1) was divided into subimages, each with a size of 256 x 256 pixels (Fig. 2).  These subimages were then converted to binary images based on their histogram values.  Since there weren’t pictures of cells lying around, images of paper puncher cutouts were used.

Figure 1. Original image of circles used to determine normal cell size.

Figure 2.  Subimages of size 256 x 256 pixels, shown in their respective locations to the original image

Figure 3.  Results of the binarized subimages obtained through each subimage’s histogram value.

The binarized subimages then went through the two morphological operations, first closing and then opening (Figures 4 and 5).  If I say so  myself, the opening operation cleans up the subimages while the closing operator combines overlapping circles.

Figure 4.  Results of the binarized subimages undergoing the opening morphological operation.

Figure 5. Results of each subimage in Figure 3 undergoing the closing morphological operation.

To determine the actual normal cell size, the area (in pixels²) of each circle – at least each distinguishable area in the image – is obtained (Figure 6).  We can take the peak in the left region to be the percentage of the individual cells and not the clumps which are taken into account by the other histogram peaks, especially those beyond the 2000 pixel count.  The normal size cell is then taken from the assumed individual cells, returning a value of 525. 

Figure 6. The histogram of the number of pixels in each distinguishable area of the image for (top) all pixel numbers and (bottom) zoomed in at 470 to 570 pixels.

The next, and most crucial, part of this whole activity is to detect cancer cells when it’s surrounded by normal cells (Figure 7). 

Figure 7.  Images of normal cells with cancer cells located within them.

Much like the previous step, the image of the cancer and normal cells was binarized based on the histogram values and applied with the opening then closing morphological operation.  Unlike the previous procedure though, the structuring element that was used was a circle that contained more than 525 pixels but lesser than what the approximated size of the cancer cells would be.  I have to say so myself, I isolated those cancer cells pretty well.  But then, I only had my visual count to compare it with.

Figure 8.  Image of the isolated cancer cells from Figure 7.

This activity reminded me so much of the many CSI and Bones episodes I’ve glued my eyes to over the years.  Now, I kind of want to apply this technique on actual medical images.  You can’t always imagine that, in medical images, the cancer cells will be photographed separately from normal cells.  That’s something to look forward to.

9/10 for being able to complete the objectives with the 1-point deduction for being late.

REFERENCES:

1. A10 – Binary Operations, Applied Physics 186 Manual