Beyond Photoshop. Activity 8, Enhancement in Frequency Domain

Photoshop is amazing. It is a powerful software that can enhance images in many ways. However, as far as I know, what we're trying to do in this activity, can't be done by Photoshop.. or probably, I just don't know how.

To begin, we have to be familiar with this frequency domain. I explored how certain patterns look in the frequency domain. First up are the Fourier transform of 2 circles of varying sizes. These are, from left to right, circles from radius 0.01 to 0.09 with a distance from the center of 0.1. Note that these are arbitrary units given that the image is 2x2 arbitrary units and 200x200 pixels big.

The Fourier space is not as mysterious as it seems

The first image contains the smallest circles resulting in just a pair of pixels. This represents 2 Dirac deltas and we (should) know that the Fourier transform of this is a sine wave as seen in the next image. As the circles increase in radius, the Fourier transform becomes a sine wave enveloped in a circle. This is better seen in the 3d plots shown below.

everything is better in 3d

These are the 3d plots of the Fourier transform of circles with radius 0.03, 0.06 and 0.09. As the radius of the circles increase, the resulting circle that envelopes the sine wave becomes smaller as well. Once the circle is big enough, the Fourier transform starts to look like the familiar Airy pattern.

I also tried to explore the effect of increasing the distance of the circles from the center of the image. In 1d space, we (should) know by now that what we are doing here is increasing the value of the dirac delta which translates to an increase in frequency in Fourier space. The result, as expected is a sine wave with a higher frequency.

increasing the distance of the circles from the center increases the frequency of the sine wave in Fourier space

The figures above are the set of images for the Fourier transform of the corresponding circles still with increasing radius from 0.01 to 0.09 but this time with distance 0.2 and 0.3 from the center respectively.

Again, probably it's the mood since it's hot right now since the aircon broke down but I just can't think of anything funny that's related to the lesson. I also did the 3d plot of the Fourier transform of the circles with distance 0.3 from the center shown below.

part of the 3d experience is being able to pan or 3d orbit

Again, they appear similar to that of the smaller distance from the center but this time the sine waves are closer to each other. It is better to explore these 3d plots if you have the code and are able to rotate it on your own.

The results of using squares are similar but the envelope is also more square like.

The results for the Gaussian are also similar but cleaner. When I got the Fourier transform of circles, the result was sine waves enclosed by circles. Also, as you can in the 3d plot, the central spot is also surrounded by smaller waves. The difference in the Gaussian is cleaner in a sense that it only has sine waves enclosed in a circle and nothing else. The images on the left are the Gaussian and to its right is the corresponding Fourier transform.

 like eyes in the dark

 I also tried to sizes. increasing the size of the Gaussian decreases the size of the Fourier transform. And since everything is better in 3d,

The first 3d image is that of two Gaussian with 0.2 units from the center and sigma 0.1. The second image is the Fourier transform of two Gaussian with sigma 0.05 and the last is for 0.01. These clearly show waves enclosed in a single circle and nothing else.


Another exercise for familiarity in this space is by using convolution. We were asked to make dots scattered randomly and then to convolve it with a 5x5 image shown in the first image below. I admit I'm not the most creative person here. I just made a pattern that's easy to code that resulted in a letter M. This is shown in the middle image below while the convoluted image is the right most.

M for Maricor. Hihi :>

This is also an expected result, the scattered dots acting as dirac deltas. Convolving it with the pattern results in the pattern being repeated in the position of the dots. Of course the M became a W since (we know that) applying Fourier transform jumbles up the matrix and thus changes the orientation.

I also explored the Fourier transform of lines of various spacing and orientation. For each set, I increased the separation of the lines. The first image in each set has alternating white and black lines with width 2 pixels. The next images have black spaces increasing the distance between white lines by 2 pixels each.

There are dots there. Just look closer

As it turns out, the Fourier transform of horizontal lines are dots along the x-axis while the dots are along the y-axis for vertical lines and both are present for the grid. It was also observed that as the frequency of the lines decrease, the frequency of the dots increase.

The interesting part is applying what I learned from the exercise to real life problems such as images with artifacts.

We have this image of a crater of the moon I think, taken quite a long time ago.

nothing spectacular about this image itself but just research on the science and history

This is probably a milestone for Science, being able to take photographs like this from the moon. I should research more about this image. However, the most obvious thing about this picture is the horizontal lines that run across it. I've read somewhere that this because this is a composite image of a number of images stitched together but again, I have to research more to confirm. Anyway, wouldn't it be nice if they did a bit of image processing to take it away?

try that in Photoshop

In fact, it was just recently that Floyd was telling me about how Photoshop appears like magic. Indeed, before this course, we never thought about the underlying codes and science that Photoshop employs to enhance or create images. The process of course was not as simple as pressing a button in Photoshop. On the other hand, it's not as hard as I thought it would be.

I just love how colorful Scilab is

The process is actually pretty short and outlined by the comments I placed in the code. The image was transformed to Fourier space for manipulation.

Fourier space doesn't look as interesting as it really is

What we have is a bright spot in the center with vertical and horizontal lines passing through it. Most of the information from the image is found in this bright spot. Then, from our exploration of the Fourier space above, we know that these lines further away from the center represent repeating artifacts or probably unwanted elements. I filtered these lines by a few lines of code denoted by the comments as filter. Of course, I left out the part of the lines passing through the center with a few pixels given by the threshold. In this case, 3. I also tried other values but these don't have much different unless of course you make it 0 and thus remove the important information. otherwise, a threshold as small as even 1 would appear the same as other values. And again, of course, make it too big and you won't be able to filter out the artifacts. Another note is that removing the horizontal line through the center won't make much of a difference in the resulting image. The most obvious artifacts visually are horizontal lines and again from the exploration part, we know that dots along a vertical line result in horizontal lines after applying Fourier transform. Below is the filtered image in Fourier space. I still removed the horizontal line as well just to remove all the artifacts, even those that are not so obvious.

Next, we applied the same technique to a more challenging image, a painting. It seems that the texture of the canvas resulted in a noisy image. The resulting texture of the painting shows the roughness of the canvas. I can't explain it well. As they say, a picture paints a thousand words so just look at the picture below and you'll get what I mean.

I'm not really an artsy person

Note however that the texture or roughness or whatever you call it is repetitive.. and you (should) know what that means- it can be removed in the Fourier space.

 walang misteryoso diyan (filter r=0.15)

As a brilliant wise man always said, walang misteryoso diyan. I just applied the same technique used in the lunar image. There are a few differences however. For one, the image in Fourier space is much different. It is of course related to the nature of the repetitive elements in the image.

The Fourier transform of this image is more interesting

Credits to Chester for suggesting that I show the log of the values in Fourier space. Otherwise, all we'll see is a single bright pixel in the center. This is the same thing as what I did in the lunar Fourier space. I'd also like to thank Chester for helping me debug my code. Apparently, when I was returning to real space, I was taking the Fourier transform of only the real part of the image in Fourier space resulting in also just one super bright pixel. Thank you Chester :)

Next I remove the unwanted elements in Fourier space. I just used a circular aperture centered on the center of the Fourier transform and set the other parts to zero.

Like a spotlight on the main character

This is the image in Fourier space with the filter. The important information is retained in the center while the repetitive noise is removed. The circular aperture shown above has a radius of 0.15 units. I also tried varying the aperture. As expected, some of the information for other details were removed by the filter. Increasing the aperture allowed more information to be saved.

This is the same Fourier transform but this time with a filter with a bigger aperture, 0.22 units. Note that the filter still doesn't allow the bright dots scattered in the Fourier space. These dots contain the information for the texture of the canvas. Increasing the aperture greater than 0.22 units would allow these dots to be expressed in the image again, and thus, negating the effects of the filter.

(filter r = 0.2)

I'm not sure if you can see it clearly here. I have to apologize for uploading so many hi-res images but I did it so the difference can be clear. Maybe if you open the images individually, the difference would be more obvious. Anyway, the trend is clear. Increasing the size of the aperture allows more detail up to 0.22 units. Increase it beyond 0.22 and you'll get the original image. The image below is the result of using an aperture with radius 0.22 while the one above is 0.2. Scroll up some more and you'll find the result of having an aperture of 0.15. Decreasing the aperture removes some details making the image blurred. Decrease it beyond 0.15 and the blurring becomes apparent.

filter r = 0.22

I guess it depends on personal preference on what aperture size to use. There are other options though too. Probably using other types of filters would result in better images but I'm too lazy to go that far. I believe that I've placed a lot of effort on this activity going way beyond what we were asked to do. In fact I think I've spent the most time in this activity so far. I think (most of) my peers would agree that this activity is probably worth four days. It is true that each of the codes are quite short but I've also spent some time debugging like when I was returning only the real part of the Fourier transform. I give myself a 12.

I also realized the method is only limited to periodic artifacts. For artifacts that are found only in a single spot, I would recommend Photoshop specifically using content aware. It is an awesome tool that allows users to remove certain unwanted elements in the image and fill the space up with the background. It is so awesome. I remember being so excited to get to use Photoshop CS 5 when I read about this before it was released. Read more about it in this link. http://www.photoshopessentials.com/photo-editing/content-aware-fill-cs5/. It started with CS 5 and continues on CS6. Probably my favorite tool in the world. It is probably the closest thing to what you can consider as magic in image enhancement.

Fourier glasses. A7, FT Model of Image Formation

As it turns out, this activity looks like we are trying to blur images on purpose. I'm not yet done of course but I do have some preliminary images. Been coding for a while now and will take a break by blogging about it. This is also to encourage myself that I'm making some progress.

The first figure is a set of images that shows the effects of the Fourier transform, fft2 in Scilab. The first is the original image, a 128 x 128 bitmap image from paint. The second is the output when fft2 is applied twice to the image. The result is an inverted circle. You won't notice much of a difference other than it appears to be lower than the original since the circle is not centered. the effect is more obvious in the second sample, the letter which is shown on the fourth and fifth image. The third and the last image are the result when applying the fft2 to the respective image once. It doesn't seem like much.

The next figure is the result of the convolution of the word NIP with an aperture. Again, the word appears is inverted since fft2 is applied twice. from left to right, the images are the result of increasing aperture size from radius 0.1 to 0.55.

blurred on purpose

The next figure is the result of template matching using correlation. The original image, the leftmost image is black text saying "THE RAIN IN SPAIN STAYS MAINLY IN THE PLAIN" on a white background. Again, this was done in paint saved as a 128x128 bitmap. A black letter 'A' was also saved in paint with the same settings, font and font size. What the code does is it finds letter A's in the phrase using correlation. It finds shapes in the first image that are similar to the second image. The result is still the blurred phrase wherein the letter As are less blurred. The third and fourth images are the same. I just tried if it would still work when the colors are inverted. Based on the result, it seems it does.

what does this phrase mean anyway?

The result is not exactly just purely letter As. The result of the correlation also contains some parts of the other letters since some parts of the letter A are also found in the other letters. I tried to remove the other letters by using the find function to remove lower values. The correlation of the letter A with the other letters are better exhibited here, in the third image of the figure below.

I guess if the Spanish plains are surrounded by mountains, clouds couldn't pass through... 

The letter H, E and R contain a structure similar to the central part of the letter A and the letters N and M contains a similar triangular shape. That is why they are not completely removed from the image. For this to work perfectly, the template should be very different to all the other structures in the image. Otherwise, it would correlate to some degree and reduce the accuracy.

A short break before the edge detection. I think this post is one of the more serious in this blog. I can't think of anything funny to say. Probably, it would be better if the phrase used were more interesting. It could also be the mood as I write this. This is a sad day.. It also feels like I'm just pushing to finish everything on time and even not so late.
...


The next set of images are the result of edge detection using the built in convolution function in scilab, conv2. The image NIP was recycled for this. I just remembered that some of us used NIP instead of VIP since we are not part of the VIP lab. It just occurred to me that we use VIP which could also mean what we are doing and not the lab. Anyway, I'm proud to be an NIP student.. I think. From left to right, the patterns used were a horizontal line, a vertical line and a diagonal line. Personally, I think these are good results. The resulting images only show the parts of the letters that contain the pattern used.

a few shades of gray 

I also tried it on the Spanish rain phrase as a measly attempt to 'extend the learning'. The results also appear fine. The first image shows horizontal lines clear enough to be distinguished. Similarly the vertical lines are obvious in the second image and diagonal lines in the third.

I called the phrase "Spanish rain" for short

Credits to Floyd for telling me about the conv2 function in Scilab and to Alix for a few syntax help and other bits of information such as to use the mat2gray function. Lesser credits to Chester for rearranging my Scilab console even if I didn't ask. I appreciate it :)

I would like to give myself an 11 since I believe I was able to do everything required and extend it a bit further. I would also like to encourage myself for completing this blog before the deadline. It is exactly 12 noon of July 11, 2013 as I'm typing this sentence.

Don't be Racist. Actvity 6, Histogram Manipulation

There are some parts of some pictures that appear dark but there may still be some details recorded on the camera. As mam said, the sensitivity of the camera allows it to record the detail. However, in the process of reproduction and detection by the eyes, some information is lost.

Our lecture in our other class, AP 187 can be applied here regarding the process of how we perceive images. The image you see in the monitor such as in this picture is a result of multiple factors.

A night to regret este remember :)) :P

If only we could see everyone in this picture...

And that's why candle light is not the preferred method of lighting in this day and age

The first and probably most obvious factor of why the image is so is because of the poor lighting. The main source of light in this picture is a single candle. That's probably the reason why the light bulb was such a bright idea. If you're a romantic, you would probably factor in the light from the stars or the moon too. That's basically the reason why we can still see some things outside at night. All these light are then reflected by the surfaces that the light hits. These include the happy faces of the guests, the confused faces of the newly weds and their borrowed wedding clothes and of course the glasses of our hilarious priest, doc Sarmago. The reflected light is then collected by the camera, this time Anjali's Olympus camera. It was probably the day I regret the most for not bringing my own camera. The light is turned into data by the camera and read by the computer. The data is transformed back into light by your computer screen. Finally, the light from the screen is collected by the eyes and processed by the brain. In this case, there's not much to process since it's so dark. But don't worry, there are still a few things we can do with this picture. We still have ways to save the proof that this really happened. We still have ways to remind Aeriel and Tor that they made a vow to love each other for the rest of their lives <3 Hihi :>

The image may appear dark to our eyes but as we should already know by now, our eyes are liars. That's why we shouldn't judge a book by its cover nor an image by its color. Haha. We judge an image by its histogram!

Based on this histogram, there is still hope for this picture.

Interestingly, it only takes a few lines of code in Scilab to output the histogram. It is easy to see how Scilab is optimized for these kinds of image processing. Of course, I'd still prefer Photoshop or Gimp for image manipulation. On the other hand, there is still a different sense of accomplishment of actually typing your own code to edit your own pictures. I'm proud to say that I was able to write the code on my own with minimal help from others. I can really feel that I'm learning a lot about Scilab. I almost finished the entire code in one sitting during the class time. It's not absolutely fast. I can imagine Chester laughing at me but this is a big improvement for me. I still encountered a few problems though. One of them is that I was unable to normalize the histogram. I plotted it too so that I could normalize it.

I'm still learning okay?

The CDF was easily computed using cumsum.

almost 80% of the pixels have a value of around 50

Actually, the heart of this code is in the process of redistributing the numerical values of each pixel in the image to follow a ramp cdf. After some thought, I found that it could be coded into a single line:

ni(find(ni<256)) = 255*cdf(ni(1:length(ni))+1);

Credits to Alix for suggesting using the find function as an easier way of replacing the values instead of making a new array. Again, there is a warm sense of accomplishment of finding a solution and in just a single line. I would also like to boast about a solution to the one imshow per run problem. Again, it may not be a big deal to you, but I'm happy I found it. The workaround is simple. Just put the two arrays together such as in this line:

both = [gi ni];

where gi is the original grayscale image and ni is the new image. Then I just imshow both. Doing that, we have the images side by side for comparison. Of course the image below is the result of imwrite.

Now you don't see me, now you do. Seriously. Comment if you get it.

Now that I think about it, there is a flaw in this procedure namely the color was lost. Now I'm wondering if there was a way to get the color back.. Of course, you can do image enhancements like this in Photoshop or Gimp. I then realized that probably, they use similar algorithms to carry out such processes.

Getting back to the resulting comparison. Can I just say that I'm not that dark. Seriously. Did you notice me in the original image? I'm in the upper right corner of both pictures but I can only be seen on the edited image. I guess I'm just too far from the light source. Moving on to other observations, of course, the image is much brighter. Mam would argue that the contrast improved. Actually, I think it's both. This can be seen in the new histogram and the CDF.

The new histogram shows better contrast and brightness

The histogram shows how the pixel values are redistributed to improve the image. Contrast is basically the difference between light and dark. The bright is represented by the higher x values while black is represented by 0. Previously, as I said, majority of the values are below 50, which is dark. The new image now shows a bigger range. As seen in the CDF as well, there is now a linear distribution of values from 0 to 255.

The new CDF follows a linear trend as expected

Brightness, on the other hand, is the number of pixels in the higher values. Again, as we see in the histogram, the values are shifted to higher values and there are now more pixels in the higher regions.

Another observation in the image is the noise made in the very dark areas such as in Tor's suit. The answer, I think, can be seen in the new histogram as well. As mentioned before, the code shifted the the values to higher values. What is notable in the histogram is how the very dark pixels are much more spread apart. This means that some pixel values that were originally very similar were suddenly shifted to have very different values. The result is that adjacent pixels alternate between light and dark resulting in a grainy image. The start of the new CDF that looks like steps is also related to this. Since large amounts of a certain value are shifted, there is no fine adjustment. I get it but am having a hard time explaining it. Anyway, I guess one way to improve this is to have a smaller bin size or more values, not just 0-255. Of course, I realize that if a similar process is used in commercially available image enhancement software, they find ways of improving the algorithm.

coded in Scilab for more than 3 hours. got the same thing in Gimp in less than 3 minutes.

However, the sense of accomplishment can be approximated as inversely proportional to time.

A similar result can be obtained by manipulating the input output or response curve in Gimp. Basically, the input is the image and the output is linear. changing the response curve changes the brightness of the image.

Again, I enjoyed coding. As it turns out, the code is not that long and just challenging enough for me to figure out with minimal help.

//Activity 6, Histogram Manipulation code
//by Joshua Beringuela completed on July 2, 2013
//read image, convert to gray scale,
//output histogram and CDF, and apply ramp CDF

oi = imread('C:\Users\Shua\Desktop\di2s.jpg');
gi = rgb2gray(oi);
ni = gi
scf(1);
h = imhist(gi,256,'')/length(gi);
scf(2);
plot(h);
cdf = cumsum(h);
scf(3);
plot(cdf);
ni(find(ni<256)) = 255*cdf(ni(1:length(ni))+1);
both = [gi ni];
imshow(both);
scf(4);
nh = imhist(ni,256,'')/length(ni);
cdf2 = cumsum(nh);
scf(5);
plot(cdf2);
//imwrite(both,'a6.png')

I am also happy with this activity and this blog because to be honest, this is the first entry I actually finished so far and on time! and in the process, I enhance an image of an event that has been imprinted in the minds of all my batchmates. This is the activity that I completed and even also applied a few ideas from 187. I give myself an 10 :)