Beyond Photoshop. Activity 8, Enhancement in Frequency Domain

Photoshop is amazing. It is a powerful software that can enhance images in many ways. However, as far as I know, what we're trying to do in this activity, can't be done by Photoshop.. or probably, I just don't know how.

To begin, we have to be familiar with this frequency domain. I explored how certain patterns look in the frequency domain. First up are the Fourier transform of 2 circles of varying sizes. These are, from left to right, circles from radius 0.01 to 0.09 with a distance from the center of 0.1. Note that these are arbitrary units given that the image is 2x2 arbitrary units and 200x200 pixels big.

The Fourier space is not as mysterious as it seems

The first image contains the smallest circles resulting in just a pair of pixels. This represents 2 Dirac deltas and we (should) know that the Fourier transform of this is a sine wave as seen in the next image. As the circles increase in radius, the Fourier transform becomes a sine wave enveloped in a circle. This is better seen in the 3d plots shown below.

everything is better in 3d

These are the 3d plots of the Fourier transform of circles with radius 0.03, 0.06 and 0.09. As the radius of the circles increase, the resulting circle that envelopes the sine wave becomes smaller as well. Once the circle is big enough, the Fourier transform starts to look like the familiar Airy pattern.

I also tried to explore the effect of increasing the distance of the circles from the center of the image. In 1d space, we (should) know by now that what we are doing here is increasing the value of the dirac delta which translates to an increase in frequency in Fourier space. The result, as expected is a sine wave with a higher frequency.

increasing the distance of the circles from the center increases the frequency of the sine wave in Fourier space

The figures above are the set of images for the Fourier transform of the corresponding circles still with increasing radius from 0.01 to 0.09 but this time with distance 0.2 and 0.3 from the center respectively.

Again, probably it's the mood since it's hot right now since the aircon broke down but I just can't think of anything funny that's related to the lesson. I also did the 3d plot of the Fourier transform of the circles with distance 0.3 from the center shown below.

part of the 3d experience is being able to pan or 3d orbit

Again, they appear similar to that of the smaller distance from the center but this time the sine waves are closer to each other. It is better to explore these 3d plots if you have the code and are able to rotate it on your own.

The results of using squares are similar but the envelope is also more square like.

The results for the Gaussian are also similar but cleaner. When I got the Fourier transform of circles, the result was sine waves enclosed by circles. Also, as you can in the 3d plot, the central spot is also surrounded by smaller waves. The difference in the Gaussian is cleaner in a sense that it only has sine waves enclosed in a circle and nothing else. The images on the left are the Gaussian and to its right is the corresponding Fourier transform.

 like eyes in the dark

 I also tried to sizes. increasing the size of the Gaussian decreases the size of the Fourier transform. And since everything is better in 3d,

The first 3d image is that of two Gaussian with 0.2 units from the center and sigma 0.1. The second image is the Fourier transform of two Gaussian with sigma 0.05 and the last is for 0.01. These clearly show waves enclosed in a single circle and nothing else.


Another exercise for familiarity in this space is by using convolution. We were asked to make dots scattered randomly and then to convolve it with a 5x5 image shown in the first image below. I admit I'm not the most creative person here. I just made a pattern that's easy to code that resulted in a letter M. This is shown in the middle image below while the convoluted image is the right most.

M for Maricor. Hihi :>

This is also an expected result, the scattered dots acting as dirac deltas. Convolving it with the pattern results in the pattern being repeated in the position of the dots. Of course the M became a W since (we know that) applying Fourier transform jumbles up the matrix and thus changes the orientation.

I also explored the Fourier transform of lines of various spacing and orientation. For each set, I increased the separation of the lines. The first image in each set has alternating white and black lines with width 2 pixels. The next images have black spaces increasing the distance between white lines by 2 pixels each.

There are dots there. Just look closer

As it turns out, the Fourier transform of horizontal lines are dots along the x-axis while the dots are along the y-axis for vertical lines and both are present for the grid. It was also observed that as the frequency of the lines decrease, the frequency of the dots increase.

The interesting part is applying what I learned from the exercise to real life problems such as images with artifacts.

We have this image of a crater of the moon I think, taken quite a long time ago.

nothing spectacular about this image itself but just research on the science and history

This is probably a milestone for Science, being able to take photographs like this from the moon. I should research more about this image. However, the most obvious thing about this picture is the horizontal lines that run across it. I've read somewhere that this because this is a composite image of a number of images stitched together but again, I have to research more to confirm. Anyway, wouldn't it be nice if they did a bit of image processing to take it away?

try that in Photoshop

In fact, it was just recently that Floyd was telling me about how Photoshop appears like magic. Indeed, before this course, we never thought about the underlying codes and science that Photoshop employs to enhance or create images. The process of course was not as simple as pressing a button in Photoshop. On the other hand, it's not as hard as I thought it would be.

I just love how colorful Scilab is

The process is actually pretty short and outlined by the comments I placed in the code. The image was transformed to Fourier space for manipulation.

Fourier space doesn't look as interesting as it really is

What we have is a bright spot in the center with vertical and horizontal lines passing through it. Most of the information from the image is found in this bright spot. Then, from our exploration of the Fourier space above, we know that these lines further away from the center represent repeating artifacts or probably unwanted elements. I filtered these lines by a few lines of code denoted by the comments as filter. Of course, I left out the part of the lines passing through the center with a few pixels given by the threshold. In this case, 3. I also tried other values but these don't have much different unless of course you make it 0 and thus remove the important information. otherwise, a threshold as small as even 1 would appear the same as other values. And again, of course, make it too big and you won't be able to filter out the artifacts. Another note is that removing the horizontal line through the center won't make much of a difference in the resulting image. The most obvious artifacts visually are horizontal lines and again from the exploration part, we know that dots along a vertical line result in horizontal lines after applying Fourier transform. Below is the filtered image in Fourier space. I still removed the horizontal line as well just to remove all the artifacts, even those that are not so obvious.

Next, we applied the same technique to a more challenging image, a painting. It seems that the texture of the canvas resulted in a noisy image. The resulting texture of the painting shows the roughness of the canvas. I can't explain it well. As they say, a picture paints a thousand words so just look at the picture below and you'll get what I mean.

I'm not really an artsy person

Note however that the texture or roughness or whatever you call it is repetitive.. and you (should) know what that means- it can be removed in the Fourier space.

 walang misteryoso diyan (filter r=0.15)

As a brilliant wise man always said, walang misteryoso diyan. I just applied the same technique used in the lunar image. There are a few differences however. For one, the image in Fourier space is much different. It is of course related to the nature of the repetitive elements in the image.

The Fourier transform of this image is more interesting

Credits to Chester for suggesting that I show the log of the values in Fourier space. Otherwise, all we'll see is a single bright pixel in the center. This is the same thing as what I did in the lunar Fourier space. I'd also like to thank Chester for helping me debug my code. Apparently, when I was returning to real space, I was taking the Fourier transform of only the real part of the image in Fourier space resulting in also just one super bright pixel. Thank you Chester :)

Next I remove the unwanted elements in Fourier space. I just used a circular aperture centered on the center of the Fourier transform and set the other parts to zero.

Like a spotlight on the main character

This is the image in Fourier space with the filter. The important information is retained in the center while the repetitive noise is removed. The circular aperture shown above has a radius of 0.15 units. I also tried varying the aperture. As expected, some of the information for other details were removed by the filter. Increasing the aperture allowed more information to be saved.

This is the same Fourier transform but this time with a filter with a bigger aperture, 0.22 units. Note that the filter still doesn't allow the bright dots scattered in the Fourier space. These dots contain the information for the texture of the canvas. Increasing the aperture greater than 0.22 units would allow these dots to be expressed in the image again, and thus, negating the effects of the filter.

(filter r = 0.2)

I'm not sure if you can see it clearly here. I have to apologize for uploading so many hi-res images but I did it so the difference can be clear. Maybe if you open the images individually, the difference would be more obvious. Anyway, the trend is clear. Increasing the size of the aperture allows more detail up to 0.22 units. Increase it beyond 0.22 and you'll get the original image. The image below is the result of using an aperture with radius 0.22 while the one above is 0.2. Scroll up some more and you'll find the result of having an aperture of 0.15. Decreasing the aperture removes some details making the image blurred. Decrease it beyond 0.15 and the blurring becomes apparent.

filter r = 0.22

I guess it depends on personal preference on what aperture size to use. There are other options though too. Probably using other types of filters would result in better images but I'm too lazy to go that far. I believe that I've placed a lot of effort on this activity going way beyond what we were asked to do. In fact I think I've spent the most time in this activity so far. I think (most of) my peers would agree that this activity is probably worth four days. It is true that each of the codes are quite short but I've also spent some time debugging like when I was returning only the real part of the Fourier transform. I give myself a 12.

I also realized the method is only limited to periodic artifacts. For artifacts that are found only in a single spot, I would recommend Photoshop specifically using content aware. It is an awesome tool that allows users to remove certain unwanted elements in the image and fill the space up with the background. It is so awesome. I remember being so excited to get to use Photoshop CS 5 when I read about this before it was released. Read more about it in this link. http://www.photoshopessentials.com/photo-editing/content-aware-fill-cs5/. It started with CS 5 and continues on CS6. Probably my favorite tool in the world. It is probably the closest thing to what you can consider as magic in image enhancement.