Fisheye to Rectilinear Conversion

911 Ground Zero. Original 35mm full frame 16mm fisheye image
The area in red is that recorded on a 22.7 x 15.1mm DSLR sensor

Here's the original image, scanned from 35mm film. The lens was a 16mm fisheye and it's quite evident that the image suffers form "fisheye distortion". The fisheye results in straight lines that are curved, especially near the edges of the frame. The horizontal field of view is around 140 degrees, the vertical field of view is around 90 degrees and the diagonal field of view is 180 degrees. This is much wider than any available rectilinear lens. For example, a 14mm rectilinear lens (the widest such lens available for a 35mm SLR) only has a horizontal field of view of 104 degrees (vs. 140 degrees for the fisheye).

This image also suffers from perspective distortion because the camera was tilted upwards. This is evident by the fact that the buildings "lean in" towards the center of the image. This is more clearly seen in the image below, which has been corrected for "fisheye distortion". This perspective distortion is not a result of the fisheye lens. A normal wide-angle rectilinear lens would show the same distortion

This the image after fisheye to rectilinear conversion

In this image the "fisheye"distortion has been corrected to form a rectilinear image, though the shape of the image is very odd indeed! The corners have been really stretched out, though that's not surprising since as I said above, the diagonal field of view is 180 degrees. It turns out that you can't have a single rectilinear image formed on film with a static lens which has a 180 degree field of view, or if you did the image would have to be infinitely wide! So it's no wonder that the corners are stretched so much, and in fact are slightly cropped.

Luckily the area recorded by the small DSLR sensor does not require such extreme correction and the stretching is relatively moderate.. You can now clearly see the perspective distortion of the buildings leaning inwards. The red frame which shows the image area recorded on the DSLR sensor is no longer rectangular. We will deal with that later, but first let's crop the image to a rectangular format (yellow frame), then correct the perspective distortion.

This is the above image, cropped to the maximum rectangular size

This is the above image,  this time corrected for perspective

As you can see, the top of the image has been stretched, pulling the buildings back into the vertical. The image has now been corrected for both perspective and fisheye distortion. All that remains is to crop the area inside the red DSLR frame lines into a rectangular format.

Here is the final crop, corrected for both fisheye distortion and perspective, including only that part of the imaged recorded by a 22.7 x 15.1mm DSLR sensor This view is approximately that which would be seen with a 20mm lens on a full frame 35mm SLR. Figures at the edges of the frame are stretched, but that's exactly what happens with a rectilinear lens. The fisheye lens doesn't stretch objects in this way, but it bends straight lines.  Comparing this shot with the fisheye shot you can see that each has it's own type of distortion and neither represents the scene in quite the same was as we see it. However people are more used to seeing rectilinear distortion, so that's what people usually prefer to see. Note that the aspect ratio of the image has been changed by the transformation.

Though the transforms here were done on a full frame 35mm image, they could just as easily have been done on the image recorded by a small sensor DSLR (i.e. the first image cropped to the red lines). Image quality can be quite good since we are working with the center part of the image and extreme corrections are not needed (as is the case for full frame 35mm images). 

How to do the corrections

• Fisheye to Rectilinear correction can be done using a number of programs. For the image shown here I used Panotools. It has two things going for it. First it's probably the most powerful tool available for image correction and panoramic stitching as well as a number of other functions. Second it's free! The only downside is that it is slightly "geekware" in that it does not have a simple user interface, indeed it doesn't even self install (you have to move files into appropriate directories manually). There'salso not much in the way of a manual or tutorial. It works either as a stand alone JAVA program or as a PhotoShop plugin. I used the PhotoShop configuration. There are graphical front ends available for the program, but the most user friendly Windows version, PTGUI,   costs $40. To get help with the program, try the Panotools FAQ website, this FAQ or this Panorama Tools website. If there is a complete Panotools manual, so far I haven't found it! I'm no expert with this program, so asking me questions about it isn't likely to get you much of an answer I'm afraid.
  
  There are also a number of commercial programs which can make corrections for image distortions, for example checkout ImageAlign (though note it costs $300).

• Perspective correction can be done by most popular image editing programs. I used Picture Publisher 8.0's transform tool for this example, but you could just as easily use PhotoShop using the crop tool's perspective option.

One of the problems of small sensor DSLRs used with regular 35mm lenses is that it's very difficult to get a really wide angle image. The small sensor results in an effective "lens multiplier", so lenses act as though they have a longer focal length as far as field of view is concerned. Using the common 22.7 x 15.1 mm sensor size (which is used in the Canon EOS D30/D60/10D bodies) results in a 1.6x lens multiplier. So a 20mm lens gives the same view on such a DSLR as a 32mm lens does on a regular full frame SLR. A 17mm lens gives the view of a 28mm lens and a 14mm lens (the widest available) gives the view of a 22mm lens.

Nikon have announced (but have not yet made available as of 03/03) a special lens which only covers the small sensor (not the full 35mm frame), which is a 12-24mm, corresponding to 19-38mm in normal 35mm terms.

Is there an alternative? Well, the answer is yes. Since we are working directly in the digital domain, not on film, we can do a lot of image manipulation. Fisheye lenses have VERY wide fields of view, but distort the image in an unnatural looking way. However in the digital domain such distortions are fairly easily corrected, or at least transformed into the more normal looking distortions given by rectilinear lenses!

There are two types of fisheye lens. The first has a 180 degree field of view in all directions and results in a circular image on 35mm film. These lenses typically have a focal length around 8mm. These lenses are not so useful in this application. The second type of fisheye is known as a "full frame fisheye" and they cover the entire 35mm frame, with a 180 degree diagonal field of view. Typical horizontal coverage is around 140 degrees, typical vertical coverage is around 90 degrees and typical focal length is 15 or 16mm.

Nikon and Canon (and other major brand)  fisheye lenses are not cheap, but there is a Russian (actually Ukranian) Zenitar 16/2.8 fisheye available for many cameras at a cost of around $150. which is a decent all manual lens. It comes in both Nikon and Pentax screw mounts, with adaptors for a number of other cameras such as the Canon EOS line. The best way to find a vendor of this lens is to do a web search or look on Ebay. It's typically not stocked by the usual mail order houses such as Ritz, Adorama or B&H.

Readers' Comments

Add a comment

David Littleboy (Tokyo, Japan) , March 07, 2003; 10:39 A.M.

FWIW, Picture Window Pro includes a lens distortion correction funtion that will fix fisheye distortion. It seems quite easy to use. It also has a perspective correction function, but I usually find that if the buildings at the edges are straight, the building in the center has an enlarged top section...

Question: doesn't image quality get seriously funky around the edges?

Bob Atkins , March 07, 2003; 02:34 P.M.

Image quality does get seriously funky if you do this on a full frame fisheye image with 180 degrees diagonal coverage. However if you only do the transform on a cropped section, which has a smaller angular coverage, the transformed image is a lot better. I'll try to post an example later of a transformed fisheye image from a D30.

Joseph Wisniewski , March 08, 2003; 10:19 A.M.

Sounds like a combo I use occasionally, 8mm fisheye on a Nikon D100. Here's a portrait, after conversion in PanoTools to the equivelant of a 6mm ultrawide on 35mm film.

http://www.pbase.com/image/7219139

I should have spend more time futzing with the CA corrections on this one.

Ciao!

Joe

Edgar Martinez , March 09, 2003; 01:28 P.M.

I think it's clear in the "final crop" picture, that one more step is needed: to strech the image vertically to get the proper ratio. Using the perspective tool to adjust for tilt seems to squish images, making everything look shorter than it is.

Bob Atkins , March 09, 2003; 02:27 P.M.

Final image with extra vertical stretch

It's possible the vertical stretching of this image wasn't optimized, but in general you will tend to get objects at the sides of the frame looking "squat" since they are stretched more horizontally than vertically.

All flat projections distort, just as all maps of the globe distort. Rectilinear projections preserve straight lines, but at the cost of distorting areas. Fisheye projections preserve areas, but at the cost of bending straight lines. We tend to view rectilinear projections as "more correct", but in fact they are just different.

Evrim Icoz , April 11, 2003; 12:21 P.M.

This is good, however, how can one go about doing a real fisheye image on a DSLR with the drop factor. That wouold have been more interesting, to me. You can get a Sigma 15-30mm and your wide angle worries are solved (at a price). I have the Zenitar but never use it.

Andrew Zabolotny , October 01, 2007; 01:27 P.M.

Maybe I'm a little late but actually Zenitar is a line of really Russian lenses (the factory is called KMZ, http://www.zenit-camera.com/ and it's located in Krasnogorsk, a Moscow suburby), Ukrainian lenses are called Arsat (the factory is called "Arsenal") and the factory is located in Kiev.

Kjeld Olesen , November 16, 2007; 03:47 A.M.

RectFish

Another option for remapping fisheye images is provided by RectFish which on top of rectilinear remapping with perspective control also provides a unique "rectangular" projection mode, that minimizes stretching but will get rid of dark corners.