Ten Years of Image Stitching in Numbers

May 2011

Introduction

I have been using PTAssembler to create create stitched, high-resolution and/or panoramic images for a little more than 10 years. For the first year or two, I used stitching only occasionally, typically when I wanted to capture a scene that was too wide for a single image taken by my camera, or if I was particularly inspired by a magnificent landscape. However, most of my photography remained of the "single shot" variety.

After a couple of years, my approach to photography changed, and I started using image stitching much more frequently. I continued to use image stitching to capture wide landscapes in a panoramic format, but I began to use it more and more to create large, high resolution images. I came to appreciate other important benefits of using image stitching, such as control over image projection and perspective. For the last few years, all of my "serious" photography has utilized image stitching, and "single-shot" photography is usually only for casual snapshots and test shots.

22 component images, 250 degree field of view, rectilindrical projection, 3.66:1 aspect ratio, 300 megapixels

Analyzing Ten Years of History

I recently spent a few weeks reviewing, tidying up and organizing the stitched (aka "panoramic") images that I have taken over the last 10 years. I have been reasonably diligent about organizing my images in a semi-consistent fashion as I have created them, but far from perfect. Going back through 10 years of images I found all sorts of unfinished panoramas and tens of thousands of images that I had never stitched together (and probably never will). I found numerous duplicates, images that I stitched together with different variants, and so on. I tidied up as much as I thought was reasonable, and then decided to try and catalog and analyze the images in my collection. For each panorama, I gathered information such as the size of panorama, the number of component images, the projection type, the field of view, and so on.

The analysis is far from perfect, and I ended up with some missing data; about 10% of the images have missing data, mostly because I didn't want to spend the time digging through hundreds of old CDs and DVDs (on which some older image files and project data are stored) to track down every last piece of information. Nonetheless, I ended up with a reasonably complete dataset of most of the panoramic and stitched images I have taken since about the middle of 2002. (Although I took many panoramic images prior to the middle of 2002, they weren't easy to access, and I have left them out of this analysis.) I was curious to see what sort of interesting data I could find in this inventory. This article summarizes some of that analysis.

Cameras

I used a total of 10 cameras (6 DSLRs and 4 compacts) over this period. Until the middle of 2006, all of the images were taken with a DSLR. Since then I have used a combination of a DSLR and a compact camera. As technology has improved, so has the resolution of the cameras I have used. The DSLR I used in 2002 had a sensor with 6 megapixels, and successive cameras increased the sensor size to 8 megapixels, 10 megapixels, 12 megapixels, 15 megapixels and my current camera has 18 megapixels. My first compact camera (in 2006) had a sensor with 7 megapixels, while my current compact camera has a resolution of 15 megapixels.

Productivity

The catalog contains 4371 panoramas, comprised from about 50,000 component images. (Note: I say "about" because I have had to estimate the number of component images for a small portion of the panoramas where data was missing...I make similar estimates elsewhere in this article). About 650 of these images (about 8%) are on display in my gallery.

The chart below shows that the number of panoramas that I have created per year varies considerably. 2009 was a banner year, but 2011 (for which the data only covers the first 4.5 months of the year) is also shaping up to be a good one! This chart only shows the number of stitched images that I have finished creating, and for which a final stitched composite image exists. There are thousands of panoramas for which I have the component images, but have not stitched together. Most of these unstitched panoramas were "alternate takes" of a scene, others were "problem" panoramas (e.g. focus errors, lighting changes, etc.), and I'm sure that there are more than a few that I have overlooked, some of which I may go back and stitch together in the future.

The next chart shows the length of time that elapses between when I photograph a scene, and when I have finished creating the stitched result. I frequently start stitching images together on the day I photograph, but rarely manage to process an entire shooting session on the same day as a I photograph. I am able to stitch together a little less than half of the images I create within a week of photographing, and about 90% are complete within four months. However, nearly 4% of the images wait for a year or more before I get around to stitching them together!

The number of days on which I photograph panoramic/stitched images has varied considerably, but has averaged about 90 days per year. I don't have statistics on the amount of time spent photographing, but my guess is that the duration of an "average" shooting session might last for an hour or two.

There are more days of the year that I spend assembling images than photographing. Over the last four years, I have averaged about 190 days per year on which I spend some time assembling images. I don't have statistics on the amount of time spent working on images, but on most days I only spend a few minutes assembling one or two panoramas, often while checking e-mail or performing some other activity on the computer.

Image Sizes and Counts

The average size (in megapixels) of the panoramic/stitched images I create has been increasing over time. Although I created my first gigapixel image (1000 megapixels) in 2003, the average image I created in that was year was just under 60 megapixels. By 2010, the average image size had increased to 132 megapixels.

To a large extent, the increase in the image size has been driven by an increase in the sensor resolution of the cameras I have used, rather than an increase in the number of component images per panorama. The next chart shows that I have been fairly consistent in the number of component images taken to create each stitched/panoramic image. The average number of component images for each panorama is just over 10.

The next chart shows the distribution of the number of components images used to create each panorama. Although the average is about 10, a large number of my panoramas are assembled using only 3-7 images (often a "single row" panorama consisting of 5 or 6 images). Only about 2 percent of my panoramas are comprised of 50 or more component images.

The next chart shows the distribution of the number of megapixels in the assembled panoramas. The average is about 100 megapixels, although that number has been increasing, as I have upgraded to higher resolution cameras over time. Although the average panorama size is about 100 megapixels, about 75 percent of the panoramas I have created contain less than 100 megapixels, and only about 9 percent contain more than 200 megapixels.

Lenses, Projections and Aspect Ratios

I am often asked what is the "best" focal length lens to use for creating panoramic images. Many beginners assume that they need a wide-angle (i.e. short focal length) lens to create panoramic images. Other assume that they shouldn't use a wide-angle lens because it has too much "distortion". Neither assumption is correct. Using a wide-angle lens just means that you'll be able to capture any given scene with fewer component images. And, this means that you'll have less stitching to do, but will end up with an image with fewer megapixels (all else being equal). This next chart shows the distribution of focal lengths that I use when shooting panoramas. Most of my panoramas are shot with focal lengths between 17mm and 75mm, in large part reflecting the fact that my two favorite lenses cover this range. However, I also shoot quite often with longer focal lengths (sometimes with focal lengths over 300mm), particularly when assembling very high resolution images comprised of lots of component images.

Related to the focal length of the lens, is the field of view that the final panorama encompasses. The next chart shows that the panoramas I create vary considerably in their field of view...all the way from less than 10 degrees, up to 360 degrees. The average field of view is almost 120 degrees. This is quite a lot wider than the field of view captured in a single image using conventional lenses. For example, a "normal" lens (e.g. 50mm on a "full-frame" SLR) only covers a field of view of about 40 degrees, and even a very wide lens (like a 17mm lens on a "full-frame" SLR) only covers a field of view of about 93 degrees. Almost 60 percent of my panoramas capture a field of view that is more than 100 degrees. Capturing images with such wide field of view is both an advantage and a challenge: It can lead to unusual and not-often-copied images, but can be difficult to compose and capture.

Just as capturing images with a wide field of view can be a challenge, so can stitching them to look visually pleasing. One of the techniques that I use is to choose different projections depending on the scene and its field of view. Most conventional camera lenses use a rectilinear projection to ensure that straight lines in the scene are rendered as straight in image that they produce. I also use this projection when creating panoramic images, but relatively rarely...only about 15 percent of the time. One of the drawbacks to this projection is that it produces images that appear increasingly stretched at the edges as the field of view increase (i.e. as the focal length decreases). Beyond 180 degrees rectilinear projection is a mathematical impossibility as the amount of stretching becomes infinite, but even at fields of view around 100 or 120 degrees, the appearance of this stretching at the edges can become disconcerting. As a result, I often use different projections. For many years, I used cylindrical projection very frequently, but once I invented rectilindrical projection, it soon became my favorite projection for handling scenes that were too wide for rectilinear projection. In many cases, it allows me to render a scene with a view wide field of view but have it appear to the casual observer as if it were taken with a standard rectilinear lens because of the way it can diminish some of the tell-tale visual signs of cylindrical projection.

Although I often refer to my stitched images as "panoramic" images, I should probably be more precise in my terminology. Many people think of panoramic images as covering a very wide field of view and/or having a "letterbox" shape (i.e. a large "aspect-ratio" where the ratio of the width to height is large). For a lot of my images both of these (large field of view and large aspect-ratio) are true, but not all. The chart below shows the distribution of aspect-ratios for the images I create. The average aspect ratio is about 2.64, meaning the width of the image is 2.64 times as large as the height of the image. However, many of the images I take are closer to much more "conventional" aspect-ratios of 4:3 (i.e. 1.33), 3:2 (i.e. 1.5) or 16:9 (i.e. 1.78). On occasion, I create images with an aspect ratio of more than 4:1, but in general I find it difficult to create visually pleasing images with an aspect ratio of much more than 4:1.

The Big Picture

Finally, the last chart (below) shows one dot for each panorama I have taken. The date on which the panorama was taken is shown on horizontal axis (x-axis), and the size of the panorama in pixels is shown on the vertical (y-axis). I have truncated the chart to only show panoramas with less than a billion pixels (i.e. a gigapixel) in order to keep the chart to a reasonable height. The red line shows the trend in pixel size over time, sloping upwards to reflect the fact that the panoramas I create have been increasing in size, on average, over time.