PTAPicker

Introduction. PTAPicker is a Windows command line program that detects control points (common features in overlapping images) from a set of input images. It is part of (and distributed with PTAssembler). PTAPicker offers the user a choice of several different detection algorithms, none of which are encumbered with legal patents. This page gives an overview of PTAPicker, its program options and presents a comparison with some other control point detector programs.

PTAssembler showing 2 images with control points detected by PTAPicker.

Overview of Control Point Detection Algorithm. PTAPicker takes a set of images as input, and produces a file with a list of control points that can be opened by PTAssembler (and possibly other programs) as its output. In order to understand some of PTAPicker's options, a basic understanding of how PTAPicker's control point detection algorithm works is useful. In brief, the algorithm works follows:

Input -> Feature detection -> Description -> Matching -> Filtering/Checking -> Thinning -> Output

The main steps of the algorithm are described below:

1. Feature Detection. In this stage, PTAPicker scans each image, and looks for "features". A feature could be any point in the image, but in this context refers to a point that is distinctive in some fashion. For example, a point that is much brighter than adjacent points (such as a star in the sky, or a point of light in a dark scene), might be a good feature. PTAPicker offers the ability to choose from several different types of feature detectors, and these detectors all identify different regions as features. The number of features that are detected depend on the size of the image, the subject matter, the type of detector used and its configuration. In some cases, several thousand features may be detected; in others only a handful (or none) may be detected.
2. Feature Description. Having detected a number of features in each image, PTAPicker "describes" the features. Like the detection stage, this can also be done in a number of different ways, and PTAPicker offers several different "descriptors" to perform this task. All of the descriptors "describe" a feature using a sequence of numbers. Ideally, each description should be sufficiently unique as to allow for easy differentiation between descriptions. Ideally, descriptors would be similar to telephone numbers in that no two people can share the same number. In practice, descriptors are not so unique, and different features are sometimes described in very similar ways. Conversely, the same feature that appears in two different images may be described quite differently depending on how each image is scaled, rotated, illuminated, distorted, etc.
3. Feature Matching. Once all of the features in all images have been described, this stage attempts to find matching features in pairs of images. PTAPicker compares each image against every other image in the input set to look for matches, i.e. features that are characterized by the same (or very similar) descriptors. If two images overlap, then one would expect to find some features in the two images that are described the same way (or very close to the same way). When these matching features are found they are recorded as matches or "candidate control points".
4. Filtering and Checking. The next stage of the algorithm is to validate the candidate control points, and reject any candidate control points that are likely to be incorrect. In practice, feature description and matching is a less-than-perfect science, and sometimes two different features may be described sufficiently similarly that they end up being "matched" to each other and identified as a candidate control point. Because a computer does not see images that same way as a human, the computer may identify matches that are obviously incorrect to a human (e.g. matching a feature on a car in one image to a feature on a building in another image). The filtering and checking stage applies a number of different tests to try and determine if any of the candidate control points are incorrect and should be eliminated. One important test that PTAPicker applies is called RANSAC. It attempts to determine which candidate control points are incorrect by looking for control points that appear to be positioned in regions of the image that are distant from control points that appear to be positioned correctly.
5. Thinning. The last stage is only used if the number of control points that are produced after the previous stage exceeds the number requested by the user. If so, some control points are discarded. PTAPicker discards control points that are close to each other, trying to keep control points that are spread across different areas of the image.

PTAPicker Usage and Options. PTAPicker can be used by PTAssembler or invoked directly from the command line. Program usage and options are summarized here. More information about each option is presented later in the document.

 Usage:

  PTAPicker [options] [input files]

 Options:

  -c       Use color descriptors
  -d NAME  Specify detector name (see below)
  -D NAME  Specify descriptor name (see below)
  -f n     Maximum number of features to retain after detection (e.g. 2048)
  -g n     Number of regions each axis is subdivided before matching (e.g. 1)
  -i n     Number of times to downsize image before processing (e.g. 2)
  -j n     Downsize until longer dimension is less than n pixels (e.g. 1200)
  -k n     Maximum number of control points to create per image pair (e.g. 30)
  -m NAME  Specify matcher name (see below)
  -n n     Number of threads to execute in parallel (e.g. 2)
  -o FILE  Specify output filename
  -p FILE  Specify input PTAssembler project filename
  -q n     Number of matches to use for RANSAC
  -r RECT  Rectangular area specifying sub-region of image (see below)
  -t n     Maximum allowable pixel error during RANSAC (e.g. 3.0)
  --a n    Maximum control point angular change in degrees (e.g. 20)
  --c n    Maximum homography angular change in degrees (e.g. 20)
  --l n    Maximum homography line length ratio (e.g. 1.2)
  --p n    Maximum homography perspective (e.g. 0.01)
  --r n    Maximum homography rectangle area ratio (e.g. 6.0)
  --t n    Feature Detection Threshold (e.g 20)
  --x n    RANSAC threshold expansion factor (e.g. 3.0)

 Notes:

  * Input files can be images (e.g. JPEG, TIFF) or (using the -p option) a
    PTAssembler project file that contains a list of images.

  * Options -i or -j cannot be specified together.  If so, -i is ignored.

  * The -r RECT option can be used to specify a region-of-interest (ROI)
    within an image to process.  If specified, only the ROI is searched
    for control points.  A RECT is specified as follows (see below for
    usage example):

      x:y:width:height:imageNumber

    where:

      x = The x coordinate of the top-left corner of the ROI
      y = The y coordinate of the top-left corner of the ROI
      width = The width of the ROI
      height = The height of the ROI
      imageNumber = The image number for the ROI (images are numbered
        in the order in which they are present on the command line or
        project file, starting at number zero).

    If the imageNumber parameter is missing, the ROI is applied to all
    images specified on the command line or in the project.


 Usage Examples:

   PTAPicker -o out.txt image_a.jpg image_b.jpg
   PTAPicker -o out.txt -p ptasmblr.ptp
   PTAPicker -d ORB -D BRISK -o out.txt -n 4 -k 50 -p ptasmblr.ptp
   PTAPicker -o out.txt -r 900:0:300:200:0 -r 0:0:300:200:1 img0.tif img1.tif


 Detector Names:

   BRISK: Binary Robust Invariant Scalable Keypoints
   Dense: Dense Feature Detector
   FAST: Features from Accelerated Segment Test
   GFTT: Good Features To Track Detector
   HARRIS: Good Features To Track Detector with Harris detector enabled
   MSER: Maximally Stable Extremal Regions
   ORB: Oriented FAST and Rotated BRIEF
   STAR: StarFeatureDetector
   SimpleBlob: Simple Blob Detector

 Descriptor Names:

   BRIEF: Binary Robust Independent Elementary Features
   BRISK: Binary Robust Invariant Scalable Keypoints
   FREAK: Fast Retina Keypoint
   ORB: Oriented FAST and Rotated BRIEF

 Matcher Names:

   BruteForce: (L2 Norm)
   BruteForce-L1: (L1 Norm)
   BruteForce-Hamming: Hamming distance
   BruteForce-Hamming(2): Hamming distance (used with ORB when WTA_K==3)
   FlannBased: Fast Library for Approximate Nearest Neighbors

• -c. By default, PTAPicker converts input image to greyscale before detecting features. If this option is specified, then the program also uses color information. This can lead to more distinctive features being detected, but can also slow program operation.
• -d NAME. The NAME parameter specifies the feature detector algorithm to use. By default, PTAPicker uses the BRISK detector.
• -D NAME. The NAME parameter specifies the feature descriptor algorithm to use. By default, PTAPicker uses the BRISK descriptor.
• -f n. Maximum number of features to retain after detection (e.g. 500). If more than this number of features were detected, features with the weakest response are removed until this number of detected features remains.
• -g n. If a value of more than 1 is specified, each is image is subdivided into multiple regions before matching, and each region of each image is compared against every other region in every other image. The image is divided into regions along both axes (i.e. width and height). This slows down processing, but usually results in more matches being found, particularly in images with minimal overlap, or repetitive patterns.
• -i n. The n parameter specifies the number of times by which to downsize the image by 50% before detecting features. downsizing the image size reduces the number of features that are detected, but speeds up the program.
• -j n. If specified, PTApicker will downsize the image by 50% repeatedly until the longer dimension is less than the value specified by n.
• -k n. The maxmum number of control points to keep (after thinning).
• -m NAME. the NAME parameter specifies the feature matching algorithm to use. By default, PTAPicker chooses the most appropriate algorithm to use depending on the detector and descriptor algorithms. The FlannBased algorithm is considerably faster than the BruteForce Algorithms, but is not quite as accurate. In some cases, the FlannBased algorithm may incorrectly match some features. The "Hamming" variants of the BruteForce algorithms should only be used with description algorithms that use binary strings (e.g. BRISK, BRIEF, FREAK, ORB). Some other description algorithms (e.g. SIFT, SURF which are not included in this program due to patent restrictions) require the older "non-hamming" variants that are included here for sake of completeness. In general, these variants are not useful.
• -n n. The n parameter specifies the maxiumum number of threads to execute in parallel. By default, PTAPicker chooses an amount based on your machine hardware, so you typically should not need to adjust this parameter.
• -o FILE. Specifies the name of the file that PTAPicker outputs containing resulting control point information.
• -p FILE. Specifies the name of a PTAssembler project file containing image information. this is an alternative to specifying image names directly on the command line.
• -q n. Specifies the number of matches that are used as input to RANSAC testing. All matches are ordered by how uniquely the descriptors match. If more than the number of matches specified by this parameter are found during the matching stage, weaker/poorer matches are discarded.
• -r RECT. This option can be used to specify a region-of-interest (ROI) within an image to process. If specified, only the ROI is searched for control points. A RECT is specified as follows:
  
  x:y:width:height:imageNumber
  
  Where:
  
  
  • x = The x coordinate of the top-left corner of the ROI
  • y = The y coordinate of the top-left corner of the ROI
  • width = The width of the ROI
  • height = The height of the ROI
  • imageNumber = The image number for the ROI (images are numbered in the order in which they are present on the command line or project file, starting at number zero). If the imageNumber parameter is missing, the ROI is applied to all images specified on the command line or in the project.
• -t n. The n parameter specifies the maximum allowable pixel error during RANSAC testing. Larger values retain more candidate control points, but increase the chance of inadvertently retaining incorrect control points.
• --a n. PTAPicker looks for incorrectly positioned control points by calculating the angles between sets of 3 control points in image pairs. The n value specifies the maximum difference in angles between images pairs before a control point is flagged as suspicious and possibly rejected. Larger values retain more control points, at the expense of increasing the chance of retaining incorrect points.
• --c n. PTAPicker analyzes the homography martix calculated during RANSAC testing. If the angles in a small region transformed by the homography matrix exceed this threshold, then all candidate control points for this image pair are rejected. Larger values retain more control points, at the expense of increasing the chance of retaining incorrect points.
• --l n. Similar to the --c option, but this tests the ratio of line lengths in a small region transformed by the homography matrix. Larger values retain more control points, at the expense of increasing the chance of retaining incorrect points.
• --p n. Similar to the --c option, but this tests the absolute values of the sums of the two perspective parameters in the homography matrix. Larger values retain more control points, at the expense of increasing the chance of retaining incorrect points.
• --r n. Similar to the --c option, but this tests that ratio of the area encompassed by a small region before and after transformation by the homography martix. Larger values retain more control points, at the expense of increasing the chance of retaining incorrect points.
• --t n. This value can be used to adjust the sensitivity of the feature detector. (This option only works with the BRISK, FAST, GFTT, HARRIS and STAR detectors). Higher values will result in fewer features being detected. Lower values result in more features being detected, but these features are less "distinct", and are more likely to produce incorrect matches. Legal values range from 1 to 255, but values between 10-100 are likely to yield the best results.
• --x n. This parameter adjusts the RANSAC threshold specified by the -t option by an amount that is a function of a point's distance from the central position of all points that passed RANSAC. This is useful for images that do not exhibit perfect rectilinear projection (e.g. images with distortion or fisheye projection). In these cases, higher values for this option will typically cause PTAPicker to retain more points near the edges and corners of the images.

Helpful Hints and Information. PTAPicker's central task is accurately identifying valid control points while simultaneously rejecting incorrectly identified invalid control points. And, if possible, do it quickly! Almost all of PTAPicker's options are targeted towards adjusting the tradeoff between the acceptance of valid control points, the rejection of invalid control points, and the time spent doing so. PTAPicker's default choices should work well for most "real-world" images, but the information in this section should allow you to make adjustments if the results are not satisfactory.

• The -v option makes PTAPicker print information about the number of features detected, matched and tested. This information should help you to make informed decisions about which other options may need adjustment.
• The number of features that are detected depends on the size of the input images after resizing (small images tend to produce fewer features), the image subject matter (low contrast images tend to produce fewer features) and the chosen feature detector. PTAPicker works best when at least several hundred features are detected for each image. To increase the number of features, use a lower detection threshold (--t option) and/or larger images (-i or -j options). Note that using a lower detection threshold and/or using larger images slows program execution.
• The matching stage works quicker when fewer features are used. To change the number of features used by the matching stage, use the -f option. Features with the weakest response are discarded first.
• The filterting and checking stage works best with a limited number of matches or "candidate control points". The -q option governs how many matches are used during filtering/testing, by retaining only the "best" matches. Using too many matches during the filtering/testing stage not only slows the program operation, but can also increase the chance of bad results as the RANSAC algorithm may fail to distinguish the valid control points from the invalid candidate control points.
• The --a, --c, --l, --p, --r all govern how strictly PTAPicker analyzes candidate control points during the filtering and checking stage. Higher values cause PTAPicker to reject fewer candidate control points at the expense of increasing the chance of retaining incorrect points.
• If the input images have distortion or were taken with a fisheye lens, use the --x option to allow PTAPicker to be more lenient about positioning errors near the edges and corners of images.
• PTAPicker chooses the most appropriate matching algorithm by itself. Newer feature description algorithms (e.g. BRISK, FREAK, BRIEF, ORB) use "binary strings" as descriptors which allow for very quick and efficient matching using the Hamming-Distance. Only the older feature detection algorithms (e.g. SIFT,SURF) algorithms require the non-Hamming matchers, and although these matchers will work with the newer binary string descriptors, they are available in PTAPicker primarily for sake of completeness.
• Matching speed can be increased considerably by using the FlannBased matcher. This matcher can be much faster than the brute-force methods, but is not quite as accurate as the brute-force methods. In most cases, this doesn't matter, as the filtering and checking stage will eliminate any incorrectly idenfitied matches produced by the FlannBased Matcher. However, in a small number of cases, the FlannBased matcher can produce just enough incorrect matches that control points in an image pair are rejected as being invalid. The choice of using the FlannBased matcher is one of speed versus accuracy.
• If no control points are detected in images which are known to overlap, using the -g option to sub-divide the images into regions before matching can often help PTAPicker find control points. Values of 2 or 3 are often useful. Note that this slows program operation, and may result in a higher chance of invalid matches being detected.
• Similarly, using the -r option to specify a sub-region of the image to process can help PTAPicker find control points in image pairs for which no control points are found.