Mathematical Problems in Industry Workshop

	Home \| Olin College

In cooperation with SIAM

Mosaic on Map Generation for a Forward Looking Camera

Industrial Presenter: Roger Berry, AMRDEC, Redstone Arsenel

AMREC has developed and successfully demonstrated mosaic software using a small Unmanned Aerial Vehicle (UAV) with a 30° Field of View down-looking camera. Small UAVs of this type normally fly at an altitude of 100 to 500 feet with a forward velocity of 30 to 60 mph. Their endurance time ranges from 30 to 80 minutes and they have a typical range of 6 miles. In currently fielded UAVs a streaming video from the UAV camera is seen on the ground station display. AMRDEC’s mosaic software stitches images seen by the UAV camera providing a composite map of an area of interest as shown in Figure 1. In general, mosaic is the process of combining multiple images to form a single image, or image mosaic. The image mosaic process involves geometrically aligning single images to create a composite image of a larger area. AMRDEC’s mosaic software provides an almost real-time, composite map of all the images seen by the UAV camera. This non-georeferenced map can be georeferenced using a satellite map of the area and either manually or automatically matching geometric features. The end result is that the UAV operator has a current, geo-referenced map, of all the area seen by the UAV camera. The operator can then zoom in on areas of interest contained on the mosaic map.

The problem needing a solution is a small UAV with a forward looking camera mounted at a look-down angle of approximately 30 degrees from horizontal. Many small UAVs currently fielded by the military are of this type. Adding a down-looking camera to fielded UAVs would be an easy solution that would facilitate AMRDEC’s current mosaic software capabilities. However, to retrofit all the fielded systems would be very expense.

Figure 1: Forming a mosaic from a sequence of still images. This example shows the stitching for a series of planar images. The current problem is to develop an algorithm to take images with different perspectives of the same object within the stills and form a two-dimensional mosaic.

When a UAV with a forward-looking camera flies over a building the camera initially sees the side of the building, followed by the side and top of the building, and finally the top of the building only. This sequence generates a problem for AMREC’s mosaic software. Images containing both the side and top of the building can be mosaiked by the process of image matching. However, the mosaic software cannot distinguish between the side and top of the building, making it impossible to translate the image into a down-looking view. Since the image cannot be translated into a down looking view, it is impossible to tie the mosaic image to a georeferenced down-looking satellite map. What is desired is the development of a perspective correction routine that will allow images taken from a forward looking camera to be translated into a down looking camera view. The down-looking camera images could then be mosaicked and geo-referenced to satellite images.

One approach to the forward-looking camera problem is to use a 3-Dimensional reference model of the area being surveyed. The 2-Dimensional mosaic image can then be matched to the 3-Dimensional reference model using permanent structures in the image. Once this is done, a real-time down looking view can be obtained from the mosaic image. The problem with this approach is that 3-Dimensional map information is not available for many areas of interest

In solving the forward-looking camera problem it may be assumed that the position of the UAV is know to within the accuracy of commercially available GPS.
Useful References

Pyramid Vision Incorporated http://www.pyramidvision.com

Mediaware Solutions http://www.mediaware.com

Kalman-filter Based Method for Creation of Super-resolved Mosaicks,” Bryce B. Ready, Clark N. Taylor, and Randal W. Beard, Electrical and Computer Engineering, Brigham Young University

“University Applications Using a New Tool to Mosaic and Georeference Imagery: “New Tool = DIME Software,” Dale R. Johnson, Positive Systems Inc. Whitefish, MT

“Map Construction for Mosaic-Based Vehicle Position Estimation,” Patrick Rowe and Alonzo Kelly, Robotics Institute, Carnegie Mellon University