Study and Implementation of an Efficient Method for Multi-Row Image Stitching
Supervisors:
Prof. Dr.-Ing. Klaus-Dieter Kuhnert
Dipl.-Inform. Klaus Müller
Submitted by:
Hardik Karelia
Matrikel Nr: 1059568
Branch: Master Studies, Mechatronics
Description
Nowadays smartphones and tablets have the ability to create panoramic images. The purpose of this thesis is to investigate and to look for an efficient technique to develop an application for automatic multi-row image stitching for the generation of panoramas. Through the use of modern algorithms and methods it is now possible to do this task by taking two or more pictures from an image capturing device and stitching them together to form a panorama with relatively larger field of view. OpenCV is an open source software library which allows us to do the integration of the image stitching and other related operations with relative ease. In this thesis an interactive application is presented which allows a user to experiment with image stitching process by giving options to select the different algorithms for various steps involved in the stitching process.
Related Work and Results
An application was developed to achieve the goal of multi-row image stitching for both Android and Linux Operating System. For the Android Application,
the front-end was programmed in JAVA and the back-end which consists of core stitching functions was programmed in C++. Figure 1 shows
the program architecture of the Android Application. Figure 2 shows the Java Native Interface (JNI) functioning and all related stitching methods were natively programmed
in C++ using OpenCV library. A module for stitching the images taken by translating camera in horizontal and vertical directions was developed as OpenCV stitching module only supports stitching of images
taken by rotating camera model. Translation camera parameters were estimated and optimized using Levenberg-Marquadt Algorithm.
Note:
Click on Images for Better View
Click here to view results of translational panoramas and here to view results of rotational panoramas.
Go to bottom of the page
Fig 1: Android Application Architecture
Fig 2: JNI Interface
Figure 3 shows the main UI of the Android Application and the figure 4 shows the Preferences Dialog, through which user can select different algorithms for each step involved in the stitching process. All the available settins are shown in the Table 1.
Fig 3: User Interface.
Fig 4: Preferences Dialog where user can select different settings for stitching process.
| Setting | Description |
|---|---|
| Compositing Resolution | Resolution for compositing step. |
| Warp Surface Type | Applies the chosen perspective transformation to all final panorama |
| Feature Detector Type | Type of algorithm to be used to detect features in the images |
| Feature Detector Type | Type of algorithm to be used to detect features in the images |
| Blending Type | Type of algorithm to be used to blend the overlapping regions of images |
| Bundle Adjuster Cost Function | Type of Cost Function to be used for the optimization of estimated camera intrinsic and extrinsic parameters |
| Seam Estimation Resolution | Resolution of images for seam estimation step |
| Seam Estimation Method | Type of method to be used to estimate seam in overlapping regions |
| Registration Resolution | Resolution of images for registration step |
| Feature Matching Confidence Threshold | Matching confidence threshold for filtering out unwanted matches |
| Confidence Threshold | Threshold for two images are from the same panorama confidence |
| Type of Camera Motion | Set the type of movement of camera while capturing images |
| Video Matching Mode | Set the no. of consecutive images to be matched with current image |
| Wave Correction | Removes the wavy effect of the resultant panorama |
Table 1: Preferences
Figure 5 shows the final output panorama taken by a translating camera in horizontal and vertical directions stitched using the application developed for Linux. Figure
6 shows the graph Tx vs Ty (in pixels) of the optimized and estimated translational camera parameters.
Panorama Image Size: 2626 x 5781 = 20 Megapixels
Time Taken: 560 sec
Fig 5: Translational Panorama, Images taken infront of F-Building, Hölderlin Campus, University of Siegen.
Fig 6: Estimated camera translations for above panorama
Fig 7: Panorama created using the Android Application
Fig 8: Panorama generated from the images taken from UAV, F and E Building, Hölderlin Campus, University of Siegen.
Various other panorama outputs for rotational camera model:
Go to Top of the Page
Fig 9: Rotational Panorama, Geisweid, Final Panorama Image Size:16881 x 4784 = 77 Megapixels
Fig 10: Rotational Panorama, Meiswinkel, Final Panorama Image Size: 28997 x 6725 = 186 Megapixels,
Fig 11: Preview Panorama (Reduced Size) - Images taken from Monte Schlacko near University of Siegen Final Panorama Image Size: 61040 x 8810 = 512 Megapixels