Intermodal image-based recognition of planar kinematic mechanisms. (April 2015)

• Record Type: Journal Article
• Authors: Eicholtz, Matthew; Burak Kara, Levent
• Is Part Of: Journal of visual languages & computing. Volume 27(2015)
• Page Start: 38
• Page End: 48
• Publication Date: 2015-04
• Subjects: Computer vision -- Evolutionary multiobjective optimization -- Kinematic simulation -- Object recognition; Visual programming languages (Computer science) -- Periodicals; Visual programming (Computer science) -- Periodicals; Programming languages (Electronic computers) -- Semantics -- Periodicals; Langages de programmation visuelle -- Périodiques; Programmation visuelle -- Périodiques; Langages de programmation -- Sémantique -- Périodiques; Programming languages (Electronic computers) -- Semantics; Visual programming (Computer science); Visual programming languages (Computer science); Periodicals; Electronic journals; 005
• DOI: 10.1016/j.jvlc.2014.10.024
• Languages: English
• ISSNs: 1045-926X

Abstract

Abstract: We present a data-driven exploratory study to investigate whether trained object detectors generalize well to test images from a different modality. We focus on the domain of planar kinematic mechanisms, which can be viewed as a set of rigid bodies connected by joints, and use textbook graphics and images of hand-drawn sketches as input modalities. The goal of our algorithm is to automatically recognize the underlying mechanical structure shown in an input image by leveraging well-known computer vision methods for object recognition with the optimizing power of multiobjective evolutionary algorithms. Taking a raw image as input, we detect pin joints using local feature descriptors in a support vector machine framework. Improving upon previous work, detection confidence depends on multiple context-based classifiers of varying image patch size and greedy foreground extraction. The likelihood of rigid body connections is approximated using normalized geodesic time, and NSGA-II is used to evolve optimal mechanisms using this data. The present work is motivated by the observation that textbook diagrams and hand-drawn sketches of mechanisms exhibit similar object structure, yet have different visual characteristics. We apply our method using various combinations of images for training and testing, and the results demonstrate a trade-off between solvability and accuracy. Highlights: We describe a system that automatically recognizes mechanical structures from images. Contextual features and foreground extraction enhance previous detection results. We extend the recognition pipeline to include hand-drawn sketches as input. Trained object detectors can perform well on images from a different modality.