Developing and applying a benchmark for evaluating image stylization. (October 2017)

• Record Type: Journal Article
• Authors: Mould, David; Rosin, Paul L.
• Is Part Of: Computers & graphics. Volume 67(2017)
• Page Start: 58
• Page End: 76
• Publication Date: 2017-10
• Subjects: Non-photorealistic rendering -- Image processing -- Image stylization -- Abstraction -- Evaluation -- Benchmarks; Computer graphics -- Periodicals; 006.6
• DOI: 10.1016/j.cag.2017.05.025
• Languages: English
• ISSNs: 0097-8493

Abstract

Highlights: Proposing a more systematic approach to image stylization evaluation, with a common set of images. Presenting a 20-image benchmark for helping NPR practitioners evaluate their methods. Demonstrating the use of the benchmark set with six existing image stylization methods. Graphical abstract: Abstract: The non-photorealistic rendering community has had difficulty evaluating its research results. Other areas of computer graphics, and related disciplines such as computer vision, have made progress by comparing algorithms' performance on common datasets, or benchmarks . We argue for the benefits of establishing a benchmark image set to which image stylization methods can be applied, simplifying the comparison of methods, and broadening the testing to which a given method is subjected. We propose a set of benchmark images, representing a range of possible subject matter and image features of interest to researchers, and we describe the policies, tradeoffs, and reasoning that led us to the particular images in the set. Then, we apply six previously existing stylization algorithms to the benchmark images; we discuss observations arising from the interactions between the algorithms and the benchmark images. Inasmuch as the benchmark images were able to thoroughly exercise the algorithms and produce new material for discussion, we can conclude that the benchmark will be effective for its stated aim.