Domain bias in distinguishing Flemish and Dutch subtitles. (15th September 2020)

• Record Type: Journal Article
• Authors: van Halteren, Hans
• Is Part Of: Natural language engineering. Volume 26:Part 5(2020)
• Page Start: 493
• Page End: 510
• Publication Date: 2020-09-15
• Subjects: Text classification, -- Dutch, -- Methodology, -- Dialect recognition, -- Topic bias; Natural language processing (Computer science) -- Periodicals; Software engineering -- Periodicals; 006.35
• DOI: 10.1017/S1351324919000445
• Languages: English
• ISSNs: 1351-3249

Abstract

Abstract: This paper describes experiments in which I tried to distinguish between Flemish and Netherlandic Dutch subtitles, as originally proposed in the VarDial 2018 Dutch–Flemish Subtitle task. However, rather than using all data as a monolithic block, I divided them into two non-overlapping domains and then investigated how the relation between training and test domains influences the recognition quality. I show that the best estimate of the level of recognizability of the language varieties is derived when training on one domain and testing on another. Apart from the quantitative results, I also present a qualitative analysis, by investigating in detail the most distinguishing features in the various scenarios. Here too, it is with the out-of-domain recognition that some genuine differences between Flemish and Netherlandic Dutch can be found.