Emerging trends: Deep nets for poets. (September 2021)

• Record Type: Journal Article
• Authors: Church, Kenneth Ward; Yuan, Xiaopeng; Guo, Sheng; Wu, Zewu; Yang, Yehua; Chen, Zeyu
• Is Part Of: Natural language engineering. Volume 27:Part 5(2021)
• Page Start: 631
• Page End: 645
• Publication Date: 2021-09
• Subjects: Deep nets -- training -- inference -- hubs -- pretrained models; Natural language processing (Computer science) -- Periodicals; Software engineering -- Periodicals; 006.35
• DOI: 10.1017/S1351324921000231
• Languages: English
• ISSNs: 1351-3249

Abstract

Abstract: Deep nets have done well with early adopters, but the future will soon depend on crossing the chasm. The goal of this paper is to make deep nets more accessible to a broader audience including people with little or no programming skills, and people with little interest in training new models. A github is provided with simple implementations of image classification, optical character recognition, sentiment analysis, named entity recognition, question answering (QA/SQuAD), machine translation, speech to text (SST), and speech recognition (STT). The emphasis is on instant gratification. Non-programmers should be able to install these programs and use them in 15 minutes or less (per program). Programs are short (10–100 lines each) and readable by users with modest programming skills. Much of the complexity is hidden behind abstractions such as pipelines and auto classes, and pretrained models and datasets provided by hubs: PaddleHub, PaddleNLP, HuggingFaceHub, and Fairseq. Hubs have different priorities than research. Research is training models from corpora and fine-tuning them for tasks. Users are already overwhelmed with an embarrassment of riches (13k models and 1k datasets). Do they want more? We believe the broader market is more interested in inference (how to run pretrained models on novel inputs) and less interested in training (how to create even more models).