An overview of the BIOASQ large-scale biomedical semantic indexing and question answering competition. Issue 1 (December 2015)
- Record Type:
- Journal Article
- Title:
- An overview of the BIOASQ large-scale biomedical semantic indexing and question answering competition. Issue 1 (December 2015)
- Main Title:
- An overview of the BIOASQ large-scale biomedical semantic indexing and question answering competition
- Authors:
- Tsatsaronis, George
Balikas, Georgios
Malakasiotis, Prodromos
Partalas, Ioannis
Zschunke, Matthias
Alvers, Michael
Weissenborn, Dirk
Krithara, Anastasia
Petridis, Sergios
Polychronopoulos, Dimitris
Almirantis, Yannis
Pavlopoulos, John
Baskiotis, Nicolas
Gallinari, Patrick
Artiéres, Thierry
Ngomo, Axel-Cyrille
Heino, Norman
Gaussier, Eric
Barrio-Alvers, Liliana
Schroeder, Michael
Androutsopoulos, Ion
Paliouras, Georgios - Abstract:
- Abstract Background This article provides an overview of the firstBioASQ challenge, a competition on large-scale biomedical semantic indexing and question answering (QA ), which took place between March and September 2013.BioASQ assesses the ability of systems to semantically index very large numbers of biomedical scientific articles, and to return concise and user-understandable answers to given natural language questions by combining information from biomedical articles and ontologies. Results The 2013BioASQ competition comprised two tasks, Task 1a and Task 1b. In Task 1a participants were asked to automatically annotate newPubMed documents withMeSH headings. Twelve teams participated in Task 1a, with a total of 46 system runs submitted, and one of the teams performing consistently better than theMTI indexer used byNLM to suggestMeSH headings to curators. Task 1b used benchmark datasets containing 29 development and 282 test English questions, along with gold standard (reference) answers, prepared by a team of biomedical experts from around Europe and participants had to automatically produce answers. Three teams participated in Task 1b, with 11 system runs. TheBioASQ infrastructure, including benchmark datasets, evaluation mechanisms, and the results of the participants and baseline methods, is publicly available. Conclusions A publicly available evaluation infrastructure for biomedical semantic indexing andQA has been developed, which includes benchmark datasets, and canAbstract Background This article provides an overview of the firstBioASQ challenge, a competition on large-scale biomedical semantic indexing and question answering (QA ), which took place between March and September 2013.BioASQ assesses the ability of systems to semantically index very large numbers of biomedical scientific articles, and to return concise and user-understandable answers to given natural language questions by combining information from biomedical articles and ontologies. Results The 2013BioASQ competition comprised two tasks, Task 1a and Task 1b. In Task 1a participants were asked to automatically annotate newPubMed documents withMeSH headings. Twelve teams participated in Task 1a, with a total of 46 system runs submitted, and one of the teams performing consistently better than theMTI indexer used byNLM to suggestMeSH headings to curators. Task 1b used benchmark datasets containing 29 development and 282 test English questions, along with gold standard (reference) answers, prepared by a team of biomedical experts from around Europe and participants had to automatically produce answers. Three teams participated in Task 1b, with 11 system runs. TheBioASQ infrastructure, including benchmark datasets, evaluation mechanisms, and the results of the participants and baseline methods, is publicly available. Conclusions A publicly available evaluation infrastructure for biomedical semantic indexing andQA has been developed, which includes benchmark datasets, and can be used to evaluate systems that: assignMeSH headings to published articles or to English questions; retrieve relevantRDF triples from ontologies, relevant articles and snippets fromPubMed Central; produce "exact" and paragraph-sized "ideal" answers (summaries). The results of the systems that participated in the 2013BioASQ competition are promising. In Task 1a one of the systems performed consistently better from theNLM 'sMTI indexer. In Task 1b the systems received high scores in the manual evaluation of the "ideal" answers; hence, they produced high quality summaries as answers. Overall, BioASQ helped obtain a unified view of how techniques from text classification, semantic indexing, document and passage retrieval, question answering, and text summarization can be combined to allow biomedical experts to obtain concise, user-understandable answers to questions reflecting their real information needs. … (more)
- Is Part Of:
- BMC bioinformatics. Volume 16:Issue 1(2015)
- Journal:
- BMC bioinformatics
- Issue:
- Volume 16:Issue 1(2015)
- Issue Display:
- Volume 16, Issue 1 (2015)
- Year:
- 2015
- Volume:
- 16
- Issue:
- 1
- Issue Sort Value:
- 2015-0016-0001-0000
- Page Start:
- 1
- Page End:
- 28
- Publication Date:
- 2015-12
- Subjects:
- BioASQ Competition -- Hierarchical Text Classification -- Semantic indexing -- Information retrieval -- Passage retrieval -- Question answering -- Multi-document text summarization
Bioinformatics -- Periodicals
Computational biology -- Periodicals
570.285 - Journal URLs:
- http://www.biomedcentral.com/bmcbioinformatics/ ↗
http://www.pubmedcentral.nih.gov/tocrender.fcgi?journal=13 ↗
http://link.springer.com/ ↗ - DOI:
- 10.1186/s12859-015-0564-6 ↗
- Languages:
- English
- ISSNs:
- 1471-2105
- Deposit Type:
- Legaldeposit
- View Content:
- Available online (eLD content is only available in our Reading Rooms) ↗
- Physical Locations:
- British Library DSC - BLDSS-3PM
British Library HMNTS - Digital store
British Library HMNTS - ELD Digital store - Ingest File:
- 9957.xml