Learner excellence biased by data set selection: A case for data characterisation and artificial data sets

Núria MacIà; Ester Bernadó-Mansilla; Albert Orriols-Puig; Tin Kam Ho

doi:10.1016/j.patcog.2012.09.022

Learner excellence biased by data set selection: A case for data characterisation and artificial data sets

Núria MacIà^*
, Ester Bernadó-Mansilla
, Albert Orriols-Puig
, Tin Kam Ho

^*Corresponding author for this work

Research output: Indexed journal article › Article › peer-review

37 Citations (Scopus)

Abstract

The excellence of a given learner is usually claimed through a performance comparison with other learners over a collection of data sets. Too often, researchers are not aware of the impact of their data selection on the results. Their test beds are small, and the selection of the data sets is not supported by any previous data analysis. Conclusions drawn on such test beds cannot be generalised, because particular data characteristics may favour certain learners unnoticeably. This work raises these issues and proposes the characterisation of data sets using complexity measures, which can be helpful for both guiding experimental design and explaining the behaviour of learners.

Original language	English
Pages (from-to)	1054-1066
Number of pages	13
Journal	Pattern Recognition
Volume	46
Issue number	3
DOIs	https://doi.org/10.1016/j.patcog.2012.09.022
Publication status	Published - Mar 2013

Keywords

Data complexity
Learner assessment
Supervised learning

Access to Document

10.1016/j.patcog.2012.09.022

Cite this

@article{cfb01a43073b4c78889f9c506d567879,

title = "Learner excellence biased by data set selection: A case for data characterisation and artificial data sets",

abstract = "The excellence of a given learner is usually claimed through a performance comparison with other learners over a collection of data sets. Too often, researchers are not aware of the impact of their data selection on the results. Their test beds are small, and the selection of the data sets is not supported by any previous data analysis. Conclusions drawn on such test beds cannot be generalised, because particular data characteristics may favour certain learners unnoticeably. This work raises these issues and proposes the characterisation of data sets using complexity measures, which can be helpful for both guiding experimental design and explaining the behaviour of learners.",

keywords = "Data complexity, Learner assessment, Supervised learning",

author = "N{\'u}ria MacI{\`a} and Ester Bernad{\'o}-Mansilla and Albert Orriols-Puig and \{Kam Ho\}, Tin",

year = "2013",

month = mar,

doi = "10.1016/j.patcog.2012.09.022",

language = "English",

volume = "46",

pages = "1054--1066",

journal = "Pattern Recognition",

issn = "0031-3203",

publisher = "Elsevier Ltd.",

number = "3",

}

TY - JOUR

T1 - Learner excellence biased by data set selection

T2 - A case for data characterisation and artificial data sets

AU - MacIà, Núria

AU - Bernadó-Mansilla, Ester

AU - Orriols-Puig, Albert

AU - Kam Ho, Tin

PY - 2013/3

Y1 - 2013/3

N2 - The excellence of a given learner is usually claimed through a performance comparison with other learners over a collection of data sets. Too often, researchers are not aware of the impact of their data selection on the results. Their test beds are small, and the selection of the data sets is not supported by any previous data analysis. Conclusions drawn on such test beds cannot be generalised, because particular data characteristics may favour certain learners unnoticeably. This work raises these issues and proposes the characterisation of data sets using complexity measures, which can be helpful for both guiding experimental design and explaining the behaviour of learners.

AB - The excellence of a given learner is usually claimed through a performance comparison with other learners over a collection of data sets. Too often, researchers are not aware of the impact of their data selection on the results. Their test beds are small, and the selection of the data sets is not supported by any previous data analysis. Conclusions drawn on such test beds cannot be generalised, because particular data characteristics may favour certain learners unnoticeably. This work raises these issues and proposes the characterisation of data sets using complexity measures, which can be helpful for both guiding experimental design and explaining the behaviour of learners.

KW - Data complexity

KW - Learner assessment

KW - Supervised learning

UR - https://www.scopus.com/pages/publications/84870236033

U2 - 10.1016/j.patcog.2012.09.022

DO - 10.1016/j.patcog.2012.09.022

M3 - Article

AN - SCOPUS:84870236033

SN - 0031-3203

VL - 46

SP - 1054

EP - 1066

JO - Pattern Recognition

JF - Pattern Recognition

IS - 3

ER -

Learner excellence biased by data set selection: A case for data characterisation and artificial data sets

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this