Analysis and Application of Normalization Methods with Supervised Feature Weighting to Improve K-means Accuracy

Iratxe Niño-Adan; Itziar Landa-Torres; Eva Portillo; Diana Manjarres

doi:10.1007/978-3-030-20055-8_2

Analysis and Application of Normalization Methods with Supervised Feature Weighting to Improve K-means Accuracy

Iratxe Niño-Adan^*, Itziar Landa-Torres, Eva Portillo, Diana Manjarres

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

4 Citations (Scopus)

Abstract

Normalization methods are widely employed for transforming the variables or features of a given dataset. In this paper three classical feature normalization methods, Standardization (St), Min-Max (MM) and Median Absolute Deviation (MAD), are studied in different synthetic datasets from UCI repository. An exhaustive analysis of the transformed features’ ranges and their influence on the Euclidean distance is performed, concluding that knowledge about the group structure gathered by each feature is needed to select the best normalization method for a given dataset. In order to effectively collect the features’ importance and adjust their contribution, this paper proposes a two-stage methodology for normalization and supervised feature weighting based on a Pearson correlation coefficient and on a Random Forest Feature Importance estimation method. Simulations on five different datasets reveal that our two-stage proposed methodology, in terms of accuracy, outperforms or at least maintains the K-means performance obtained if only normalization is applied.

Original language	English
Title of host publication	14th International Conference on Soft Computing Models in Industrial and Environmental Applications SOCO 2019, Proceedings
Editors	Francisco Martínez Álvarez, Alicia Troncoso Lora, Héctor Quintián, José António Sáez Muñoz, Emilio Corchado
Publisher	Springer Verlag
Pages	14-24
Number of pages	11
ISBN (Print)	9783030200541
DOIs	https://doi.org/10.1007/978-3-030-20055-8_2
Publication status	Published - 2020
Event	14th International Conference on Soft Computing Models in Industrial and Environmental Applications, SOCO 2019 - Seville, Spain Duration: 13 May 2019 → 15 May 2019

Publication series

Name	Advances in Intelligent Systems and Computing
Volume	950
ISSN (Print)	2194-5357
ISSN (Electronic)	2194-5365

Conference

Conference	14th International Conference on Soft Computing Models in Industrial and Environmental Applications, SOCO 2019
Country/Territory	Spain
City	Seville
Period	13/05/19 → 15/05/19

Funding

Acknowledgement. This work has been supported in part by the ELKARTEK program (SeNDANEU KK-2018/00032), the HAZITEK program (DATALYSE ZL-2018/00765) of the Basque Government and a TECNALIA Research and Innovation PhD Scholarship.

Funders	Funder number
Eusko Jaurlaritza

Keywords

K-means
Normalization
Pearson correlation
Random Forest
Standardization
Weighted Euclidean Distance

Access to Document

10.1007/978-3-030-20055-8_2

Cite this

Niño-Adan, I., Landa-Torres, I., Portillo, E., & Manjarres, D. (2020). Analysis and Application of Normalization Methods with Supervised Feature Weighting to Improve K-means Accuracy. In F. Martínez Álvarez, A. Troncoso Lora, H. Quintián, J. A. Sáez Muñoz, & E. Corchado (Eds.), 14th International Conference on Soft Computing Models in Industrial and Environmental Applications SOCO 2019, Proceedings (pp. 14-24). (Advances in Intelligent Systems and Computing; Vol. 950). Springer Verlag. https://doi.org/10.1007/978-3-030-20055-8_2

Niño-Adan, Iratxe ; Landa-Torres, Itziar ; Portillo, Eva et al. / Analysis and Application of Normalization Methods with Supervised Feature Weighting to Improve K-means Accuracy. 14th International Conference on Soft Computing Models in Industrial and Environmental Applications SOCO 2019, Proceedings. editor / Francisco Martínez Álvarez ; Alicia Troncoso Lora ; Héctor Quintián ; José António Sáez Muñoz ; Emilio Corchado. Springer Verlag, 2020. pp. 14-24 (Advances in Intelligent Systems and Computing).

@inproceedings{8c3f487376fb4a83b86034093ad5c61a,

title = "Analysis and Application of Normalization Methods with Supervised Feature Weighting to Improve K-means Accuracy",

abstract = "Normalization methods are widely employed for transforming the variables or features of a given dataset. In this paper three classical feature normalization methods, Standardization (St), Min-Max (MM) and Median Absolute Deviation (MAD), are studied in different synthetic datasets from UCI repository. An exhaustive analysis of the transformed features{\textquoteright} ranges and their influence on the Euclidean distance is performed, concluding that knowledge about the group structure gathered by each feature is needed to select the best normalization method for a given dataset. In order to effectively collect the features{\textquoteright} importance and adjust their contribution, this paper proposes a two-stage methodology for normalization and supervised feature weighting based on a Pearson correlation coefficient and on a Random Forest Feature Importance estimation method. Simulations on five different datasets reveal that our two-stage proposed methodology, in terms of accuracy, outperforms or at least maintains the K-means performance obtained if only normalization is applied.",

keywords = "K-means, Normalization, Pearson correlation, Random Forest, Standardization, Weighted Euclidean Distance",

author = "Iratxe Ni{\~n}o-Adan and Itziar Landa-Torres and Eva Portillo and Diana Manjarres",

note = "Publisher Copyright: {\textcopyright} 2020, Springer Nature Switzerland AG.; 14th International Conference on Soft Computing Models in Industrial and Environmental Applications, SOCO 2019 ; Conference date: 13-05-2019 Through 15-05-2019",

year = "2020",

doi = "10.1007/978-3-030-20055-8_2",

language = "English",

isbn = "9783030200541",

series = "Advances in Intelligent Systems and Computing",

publisher = "Springer Verlag",

pages = "14--24",

editor = "{Mart{\'i}nez {\'A}lvarez}, Francisco and {Troncoso Lora}, Alicia and H{\'e}ctor Quinti{\'a}n and {S{\'a}ez Mu{\~n}oz}, {Jos{\'e} Ant{\'o}nio} and Emilio Corchado",

booktitle = "14th International Conference on Soft Computing Models in Industrial and Environmental Applications SOCO 2019, Proceedings",

address = "Germany",

}

Niño-Adan, I, Landa-Torres, I, Portillo, E & Manjarres, D 2020, Analysis and Application of Normalization Methods with Supervised Feature Weighting to Improve K-means Accuracy. in F Martínez Álvarez, A Troncoso Lora, H Quintián, JA Sáez Muñoz & E Corchado (eds), 14th International Conference on Soft Computing Models in Industrial and Environmental Applications SOCO 2019, Proceedings. Advances in Intelligent Systems and Computing, vol. 950, Springer Verlag, pp. 14-24, 14th International Conference on Soft Computing Models in Industrial and Environmental Applications, SOCO 2019, Seville, Spain, 13/05/19. https://doi.org/10.1007/978-3-030-20055-8_2

Analysis and Application of Normalization Methods with Supervised Feature Weighting to Improve K-means Accuracy. / Niño-Adan, Iratxe; Landa-Torres, Itziar; Portillo, Eva et al.
14th International Conference on Soft Computing Models in Industrial and Environmental Applications SOCO 2019, Proceedings. ed. / Francisco Martínez Álvarez; Alicia Troncoso Lora; Héctor Quintián; José António Sáez Muñoz; Emilio Corchado. Springer Verlag, 2020. p. 14-24 (Advances in Intelligent Systems and Computing; Vol. 950).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Analysis and Application of Normalization Methods with Supervised Feature Weighting to Improve K-means Accuracy

AU - Niño-Adan, Iratxe

AU - Landa-Torres, Itziar

AU - Portillo, Eva

AU - Manjarres, Diana

PY - 2020

Y1 - 2020

N2 - Normalization methods are widely employed for transforming the variables or features of a given dataset. In this paper three classical feature normalization methods, Standardization (St), Min-Max (MM) and Median Absolute Deviation (MAD), are studied in different synthetic datasets from UCI repository. An exhaustive analysis of the transformed features’ ranges and their influence on the Euclidean distance is performed, concluding that knowledge about the group structure gathered by each feature is needed to select the best normalization method for a given dataset. In order to effectively collect the features’ importance and adjust their contribution, this paper proposes a two-stage methodology for normalization and supervised feature weighting based on a Pearson correlation coefficient and on a Random Forest Feature Importance estimation method. Simulations on five different datasets reveal that our two-stage proposed methodology, in terms of accuracy, outperforms or at least maintains the K-means performance obtained if only normalization is applied.

AB - Normalization methods are widely employed for transforming the variables or features of a given dataset. In this paper three classical feature normalization methods, Standardization (St), Min-Max (MM) and Median Absolute Deviation (MAD), are studied in different synthetic datasets from UCI repository. An exhaustive analysis of the transformed features’ ranges and their influence on the Euclidean distance is performed, concluding that knowledge about the group structure gathered by each feature is needed to select the best normalization method for a given dataset. In order to effectively collect the features’ importance and adjust their contribution, this paper proposes a two-stage methodology for normalization and supervised feature weighting based on a Pearson correlation coefficient and on a Random Forest Feature Importance estimation method. Simulations on five different datasets reveal that our two-stage proposed methodology, in terms of accuracy, outperforms or at least maintains the K-means performance obtained if only normalization is applied.

KW - K-means

KW - Normalization

KW - Pearson correlation

KW - Random Forest

KW - Standardization

KW - Weighted Euclidean Distance

UR - http://www.scopus.com/inward/record.url?scp=85065923886&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-20055-8_2

DO - 10.1007/978-3-030-20055-8_2

M3 - Conference contribution

AN - SCOPUS:85065923886

SN - 9783030200541

T3 - Advances in Intelligent Systems and Computing

SP - 14

EP - 24

BT - 14th International Conference on Soft Computing Models in Industrial and Environmental Applications SOCO 2019, Proceedings

A2 - Martínez Álvarez, Francisco

A2 - Troncoso Lora, Alicia

A2 - Quintián, Héctor

A2 - Sáez Muñoz, José António

A2 - Corchado, Emilio

PB - Springer Verlag

T2 - 14th International Conference on Soft Computing Models in Industrial and Environmental Applications, SOCO 2019

Y2 - 13 May 2019 through 15 May 2019

ER -

Niño-Adan I, Landa-Torres I, Portillo E, Manjarres D. Analysis and Application of Normalization Methods with Supervised Feature Weighting to Improve K-means Accuracy. In Martínez Álvarez F, Troncoso Lora A, Quintián H, Sáez Muñoz JA, Corchado E, editors, 14th International Conference on Soft Computing Models in Industrial and Environmental Applications SOCO 2019, Proceedings. Springer Verlag. 2020. p. 14-24. (Advances in Intelligent Systems and Computing). doi: 10.1007/978-3-030-20055-8_2

Analysis and Application of Normalization Methods with Supervised Feature Weighting to Improve K-means Accuracy

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this