Research Article Open Access

HYPERPARAMETER SELECTION IN KERNEL PRINCIPAL COMPONENT ANALYSIS

Md. Ashad Alam1 and Kenji Fukumizu2
  • 1 Graduate University for Advanced Studies, Japan
  • 2 , Japan

Abstract

In kernel methods, choosing a suitable kernel is indispensable for favorable results. No well-founded methods, however, have been established in general for unsupervised learning. We focus on kernel Principal Component Analysis (kernel PCA), which is a nonlinear extension of principal component analysis and has been used electively for extracting nonlinear features and reducing dimensionality. As a kernel method, kernel PCA also suffers from the problem of kernel choice. Although cross-validation is a popular method for choosing hyperparameters, it is not applicable straightforwardly to choose a kernel in kernel PCA because of the incomparable norms given by different kernels. It is important, thus, to develop a well-founded method for choosing a kernel in kernel PCA. This study proposes a method for choosing hyperparameters in kernel PCA (kernel and the number of components) based on cross-validation for the comparable reconstruction errors of pre-images in the original space. The experimental results on synthesized and real-world datasets demonstrate that the proposed method successfully selects an appropriate kernel and the number of components in kernel PCA in terms of visualization and classification errors on the principal components. The results imply that the proposed method enables automatic design of hyperparameters in kernel PCA.

Journal of Computer Science
Volume 10 No. 7, 2014, 1139-1150

DOI: https://doi.org/10.3844/jcssp.2014.1139.1150

Submitted On: 2 December 2013 Published On: 15 February 2014

How to Cite: Alam, M. A. & Fukumizu, K. (2014). HYPERPARAMETER SELECTION IN KERNEL PRINCIPAL COMPONENT ANALYSIS. Journal of Computer Science, 10(7), 1139-1150. https://doi.org/10.3844/jcssp.2014.1139.1150

  • 3,691 Views
  • 6,624 Downloads
  • 28 Citations

Download

Keywords

  • Kernel Principal Component Analysis
  • Pre-Image
  • Kernel Choice
  • Cross-Validation