Kernel-based framework for spectral dimensionality reduction and clustering formulationa theoretical study

  1. BLANCO VALENCIA, Xiomara Patricia 1
  2. BECERRA, M. A. 2
  3. CASTRO OSPINA, A. E. 3
  4. ORTEGA ADARME, M. 4
  5. VIVEROS MELO, D. 5
  6. PELUFFO ORDÓÑEZ, D. H. 6
  7. Juan Carlos Alvarado Pérez
  1. 1 Universidad de Salamanca
    info

    Universidad de Salamanca

    Salamanca, España

    ROR https://ror.org/02f40zc51

  2. 2 Institución Universitaria Salazar y Herrera
    info

    Institución Universitaria Salazar y Herrera

    Medellín, Colombia

    ROR https://ror.org/047xxr477

  3. 3 Research Center of the Instituto Tecnológico Metropolitano
  4. 4 Universidad de Nariño
    info

    Universidad de Nariño

    Pasto, Colombia

    ROR https://ror.org/050bg0846

  5. 5 Coorporación Universitaria Autónoma de Nariño
  6. 6 Universidad Técnica del Norte
    info

    Universidad Técnica del Norte

    Ibarra, Ecuador

    ROR https://ror.org/03f0t8b71

Mostrar afiliaciones +
Revista:
ADCAIJ: Advances in Distributed Computing and Artificial Intelligence Journal

ISSN: 2255-2863

Año de publicación: 2017

Volumen: 6

Número: 1

Páginas: 31-40

Tipo: Artículo

DOI: 10.14201/ADCAIJ2017613140 DIALNET GOOGLE SCHOLAR lock_openAcceso abierto editor

Otras publicaciones en: ADCAIJ: Advances in Distributed Computing and Artificial Intelligence Journal

Resumen

This work outlines a unified formulation to represent spectral approaches for both dimensionality reduction and clustering. Proposed formulation starts with a generic latent variable model in terms of the projected input data matrix.Particularly, such a projection maps data onto a unknown high-dimensional space. Regarding this model, a generalized optimization problem is stated using quadratic formulations and a least-squares support vector machine.The solution of the optimization is addressed through a primal-dual scheme.Once latent variables and parameters are determined, the resultant model outputs a versatile projected matrix able to represent data in a low-dimensional space, as well as to provide information about clusters. Particularly, proposedformulation yields solutions for kernel spectral clustering and weighted-kernel principal component analysis.

Referencias bibliográficas

  • Aldrich, C. and Auret, L., 2013. Statistical Learning Theory and Kernel-Based Methods. In Unsupervised Process Monitoring and Fault Diagnosis with Machine Learning Methods, pages 117–181. Springer. https://doi.org/10.1007/978-1-4471-5185-2_4
  • Alvarado-Pérez, J. C. and Peluffo-Ordó’nez, D. H., 2015. Artificial and Natural Intelligence Integration. In 12th International Conference on Distributed Computing and Artificial Intelligence (DCAI 2016), pages 167–173. Springer. https://doi.org/10.1007/978-3-319-19638-1_19
  • Alvarado-Pérez, J. C., Peluffo-Ordó-ez, D. H., and Therón, R., 2015. Bridging the gap between human knowledge and machine learning. ADCAIJ: Advances in Distributed Computing and Artificial Intelligence Journal, 4(1):54–64. https://doi.org/10.14201/ADCAIJ2015415464
  • Alzate, C. and Suykens, J. A. K., 2010. Multiway spectral clustering with out-of-sample extensions through weighted kernel PCA. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 32(2):335–347.
  • Belanche Mu-oz, L. A., 2013. Developments in kernel design. In European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN 2013), Bruges, Belgium, pages 369–378.
  • Domeniconi, C., Peng, J., and Yan, B., 2011. Composite kernels for semi-supervised clustering. Knowledge and information systems, 28(1):99–116. https://doi.org/10.1007/s10115-010-0318-8
  • Filippone, M., Camastra, F., Masulli, F., and Rovetta, S., 2008. A survey of kernel and spectral methods for clustering. Pattern recognition, 41(1):176–190. https://doi.org/10.1016/j.patcog.2007.05.018
  • González, F., Bermeo, D., Ramos, L., and Nasraoui, O., 2012. On the Robustness of Kernel-Based Clustering. Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, pages 122–129. https://doi.org/10.1007/978-3-642-33275-3_15
  • Ham, J., Lee, D. D., Mika, S., and Schölkopf, B., 2004. A kernel view of the dimensionality reduction of manifolds. In Proceedings of the twenty-first international conference on Machine learning, page 47. ACM. https://doi.org/10.1145/1015330.1015417
  • Harchaoui, Z., Bach, F., Cappé, O., and Moulines, E., 2013. Kernel-based methods for hypothesis testing: a unified view. IEEE Signal Processing Magazine, 30(4):87–97. https://doi.org/10.1109/MSP.2013.2253631
  • Huang, H., Chuang, Y., and Chen, C., 2012. Multiple Kernel Fuzzy Clustering. Fuzzy Systems, IEEE Transactions on, 20(1):120–134.
  • Langone, R., Alzate, C., and Suykens, J. A., 2013. Kernel spectral clustering with memory effect. Physica A: Statistical Mechanics and its Applications. https://doi.org/10.1016/j.physa.2013.01.058
  • Molina-Giraldo, S., Álvarez-Meza, A., Peluffo-Ordo-ez, D., and Castellanos-Domínguez, G., 2012. Image Segmentation Based on Multi-Kernel Learning and Feature Relevance Analysis. Advances in Artificial Intelligence–IBERAMIA 2012, pages 501–510. https://doi.org/10.1007/978-3-642-34654-5_51
  • Peluffo-Ordó-ez, D. H., Lee, J. A., Verleysen, M., Rodr?guez, J. L., and Castellanos-Dom?nguez, G., 2014. Unsupervised relevance analysis for feature extraction and selection. In ICPRAM 2014, pages 310–315.
  • Peluffo-Ordónez,D., Garcia-Vega, S., Langone, R., Suykens, J., Castellanos-Dominguez,G. et al., 2013. Kernel spectral clustering for dynamic data using multiple kernel learning. In Neural Networks (IJCNN), The 2013 International Joint Conference on, pages 1–6. IEEE. https://doi.org/10.1109/ijcnn.2013.6706858
  • Peluffo-Ordonez,D. H., Aldo Lee, J., and Verleysen,M., 2014. Generalized kernel framework for unsupervised spectral methods of dimensionality reduction. In 2014 IEEE Symposium on Computational Intelligence and Data Mining (CIDM), pages 171–177. IEEE. https://doi.org/10.1109/cidm.2014.7008664
  • Peluffo-Ordónez, D. H., Alvarado-Pérez, J. C., Lee, J. A., and Verleysen, M., 2015. Geometrical homotopy for data visualization. In European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN 2015).
  • Peluffo-Ordó-ez, D. H., Alzate, C., Suykens, J. A., and Castellanos-Domínguez, G., 2014a. Optimal Data Projection for Kernel Spectral Clustering. In European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, pages 553–558.
  • Peluffo-Ordó-ez, D. H., Lee, J. A., and Verleysen,M., 2014b. Recent methods for dimensionality reduction: A brief comparative analysis. In European Symposium on Artificial Neural Networks (ESANN). Citeseer.
  • Peluffo Ordo-ez, D. H., Lee, J. A., Verleysen, M., Rodriguez, J. L., Castellanos-Dominguez, G. et al., 2015. Unsupervised relevance analysis for feature extraction and selection. A distance-based approach for feature relevance. In 3rd International Conference on Pattern Recognition Applications and Methods (ICPRAM2014).
  • Schölkopf, B. and Smola, A. J., 2002. Learning with Kernels.
  • Seeland, M., Karwath, A., and Kramer, S., 2012. A structural cluster kernel for learning on graphs. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 516–524. ACM. https://doi.org/10.1145/2339530.2339614
  • Wolf, L. and Bileschi, S., 2005. Combining variable selection with dimensionality reduction. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), volume 2, pages 801–806. IEEE. https://doi.org/10.1109/cvpr.2005.103
  • Wolf, L. and Shashua, A., 2005. Feature selection for unsupervised and supervised inference: The emergence of sparsity in a weight-based approach. Journal of machine learning, 6:1855 – 1887.
  • Wu, Y., Ma, W., Gong, M., Li, H., and Jiao, L., 2015. Novel Fuzzy Active Contour Model With Kernel Metric for Image Segmentation. Applied Soft Computing. https://doi.org/10.1016/j.asoc.2015.04.058
  • Zelnik-manor, L. and Perona, P., 2004. Self-tuning spectral clustering. In Advances in Neural Information Processing Systems 17, pages 1601–1608.MIT Press.
Fuente de los datos: Dialnet