Evaluating automatically parallelized versions of the support vector machine

Valeriu Codreanu, Bob Droge, David Williams, Burhan Yasar, Po Yang, Baoquan Liu, Feng Dong, Olarik Surinta, Lambert Schomaker, Jos Roerdink, Marco Wiering

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

The support vector machine (SVM) is a supervised learning algorithm used for recognizing patterns in data. It is a very popular technique in machine learning and has been successfully used in applications such as image classification, protein classification, and handwriting recognition. However, the computational complexity of the kernelized version of the algorithm grows quadratically with the number of training examples. To tackle this high computational complexity, we have developed a directive-based approach that converts a gradient-ascent based training algorithm for the CPU to an efficient graphics processing unit (GPU) implementation. We compare our GPU-based SVM training algorithm to the standard LibSVM CPU implementation, a highly optimized GPU-LibSVM implementation, as well as to a directive-based OpenACC implementation. The results on different handwritten digit classification datasets demonstrate an important speed-up for the current approach when compared to the CPU and OpenACC versions.Furthermore, our solution is almost as fast and sometimes even faster than the highly optimized CUBLAS-based GPU-LibSVM implementation, without sacrificing the algorithm’s accuracy. Copyright © 2014 JohnWiley & Sons, Ltd.
Original languageEnglish
Pages (from-to)2274-2294
Number of pages21
JournalConcurrency and Computation: Practice and Experience
Volume26
Issue number7
Early online date9 Oct 2014
DOIs
Publication statusPublished - 2016

Fingerprint

Support vector machines
Graphics Processing Unit
Support Vector Machine
Program processors
Training Algorithm
Computational complexity
Computational Complexity
Protein Classification
Image classification
Handwriting Recognition
Supervised learning
Ascent
Learning algorithms
Image Classification
Learning systems
Supervised Learning
Digit
Convert
Learning Algorithm
Proteins

Keywords

  • support vector machine (SVM)
  • machine learning
  • computational complexity

Cite this

Codreanu, V., Droge, B., Williams, D., Yasar, B., Yang, P., Liu, B., ... Wiering, M. (2016). Evaluating automatically parallelized versions of the support vector machine. Concurrency and Computation: Practice and Experience, 26(7), 2274-2294. https://doi.org/10.1002/cpe.3413
Codreanu, Valeriu ; Droge, Bob ; Williams, David ; Yasar, Burhan ; Yang, Po ; Liu, Baoquan ; Dong, Feng ; Surinta, Olarik ; Schomaker, Lambert ; Roerdink, Jos ; Wiering, Marco. / Evaluating automatically parallelized versions of the support vector machine. In: Concurrency and Computation: Practice and Experience. 2016 ; Vol. 26, No. 7. pp. 2274-2294.
@article{a7cf7bd0c25f44f691d21fac346c97da,
title = "Evaluating automatically parallelized versions of the support vector machine",
abstract = "The support vector machine (SVM) is a supervised learning algorithm used for recognizing patterns in data. It is a very popular technique in machine learning and has been successfully used in applications such as image classification, protein classification, and handwriting recognition. However, the computational complexity of the kernelized version of the algorithm grows quadratically with the number of training examples. To tackle this high computational complexity, we have developed a directive-based approach that converts a gradient-ascent based training algorithm for the CPU to an efficient graphics processing unit (GPU) implementation. We compare our GPU-based SVM training algorithm to the standard LibSVM CPU implementation, a highly optimized GPU-LibSVM implementation, as well as to a directive-based OpenACC implementation. The results on different handwritten digit classification datasets demonstrate an important speed-up for the current approach when compared to the CPU and OpenACC versions.Furthermore, our solution is almost as fast and sometimes even faster than the highly optimized CUBLAS-based GPU-LibSVM implementation, without sacrificing the algorithm’s accuracy. Copyright {\circledC} 2014 JohnWiley & Sons, Ltd.",
keywords = "support vector machine (SVM), machine learning, computational complexity",
author = "Valeriu Codreanu and Bob Droge and David Williams and Burhan Yasar and Po Yang and Baoquan Liu and Feng Dong and Olarik Surinta and Lambert Schomaker and Jos Roerdink and Marco Wiering",
year = "2016",
doi = "10.1002/cpe.3413",
language = "English",
volume = "26",
pages = "2274--2294",
journal = "Concurrency and Computation: Practice and Experience",
issn = "1532-0626",
number = "7",

}

Codreanu, V, Droge, B, Williams, D, Yasar, B, Yang, P, Liu, B, Dong, F, Surinta, O, Schomaker, L, Roerdink, J & Wiering, M 2016, 'Evaluating automatically parallelized versions of the support vector machine', Concurrency and Computation: Practice and Experience, vol. 26, no. 7, pp. 2274-2294. https://doi.org/10.1002/cpe.3413

Evaluating automatically parallelized versions of the support vector machine. / Codreanu, Valeriu; Droge, Bob; Williams, David; Yasar, Burhan; Yang, Po; Liu, Baoquan; Dong, Feng; Surinta, Olarik; Schomaker, Lambert; Roerdink, Jos; Wiering, Marco.

In: Concurrency and Computation: Practice and Experience, Vol. 26, No. 7, 2016, p. 2274-2294.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Evaluating automatically parallelized versions of the support vector machine

AU - Codreanu, Valeriu

AU - Droge, Bob

AU - Williams, David

AU - Yasar, Burhan

AU - Yang, Po

AU - Liu, Baoquan

AU - Dong, Feng

AU - Surinta, Olarik

AU - Schomaker, Lambert

AU - Roerdink, Jos

AU - Wiering, Marco

PY - 2016

Y1 - 2016

N2 - The support vector machine (SVM) is a supervised learning algorithm used for recognizing patterns in data. It is a very popular technique in machine learning and has been successfully used in applications such as image classification, protein classification, and handwriting recognition. However, the computational complexity of the kernelized version of the algorithm grows quadratically with the number of training examples. To tackle this high computational complexity, we have developed a directive-based approach that converts a gradient-ascent based training algorithm for the CPU to an efficient graphics processing unit (GPU) implementation. We compare our GPU-based SVM training algorithm to the standard LibSVM CPU implementation, a highly optimized GPU-LibSVM implementation, as well as to a directive-based OpenACC implementation. The results on different handwritten digit classification datasets demonstrate an important speed-up for the current approach when compared to the CPU and OpenACC versions.Furthermore, our solution is almost as fast and sometimes even faster than the highly optimized CUBLAS-based GPU-LibSVM implementation, without sacrificing the algorithm’s accuracy. Copyright © 2014 JohnWiley & Sons, Ltd.

AB - The support vector machine (SVM) is a supervised learning algorithm used for recognizing patterns in data. It is a very popular technique in machine learning and has been successfully used in applications such as image classification, protein classification, and handwriting recognition. However, the computational complexity of the kernelized version of the algorithm grows quadratically with the number of training examples. To tackle this high computational complexity, we have developed a directive-based approach that converts a gradient-ascent based training algorithm for the CPU to an efficient graphics processing unit (GPU) implementation. We compare our GPU-based SVM training algorithm to the standard LibSVM CPU implementation, a highly optimized GPU-LibSVM implementation, as well as to a directive-based OpenACC implementation. The results on different handwritten digit classification datasets demonstrate an important speed-up for the current approach when compared to the CPU and OpenACC versions.Furthermore, our solution is almost as fast and sometimes even faster than the highly optimized CUBLAS-based GPU-LibSVM implementation, without sacrificing the algorithm’s accuracy. Copyright © 2014 JohnWiley & Sons, Ltd.

KW - support vector machine (SVM)

KW - machine learning

KW - computational complexity

U2 - 10.1002/cpe.3413

DO - 10.1002/cpe.3413

M3 - Article

VL - 26

SP - 2274

EP - 2294

JO - Concurrency and Computation: Practice and Experience

JF - Concurrency and Computation: Practice and Experience

SN - 1532-0626

IS - 7

ER -