A Caltech Library Service

Infinite Ensemble Learning with Support Vector Machines


Lin, Hsuan-Tien (2005) Infinite Ensemble Learning with Support Vector Machines. Master's thesis, California Institute of Technology. doi:10.7907/E03R-EN93.


Ensemble learning algorithms such as boosting can achieve better performance by averaging over the predictions of base learners. However, existing algorithms are limited to combining only a finite number of base learners, and the generated ensemble is usually sparse. It is not clear whether we should construct an ensemble classifier with a larger or even an infinite number of base learners.

In addition, constructing an infinite ensemble itself is a challenging task. In this paper, we formulate an infinite ensemble learning framework based on SVM. The framework could output an infinite and nonsparse ensemble, and can be applied to construct new kernels for SVM as well as to interpret existing ones. We demonstrate the framework with a concrete application, the stump kernel, which embodies infinitely many decision stumps. The stump kernel is simple, yet powerful.

Experimental results show that SVM with the stump kernel usually achieves better performance than boosting, even with noisy data.

Item Type:Thesis (Master's thesis)
Subject Keywords:boosting; ensemble learning; Kernel method; large margin classifier; support vector machine
Degree Grantor:California Institute of Technology
Division:Engineering and Applied Science
Major Option:Computer Science
Thesis Availability:Public (worldwide access)
Research Advisor(s):
  • Abu-Mostafa, Yaser S.
Thesis Committee:
  • Unknown, Unknown
Defense Date:18 May 2005
Non-Caltech Author Email:htlin (AT)
Record Number:CaltechETD:etd-05262005-030549
Persistent URL:
Lin, Hsuan-Tien0000-0003-2968-0671
Default Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:2087
Deposited By: Imported from ETD-db
Deposited On:26 May 2005
Last Modified:07 May 2020 22:55

Thesis Files

PDF - Final Version
See Usage Policy.


Repository Staff Only: item control page