Lecture 7

Support vector machines

Support vector machines (SVMs)
Constrained optimization
Lagrange multipliers
Structural risk minimization
VC dimension

Constrained optimization and Lagrange multipliers

• Vapnik, Vladimir N. (1998) Statistical Learning Theory. New York: John Wiley.
  • Section 9.5: Three theorems of optimization theory, pp. 390-394. CC
  • http://en.wikipedia.org/wiki/Optimization_%28mathematics%29

• Strang, Gilbert (1988) Linear Algebra and its Applications, Third Edition. New York: Harcourt Brace Jovanovich College Publishers.
  • Section 8.3: The theory of duality, pp. 412-423. CC
  • http://en.wikipedia.org/wiki/Dual_space

• Duda, Richard O., Hart, Peter E., & Stork, David (2001) Pattern Classification. Second Edition. New York: Wiley.
  • Section A.3: Lagrange optimization, p. 610.
  • http://en.wikipedia.org/wiki/Constrained_Optimization_and_lagrange_Multipliers

• Bishop, Christopher M. (2006) Pattern Recognition and Machine Learning. Springer.
  • Appendix E: Lagrange multipliers, pp. 707-710. CC

Support vector machines

• http://en.wikipedia.org/wiki/Support_vector_machines

• Duda, Richard O., Hart, Peter E., & Stork, David (2001) Pattern Classification. Second Edition. New York: Wiley.
  • Section 5.11: Support vector machines, pp. 259-265

• Vapnik, Vladimir N. (1998) Statistical Learning Theory. New York: John Wiley. Chapter 10: The support vector method for estimating indicator functions, pp. 401 - 441. CC

V-C dimension

• Bartlett, Peter L., & Maass, Wolfgang (2003) Vapnik-Chervonenkis dimension of neural nets. In Arbib, Michael A. (2003) The Handbook of Brain Theory and Neural Networks, Second Edition. Cambridge, Mass.: MIT Press. pp. 1188-1192. CC

Additional SVM resources

• CN710 Readings re: SVM, especially the Bennett and Campbell paper