Introduction
As we mentioned earlier, the SVM algorithm is quite versatile: not only does it support linear and nonlinear classification, but it also supports linear and nonlinear regression. The trick is to reverse the objective: instead of trying to fit the largest possible street between two classes while limiting margin violations, SVM Regression tries to fit as many instances as possible on the street while limiting margin violations (i.e., instances o the street).
The width of the street is controlled by a hyperparameter ϵ. Figure below shows two linear SVM Regression models trained on some random linear data, one with a large margin (ϵ = 1.5) and the other with a small margin (ϵ =0.5).
Visualization
Implementation
Adding more training instances within the margin does not affect the model’s predictions; thus, the model is said to be ϵ-insensitive. You can use Scikit-Learn’s LinearSVR class to perform linear SVM Regression. The following code produces the model represented on the left of Figure 5-10 (the training data should be scaled and centered first):
from sklearn.svm import LinearSVR svm_reg = LinearSVR(epsilon=1.5) svm_reg.fit(X, y)
To tackle nonlinear regression tasks, you can use a kernelized SVM model. For example, Figure below shows SVM Regression on a random quadratic training set, using a 2nd-degree polynomial kernel. There is little regularization on the left plot (i.e., a large C value), and much more regularization on the right plot (i.e., a small C value).
The following code produces the model represented on the left of Figure above using Scikit-Learn’s SVR class (which supports the kernel trick). The SVR class is the regression equivalent of the SVC class, and the Linear SVR class is the regression equivalent of the Linear SVC class. The LinearSVR class scales linearly with the size of the training set (just like the Linear SVC class), while the SVR class gets much too slow when the training set grows large (just like the Support Vector Classification class).
from sklearn.svm import SVR svm_poly_reg = SVR(kernel="poly", degree=2, C=100, epsilon=0.1) svm_poly_reg.fit(X, y)
Conclusion
SVMs can also be used for outlier detection; see Scikit-Learn’s documentation for more details.