Support Vector Machine in Machine Learning

Support Vector Machine (SVM):

Support Vector Machine (SVM) is a supervised machine learning algorithm that can be used for both classification and regression challenges. However, it is mostly used in classification problems. In this algorithm, we plot each data item as a point in n-dimensional space with the value of each feature being the value of a particular coordinate. Then, we perform classification by finding the hyper-plane that differentiates the two classes very well.

Support vectors are simply the coordinates of individual observation. Support Vector Machine is a frontier that best segregates the two classes (hyper-plane/line). You can use a support vector machine (SVM) when your data has exactly two classes. An SVM classifies the data by finding the best hyperplane that separates all data points of one class from those of the other class. The best hyperplane for an SVM is the one with the largest margin between the two classes. Margin means the maximal width of the slab parallel to the hyperplane that has no interior data points.

Determining the right hyper-plane:

Case-1: Here, we have three hyper-planes (A, B, and C) segregation two distinct data groups to classify stars and circles. Select the hyper-plane which segregates the two classes better. In this scenario, hyper-plane “B” is the best option among the three.

Case-2: In this case, three hyper-planes (A, B, and C) are seemingly segregating the classes well and we have to identify the right hyper-plane. We have to use the technique of optimizing the distances between groups of points of either of the class and classifier hyperplane. This distance is called Margin.

Case-3: Adopt the procedure as earlier but carefully to identify the right hyper-plane. Some of us may tend to select hyperplane B as it has a higher margin compared to A. But, here is the catch, SVM selects the hyper-plane which classifies the classes accurately prior to maximizing the margin.

Here, hyper-plane B has a classification error as it forgot to include point C, and A has classified it all correctly. Therefore, the right hyper-plane is A. But keep in mind that sometimes point C may be an outlier also.

Case-4: The programmer may be unable to segregate the two classes using a straight line, as one of the stars lies in the territory of the other class as an outlier. One star at the other end is like an outlier for the star class. SVM has a feature to ignore outliers and find the hyper-plane that has the maximum margin. As such we can say that SVM is robust to outliers.

Case-5: Inter mixing classes of Dataset.