6.2.14.3. scikits.learn.linear_model.sparse.SGDClassifier¶
- class scikits.learn.linear_model.sparse.SGDClassifier(loss='hinge', penalty='l2', alpha=0.0001, rho=0.85, fit_intercept=True, n_iter=5, shuffle=False, verbose=0, n_jobs=1)¶
Linear model fitted by minimizing a regularized empirical loss with SGD
SGD stands for Stochastic Gradient Descent: the gradient of the loss is estimated each sample at a time and the model is updated along the way with a decreasing strength schedule (aka learning rate).
The regularizer is a penalty added to the loss function that shrinks model parameters towards the zero vector using either the squared euclidean norm L2 or the absolute norm L1 or a combination of both (Elastic Net). If the parameter update crosses the 0.0 value because of the regularizer, the update is truncated to 0.0 to allow for learning sparse models and achieve online feature selection.
This implementation works on scipy.sparse X and dense coef_.
Parameters : loss : str, ‘hinge’ or ‘log’ or ‘modified_huber’
The loss function to be used. Defaults to ‘hinge’. The hinge loss is a margin loss used by standard linear SVM models. The ‘log’ loss is the loss of logistic regression models and can be used for probability estimation in binary classifiers. ‘modified_huber’ is another smooth loss that brings tolerance to outliers.
penalty : str, ‘l2’ or ‘l1’ or ‘elasticnet’
The penalty (aka regularization term) to be used. Defaults to ‘l2’ which is the standard regularizer for linear SVM models. ‘l1’ and ‘elasticnet’ migh bring sparsity to the model (feature selection) not achievable with ‘l2’.
alpha : float
Constant that multiplies the regularization term. Defaults to 0.0001
rho : float
The Elastic Net mixing parameter, with 0 < rho <= 1. Defaults to 0.85.
fit_intercept: bool :
Whether the intercept should be estimated or not. If False, the data is assumed to be already centered. Defaults to True.
n_iter: int :
The number of passes over the training data (aka epochs). Defaults to 5.
shuffle: bool :
Whether or not the training data should be shuffled after each epoch. Defaults to False.
verbose: integer, optional :
The verbosity level
n_jobs: integer, optional :
The number of CPUs to use to do the OVA (One Versus All, for multi-class problems) computation. -1 means ‘all CPUs’. Defaults to 1.
See also
LinearSVC, LogisticRegression
Examples
>>> import numpy as np >>> from scikits.learn import linear_model >>> X = np.array([[-1, -1], [-2, -1], [1, 1], [2, 1]]) >>> y = np.array([1, 1, 2, 2]) >>> clf = linear_model.sparse.SGDClassifier() >>> clf.fit(X, y) SGDClassifier(loss='hinge', n_jobs=1, shuffle=False, verbose=0, n_iter=5, fit_intercept=True, penalty='l2', rho=1.0, alpha=0.0001) >>> print clf.predict([[-0.8, -1]]) [ 1.]
Attributes
coef_ array, shape = [n_features] if n_classes == 2 else [n_classes, n_features] Weights asigned to the features. intercept_ array, shape = [1] if n_classes == 2 else [n_classes] Constants in decision function. Methods
decision_function(X) Predict signed ‘distance’ to the hyperplane (aka confidence score). fit(X, y[, coef_init, intercept_init, ...]) Fit linear model with Stochastic Gradient Descent predict(X) Predict using the linear model predict_proba(X) Predict class membership probability score(X, y) Returns the mean error rate on the given test data and labels. - __init__(loss='hinge', penalty='l2', alpha=0.0001, rho=0.85, fit_intercept=True, n_iter=5, shuffle=False, verbose=0, n_jobs=1)¶
- decision_function(X)¶
Predict signed ‘distance’ to the hyperplane (aka confidence score).
Parameters : X : scipy.sparse matrix of shape [n_samples, n_features]
Returns : array, shape = [n_samples] if n_classes == 2 else [n_samples, n_classes] :
The signed ‘distances’ to the hyperplane(s).
- fit(X, y, coef_init=None, intercept_init=None, class_weight={}, **params)¶
Fit linear model with Stochastic Gradient Descent
X is expected to be a sparse matrix. For maximum efficiency, use a sparse matrix in CSR format (scipy.sparse.csr_matrix)
Parameters : X : scipy sparse matrix of shape [n_samples,n_features]
Training data
y : numpy array of shape [n_samples]
Target values
coef_init : array, shape = [n_features] if n_classes == 2 else
[n_classes,n_features] :
The initial coeffients to warm-start the optimization.
intercept_init : array, shape = [1] if n_classes == 2 else [n_classes]
The initial intercept to warm-start the optimization.
class_weight : dict, {class_label
Weights associated with classes. If not given, all classes are supposed to have weight one.
The “auto” mode uses the values of y to automatically adjust weights inversely proportional to class frequencies.
Returns : self : returns an instance of self.
- predict(X)¶
Predict using the linear model
Parameters : X : array or scipy.sparse matrix of shape [n_samples, n_features]
Whether the numpy.array or scipy.sparse matrix is accepted dependes on the actual implementation
Returns : array, shape = [n_samples] :
Array containing the predicted class labels.
- predict_proba(X)¶
Predict class membership probability
Parameters : X : array or scipy.sparse matrix of shape [n_samples, n_features]
Returns : array, shape = [n_samples] if n_classes == 2 else [n_samples, :
n_classes] :
Contains the membership probabilities of the positive class.
- score(X, y)¶
Returns the mean error rate on the given test data and labels.
Parameters : X : array-like, shape = [n_samples, n_features]
Training set.
y : array-like, shape = [n_samples]
Labels for X.
Returns : z : float