8.17.26. sklearn.linear_model.SGDClassifier

class sklearn.linear_model.SGDClassifier(loss='hinge', penalty='l2', alpha=0.0001, l1_ratio=0.15, fit_intercept=True, n_iter=5, shuffle=False, verbose=0, epsilon=0.1, n_jobs=1, random_state=None, learning_rate='optimal', eta0=0.0, power_t=0.5, class_weight=None, warm_start=False, rho=None, seed=None)

Linear model fitted by minimizing a regularized empirical loss with SGD.

SGD stands for Stochastic Gradient Descent: the gradient of the loss is estimated each sample at a time and the model is updated along the way with a decreasing strength schedule (aka learning rate).

The regularizer is a penalty added to the loss function that shrinks model parameters towards the zero vector using either the squared euclidean norm L2 or the absolute norm L1 or a combination of both (Elastic Net). If the parameter update crosses the 0.0 value because of the regularizer, the update is truncated to 0.0 to allow for learning sparse models and achieve online feature selection.

This implementation works with data represented as dense or sparse arrays of floating point values for the features.

Parameters:

loss : str, ‘hinge’ or ‘log’ or ‘modified_huber’

The loss function to be used. Defaults to ‘hinge’. The hinge loss is a margin loss used by standard linear SVM models. The ‘log’ loss is the loss of logistic regression models and can be used for probability estimation in binary classifiers. ‘modified_huber’ is another smooth loss that brings tolerance to outliers.

penalty : str, ‘l2’ or ‘l1’ or ‘elasticnet’

The penalty (aka regularization term) to be used. Defaults to ‘l2’ which is the standard regularizer for linear SVM models. ‘l1’ and ‘elasticnet’ migh bring sparsity to the model (feature selection) not achievable with ‘l2’.

alpha : float

Constant that multiplies the regularization term. Defaults to 0.0001

l1_ratio : float

The Elastic Net mixing parameter, with 0 <= l1_ratio <= 1. l1_ratio=0 corresponds to L2 penalty, l1_ratio=1 to L1. Defaults to 0.15.

fit_intercept: bool :

Whether the intercept should be estimated or not. If False, the data is assumed to be already centered. Defaults to True.

n_iter: int, optional :

The number of passes over the training data (aka epochs). Defaults to 5.

shuffle: bool, optional :

Whether or not the training data should be shuffled after each epoch. Defaults to False.

random_state: int seed, RandomState instance, or None (default) :

The seed of the pseudo random number generator to use when shuffling the data.

verbose: integer, optional :

The verbosity level

epsilon: float :

Epsilon in the epsilon-insensitive loss functions; only if loss==’huber’ or loss=’epsilon_insensitive’. If the difference between the current prediction and the correct label is below this threshold, the model is not updated.

n_jobs: integer, optional :

The number of CPUs to use to do the OVA (One Versus All, for multi-class problems) computation. -1 means ‘all CPUs’. Defaults to 1.

learning_rate : string, optional

The learning rate: constant: eta = eta0 optimal: eta = 1.0/(t+t0) [default] invscaling: eta = eta0 / pow(t, power_t)

eta0 : double

The initial learning rate [default 0.01].

power_t : double

The exponent for inverse scaling learning rate [default 0.5].

class_weight : dict, {class_label

Preset for the class_weight fit parameter.

Weights associated with classes. If not given, all classes are supposed to have weight one.

The “auto” mode uses the values of y to automatically adjust weights inversely proportional to class frequencies.

warm_start : bool, optional

When set to True, reuse the solution of the previous call to fit as initialization, otherwise, just erase the previous solution.

See also

LinearSVC, LogisticRegression, Perceptron

Examples

>>> import numpy as np
>>> from sklearn import linear_model
>>> X = np.array([[-1, -1], [-2, -1], [1, 1], [2, 1]])
>>> Y = np.array([1, 1, 2, 2])
>>> clf = linear_model.SGDClassifier()
>>> clf.fit(X, Y)
... 
SGDClassifier(alpha=0.0001, class_weight=None, epsilon=0.1, eta0=0.0,
        fit_intercept=True, l1_ratio=0.15, learning_rate='optimal',
        loss='hinge', n_iter=5, n_jobs=1, penalty='l2', power_t=0.5,
        random_state=None, rho=None, shuffle=False,
        verbose=0, warm_start=False)
>>> print(clf.predict([[-0.8, -1]]))
[1]

Attributes

coef_	array, shape = [1, n_features] if n_classes == 2 else [n_classes,
n_features]		Weights assigned to the features.
intercept_	array, shape = [1] if n_classes == 2 else [n_classes]	Constants in decision function.

Methods

decision_function(X)	Predict confidence scores for samples.
fit(X, y[, coef_init, intercept_init, ...])	Fit linear model with Stochastic Gradient Descent.
fit_transform(X[, y])	Fit to data, then transform it
get_params([deep])	Get parameters for the estimator
partial_fit(X, y[, classes, sample_weight])	Fit linear model with Stochastic Gradient Descent.
predict(X)	Predict class labels for samples in X.
predict_log_proba(X)	Log of probability estimates.
predict_proba(X)	Probability estimates.
score(X, y)	Returns the mean accuracy on the given test data and labels.
set_params(*args, **kwargs)
transform(X[, threshold])	Reduce X to its most important features.

__init__(loss='hinge', penalty='l2', alpha=0.0001, l1_ratio=0.15, fit_intercept=True, n_iter=5, shuffle=False, verbose=0, epsilon=0.1, n_jobs=1, random_state=None, learning_rate='optimal', eta0=0.0, power_t=0.5, class_weight=None, warm_start=False, rho=None, seed=None)

decision_function(X)

Predict confidence scores for samples.

The confidence score for a sample is the signed distance of that sample to the hyperplane.

Parameters:

X : {array-like, sparse matrix}, shape = [n_samples, n_features]

Samples.

Returns:

array, shape = [n_samples] if n_classes == 2 else [n_samples,n_classes] :

Confidence scores per (sample, class) combination. In the binary case, confidence score for the “positive” class.

fit(X, y, coef_init=None, intercept_init=None, class_weight=None, sample_weight=None)

Fit linear model with Stochastic Gradient Descent.

Parameters:

X : {array-like, sparse matrix}, shape = [n_samples, n_features]

Training data

y : numpy array of shape [n_samples]

Target values

coef_init : array, shape = [n_classes,n_features]

The initial coeffients to warm-start the optimization.

intercept_init : array, shape = [n_classes]

The initial intercept to warm-start the optimization.

sample_weight : array-like, shape = [n_samples], optional

Weights applied to individual samples. If not provided, uniform weights are assumed.

Returns:

self : returns an instance of self.

fit_transform(X, y=None, **fit_params)

Fit to data, then transform it

Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.

Parameters:

X : numpy array of shape [n_samples, n_features]

Training set.

y : numpy array of shape [n_samples]

Target values.

Returns:

X_new : numpy array of shape [n_samples, n_features_new]

Transformed array.

get_params(deep=True)

Get parameters for the estimator

Parameters:

deep: boolean, optional :

If True, will return the parameters for this estimator and contained subobjects that are estimators.

partial_fit(X, y, classes=None, sample_weight=None)

Fit linear model with Stochastic Gradient Descent.

Parameters:

X : {array-like, sparse matrix}, shape = [n_samples, n_features]

Subset of the training data

y : numpy array of shape [n_samples]

Subset of the target values

classes : array, shape = [n_classes]

Classes across all calls to partial_fit. Can be obtained by via np.unique(y_all), where y_all is the target vector of the entire dataset. This argument is required for the first call to partial_fit and can be omitted in the subsequent calls. Note that y doesn’t need to contain all labels in classes.

sample_weight : array-like, shape = [n_samples], optional

Weights applied to individual samples. If not provided, uniform weights are assumed.

Returns:

self : returns an instance of self.

predict(X)

Predict class labels for samples in X.

Parameters:

X : {array-like, sparse matrix}, shape = [n_samples, n_features]

Samples.

Returns:

C : array, shape = [n_samples]

Predicted class label per sample.

predict_log_proba(X)

Log of probability estimates.

Log probability estimates are only supported for binary classification.

Parameters:

X : array-like, shape = [n_samples, n_features]

Returns:

T : array-like, shape = [n_samples, n_classes]

Returns the log-probability of the sample for each class in the model, where classes are ordered as they are in self.classes_.

predict_proba(X)

Probability estimates.

Probability estimates are only supported for binary classification.

Parameters:

X : {array-like, sparse matrix}, shape = [n_samples, n_features]

Returns:

array, shape = [n_samples, n_classes] :

Returns the probability of the sample for each class in the model, where classes are ordered as they are in self.classes_.

References

The justification for the formula in the loss=”modified_huber” case is in the appendix B in: http://jmlr.csail.mit.edu/papers/volume2/zhang02c/zhang02c.pdf

score(X, y)

Returns the mean accuracy on the given test data and labels.

Parameters:

X : array-like, shape = [n_samples, n_features]

Training set.

y : array-like, shape = [n_samples]

Labels for X.

Returns:

z : float

seed

DEPRECATED: Parameter ‘seed’ war renamed to ‘random_state’ for consistency and will be removed in 0.15

transform(X, threshold=None)

Reduce X to its most important features.

Parameters:

X : array or scipy sparse matrix of shape [n_samples, n_features]

The input samples.

threshold : string, float or None, optional (default=None)

The threshold value to use for feature selection. Features whose importance is greater or equal are kept while the others are discarded. If “median” (resp. “mean”), then the threshold value is the median (resp. the mean) of the feature importances. A scaling factor (e.g., “1.25*mean”) may also be used. If None and if available, the object attribute threshold is used. Otherwise, “mean” is used by default.

Returns:

X_r : array of shape [n_samples, n_selected_features]

The input samples with only the selected features.