skoot.feature_selection.LinearCombinationFilter

class skoot.feature_selection.LinearCombinationFilter(cols=None, as_df=True)

Filter any perfect linear combinations in a matrix.

The LinearCombinationFilter will resolve linear combinations in a numeric matrix. The QR decomposition is used to determine whether the matrix is full rank, and then identify the sets of columns that are involved in the dependencies. This class is adapted from the implementation in the R package, caret.

Parameters:

cols : array-like, shape=(n_features,), optional (default=None)

The names of the columns on which to apply the transformation. If no column names are provided, the transformer will be fit on the entire frame. Note that the transformation will also only apply to the specified columns, and any other non-specified columns will still be present after transformation. Note that since this transformer can only operate on numeric columns, not explicitly setting the cols parameter may result in errors for categorical data.

as_df : bool, optional (default=True)

Whether to return a Pandas DataFrame in the transform method. If False, will return a Numpy ndarray instead. Since most skoot transformers depend on explicitly-named DataFrame features, the as_df parameter is True by default.

Attributes

drop_

(array-like, shape=(n_features,)) Assigned after calling fit. These are the features that are designated as “bad” and will be dropped in the transform method.

References

[R13]

Caret’s filterLinearCombos script - https://bit.ly/2uA6vSX

Examples

An example linear combination filter:

>>> from skoot.datasets import load_iris_df
>>>
>>> X = load_iris_df(include_tgt=False)
>>> filterer = LinearCombinationFilter()
>>> X_transform = filterer.fit_transform(X)
>>> assert X_transform.shape[1] == 4 # no combos in iris...

Methods

fit(X[, y])	Fit the transformer.
fit_transform(X[, y])	Fit to data, then transform it.
get_params([deep])	Get parameters for this estimator.
set_params(**params)	Set the parameters of this estimator.
transform(X)	Transform a test dataframe.

__init__(cols=None, as_df=True)

Initialize self. See help(type(self)) for accurate signature.

fit(X, y=None)

Fit the transformer.

Parameters:

X : pd.DataFrame, shape=(n_samples, n_features)

The Pandas frame to fit. The frame will only be fit on the prescribed cols (see __init__) or all of them if cols is None. Furthermore, X will not be altered in the process of the fit.

y : array-like or None, shape=(n_samples,), optional (default=None)

Pass-through for sklearn.pipeline.Pipeline. Even if explicitly set, will not change behavior of fit.

fit_transform(X, y=None, **fit_params)

Fit to data, then transform it.

Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.

Parameters:

X : numpy array of shape [n_samples, n_features]

Training set.

y : numpy array of shape [n_samples]

Target values.

Returns:

X_new : numpy array of shape [n_samples, n_features_new]

Transformed array.

get_params(deep=True)

Get parameters for this estimator.

Parameters:

deep : boolean, optional

If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

params : mapping of string to any

Parameter names mapped to their values.

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as pipelines). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Returns:self

transform(X)

Transform a test dataframe.

Parameters:

X : pd.DataFrame, shape=(n_samples, n_features)

The Pandas frame to transform. The operation will be applied to a copy of the input data, and the result will be returned.

Returns:

X_select : pd.DataFrame, shape=(n_samples, n_features)

The selected columns from X.