skoot.preprocessing.BinningTransformer

class skoot.preprocessing.BinningTransformer(cols=None, as_df=True, n_bins=10, strategy='uniform', return_bin_label=True, overwrite=True, n_jobs=1)

Bin continuous variables.

The BinningTransformer will create buckets for continuous variables, effectively transforming continuous features into categorical features.

Pros of binning:

• Particularly useful in the case of very skewed data where an algorithm may make assumptions on the underlying distribution of the variables
• Quick and easy way to take curvature into account

There are absolutely some negatives to binning:

• You can tend to throw away information from continuous variables
• You might end up fitting “wiggles” rather than a linear relationship itself
• You use up a lot of degrees of freedom

For a more exhaustive list of detrimental effects of binning, take a look at [1].

Parameters:

cols : array-like, shape=(n_features,), optional (default=None)

The names of the columns on which to apply the transformation. Optional. If None, will be applied to all features (which could prove to be expensive)

as_df : bool, optional (default=True)

Whether to return a Pandas DataFrame in the transform method. If False, will return a Numpy ndarray instead. Since most skoot transformers depend on explicitly-named DataFrame features, the as_df parameter is True by default.

n_bins : int or iterable, optional (default=10)

The number of bins into which to separate each specified feature. Default is 20, but can also be an iterable or dict of the same length as cols, where positional integers indicate a different bin size for that feature.

strategy : str or unicode, optional (default=”uniform”)

The strategy for binning. Default is “uniform”, which uniformly segments a feature. Alternatives include “percentile” which uses n_bins to compute quantiles (for n_bins=5), quartiles (for n_bins=4), etc. Note that for percentile binning, the outer bin boundaries (low boundary of lowest bin and high boundary of the highest bin) will be set to -inf and inf, respectively, to behave similar to other binning strategies.

return_bin_label : bool, optional (default=True)

Whether to return the string representation of the bin (i.e., “<25.2”) rather than the bin level, an integer.

overwrite : bool, optional (default=True)

Whether to overwrite the original feature with the binned feature. Default is True so that the output names match the input names. If False, the output columns will be appended to the right side of the frame with “_binned” appended.

n_jobs : int, 1 by default

The number of jobs to use for the encoding. This works by fitting each incremental LabelEncoder in parallel.

If -1 all CPUs are used. If 1 is given, no parallel computing code is used at all, which is useful for debugging. For n_jobs below -1, (n_cpus + 1 + n_jobs) are used. Thus for n_jobs = -2, all CPUs but one are used.

Attributes

bins_	(dict) A dictionary mapping the column names to the corresponding bins, which are internal _Bin objects that store data on upper and lower bounds.
fit_cols_	(list) The list of column names on which the transformer was fit. This is used to validate the presence of the features in the test set during the transform stage.

Notes

If a feature has fewer than n_bins unique values, it will raise a ValueError in the fit procedure.

References

[R16]

“Problems Caused by Categorizing Continuous Variables” http://biostat.mc.vanderbilt.edu/wiki/Main/CatContinuous

Examples

Bin two features in iris:

>>> from skoot.datasets import load_iris_df
>>> iris = load_iris_df(include_tgt=False, names=['a', 'b', 'c', 'd'])
>>> binner = BinningTransformer(cols=["a", "b"], strategy="uniform")
>>> trans = binner.fit_transform(iris)
>>> trans.head()
              a             b    c    d
0  (5.10, 5.50]  (3.40, 3.60]  1.4  0.2
1  (4.70, 5.10]  (3.00, 3.20]  1.4  0.2
2  (4.70, 5.10]  (3.20, 3.40]  1.3  0.2
3  (-Inf, 4.70]  (3.00, 3.20]  1.5  0.2
4  (4.70, 5.10]  (3.60, 3.80]  1.4  0.2
>>> trans.dtypes
a     object
b     object
c    float64
d    float64
dtype: object

Methods

fit(X[, y])	Fit the transformer.
fit_transform(X[, y])	Fit to data, then transform it.
get_params([deep])	Get parameters for this estimator.
set_params(**params)	Set the parameters of this estimator.
transform(X)	Apply the transformation to a dataframe.

__init__(cols=None, as_df=True, n_bins=10, strategy='uniform', return_bin_label=True, overwrite=True, n_jobs=1)

Initialize self. See help(type(self)) for accurate signature.

fit(X, y=None)

Fit the transformer.

Parameters:

X : pd.DataFrame, shape=(n_samples, n_features)

The Pandas frame to fit. The frame will only be fit on the prescribed cols (see __init__) or all of them if cols is None.

y : array-like or None, shape=(n_samples,), optional (default=None)

Pass-through for sklearn.pipeline.Pipeline.

fit_transform(X, y=None, **fit_params)

Fit to data, then transform it.

Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.

Parameters:

X : numpy array of shape [n_samples, n_features]

Training set.

y : numpy array of shape [n_samples]

Target values.

Returns:

X_new : numpy array of shape [n_samples, n_features_new]

Transformed array.

get_params(deep=True)

Get parameters for this estimator.

Parameters:

deep : boolean, optional

If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

params : mapping of string to any

Parameter names mapped to their values.

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as pipelines). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Returns:self

transform(X)

Apply the transformation to a dataframe.

This method will bin the continuous values in the test frame with the bins designated in the fit stage.

Parameters:

X : pd.DataFrame, shape=(n_samples, n_features)

The Pandas frame to transform. The operation will be applied to a copy of the input data, and the result will be returned.

Returns:

X : pd.DataFrame or np.ndarray, shape=(n_samples, n_features)

The operation is applied to a copy of X, and the result set is returned.

Examples using skoot.preprocessing.BinningTransformer

Binning transformer