Categorical feature engineering
Instruction and application
In ProgressCategorical feature engineering
Not all useful data arrives as numbers. Product type, plan name, job title and browser version can all carry important signal, but only if you encode them in a way the model can use.
Each encoding method suits a different type of categorical problem.
| Technique | Best used when | Key caution |
|---|---|---|
| One-hot encoding | Low-cardinality features with no natural order | Can create too many columns for high-cardinality features |
| Label encoding | Ordinal categories or tree-based models | Can create false order for unordered categories |
| Target encoding | High-cardinality features with strong target signal | Must guard against leakage with cross-validation |
| Embeddings | Large-scale deep learning tasks | Lower interpretability |
| Feature hashing | Streaming or open-ended category spaces | Hash collisions can merge different categories |
One-hot encoding
One-hot encoding creates a new binary column for each category. It works well for small, unordered category sets such as subscription plans or device types.
For example, a subscription_type feature with values Basic, Standard and Premium becomes three binary columns.
Label encoding
Label encoding assigns an integer to each category. It is especially useful for ordinal variables like risk level or education level, where the ranking itself carries meaning.
Target encoding
Target encoding replaces a category with a statistic derived from the target, such as mean conversion rate or churn rate. This can be powerful for high-cardinality features like product IDs, but it must be computed carefully to avoid leaking future information.
Embeddings
Embeddings let a deep model learn dense vector representations for large category spaces. They are common in recommendation systems and other settings where similarity between categories matters.
Feature hashing
Feature hashing maps categories into a fixed number of bins with a hash function. This is efficient when you do not know every possible category ahead of time, such as with user-agent strings or campaign names.
Practical rule of thumb
Start with the simplest encoding that fits the data shape. Move to target encoding, hashing or embeddings only when the number of unique categories, the model type or the production setting makes simple methods impractical.