Async review
Instruction and application
In ProgressRecap core topics:
Unit 2:
Feature Engineering for Different Data Types
Unit 3:
Advanced Feature Engineering Strategies
Unit 2: Feature Engineering for Different Data Types
In Unit 2, you explored…
Numerical feature engineering:
Polynomial features, mathematical transformations, binning strategies and specialised transformations (Box-Cox, Yeo-Johnson).
Categorical feature engineering:
One-hot encoding, label encoding, target encoding, embedding techniques and feature hashing.
Text-based feature engineering:
Bag-of-words, TF-IDF, word embeddings (Word2Vec, GloVe, FastText), topic modelling (LDA, NMF) and named entity recognition.
Image-based feature engineering:
Colour histograms, texture descriptors, edge detection, feature descriptors (SIFT, SURF, ORB) and dimensionality reduction.
Time-based feature engineering:
Lag features, rolling window statistics, seasonality extraction, time decomposition and event encoding.
Unit 3: Advanced Feature Engineering Strategies
In Unit 3, you explored…
Feature selection and dimensionality reduction:
Methods including filter, wrapper and embedded selection (LASSO, tree-based models), along with dimensionality reduction techniques (PCA, t-SNE, UMAP) to optimise model performance and efficiency.
Automated feature engineering:
AI-driven tools like Featuretools and tsfresh for generating and selecting features efficiently.
AI-driven feature engineering:
Automated tools including Featuretools and tsfresh for feature extraction, deep feature synthesis and AI-powered methods such as LLM-based feature generation, domain-specific embeddings and synthetic data augmentation.
Domain-specific feature engineering:
Industry-focused techniques for financial data (credit risk indicators, ratios), health care (comorbidity indices), retail (customer segmentation), IoT (sensor feature extraction) and image data (non-deep learning methods).
Feature engineering in production:
Best practices for deploying engineered features, managing feature pipelines, monitoring feature drift, handling versioning and ensuring scalability in real-time and batch ML workflows.
Handling temporal data
Time-series data requires specialised feature engineering techniques to capture trends and seasonality. Key methods include:
Lag features:
Using past values as predictive inputs, such as using yesterday’s sales to predict today’s.
Rolling statistics:
Computing moving averages, minimum/maximum values or standard deviation over a specified time window to smooth fluctuations.
Date part extraction:
Breaking down timestamps into useful components like hour, day, month, weekday or weekend flag to capture periodic patterns.
Automated feature engineering
Automating feature generation can save time and improve efficiency when working with large datasets. Common approaches include:
Tools like
tsfresh: Automatically extract relevant statistical features from time-series data, reducing manual effort.
Deep Feature Synthesis (DFS):
Creates new features by programmatically combining existing ones, making it valuable for relational datasets and time-series analysis.
Feature selection and dimensionality reduction
Selecting the right features improves model efficiency and performance. Important techniques include:
Correlation analysis:
Identifies and removes highly correlated features to reduce redundancy.
Principal Component Analysis (PCA):
Reduces the feature space while retaining key variance, helping to simplify complex datasets.
Embedded selection:
Uses machine learning models like LASSO (L1 regularisation) or tree-based feature importance methods to rank and select the most relevant features.
Action item: Time and categories Poll
Let’s do a quick poll to check our understanding of time-based and categorical features in machine learning. No pressure—just pick the best answer!