Text-based feature engineering
Instruction and application
In ProgressText-based feature engineering
Text data is everywhere, from support tickets to reviews and emails, but models cannot learn from raw language directly. The first task is to convert language into structured numeric signals without losing too much meaning.
Vectorisation: turning words into numbers
Vectorisation is the foundation of text feature engineering.
Bag of words
Bag of words builds a vocabulary and represents each document as counts of the words it contains. It ignores order, but it is fast, interpretable and often surprisingly effective for tasks like spam detection or simple sentiment analysis.
N-grams
N-grams capture sequences of words, such as bigrams or trigrams. They help preserve short-range context and are useful when phrases such as “not happy” or “very poor” carry more meaning than the individual words alone.
TF-IDF
TF-IDF highlights words that are distinctive in a document while downweighting words that are common everywhere. This often improves precision over plain bag of words while keeping the representation interpretable.
Word embeddings
Embeddings such as Word2Vec, GloVe and transformer-based representations map words or documents into dense vectors that capture context and semantic similarity. They are especially useful when you need the model to generalise beyond exact word matches.
Choosing a representation
Use simpler representations when interpretability and speed matter. Move to embeddings when semantic meaning, synonym handling or large-scale generalisation becomes more important.
Topic modelling
Topic modelling groups documents by recurring word patterns, helping you discover themes without labelled data. Techniques such as LDA and NMF are useful for summarising large sets of comments, reviews or tickets.
A review corpus might surface themes like billing, delivery delays or product quality, which can then become structured features or reporting categories.
Named entity recognition
Named entity recognition extracts people, organisations, locations, dates and other specific entities from raw text. Those entities can then be counted, tagged or linked into downstream models.
For example, in a customer complaint, entities can reveal which product, team or event is associated with a negative outcome.