Everyone talks about LLMs.
Nobody explains how they actually work under the hood.
GPT. Claude. Gemini. Llama.
They all come from the same 5-stage pipeline.
And once you understand it, you can build one yourself.
Not a GPT-4 clone.
But a real working language model.
One that learns from text, generates new text, and actually makes sense.
I built one. Here's exactly how it works.
No PhD required. Code included.
The lie everyone believes about LLMs
Most people think building an LLM is about the architecture.
Transformers. Attention heads. Layers.
It is not.
The transformer architecture is publicly published. Every major lab uses roughly the same building blocks.
If architecture were the secret, everyone would have GPT-4.
The real secret: data, training, and alignment.
Architecture is one paragraph. Everything else is where real models are won and lost.
Here are the 5 stages.
Stage 1 — Data (Where models are actually won)
Raw internet text is filthy.
You cannot train on it directly.
Common Crawl — the public web scrape used to train most LLMs — contains 250 billion pages and over a million gigabytes.
But most of it is garbage.
Duplicate headers. Spam. Toxic content. Personal data. Low-quality pages with 3 words.
Before any training happens, you run a brutal multi-step filter:
→ Extract clean text from raw HTML
→ Filter harmful, NSFW, and personal data
→ Deduplicate by URL, document, and line
→ Remove low-quality docs by word count and token density
→ Run model-based quality scoring — would a Wikipedia editor cite this page?
→ Balance the data mix across code, books, science, and web
The result: a clean dataset that's a fraction of the original size but dramatically better.
The rule to burn in:
Data quality beats data quantity. Every time.
The most guarded secret in the field is not the architecture.
It's how the data was cleaned.
Stage 2 — Tokenization (Break text into pieces the model can actually learn from)
The model never reads raw text.
It reads tokens.
A token is not always a full word. It is a piece of a word — a fragment the model learned to treat as a unit.
"playing" → ["play", "ing"] "unbelievable" → ["un", "believ", "able"] "dog" → ["dog"]
The standard method is called Byte-Pair Encoding (BPE).
It starts with individual characters and merges the most common pairs repeatedly until it has a fixed vocabulary — usually 32,000 to 100,000 tokens.
Here is a minimal tokenizer in Python:
from tokenizers import Tokenizer, models, trainers, pre_tokenizers
# Initialize BPE tokenizer
tokenizer = Tokenizer(models.BPE())
tokenizer.pre_tokenizer = pre_tokenizers.Whitespace()
# Train on your corpus
trainer = trainers.BpeTrainer(
vocab_size=32000,
special_tokens=["<PAD>", "<BOS>", "<EOS>", "<UNK>"]
)
tokenizer.train(files=["your_data.txt"], trainer=trainer)
tokenizer.save("tokenizer.json")
# Test it
output = tokenizer.encode("Building an LLM from scratch is powerful")
print(output.tokens)
# ['Building', 'an', 'LL', 'M', 'from', 'scratch', 'is', 'powerful']
print(output.ids)
# [4821, 271, 3728, 44, 505, 8905, 318, 6787]
Rule of thumb: 1 token ≈ 0.75 words.
1,000 tokens ≈ 750 words.
A 100k context window = roughly a full novel.
Stage 3 — Training (One deceptively simple objective)
The entire training task sounds too simple to be powerful:
Predict the next token.
That is it.
Given "The cat sat on the", predict "mat".
Do this across trillions of examples and something remarkable happens.
The model learns grammar. Then facts. Then reasoning. Then how to write code, translate languages, solve math.
Nobody taught it any of that.
It emerged from next-token prediction at massive scale.
Here is a minimal decoder-only transformer in PyTorch — the same architecture behind every major LLM:
import torch
import torch.nn as nn
import math
class CausalSelfAttention(nn.Module):
def __init__(self, embed_dim, num_heads):
super().__init__()
self.num_heads = num_heads
self.head_dim = embed_dim // num_heads
self.qkv = nn.Linear(embed_dim, 3 * embed_dim, bias=False)
self.proj = nn.Linear(embed_dim, embed_dim, bias=False)
def forward(self, x):
B, T, C = x.shape
q, k, v = self.qkv(x).chunk(3, dim=-1)
# Split into heads
q = q.view(B, T, self.num_heads, self.head_dim).transpose(1, 2)
k = k.view(B, T, self.num_heads, self.head_dim).transpose(1, 2)
v = v.view(B, T, self.num_heads, self.head_dim).transpose(1, 2)
# Scaled dot-product attention with causal mask
scale = math.sqrt(self.head_dim)
attn = (q @ k.transpose(-2, -1)) / scale
# Causal mask: can only attend to past tokens
mask = torch.tril(torch.ones(T, T, device=x.device))
attn = attn.masked_fill(mask == 0, float('-inf'))
attn = torch.softmax(attn, dim=-1)
out = (attn @ v).transpose(1, 2).contiguous().view(B, T, C)
return self.proj(out)
class TransformerBlock(nn.Module):
def __init__(self, embed_dim, num_heads, ff_dim, dropout=0.1):
super().__init__()
self.attn = CausalSelfAttention(embed_dim, num_heads)
self.ff = nn.Sequential(
nn.Linear(embed_dim, ff_dim),
nn.GELU(),
nn.Linear(ff_dim, embed_dim),
nn.Dropout(dropout)
)
self.ln1 = nn.LayerNorm(embed_dim)
self.ln2 = nn.LayerNorm(embed_dim)
def forward(self, x):
x = x + self.attn(self.ln1(x)) # attention + residual
x = x + self.ff(self.ln2(x)) # feedforward + residual
return x
class MiniLLM(nn.Module):
def __init__(self, vocab_size, embed_dim, num_heads,
ff_dim, num_layers, max_seq_len, dropout=0.1):
super().__init__()
self.token_emb = nn.Embedding(vocab_size, embed_dim)
self.pos_emb = nn.Embedding(max_seq_len, embed_dim)
self.blocks = nn.ModuleList([
TransformerBlock(embed_dim, num_heads, ff_dim, dropout)
for _ in range(num_layers)
])
self.ln_final = nn.LayerNorm(embed_dim)
self.output = nn.Linear(embed_dim, vocab_size, bias=False)
self.dropout = nn.Dropout(dropout)
def forward(self, token_ids):
B, T = token_ids.shape
positions = torch.arange(T, device=token_ids.device).unsqueeze(0)
x = self.dropout(
self.token_emb(token_ids) + self.pos_emb(positions)
)
for block in self.blocks:
x = block(x)
x = self.ln_final(x)
return self.output(x) # logits over vocabulary
# Initialize a small but real model
model = MiniLLM(
vocab_size=32000,
embed_dim=512,
num_heads=8,
ff_dim=2048,
num_layers=6,
max_seq_len=1024
)
total_params = sum(p.numel() for p in model.parameters())
print(f"Parameters: {total_params:,}")
# Parameters: 44,082,176 — a 44M parameter model
Now the training loop:
import torch.optim as optim
from torch.nn.utils import clip_grad_norm_
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)
optimizer = optim.AdamW(model.parameters(), lr=3e-4, weight_decay=0.01)
criterion = nn.CrossEntropyLoss()
def train_epoch(model, dataloader):
model.train()
total_loss = 0
for input_ids, target_ids in dataloader:
input_ids = input_ids.to(device)
target_ids = target_ids.to(device)
# Forward pass
logits = model(input_ids)
# Flatten for loss calculation
loss = criterion(
logits.view(-1, logits.size(-1)), # (batch * seq_len, vocab)
target_ids.view(-1) # (batch * seq_len)
)
# Backward pass
optimizer.zero_grad()
loss.backward()
clip_grad_norm_(model.parameters(), max_norm=1.0) # prevent explosions
optimizer.step()
total_loss += loss.item()
return total_loss / len(dataloader)
What the model is actually learning:
→ Every input token attends to every previous token
→ The causal mask prevents looking into the future
→ Loss = how surprised the model was by the real next token
→ Lower loss = better predictions = model is learning language
Stage 4 — Alignment (Turn a text predictor into an assistant)
After pretraining you have something impressive but useless for chat.
Ask it a question and it might reply with three more questions.
Because predicting the next token doesn't mean understanding what you want.
Two steps fix this.
Step 1: Supervised Fine-Tuning (SFT)
Show the model thousands of examples:
prompt → ideal response
The model learns to imitate the format of a good answer.
The surprising part: you need very little data.
A few thousand examples is enough because the knowledge is already inside the pretrained model.
SFT just teaches it to express that knowledge in the right format.
# SFT training example structure
sft_examples = [
{
"prompt": "Explain what an API is in simple terms.",
"response": "An API is like a waiter in a restaurant. You (the app) tell the waiter (API) what you want. The waiter goes to the kitchen (server), gets it, and brings it back. You never go to the kitchen directly."
},
{
"prompt": "What is the capital of France?",
"response": "The capital of France is Paris."
}
# A few thousand of these is enough
]
# Format as: <prompt> [SEP] <response> <EOS>
# Fine-tune the pretrained model on these pairs
# Same training loop as pretraining — just different data
Step 2: RLHF (Reinforcement Learning from Human Feedback)
SFT teaches format. RLHF teaches preference.
The model generates two answers. A human picks the better one. Those preferences train a reward model. The LLM is optimized to maximize that reward.
This is why ChatGPT and Claude feel like assistants — not random text generators.
Without RLHF:
→ Fluent. Capable. But unreliable.
→ Confidently wrong.
→ Doesn't know when to say "I don't know."
With RLHF:
→ Helpful. Clear. Safe.
→ Learns what "a good answer" actually means.
Stage 5 — Evaluation (Prove it actually works)
Building a model without measuring it is guessing.
During pretraining — measure perplexity.
Perplexity measures how "surprised" the model is by real text.
Lower perplexity = model predicts text better = it's learning.
Between 2017 and 2023, the best models dropped from perplexing among ~70 possible tokens to fewer than 10.
import torch
import math
def calculate_perplexity(model, dataloader, device):
model.eval()
total_loss = 0
total_tokens = 0
criterion = nn.CrossEntropyLoss(reduction='sum')
with torch.no_grad():
for input_ids, target_ids in dataloader:
input_ids = input_ids.to(device)
target_ids = target_ids.to(device)
logits = model(input_ids)
loss = criterion(
logits.view(-1, logits.size(-1)),
target_ids.view(-1)
)
total_loss += loss.item()
total_tokens += target_ids.numel()
avg_loss = total_loss / total_tokens
perplexity = math.exp(avg_loss)
return perplexity
# Example output progression:
# Epoch 1: Perplexity = 847.3 (model knows almost nothing)
# Epoch 5: Perplexity = 124.6 (getting better)
# Epoch 20: Perplexity = 23.4 (actually learning language)
After alignment — perplexity stops working.
Fine-tuned models score worse on perplexity while being far more useful.
You need human benchmarks:
→ MMLU: 57 academic subjects, multiple choice — measures knowledge
→ Chatbot Arena: humans blind-compare two models and vote — measures preference
→ AlpacaEval: LLM judges LLM — 98% correlation with human judges, costs $10
The honest truth: no single score captures a good model.
The same model scores 0.637 or 0.488 on the same benchmark depending only on how the prompt is formatted.
Evaluation is genuinely hard.
And no one has fully solved it.
How to generate text from your model
The model is trained.
Now make it generate something.
def generate(model, tokenizer, prompt, max_new_tokens=100,
temperature=0.8, device='cuda'):
model.eval()
# Encode prompt to token IDs
token_ids = tokenizer.encode(prompt).ids
input_tensor = torch.tensor(token_ids, dtype=torch.long,
device=device).unsqueeze(0)
with torch.no_grad():
for _ in range(max_new_tokens):
# Keep only last max_seq_len tokens (context window)
context = input_tensor[:, -1024:]
# Forward pass
logits = model(context)
# Get logits for last token only
next_token_logits = logits[:, -1, :]
# Apply temperature (higher = more creative)
next_token_logits = next_token_logits / temperature
# Sample from probability distribution
probs = torch.softmax(next_token_logits, dim=-1)
next_token = torch.multinomial(probs, num_samples=1)
# Append and continue
input_tensor = torch.cat([input_tensor, next_token], dim=1)
# Stop at end of sequence token
if next_token.item() == tokenizer.token_to_id("<EOS>"):
break
# Decode back to text
generated_ids = input_tensor[0].tolist()
return tokenizer.decode(generated_ids)
# Test it
output = generate(
model, tokenizer,
prompt="The most important thing about machine learning is",
max_new_tokens=100,
temperature=0.8
)
print(output)
Temperature controls creativity:
→ temperature = 0.1 → safe, predictable, repetitive
→ temperature = 0.8 → natural, varied, good default
→ temperature = 1.5 → creative, surprising, sometimes incoherent
What the full pipeline looks like
Before: raw internet text, 1 million GB, completely unusable.
After Stage 1: clean filtered dataset, ready to train on.
Before: raw text, meaningless to a model.
After Stage 2: tokens with IDs, the model's native language.
Before: random weights, outputs garbage.
After Stage 3: a model that understands language patterns.
Before: a text predictor that answers questions with more questions.
After Stage 4: an assistant that follows instructions and is safe.
Before: no idea if the model is actually good.
After Stage 5: benchmarks, perplexity scores, human evaluations.
Each stage builds on the last.
Skip any one and the whole thing breaks.
The 5 mistakes that sink LLM projects
1. Obsessing over architecture.
Transformers are standardized. Published. Copied.
The architecture is the least important part.
2. Treating data as a commodity.
Dirty data caps your ceiling regardless of compute.
The top labs spend more on data cleaning than model design.
3. Skipping the scaling math.
A model too big for its data is undertrained and wastes compute.
Optimal ratio: ~20 tokens of training data per parameter.
4. Stopping at SFT.
A fine-tuned model imitates. Without RLHF it never learns what people actually prefer.
5. Trusting perplexity after alignment.
Post-training changes the distribution.
Perplexity stops being meaningful the moment you run SFT.
Switch to human benchmarks immediately.
The uncomfortable truth
A great LLM is not trained.
It is engineered.
5 stages. Not 1.
Architecture is one paragraph inside Stage 3.
Everything that matters is in the other four.
Data quality. Scaling math. Alignment. Honest evaluation.
That's what separates GPT-4 from a hobby model.
Two labs with identical architecture produce wildly different models.
The architecture is shared.
Everything that actually matters is not.
Start here
Want to run this yourself? Here is the minimal setup:
# Full minimal LLM training setup
# Requirements: pip install torch tokenizers datasets
# 1. Get data
from datasets import load_dataset
dataset = load_dataset("wikitext", "wikitext-103-v1", split="train")
text = "\n\n".join([t.strip() for t in dataset['text'] if t.strip()])
# 2. Train tokenizer
from tokenizers import Tokenizer, models, trainers, pre_tokenizers
tokenizer = Tokenizer(models.BPE())
tokenizer.pre_tokenizer = pre_tokenizers.Whitespace()
trainer = trainers.BpeTrainer(vocab_size=8000,
special_tokens=["<PAD>","<BOS>","<EOS>"])
with open("corpus.txt", "w") as f:
f.write(text[:5_000_000]) # use 5MB to start
tokenizer.train(["corpus.txt"], trainer)
# 3. Build model (use MiniLLM class from Stage 3 above)
model = MiniLLM(
vocab_size=8000,
embed_dim=256, # small but real
num_heads=8,
ff_dim=1024,
num_layers=4,
max_seq_len=256
)
# ~15M parameters — trains on a laptop GPU in hours
# 4. Train (use train_epoch from Stage 3 above)
# 5. Generate (use generate() from Stage 5 above)
print("Your LLM is running.")
Start small. 15M parameters. WikiText dataset. Free GPU on Google Colab.
Watch the perplexity drop from 800 to 50 over a few hours.
That drop is the model learning language.
In real time.
That's the moment everything clicks.
Now make it useful: build a real niche LLM
WikiText is a learning dataset.
The real money — and the real fun — is training on a specific domain and watching your model become an expert in exactly one thing.
Here are 5 niches you can build right now. Same pipeline. Different data.
Niche 1 — Coding Assistant LLM (The main example: highest impact, most dramatic results)
This is the one to build first.
The pain is universal. Every developer has hit this:
→ You stare at a function that isn't working.
→ Stack Overflow has 12 answers, all from 2014.
→ You paste into ChatGPT and get something almost right.
A coding LLM trained on the right data does this natively, offline, and tuned to your exact stack.
The data:
from datasets import load_dataset
# The Code dataset — 6.4M Python files from GitHub
code_dataset = load_dataset("codeparrot/github-code",
languages=["Python"],
streaming=True,
split="train")
# Stack Overflow Q&A pairs — real developer problems + accepted answers
so_dataset = load_dataset("koutch/stackoverflow_python",
split="train")
# The Stack — 30 programming languages, cleaned and deduplicated
the_stack = load_dataset("bigcode/the-stack",
data_dir="data/python",
split="train",
streaming=True)
What the training pairs look like:
# Format 1: Code completion
# Input: function signature + docstring
# Target: complete implementation
input_example = """
def calculate_compound_interest(principal, rate, time, n=12):
\"\"\"
Calculate compound interest.
Args:
principal: Initial investment amount
rate: Annual interest rate (as decimal, e.g. 0.05 for 5%)
time: Time period in years
n: Compounding frequency per year (default: monthly)
Returns:
Final amount after compound interest
\"\"\"
"""
target_example = """
amount = principal * (1 + rate/n) ** (n * time)
return round(amount, 2)
"""
# Format 2: Error → Fix pairs (from Stack Overflow)
input_example_2 = """
# ERROR: TypeError: unsupported operand type(s) for +: 'int' and 'str'
user_age = input("Enter your age: ")
next_year_age = user_age + 1
print(f"Next year you will be {next_year_age}")
# Fix this code:
"""
target_example_2 = """
user_age = int(input("Enter your age: ")) # convert string to int
next_year_age = user_age + 1
print(f"Next year you will be {next_year_age}")
"""
# Format 3: Natural language → Code (instruction following)
input_example_3 = """
# Write a Python function that:
# - Takes a list of dictionaries
# - Filters by a given key-value pair
# - Returns sorted results by a specified field
"""
target_example_3 = """
def filter_and_sort(data, filter_key, filter_value, sort_field):
filtered = [item for item in data
if item.get(filter_key) == filter_value]
return sorted(filtered, key=lambda x: x.get(sort_field, ''))
"""
The before/after that sells this:
Before training:
Prompt: "Write a Python decorator that retries a function on failure"
Output: "A decorator is a design pattern in Python that allows..."
# Generic. Useless. Textbook answer.
After 10 hours of training on GitHub + Stack Overflow:
Prompt: "Write a Python decorator that retries a function on failure" Output:
import time
from functools import wraps
def retry(max_attempts=3, delay=1.0, exceptions=(Exception,)):
def decorator(func):
@wraps(func)
def wrapper(*args, **kwargs):
for attempt in range(max_attempts):
try:
return func(*args, **kwargs)
except exceptions as e:
if attempt == max_attempts - 1:
raise
time.sleep(delay * (attempt + 1))
return wrapper
return decorator
# Usage:
@retry(max_attempts=3, delay=0.5, exceptions=(ConnectionError,))
def fetch_data(url):
...
# Correct. Production-ready. Exponential backoff included.
The model learned what a senior developer would write.
Not from being told. From reading 6 million Python files and every accepted Stack Overflow answer.
Niche 2 — SQL Query Generator
The pain: every non-technical founder has data they cannot access.
The data is there. In their database. They just cannot write the query.
They describe what they want in plain English. Your model writes the SQL.
The data:
# Spider dataset — 10,000+ SQL queries with natural language descriptions
# Covers 200 databases across 138 domains
spider = load_dataset("spider", split="train")
# WikiSQL — 80,654 natural language + SQL pairs
wikisql = load_dataset("wikisql", split="train")
# What a training pair looks like:
example = {
"question": "Show me all users who signed up in the last 30 days and made at least one purchase",
"sql": """
SELECT u.id, u.email, u.created_at, COUNT(p.id) as purchase_count
FROM users u
JOIN purchases p ON u.id = p.user_id
WHERE u.created_at >= NOW() - INTERVAL '30 days'
GROUP BY u.id, u.email, u.created_at
HAVING COUNT(p.id) >= 1
ORDER BY u.created_at DESC;
"""
}
The before/after:
# Non-technical founder types:
"Which of my customers spent the most money last quarter
but haven't bought anything in the last 60 days?"
# Model outputs:
SELECT
c.customer_id,
c.email,
SUM(o.total_amount) as q_spend,
MAX(o.created_at) as last_order_date
FROM customers c
JOIN orders o ON c.customer_id = o.customer_id
WHERE o.created_at BETWEEN DATE_TRUNC('quarter', NOW() - INTERVAL '3 months')
AND DATE_TRUNC('quarter', NOW())
GROUP BY c.customer_id, c.email
HAVING MAX(o.created_at) < NOW() - INTERVAL '60 days'
ORDER BY q_spend DESC
LIMIT 20;
Who pays for this: every SaaS founder, every e-commerce operator, every analyst who sits between a business owner and a database.
They will pay $20/month without thinking.
Niche 3 — US Legal Document Summarizer
The pain: a 40-page contract. A lease agreement. An NDA.
Most people sign without understanding it.
A lawyer charges $300/hour to read it.
Your model reads it in 3 seconds.
The data:
# Free Law Project — millions of US court opinions, public domain
free_law = load_dataset("free-law/courtlistener-opinion-clustering",
split="train")
# MultiLegalPile — legal text across US jurisdictions
multi_legal = load_dataset("joelniklaus/multi_legal_pile",
languages=["en"],
split="train")
# What training pairs look like:
example = {
"input": """
SECTION 12.4 INDEMNIFICATION. Licensee shall defend, indemnify,
and hold harmless Licensor and its officers, directors, employees,
agents, and successors from and against any and all losses, damages,
liabilities, deficiencies, claims, actions, judgments, settlements,
interest, awards, penalties, fines, costs, or expenses of whatever
kind, including reasonable attorneys' fees, that are incurred by
Licensor arising out of or relating to Licensee's breach of any
representation, warranty, covenant, or obligation under this Agreement...
[continues for 3 more paragraphs of dense legalese]
""",
"summary": """
PLAIN ENGLISH: If you break any part of this contract and it causes
the other party legal trouble or financial loss, YOU pay for everything —
their lawyers, court costs, damages, all of it. This is a broad
indemnification clause that heavily favors the licensor.
RED FLAGS:
• "any and all losses" — extremely broad, no cap on liability
• "reasonable attorneys' fees" — you pay their legal bills too
• No carve-out for licensor's own negligence
NEGOTIATE: Ask to add "except to the extent caused by Licensor's
gross negligence or willful misconduct" before signing.
"""
}
The output format that makes this worth paying for:
Input: [paste any US contract clause]
Output:
━━━━━━━━━━━━━━━━━━
PLAIN ENGLISH SUMMARY
━━━━━━━━━━━━━━━━━━
[2-3 sentences anyone can understand]
RED FLAGS 🚩
• [specific risks in this clause]
• [what it lets the other party do]
MISSING PROTECTIONS ⚠️
• [what a balanced clause would include]
• [what a lawyer would ask to add]
RISK LEVEL: High / Medium / Low
━━━━━━━━━━━━━━━━━━
Who pays for this: freelancers reviewing client contracts, small business owners, renters reviewing leases, startup founders signing their first SaaS agreements.
The market is everyone who cannot afford a lawyer but still has to sign things.
Niche 4 — Medical Symptom Explainer
The pain: you Google a symptom and WebMD tells you that you have 3 days to live.
A medical LLM trained on clinical notes and patient education materials does something different.
It explains what the symptom actually means. In plain English. Without catastrophizing.
# PubMed abstracts — 35 million medical research papers
pubmed = load_dataset("pubmed", split="train")
# MedQA — US Medical Licensing Exam questions + answers
medqa = load_dataset("bigbio/med_qa", split="train")
# Training pair format:
example = {
"patient_question": "I've had a dull ache behind my left eye for 3 days. It gets worse when I lean forward. No fever. What could this be?",
"clinical_explanation": """
Based on your description — dull ache behind one eye, worsening when leaning forward,
no fever, lasting 3 days — the most likely explanation is:
MOST LIKELY: Sinusitis (sinus pressure)
The maxillary or frontal sinuses sit directly behind and above your eyes.
Inflammation or congestion increases pressure when you change head position.
The fact that it worsens leaning forward is a classic sinusitis pattern.
ALSO POSSIBLE: Tension headache, eye strain, or mild migraine variant
LESS LIKELY but worth noting: If you also have vision changes, nausea,
or the pain is sudden and severe — see a doctor same day.
WHAT TO TRY: Saline nasal rinse, steam inhalation, ibuprofen for pressure relief.
If no improvement in 7-10 days or fever develops → see a doctor.
NOTE: This is general health information, not a diagnosis.
Always consult a licensed physician for medical advice.
"""
}
The key: every response ends with a clear disclaimer and escalation signal.
That is what separates useful health information from dangerous advice.
Niche 5 — E-commerce Product Description Writer
The pain: a Shopify store with 500 products. Every description is a wall of spec sheet text nobody reads.
Good product descriptions do one thing: make someone feel something.
An LLM trained on high-converting product copy learns exactly what words drive clicks.
# Training data structure:
example = {
"product_specs": """
Product: Ceramic Coffee Mug
Material: Stoneware ceramic
Capacity: 14oz
Dimensions: 3.5" diameter x 4.2" height
Colors: Matte black, cream white, sage green
Dishwasher safe: Yes
Microwave safe: Yes
Weight: 0.8 lbs
""",
"high_converting_description": """
Some mornings deserve better than a paper cup.
This is the mug that stays on your desk. The one your coworkers
ask about. Heavy enough to feel intentional. Smooth enough to
actually enjoy holding at 7am.
14oz — the right amount. Not the novelty bucket. Not the tiny
espresso thing. The one you actually finish.
Matte stoneware that doesn't show fingerprints. Dishwasher safe
because life is short. Three colors that work with any kitchen
that isn't trying too hard.
You already have mugs.
You don't have this one.
""",
"meta_description": "Stoneware ceramic mug, 14oz, matte finish. Dishwasher and microwave safe. The coffee mug that stays.",
"keywords": ["ceramic coffee mug", "stoneware mug", "matte black mug",
"14oz mug", "minimalist coffee mug", "handmade style mug"]
}
The data source: scrape the top 1,000 Shopify stores by traffic. Extract product titles, specs, and descriptions. Filter for products with high review counts — those descriptions are proven to convert. Train on those.
Your model learns the difference between spec sheets and copy that sells.
The pattern across all 5 niches
Look at what they share:
→ A clear, specific pain the user feels every day
→ A data source that already exists and is publicly available
→ A before/after that is immediately obvious to anyone in that profession
→ A willingness to pay because the alternative costs more time or money
The 5-stage pipeline is identical for every single one.
You change only one thing: the training data.
Same tokenizer setup. Same transformer architecture. Same training loop. Same evaluation method.
Different data → different expert → different product.
That is the leverage.
One pipeline. Five products. Five revenue streams.
If this was useful:
→ Repost to share it with every developer learning AI
→ Follow @sairahul1 for more breakdowns like this
→ Bookmark this — the code works, run it tonight
I write about AI, building products, and systems that work while you sleep.