Epochs, Batches, and Iterations — Explained Simply

What Epochs, Batches, and Iterations Actually Are

You Are Studying For a Very Important Exam

You have a textbook with 1000 pages. Three decisions to make:

• How many times do you read the whole textbook? → Epochs
• How many pages do you read before stopping to take notes? → Batch size
• How many note-taking sessions does that create? → Iterations

Epoch — Reading the Whole Textbook Once

One epoch means your model has seen every single training example exactly once.

1000 training samples + batch size 32 + 50 epochs:

After epoch 1  → Model has seen everything once
After epoch 10 → Model has seen everything ten times
After epoch 50 → Model has seen everything fifty times

for epoch in range(50):          # 50 epochs
    for X_batch, y_batch in train_loader:
        pass                     # Train on each batch
    print(f"Epoch {epoch + 1} complete")

How Loss Changes Across Epochs

Epoch 1:   Loss = 1.842  ← big improvement
Epoch 2:   Loss = 0.923  ← still improving
Epoch 5:   Loss = 0.412  ← slowing down
Epoch 10:  Loss = 0.187  ← small refinements
Epoch 20:  Loss = 0.093  ← almost converged
Epoch 50:  Loss = 0.043  ← tiny improvements
Epoch 100: Loss = 0.041  ← barely changing — stop here

Batch — How Many Pages Before Taking Notes

A batch is a chunk of training samples the model processes before updating its weights.

Total training samples: 1000
Batch size:             32
→ Model updates weights after every 32 samples

loader = DataLoader(
    dataset,
    batch_size=32,     # 32 samples per batch
    shuffle=True       # Mix order each epoch
)

Why Not All at Once?

1 million samples → wait to see all before one update → incredibly slow.

Why Not One at a Time?

One sample → very noisy unreliable signal → unstable training.

Batch size 1    → Very noisy   → unstable
Batch size 32   → Balanced     → stable and fast ← start here
Batch size 256  → Smooth       → needs more memory

Iteration — One Note-Taking Session

One iteration = one weight update = model sees one batch, computes loss, runs backprop, adjusts weights.

Total samples:           1000
Batch size:              32
Iterations per epoch:    1000 / 32 = 32 iterations
Epochs:                  50
Total iterations:        32 × 50 = 1600 weight updates

All Three Together in Code

dataset      = NigerianReviewDataset(reviews, labels)
train_loader = DataLoader(dataset, batch_size=32, shuffle=True)

for epoch in range(50):                                        # EPOCH
    for iteration, (X_batch, y_batch) in enumerate(train_loader):  # ITERATION
                                                               # X_batch = BATCH
        optimizer.zero_grad()
        predictions = model(X_batch)
        loss        = loss_fn(predictions, y_batch)
        loss.backward()
        optimizer.step()

        if iteration % 10 == 0:
            print(f"Epoch {epoch+1} | Iter {iteration+1} | Loss: {loss:.4f}")

    print(f"──── Epoch {epoch+1} complete ────")

How Many Epochs Should You Train For

Too few  → underfitting → model has not learned enough
Too many → overfitting  → model has memorised training data

Signs to stop:
  ✅ Validation loss has stopped improving
  ✅ Both train and val loss are low
  ✅ Gap between them is small

Signs of too long:
  ❌ Train loss keeps going down
  ❌ Val loss starts going back UP

Early Stopping

best_val_loss = float('inf')
patience      = 5
no_improve    = 0

for epoch in range(100):
    train_loss = train_epoch(model, train_loader)
    val_loss   = evaluate(model, val_loader)

    if val_loss < best_val_loss:
        best_val_loss = val_loss
        torch.save(model.state_dict(), 'best_model.pth')
        no_improve    = 0
    else:
        no_improve += 1

    if no_improve >= patience:
        print(f"Early stopping at epoch {epoch+1}")
        break

Common Batch Sizes

The Real Words Mapped to the Story

The One Thing to Remember

An epoch is one full pass through your data. A batch is the chunk you process at once. An iteration is one weight update. 1000 samples + batch size 32 + 50 epochs = 1600 weight updates total.