NVIDIA · NCP-GENL

The regression-measurement thread

Knowing whether a change actually helped. Opens in M3's five-step EDA checklist before any fine-tuning starts, runs through M6's perplexity, BLEU/ROUGE/METEOR, and RAG-specific metrics, and closes in M9 where offline regression against prior versions and drift detection extend the same discipline into production.

NCPG-T3 · 15 lessons across 3 modules

  1. M3M3-01Cleaning and curating a dataset: dedup, imbalance, and NeMo Curator
  2. M3M3-02Organizing and formatting datasets for pretraining, fine-tuning, and RAG
  3. M3M3-03Subword tokenization: BPE vs. WordPiece
  4. M3M3-04The vocabulary-size tradeoff and why it caps perplexity comparisons
  5. M3M3-05Exploratory data analysis before fine-tuning: the five-step checklist
  6. M6M6-01Perplexity: what it measures and where it does not apply
  7. M6M6-02BLEU vs. ROUGE vs. METEOR
  8. M6M6-03LLM-as-a-judge, human-in-the-loop review, and error analysis
  9. M6M6-04Benchmarking and scalable evaluation frameworks
  10. M6M6-05RAG evaluation: faithfulness, answer relevancy, context precision, and context recall
  11. M9M9-01Reliability and performance metrics: latency percentiles, throughput, error rate
  12. M9M9-02Logging, anomalies, and root-cause analysis
  13. M9M9-03Drift: data drift and concept drift
  14. M9M9-04Offline eval-set regression: benchmarking a deployment against prior versions
  15. M9M9-05Automated model retraining, tuning, and versioning

Part of the throughlines running across the NCP-GENL prep course.