LLM Fine-Tuning, Prompt Engineering & Model Evaluation

Please contact the course administrator to sign up for this course.
1/5 – (1 vote)

LLM Fine-Tuning, Prompt Engineering & Model Evaluation - DataLens Africa

đź”° Course Introduction

Large Language Models (LLMs) such as Llama, Mistral, and GPT have revolutionized how AI is developed, deployed, and used across the world. However, for these models to work effectively in contextual scenarios where languages, cultures, regulations, and user behaviors are unique, they must be fine-tuned, evaluated, and optimized using high-quality, localized datasets.

This advanced-level course is designed for learners who want to move beyond annotation into the world of model training, prompt engineering, LLM customization, and evaluation. It bridges theory and practice, enabling participants to fine-tune real open-source LLMs, build instruction datasets, engineer prompts for domain-specific tasks, and conduct safety and performance evaluations.

By the end of this course, learners will be capable of contributing to the development of African-focused LLMs—such as models for banking, agriculture, insurance, government services, and multi-language chatbots. This is one of the most valuable and future-proof skill sets in AI today.

Difficulty Level: Advanced Level — Designed for AI practitioners

Course Goal: Train learners to prepare data, fine-tune LLMs, design effective prompts, evaluate model performance, and apply safety frameworks—optimized for African contexts.

Target Audience: Senior data professionals, ML Engineers, project managers, and lead QA trainers.

Prerequisites:

  • Intermediate Python programming
  • Basic understanding of machine learning
  • Course 1 & Course 2 recommended but not required

Learning Outcomes:

  • Prepare and structure datasets for LLM fine-tuning.
  • Design effective prompts for different NLP tasks.
  • Fine-tune and deploy open-source LLMs using modern techniques.
  • Evaluate LLM performance, safety, and RAG-based workflows.

đź“… Hands-On Labs Webinar

To transform your theoretical knowledge into practical, job-ready skills, this course includes a mandatory 120mins instructor-led live webinar lab. This interactive session is designed to simulate real-world annotation tasks under the guidance of an expert instructor. You will use professional annotation tools, work with authentic datasets, and receive immediate feedback.

The Hands-On Lab webinar is a paid feature of this course. To gain access, you must register for the next cohort of our Hands-On Lab webinar for this course.

General Instructions for the Hands-On Lab Webinar:

  • Platform: Labs will be conducted via Zoom. Once registration is complete and payment is confirmed, the webinar details including the meeting link / invitation will be sent via email.
  • Preparation: Before the date of the Hands-On Lab webinar, ensure you have completed the all module lessons and quizzes in the course.
  • Participation: These are interactive sessions. You are encouraged to ask questions and share your screen (if comfortable) for troubleshooting.
  • Recording: Sessions will be recorded and made available for 48 hours for registered participants who cannot attend live.

Sign Up for the Live Webinar Series Here:

Click to Register for the LLM Fine-Tuning, Prompt Engineering and Model Evaluation Lab.

🥇🏆 Completion Certificate

Upon successful completion of both the online course content and the Hands-On Lab webinar, you will be issued a Certificate of Completion.

Certificate Requirements:
  • Complete all course modules and knowledge check quizzes.
  • Attend and actively participate in the mandatory Hands-On Lab webinar.
  • Submit lab exercises as required by the instructor.
Certificate Issuance:
  • Certificates will be issued digitally within 7 business days after lab completion.
  • You will receive your certificate via the email registered on your course account.
  • The certificate can be verified online and shared on professional platforms like LinkedIn.

Need Help?

  • Technical Issues?
    If you encounter any technical difficulties during while taking this course or during registration for the hands-on lab, please submit a support ticket here. Include the course name, screenshots, and detailed description of the issue.

Module 1 – Foundations of Large Language Models

This module covers the core architecture of LLMs, explains the differences between pre-training, fine-tuning, and instruction tuning, and provides a hands-on lab for setting up a development environment.

Lessons

Module 1 – Overview Preview Transformer Architecture Overview Preview Pre-training vs Fine-Tuning vs Instruction Tuning Preview Tokenization and Embeddings Preview Model Families: GPT, Llama, Claude, etc. Preview Compute Requirements and Infrastructure Preview Module 1 – Knowledge Check Quizzes Preview

Module 2 – Prompt Engineering Mastery

Learners will explore various prompt structures, optimization techniques, and ethical considerations, applying them in hands-on labs designed for tasks like summarization and reasoning.

Lessons

Module 2 – Overview Prompt Structures and Templates Zero-Shot, Few-Shot, and Chain-of-Thought Prompting System Prompts and Role Assignment Prompt Optimization Techniques Context Windows and Memory Safety, Bias and Ethical Prompting (Prompt injection defense) Hands-on Lab: Design prompts for summarization, extraction, reasoning Module 2 – Knowledge Check Quizzes

Module 3 – Preparing Data for Fine-Tuning

This section focuses on the strategies for collecting, curating, and preprocessing training data, with a special emphasis on handling low-resource languages and a hands-on lab using African language datasets.

Lessons

Module 3 – Overview Data Collection and Curation Strategies Formatting Training Data (JSONL, CSV, etc.) Data Cleaning and Preprocessing Handling Low-Resource Languages Hands-on Lab: Preparing African Language Dataset Module 3 – Knowledge Check Quizzes

Module 4 – Supervised Fine-Tuning (SFT)

Students will learn when and how to fine-tune models, explore training pipelines like LoRA, configure hyperparameters, and complete a hands-on lab fine-tuning an open-source model.

Lessons

Module 4 – Overview When to Fine-Tune vs Not Fine-Tune Training Pipelines (LoRA, QLoRA) Hyperparameter Configuration Training with Open-Source Models (Llama, Mistral, Gemma) Deployment Considerations Hands-on Lab: Fine-tune a small LLaMA-based model using LoRA Module 4 – Knowledge Check Quizzes

Module 5 – Retrieval-Augmented Generation (RAG)

This module compares RAG with fine-tuning, introduces vector databases and chunking strategies, and guides learners through building a simple RAG chatbot in a hands-on lab.

Lessons

Module 5 – Overview What's RAG? RAG vs Fine-Tuning Vector Databases (FAISS, Milvus, Pinecone, Weaviate) Chunking Strategies and Embeddings Agentic RAG Systems: From Retrieval to Reasoning Hands-on Lab: Build a simple RAG chatbot using open-source tools Module 5 – Knowledge Check Quizzes

Module 6 – Model Evaluation Techniques

Focusing on both automated metrics and human evaluation, this module teaches learners to assess model performance, safety, and bias through practical evaluation labs.

Lessons

Module 6 – Overview Evaluation Metrics: BLEU, ROUGE, BERTScore, G-Eval Human Evaluation Design and Frameworks LLM Safety Testing (toxicity, hallucination, bias & fairness assessment) LLM Benchmarks, Model Comparisons and Leaderboard Design Hands-on Lab: Evaluating Model Outputs Across Metrics Module 6 – Knowledge Check Quizzes