Machine Learning Engineer and Software Architect with 15+ years of experience, specializing in deep learning, NLP, and LLM-based systems. Strong focus on designing and deploying end-to-end AI pipelines, including model fine-tuning (LoRA/PEFT), retrieval-augmented generation (RAG), and high-performance inference with modern frameworks such as PyTorch, Hugging Face Transformers, and vLLM. Experienced in building agentic systems using LangChain and LangGraph, as well as developing scalable backend services in Python. Combines solid software engineering principles with hands-on expertise in data-centric AI to deliver robust, production-ready solutions.
This hands-on workshop guides participants through the full process of building a GPT-style large language model in PyTorch, starting from core tensor operations and ending with pretraining, text generation, and fine-tuning for practical tasks. The workshop is designed for beginners and uses clear, step-by-step instruction, guided labs, and starter code throughout.
Participants will learn how raw text becomes model-ready data, how embeddings and attention work, how transformer blocks are assembled into a GPT architecture, and how to train and adapt the model for real use cases such as classification and instruction following.
The teaching approach emphasizes:
Foundational concepts first
Hands-on coding in PyTorch
Guided exercises with starter code
Simple explanations of core LLM building blocks
Laptop-friendly examples and workflows
Learning Objectives
By the end of this workshop, participants will be able to:
Explain, in simple terms, what a large language model is and how GPT-style models are built
Use core PyTorch concepts including tensors, modules, parameters, and training loops
Convert raw text into tokenized, padded, and batched training examples
Describe and implement tokenization approaches, including Byte Pair Encoding concepts used in GPT-style tokenizers
Create next-token prediction datasets using sliding windows over token sequences
Implement token embeddings and positional embeddings in PyTorch
Explain the intuition behind attention and why it improves sequence modeling
Code self-attention, causal masking, and multi-head attention step by step
Build transformer blocks and assemble a small GPT-style language model from scratch
Generate text from model outputs using decoding loops
Train a small language model on unlabeled text data and track loss during pretraining
Improve text generation with practical decoding methods and load pretrained weights
Agenda
Day 1 - PyTorch Foundations and Text Preparation
Topics
Welcome, workshop setup, and roadmap for building an LLM from scratch
What large language models are and how GPT-style systems work at a high level
Core PyTorch concepts:
tensors
shapes
basic tensor operations
modules and parameters
forward passes
From raw text to model-ready data
Simple tokenization and vocabulary building
Token IDs, padding, batching, and sequence formatting
Byte Pair Encoding intuition and GPT-style tokenization
Creating next-token prediction examples with sliding windows
Token embeddings and positional embeddings
Activities
Guided PyTorch warm-up exercises
Walkthrough: converting text into token sequences step by step
Lab: build a simple tokenizer and inspect tokenized outputs
Lab: create input-target pairs for next-token prediction
Lab: implement embedding layers and visualize tensor shapes
Day 1 Outcome
Participants understand the end-to-end data pipeline that turns text into vectors suitable for a GPT-style model.
Day 2 - Attention and GPT-Style Sequence Modeling
Topics
Why older sequence models struggle with long-range dependencies
Attention intuition: letting each token focus on relevant context
Self-attention concepts using small numerical examples
Coding self-attention manually in PyTorch
Queries, keys, values, and attention scores
Causal masking for next-token prediction
Multi-head attention and why multiple heads help
Activities
Instructor-led visual explanation of attention flow
Starting from
€2,990.00
Topics
Customizable to your team's needs
Expert trainers with industry experience
Hands-on exercises and real-world projects
Interested in this workshop?
Get in touch to discuss dates, customization options, and group pricing for your team.
Fine-tune a GPT model on instruction-response data and evaluate generated outputs
Target Audience
This workshop is intended for:
Developers who are new to large language models
Data and ML beginners who want a practical introduction to GPT-style architectures
Python users who want to understand how modern text generation models work internally
Educators, technical professionals, and students seeking a hands-on introduction to PyTorch-based LLM development
Prerequisites
Participants should have:
Basic Python programming experience
Comfort running Python scripts or notebooks
A general understanding of variables, functions, loops, and lists
Interest in machine learning concepts, but no prior LLM or deep learning experience is required
Helpful but optional:
Familiarity with Jupyter notebooks
Basic command-line usage
Introductory exposure to linear algebra concepts such as vectors and matrices
Workshop Format
The workshop combines:
Short instructor-led concept sessions
Live coding demonstrations
Guided labs with starter code
Structured checkpoints and troubleshooting help
Incremental model building across the week
Participants leave with a working end-to-end project and a clear mental model of how GPT-style language models are built, trained, and adapted in PyTorch.
Live coding: self-attention from scratch
Lab: compute attention scores and weights by hand and in PyTorch
Lab: add causal masking to prevent access to future tokens
Lab: refactor into reusable PyTorch modules
ASCII Concept Sketch
Token t_i
|
+--> Query
+--> Key
+--> Value
Attention(t_i) = weighted sum of Values from allowed tokens
Day 2 Outcome
Participants can explain and implement self-attention, causal attention, and multi-head attention in PyTorch.
Day 3 - Building Transformer Blocks
Topics
The overall GPT architecture
Transformer blocks as the core reusable component
Layer composition:
embeddings
attention
feed-forward network
normalization
residual connections
Implementing feed-forward layers in PyTorch
Understanding residual paths and layer normalization
Inspecting tensor shapes and outputs at each stage
Activities
Live coding: assemble a transformer block piece by piece
Lab: test each block independently with starter code
Lab: trace activations through a transformer block
Debugging session: inspect tensor shapes and intermediate outputs
Day 3 Outcome
Participants can build and validate the main components of a transformer block in PyTorch.
Day 4 - Assembling a GPT Model
Topics
Connecting embeddings, transformer blocks, and output heads
Building a small GPT model from scratch in PyTorch
Configuring vocabulary size, context length, embedding size, and depth
Forward propagation through the full model
Producing next-token logits from hidden states
Comparing outputs from different model sizes and settings
Activities
Live coding: connect blocks into a working GPT model
Lab: run sample inputs through the full architecture
Lab: verify model outputs and tensor dimensions
Lab: experiment with small architectural configurations
ASCII Architecture Sketch
Input Tokens
|
[Token Embeddings + Positional Embeddings]
|
[Transformer Block] x N
|
Linear Head
|
Next-Token Logits
Day 4 Outcome
Participants build and run a complete GPT-style model that produces next-token predictions.
Day 5 - Text Generation Basics
Topics
Autoregressive generation and decoding loops
Logits, probabilities, and token selection
Greedy decoding and temperature scaling
Top-k sampling and simple sampling strategies
Comparing generation from untrained and partially trained models
Practical debugging of repetitive or unstable generations
Activities
Live coding: implement autoregressive text generation
Lab: generate text with different decoding settings
Lab: compare outputs from untrained and partially trained models
Debugging session: inspect logits, predictions, and generated text
Day 5 Outcome
Participants can generate text from a GPT-style model and understand how decoding choices affect output.
Day 6 - Pretraining Objectives and Training Loops
Topics
Language modeling objective and next-token prediction loss
Pretraining on unlabeled text data
Training loops in PyTorch:
forward pass
loss computation
backward pass
optimizer step
Gradient flow and parameter updates
Tracking progress with training and validation loss
Activities
Live coding: implement a simple pretraining loop
Lab: compute loss for next-token prediction
Lab: step through backpropagation and optimizer updates
Lab: record and visualize training and validation loss
Day 6 Outcome
Participants can implement the core training loop needed to pretrain a small language model.
Day 7 - Running Small-Scale Pretraining Experiments
Topics
Running laptop-friendly pretraining experiments
Choosing batch size, sequence length, and learning rate
Monitoring overfitting and underfitting
Inspecting generated samples during training
Saving checkpoints and resuming training
Comparing from-scratch models with pretrained checkpoints
Activities
Lab: train a small model on a sample corpus
Lab: evaluate loss and inspect generated samples during training
Lab: save and reload checkpoints
Lab: compare a freshly initialized model with a pretrained model
Day 7 Outcome
Participants can run, monitor, and manage practical small-scale pretraining experiments in PyTorch.
Day 8 - Using Pretrained Weights and Practical Inference
Topics
Loading and using pretrained model weights
Preparing a model for inference
Efficient text generation workflows
Prompting patterns for better outputs
Trying different decoding options for higher-quality text
Practical tradeoffs between custom training and pretrained reuse
Activities
Lab: load pretrained weights and run inference
Lab: compare outputs under different prompting styles
Lab: try different decoding options for better text output
Lab: benchmark simple inference settings
Day 8 Outcome
Participants can use pretrained GPT-style models effectively for practical text generation tasks.
Day 9 - Fine-Tuning GPT for Classification
Topics
Fine-tuning versus pretraining
Adapting GPT for text classification
Guided example: spam classification with a pretrained GPT model
Preparing labeled datasets for supervised fine-tuning
Training loops for classification tasks
Evaluating accuracy, errors, and prediction confidence
Activities
Lab: fine-tune GPT for classification with starter code
Lab: evaluate accuracy and inspect prediction errors
Lab: compare classification heads and output strategies
Debugging session: analyze common failure cases
Day 9 Outcome
Participants gain hands-on experience adapting GPT models for classification tasks using PyTorch.
Day 10 - Instruction Fine-Tuning, Evaluation, and Wrap-Up
Topics
Batching and formatting instruction-response data
Fine-tuning GPT to follow instructions
Evaluating outputs for generation tasks
Comparing base model behavior with fine-tuned behavior
Practical discussion: limitations, responsible use, and next steps for continued learning
Final review of the full LLM pipeline from data preparation to deployment-ready workflows
Activities
Lab: prepare an instruction dataset for tuning
Lab: fine-tune GPT on instruction-following examples
Final exercise: compare base model behavior with fine-tuned behavior
Wrap-up and Q&A
Day 10 Outcome
Participants leave with hands-on experience fine-tuning GPT models for instruction-following and a clear roadmap for continued learning with PyTorch.
