LLMs Under the Hood - Applied Engineering Workshop
Oct 28, 2026, 9:00 AM · 8 hr 5 min · Room B
Session description
Large Language Models are powerful tools, but using them effectively requires more than prompt crafting. This workshop builds practical intuition for how modern AI systems work by examining the mechanics behind tokenization, embeddings, vector search, transformer architectures, and retrieval‑augmented generation. Through hands‑on exercises, participants learn how to evaluate use‑cases, reduce risk, and design applications that integrate AI responsibly and reliably. No prior AI experience is required; familiarity with software development concepts is helpful.
This full‑day workshop provides a practical, engineering‑focused understanding of how modern language models operate and how to apply them effectively in real systems. Participants explore the full lifecycle of text processing, from tokenization to embeddings to transformer inference, and learn how these components shape model behavior.
The workshop emphasizes applied techniques: building and comparing embeddings, performing semantic search, integrating vector databases, designing retrieval‑augmented generation pipelines, and evaluating model output for reliability and safety. Attendees work through hands‑on exercises using local or cloud‑based tools to reinforce concepts and build confidence in applying AI to real‑world problems.
By the end of the day, participants will understand when and why to use LLMs, how to structure data for them, how to reduce hallucinations, and how to design systems that are transparent, ethical, and maintainable.
What you'll cover
Module 1 – Foundations of Modern NLP (60–75 min)
Evolution of text models (word2vec → RankBrain → Transformers)
Keyword search vs vector search
Dense vs sparse representations
Tokenization (cl100K, Unicode, sequences)
Embedding math: cosine similarity, distance, clustering
Dimensionality reduction examples (T‑SNE projections)
Why embeddings are the foundation of all modern LLM behavior
Hands‑On: Generating Embeddings Locally
Module 2 – Transformer Architecture in Practice (60 min)
Self‑attention, Q/K/V matrices
Multi‑head attention
Positional encoding
Transformer blocks and parallelism
Why transformers generalize well
Practical limitations (context windows, O(n²) attention)
Hands‑On: Visualizing Attention
Module 3 – Embeddings for Real Applications (60–75 min)
Semantic similarity and clustering
Embedding‑based reasoning
Bias, fairness, and normalization
Resume‑scanning example from the deck
Privacy and reversibility considerations
Hands‑On: Exploring similarity in‑memory
Module 4 – Vector Search and Knowledge Architectures (60-75 min)
Vector databases (Azure AI Search, Redis, Qdrant, Pinecone)
KD‑trees and nearest‑neighbor search
Knowledge graphs and hybrid retrieval
When vector search outperforms attention and when it doesn’t
Hands‑On: Implementing Data Clustering
Module 5 – Retrieval‑Augmented Generation (60–75 min)
Dense RAG
GraphRAG
MCP‑based RAG integration
Example architecture from the deck
Reducing hallucinations
Designing prompts for contextual relevance
Hands‑On: Implementing RAG
Module 6 – Responsible AI and System Design (30–45 min)
Copyright, carbon footprint, and model creation concerns
Bias, fairness, and transparency
When not to use AI
Designing safe, predictable user experiences
Hands‑On: Implementing safety patterns
Requirements
Laptop with LM Studio and Postman installed
An IDE for the language of your choice
A containerization environment (Docker or Podman)
The Qdrant container downloaded
Takeaways
Understand how tokenization, embeddings, and transformer architectures influence model behavior
Apply embeddings for semantic search, clustering, and context retrieval
Build and evaluate retrieval‑augmented generation systems
Identify risks, limitations, and ethical considerations when integrating AI into applications