Deeply Learning

Notes on Deep Learning Systems and AI Research

Visualizing Attention: See what an LLM sees.

Learn how attention mechanisms work in transformers by visualizing what LLMs see when processing text. Discover how attention connects semantically related tokens (like Paris → French), understand the Query-Key-Value framework, and explore how different attention heads specialize in syntax, semantics, and coreference.

26 min read · December 19, 2025

2025 · Attention Transformers Large Language Models LLMs Natural Language Processing NLP · Natural Language Processing NLP Large Language Models LLMs Transformers
Supervised Finetuning in LLM training workflow

Learn how supervised fine-tuning (SFT) fits into the LLM training pipeline. This post explains the three-step process (pretraining → SFT → alignment), demonstrates SFT implementation with a practical example, and shows how fine-tuning transforms a base model into a task-specific assistant.

38 min read · December 18, 2025

2025 · Supervised Fine-tuning SFT LLM Large Language Models LLMs Transformers Natural Language Processing NLP · Natural Language Processing NLP Large Language Models LLMs Transformers
From Words to Meaning: Implementing Word2Vec from Scratch

Word embeddings are one of the most transformative developments in Natural Language Processing (NLP). They solve a fundamental problem: how can we rep...

168 min read · December 17, 2025

2025 · Word Embeddings Word2Vec Embeddings Embedding Models Natural Language Processing NLP Large Language Models Deep Learning Neural Networks · Natural Language Processing
Primer on Large Language Model (LLM) Inference Optimizations: 3. Model Architecture Optimizations

Exploring model architecture optimizations for Large Language Model (LLM) inference, focusing on Group Query Attention (GQA) and Mixture of Experts (MoE) techniques.

11 min read · November 15, 2024

2024 · LLM Inference Optimization Transformer Attention Mechanism Multi-Head Attention K-V Caching Memory Calculation Optimization Metrics Optimization Techniques Mixture of Experts Group Query Attention GQA MoE AI Accelerators Hardware Acceleration Model Architecture Optimizations Natural Language Processing NLP Large Language Models LLMs Transformers · Large Language Model Inference Optimization Natural Language Processing
Scaling Laws in Large Language Models

Scaling laws in AI offer a quantitative framework for understanding the relationship between model size, data, and compute resources. Learn about the Chinchilla scaling law, power laws, and the future of large models.

10 min read · November 07, 2024

2024 · LLM Scaling Laws Emergent Capabilities Transformers Natural Language Processing NLP Large Language Models LLMs Transformers · Large Language Model Emegent Capabilities Natural Language Processing
Primer on Large Language Model (LLM) Inference Optimizations: 2. Introduction to Artificial Intelligence (AI) Accelerators

Exploration of AI accelerators and their impact on deploying Large Language Models (LLMs) at scale.

12 min read · November 06, 2024

2024 · LLM Inference Optimization Transformer Attention Mechanism Multi-Head Attention AI Accelerators GPUs TPUs FPGAs ASICs Parallel Processing Data Parallelism Model Parallelism Task Parallelism Co-Processing Mode Intelligent Processing Units Reconfigurable Dataflow Units Neural Processing Units Large Language Models LLMs Transformers Natural Language Processing NLP · Large Language Model Inference Optimization AI Accelerators Natural Language Processing
Primer on Large Language Model (LLM) Inference Optimizations: 1. Background and Problem Formulation

Overview of Large Language Model (LLM) inference, its importance, challenges, and key problem formulations.

15 min read · October 31, 2024

2024 · LLM Inference Optimization Transformer Attention Mechanism Multi-Head Attention K-V Caching Memory Calculation Optimization Metrics Optimization Techniques Natural Language Processing NLP Large Language Models LLMs Transformers · Large Language Model Inference Optimization Natural Language Processing