getting rid of dead commented text

contents/core/dnn_architectures/dnn_architectures.qmd

@@ -61,15 +61,6 @@ Dense pattern processing addresses this fundamental need by enabling several key
 
 For example, in the MNIST digit recognition task, while humans might focus on specific parts of digits (like loops in '6' or crossings in '8'), we cannot definitively say which pixel combinations are important for classification. A '7' written with a serif could share pixel patterns with a '2', while variations in handwriting mean discriminative features might appear anywhere in the image. This uncertainty about feature relationships necessitates a dense processing approach where every pixel can potentially influence the classification decision.
 
-<!-- The need for processing arbitrary relationships, however, comes with significant computational implications. When every output potentially depends on every input, the system must:
-
-* Access all input values for each computation
-* Store weights for all possible connections
-* Compute across all these connections
-* Move data between all elements of the network
-
-These requirements directly influence how we structure both algorithms and computer systems to handle dense pattern processing efficiently. -->
-
 ### Algorithmic Structure
 
 To enable unrestricted feature interactions, MLPs implement a direct algorithmic solution: connect everything to everything. This is realized through a series of fully-connected layers, where each neuron connects to every neuron in adjacent layers. The dense connectivity pattern translates mathematically into matrix multiplication operations. As shown in @fig-mlp, each layer transforms its input through matrix multiplication followed by element-wise activation:
@@ -133,15 +124,6 @@ This translation from mathematical abstraction to concrete computation exposes h
 
 In the MNIST example, each output neuron requires 784 multiply-accumulate operations and at least 1,568 memory accesses (784 for inputs, 784 for weights). While actual implementations use sophisticated optimizations through libraries like [BLAS](https://www.netlib.org/blas/) or [cuBLAS](https://developer.nvidia.com/cublas), these fundamental patterns drive key system design decisions.
 
-<!-- The computational mapping reveals several critical patterns that influence system design:
-
-1. Each output depends on every input, creating an all-to-all communication pattern
-2. Memory access is extensive and regular, with complete rows and columns being accessed
-3. The basic operation (multiply-accumulate) repeats many times with different data
-4. Computation can be parallelized across batches and output neurons
-
-These patterns create both challenges in implementation and opportunities for optimization, which we'll examine in the next [section](#system-implications) -->
-
 ### System Implications
 
 When analyzing how computational patterns impact computer systems, we typically examine three fundamental dimensions: memory requirements, computation needs, and data movement. This framework enables a systematic analysis of how algorithmic patterns influence system design decisions. We will use this framework for analyzing other network architectures, allowing us to compare and contrast their different characteristics.
@@ -195,8 +177,6 @@ Taking image processing as an example, if we want to detect a cat in an image, c
 
 This leads us to the convolutional neural network architecture (CNN), introduced by @lecun1989backpropagation. CNNs address spatial pattern processing through a fundamentally different connection pattern than MLPs. Instead of connecting every input to every output, CNNs use a local connection pattern where each output connects only to a small, spatially contiguous region of the input. This local receptive field moves across the input space, applying the same set of weights at each position—a process known as convolution.
 
-<!-- These requirements create specific demands on our processing architecture. The system needs to support local connectivity to detect spatial patterns while enabling parameter sharing to recognize patterns independent of position. It must facilitate hierarchical processing to combine simple patterns into complex features, and efficiently handle shifting patterns across the input space. Unlike the dense connectivity of MLPs, spatial pattern processing suggests an architecture that explicitly encodes these spatial relationships while maintaining computational efficiency. This leads us to the convolutional neural network architecture, which we'll examine next. -->
-
 ### Algorithmic Structure
 
 The core operation in a CNN can be expressed mathematically as:
@@ -407,12 +387,6 @@ For our example with a 128-dimensional hidden state, each time step must: load t
 
 Different architectures handle this sequential data movement through specialized mechanisms. CPUs maintain weight matrices in cache while streaming through sequence elements and managing hidden state updates. GPUs employ memory architectures optimized for maintaining state information across sequential operations while processing multiple sequences in parallel. Deep learning frameworks orchestrate these movements by managing data transfers between time steps and optimizing batch operations.
 
-<!-- ### Summary and Next Steps
-
-The analysis of RNNs demonstrates how sequential pattern processing creates fundamentally different computational patterns from both the dense connectivity of MLPs and the spatial operations of CNNs. While MLPs process all inputs simultaneously and CNNs reuse weights across spatial positions, RNNs must handle temporal dependencies that create inherent sequential processing requirements. This sequential nature manifests in distinct system demands: memory systems must manage both weight reuse across time steps and hidden state updates, computation must balance sequential dependencies with parallel execution, and data movement centers around maintaining and updating state information efficiently.
-
-These characteristics illustrate why different optimization strategies have evolved for RNN processing, and why certain applications began shifting toward alternative architectures like attention mechanisms, which we'll examine next. As we explore these newer architectural patterns, we'll see how they address some of the fundamental challenges of sequential processing while creating their own unique demands on computer systems. -->
-
 ## Attention Mechanisms: Dynamic Pattern Processing
 
 While previous architectures process patterns in fixed ways—MLPs with dense connectivity, CNNs with spatial operations, and RNNs with sequential updates—many tasks require dynamic relationships between elements that change based on content. Language understanding, for instance, needs to capture relationships between words that depend on meaning rather than just position. Graph analysis requires understanding connections that vary by node. These dynamic relationships suggest we need an architecture that can learn and adapt its processing patterns based on the data itself.
@@ -567,16 +541,6 @@ Finally, self-attention generates memory-intensive intermediate results. The att
 
 These computational patterns create a unique profile for Transformer self-attention, distinct from previous architectures. The parallel nature of the computations makes Transformers well-suited for modern parallel processing hardware, but the quadratic complexity with sequence length poses challenges for processing long sequences. As a result, much research has focused on developing optimization techniques, such as sparse attention patterns or low-rank approximations, to address these challenges. Each of these optimizations presents its own trade-offs between computational efficiency and model expressiveness, a balance that must be carefully considered in practical applications.
 
-<!-- ### Summary
-
-Attention mechanisms and Transformers have ushered in a paradigm shift in neural network information processing. Unlike the fixed patterns of MLPs, CNNs, or RNNs, these architectures introduce dynamic, content-dependent computation, bringing both unprecedented capabilities and unique system challenges. The basic attention mechanism laid the groundwork for content-based weighting of information, allowing models to dynamically focus on relevant parts of the input. Transformers then extended this concept with self-attention, enabling each element in a sequence to interact with every other element, capturing complex dependencies regardless of positional distance.
-
-This dynamic pattern processing manifests in distinctive system demands. Memory systems must contend with the quadratic scaling of attention weights with sequence length, a challenge that becomes particularly acute for longer sequences. Computation needs center around intensive matrix multiplications for query-key interactions and value combining, operations that benefit from parallelization but scale quadratically with sequence length. Data movement patterns revolve around the frequent access and update of dynamically generated weights and intermediate results, creating unique bandwidth requirements.
-
-These characteristics elucidate both the strengths and challenges of Transformer architectures. Their ability to capture dynamic, long-range relationships has enabled breakthrough performance across a wide range of tasks, from natural language processing to computer vision and beyond. However, their computational intensity necessitates specialized hardware and optimized implementations to manage their resource demands effectively. This trade-off between expressive power and memory efficiency is a key consideration when choosing architectures for different tasks. While Transformers excel at capturing complex dependencies, their memory demands necessitate careful system design and optimization, especially for resource-constrained environments.
-
-The advent of attention mechanisms and Transformers has opened new frontiers in machine learning, challenging our previous notions of architectural design and efficiency. As we continue to push the boundaries of what's possible with these models, a deep understanding of their computational patterns and system implications will be important in guiding future innovations in both model architecture and hardware design. -->
-
 ## Architectural Building Blocks
 
 Deep learning architectures, while we presented them as distinct approaches in the previous sections, are better understood as compositions of fundamental building blocks that evolved over time. Much like how complex LEGO structures are built from basic bricks, modern neural networks combine and iterate on core computational patterns that emerged through decades of research [@lecun2015deep]. Each architectural innovation introduced new building blocks while finding novel ways to use existing ones.

contents/core/generative_ai/generative_ai.qmd

@@ -11,12 +11,3 @@ Imagine a chapter that writes itself and adapts to your curiosity, generating ne
 This chapter will transform how you read and learn, dynamically generating content as you go. While we fine-tune this exciting new feature, we hope users get ready for an educational experience that's as dynamic and unique as you are. Mark your calendars for the big reveal and bookmark this page.
 
 _The future of **generative learning** is here! — Vijay Janapa Reddi_
-
-<!--
-::: {.callout-tip}
-
-## Learning Objectives
-
-* *Coming soon.*
-:::
--->