Adding in Colabs and fixing learning objectives

contents/core/dl_architectures/dl_architectures.qmd

@@ -134,6 +134,23 @@ The implementation of MLPs presents several key challenges and opportunities tha
 
 These challenges and opportunities drive the development of specialized neural processing engines fo machine learning systems. While memory bandwidth limitations push designs toward sophisticated memory hierarchies (needing to handle >50,000 memory operations efficiently), the regular patterns and parallel opportunities enable efficient implementations through specialized processing units. The patterns established by MLPs form a baseline that more specialized neural architectures must consider in their implementations.
 
+:::{.callout-caution #exr-mlp collapse="false"}
+
+##### Multilayer Perceptrons (MLPs)
+
+We've just scratched the surface of neural networks. Now, you'll get to try and apply these concepts in practical examples. In the provided Colab notebooks, you'll explore:
+
+**Predicting house prices:** Learn how neural networks can analyze housing data to estimate property values.
+[![](https://colab.research.google.com/assets/colab-badge.png)](https://colab.research.google.com/github/Mjrovai/UNIFEI-IESTI01-TinyML-2022.1/blob/main/00_Curse_Folder/1_Fundamentals/Class_07/TF_Boston_Housing_Regression.ipynb)
+
+**Image Classification:** Discover how to build a network to understand the famous MNIST handwritten digit dataset.
+[![](https://colab.research.google.com/assets/colab-badge.png)](https://colab.research.google.com/github/Mjrovai/UNIFEI-IESTI01-TinyML-2022.1/blob/main/00_Curse_Folder/1_Fundamentals/Class_09/TF_MNIST_Classification_v2.ipynb)
+
+**Real-world medical diagnosis:** Use deep learning to tackle the important task of breast cancer classification.
+[![](https://colab.research.google.com/assets/colab-badge.png)](https://colab.research.google.com/github/Mjrovai/UNIFEI-IESTI01-TinyML-2022.1/blob/main/00_Curse_Folder/1_Fundamentals/Class_13/docs/WDBC_Project/Breast_Cancer_Classification.ipynb)
+
+:::
+
 ### Convolutional Neural Networks: Spatial Pattern Processing
 
 Convolutional Neural Networks (CNNs) represent a specialized neural architecture designed to efficiently process data with spatial relationships, such as images. While MLPs treat each input independently, CNNs exploit local patterns and spatial hierarchies, establishing computational patterns that have revolutionized computer vision and spatial data processing.

contents/core/nn_primer/nn_primer.qmd

@@ -16,8 +16,6 @@ This chapter focuses on neural networks, the foundational building blocks of mod
 
 Our exploration begins with the biological inspiration behind artificial neurons and extends to the practical design and operation of neural networks. We examine the structure and topology of these networks, the flow of data through their layers, and the mechanisms by which they learn from data. These fundamental concepts provide the necessary context for understanding the broader field of deep learning, which we will explore in subsequent chapters.
 
-By focusing on the fundamentals of neural networks, this chapter will start to bridge the gap between theoretical concepts and engineering ML systems. This focus allows us to build a strong foundation for examining modern architectures, computational patterns, and the system-level considerations that drive the development of advanced machine learning systems, which we will explore in future chapters.
-
 ::: {.callout-tip}
 
 ## Learning Objectives
@@ -28,11 +26,16 @@ By focusing on the fundamentals of neural networks, this chapter will start to b
 
 * Examine the processes of forward propagation, backward propagation, and optimization as the core mechanisms of learning.
 
+* Understand the complete machine learning pipeline, from pre-processing through neural computation to post-processing.
+
+* Compare and contrast training and inference phases, understanding their distinct computational requirements and optimizations.
+
 * Learn how neural networks process data to extract patterns and make predictions, bridging theoretical concepts with computational implementations.
 
-* Build a foundational understanding of neural networks as the essential building blocks for deep learning architectures, setting the stage for more advanced exploration in later chapters.  
 :::
 
+The updated objectives now include the prediction phase, processing pipeline, and real-world deployment aspects we covered in the chapter. Would you like me to refine any of these objectives further?
+
 ## Overview
 
 Neural networks, a foundational concept within machine learning and artificial intelligence, are computational models inspired by the structure and function of biological neural systems. These networks represent a critical intersection of algorithms, mathematical frameworks, and computing infrastructure, making them integral to solving complex problems in AI.
@@ -201,13 +204,13 @@ Each element of a biological neuron has a computational analog in artificial sys
 
 Each component serves a similar function, albeit through vastly different mechanisms. Here, we explain these mappings and their implications for artificial neural networks.
 
-1. **Cell ↔ Neuron/Node**: The artificial neuron or node serves as the fundamental computational unit, mirroring the cell's role in biological systems.
+1. **Cell ↔ Neuron/Node:** The artificial neuron or node serves as the fundamental computational unit, mirroring the cell's role in biological systems.
 
-2. **Dendrites/Synapse ↔ Weights**  Weights in artificial neurons represent connection strengths, analogous to synapses in biological neurons. These weights are adjustable, enabling learning and optimization over time.
+2. **Dendrites/Synapse ↔ Weights:**  Weights in artificial neurons represent connection strengths, analogous to synapses in biological neurons. These weights are adjustable, enabling learning and optimization over time.
 
-3. **Soma ↔ Net Input**  The net input in artificial neurons sums weighted inputs to determine activation, similar to how the soma integrates signals in biological neurons.
+3. **Soma ↔ Net Input:**  The net input in artificial neurons sums weighted inputs to determine activation, similar to how the soma integrates signals in biological neurons.
 
-4. **Axon ↔ Output** The output of an artificial neuron passes processed information to subsequent network layers, much like an axon transmits signals to other neurons.
+4. **Axon ↔ Output:** The output of an artificial neuron passes processed information to subsequent network layers, much like an axon transmits signals to other neurons.
 
 This mapping illustrates how artificial neural networks simplify and abstract biological processes while preserving their essential computational principles. However, understanding individual neurons is just the beginning—the true power of neural networks emerges from how these basic units work together in larger systems.
 
@@ -987,7 +990,7 @@ The structural difference between training loops and inference passes significan
 
 Memory and computation requirements differ substantially between training and inference. Training demands considerable memory to store intermediate activations for backpropagation, gradients for weight updates, and optimization states. Inference eliminates these memory-intensive requirements, needing only enough memory to store the model parameters and compute a single forward pass. This reduction in memory footprint, coupled with simpler computation patterns, enables inference to run efficiently on a broader range of devices, from powerful servers to resource-constrained edge devices.
 
-#tbl-train-vs-inference summarizes the key differences between training and inference:
+In general, the training phase requires more computational resources and memory for learning, while inference is streamlined for efficient prediction. @tbl-train-vs-inference summarizes the key differences between training and inference. 
 
 +-------------------------+------------------------------+--------------------------------+
 | Aspect                  | Training                     | Inference                      |
@@ -1011,19 +1014,19 @@ Memory and computation requirements differ substantially between training and in
 |                         | for efficient training       | including mobile/edge          |
 +-------------------------+------------------------------+--------------------------------+
 
-: Key differences between training and inference phases in neural networks. The training phase requires more computational resources and memory for learning, while inference is streamlined for efficient prediction. {#tbl-train-vs-inference}
+: Key differences between training and inference phases in neural networks. {#tbl-train-vs-inference}
 
 This stark contrast between training and inference phases highlights why system architectures often differ significantly between development and deployment environments. While training requires substantial computational resources and specialized hardware, inference can be optimized for efficiency and deployed across a broader range of devices.
 
 #### Basic Pipeline
 
-The implementation of neural networks in practical applications requires a complete processing pipeline that extends beyond the network itself. This pipeline transforms raw inputs into meaningful outputs through a series of distinct stages, each essential for the system's operation. Understanding this complete pipeline provides critical insights into the design and deployment of machine learning systems.
+The implementation of neural networks in practical applications requires a complete processing pipeline that extends beyond the network itself. This pipeline, which is illustrated in @fig-inference-pipeline. transforms raw inputs into meaningful outputs through a series of distinct stages, each essential for the system's operation. Understanding this complete pipeline provides critical insights into the design and deployment of machine learning systems.
 
-![End-to-end workflow for the inference prediction phase.](images/png/inference_pipeline.png)
+![End-to-end workflow for the inference prediction phase.](images/png/inference_pipeline.png){#fig-inference-pipeline}
 
-Machine learning systems operate as hybrid architectures that combine conventional computing operations with neural network computations. The neural network component, focused on learned transformations through matrix operations, represents just one element within a broader computational framework. This framework encompasses both the preparation of input data and the interpretation of network outputs, processes that rely primarily on traditional computing methods.
+The key thing to notice from the figure is that machine learning systems operate as hybrid architectures that combine conventional computing operations with neural network computations. The neural network component, focused on learned transformations through matrix operations, represents just one element within a broader computational framework. This framework encompasses both the preparation of input data and the interpretation of network outputs, processes that rely primarily on traditional computing methods.
 
-Consider how data flows through this pipeline:
+Consider how data flows through the pipeline in @fig-inference-pipeline:
 
 1. Raw inputs arrive in their original form, which might be images, text, sensor readings, or other data types
 2. Pre-processing transforms these inputs into a format suitable for neural network consumption
@@ -1037,9 +1040,9 @@ The hybrid nature of this architecture has significant implications for system i
 
 ### Pre-processing
 
-The pre-processing stage transforms raw inputs into a format suitable for neural network computation. While often overlooked in theoretical discussions, this stage forms a critical bridge between real-world data and neural network operations. Consider our MNIST digit recognition example: before a handwritten digit image can be processed by the neural network we designed earlier, it must undergo several transformations. Raw images of handwritten digits arrive in various formats, sizes, and pixel value ranges.
+The pre-processing stage transforms raw inputs into a format suitable for neural network computation. While often overlooked in theoretical discussions, this stage forms a critical bridge between real-world data and neural network operations. Consider our MNIST digit recognition example: before a handwritten digit image can be processed by the neural network we designed earlier, it must undergo several transformations. Raw images of handwritten digits arrive in various formats, sizes, and pixel value ranges. For instance, in @fig-handwritten, we see that the digits are all of different sizes, and even the number 6 is written differently by the same person. 
 
-![Images of handwritten digits. Source: O. Augereau](images/png/handwritten_digits.png)
+![Images of handwritten digits. Source: O. Augereau](images/png/handwritten_digits.png){#fig-handwritten}
 
 The pre-processing stage standardizes these inputs through conventional computing operations:
 
@@ -1229,7 +1232,13 @@ Perhaps most importantly, this implementation demonstrated how theoretical princ
 
 ### Takeaway
 
-The USPS ZIP code recognition system exemplifies the journey from biological inspiration to practical neural network deployment that we've explored throughout this chapter. It demonstrates how the basic principles of neural computation---from pre-processing through inference to post-processing---come together in solving real-world problems. The system's development shows why understanding both the theoretical foundations and practical considerations is crucial. While the biological visual system processes handwritten digits effortlessly, translating this capability into an artificial system required careful consideration of network architecture, training procedures, and system integration. The success of this early large-scale neural network deployment helped establish many practices we now consider standard: the importance of comprehensive training data, the need for confidence metrics, the role of pre- and post-processing, and the critical nature of system-level optimization. As we move forward to explore more complex architectures and applications in subsequent chapters, this case study reminds us that successful deployment requires mastery of both fundamental principles and practical engineering considerations.
+The USPS ZIP code recognition system is an excellent example of the journey from biological inspiration to practical neural network deployment that we've explored throughout this chapter. It demonstrates how the basic principles of neural computation---from pre-processing through inference to post-processing---come together in solving real-world problems. 
+
+The system's development shows why understanding both the theoretical foundations and practical considerations is crucial. While the biological visual system processes handwritten digits effortlessly, translating this capability into an artificial system required careful consideration of network architecture, training procedures, and system integration. 
+
+The success of this early large-scale neural network deployment helped establish many practices we now consider standard: the importance of comprehensive training data, the need for confidence metrics, the role of pre- and post-processing, and the critical nature of system-level optimization. 
+
+As we move forward to explore more complex architectures and applications in subsequent chapters, this case study reminds us that successful deployment requires mastery of both fundamental principles and practical engineering considerations.
 
 ## Conclusion