diff --git a/contents/core/dl_architectures/dl_architectures.qmd b/contents/core/dl_architectures/dl_architectures.qmd index 8a8a2676..16276936 100644 --- a/contents/core/dl_architectures/dl_architectures.qmd +++ b/contents/core/dl_architectures/dl_architectures.qmd @@ -10,6 +10,15 @@ Resources: [Slides](#sec-deep-learning-primer-resource), [Videos](#sec-deep-lear ![_DALLĀ·E 3 Prompt: A visually striking rectangular image illustrating the interplay between deep learning algorithms like CNNs, RNNs, and Attention Networks, interconnected with machine learning systems. The composition features neural network diagrams blending seamlessly with representations of computational systems such as processors, graphs, and data streams. Bright neon tones contrast against a dark futuristic background, symbolizing cutting-edge technology and intricate system complexity._](images/png/cover_dl_arch.png) +## Purpose {.unnumbered} + +_What are the modern deep learning architectures, what fundamental computational patterns underlie them, and how can these patterns be leveraged to build adaptable and efficient AI systems?_ + +This chapter focuses on real-world deep neural network (DNN) architectures, including convolutional networks (CNNs), recurrent networks (RNNs), and transformers, with a unique focus on the foundational computational patterns they share. Despite rapid advancements in AI, the core building blocks---or "LEGO pieces"---have remained relatively stable. Understanding these components is essential for constructing AI systems that support diverse model architectures while enabling efficient and scalable deployments. By bridging theoretical concepts with practical system design, this chapter provides a solid foundation for advancing machine learning systems and engineering. + +## Purpose {.unnumbered} + + ::: {.callout-tip} ## Learning Objectives @@ -128,23 +137,6 @@ The implementation of MLPs presents several key challenges and opportunities tha These challenges and opportunities drive the development of specialized neural processing engines fo machine learning systems. While memory bandwidth limitations push designs toward sophisticated memory hierarchies (needing to handle >50,000 memory operations efficiently), the regular patterns and parallel opportunities enable efficient implementations through specialized processing units. The patterns established by MLPs form a baseline that more specialized neural architectures must consider in their implementations. -:::{.callout-caution #exr-mlp collapse="false"} - -##### Multilayer Perceptrons (MLPs) - -We've just scratched the surface of neural networks. Now, you'll get to try and apply these concepts in practical examples. In the provided Colab notebooks, you'll explore: - -**Predicting house prices:** Learn how neural networks can analyze housing data to estimate property values. -[![](https://colab.research.google.com/assets/colab-badge.png)](https://colab.research.google.com/github/Mjrovai/UNIFEI-IESTI01-TinyML-2022.1/blob/main/00_Curse_Folder/1_Fundamentals/Class_07/TF_Boston_Housing_Regression.ipynb) - -**Image Classification:** Discover how to build a network to understand the famous MNIST handwritten digit dataset. -[![](https://colab.research.google.com/assets/colab-badge.png)](https://colab.research.google.com/github/Mjrovai/UNIFEI-IESTI01-TinyML-2022.1/blob/main/00_Curse_Folder/1_Fundamentals/Class_09/TF_MNIST_Classification_v2.ipynb) - -**Real-world medical diagnosis:** Use deep learning to tackle the important task of breast cancer classification. -[![](https://colab.research.google.com/assets/colab-badge.png)](https://colab.research.google.com/github/Mjrovai/UNIFEI-IESTI01-TinyML-2022.1/blob/main/00_Curse_Folder/1_Fundamentals/Class_13/docs/WDBC_Project/Breast_Cancer_Classification.ipynb) - -::: - ### Convolutional Neural Networks: Spatial Pattern Processing Convolutional Neural Networks (CNNs) represent a specialized neural architecture designed to efficiently process data with spatial relationships, such as images. While MLPs treat each input independently, CNNs exploit local patterns and spatial hierarchies, establishing computational patterns that have revolutionized computer vision and spatial data processing.