diff --git a/contents/core/ai_for_good/ai_for_good.qmd b/contents/core/ai_for_good/ai_for_good.qmd index b4c5d4e2..efffc127 100644 --- a/contents/core/ai_for_good/ai_for_good.qmd +++ b/contents/core/ai_for_good/ai_for_good.qmd @@ -10,9 +10,11 @@ Resources: [Slides](#sec-ai-for-good-resource), [Videos](#sec-ai-for-good-resour ![_DALL·E 3 Prompt: Illustration of planet Earth wrapped in shimmering neural networks, with diverse humans and AI robots working together on various projects like planting trees, cleaning the oceans, and developing sustainable energy solutions. The positive and hopeful atmosphere represents a united effort to create a better future._](images/png/cover_ai_good.png) -By aligning AI progress with human values, goals, and ethics, the ultimate goal of ML systems (at any scale) is to be a technology that reflects human principles and aspirations. Initiatives under "AI for Good" promote the development of AI to tackle the [UN Sustainable Development Goals](https://www.undp.org/sustainable-development-goals) (SDGs) using embedded AI technologies, expanding access to AI education, amongst other things. While it is now clear that AI will be an instrumental part of progress towards the SDGs, its adoption and impact are limited by the immense power consumption, strong connectivity requirements, and high costs of cloud-based deployments. TinyML can circumvent many of these issues by allowing ML models to run on low-cost and low-power microcontrollers. +## Purpose {.unnumbered} -> The "AI for Good" movement is critical in cultivating a future where an AI-empowered society is more just, sustainable, and prosperous for all humanity. +_How can we harness machine learning systems to address critical societal challenges, and what principles guide the development of solutions that create lasting positive impact?_ + +The application of AI systems to societal challenges represents the culmination of technical capability and social responsibility. Impact-driven development reveals essential patterns for translating technological potential into meaningful change, highlighting critical relationships between system design and societal outcomes. The implementation of solutions for social good showcases pathways for addressing complex challenges while maintaining technical rigor and operational effectiveness. Understanding these impact dynamics provides insights into creating transformative systems, establishing core principles for designing AI solutions that advance human welfare, and promote positive societal transformation. ::: {.callout-tip} @@ -32,6 +34,10 @@ By aligning AI progress with human values, goals, and ethics, the ultimate goal ## Overview +By aligning AI progress with human values, goals, and ethics, the ultimate goal of ML systems (at any scale) is to be a technology that reflects human principles and aspirations. Initiatives under "AI for Good" promote the development of AI to tackle the [UN Sustainable Development Goals](https://www.undp.org/sustainable-development-goals) (SDGs) using embedded AI technologies, expanding access to AI education, amongst other things. While it is now clear that AI will be an instrumental part of progress towards the SDGs, its adoption and impact are limited by the immense power consumption, strong connectivity requirements, and high costs of cloud-based deployments. TinyML can circumvent many of these issues by allowing ML models to run on low-cost and low-power microcontrollers. + +> The "AI for Good" movement is critical in cultivating a future where an AI-empowered society is more just, sustainable, and prosperous for all humanity. + To give ourselves a framework around which to think about AI for social good, we will be following the UN Sustainable Development Goals (SDGs). The UN SDGs are a collection of 17 global goals, shown in @fig-sdg, adopted by the United Nations in 2015 as part of the 2030 Agenda for Sustainable Development. The SDGs address global challenges related to poverty, inequality, climate change, environmental degradation, prosperity, and peace and justice. ![United Nations Sustainable Development Goals (SDG). Source: [United Nations](https://sdgs.un.org/goals).](https://www.un.org/sustainabledevelopment/wp-content/uploads/2015/12/english_SDG_17goals_poster_all_languages_with_UN_emblem_1.png){#fig-sdg} diff --git a/contents/core/benchmarking/benchmarking.qmd b/contents/core/benchmarking/benchmarking.qmd index af5546e6..c631d540 100644 --- a/contents/core/benchmarking/benchmarking.qmd +++ b/contents/core/benchmarking/benchmarking.qmd @@ -10,9 +10,11 @@ Resources: [Slides](#sec-benchmarking-ai-resource), [Videos](#sec-benchmarking-a ![_DALL·E 3 Prompt: Photo of a podium set against a tech-themed backdrop. On each tier of the podium, there are AI chips with intricate designs. The top chip has a gold medal hanging from it, the second one has a silver medal, and the third has a bronze medal. Banners with 'AI Olympics' are displayed prominently in the background._](images/png/cover_ai_benchmarking.png) -Benchmarking is critical to developing and deploying machine learning systems, especially TinyML applications. Benchmarks allow developers to measure and compare the performance of different model architectures, training procedures, and deployment strategies. This provides key insights into which approaches work best for the problem at hand and the constraints of the deployment environment. +## Purpose {.unnumbered} -This chapter will provide an overview of popular ML benchmarks, best practices for benchmarking, and how to use benchmarks to improve model development and system performance. It provides developers with the right tools and knowledge to effectively benchmark and optimize their systems, especially for TinyML systems. +_How can quantitative evaluation reshape the development of machine learning systems, and what metrics reveal true system capabilities?_ + +The measurement and analysis of AI system performance represent a critical element in bridging theoretical capabilities with practical outcomes. Systematic evaluation approaches reveal fundamental relationships between model behavior, resource utilization, and operational reliability. These measurements draw out the essential trade-offs across accuracy, efficiency, and scalability, providing insights that guide architectural decisions throughout the development lifecycle. These evaluation frameworks establish core principles for assessing and validating system design choices and enable the creation of robust solutions that meet increasingly complex performance requirements across diverse deployment scenarios. ::: {.callout-tip} diff --git a/contents/core/data_engineering/data_engineering.qmd b/contents/core/data_engineering/data_engineering.qmd index 575efe0f..2388edfb 100644 --- a/contents/core/data_engineering/data_engineering.qmd +++ b/contents/core/data_engineering/data_engineering.qmd @@ -10,7 +10,13 @@ Resources: [Slides](#sec-data-engineering-resource), [Videos](#sec-data-engineer ![_DALL·E 3 Prompt: Create a rectangular illustration visualizing the concept of data engineering. Include elements such as raw data sources, data processing pipelines, storage systems, and refined datasets. Show how raw data is transformed through cleaning, processing, and storage to become valuable information that can be analyzed and used for decision-making._](images/png/cover_data_engineering.png) -Data is the lifeblood of AI systems. Without good data, even the most advanced machine-learning algorithms will not succeed. However, TinyML models operate on devices with limited processing power and memory. This section explores the intricacies of building high-quality datasets to fuel our AI models. Data engineering involves collecting, storing, processing, and managing data to train machine learning models. +Let me refine this to strengthen the academic tone and remove direct reader references while maintaining the focus on data engineering principles: + +## Purpose {.unnumbered} + +_Why does data infrastructure form the foundation of AI system success, and how does its design influence system capabilities?_ + +At the heart of every AI system lies its data infrastructure, determining both its potential and limitations. While architectural patterns and workflows define the system's structure, data quality and accessibility ultimately govern its effectiveness. The engineering of data systems requires careful consideration of acquisition, processing, and delivery mechanisms, establishing the bedrock upon which sophisticated AI capabilities can be built. This perspective reveals how data engineering decisions ripple through every aspect of system performance and reliability, shaping the possibilities for model development, training efficiency, and deployment success. ::: {.callout-tip} diff --git a/contents/core/dl_architectures/dl_architectures.qmd b/contents/core/dl_architectures/dl_architectures.qmd index 8a8a2676..21d1660e 100644 --- a/contents/core/dl_architectures/dl_architectures.qmd +++ b/contents/core/dl_architectures/dl_architectures.qmd @@ -10,6 +10,12 @@ Resources: [Slides](#sec-deep-learning-primer-resource), [Videos](#sec-deep-lear ![_DALL·E 3 Prompt: A visually striking rectangular image illustrating the interplay between deep learning algorithms like CNNs, RNNs, and Attention Networks, interconnected with machine learning systems. The composition features neural network diagrams blending seamlessly with representations of computational systems such as processors, graphs, and data streams. Bright neon tones contrast against a dark futuristic background, symbolizing cutting-edge technology and intricate system complexity._](images/png/cover_dl_arch.png) +## Purpose {.unnumbered} + +_What recurring patterns emerge across modern deep learning architectures, and how do these patterns enable systematic approaches to AI system design?_ + +Deep learning architectures represent a convergence of computational patterns that form the building blocks of modern AI systems. Understanding these foundational patterns—from convolutional structures to attention mechanisms—reveals how complex models arise from simple, repeatable components. The examination of these architectural elements provides insights into the systematic construction of flexible, efficient AI systems, establishing core principles that influence every aspect of system design and deployment. These structural insights illuminate the path toward creating scalable, adaptable solutions across diverse application domains. + ::: {.callout-tip} ## Learning Objectives @@ -128,23 +134,6 @@ The implementation of MLPs presents several key challenges and opportunities tha These challenges and opportunities drive the development of specialized neural processing engines fo machine learning systems. While memory bandwidth limitations push designs toward sophisticated memory hierarchies (needing to handle >50,000 memory operations efficiently), the regular patterns and parallel opportunities enable efficient implementations through specialized processing units. The patterns established by MLPs form a baseline that more specialized neural architectures must consider in their implementations. -:::{.callout-caution #exr-mlp collapse="false"} - -##### Multilayer Perceptrons (MLPs) - -We've just scratched the surface of neural networks. Now, you'll get to try and apply these concepts in practical examples. In the provided Colab notebooks, you'll explore: - -**Predicting house prices:** Learn how neural networks can analyze housing data to estimate property values. -[![](https://colab.research.google.com/assets/colab-badge.png)](https://colab.research.google.com/github/Mjrovai/UNIFEI-IESTI01-TinyML-2022.1/blob/main/00_Curse_Folder/1_Fundamentals/Class_07/TF_Boston_Housing_Regression.ipynb) - -**Image Classification:** Discover how to build a network to understand the famous MNIST handwritten digit dataset. -[![](https://colab.research.google.com/assets/colab-badge.png)](https://colab.research.google.com/github/Mjrovai/UNIFEI-IESTI01-TinyML-2022.1/blob/main/00_Curse_Folder/1_Fundamentals/Class_09/TF_MNIST_Classification_v2.ipynb) - -**Real-world medical diagnosis:** Use deep learning to tackle the important task of breast cancer classification. -[![](https://colab.research.google.com/assets/colab-badge.png)](https://colab.research.google.com/github/Mjrovai/UNIFEI-IESTI01-TinyML-2022.1/blob/main/00_Curse_Folder/1_Fundamentals/Class_13/docs/WDBC_Project/Breast_Cancer_Classification.ipynb) - -::: - ### Convolutional Neural Networks: Spatial Pattern Processing Convolutional Neural Networks (CNNs) represent a specialized neural architecture designed to efficiently process data with spatial relationships, such as images. While MLPs treat each input independently, CNNs exploit local patterns and spatial hierarchies, establishing computational patterns that have revolutionized computer vision and spatial data processing. diff --git a/contents/core/dl_primer/dl_primer.qmd b/contents/core/dl_primer/dl_primer.qmd index e27d3fb9..96736f5c 100644 --- a/contents/core/dl_primer/dl_primer.qmd +++ b/contents/core/dl_primer/dl_primer.qmd @@ -10,7 +10,11 @@ Resources: [Slides](#sec-deep-learning-primer-resource), [Videos](#sec-deep-lear ![_DALL·E 3 Prompt: A rectangular illustration divided into two halves on a clean white background. The left side features a detailed and colorful depiction of a biological neural network, showing interconnected neurons with glowing synapses and dendrites. The right side displays a sleek and modern artificial neural network, represented by a grid of interconnected nodes and edges resembling a digital circuit. The transition between the two sides is distinct but harmonious, with each half clearly illustrating its respective theme: biological on the left and artificial on the right._](images/png/cover_nn_primer.png) -This chapter bridges fundamental neural network concepts with real-world system implementations by exploring how different architectural patterns process information and influence system design. Instead of concentrating on algorithms or model accuracy—typical topics in deep learning algorithms courses or books—this chapter focuses on how architectural choices shape distinct computational patterns that drive system-level decisions, such as memory hierarchy, processing units, and hardware acceleration. By understanding these relationships, readers will gain the insight needed to make informed decisions about model selection, system optimization, and hardware/software co-design in the chapters that follow. +## Purpose {.unnumbered} + +_What inspiration from nature drives the development of machine learning systems, and how do biological neural processes inform their fundamental design?_ + +The neural systems of nature offer profound insights into information processing and adaptation, inspiring the core principles of modern machine learning. Translating biological mechanisms into computational frameworks illuminates fundamental patterns that shape artificial neural networks. These patterns reveal essential relationships between biological principles and their digital counterparts, establishing building blocks for understanding more complex architectures. Analyzing these mappings from natural to artificial provides critical insights into system design, laying the foundation for exploring advanced neural architectures and their practical implementations. ::: {.callout-tip} diff --git a/contents/core/efficient_ai/efficient_ai.qmd b/contents/core/efficient_ai/efficient_ai.qmd index bb79f36c..3054678f 100644 --- a/contents/core/efficient_ai/efficient_ai.qmd +++ b/contents/core/efficient_ai/efficient_ai.qmd @@ -10,7 +10,11 @@ Resources: [Slides](#sec-efficient-ai-resource), [Videos](#sec-efficient-ai-reso ![_DALL·E 3 Prompt: A conceptual illustration depicting efficiency in artificial intelligence using a shipyard analogy. The scene shows a bustling shipyard where containers represent bits or bytes of data. These containers are being moved around efficiently by cranes and vehicles, symbolizing the streamlined and rapid information processing in AI systems. The shipyard is meticulously organized, illustrating the concept of optimal performance within the constraints of limited resources. In the background, ships are docked, representing different platforms and scenarios where AI is applied. The atmosphere should convey advanced technology with an underlying theme of sustainability and wide applicability._](images/png/cover_efficient_ai.png) -Efficiency in artificial intelligence is not simply a luxury but a necessity. In this chapter, we dive into the key concepts underpinning AI systems' efficiency. The computational demands on neural networks can be daunting, even for minimal systems. For AI to be seamlessly integrated into everyday devices and essential systems, it must perform optimally within the constraints of limited resources while maintaining its efficacy. The pursuit of efficiency guarantees that AI models are streamlined, rapid, and sustainable, thereby widening their applicability across various platforms and scenarios. +## Purpose {.unnumbered} + +_What fundamental resource constraints challenge machine learning systems, and how do they shape architectural decisions?_ + +The interplay between computational demands and system resources drives fundamental design choices in machine learning systems. Each efficiency decision---from arithmetic precision to memory access patterns---cascades through multiple layers of the system stack, revealing essential relationships between resource utilization and model capability. These relationships guide architectural choices that balance system performance against resource constraints, establishing core principles for designing AI solutions that achieve maximum impact with minimum resources. ::: {.callout-tip} diff --git a/contents/core/frameworks/frameworks.qmd b/contents/core/frameworks/frameworks.qmd index 1288d463..953cb619 100644 --- a/contents/core/frameworks/frameworks.qmd +++ b/contents/core/frameworks/frameworks.qmd @@ -10,9 +10,13 @@ Resources: [Slides](#sec-ai-frameworks-resource), [Videos](#sec-ai-frameworks-re ![_DALL·E 3 Prompt: Illustration in a rectangular format, designed for a professional textbook, where the content spans the entire width. The vibrant chart represents training and inference frameworks for ML. Icons for TensorFlow, Keras, PyTorch, ONNX, and TensorRT are spread out, filling the entire horizontal space, and aligned vertically. Each icon is accompanied by brief annotations detailing their features. The lively colors like blues, greens, and oranges highlight the icons and sections against a soft gradient background. The distinction between training and inference frameworks is accentuated through color-coded sections, with clean lines and modern typography maintaining clarity and focus._](images/png/cover_ml_frameworks.png) -This chapter explores the landscape of AI frameworks that serve as the foundation for developing machine learning systems. AI frameworks provide the tools, libraries, and environments to design, train, and deploy machine learning models. We explore the evolutionary trajectory of these frameworks, dissect the workings of TensorFlow, and provide insights into the core components and advanced features that define these frameworks. +Let me refine this to maintain a textbook tone, remove the "By" construction, and strengthen the systems perspective: -Furthermore, we investigate the specialization of frameworks tailored to specific needs, the emergence of frameworks specifically designed for embedded AI, and the criteria for selecting the most suitable framework for your project. This exploration will be rounded off by a glimpse into the future trends expected to shape the landscape of ML frameworks in the coming years. +## Purpose {.unnumbered} + +_How do AI frameworks bridge the gap between theoretical design and practical implementation, and what role do they play in enabling scalable machine learning systems?_ + +AI frameworks represent critical middleware that transform abstract model specifications into executable implementations. The evolution of these frameworks reveals fundamental patterns for translating high-level designs into efficient computational workflows. Their architecture illuminates essential trade-offs between abstraction, performance, and portability, providing systematic approaches to managing complexity in machine learning systems. Understanding framework capabilities and constraints offers crucial insights into the engineering decisions that shape system scalability, enabling the development of robust, deployable solutions across diverse computing environments. ::: {.callout-tip} diff --git a/contents/core/hw_acceleration/hw_acceleration.qmd b/contents/core/hw_acceleration/hw_acceleration.qmd index f247bcae..d160945d 100644 --- a/contents/core/hw_acceleration/hw_acceleration.qmd +++ b/contents/core/hw_acceleration/hw_acceleration.qmd @@ -10,7 +10,11 @@ Resources: [Slides](#sec-ai-acceleration-resource), [Videos](#sec-ai-acceleratio ![_DALL·E 3 Prompt: Create an intricate and colorful representation of a System on Chip (SoC) design in a rectangular format. Showcase a variety of specialized machine learning accelerators and chiplets, all integrated into the processor. Provide a detailed view inside the chip, highlighting the rapid movement of electrons. Each accelerator and chiplet should be designed to interact with neural network neurons, layers, and activations, emphasizing their processing speed. Depict the neural networks as a network of interconnected nodes, with vibrant data streams flowing between the accelerator pieces, showcasing the enhanced computation speed._](images/png/cover_ai_hardware.png) -Deploying ML on edge devices presents challenges such as limited processing speed, memory constraints, and stringent energy efficiency requirements. To overcome these challenges, specialized hardware acceleration is key. Hardware accelerators are designed to optimize compute-intensive tasks like inference by using custom silicon chips tailored for matrix multiplications, providing significant speedups compared to general-purpose CPUs. This enables real-time execution of advanced models on devices with strict constraints on size, weight, and power. +## Purpose {.unnumbered} + +_How does specialized computing transform the performance frontier of machine learning systems, and what principles guide the design of acceleration strategies?_ + +The evolution of computational acceleration in machine learning systems represents a fundamental shift in how processing resources are architected and utilized. Each acceleration approach introduces unique patterns for matching algorithmic structure with computational capabilities, revealing essential relationships between model design and execution efficiency. The integration of specialized computing elements demonstrates trade-offs between performance, power efficiency, and system complexity. These architectural interactions provide insights into system-level optimization strategies, establishing core principles for designing AI solutions that effectively leverage advanced computation across diverse deployment environments. ::: {.callout-tip} diff --git a/contents/core/ml_systems/ml_systems.qmd b/contents/core/ml_systems/ml_systems.qmd index 18430f20..926ca4c3 100644 --- a/contents/core/ml_systems/ml_systems.qmd +++ b/contents/core/ml_systems/ml_systems.qmd @@ -10,11 +10,11 @@ Resources: [Slides](#sec-ml-systems-resource), [Videos](#sec-ml-systems-resource ![*DALL·E 3 Prompt: Illustration in a rectangular format depicting the merger of embedded systems with Embedded AI. The left half of the image portrays traditional embedded systems, including microcontrollers and processors, detailed and precise. The right half showcases the world of artificial intelligence, with abstract representations of machine learning models, neurons, and data flow. The two halves are distinctly separated, emphasizing the individual significance of embedded tech and AI, but they come together in harmony at the center.*](images/png/cover_ml_systems.png) -The convergence of machine learning and computing systems has ushered in a new era of intelligent computing that extends from powerful cloud infrastructures to tiny embedded devices. Machine learning (ML) systems represent this intersection where algorithmic intelligence meets hardware constraints, creating solutions that must carefully balance computational power, energy efficiency, and real-world practicality. As ML continues to transform various sectors, understanding how to effectively deploy these systems across different computing platforms has become increasingly crucial. +## Purpose {.unnumbered} -Modern ML systems span a remarkable spectrum of capabilities and constraints. At one end, cloud-based systems harness vast computational resources to train and deploy complex models. At the other end, tiny embedded systems bring ML capabilities to resource-constrained devices that operate on minimal power. Between these extremes lie edge and mobile computing solutions, each offering unique advantages for specific use cases. This diversity in deployment options presents both opportunities and challenges for system designers and ML practitioners. +_How do the diverse environments where machine learning operates shape the fundamental nature of these systems, and what drives their widespread deployment across computing platforms?_ -This chapter explores this diverse landscape of ML systems, beginning with a comprehensive overview of different deployment paradigms. We'll examine how each approach addresses specific challenges and requirements, from processing power and memory constraints to energy efficiency and real-time performance. Through detailed comparisons and real-world examples, we'll develop a deep understanding of when and how to employ each type of ML system effectively. +The deployment of machine learning systems across varied computing environments reveals essential insights into the relationship between theoretical principles and practical implementation. Each computing environment - from large-scale distributed systems to resource-constrained devices - introduces distinct requirements that influence both system architecture and algorithmic approaches. Understanding these relationships reveals core engineering principles that govern the design of machine learning systems. This understanding provides a foundation for examining how theoretical concepts translate into practical implementations, and how system designs adapt to meet diverse computational, memory, and energy constraints. :::{.callout-tip} diff --git a/contents/core/ondevice_learning/ondevice_learning.qmd b/contents/core/ondevice_learning/ondevice_learning.qmd index 09a7d192..fa9e8ed0 100644 --- a/contents/core/ondevice_learning/ondevice_learning.qmd +++ b/contents/core/ondevice_learning/ondevice_learning.qmd @@ -10,7 +10,11 @@ Resources: [Slides](#sec-on-device-learning-resource), [Videos](#sec-on-device-l ![_DALL·E 3 Prompt: Drawing of a smartphone with its internal components exposed, revealing diverse miniature engineers of different genders and skin tones actively working on the ML model. The engineers, including men, women, and non-binary individuals, are tuning parameters, repairing connections, and enhancing the network on the fly. Data flows into the ML model, being processed in real-time, and generating output inferences._](images/png/cover_ondevice_learning.png) -On-device Learning represents a significant innovation for embedded and edge IoT devices, enabling models to train and update directly on small local devices. This contrasts with traditional methods, where models are trained on expansive cloud computing resources before deployment. With on-device learning, devices like smart speakers, wearables, and industrial sensors can refine models in real-time based on local data without needing to transmit data externally. For example, a voice-enabled smart speaker could learn and adapt to its owner's speech patterns and vocabulary right on the device. However, there is no such thing as a free lunch; therefore, in this chapter, we will discuss both the benefits and the limitations of on-device learning. +## Purpose {.unnumbered} + +_How does learning at the edge transform traditional machine learning paradigms, and what principles enable effective adaptation in resource-constrained environments?_ + +The migration of learning capabilities to edge devices represents a fundamental shift in how AI systems evolve and adapt. Local learning introduces unique patterns for balancing model adaptation with resource limitations, revealing essential relationships between computational constraints and system autonomy. The implementation of on-device training emphasizes the trade-offs between learning capability, energy efficiency, and operational independence. These adaptation mechanisms provide insights into designing self-evolving systems, establishing core principles for creating AI solutions that can learn and improve within the constraints of local computing environments. ::: {.callout-tip} diff --git a/contents/core/ops/ops.qmd b/contents/core/ops/ops.qmd index bdceaf3f..89b74e44 100644 --- a/contents/core/ops/ops.qmd +++ b/contents/core/ops/ops.qmd @@ -10,7 +10,11 @@ Resources: [Slides](#sec-embedded-aiops-resource), [Videos](#sec-embedded-aiops- ![_DALL·E 3 Prompt: Create a detailed, wide rectangular illustration of an AI workflow. The image should showcase the process across six stages, with a flow from left to right: 1. Data collection, with diverse individuals of different genders and descents using a variety of devices like laptops, smartphones, and sensors to gather data. 2. Data processing, displaying a data center with active servers and databases with glowing lights. 3. Model training, represented by a computer screen with code, neural network diagrams, and progress indicators. 4. Model evaluation, featuring people examining data analytics on large monitors. 5. Deployment, where the AI is integrated into robotics, mobile apps, and industrial equipment. 6. Monitoring, showing professionals tracking AI performance metrics on dashboards to check for accuracy and concept drift over time. Each stage should be distinctly marked and the style should be clean, sleek, and modern with a dynamic and informative color scheme._](images/png/cover_ml_ops.png) -In this chapter, we will dive into the practices and frameworks needed to successfully develop, deploy, and manage machine learning models from start to finish. You will learn about each stage in the ML workflow, from data collection and model training to evaluation, deployment, and ongoing monitoring. We will discuss the role of automation, collaboration, and continuous improvement, highlighting why they are essential for keeping ML systems efficient and reliable. We will also explore different deployment environments, from powerful cloud servers to resource-limited edge devices, looking at the unique challenges each presents. Through concrete examples, you will see how to design and operate ML systems that deliver consistent, reliable performance, no matter where they are deployed. By the end of this chapter, you will have a solid grasp of ML model management and be ready to build and maintain ML applications that provide lasting value. +## Purpose {.unnumbered} + +_What systematic principles enable continuous evolution of machine learning systems in production, and how do operational requirements shape deployment architecture?_ + +The transition from experimental to production environments represents a fundamental challenge in scaling machine learning systems. Operational practices reveal essential patterns for maintaining reliability and adaptability throughout the system lifecycle, highlighting critical relationships between deployment architecture and continuous delivery. The implementation of MLOps frameworks draws out the key trade-offs between automation, monitoring, and governance that shape system evolution. Understanding these operational dynamics provides insights into managing complex AI workflows, establishing core principles for creating systems that remain robust and responsive in dynamic production environments. ::: {.callout-tip} diff --git a/contents/core/optimizations/optimizations.qmd b/contents/core/optimizations/optimizations.qmd index e9b721eb..1cd511f7 100644 --- a/contents/core/optimizations/optimizations.qmd +++ b/contents/core/optimizations/optimizations.qmd @@ -10,7 +10,11 @@ Resources: [Slides](#sec-model-optimizations-resource), [Videos](#sec-model-opti ![_DALL·E 3 Prompt: Illustration of a neural network model represented as a busy construction site, with a diverse group of construction workers, both male and female, of various ethnicities, labeled as 'pruning', 'quantization', and 'sparsity'. They are working together to make the neural network more efficient and smaller, while maintaining high accuracy. The 'pruning' worker, a Hispanic female, is cutting unnecessary connections from the middle of the network. The 'quantization' worker, a Caucasian male, is adjusting or tweaking the weights all over the place. The 'sparsity' worker, an African female, is removing unnecessary nodes to shrink the model. Construction trucks and cranes are in the background, assisting the workers in their tasks. The neural network is visually transforming from a complex and large structure to a more streamlined and smaller one._](images/png/cover_model_optimizations.png) -When machine learning models are deployed on systems, especially on resource-constrained embedded systems, the optimization of models is a necessity. While machine learning inherently often demands substantial computational resources, the systems are inherently limited in memory, processing power, and energy. This chapter will dive into the art and science of optimizing machine learning models to ensure they are lightweight, efficient, and effective when deployed in TinyML scenarios. +## Purpose {.unnumbered} + +_How do theoretical neural network models transform into practical solutions, and what techniques bridge implementation gaps?_ + +The adaptation of machine learning models for real-world deployment demands systematic transformation approaches. Each optimization technique introduces unique methods for preserving model capabilities while meeting deployment constraints, revealing fundamental patterns in the relationship between theoretical design and practical implementation. These patterns highlight strategies for translating research advances into production systems, establishing core principles for creating solutions that maintain effectiveness while satisfying real-world requirements. ::: {.callout-tip} diff --git a/contents/core/privacy_security/privacy_security.qmd b/contents/core/privacy_security/privacy_security.qmd index e0c9d28f..11045fad 100644 --- a/contents/core/privacy_security/privacy_security.qmd +++ b/contents/core/privacy_security/privacy_security.qmd @@ -10,7 +10,11 @@ Resources: [Slides](#sec-security-and-privacy-resource), [Videos](#sec-security ![_DALL·E 3 Prompt: An illustration on privacy and security in machine learning systems. The image shows a digital landscape with a network of interconnected nodes and data streams, symbolizing machine learning algorithms. In the foreground, there's a large lock superimposed over the network, representing privacy and security. The lock is semi-transparent, allowing the underlying network to be partially visible. The background features binary code and digital encryption symbols, emphasizing the theme of cybersecurity. The color scheme is a mix of blues, greens, and grays, suggesting a high-tech, digital environment._](images/png/cover_security_privacy.png) -Security and privacy are critical when developing real-world machine learning systems. As machine learning is increasingly applied to sensitive domains like healthcare, finance, and personal data, protecting confidentiality and preventing misuse of data and models becomes imperative. Anyone aiming to build robust and responsible ML systems must grasp potential security and privacy risks such as data leaks, model theft, adversarial attacks, bias, and unintended access to private information. We also need to understand best practices for mitigating these risks. Most importantly, security and privacy cannot be an afterthought and must be proactively addressed throughout the ML system development lifecycle - from data collection and labeling to model training, evaluation, and deployment. Embedding security and privacy considerations into each stage of building, deploying, and managing machine learning systems is essential for safely unlocking the benefits of AI. +## Purpose {.unnumbered} + +_What principles guide the protection of machine learning systems, and how do security and privacy requirements transform system architecture?_ + +The integration of protection mechanisms into AI systems represents a fundamental dimension of modern system design. Security considerations reveal essential patterns for safeguarding data, models, and infrastructure while maintaining operational effectiveness. The implementation of defensive strategies brings to light the trade-offs between protection, performance, and usability that shape architectural decisions across the AI lifecycle. Understanding these security dynamics provides insights into creating trustworthy systems, establishing core principles for designing solutions that preserve privacy and resist adversarial threats while meeting functional requirements in production environments. ::: {.callout-tip} diff --git a/contents/core/responsible_ai/responsible_ai.qmd b/contents/core/responsible_ai/responsible_ai.qmd index b85e5ee6..7a0c9cfc 100644 --- a/contents/core/responsible_ai/responsible_ai.qmd +++ b/contents/core/responsible_ai/responsible_ai.qmd @@ -10,7 +10,11 @@ Resources: [Slides](#sec-responsible-ai-resource), [Videos](#sec-responsible-ai- ![_DALL·E 3 Prompt: Illustration of responsible AI in a futuristic setting with the universe in the backdrop: A human hand or hands nurturing a seedling that grows into an AI tree, symbolizing a neural network. The tree has digital branches and leaves, resembling a neural network, to represent the interconnected nature of AI. The background depicts a future universe where humans and animals with general intelligence collaborate harmoniously. The scene captures the initial nurturing of the AI as a seedling, emphasizing the ethical development of AI technology in harmony with humanity and the universe._](images/png/cover_responsible_ai.png) -As machine learning models grow across various domains, these algorithms have the potential to perpetuate historical biases, breach privacy, or enable unethical automated decisions if developed without thoughtful consideration of their societal impacts. Even systems created with good intentions can ultimately discriminate against certain demographic groups, enable surveillance, or lack transparency into their behaviors and decision-making processes. As such, machine learning engineers and companies have an ethical responsibility to proactively ensure principles of fairness, accountability, safety, and transparency are reflected in their models to prevent harm and build public trust. +## Purpose {.unnumbered} + +_How do human values translate into system architecture, and what principles enable the development of AI systems that embody ethical considerations?_ + +Embedding ethical principles into machine learning systems represents a fundamental engineering challenge. Each design decision carries moral implications, revealing essential patterns in how technical choices influence societal outcomes. The implementation of ethical frameworks demonstrates the need for new approaches to system architecture that consider human values as fundamental design constraints. Understanding these moral-technical relationships provides insights into creating principled systems, establishing core methodologies for designing AI solutions that advance capabilities while upholding human values. ::: {.callout-tip} diff --git a/contents/core/robust_ai/robust_ai.qmd b/contents/core/robust_ai/robust_ai.qmd index 91b24604..ec0d397f 100644 --- a/contents/core/robust_ai/robust_ai.qmd +++ b/contents/core/robust_ai/robust_ai.qmd @@ -10,11 +10,11 @@ Resources: [Slides](#sec-robust-ai-resource), [Videos](#sec-robust-ai-resource), ![_DALL·E 3 Prompt: Create an image featuring an advanced AI system symbolized by an intricate, glowing neural network, deeply nested within a series of progressively larger and more fortified shields. Each shield layer represents a layer of defense, showcasing the system's robustness against external threats and internal errors. The neural network, at the heart of this fortress of shields, radiates with connections that signify the AI's capacity for learning and adaptation. This visual metaphor emphasizes not only the technological sophistication of the AI but also its resilience and security, set against the backdrop of a state-of-the-art, secure server room filled with the latest in technological advancements. The image aims to convey the concept of ultimate protection and resilience in the field of artificial intelligence._](./images/png/cover_robust_ai.png) -The development of robust machine learning systems has become increasingly crucial. As these systems are deployed in various critical applications, from autonomous vehicles to healthcare diagnostics, ensuring their resilience to faults and errors is paramount. +## Purpose {.unnumbered} -Robust AI, in the context of hardware faults, software faults, and errors, plays an important role in maintaining the reliability, safety, and performance of machine learning systems. By addressing the challenges posed by transient, permanent, and intermittent hardware faults [@ahmadilivani2024systematic], as well as bugs, design flaws, and implementation errors in software [@zhang2008distribution], robust AI techniques enable machine learning systems to operate effectively even in adverse conditions. +_How do uncertainty and variability shape machine learning system design, and what principles enable reliable operation in challenging conditions?_ -This chapter explores the fundamental concepts, techniques, and tools for building fault-tolerant and error-resilient machine learning systems. It empowers researchers and practitioners to develop AI solutions that can withstand the complexities and uncertainties of real-world environments. +The exposure of AI systems to real-world conditions presents fundamental challenges in maintaining consistent performance. Operational patterns reveal essential relationships between system stability and environmental variability, highlighting critical trade-offs between resilience and efficiency. The implementation of robust architectures emphasizes the need for strategies to maintain reliability across diverse and unpredictable scenarios while preserving core functionality. Understanding these resilience dynamics provides insights into creating dependable systems, establishing principles for designing AI solutions that maintain effectiveness even when faced with distributional shifts, noise, and adversarial conditions. ::: {.callout-tip} @@ -34,7 +34,7 @@ This chapter explores the fundamental concepts, techniques, and tools for buildi ## Overview -Robust AI refers to a system's ability to maintain its performance and reliability in the presence of errors. A robust machine learning system is designed to be fault-tolerant and error-resilient, capable of operating effectively even under adverse conditions. +Robust AI refers to a system's ability to maintain its performance and reliability in the presence of errors. A robust machine learning system is designed to be fault-tolerant and error-resilient, capable of operating effectively even under adverse conditions. As ML systems become increasingly integrated into various aspects of our lives, from cloud-based services to edge devices and embedded systems, the impact of hardware and software faults on their performance and reliability becomes more significant. In the future, as ML systems become more complex and are deployed in even more critical applications, the need for robust and fault-tolerant designs will be paramount. diff --git a/contents/core/sustainable_ai/sustainable_ai.qmd b/contents/core/sustainable_ai/sustainable_ai.qmd index 70e1563f..00f18bde 100644 --- a/contents/core/sustainable_ai/sustainable_ai.qmd +++ b/contents/core/sustainable_ai/sustainable_ai.qmd @@ -10,6 +10,12 @@ Resources: [Slides](#sec-sustainable-ai-resource), [Videos](#sec-sustainable-ai- ![_DALL·E 3 Prompt: 3D illustration on a light background of a sustainable AI network interconnected with a myriad of eco-friendly energy sources. The AI actively manages and optimizes its energy from sources like solar arrays, wind turbines, and hydro dams, emphasizing power efficiency and performance. Deep neural networks spread throughout, receiving energy from these sustainable resources._](images/png/cover_sustainable_ai.png) +## Purpose {.unnumbered} + +_What principles emerge when we consider machine learning systems through an ecological lens, and how does environmental stewardship reshape system architecture?_ + +The ecological footprint of AI computation presents a fundamental challenge to technological advancement. Each design decision carries environmental implications, revealing essential tensions between computational capability and ecological responsibility. The pursuit of sustainable practices illuminates new approaches to system architecture that consider environmental impact as a primary design constraint. Understanding these ecological relationships provides insights into creating next-generation systems, establishing core principles for designing AI solutions that advance technology while preserving planetary resources. + ::: {.callout-tip} ## Learning Objectives diff --git a/contents/core/training/training.qmd b/contents/core/training/training.qmd index e8272200..5af40b7e 100644 --- a/contents/core/training/training.qmd +++ b/contents/core/training/training.qmd @@ -10,7 +10,11 @@ Resources: [Slides](#sec-ai-training-resource), [Videos](#sec-ai-frameworks-reso ![_DALL·E 3 Prompt: An illustration for AI training, depicting a neural network with neurons that are being repaired and firing. The scene includes a vast network of neurons, each glowing and firing to represent activity and learning. Among these neurons, small figures resembling engineers and scientists are actively working, repairing and tweaking the neurons. These miniature workers symbolize the process of training the network, adjusting weights and biases to achieve convergence. The entire scene is a visual metaphor for the intricate and collaborative effort involved in AI training, with the workers representing the continuous optimization and learning within a neural network. The background is a complex array of interconnected neurons, creating a sense of depth and complexity._](images/png/ai_training.png) -Training is central to developing accurate and useful AI systems using machine learning techniques. At a high level, training involves feeding data into machine learning algorithms so they can learn patterns and make predictions. However, effectively training models requires tackling various challenges around data, algorithms, optimization of model parameters, and enabling generalization. This chapter will explore the nuances and considerations around training machine learning models. +## Purpose {.unnumbered} + +_What essential principles guide the training of machine learning systems, and how do computational environments shape training strategies?_ + +The transformation of data into actionable models through training represents a fundamental challenge in machine learning systems. Training architectures emphasize the delicate balance between algorithmic optimization and computational resource management, illuminating critical relationships between data movement, model convergence, and system performance. These relationships shape core engineering decisions regarding distributed computation, memory hierarchy, and processing efficiency. Understanding these training dynamics offers insights into the practical constraints that impact model development, establishing key principles for designing robust training pipelines that can scale across various computational environments. ::: {.callout-tip} diff --git a/contents/core/workflow/workflow.qmd b/contents/core/workflow/workflow.qmd index 41bf1b73..fe2ac505 100644 --- a/contents/core/workflow/workflow.qmd +++ b/contents/core/workflow/workflow.qmd @@ -10,9 +10,11 @@ Resources: [Slides](#sec-ai-workflow-resource), [Videos](#sec-ai-workflow-resour ![_DALL·E 3 Prompt: Create a rectangular illustration of a stylized flowchart representing the AI workflow/pipeline. From left to right, depict the stages as follows: 'Data Collection' with a database icon, 'Data Preprocessing' with a filter icon, 'Model Design' with a brain icon, 'Training' with a weight icon, 'Evaluation' with a checkmark, and 'Deployment' with a rocket. Connect each stage with arrows to guide the viewer horizontally through the AI processes, emphasizing these steps' sequential and interconnected nature._](images/png/cover_ai_workflow.png) -The ML workflow is a structured approach that guides professionals and researchers through developing, deploying, and maintaining ML models. This workflow is generally divided into several crucial stages, each contributing to the effective development of intelligent systems. +## Purpose {.unnumbered} -In this chapter, we will explore the machine learning workflow, setting the stage for subsequent chapters that go deeper into the specifics. This chapter focuses only presenting a high-level overview of the steps involved in the ML workflow. +_Beyond individual components, how do the diverse elements of AI systems combine to create effective machine learning system solutions?_ + +The creation of practical AI solutions requires the orchestration of multiple system components into coherent workflows. Architectural patterns serve as building blocks, while workflow design highlights the connections and interactions that animate these components. This systematic perspective reveals how data flow, model training, and deployment considerations are intertwined to form robust AI systems. Analyzing these interconnections offers crucial insights into system-level design choices, establishing a framework for understanding how theoretical concepts can be translated into deployable solutions that meet real-world needs. ::: {.callout-tip} @@ -36,6 +38,10 @@ In this chapter, we will explore the machine learning workflow, setting the stag ![Multi-step design methodology for the development of a machine learning model. Commonly referred to as the machine learning lifecycle](images/png/ML_life_cycle.png){#fig-ml-life-cycle} +The ML workflow is a structured approach that guides professionals and researchers through developing, deploying, and maintaining ML models. This workflow is generally divided into several crucial stages, each contributing to the effective development of intelligent systems. + +In this chapter, we will explore the machine learning workflow, setting the stage for subsequent chapters that go deeper into the specifics. This chapter focuses only presenting a high-level overview of the steps involved in the ML workflow. + @fig-ml-life-cycle illustrates the systematic workflow required for developing a successful machine learning model. This end-to-end process, commonly referred to as the machine learning lifecycle, enables you to build, deploy, and maintain models effectively. It typically involves the following key steps: 1. **Problem Definition** - Start by clearly articulating the specific problem you want to solve. This focuses on your efforts during data collection and model building.