From 7998b68f2aadf0096644af30ee606e7fcca128fe Mon Sep 17 00:00:00 2001 From: Vijay Janapa Reddi Date: Mon, 23 Dec 2024 20:55:01 -0500 Subject: [PATCH] Updated purpose --- contents/core/ai_for_good/ai_for_good.qmd | 10 ++++++++-- contents/core/benchmarking/benchmarking.qmd | 6 ++++-- contents/core/data_engineering/data_engineering.qmd | 6 ++++-- contents/core/dl_architectures/dl_architectures.qmd | 7 ++----- contents/core/dl_primer/dl_primer.qmd | 4 ++-- contents/core/efficient_ai/efficient_ai.qmd | 4 ++-- contents/core/frameworks/frameworks.qmd | 8 +++++--- contents/core/hw_acceleration/hw_acceleration.qmd | 6 +++++- contents/core/ml_systems/ml_systems.qmd | 4 ++-- contents/core/ondevice_learning/ondevice_learning.qmd | 6 +++++- contents/core/ops/ops.qmd | 6 +++++- contents/core/optimizations/optimizations.qmd | 6 +++++- contents/core/privacy_security/privacy_security.qmd | 6 +++++- contents/core/responsible_ai/responsible_ai.qmd | 6 +++++- contents/core/robust_ai/robust_ai.qmd | 8 ++++---- contents/core/sustainable_ai/sustainable_ai.qmd | 6 ++++++ contents/core/training/training.qmd | 6 +++--- contents/core/workflow/workflow.qmd | 4 ++-- 18 files changed, 74 insertions(+), 35 deletions(-) diff --git a/contents/core/ai_for_good/ai_for_good.qmd b/contents/core/ai_for_good/ai_for_good.qmd index b4c5d4e2..efffc127 100644 --- a/contents/core/ai_for_good/ai_for_good.qmd +++ b/contents/core/ai_for_good/ai_for_good.qmd @@ -10,9 +10,11 @@ Resources: [Slides](#sec-ai-for-good-resource), [Videos](#sec-ai-for-good-resour ![_DALL·E 3 Prompt: Illustration of planet Earth wrapped in shimmering neural networks, with diverse humans and AI robots working together on various projects like planting trees, cleaning the oceans, and developing sustainable energy solutions. The positive and hopeful atmosphere represents a united effort to create a better future._](images/png/cover_ai_good.png) -By aligning AI progress with human values, goals, and ethics, the ultimate goal of ML systems (at any scale) is to be a technology that reflects human principles and aspirations. Initiatives under "AI for Good" promote the development of AI to tackle the [UN Sustainable Development Goals](https://www.undp.org/sustainable-development-goals) (SDGs) using embedded AI technologies, expanding access to AI education, amongst other things. While it is now clear that AI will be an instrumental part of progress towards the SDGs, its adoption and impact are limited by the immense power consumption, strong connectivity requirements, and high costs of cloud-based deployments. TinyML can circumvent many of these issues by allowing ML models to run on low-cost and low-power microcontrollers. +## Purpose {.unnumbered} -> The "AI for Good" movement is critical in cultivating a future where an AI-empowered society is more just, sustainable, and prosperous for all humanity. +_How can we harness machine learning systems to address critical societal challenges, and what principles guide the development of solutions that create lasting positive impact?_ + +The application of AI systems to societal challenges represents the culmination of technical capability and social responsibility. Impact-driven development reveals essential patterns for translating technological potential into meaningful change, highlighting critical relationships between system design and societal outcomes. The implementation of solutions for social good showcases pathways for addressing complex challenges while maintaining technical rigor and operational effectiveness. Understanding these impact dynamics provides insights into creating transformative systems, establishing core principles for designing AI solutions that advance human welfare, and promote positive societal transformation. ::: {.callout-tip} @@ -32,6 +34,10 @@ By aligning AI progress with human values, goals, and ethics, the ultimate goal ## Overview +By aligning AI progress with human values, goals, and ethics, the ultimate goal of ML systems (at any scale) is to be a technology that reflects human principles and aspirations. Initiatives under "AI for Good" promote the development of AI to tackle the [UN Sustainable Development Goals](https://www.undp.org/sustainable-development-goals) (SDGs) using embedded AI technologies, expanding access to AI education, amongst other things. While it is now clear that AI will be an instrumental part of progress towards the SDGs, its adoption and impact are limited by the immense power consumption, strong connectivity requirements, and high costs of cloud-based deployments. TinyML can circumvent many of these issues by allowing ML models to run on low-cost and low-power microcontrollers. + +> The "AI for Good" movement is critical in cultivating a future where an AI-empowered society is more just, sustainable, and prosperous for all humanity. + To give ourselves a framework around which to think about AI for social good, we will be following the UN Sustainable Development Goals (SDGs). The UN SDGs are a collection of 17 global goals, shown in @fig-sdg, adopted by the United Nations in 2015 as part of the 2030 Agenda for Sustainable Development. The SDGs address global challenges related to poverty, inequality, climate change, environmental degradation, prosperity, and peace and justice. ![United Nations Sustainable Development Goals (SDG). Source: [United Nations](https://sdgs.un.org/goals).](https://www.un.org/sustainabledevelopment/wp-content/uploads/2015/12/english_SDG_17goals_poster_all_languages_with_UN_emblem_1.png){#fig-sdg} diff --git a/contents/core/benchmarking/benchmarking.qmd b/contents/core/benchmarking/benchmarking.qmd index af5546e6..c631d540 100644 --- a/contents/core/benchmarking/benchmarking.qmd +++ b/contents/core/benchmarking/benchmarking.qmd @@ -10,9 +10,11 @@ Resources: [Slides](#sec-benchmarking-ai-resource), [Videos](#sec-benchmarking-a ![_DALL·E 3 Prompt: Photo of a podium set against a tech-themed backdrop. On each tier of the podium, there are AI chips with intricate designs. The top chip has a gold medal hanging from it, the second one has a silver medal, and the third has a bronze medal. Banners with 'AI Olympics' are displayed prominently in the background._](images/png/cover_ai_benchmarking.png) -Benchmarking is critical to developing and deploying machine learning systems, especially TinyML applications. Benchmarks allow developers to measure and compare the performance of different model architectures, training procedures, and deployment strategies. This provides key insights into which approaches work best for the problem at hand and the constraints of the deployment environment. +## Purpose {.unnumbered} -This chapter will provide an overview of popular ML benchmarks, best practices for benchmarking, and how to use benchmarks to improve model development and system performance. It provides developers with the right tools and knowledge to effectively benchmark and optimize their systems, especially for TinyML systems. +_How can quantitative evaluation reshape the development of machine learning systems, and what metrics reveal true system capabilities?_ + +The measurement and analysis of AI system performance represent a critical element in bridging theoretical capabilities with practical outcomes. Systematic evaluation approaches reveal fundamental relationships between model behavior, resource utilization, and operational reliability. These measurements draw out the essential trade-offs across accuracy, efficiency, and scalability, providing insights that guide architectural decisions throughout the development lifecycle. These evaluation frameworks establish core principles for assessing and validating system design choices and enable the creation of robust solutions that meet increasingly complex performance requirements across diverse deployment scenarios. ::: {.callout-tip} diff --git a/contents/core/data_engineering/data_engineering.qmd b/contents/core/data_engineering/data_engineering.qmd index 34666244..2388edfb 100644 --- a/contents/core/data_engineering/data_engineering.qmd +++ b/contents/core/data_engineering/data_engineering.qmd @@ -10,11 +10,13 @@ Resources: [Slides](#sec-data-engineering-resource), [Videos](#sec-data-engineer ![_DALL·E 3 Prompt: Create a rectangular illustration visualizing the concept of data engineering. Include elements such as raw data sources, data processing pipelines, storage systems, and refined datasets. Show how raw data is transformed through cleaning, processing, and storage to become valuable information that can be analyzed and used for decision-making._](images/png/cover_data_engineering.png) +Let me refine this to strengthen the academic tone and remove direct reader references while maintaining the focus on data engineering principles: + ## Purpose {.unnumbered} -_How does sourcing, preparation, and the quality of data influence the design, performance, and scalability of machine learning systems?_ +_Why does data infrastructure form the foundation of AI system success, and how does its design influence system capabilities?_ -Data is the lifeblood of AI systems. This chapter explores the engineering aspects of data sourcing, processing, and quality assurance, highlighting their impact on the scalability, efficiency, and adaptability of machine learning workflows. Building on earlier discussions about neural network architectures, this chapter addresses the design challenges of creating reliable data pipelines that integrate seamlessly with machine learning systems. The insights gained here will equip readers with the skills to engineer data solutions that support optimization and deployment, paving the way for advanced applications in complex environments. +At the heart of every AI system lies its data infrastructure, determining both its potential and limitations. While architectural patterns and workflows define the system's structure, data quality and accessibility ultimately govern its effectiveness. The engineering of data systems requires careful consideration of acquisition, processing, and delivery mechanisms, establishing the bedrock upon which sophisticated AI capabilities can be built. This perspective reveals how data engineering decisions ripple through every aspect of system performance and reliability, shaping the possibilities for model development, training efficiency, and deployment success. ::: {.callout-tip} diff --git a/contents/core/dl_architectures/dl_architectures.qmd b/contents/core/dl_architectures/dl_architectures.qmd index 16276936..21d1660e 100644 --- a/contents/core/dl_architectures/dl_architectures.qmd +++ b/contents/core/dl_architectures/dl_architectures.qmd @@ -12,12 +12,9 @@ Resources: [Slides](#sec-deep-learning-primer-resource), [Videos](#sec-deep-lear ## Purpose {.unnumbered} -_What are the modern deep learning architectures, what fundamental computational patterns underlie them, and how can these patterns be leveraged to build adaptable and efficient AI systems?_ - -This chapter focuses on real-world deep neural network (DNN) architectures, including convolutional networks (CNNs), recurrent networks (RNNs), and transformers, with a unique focus on the foundational computational patterns they share. Despite rapid advancements in AI, the core building blocks---or "LEGO pieces"---have remained relatively stable. Understanding these components is essential for constructing AI systems that support diverse model architectures while enabling efficient and scalable deployments. By bridging theoretical concepts with practical system design, this chapter provides a solid foundation for advancing machine learning systems and engineering. - -## Purpose {.unnumbered} +_What recurring patterns emerge across modern deep learning architectures, and how do these patterns enable systematic approaches to AI system design?_ +Deep learning architectures represent a convergence of computational patterns that form the building blocks of modern AI systems. Understanding these foundational patterns—from convolutional structures to attention mechanisms—reveals how complex models arise from simple, repeatable components. The examination of these architectural elements provides insights into the systematic construction of flexible, efficient AI systems, establishing core principles that influence every aspect of system design and deployment. These structural insights illuminate the path toward creating scalable, adaptable solutions across diverse application domains. ::: {.callout-tip} diff --git a/contents/core/dl_primer/dl_primer.qmd b/contents/core/dl_primer/dl_primer.qmd index e52c42d8..8be92bc2 100644 --- a/contents/core/dl_primer/dl_primer.qmd +++ b/contents/core/dl_primer/dl_primer.qmd @@ -12,9 +12,9 @@ Resources: [Slides](#sec-deep-learning-primer-resource), [Videos](#sec-deep-lear ## Purpose {.unnumbered} -_How can we understand neural networks as digital representations of biological concepts, and what implications do these artificial neural networks have on system design?_ +_What inspiration from nature drives the development of machine learning systems, and how do biological neural processes inform their fundamental design?_ -This chapter is an essential primer on artificial neural networks. It focuses on translating the biological inspirations underlying these models into their computational counterparts. The goal is to bridge the gap between the intuitive principles of biological neural systems and their digital implementations. Newcomers gain foundational insights into the challenges and considerations at the intersection of biology and computation, while seasoned practitioners deepen their understanding of the systemic implications of these digital constructs. In doing so, this chapter supports the exploration of advanced deep learning architectures and methodologies in subsequent sections. It lays the groundwork for the terminology and concepts that will be central throughout the book. +The neural systems of nature offer profound insights into information processing and adaptation, inspiring the core principles of modern machine learning. Translating biological mechanisms into computational frameworks illuminates fundamental patterns that shape artificial neural networks. These patterns reveal essential relationships between biological principles and their digital counterparts, establishing building blocks for understanding more complex architectures. Analyzing these mappings from natural to artificial provides critical insights into system design, laying the foundation for exploring advanced neural architectures and their practical implementations. ::: {.callout-tip} diff --git a/contents/core/efficient_ai/efficient_ai.qmd b/contents/core/efficient_ai/efficient_ai.qmd index 9d8645bc..3054678f 100644 --- a/contents/core/efficient_ai/efficient_ai.qmd +++ b/contents/core/efficient_ai/efficient_ai.qmd @@ -12,9 +12,9 @@ Resources: [Slides](#sec-efficient-ai-resource), [Videos](#sec-efficient-ai-reso ## Purpose {.unnumbered} -_Why is efficiency important in evaluating and deploying AI systems, and what general strategies do machine learning system engineers adopt to enhance performance, energy efficiency and resource utilization?_ +_What fundamental resource constraints challenge machine learning systems, and how do they shape architectural decisions?_ -This chapter provides an overview of the importance of efficiency in AI, focusing on model architectures, compression techniques, and inference hardware. It builds on previous discussions of AI workflows and data handling, highlighting how efficient numerics and formats enhance computational performance. This foundation helps readers appreciate the strategies essential for optimizing AI systems as they progress through the book. +The interplay between computational demands and system resources drives fundamental design choices in machine learning systems. Each efficiency decision---from arithmetic precision to memory access patterns---cascades through multiple layers of the system stack, revealing essential relationships between resource utilization and model capability. These relationships guide architectural choices that balance system performance against resource constraints, establishing core principles for designing AI solutions that achieve maximum impact with minimum resources. ::: {.callout-tip} diff --git a/contents/core/frameworks/frameworks.qmd b/contents/core/frameworks/frameworks.qmd index 8a2ffd5a..953cb619 100644 --- a/contents/core/frameworks/frameworks.qmd +++ b/contents/core/frameworks/frameworks.qmd @@ -10,11 +10,13 @@ Resources: [Slides](#sec-ai-frameworks-resource), [Videos](#sec-ai-frameworks-re ![_DALL·E 3 Prompt: Illustration in a rectangular format, designed for a professional textbook, where the content spans the entire width. The vibrant chart represents training and inference frameworks for ML. Icons for TensorFlow, Keras, PyTorch, ONNX, and TensorRT are spread out, filling the entire horizontal space, and aligned vertically. Each icon is accompanied by brief annotations detailing their features. The lively colors like blues, greens, and oranges highlight the icons and sections against a soft gradient background. The distinction between training and inference frameworks is accentuated through color-coded sections, with clean lines and modern typography maintaining clarity and focus._](images/png/cover_ml_frameworks.png) -## Purpose +Let me refine this to maintain a textbook tone, remove the "By" construction, and strengthen the systems perspective: -_What role do AI frameworks play in enabling scalable and efficient machine learning systems, and how do their features and capabilities influence the design, training, and deployment of models across diverse environments?_ +## Purpose {.unnumbered} -AI frameworks are the backbone of modern machine learning workflows, providing the tools and abstractions necessary to design, train, and deploy complex models across diverse environments. These frameworks bridge the gap between model development and system execution, allowing engineers to take models designed by developers and efficiently execute them on various underlying systems. By understanding the components, capabilities, and limitations of these frameworks, machine learning practitioners can make informed decisions about the tools that best enable scalability, efficiency, and adaptability. This knowledge prepares us to address the challenges of machine learning deployment, optimization, and system-level integration. +_How do AI frameworks bridge the gap between theoretical design and practical implementation, and what role do they play in enabling scalable machine learning systems?_ + +AI frameworks represent critical middleware that transform abstract model specifications into executable implementations. The evolution of these frameworks reveals fundamental patterns for translating high-level designs into efficient computational workflows. Their architecture illuminates essential trade-offs between abstraction, performance, and portability, providing systematic approaches to managing complexity in machine learning systems. Understanding framework capabilities and constraints offers crucial insights into the engineering decisions that shape system scalability, enabling the development of robust, deployable solutions across diverse computing environments. ::: {.callout-tip} diff --git a/contents/core/hw_acceleration/hw_acceleration.qmd b/contents/core/hw_acceleration/hw_acceleration.qmd index f247bcae..d160945d 100644 --- a/contents/core/hw_acceleration/hw_acceleration.qmd +++ b/contents/core/hw_acceleration/hw_acceleration.qmd @@ -10,7 +10,11 @@ Resources: [Slides](#sec-ai-acceleration-resource), [Videos](#sec-ai-acceleratio ![_DALL·E 3 Prompt: Create an intricate and colorful representation of a System on Chip (SoC) design in a rectangular format. Showcase a variety of specialized machine learning accelerators and chiplets, all integrated into the processor. Provide a detailed view inside the chip, highlighting the rapid movement of electrons. Each accelerator and chiplet should be designed to interact with neural network neurons, layers, and activations, emphasizing their processing speed. Depict the neural networks as a network of interconnected nodes, with vibrant data streams flowing between the accelerator pieces, showcasing the enhanced computation speed._](images/png/cover_ai_hardware.png) -Deploying ML on edge devices presents challenges such as limited processing speed, memory constraints, and stringent energy efficiency requirements. To overcome these challenges, specialized hardware acceleration is key. Hardware accelerators are designed to optimize compute-intensive tasks like inference by using custom silicon chips tailored for matrix multiplications, providing significant speedups compared to general-purpose CPUs. This enables real-time execution of advanced models on devices with strict constraints on size, weight, and power. +## Purpose {.unnumbered} + +_How does specialized computing transform the performance frontier of machine learning systems, and what principles guide the design of acceleration strategies?_ + +The evolution of computational acceleration in machine learning systems represents a fundamental shift in how processing resources are architected and utilized. Each acceleration approach introduces unique patterns for matching algorithmic structure with computational capabilities, revealing essential relationships between model design and execution efficiency. The integration of specialized computing elements demonstrates trade-offs between performance, power efficiency, and system complexity. These architectural interactions provide insights into system-level optimization strategies, establishing core principles for designing AI solutions that effectively leverage advanced computation across diverse deployment environments. ::: {.callout-tip} diff --git a/contents/core/ml_systems/ml_systems.qmd b/contents/core/ml_systems/ml_systems.qmd index 2630252b..926ca4c3 100644 --- a/contents/core/ml_systems/ml_systems.qmd +++ b/contents/core/ml_systems/ml_systems.qmd @@ -12,9 +12,9 @@ Resources: [Slides](#sec-ml-systems-resource), [Videos](#sec-ml-systems-resource ## Purpose {.unnumbered} -_Why are machine learning systems deployed so broadly across different environments ranging from cloud to edge, mobile, and tiny devices, and how does this influence the way we design and engineer these systems?_ +_How do the diverse environments where machine learning operates shape the fundamental nature of these systems, and what drives their widespread deployment across computing platforms?_ -The diversity of machine learning deployment environments offers a critical perspective on how theoretical concepts are implemented in practice. Ecosystems ranging from cloud infrastructures to edge solutions, mobile platforms, and tiny devices influence not only the design of machine learning systems but also the neural network algorithms that drive them. Understanding the constraints and opportunities within these environments highlights the practical considerations required to adapt algorithms for real-world applications. This chapter establishes a foundational framework for analyzing scalability, efficiency, and adaptability, preparing us to tackle the challenges of optimization, integration, and deployment in complex systems. +The deployment of machine learning systems across varied computing environments reveals essential insights into the relationship between theoretical principles and practical implementation. Each computing environment - from large-scale distributed systems to resource-constrained devices - introduces distinct requirements that influence both system architecture and algorithmic approaches. Understanding these relationships reveals core engineering principles that govern the design of machine learning systems. This understanding provides a foundation for examining how theoretical concepts translate into practical implementations, and how system designs adapt to meet diverse computational, memory, and energy constraints. :::{.callout-tip} diff --git a/contents/core/ondevice_learning/ondevice_learning.qmd b/contents/core/ondevice_learning/ondevice_learning.qmd index 09a7d192..fa9e8ed0 100644 --- a/contents/core/ondevice_learning/ondevice_learning.qmd +++ b/contents/core/ondevice_learning/ondevice_learning.qmd @@ -10,7 +10,11 @@ Resources: [Slides](#sec-on-device-learning-resource), [Videos](#sec-on-device-l ![_DALL·E 3 Prompt: Drawing of a smartphone with its internal components exposed, revealing diverse miniature engineers of different genders and skin tones actively working on the ML model. The engineers, including men, women, and non-binary individuals, are tuning parameters, repairing connections, and enhancing the network on the fly. Data flows into the ML model, being processed in real-time, and generating output inferences._](images/png/cover_ondevice_learning.png) -On-device Learning represents a significant innovation for embedded and edge IoT devices, enabling models to train and update directly on small local devices. This contrasts with traditional methods, where models are trained on expansive cloud computing resources before deployment. With on-device learning, devices like smart speakers, wearables, and industrial sensors can refine models in real-time based on local data without needing to transmit data externally. For example, a voice-enabled smart speaker could learn and adapt to its owner's speech patterns and vocabulary right on the device. However, there is no such thing as a free lunch; therefore, in this chapter, we will discuss both the benefits and the limitations of on-device learning. +## Purpose {.unnumbered} + +_How does learning at the edge transform traditional machine learning paradigms, and what principles enable effective adaptation in resource-constrained environments?_ + +The migration of learning capabilities to edge devices represents a fundamental shift in how AI systems evolve and adapt. Local learning introduces unique patterns for balancing model adaptation with resource limitations, revealing essential relationships between computational constraints and system autonomy. The implementation of on-device training emphasizes the trade-offs between learning capability, energy efficiency, and operational independence. These adaptation mechanisms provide insights into designing self-evolving systems, establishing core principles for creating AI solutions that can learn and improve within the constraints of local computing environments. ::: {.callout-tip} diff --git a/contents/core/ops/ops.qmd b/contents/core/ops/ops.qmd index bdceaf3f..89b74e44 100644 --- a/contents/core/ops/ops.qmd +++ b/contents/core/ops/ops.qmd @@ -10,7 +10,11 @@ Resources: [Slides](#sec-embedded-aiops-resource), [Videos](#sec-embedded-aiops- ![_DALL·E 3 Prompt: Create a detailed, wide rectangular illustration of an AI workflow. The image should showcase the process across six stages, with a flow from left to right: 1. Data collection, with diverse individuals of different genders and descents using a variety of devices like laptops, smartphones, and sensors to gather data. 2. Data processing, displaying a data center with active servers and databases with glowing lights. 3. Model training, represented by a computer screen with code, neural network diagrams, and progress indicators. 4. Model evaluation, featuring people examining data analytics on large monitors. 5. Deployment, where the AI is integrated into robotics, mobile apps, and industrial equipment. 6. Monitoring, showing professionals tracking AI performance metrics on dashboards to check for accuracy and concept drift over time. Each stage should be distinctly marked and the style should be clean, sleek, and modern with a dynamic and informative color scheme._](images/png/cover_ml_ops.png) -In this chapter, we will dive into the practices and frameworks needed to successfully develop, deploy, and manage machine learning models from start to finish. You will learn about each stage in the ML workflow, from data collection and model training to evaluation, deployment, and ongoing monitoring. We will discuss the role of automation, collaboration, and continuous improvement, highlighting why they are essential for keeping ML systems efficient and reliable. We will also explore different deployment environments, from powerful cloud servers to resource-limited edge devices, looking at the unique challenges each presents. Through concrete examples, you will see how to design and operate ML systems that deliver consistent, reliable performance, no matter where they are deployed. By the end of this chapter, you will have a solid grasp of ML model management and be ready to build and maintain ML applications that provide lasting value. +## Purpose {.unnumbered} + +_What systematic principles enable continuous evolution of machine learning systems in production, and how do operational requirements shape deployment architecture?_ + +The transition from experimental to production environments represents a fundamental challenge in scaling machine learning systems. Operational practices reveal essential patterns for maintaining reliability and adaptability throughout the system lifecycle, highlighting critical relationships between deployment architecture and continuous delivery. The implementation of MLOps frameworks draws out the key trade-offs between automation, monitoring, and governance that shape system evolution. Understanding these operational dynamics provides insights into managing complex AI workflows, establishing core principles for creating systems that remain robust and responsive in dynamic production environments. ::: {.callout-tip} diff --git a/contents/core/optimizations/optimizations.qmd b/contents/core/optimizations/optimizations.qmd index e9b721eb..1cd511f7 100644 --- a/contents/core/optimizations/optimizations.qmd +++ b/contents/core/optimizations/optimizations.qmd @@ -10,7 +10,11 @@ Resources: [Slides](#sec-model-optimizations-resource), [Videos](#sec-model-opti ![_DALL·E 3 Prompt: Illustration of a neural network model represented as a busy construction site, with a diverse group of construction workers, both male and female, of various ethnicities, labeled as 'pruning', 'quantization', and 'sparsity'. They are working together to make the neural network more efficient and smaller, while maintaining high accuracy. The 'pruning' worker, a Hispanic female, is cutting unnecessary connections from the middle of the network. The 'quantization' worker, a Caucasian male, is adjusting or tweaking the weights all over the place. The 'sparsity' worker, an African female, is removing unnecessary nodes to shrink the model. Construction trucks and cranes are in the background, assisting the workers in their tasks. The neural network is visually transforming from a complex and large structure to a more streamlined and smaller one._](images/png/cover_model_optimizations.png) -When machine learning models are deployed on systems, especially on resource-constrained embedded systems, the optimization of models is a necessity. While machine learning inherently often demands substantial computational resources, the systems are inherently limited in memory, processing power, and energy. This chapter will dive into the art and science of optimizing machine learning models to ensure they are lightweight, efficient, and effective when deployed in TinyML scenarios. +## Purpose {.unnumbered} + +_How do theoretical neural network models transform into practical solutions, and what techniques bridge implementation gaps?_ + +The adaptation of machine learning models for real-world deployment demands systematic transformation approaches. Each optimization technique introduces unique methods for preserving model capabilities while meeting deployment constraints, revealing fundamental patterns in the relationship between theoretical design and practical implementation. These patterns highlight strategies for translating research advances into production systems, establishing core principles for creating solutions that maintain effectiveness while satisfying real-world requirements. ::: {.callout-tip} diff --git a/contents/core/privacy_security/privacy_security.qmd b/contents/core/privacy_security/privacy_security.qmd index e0c9d28f..11045fad 100644 --- a/contents/core/privacy_security/privacy_security.qmd +++ b/contents/core/privacy_security/privacy_security.qmd @@ -10,7 +10,11 @@ Resources: [Slides](#sec-security-and-privacy-resource), [Videos](#sec-security ![_DALL·E 3 Prompt: An illustration on privacy and security in machine learning systems. The image shows a digital landscape with a network of interconnected nodes and data streams, symbolizing machine learning algorithms. In the foreground, there's a large lock superimposed over the network, representing privacy and security. The lock is semi-transparent, allowing the underlying network to be partially visible. The background features binary code and digital encryption symbols, emphasizing the theme of cybersecurity. The color scheme is a mix of blues, greens, and grays, suggesting a high-tech, digital environment._](images/png/cover_security_privacy.png) -Security and privacy are critical when developing real-world machine learning systems. As machine learning is increasingly applied to sensitive domains like healthcare, finance, and personal data, protecting confidentiality and preventing misuse of data and models becomes imperative. Anyone aiming to build robust and responsible ML systems must grasp potential security and privacy risks such as data leaks, model theft, adversarial attacks, bias, and unintended access to private information. We also need to understand best practices for mitigating these risks. Most importantly, security and privacy cannot be an afterthought and must be proactively addressed throughout the ML system development lifecycle - from data collection and labeling to model training, evaluation, and deployment. Embedding security and privacy considerations into each stage of building, deploying, and managing machine learning systems is essential for safely unlocking the benefits of AI. +## Purpose {.unnumbered} + +_What principles guide the protection of machine learning systems, and how do security and privacy requirements transform system architecture?_ + +The integration of protection mechanisms into AI systems represents a fundamental dimension of modern system design. Security considerations reveal essential patterns for safeguarding data, models, and infrastructure while maintaining operational effectiveness. The implementation of defensive strategies brings to light the trade-offs between protection, performance, and usability that shape architectural decisions across the AI lifecycle. Understanding these security dynamics provides insights into creating trustworthy systems, establishing core principles for designing solutions that preserve privacy and resist adversarial threats while meeting functional requirements in production environments. ::: {.callout-tip} diff --git a/contents/core/responsible_ai/responsible_ai.qmd b/contents/core/responsible_ai/responsible_ai.qmd index b85e5ee6..7a0c9cfc 100644 --- a/contents/core/responsible_ai/responsible_ai.qmd +++ b/contents/core/responsible_ai/responsible_ai.qmd @@ -10,7 +10,11 @@ Resources: [Slides](#sec-responsible-ai-resource), [Videos](#sec-responsible-ai- ![_DALL·E 3 Prompt: Illustration of responsible AI in a futuristic setting with the universe in the backdrop: A human hand or hands nurturing a seedling that grows into an AI tree, symbolizing a neural network. The tree has digital branches and leaves, resembling a neural network, to represent the interconnected nature of AI. The background depicts a future universe where humans and animals with general intelligence collaborate harmoniously. The scene captures the initial nurturing of the AI as a seedling, emphasizing the ethical development of AI technology in harmony with humanity and the universe._](images/png/cover_responsible_ai.png) -As machine learning models grow across various domains, these algorithms have the potential to perpetuate historical biases, breach privacy, or enable unethical automated decisions if developed without thoughtful consideration of their societal impacts. Even systems created with good intentions can ultimately discriminate against certain demographic groups, enable surveillance, or lack transparency into their behaviors and decision-making processes. As such, machine learning engineers and companies have an ethical responsibility to proactively ensure principles of fairness, accountability, safety, and transparency are reflected in their models to prevent harm and build public trust. +## Purpose {.unnumbered} + +_How do human values translate into system architecture, and what principles enable the development of AI systems that embody ethical considerations?_ + +Embedding ethical principles into machine learning systems represents a fundamental engineering challenge. Each design decision carries moral implications, revealing essential patterns in how technical choices influence societal outcomes. The implementation of ethical frameworks demonstrates the need for new approaches to system architecture that consider human values as fundamental design constraints. Understanding these moral-technical relationships provides insights into creating principled systems, establishing core methodologies for designing AI solutions that advance capabilities while upholding human values. ::: {.callout-tip} diff --git a/contents/core/robust_ai/robust_ai.qmd b/contents/core/robust_ai/robust_ai.qmd index 91b24604..ec0d397f 100644 --- a/contents/core/robust_ai/robust_ai.qmd +++ b/contents/core/robust_ai/robust_ai.qmd @@ -10,11 +10,11 @@ Resources: [Slides](#sec-robust-ai-resource), [Videos](#sec-robust-ai-resource), ![_DALL·E 3 Prompt: Create an image featuring an advanced AI system symbolized by an intricate, glowing neural network, deeply nested within a series of progressively larger and more fortified shields. Each shield layer represents a layer of defense, showcasing the system's robustness against external threats and internal errors. The neural network, at the heart of this fortress of shields, radiates with connections that signify the AI's capacity for learning and adaptation. This visual metaphor emphasizes not only the technological sophistication of the AI but also its resilience and security, set against the backdrop of a state-of-the-art, secure server room filled with the latest in technological advancements. The image aims to convey the concept of ultimate protection and resilience in the field of artificial intelligence._](./images/png/cover_robust_ai.png) -The development of robust machine learning systems has become increasingly crucial. As these systems are deployed in various critical applications, from autonomous vehicles to healthcare diagnostics, ensuring their resilience to faults and errors is paramount. +## Purpose {.unnumbered} -Robust AI, in the context of hardware faults, software faults, and errors, plays an important role in maintaining the reliability, safety, and performance of machine learning systems. By addressing the challenges posed by transient, permanent, and intermittent hardware faults [@ahmadilivani2024systematic], as well as bugs, design flaws, and implementation errors in software [@zhang2008distribution], robust AI techniques enable machine learning systems to operate effectively even in adverse conditions. +_How do uncertainty and variability shape machine learning system design, and what principles enable reliable operation in challenging conditions?_ -This chapter explores the fundamental concepts, techniques, and tools for building fault-tolerant and error-resilient machine learning systems. It empowers researchers and practitioners to develop AI solutions that can withstand the complexities and uncertainties of real-world environments. +The exposure of AI systems to real-world conditions presents fundamental challenges in maintaining consistent performance. Operational patterns reveal essential relationships between system stability and environmental variability, highlighting critical trade-offs between resilience and efficiency. The implementation of robust architectures emphasizes the need for strategies to maintain reliability across diverse and unpredictable scenarios while preserving core functionality. Understanding these resilience dynamics provides insights into creating dependable systems, establishing principles for designing AI solutions that maintain effectiveness even when faced with distributional shifts, noise, and adversarial conditions. ::: {.callout-tip} @@ -34,7 +34,7 @@ This chapter explores the fundamental concepts, techniques, and tools for buildi ## Overview -Robust AI refers to a system's ability to maintain its performance and reliability in the presence of errors. A robust machine learning system is designed to be fault-tolerant and error-resilient, capable of operating effectively even under adverse conditions. +Robust AI refers to a system's ability to maintain its performance and reliability in the presence of errors. A robust machine learning system is designed to be fault-tolerant and error-resilient, capable of operating effectively even under adverse conditions. As ML systems become increasingly integrated into various aspects of our lives, from cloud-based services to edge devices and embedded systems, the impact of hardware and software faults on their performance and reliability becomes more significant. In the future, as ML systems become more complex and are deployed in even more critical applications, the need for robust and fault-tolerant designs will be paramount. diff --git a/contents/core/sustainable_ai/sustainable_ai.qmd b/contents/core/sustainable_ai/sustainable_ai.qmd index 70e1563f..00f18bde 100644 --- a/contents/core/sustainable_ai/sustainable_ai.qmd +++ b/contents/core/sustainable_ai/sustainable_ai.qmd @@ -10,6 +10,12 @@ Resources: [Slides](#sec-sustainable-ai-resource), [Videos](#sec-sustainable-ai- ![_DALL·E 3 Prompt: 3D illustration on a light background of a sustainable AI network interconnected with a myriad of eco-friendly energy sources. The AI actively manages and optimizes its energy from sources like solar arrays, wind turbines, and hydro dams, emphasizing power efficiency and performance. Deep neural networks spread throughout, receiving energy from these sustainable resources._](images/png/cover_sustainable_ai.png) +## Purpose {.unnumbered} + +_What principles emerge when we consider machine learning systems through an ecological lens, and how does environmental stewardship reshape system architecture?_ + +The ecological footprint of AI computation presents a fundamental challenge to technological advancement. Each design decision carries environmental implications, revealing essential tensions between computational capability and ecological responsibility. The pursuit of sustainable practices illuminates new approaches to system architecture that consider environmental impact as a primary design constraint. Understanding these ecological relationships provides insights into creating next-generation systems, establishing core principles for designing AI solutions that advance technology while preserving planetary resources. + ::: {.callout-tip} ## Learning Objectives diff --git a/contents/core/training/training.qmd b/contents/core/training/training.qmd index 30ffdc38..5af40b7e 100644 --- a/contents/core/training/training.qmd +++ b/contents/core/training/training.qmd @@ -10,11 +10,11 @@ Resources: [Slides](#sec-ai-training-resource), [Videos](#sec-ai-frameworks-reso ![_DALL·E 3 Prompt: An illustration for AI training, depicting a neural network with neurons that are being repaired and firing. The scene includes a vast network of neurons, each glowing and firing to represent activity and learning. Among these neurons, small figures resembling engineers and scientists are actively working, repairing and tweaking the neurons. These miniature workers symbolize the process of training the network, adjusting weights and biases to achieve convergence. The entire scene is a visual metaphor for the intricate and collaborative effort involved in AI training, with the workers representing the continuous optimization and learning within a neural network. The background is a complex array of interconnected neurons, creating a sense of depth and complexity._](images/png/ai_training.png) -## Purpose +## Purpose {.unnumbered} -_What are the foundational principles and key techniques that enable the effective training of machine learning models, and how do system-level considerations influence this process?_ +_What essential principles guide the training of machine learning systems, and how do computational environments shape training strategies?_ -Training is the cornerstone of machine learning, as it transforms data into actionable models through iterative optimization and careful tuning. This chapter focuses on the essential components of training workflows while addressing common pitfalls and challenges. Beyond theoretical principles, we focus on several system-level factors such as resource constraints, parallelization, and bottlenecks that significantly impact training efficiency and scalability. This chapter equips us with the knowledge to design robust training pipelines, while ensuring effective model performance in real-world applications. +The transformation of data into actionable models through training represents a fundamental challenge in machine learning systems. Training architectures emphasize the delicate balance between algorithmic optimization and computational resource management, illuminating critical relationships between data movement, model convergence, and system performance. These relationships shape core engineering decisions regarding distributed computation, memory hierarchy, and processing efficiency. Understanding these training dynamics offers insights into the practical constraints that impact model development, establishing key principles for designing robust training pipelines that can scale across various computational environments. ::: {.callout-tip} diff --git a/contents/core/workflow/workflow.qmd b/contents/core/workflow/workflow.qmd index e0607827..fe2ac505 100644 --- a/contents/core/workflow/workflow.qmd +++ b/contents/core/workflow/workflow.qmd @@ -12,9 +12,9 @@ Resources: [Slides](#sec-ai-workflow-resource), [Videos](#sec-ai-workflow-resour ## Purpose {.unnumbered} -_AI sytems are more than just machine learning models, so how can we systematically and more holistically approach AI systems engineering?_ +_Beyond individual components, how do the diverse elements of AI systems combine to create effective machine learning system solutions?_ -This chapter highlights the broader scope of machine learning beyond understanding neural network model architectures. It introduces readers to a comprehensive AI workflow, encompassing stages from data gathering to operational deployment. The focus extends to critical factors such as resource management, data privacy, and integration processes. By laying this foundation, the chapter serves as a preview of the detailed engineering and operational concepts that follow, guiding both newcomers and seasoned practitioners toward a holistic understanding of AI systems in practice. +The creation of practical AI solutions requires the orchestration of multiple system components into coherent workflows. Architectural patterns serve as building blocks, while workflow design highlights the connections and interactions that animate these components. This systematic perspective reveals how data flow, model training, and deployment considerations are intertwined to form robust AI systems. Analyzing these interconnections offers crucial insights into system-level design choices, establishing a framework for understanding how theoretical concepts can be translated into deployable solutions that meet real-world needs. ::: {.callout-tip}