Architecture
#Software for Open Networking in the Cloud (SONiC) Architecture
Software for Open Networking in the Cloud (SONiC) is a collection of software packages installed on Linux running on a network hardware switch which make it a complete, functional router targeted at data center networks.
The following picture presents the target SONiC Architecture.
- Fans
- Power supplies
- Media/Transceivers
The right hand side of the picture focuses on the networking components including:
- SAI
- ASIC Control Software
Figure 1 – Target SONiC High Level Software Design
The current SONiC project implements Switch State Service (SWSS) as it’s ASIC control software.
Switch hardware refers to all the physical components inside the network switch enclosure (chassis). This includes fans, power supplies, status LEDs and network transceivers. In SONiC terminology, these are called “system devices”.
The switch abstraction interface is a standardized C API to the switching ASIC. This API is normally implemented by an SDK specific to the Switch ASIC. More information on SAI is available at the SAI GitHub repository.
The SAI Host Adapter’s role is to provide a mechanism for storing and synchronizing network switch data with the Switch ASIC. This includes initialization, configuration and current status of the switch ASIC.
Network applications, such as a BGP routing protocol, use the Object Libraries API’s to get and set the state of the SONiC SAI Host Adapter.
A high level description of the Object library follows.
The SONiC Object Library (Object Library) mediates interactions between SONiC applications and external applications. The Object Library defines two types of application roles: clients and servers. Client applications execute create, set, get, and delete operations on objects. Server applications execute operations requested by clients.
In addition, the Object Library implements a publish/subscribe model. Server applications publish events; client applications subscribe to events and objects. Client applications can subscribe to events generated when objects are created, modified, or deleted.
The publish/subscribe approach and object-centric operations allow for independent operation of client and server applications. A key tenant of the SONiC architecture is to allows processes to be stopped, started, restarted and replaced.
The Switch State Service (SwSS) is a collection of software that provides a database interface for communication with and state representation of network applications and network switch hardware.
Figure 3 – Switch State Service High Level Design
Network applications read and write to APP_DB. Example applications include a netlink route syncer, quagga FPM route syncer, access control list (ACL) control, QoS control, load balancer, telemetry control and so on.
Orchestration agents read and write data between APP and ASIC databases. Orchestration agents are responsible for any necessary logic to verify and transform application data into SAI objects.
The syncd process reads and writes SAI objects between the ASIC_DB and the SAI SDK.
A key value database was chosen to provide a language independent interface, a method for data persistence, replication and multi-process communication. An API wrapper is implemented in swss/common which implements transactions, convenience methods and allows the database storage engine to be changed in the future if necessary. Redis was chosen as the underlying database engine, but this could be changed in the future.
Using the SwSS API, SONiC network applications are written to be entirely independent of the lower layer communication details to the hardware. Applications subscribe only to the data views they require and avoid implementation details that are not specific for their functionality. Examples of applications that intended to interface with SwSS include: Layer 3 routing, Layer 2 bridging, Access control lists (packet filtering), Quality of service, Telemetry streaming, tunneling, link aggregation, load balancing and policy based routing to name a few.
This process contains logic for transforming and copying data between the APP tables and the ASIC tables.
There must only be one producer for each ASIC table. Currently there is just one orchestration agent, although others could be added over time.
Only a single Orchestration Agent may write to an ASIC_DB table.
The switch sync daemon syncd copies data between the ASIC_DB tables and a SAI compliant ASIC SDK. There must only be one syncd process per SAI SDK instance.
SwSS implements the concept of a table in redis by naming keys with prefixes. A producer / consumer design is implemented to ensure integrity of data.
APP_ tables are designed for each use case. For example, ROUTE_TABLE and NEIGH_TABLE.
ASIC_ tables are created from the SAI header files. For example ASIC_sai_unicast_route_entry_t and ASIC_sai_neighbor_entry_t.
[TODO: link to github, .h files, API’s for common table operations]
Producer
SET – insert or update a key -> fields and values.
DEL – deletes a key
Consumer
POP – get a table change notification, the key name and the key->fields and values and operation [SET, DEL].
SELECT – check if a table notification exists.
SwSS implements transactions internally so producers and consumers can to stay in sync with the database using a queue-like method.
For each ‘TABLE’, there are QUEUE keys used for internal implementation of notifications. Here is an example of how it works.
The intfsyncd process performs a SET to the APP.INTF_TABLE using the swss producer API. The producer API SETs the key/value in the TABLE and SETs an equivalent entry for each of the QUEUE keys.
The orchestration agent (OA) is a CONSUMER of the APP.INTF_TABLE. OA will receive a notification from the swss consumer API that there is a data change on APP.INTF_TABLE. The consumer API will POP the KEY, VALUE and OP from the QUEUE keys. The data the intfsyncd wrote to the APP.INTF_TABLE remains untouched.
See the code for more details.
Tablename+”_KEY_QUEUE”
Tablename+”_VALUE_QUEUE”
Tablename+”_OP_QUEUE”
Two databases are defined, APP and ASIC. Applications outside of SwSS are expected to store data by adding keys with well-defined names into the APP database. The ASIC database stores data used by hardware sync agents. Keys in the ASIC database are expected named strictly following SAI attributes.
Keys must be prefixed with a string that looks like a table name. The allowed keys are “[a-z][A-Z][0-9]_” and end with “_TABLE:”
In redis, databases are only defined my numbers:
Database 0 = APP_DB
Database 1 = ASIC_DB
Database 7 = TEST (used for unit testing)
The SwSS schema is defined at: https://github.com/Azure/sonic-swss/blob/master/doc/swss-schema.md
The ASIC database stores data used by hardware sync agents. Keys in the ASIC database are named strictly following SAI attributes. See https://github.com/opencomputeproject/SAI
This section provides an example of how a BGP route is learned and propagated to the ASIC. Quagga is used as an example, but other routing applications could be used.
Figure 4 – Learn bgp route
Platform management service manages the peripheral devices such as Fan, PDU, LED and Transceivers. Common switch platform leverages I2C bus and usually has one or more I2C masters to manage these peripheral devices. These devices are usually connected to I2C master via a series of I2C muxes.
Linux kernel has very mature I2C subsystem and well-defined API. It supports almost all commonly used I2C devices (master, muxes, sensors). SONiC then leverages the Linux kernel to expose interfaces to manage these devices. In this approach, SONiC first installs kernel drivers for these I2C devices, use instantiate those devices to build the I2C tree topology, and expose the sysfs interface such as hwmon to manage those devices.
SONiC provide a few services and utilities to manage these devices
- decode-syseeprom: show system eeprom information
- sensors: monitor temperature and fan speed
- fancontrol: adjust fan speed based on the sensor temperature
- ledd: change LED lights based on the link or system status
- sfputil: manage transceivers such as show transceiver eeprom, detect presence, reset, change lpmode
The platform supplier is expected develop the their platform modules to build the I2C tree, expose the sysfs interface and develop plugins to hook their devices to SONiC platform management service. Detailed information can be found at the Platform Porting Guide.