List of major Network Operating Systems (NOS)

Cumulus Linux: is an open source Linux based NOS. It’s used for white box or bare metal or open switches. The Linux distribution used in Cumulus Linux is based on Debian & uses the full TCP/IP stack of Linux. So at its core it’s a Linux OS customised to run as NOS on any x86 or commodity hardware based switches. At the moment, there are 130+ hardware vendors which support cumulus Linux as NOS. We can use almost all the Linux commands, tools, utilities, applications, management & automation techniques on network switches running on cumulus Linux as we do for Linux based servers. This familiarity and similarity allows for significant savings in CapEX and OpEx.  Cumulus sells only software (mainly cumulus Linux as NOS) and no hardware at all. Though they collaborate with multiple hardware vendors like Dell & HP. Cumulus business model is very similar to Red Hat (for RHEL or Red Hat Enterprise Linux). Cumulus Linux costs much less than proprietary NOS but is NOT Free. On 4th May 2020, graphics processor (GPU) and AI giant Nvidia has acquired Cumulus Networks. Just about a week back, in April 2020, Nvidia completed $7 billion acquisition of another company called Mellanox which is into DC hardware business, including DC Ethernet open switches. Mellanox switches already ship with Cumulus Linux as NOS. With the acquisition of Mellanox and now Cumulus networks, Nvidia aims to get a strong foothold in next-gen DC computing business.

ONIE: Open Network Install Environment. It’s not really a full-fledged NOS as such. Instead ONIE is kind of a mini OS (boot loader + bare bone Linux kernel with busybox utilities) which is just enough to boot any bare metal or open switch. It’s very similar to BIOS and PXE (Preboot Execution Environment) which allows a workstation or PC to boot from a server on a network prior to booting the OS on the local hard drive. ONIE runs in management subsystem* of a switch. For main switch OS or NOS functionality i.e. for switching sub-system, one need to install any full-fledged NOS of on top. So ONIE just provides the “install environment” i.e. a booted switch environment for the user to install any NOS of choice. ONIE’s main purpose is for the switch h/w manufacturer to ship out a bare metal/white-box switch with a bare min. bootable OS and then leave it for the purchaser to pick & install any NOS of his choice for providing main switching functionalities. ONIE was developed by Cumulus and later in 2013 they donated it to the open community. It’s now a part of Open Compute Project (OCP)**.

ONL: Open Network Linux is an open source Linux distribution for bare metal switches. ONL was initially developed by BigSwitch Networks but is now open source and part of Open Compute Project (OCP). It uses ONIE for installation on devices. ONL follows the modular architecture of NG-NOS *** and provides only the base NOS or lower platform layer for switch hardware management. An Open switch then needs an upper layer of NOS as forwarding agent. Anyone (typically service providers or big enterprises) can develop their own  forwarding engine. Some example forwarding agents are – Facebook’s FBOSS, MS Azure SONiC, BigSwitch Network’s SwitchLight OS and ONF’s open source Stratum. ONL is Openflow centric.

OPX: OpenSwitch, abbreviated as OPX, is a Linux based open source NOS with unmodified Linux kernel. It also utilizes ONIL as the bootloader to install on Open hardware switches. OPX runs on the latest stable Debian distro.

DNOS: Dell Network OS runs on Dell switches. DNOS is evolved from PowerConnect OS and FTOS (Force10 OS). PowerConnect OS was Dells original NOS for Dell’s campus switch platform whereas FTOS was the OS developed by a company called Force10 which Dell acquired in 2011. FTOS was aimed for high end DC switching platform & is very similar to Cisco NX-OS and Juniper JUNOS. In 2014, Dell decided to open their networking hardware to 3rd party NOS as well. Dell is the 1st major networking hardware vendor in the industry that allowed this. So most of the Dell networking hardware can now also support and run on any open NOS like Cumulus Linux, SwitchLight OS etc.

Switch Light OS: An SDN centric NOS for open switches based on Open Network Linux (ONL) by a company called BigSwitch Networks. It’s a Cloud and DC networking company which is acquired by Arista networks in Feb 2020. Switch Light OS is for bare metal/physical switches. There is another equivalent software agent from BigSwitch Networks for KVM/Hypervisor based virtual switches & is called Switch Light VX.

Junos OS: A single NOS by Juniper which runs on all of its networking hardware platforms. Junos kernel is based on FreeBSD Unix OS. Architecturally Junos is divided into three planes – control plane which runs on routing engine (RE), forwarding/data plane or packet forwarding engine (PFE) and services plane. Separating service plane allows Junos to support wide variety of services in different kinds of platforms.

Nokia SROS: Service Router OS is the NOS developed by Alcatel Lucent for their Service Routers, mainly 7750. Alcatel Lucent was acquired by Nokia in 2016 so it has now become Nokia SROS

Arista EOS: Arista Extensible OS (EOS) is unmodified Linux kernel based NOS that is fully programmable and highly modular. It is programmable at all layers – Linux kernel (unrestricted access), hardware forwarding tables, switch config and CLI, switch control plane as well as management layer. At the core of Arista EOS is Sysdb, an in-memory DB that contains the complete state of the system. It’s used to coordinate system control and maintains state separation amongst multiple processes – EOS internal processes as well as user installed applications. This highly programmable access to EOS allows users to develop and install custom applications without disrupting the core NOS system.

FBOSS: Facebook Open Switching System is the NOS developed by Facebook to run on its DC switches which are also designed & developed by Facebook and named as “Wedge” switches. Facebook follows the dis-aggregated model by separating network hardware & software. FBOSS is not really a typical NOS or even an OS. Instead it’s a group of applications that are installed on top of a standard Linux OS that gives it switching functionality yet feel like a normal compute hardware or server. FBOSS is not tied to any particular Linux distro. There is no native CLI to FBOSS. Instead configuration, monitoring and even troubleshooting is done via APIs using controllers or other automation tools. This means FBOSS is designed to be managed by software and not human operators. Both the Facebook NOS FBOSS and switch hardware Wedge are now open projects available in public domain. FBOSS is available on GitHub and Wedge switch specifications are part of Open Compute Project (OCP).

SONiC:  Software for Open Networking in the Cloud or SONiC is Microsoft Azure developed open source cloud scale NOS based on Linux. This is the NOS that runs on Azure cloud infrastructure switches. It’s a modular and extensible NOS where various applications or functions like BGP are containerized. Extensibility allows various hardware vendors like Dell to develop and plug in new components and tailor SONiC to their specific scenarios. Containerization is the key enabler for serviceability and scalability. Bug fixes can be done for flawed container without impacting other services/containers. 

HP ComWare:  It runs on HP’s chassis-based DC switches (some access switches as well) and WAN routers. In 2009, Cisco moved into server space with UCS product range. That same year HP expanded their networking market business by acquiring 3Com. Soon after that HP merged their old HP ProCurve OS with 3Com Comware & the new OS became HP Comware.

HP ProVision:  OS for rest of HP’s switching platform including HP ProCurve network hardware. Mostly focussed on LAN Switching and very popular in campus networks.

ArubaOS: Network OS for Aruba Wireless LAN products like Aruba Access Points (AP’s) & Wireless LAN Controllers (WLC’s). Aruba networks was acquired by HP in 2015 and is now a HP subsidiary company focussing on wireless network products. Some wireless related features code of HP ProVIsion was merged with AurabaOS.

ArubaOS-CX: Network OS for Aruba wired LAN products like Aruba edge, aggregation & core switches e.g. Aruba 8400 series switches. It’s a programmable, cloud-native, database driven, Linux kernel based NOS.

Cisco IOS:

• Cisco Internetwork Operating System (IOS) is Cisco’s first NOS developed mainly for routers & now runs on a number of Cisco hardware platforms
• Monolithic Architecture i.e. OS running directly on the hardware. So it’s one single piece of software – platform OS + IOS

Cisco IOS-XE:

• Underlying platform OS is Linux & IOS running as a separate process/daemon on top of Linux kernel
• Distributed s/w architecture that moves many OS responsibilities out of the IOS process and allows feature components e.g. BGP to run as subsystems.
• Provides open, standards-based programmable interfaces for network automation like NETCONF, RESTCONF & REST API
• Supports Linux Containers (LXC) and Virtual Machines (VM) to be securely hosted on the network device e.g. we can run Wireshark on the router/switch along with IOS-XE
• Designed for Enterprise networks including routing, switching, wired & wireless access devices
• Introduced by Cisco with the ASR 1000 series routers and now runs on mainly on Cisco ASR, ISR, Cat9K product family & CSR1000v
• All CLI commands are the same between Cisco IOS and IOS XE

Cisco IOS-XR:

• Completely different code base & CLI command set than IOS & IOS-XE. Built on Wind River Linux, a package based distributed software architecture.
• So instead of one single IOS image over Linux, its collection of micro services / software for different features which can be installed, removed & updated independently
• Allows in-life feature installs & updates. Underlying Linux environment is opened up for direct access to end users for automation and support for containers (LXC & Docker)
• Supports programmable interfaces like NetConf, gRPC & APIs at every layer of the network stack
• Intended for Service Provider or SP networks. Very similar to Juniper architecture which also targets the same market segment
• Introduced with the Cisco CRS-1 (Carrier Routing System) router in 2004. Runs on carrier-grade high end routing platforms like CRS-1, 8K, 12K ASR9K & NCS

NX-OS:

• NOS for the Nexus Ethernet switches and Multi-layer Director Switch or MDS-series Fibre Channel SAN (Storage Area Network) switches made by Cisco
• Open and modular NOS based on Linux kernel. It evolved from the Cisco SAN-OS, originally developed for Cisco MDS switches
• Provides full access to the platform allowing users to programmatically provision and configure switches through comprehensive APIs
• Nexus switches can work in two modes – NX-OS mode & ACI mode. When in NX-OS mode, the switch acts like a standalone Ethernet switch. In ACI mode, it acts as node of a ACI cluster (Spine/leaf) and is controlled by ACI controller
• CLI of NX-OS is similar to that of Cisco IOS

[ ** In a switch, there are two independent sub-systems – management subsystem and switching subsystem. Management subsystem controls main CPU, storage, bootflash, console and management Ethernet port whereas switching subsystem controls all of the forwarding or data plane i.e. Ethernet ports and associated switching ASIC’s. Management subsystem is basically device management and programming software whereas switching subsystem is forwarding stack of the switch.]

[*** ONIE is part of Open Compute Networking Project within OCP. In server infrastructure space, open ecosystem exists for years i.e. anyone can build servers by mixing and matching components from various suppliers and then run their choice of OS on that server.  There are multiple vendors in the market for each individual component. However as long as they comply with the pre-agreed standards and specification (which are developed by OCP), they fit together nicely to provide greater choice, customization, flexibility and cost savings to customers. This same concept for the networking world now, to break open the black box of proprietary networking gear, is called “open networking”. Its main purpose is to “dis-aggregate” hardware and software or the abstraction of the NOS from the hardware layer. In this dis-aggregated model, ONIE acts as the glue which binds open switch hardware to an open NOS.]

[*** NG-NOS or Next Gen Network Operating System is a NOS which has a modular architecture and is developed for open networking hardware like open switches for Software Defined Networks (SDN). In its modular Architecture, NG-NOS is divided into two layers. Lower layer or platform layer that controls the switch hardware including switch ASIC drivers and an upper layer which comprises of management and switching sub-systems. This upper layer carries the forwarding and management of switch & leaves the open switch hardware management to the lower layer. Both these layers are interconnected with standards based API’s so that multiple vendors can develop their own software for each layer independently. Finally this upper layer can be managed & controlled locally on a per switch basis or preferably by an external SDN controller using Openflow protocol which is a type of open API.  ]