One of the consistent questions that arises during the web-scale transition is the impact of managed access to networking infrastructure. How do we take traditional management techniques and adapt them to the new operational paradigm of web-scale networking, where automation drives the majority of changes and the infrastructure is treated as a holistic entity rather than node-by-node?

Local privileges

In the most basic way, we can migrate existing workflows to the new paradigm. Though inefficient, the old way of doing things still works with the new web-scale paradigm. The easiest way to do this is to restrict access to your switches using local privileges. In Linux, users are controlled using the adduser command, and the permissions for that user are controlled using the chmod commands.

A list of all users is stored in the /etc/passwd folder of Linux:

 cumulus@leaf02:~$ cat /etc/passwd
 root:x:0:0:root:/root:/bin/bash
 daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
 bin:x:2:2:bin:/bin:/usr/sbin/nologin
 sys:x:3:3:sys:/dev:/usr/sbin/nologin
 sync:x:4:65534:sync:/bin:/bin/sync
 games:x:5:60:games:/usr/games:/usr/sbin/nologin
 man:x:6:12:man:/var/cache/man:/usr/sbin/nologin
 lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin
 mail:x:8:8:mail:/var/mail:/usr/sbin/nologin
 news:x:9:9:news:/var/spool/news:/usr/sbin/nologin
 uucp:x:10:10:uucp:/var/spool/uucp:/usr/sbin/nologin
 proxy:x:13:13:proxy:/bin:/usr/sbin/nologin
 www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin
 backup:x:34:34:backup:/var/backups:/usr/sbin/nologin
 list:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin
 irc:x:39:39:ircd:/var/run/ircd:/usr/sbin/nologin
 gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/usr/sbin/nologin
 nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin
 systemd-timesync:x:100:103:systemd Time Synchronization,,,:/run/systemd:/bin/false
 systemd-network:x:101:104:systemd Network Management,,,:/run/systemd/netif:/bin/false
 systemd-resolve:x:102:105:systemd Resolver,,,:/run/systemd/resolve:/bin/false
 systemd-bus-proxy:x:103:106:systemd Bus Proxy,,,:/run/systemd:/bin/false
 frr:x:104:109:Frr routing suite,,,:/var/run/frr/:/bin/false
 ntp:x:105:110::/home/ntp:/bin/false
 uuidd:x:106:111::/run/uuidd:/bin/false
 messagebus:x:107:112::/var/run/dbus:/bin/false
 sshd:x:108:65534::/var/run/sshd:/usr/sbin/nologin
 snmp:x:109:114::/var/lib/snmp:/usr/sbin/nologin
 dnsmasq:x:110:65534:dnsmasq,,,:/var/lib/misc:/bin/false
 _lldpd:x:111:115::/var/run/lldpd:/bin/false
 cumulus:x:1000:1000:cumulus,,,:/home/cumulus:/bin/bash

Users can be added and deleted using the adduser and deluser commands:

 cumulus@leaf02:~$ sudo adduser test
 Adding user `test' ...
 Adding new group `test' (1002) ...
 Adding new user `test' (1002) with group `test' ...
 Creating home directory `/home/test' ...
 Copying files from `/etc/skel' ...
 Enter new UNIX password:
 Retype new UNIX password:
 passwd: password updated successfully
 Changing the user information for test
 Enter the new value, or press ENTER for the default
         Full Name []:
         Room Number []:
         Work Phone []:
         Home Phone []:
         Other []:
 Is the information correct? [Y/n] Y
 cumulus@leaf02:~$ sudo deluser test
 Removing user `test' ...
 Warning: group `test' has no more members.
 Done.
 cumulus@leaf02:~$ groups

After a user is created, the new user is only part of their own group.

Privilege level

This brings us to the first major paradigm shift. As the industry moves away from proprietary operating systems and closed CLIs, the networking managed access model needs to shift and adapt more closely to the server management model. Instead of treating managed access as a series of commands that are or are not allowed to execute, the restriction in privilege migrates to the files and programs themselves that are being executed.

Linux permissions aren’t the same as traditional networking permissions. We can create the same paradigm, but each file has its own read, write and execute functionality. It isn’t about command authorization, but rather about file authorization.

For example, the two most commonly edited files on Cumulus Linux are /etc/network/interfaces and /etc/frr/frr.conf. The former is owned by root and the later is owned by frr:

 test@leaf02:/etc/network$ ls -al
 total 8
 drwxr-xr-x 1 root root 132 Feb 25 19:40 .
 drwxr-xr-x 1 root root 3572 Apr 13 19:00 ..
 drwxr-xr-x 1 root root 22 Feb 25 19:27 if-post-down.d
 drwxr-xr-x 1 root root 36 Feb 25 19:27 if-pre-up.d
 drwxr-xr-x 1 root root 88 Apr 13 18:50 if-up.d
 drwxr-xr-x 1 root root 66 Feb 25 19:26 ifupdown2
 -rw-r--r-- 1 root root 309 Apr 13 18:49 interfaces
 drwxr-xr-x 1 root root 0 Feb 20 19:11 interfaces.d
 lrwxrwxrwx 1 root root 12 Feb 25 19:40 run -> /run/network
 test@leaf02:/etc/frr$ ls -al
 total 16
 drwxrwxr-x 1 frr frr 74 Feb 25 19:26 .
 drwxr-xr-x 1 root root 3572 Apr 13 19:00 ..
 -rw-r--r-- 1 frr frr 998 Feb 21 22:02 daemons
 -rw-r--r-- 1 frr frr 1150 Feb 21 22:02 daemons.conf
 -rw-r--r-- 1 frr frr 120 Feb 21 22:02 frr.conf
 -rw-r--r-- 1 frr frrvty 60 Feb 25 19:26 vtysh.conf

These files can only be edited by those respective users and groups. In order to edit these files, I have to use elevated privileges using the sudo command:

 test@leaf02:/etc/network$ sudo nano interfaces

 We trust you have received the usual lecture from the local System
 Administrator. It usually boils down to these three things:

     #1) Respect the privacy of others.
     #2) Think before you type.
     #3) With great power comes great responsibility.

 [sudo] password for test:
 Sorry, user test is not allowed to execute '/usr/bin/nano interfaces' as root on leaf02.
 test@leaf02:/etc/network$
 test@leaf02:/etc/network$ groups
 test

Since my test user is only part of the test group, I have no privileged access to edit these files.

This circles back to the concept of privilege levels. In traditional networking, user access is always controlled via two methods:

  1. Privilege level allowing read/write vs. read-only access
  2. Role based access control restricting which commands can be executed

In open networking, this concept is expanded and allows for more flexibility by migrating it to user/group privileges per file.

Centralized user management

Moving along, the next level up is integrating the user access with using a centralized user authority like TACACS or RADIUS. Centralized user management can come in three different flavors:

  • User authentication
  • User authorization
  • Per command authorization

Applying these to the open networking paradigm, all files have local user and group permissions. So the user authentication and authorization will dictate what groups the user has associated. Generally speaking, most centralized management systems will control which users have sudo privilege, as that is the differentiation between read-only and read-write users.

In conjunction with general open networking access privileges, the privileges can be applied to restrict a user’s capability in NCLU. NCLU has its own user privilege control file:

 test@leaf02:/etc/network$ cat /etc/netd.conf
 
 # If True "net" will display the linux command that is running
 # behind the scenes
 show_linux_command = False

 # If True the diffs shown in "net pending" and "net commit" will use colors
 color_diffs = True

 # Control which components are wrappered via "net"
 enable_time = True
 enable_lnv = True
 enable_accesslist = True
 enable_ifupdown2 = True
 enable_frr = True
 enable_dot1x = True
 enable_snmp = True
 enable_syslog = True
 enable_dns = True
 enable_run = True
 enable_traffic = True
 
 # Control which users/groups are allowed to run 'add', 'del',
 # 'clear', 'net abort', 'net commit' and restart services (FRR, etc)
 # to apply those changes
 users_with_edit = root, cumulus
 groups_with_edit = netedit
 
 # Control which users/groups are allowed to run 'show' commands
 users_with_show = root, cumulus, vagrant
 groups_with_show = netshow, netedit
 
 # Hide corner case ifupdown keywords
 ifupdown_blacklist = address-purge, bond-ad-actor-sys-prio, bond-ad-actor-system, 
 bond-num-grat-arp, bond-num-unsol-na, bond-use-carrier, bond-xmit-hash-policy, bridge-bridgeprio, 
 bridge-fd, bridge-hashel, bridge-hashmax, bridge-hello, bridge-igmp-querier-src, bridge-maxage, 
 bridge-maxwait, bridge-mclmc, bridge-mclmi, bridge-mcmi, bridge-mcqi, bridge-mcqpi, bridge-mcqri, 
 bridge-mcrouter, bridge-mcsqc, bridge-mcsqi, bridge-pathcosts, bridge-port-pvids, bridge-port-vids, 
 bridge-portprios, bridge-waitport, broadcast, link-type, mstpctl-ageing, mstpctl-fdelay, 
 mstpctl-forcevers, mstpctl-hello, mstpctl-maxage, mstpctl-maxhops, mstpctl-portp2p, 
 mstpctl-portpathcost, mstpctl-portrestrrole, mstpctl-portrestrtcn, mstpctl-treeportcost, 
 mstpctl-treeportprio, mstpctl-txholdcount, netmask, preferred-lifetime, scope, vxlan-ageing, 
 vxlan-learning, up, down, bridge-gcint, bridge-mcqifaddr, bridge-mcqv4src

In the above example, the test user is not part of users_with_edit privilege, and as a result this user cannot make changes:

 test@leaf02:/etc/network$ net add interface swp1
 ERROR: User test does not have permission to make networking changes.

This control can be included as a field stored in the centralized server. In this example, the ops user only has show privileges (read-only) via NCLU:

 cumulus@oob-mgmt-server:~$ ssh ops@leaf01
 ops@leaf01's password:

 Welcome to Cumulus VX (TM)

 Cumulus VX (TM) is a community supported virtual appliance designed for
 experiencing, testing and prototyping Cumulus Networks' latest technology.
 For any questions or technical support, visit our community site at:
 http://community.cumulusnetworks.com

 The registered trademark Linux (R) is used pursuant to a sublicense from LMI,
 the exclusive licensee of Linus Torvalds, owner of the mark on a world-wide
 basis.
 ops@leaf01:~$ groups
 radius_users netshow

The admin user has edit and show privileges (read-write) via NCLU:

 cumulus@oob-mgmt-server:~$ ssh admin@leaf01
 admin@leaf01's password:

 Welcome to Cumulus VX (TM)

 Cumulus VX (TM) is a community supported virtual appliance designed for
 experiencing, testing and prototyping Cumulus Networks' latest technology.
 For any questions or technical support, visit our community site at:
 http://community.cumulusnetworks.com

 The registered trademark Linux (R) is used pursuant to a sublicense from LMI,
 the exclusive licensee of Linus Torvalds, owner of the mark on a world-wide
 basis.
 admin@leaf01:~$ groups
 radius_users netedit

Access control with automation

When using automation, the configurations are normally stored on a git server. So at the highest level, the permissions on the repo server itself can restrict how the configuration is edited and deployed. At the next level, users can have isolated access to the management server and the policy can allow users to execute only necessary automation playbooks. The playbooks can be written with a service account, and that service account can have restricted access on each of the network devices. Take a look at this workflow to understand it better:

open networking access privileges

This moves all the access control responsibility to the management server, which can be tied into the existing access control infrastructure applied to all other servers in the environment.

If you enjoyed this how-to blog, you’re going to love our web-scale networking how-to video series! Study up on automation, configuring trunks and access ports, data center interconnect and more with the help of our expert how-to instructors. You’ll see that learning can be fun!