When adding a NIC to CentOS, one normally expects to see the device appear in /etc/udev/rules.d/70-persistent-net.rules. But things don’t always go your way.

In A Perfect World…

The 70-persistent-net.rules file is a view into the kernel’s udev networking subsystem which serves to locate interfaces during startup, assign drivers, and define persistent names for their access.

  • If the unique device key — the MAC address in the case of network interfaces — does not already exist in the file, default sequential numbering is applied and an entry is added.
  • However, if the matching key for that interface already exists, then the kernel will use that NAME to create the device.

I have talked about persistent-net in the past, editing the file to restore expected interface order when a VM is cloned and the new VMNICs are detected. It can also be useful, for example, if you didn’t like the default assignment of ‘em1‘ as the first NIC on a Dell motherboard: you could edit persistent-net and change it to ‘eth0‘. (And then either reboot, or run ‘udevadm trigger’.)

Of course, once the Ethernet device names are known, you would then create the appropriate ifcfg-<Name> configuration files in /etc/sysconfig/network-scripts/:

DHCP Static


But what if persistent-net does not exist? This is the situation I found myself in when adding a quad-port NIC on several CentOS 6.4 and 6.5 physical hosts. I knew the cards were recognized because I could see them in the output of ‘lspci‘. But what did the kernel create the interfaces as? How was I to reference the ports by name? How should I define my ‘ifcfg-XXX‘ configuration files?

There are 2 ways to determine this. The first involves installing the ‘lshw‘ utility to view various hardware classes. Note that the interface name appears as the “logical name”:

The second method takes advantage of the sysfs to walk the hardware tree. (I’m betting this is just what ‘lshw‘ is doing.) Attributes for each device are kept in text files under the appropriate named-device directory.

So once I knew the discovered interface names — p1p1 thru p1p4 in the case of the quad — I created the appropriate ifcfg-p1pX files and connected my SAN.

All done, right? Not quite.

I could ping my SAN, but iSCSI discovery & login were failing with connect timeout:

After much hair pulling, I finally realized that iSCSI utilizes its own set of interface names, and these too were not auto-discovered with NIC addition. The pre-existing interfaces ‘iface0‘ and ‘iface1‘ lived under /var/lib/iscsi/ifaces, but nothing had been created for the recently-added quad.

Now it might be tempting to just add iface2, iface3, etc, but that would not be very efficient. iSCSI discovery tries each iface in sequence (with 5 retries for each), so if you put iSCSI on p1p4 and no other, you’d have to weather 6×5 timeouts before the ‘sendtargets‘ ultimately succeeded. (If you didn’t want that default search behavior, you could specify the exact interface with the ‘-I <iface-name>‘ option.)

The better method is to keep only the ifaces that are used for iSCSI. Either the MAC address (iface.hwaddress) or interface name (iface.net_ifacename) are the glue that joins iSCSI iface files to kernel device identity. So in my case, I just updated the existing iface0 to use the MAC data for p1p1. Alternatively, I could have created a new interface using the p1p1 name. Both approaches can be done by editing at the /var/lib/iscsi/ifaces file level, or through iscsiadm commands as follows:

At that point, discovery and login worked as we had originally expected when all this began:

Understanding the ways in which subsystems such as udev and iscsi work by learning as much as you can about the config files is fantastic foundational knowledge. But to be able to pair that up with the commands needed to effect a change is the icing on the cake, because it opens the door to scripting and automation. Later on, I’ll post a script that does all the work of udev‘s write_net_rules for cases when persistent-net can’t be relied upon…