Overview

Figure  illustrates how VNET+ enables traffic isolation between RAW sockets at a high level. When a packet is sent out by a process, it is tagged with the xid of the sending slice. This xid is obtained from the process in context, or from the socket in the context of which the packet is sent. The tag is used to update a connection record to associate the same xid with incoming packets that are received in the context of the connection. When an incoming packet is received, it is tagged based on the value found in the connection structure. In some cases, the first packet of a connection may be an incoming packet. In such cases, VNET+ looks for protocol sockets listening for such packets. If it finds such a socket, then it sets the tag of the packet as well as configures the corresponding connection structure based on this value.

The tag of a packet may be set differently if the program manipulating the socket runs in root context, and if it requests that the xid on the connection be set to a particular value. The program issues the request using the setsockopt system call and the SO_PEERCRED flag. Note that we use SO_PEERCRED here to avoid inventing a new socket option - but it is a misnomer. Although the peercred of a socket typically corresponds to the credentials (such as the uid or the xid) of the peer, in this case, we use this operation to set the credentials of the current side. If such a credential is set, then it overrides the assignment of the packet tag based on the corresponding connection or socket.

Once the tag of a packet has been set, it undergoes normal processing in the network stack and is processed by a set of packet taps, protocol handlers and RAW socket handlers. Protocol handling carries on as it would normally, with the packet tag ignored. The RAW socket handlers have been modified to first verify that the packet tag corresponds to the xid of the RAW socket (i.e., the xid of the process that created the RAW socket). Among the packet taps, the ones that use the packet tag include the traffic shaper (e.g. HTB) and the PACKET SOCKET handler. The traffic shaper functions independently of VNET+, simpy consuming the value of the tag as it is set by VNET. The treatment of the PACKET SOCKET handler is more complex, and discussed in a section of its own.

PlanetFlow, Planet-Lab's traffic-auditing system also uses the tag to keep track of the slices generating the packets tracked.

Figure: VNET+ and RAW sockets