02 Dec 2015

Millions of Little Minions: Using Packets for Low Latency Network Programming and Visibility

ABSTRACT. This paper presents a practical approach to rapidly introducing new dataplane functionality into networks: End-hosts embed tiny programs into packets to actively query and manipulate a network’s internal state. We show how this “tiny packet program” (TPP) interface gives end-hosts unprecedented visibility into network behavior, enabling them to work with the network to achieve a desired functionality. Our design leverages what each component does best: (a) switches forward and execute tiny packet programs (at most 5 instructions) in-band at line rate, and (b) end-hosts perform arbitrary (and easily updated) computation on network state. By implementing three different research proposals, we show that TPPs are useful. Using a hardware prototype on a NetFPGA, we show our design is feasible at a reasonable cost.

Consider a large datacenter network … the extensive use of multipath routing … difficult to triangulate problems to a single switch

examine relevant network state such as switch ID, queue occupancy, input/output ports, port utilization, and matched forwarding rules at the exact time each packet was forwarded, so as to reconstruct what exactly transpired in the dataplane.

Can packets be instrumented to access and report on switch state? To date such state has been locked inside switches. This paper describes a simple, programmable interface that enables end-hosts to query switch memory (counters, forwarding table entries, etc.) from packets, directly in the dataplane. Specifically, a subset of packets carry in their header a tiny packet program (TPP), which consists of a few instructions that read, write, or perform simple, protocol-agnostic computation using switch memory.

A key observation in this paper is that having such programmable and fast access to network state benefits a broad class of net- work tasks—congestion control, measurement, troubleshooting, and verification—which we call dataplane tasks.

TPPs can be viewed as a particular, reasoned point within the spectrum of ideas in Active Networking [33, 36].

1.1 Goals

Our main goal is to expose network state to end-hosts through the dataplane. To benefit dataplane tasks, any interface should satisfy the following requirements:

Non-Goals
It is worth noting that our goal is not to be flexible enough to implement any, and all dataplane network tasks. For instance, TPPs are not expressive enough to implement per-packet scheduling.

2 Example Programs

We start our discussion using examples of dataplane tasks that can be implemented using TPPs, showcasing the utility of exposing network state to end-hosts directly in the dataplane.

What is a TPP?

2.1 Micro-burst Detection

2.2 Rate-based Congestion Control3

2.3 Network Troubleshooting Framework

2.4 Distributed Load Balancing

2.5 Other possibilities4

3 Design of TPP-Capable Switches

reference