Mesh Networks

by Diana Thayer and Jamey Sharp

Introduction: Why Mesh? For What?

The state of the web is bleak. Net neutrality is dead. The public internet has become a common platform for the disinformation and propaganda campaigns of fringe groups and nation-states alike. Virtually every website is funded by ads and the proceeds from mining or selling data about you. The corporations who control the wires that deliver the web to those who can pay consider the service a luxury, rather than a critical utility in this interconnected era. The necessary networking infrastructure never made it to millions of people in the United States alone, because telecoms consider extending service to them to be too expensive. They call it the “last mile” problem but it smells like greed and apathy. When disasters strike and take out networking infrastructure, nobody is quick to restore it–except the people themselves.

How can we provide resilient connectivity to all and return control over that infrastructure to the communities it serves? This is a hard problem, but there’s no shortage of people and projects working on it. Many communities have turned to mesh networking technology to retake their networking infrastructure, such as in Puerto Rico¹ and Detroit². Using inexpensive hardware and free software, communities can–and do!–distribute mesh nodes that form networks providing durable connectivity to people in the area. In the next section we’ll discuss how this approach is different from today’s Internet.

In Catalonia, guifi.net³ services a network of more than 34 thousand nodes transmitting across a system of wires, wifi, and point-to-point transmissions. Freifunk⁴, based in Germany, is creating “a global movement for free infrastructure and open frequencies” and services more than 45 thousand nodes. Freedombone⁵ provides disk images for flashing inexpensive, open hardware like the Beaglebone Black with software for serving applications like chat, email, voice-over-IP, and even social media sites like a Mastodon⁶ instance over a mesh network. It is becoming easier all the time to rebuild and retake the networking infrastructure that has become critical to virtually every aspect of our lives. The state of the web may be bleak but we the people are constructing something much, much better.

So let’s talk about what mesh networks are, where they are being used today, and what challenges we face in using them to secure a future of community-governed network infrastructure.

Technical Overview

If you have internet access at home, then you’re probably familiar with paying a telecom corporation for that access. (For many people that’s either your phone company or your cable TV provider.) But we aren’t all connected to the same company. Instead, our Internet Service Providers (ISPs) maintain numerous inter-related commercial contracts with each other about who can carry whose traffic where.

Large ISPs sometimes agree to exchange traffic with each other without any money changing hands between them (this is called “settlement-free peering”) on the assumption that on average they’ll send about as much traffic as they receive. The largest, so-called “Tier 1”, ISPs have peering agreements with all of the other largest ISPs, so that every internet user in the world is indirectly a customer of one of these Tier 1 ISPs. For the ISPs who are not in Tier 1, they have to pay one or more other ISPs for “transit” when their customers’ traffic needs to cross the other ISP’s network to reach its destination.

This complicated set of contracts is programmed into the routers that keep today’s internet running. Using a routing protocol called the Border Gateway Protocol (BGP), routers advertise routing information to their neighbours via TCP on port 179. The advertised information defines the best route from point A to point B when multiple routes are available. By design, routers trust one another that the routes they announce as best are indeed best, which has increasingly led to international tensions. One issue has come up regularly: If a router falsely advertises a traffic route, routers would happily obey and pass applicable traffic along the advertised route. Merely by acting in bad faith, an actor can make portions of the Internet inaccessible, cut off access entirely, or intercept and modify communications. Because of its importance, many nation-states own or take part in maintaining the fidelity of BGP routing information. However, BGP incidents involving falsely advertised routes affecting a government’s critical systems have begged the question, has this trustful routing system become a medium of information warfare? The systems governing the declaration of routes are arcane and involved nations have so far accepted one another’s apologies for any technical issues. After all, what recourse have they got? If they escalate, the whole system comes down, and the network backbone crumbles. Bye bye Internet.

By contrast, a mesh routing protocol puts all network participants on an equal footing. Every device participating in a mesh network acts to forward traffic as needed for anyone else. There are no “Tier 1” mesh nodes, and every node peers with any other node that it can reach, without concern for business agreements. Such networks are self-organizing, ad-hoc, and may not require routers to trust each other, which avoids the complex web of contracts needed to keep today’s Internet operating, although at the cost of more complex technology.

There are many different mesh routing protocols that have been devised in the last two decades, with different strengths and use cases, but they all have this in common: mesh networks are decentralized peer-to-peer systems. Like e-mail and BitTorrent, mesh networks scale up and avoid single points of failure by distributing work across all participants. Mesh routing protocols have so much in common with popular peer-to-peer “overlay networks” that some protocols, like Virtual Ring Routing, have even drawn significant inspiration from distributed hash tables like Chord, Pastry, or the Kademlia DHT that’s used in BitTorrent.⁷

Implementation Challenges

There is no single routing protocol that works for all networks. Research on mesh routing protocols has mostly focused on special cases. In this section we’ll explore some of the trade-offs that current routing protocols make, and discuss which sorts of networks they’re suited for.

We can catalog mesh routing protocols according to a remarkably large number of trade-offs, such as:

Scale: 100 nodes? 1000? or every device on the planet?
Mobility: can the network handle nodes that move at highway speeds?
Power: can nodes run off small batteries?
Versatility: is the network specialized for only certain use cases?
Latency: how long do messages take to get delivered?
Trust: does the network need to defend against malicious or greedy users?

One major mesh research area has been for sensor networks. Sensor networks are composed of tens to hundreds of sensors scattered through an area and measuring different properties of their environment, such as weather, soil conditions, or vehicle speed. These sensors usually need to operate using very little power, so they can run for months or years on a single battery without maintenance. The nodes in a single sensor network are usually all operated by a single owner, so the nodes trust each other, and are isolated from the rest of the world. They typically have very small amounts of data to send, and do so infrequently–perhaps generating tens of bytes every few hours. Usually sensor networks can tolerate large delays in delivering data to their central data collection point–depending on the use case it may not matter if it takes hours or even days before the message arrives. In most applications, the sensor nodes don’t get moved once they’ve been placed.

For either disaster relief or military applications, which have also motivated considerable research funding, the constraints are similar in some ways and quite different in others. In these cases a single organization, or a few cooperating agencies at most, could quickly deploy a few dozen network nodes across an area and have them self-configure into a network. Because it’s a single-owner network, the nodes can trust each other. These networks are primarily for coordinating local response so they don’t need to connect to the outside world; specialists may have long-range radio or satellite communications equipment for that. On the other hand, there may be large amounts of time-critical data to send, such as voice or video conversations, photos, maps, etc. The deployments aren’t expected to last very long, so even though available power may be limited, low-power operation isn’t as high of a priority as for sensor networks. Often the nodes are being carried by people or on vehicles in the field, so their routing protocols need to support highly mobile network nodes with rapidly changing radio connections.

The way that people use today’s Internet is quite different from either of those use cases. Instead of tens or hundreds of network nodes, there are billions, owned by nearly as many different people and organizations who, on the whole, don’t trust each other. All kinds of data are carried over the internet, including low-bandwidth delay-tolerant information like in sensor networks, and time-critical voice and video calls like in disaster relief networks; but also large delay-tolerant public datasets like software updates or the collection of all photos taken through space telescopes⁸, and low-bandwidth time-critical communications like instant messaging, SSH, or multi-player games. Power isn’t a problem for a substantial portion of the current internet, because reliable power has been a prerequisite for most internet equipment, but low-power networking would make the internet accessible to more mobile devices and to more communities. There’s a significant divide between fixed and mobile portions of the internet: servers and home/office computers rarely move, but there are over two billion smartphones in use worldwide,⁹ and those are highly mobile and expected to have continuous internet access even at highway speeds.

As this is only an introduction, we can’t go into details such as how mesh routing protocols can be connected with other networks like the internet, or how they can defend routing integrity against malicious nodes; but there are comprehensive survey papers and other resources on these and many other topics in mesh routing research.¹⁰

Non-technical challenges

Beyond the design of your mesh network’s technological underpinnings, community efforts to implement a mesh may run into political or commercial difficulties.

If your goal is to replace last-mile Internet service in your neighborhood, then your neighbors will probably want your mesh network to also connect to the rest of the internet. At that point you need to find somebody willing to sell you internet access under terms that allow you to share it. If you read the Terms of Service for your home internet access, you’ll probably find language prohibiting sharing that connection with other people. At that point you may find yourself diving into the world of peering and transit contracts with backbone ISPs, getting your own Autonomous System Number and buying blocks of at least 255 IP addresses on the open market for $10-$20 each.¹¹ The roadblocks imposed on small community projects under the current structure of the Internet are absurd.

If your goal is to build a larger community network for your city or county, you may find that telecoms start lobbying your state or federal government to block your efforts. In the rural town of Wilson, North Carolina, city officials advanced a program to maintain a municipal fiber-to-the-home (FTTH) network for residents, businesses and institutions. Neighboring towns asked Wilson to expand the network, called “Greenlight”, in order to serve their communities. However, state law prohibited the network from expanding beyond county borders. When a reversal of two FCC rulings permitted the network to expand, they did so, and new communities gained better access than any commercial telecom had or would provide. Later, an appellate court reversed one of those reversals, effectively reinstating the ban on municipal networks that cross county lines, so the state legislature passed a law that allowed it to retain service with the caveat that if private provider enters the market in that area, Greenlight will have to cease operations there. In this way, a conflux of state and federal institutions prevented local communities from governing their own networking infrastructure, even when it provided better, cheaper, and more reliable service.¹²

Conclusion

Establishing robust mesh networks is a hard problem. Many questions remain unanswered and critical research is ongoing. While existing implementations like guifi.net and Freifunk demonstrate that mesh can scale, there are precious few standards. Communities are still exploring how to organize and support the labor to maintain these ad-hoc networks. Legal difficulties plague these efforts in the United States, as major telecoms fight to retain and advance their monopolies, while communities struggle to pool the resources to establish infrastructure.

But mesh networking is not a far-off dream. It is happening right now in places around the world. When networking infrastructure has failed or when telecoms have refused service, many communities have established mesh networks that continue to serve them, even in dire and resource-starved circumstances. We can rebuild the web, we must rebuild the web, and we are rebuilding the web. We got this.