Toward Automated Worldwide Monitoring of Network-level Censorship

Although Internet censorship is a well-studied topic, to date most published studies have focused
on a single aspect of the phenomenon, using methods and sources specific to each researcher.
Results are dicult to compare, and global, historical perspectives are rare. Because each group
maintains their own software, erroneous methods may continue to be used long after the error has
been discovered. Because censors continually update their equipment and blacklists, it may be
impossible to reproduce historical results even with the same vantage points and testing software.
Because “probe lists” of potentially censored material are labor-intensive to compile, requiring an
understanding of the politics and culture of each country studied, researchers discover only the most
obvious and long-lasting cases of censorship.
In this dissertation I will show that it is possible to make progress toward addressing all of
these problems at once. I will present a proof-of concept monitoring system designed to operate
continuously, in as many dierent countries as possible, using the best known techniques for
detection and analysis. I will also demonstrate improved techniques for verifying the geographic
location of a monitoring vantage point; for distinguishing innocuous network problems from
censorship and other malicious network interference; and for discovering new web pages that are
closely related to known-censored pages. These techniques improve the accuracy of a continuous
monitoring system and reduce the manual labor required to operate it.
This research has, in addition, already led to new discoveries. For example, I have confirmed
reports that a commonly-used heuristic is too sensitive and will mischaracterize a wide variety of
unrelated problems as censorship. I have been able to identify a few cases of political censorship
within a much longer list of cases of moralizing censorship. I have expanded small seed groups of
politically sensitive documents into larger groups of documents to test for censorship. Finally, I
can also detect other forms of network interference with a totalitarian motive, such as injection of
surveillance scripts.
In summary, this work demonstrates that mostly-automated measurements of Internet censorship
on a worldwide scale are feasible, and that the elusive global and historical perspective is within
reach.