An Adversarial Evaluation of Network Signaling and Control Mechanisms - Information Security and Cryptology-ICISC 2010 - page 252

An Adversarial Evaluation of Network Signaling and
Control Mechanisms

Kangkook Jee 1 , Stelios Sidiroglou-Douskos 2 ,
Angelos Stavrou 3 , and Angelos Keromytis 1

1 Department of Computer Science, Columbia University
{ jikk,angelos } @cs.columbia.edu
2 Computer Science and Artificial Intelligence Laboratory, MIT
[e-mail]
3 Department of Computer Science, George Mason University
[e-mail]

Abstract. Network signaling and control mechanisms are critical to coordinate
such diverse defense capabilities as honeypots and honeynets, host-based de-
fenses, and online patching systems, any one of which might issue an actionable
alert and provide security-critical data. Despite considerable work in exploring
the trust requirements of such defenses and in addressing the distribution speed
of alerts, little work has gone into identifying how the underlying transport sys-
tems behave under adversarial scenarios.
In this paper, we evaluate the reliability and performance trade-offs for a vari-
ety of control channel mechanisms that are suitable for coordinating large-scale
collaborative defenses when under attack. Our results show that the performance
and reliability characteristics change drastically when one evaluates the systems
under attack by a sophisticated and targeted adversary. Based on our evaluation,
we explore available design choices to reinforce the reliability of the control chan-
nel mechanisms. To that end, we propose ways to construct a control scheme to
improve network coverage without imposing additional overhead.

1 Introduction

The prevalence and effectiveness of large-scale malware phenomena (worms, botnets,
web-based malware) has led to the development of several automated defenses that
detect new threats and generate various kinds of fixes such as patches, filters. The secu-
rity literature is rife with distributed security systems [7,5] which assume that reliable,
scalable and robust Content Distribution Network (CDN) functionality is universally
available. To date, the primary metrics of effectiveness have been propagation time (la-
tency and throughput) and node coverage in the presence of “natural” phenomena such
as churn. However, the conspicuous absence of an adversarial analysis, both in terms of
performance impact and security guarantees ( e.g., susceptibility to man-in-the-middle
attacks), is of particular concern as the control channel for security data is a very attrac-
tive target for adversaries. This is especially true for systems that make design decisions
that favor performance over robustness ( e.g., using a centralized tracker in BitTorrent).
We argue that such a narrow view of system performance is inadequate and even
dangerous in the presence of malicious adversaries. In other areas of security (spam,