Fast statistical anomaly detection on streaming data

Info icon This preview shows pages 76–78. Sign up to view the full content.

View Full Document Right Arrow Icon
Fast statistical anomaly detection on streaming data has become an important area of research given the proliferation of data over the past few decades, and the need to detect quickly the event that a process has changed significantly from past behavior. Applications can be found in many areas including engineering (Chandola et al. , 2009), computer science (Forrest et al. , 1996), and, specifically, in communications networks (Mukherjee et al. , 1994; Yeung and Ding, 2003; Lambert and Liu, 2006; Chandola et al. , 2009). In many cases, the data can be represented as a graph (Kolaczyk, 2009). Nodes represent actors sending and receiving data, and edges represent communications between nodes. Anomalies can be detected in the changes to the structure of the graph (Noble and Cook, 2003; Collins and Reiter, 2007). Scan statistics for communications graphs were established in Priebe et al. (2005), and used a star shape. Paths are compared with stars in Section 3.5.1. Similar methods that aggregate at the node, examining each node’s behavior independently, include Yeung and Ding (2003) and Mukher- jee et al. (1994). In none of this work are edges modeled. Yet, different edges may have significantly different behavior over time, and attacks between nodes must happen over edges. In addition, traversal cannot be captured by analyzing node behavior separately for each node. In these cases, mod- eling each edge is desirable. Additionally, all of these graph methods tend to lack fine-grained locality, which we address by using k -paths. Because of this locality, we have discovered attacks that are not specifically traversal or star-shaped. In only one article identified, Heard et al. (2010), are the individ- ual edges modeled. A Bayesian testing framework is proposed to test the anomalousness of each edge in a social network, without consideration of other local-edge anomalousness. These edges are then passed to a secondary analysis that examines the graph constructed from the edges that were detected in the initial pass. Interesting features of the anomalous edge graph can be detected in this way, but simultaneously testing multiple local sets of edges will have increased power to detect locally anomalous behavior. For example, if two anomalous edges were connected by a non-anomalous edge, this possible traversal path would likely be missed by the technique in Heard Copyright © 2014. Imperial College Press. All rights reserved. May not be reproduced in any form without permission from the publisher, except fair uses permitted under U.S. or applicable copyright law. EBSCO Publishing : eBook Collection (EBSCOhost) - printed on 2/16/2016 3:37 AM via CGC-GROUP OF COLLEGES (GHARUAN) AN: 779681 ; Heard, Nicholas, Adams, Niall M..; Data Analysis for Network Cyber-security Account: ns224671
Image of page 76

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Statistical Detection of Intruders Within Computer Networks 77 et al. (2010), but is a valid anomaly in many settings. In addition, when data speeds are high, a fully Bayesian treatment may pose computational difficulties, unless the model is parsimonious enough for sequential Monte Carlo (Doucet et al. , 2001).
Image of page 77
Image of page 78
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern