Integration with systems and data sources Rather thansolely relying on flow

Integration with systems and data sources rather

This preview shows page 21 - 23 out of 29 pages.

Integration with systems and data sources – Rather than solely relying on flow data, data analysis usually benefits from integrating with other data sources, such as geolo- cation databases, WHOIS, blacklists, BGP information, etc. In terms of system integration, support for directory services for user authentication are a welcome feature. Analysis delay – This delay should not be confused with the processing delay of a flow collector; while the processing delay determines when flow data is made available for analysis, the analysis delay is based on the computation time of the analysis software. The shorter the computation time, the more timely the analysis. Especially for time-critical applications, such as IDSs, analysis delays are an important criterium. The market of commercial flow analysis applications con- sists of both appliance products (hardware or virtual) and software (standalone or Software as a Service ). An overview of applications that have a primary focus on flow data analysis, is provided in Table VIII. Several conclusions can be drawn from this table. First, all applications provide Flow Analysis & Reporting functionality, usually complemented with Threat Detection or Performance Monitoring functionality. Very few applications provide both threat detection and performance monitoring functionality. Second, those applications doing performance monitoring have a strong focus on application performance, which is in line with the observation that ap- plication awareness in flow monitoring is becoming more important. The number of available open-source flow data analysis applications that have been updated at least once since 2008 is rather small. We have compiled an overview in Table IX. Contrary to commercial applications, open-source alternatives are usually rather limited in functionality; although they all support flow analysis & reporting, extended functionality like performance monitoring is rare. Moreover, threat detection functionality is not supported by any open-source application. Some applications, such as NfSen, do however provide plugin- support, by means of which threat detection or performance monitoring can be implemented. As flow data consists of large volumes of essentially tabular, timestamped information that is not very semantically com- plex, existing work in data analysis for other fields may prove to be applicable to flow data as well. We have had success in using the open-source pandas 18 data analysis framework for Python, together with glue code for bridging the gap between IPFIX files and pandas data-frames [93]. Pandas was originally developed for financial analysis and visualization, and is based on the numpy numerical computing framework for Python, which provides efficient primitives for dealing 18 . pydata . org/
Image of page 21
22 TABLE VIII C OMMERCIAL F LOW D ATA A NALYSIS A PPLICATIONS Vendor Product Flow analysis & reporting Threat detection Performance monitoring Main selling point(s) Arbor Networks Pravail Network Security Intelligence (NSI) 3 3 Global threat intelligence, DDoS attack detection &
Image of page 22
Image of page 23

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture