FloCon 2020 has ended
Back To Schedule
Tuesday, January 7 • 9:00am - 9:30am
SysFlow: Scalable System Telemetry for Improved Security Analytics

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Feedback form is now closed.
In this talk, we introduce SysFlow as a new data representation for system behavior introspection for scalable security, compliance, and performance analytics. SysFlow is a compact open data format that lifts the representation of system activities into a flow-centric, object-relational mapping that records how applications interact with their environment—analogous to how NetFlow summarizes network communications. However, unlike NetFlow, which only captures network interactions, SysFlow connects network behaviors to processes and file access information, providing a richer context for analysis. This additional context facilitates deeper introspection into attack kill chains, resulting in analyses that yield lower false positives, and higher detection rates than traditional network-based approaches. SysFlow supports single-event and volumetric flow representations of process control flows, file interactions, and network communications. The new telemetry format drastically reduces storage requirements as compared to existing system telemetry sources, thereby enabling feature-filled analytics, process-level provenance tracking, and long-term data archival for threat hunting and forensics.

We present a new open-source telemetry pipeline built atop SysFlow. The pipeline provides a set of reusable components and APIs that enable ease deployment of telemetry probes for bare-host and container workload monitoring, as well as the export of SysFlow records to S3-compliant object stores feeding into distributed security analytics jobs based on Apache Spark. Specifically, the analytics framework provides an extensible policy engine that ingests customizable security policies described in a declarative input language, providing facilities for defining higher-order logic expressions that are checked against SysFlow records. This allows practitioners to easily define security and compliance policies that can be deployed on a scalable, out-of-the-box analysis toolchain while supporting extensible programmatic APIs for the implementation of custom analytics algorithms. As a result, the pipeline enables researchers and analysts to redirect their efforts to developing and sharing analytics, rather than building support infrastructure for telemetry.

The SysFlow probe has been optimized to incur minimal performance overheads and does not require program instrumentation or system call interposition for data collection, therefore having negligible impact on monitored workloads. The implementation has been validated under multiple stress test profiles. We will demonstrate use cases for the identification of advanced and persistent threats, security vulnerabilities, performance bottlenecks, and service outages.

Attendees will Learn:
  • Design principles and architectural insights influencing the SysFlow telemetry pipeline implementation;
  • How to deploy and instantiate the SysFlow pipeline in container cloud environments;
  • How to use the new telemetry pipeline to (a) create security policies for container integrity analytics and (b) write custom algorithms atop the analytics framework to identify malicious behaviors in containers.

avatar for Frederico Araujo

Frederico Araujo

Research Scientist, IBM Research
Dr. Frederico Araujo is a Research Scientist at IBM Research, where he leads the team's efforts on cloud-native security. He's an active contributor to open source and a maintainer of the SysFlow project. He's also a contributor to CNCF's Falco project. His work has been featured... Read More →
avatar for Teryl Taylor

Teryl Taylor

Research Staff Member, IBM Research
Dr. Teryl Taylor is a Research Staff Member in the Cognitive Cybersecurity Intelligence Group at IBM Research. He has ten years of experience in cybersecurity related research, including NetFlow based analytics, system telemetry and analytics, security visualization and cyber deception... Read More →

Tuesday January 7, 2020 9:00am - 9:30am EST
Regency Ballroom Hyatt Regency Savannah 2 W. Bay Street Savannah GA 31401