stream-talk - The Stanford Data Stream The Stanford Data...

Info iconThis preview shows pages 1–11. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: The Stanford Data Stream The Stanford Data Stream Management System Management System Jennifer Widom Stanford University st anfordst re amdat am anager stanfordstreamdatamanage 2 Formula for a Database Research Project Formula for a Database Research Project Pick a simple but fundamental assumption underlying traditional database systems Drop it Reconsider all aspects of data management and query processing Many Ph.D. theses Prototype from scratch stanfordstreamdatamanage 3 Following the Formula Following the Formula Dropped assumption ( LORE LORE Project): Data has a fixed schema declared in advance Dropped assumption ( TRIO TRIO Project): All data is accurate, consistent, and complete Dropped assumption ( STREAM STREAM Project) First load data, then index it, then run queries Continuous data streams Continuous data streams Continuous queries Continuous queries stanfordstreamdatamanage 4 Data Streams Data Streams Continuous, unbounded, rapid, time-varying streams of data elements Occur in a variety of modern applications Network monitoring and traffic engineering Sensor networks, RFID tags Telecom call records Financial applications Web logs and click-streams Manufacturing processes DSMS DSMS = Data Stream Management System stanfordstreamdatamanage 5 DBMS versus DSMS DBMS versus DSMS Persistent relations One-time queries Random access Access plan determined by query processor and physical DB design Transient streams (and persistent relations) Continuous queries Sequential access Unpredictable data characteristics and arrival patterns stanfordstreamdatamanage 6 DSMS Scratch Store The (Simplified) Big Picture The (Simplified) Big Picture Input streams Register Query Streamed Result Stored Result Archive Stored Relations stanfordstreamdatamanage 7 (Simplified) Network Monitoring (Simplified) Network Monitoring Register Monitoring Queries DSMS Scratch Store Network measurements, Packet traces Intrusion Warnings Online Performance Metrics Archive Lookup Tables stanfordstreamdatamanage 8 The STREAM System The STREAM System Data streams and stored relations Declarative language for registering continuous queries Flexible query plans and execution strategies Textual, graphical, and application interfaces Relational, centralized (for now) stanfordstreamdatamanage 9 STREAM System Challenges STREAM System Challenges Must cope with: Stream rates Stream rates that may be high,variable, bursty Stream data Stream data that may be unpredictable, variable Continuous query loads Continuous query loads that may be high, variable stanfordstreamdatamanage 10 STREAM System Challenges STREAM System Challenges Must cope with: Stream rates Stream rates that may be high ,variable, bursty Stream data Stream data that may be unpredictable, variable Continuous query loads Continuous query loads that may be high , variable Overload stanfordstreamdatamanage...
View Full Document

Page1 / 48

stream-talk - The Stanford Data Stream The Stanford Data...

This preview shows document pages 1 - 11. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online