csce824-lecture15 - Non­Traditional Databases...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Non­Traditional Databases Non­Traditional Databases Reading Reading 1. 2. Scientific data management at the Johns Hopkins institute for data intensive engineering and science Yanif Ahmad, Randal Burns, Michael Kazhdan, Charles Meneveau, Alex Szalay, Andreas Terzis, February 2011 SIGMOD Record , Volume 39 Issue 3 , Migrating a (large) science database to the cloud Ani Thakar, Alex Szalay, June 2010 HPDC '10: Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing , Farkas CSCE 824 ­ Spring 2011 2 Reading Reading 3. Farkas M. Stonebaker, U. Cetintemel, One Size Fits All": An Idea Whose Time Has Come and Gone, in Proceeding of CDE '05 Proceedings of the 21st International Conference on Data Engineering, IEEE Computer Society Washington, DC, USA, 2005, CSCE 824 ­ Spring 2011 3 Traditional Database Traditional Database Management Systems Farkas Focus on business data management Provide uniform capabilities regardless of the data characteristics Need: capabilities to meet new application requirements CSCE 824 ­ Spring 2011 4 Examples of New Needs Examples of New Needs Farkas Stream Data Processing Large scale scientific databases Data warehousing CSCE 824 ­ Spring 2011 5 Streaming Data Streaming Data Farkas Sensor­based applications – Real­time systems: sophisticated alerting, location­ based services, – Historical data Financial applications – Support applications, such as electronic trading, legal compliance, real­time marker analysis, etc. Performance requirements CSCE 824 ­ Spring 2011 6 Performance SDMS vs. RDMS Performance SDMS vs. RDMS Empirical results (see reference paper #3) Issues: Inbound processing model Correct primitives for stream processing (aggregates, “timeout,” “slack”) – Seamless integration of DBMS processing with application processing (client­server vs. embedded applications) – Transactional behavior (weaker notion of recovery, tolerance, no ACID requirements) – – Farkas CSCE 824 ­ Spring 2011 7 Security for Streaming Data? Security for Streaming Data? What is the difference between the security needs of streaming vs. traditional (e.g., relational) data? How to enforce security? – Security punctuation Farkas CSCE 824 ­ Spring 2011 8 Scientific Databases Scientific Databases Massive amount of data Heterogeneous data – Sensor data, satellite, scientific simulation data, etc. Goal: better understanding of physical phenomena – Genomic database, geological exploration, astronomy, etc. Farkas CSCE 824 ­ Spring 2011 9 Scientific Databases Scientific Databases Need efficient analysis and querying capabilities – Multi­dimensional indexing (e.g., genomic sequence indexing) – Specific applications (e.g., visualization of seismic data) – Specific aggregations (e.g., data mining for biological correlation) – Efficient data archiving, staging, lineage, and error propagation techniques Farkas CSCE 824 ­ Spring 2011 10 Example Scientific Data Example Scientific Data Management Reference #1 Basic research: 1. 2. 3. 4. Farkas formation of hypotheses and theories designing experiments for their validation collecting data by experimentation analyzing data to guide new insights for further research CSCE 824 ­ Spring 2011 11 Scientific Computing Scientific Computing Steps 3 and 4 are data intensive Need to improve computational power – – – – Farkas Parallel processing Grid and supercomputers Special application logic Preservation of scientific data CSCE 824 ­ Spring 2011 12 Current Technologies and Scientific Current Technologies and Scientific Databases Farkas Reference #2: How to migrate large scale scientific database to cloud environment? Difficult engineering process Limited capabilities of database user Based on commercial cloud CSCE 824 ­ Spring 2011 13 Data Warehousing Data Warehousing Repository of data providing organized and cleaned enterprise­wide data (obtained form a variety of sources) in a standardized format – – – Farkas Data mart (single subject area) Enterprise data warehouse (integrated data marts) Metadata CSCE 824 ­ Spring 2011 14 Data Warehousing Data Warehousing Farkas Difference between OLTP and OLAP Data management: updates, indexing, dependencies, etc. OLAP: needs Read Optimized storage CSCE 824 ­ Spring 2011 15 Next Class Geographical Databases Farkas Farkas CSCE 824 ­ Spring 2011 16 ...
View Full Document

This note was uploaded on 12/13/2011 for the course CSCE 824 taught by Professor Staff during the Fall '11 term at South Carolina.

Ask a homework question - tutors are online