{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

csce824-lecture15 - Non­Traditional Databases...

Info icon This preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Non­Traditional Databases Non­Traditional Databases Reading Reading 1. 2. Scientific data management at the Johns Hopkins institute for data intensive engineering and science Yanif Ahmad, Randal Burns, Michael Kazhdan, Charles Meneveau, Alex Szalay, Andreas Terzis, February 2011 SIGMOD Record , Volume 39 Issue 3 , http://dl.acm.org/citation.cfm?id=1942776.1942782&coll=DL&dl=ACM&CFID http://dl.acm.org/citation.cfm?id=1942776.1942782&coll=DL&dl=ACM&CFI Migrating a (large) science database to the cloud Ani Thakar, Alex Szalay, June 2010 HPDC '10: Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing , http://dl.acm.org/citation.cfm?id=1851539&bnc=1 Farkas CSCE 824 ­ Spring 2011 2 Reading Reading 3. Farkas M. Stonebaker, U. Cetintemel, One Size Fits All": An Idea Whose Time Has Come and Gone, in Proceeding of CDE '05 Proceedings of the 21st International Conference on Data Engineering, IEEE Computer Society Washington, DC, USA, 2005, http://www.computer.org/portal/web/csdl/abs/proceedings/icde/2005/2 CSCE 824 ­ Spring 2011 3 Traditional Database Traditional Database Management Systems Farkas Focus on business data management Provide uniform capabilities regardless of the data characteristics Need: capabilities to meet new application requirements CSCE 824 ­ Spring 2011 4 Examples of New Needs Examples of New Needs Farkas Stream Data Processing Large scale scientific databases Data warehousing CSCE 824 ­ Spring 2011 5 Streaming Data Streaming Data Farkas Sensor­based applications – Real­time systems: sophisticated alerting, location­ based services, – Historical data Financial applications – Support applications, such as electronic trading, legal compliance, real­time marker analysis, etc. Performance requirements CSCE 824 ­ Spring 2011 6 Performance SDMS vs. RDMS Performance SDMS vs. RDMS Empirical results (see reference paper #3) Issues: Inbound processing model Correct primitives for stream processing (aggregates, “timeout,” “slack”) – Seamless integration of DBMS processing with application processing (client­server vs. embedded applications) – Transactional behavior (weaker notion of recovery, tolerance, no ACID requirements) – – Farkas CSCE 824 ­ Spring 2011 7 Security for Streaming Data? Security for Streaming Data? What is the difference between the security needs of streaming vs. traditional (e.g., relational) data? How to enforce security? – Security punctuation Farkas CSCE 824 ­ Spring 2011 8 Scientific Databases Scientific Databases Massive amount of data Heterogeneous data – Sensor data, satellite, scientific simulation data, etc. Goal: better understanding of physical phenomena – Genomic database, geological exploration, astronomy, etc. Farkas CSCE 824 ­ Spring 2011 9 Scientific Databases Scientific Databases Need efficient analysis and querying capabilities – Multi­dimensional indexing (e.g., genomic sequence indexing) – Specific applications (e.g., visualization of seismic data) – Specific aggregations (e.g., data mining for biological correlation) – Efficient data archiving, staging, lineage, and error propagation techniques Farkas CSCE 824 ­ Spring 2011 10 Example Scientific Data Example Scientific Data Management Reference #1 Basic research: 1. 2. 3. 4. Farkas formation of hypotheses and theories designing experiments for their validation collecting data by experimentation analyzing data to guide new insights for further research CSCE 824 ­ Spring 2011 11 Scientific Computing Scientific Computing Steps 3 and 4 are data intensive Need to improve computational power – – – – Farkas Parallel processing Grid and supercomputers Special application logic Preservation of scientific data CSCE 824 ­ Spring 2011 12 Current Technologies and Scientific Current Technologies and Scientific Databases Farkas Reference #2: How to migrate large scale scientific database to cloud environment? Difficult engineering process Limited capabilities of database user Based on commercial cloud CSCE 824 ­ Spring 2011 13 Data Warehousing Data Warehousing Repository of data providing organized and cleaned enterprise­wide data (obtained form a variety of sources) in a standardized format – – – Farkas Data mart (single subject area) Enterprise data warehouse (integrated data marts) Metadata CSCE 824 ­ Spring 2011 14 Data Warehousing Data Warehousing Farkas Difference between OLTP and OLAP Data management: updates, indexing, dependencies, etc. OLAP: needs Read Optimized storage CSCE 824 ­ Spring 2011 15 Next Class Geographical Databases Farkas Farkas CSCE 824 ­ Spring 2011 16 ...
View Full Document

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern