weld-planet03-part2 - Planning for theWe I I b Exe cution...

Info iconThis preview shows pages 1–11. Sign up to view the full content.

View Full Document Right Arrow Icon
© Danie l S. Weld, PLANET 2003 Tutorial on Data Integration Planning for the Web II Execution & Service Integration Dan Weld University of Washington June, 2003
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
2 © Danie l S. Weld, PLANET 2003 Tutorial on Data Integration Acknowledgements Oren Etzioni Yolanda Gil Keith Golden Alon Halevy Zack Ives Tal Shaked Caveat
Background image of page 2
3 © Danie l S. Weld, PLANET 2003 Tutorial on Data Integration Outline Execution for Data Integration Coping with incomplete statistics, latency Interleaved planning & execution Convergent query processing Service Integration Web service composition Background Representational issues Planning algorithms Automated data analysis
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
4 © Danie l S. Weld, PLANET 2003 Tutorial on Data Integration Optimization and Execution Problem: Few and unreliable statistics about the data. Unexpected (possibly bursty) network transfer rates. Generally, unpredictable environment. General solution: (research area) Adaptive query processing. Interleave optimization and execution. As you get to know more about your data, you can improve your plan.
Background image of page 4
5 © Danie l S. Weld, PLANET 2003 Tutorial on Data Integration Adaptivity & Incremental Processing Query Performance Query Translation User's Query Query over Sources Query Results Tukwila Network-Based Query Processor Evaluated within the Tukwila system [Ives PhD]
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
6 © Danie l S. Weld, PLANET 2003 Tutorial on Data Integration Query Optimization: Model Query Plans’ Execution & Choose the Best op op op op Restock (R) 100 tuples Orders (O) 50 tuples Shipping (S) 90 tuples Restock (R) 100 tuples Orders (O) 50 tuples Shipping (S) 90 tuples From source sizes, stats, estimate result sizes, costs RO ~30 tuples ROS ~270 tuples 50 sec ROS ~270 tuples 30 sec OS ~15 tuples Estimates, assumptions introduce error: Exponential increase in estimation error with each join [Ioannidis & Christodoulakis 91] [Antoshekov 93,96] Worse if no detailed statistics
Background image of page 6
7 © Danie l S. Weld, PLANET 2003 Tutorial on Data Integration Why Does Data Integration Make Optimization Harder? Query optimization estimates costs using knowledge about environment and data: Data source sizes (“cardinalities”) Often unavailable or not meaningful in data integration Histograms Too expensive to maintain in data integration I/O costs Network I/O costs fluctuate Need a way to gain this sort of knowledge!
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
8 © Danie l S. Weld, PLANET 2003 Tutorial on Data Integration Some Solutions 1. Adaptive operators 2. Mid query reoptimization 3. Convergent query processing 1. Query scrambling [Franklin et al.] 2. Eddies [Hellerstein et al.]
Background image of page 8
9 © Danie l S. Weld, PLANET 2003 Tutorial on Data Integration O p t i m i z e r ( R e - ) O p t i m i z e r M e m A l l o c - F r a g m e n t e r E x e c u t i o n E n g i n e T e m p S t o r e E v e n t H a n d l e r Q u e r y O p e r a t o r s R e f o r m u l a t o r C a t a l o g s o u r c e m a p p i n g s q u e r y l o g i c a l p l a n e x e c p l a n a n s w e r d a t a e x e c r e s u l t s Tukwila Data Integration System Novel components: Event handler Optimization-execution loop Adaptive operators
Background image of page 9

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
10 © Danie l S. Weld,
Background image of page 10
Image of page 11
This is the end of the preview. Sign up to access the rest of the document.

Page1 / 78

weld-planet03-part2 - Planning for theWe I I b Exe cution...

This preview shows document pages 1 - 11. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online