Wrapping On Query Languages and Efficient XML Integration
A paper by Vassilis Christophides and Sophie Cluet
Speaker: Yu Wang
Outline
Introduction YAT System YAT XML Algebra Wrapping source query languages Optimization techniques Conclusion
Introduction
Application require integrated access to various information sources, fast deployment and low maintenance cost XML
Enable easy wrapping of external sources Enable easy wrapping of declarative integration
Advantage in using XML
Have flexible format: can be used to represent structured/semistructured information Convert data into XML easily Exist many languages allowing declarative integration of XML data( e.g. MSL, YATL) Facilitates interoperability as a standard
Hard issues
Wrapping type information
XML's current form of typing is not sufficient to capture rich type systems(e.g. an object database schema). Recent proposals( e.g. XML Schema, DCD) don't provide definitive standard yet.
Wrapping source query capabilities
TSIMMIS system: query templates are used to describe source capabilities Not allow an exhaustive description of a source capabilities
Processing XML queries efficiently
Not have a well-understood algebra
Solution
This paper propose an algebraic framework and optimization techniques to address the last 2 issues
An algebra for XML
Introduce an operational model based on a general-purpose algebra for XML
A source description language
Use the algebra to wrap full text queries/ structured query languages( e.g. OQL/ SQL)
Query processing techniques
Show the algebra is appropriate to optimize integration application
Example
Use an example to show the improvements this paper propose Goal
Integrate two sources
Highly structured: an object database Partially structured document repository: full-text indexed with Wais
Sample XML Data
<work> <artist> Claude Monet </artist> <title> Nympheas </title> <style> Impressionist </style> <size> 21 x 61 </size> <cplace>Giverny</cplace> </work> .... <work> <artist> Claude Monet </artist> <title> Waterloo Bridge </title> <style> Impressionist </style> <size> 29.2 x 46.4 </size> <history>Painted with <technique> Oil on canvas </technique> in ... </work> <object id="a1" class="artifact"> <tuple> <title> Nympheas </title> <year> 1897 </year> <creator> Claude Monet </creator> <price> 10.000.000 </price> <owners refs = "p1 p2 p3"/> </tuple> </object> ..... <object id="p3" class="person"> <tuple> <name> Doctor X </name> <auction> 10.1500.000</auction> </tuple> </object>
YAT System
A semistructured data conversion system Rely on a library of generic wrappers and a declarative integration languages, YATL Use 3 steps to setup the application example with YAT Structural information exported by the two wrappers: o2 and xmlwais
Installing Wrappers and Mediators
logos{simeon}: o2-wrapper -server gringos.inria.fr -system cultural -base art -port 6066 o2-wrapper is running at logos.inria.fr:6066 logos{simeon}: -----------------------------------------------------------------------------sappho{christop}: xmlwais-wrapper -directory ~christop/wais-sources/museum.src -port 6060 xmlwais-wrapper is running at sappho.ics.forth.gr:6060 sappho{christop}: -----------------------------------------------------------------------------cosmos{cluet}: yat-mediator -port 6666 yat-mediator is running at cosmos.inria.fr:6666 yat> connect o2artifact logos.inria.fr:6066; yat> connect xmlartwork sappho.ics.forth.gr:6060; yat> import o2artifact; yat> import xmlartwork; yat> load "/u/cluet/YAT/view1.yat";
YAT Type System
Allow to represent information at various levels of genericity( model, schema , data) Understand the connection existing between these levels
Using this feature to wrap query languages
A graphical representation of YAT data model
O2 data model
Described as atomic type/ a tuple/ a collection / a reference A tuple type is represented as a collection of linear subtrees
YAT Type System( Con't)
The representation of the document exported by the xmlwais wrapper
Described as a sequence of mandatory elements Capture partially structured information
YAT meta-model
Capture any tree Others are instances of this model
O2, XML-Wais and YAT mediator structural metadata
Integration Programs
Compose
A sequence of rules A sequence of queries : 3 clauses
MATCH
Perform pattern-matching Filters are used to navigate in the source data and bind variables
WHERE MAKE
Construct the result by creating a new tree
Integrating information about the works of Art
artworks() := MAKE doc * & artwork($t,$c) := work [ title: $t, artist: $a, year: $y, price: $p, style: $s, size: $si, owners * $o, more: $elds ] MATCH artifacts WITH set { * class: artifact: tuple { title: $t, year: $y, creator: $c, price: $p, Owners: list * class: person: tuple { name: $o, auction: $au}}}, works WITH works * work [artist: $a, title: $t', style: $s, size: $si, *($elds) ] WHERE $y > 1800 AND $c = $a AND $t = $t'
YAT XML Algebra
Overview
Characteristic
Mail tool for both the generic description of source query capabilities and the XML query optimization Provides a fixed set of predefined operations
Satisfy the requirement
Expressive power
Capture evaluation of query and integration languages
Support for flexible typing Support for optimization
An extension of one object algebra Independent of any underlying physical access structure
YAT XML Algebra( Con't)
Operators
Bind operator
Extract data from input tree according to the filter Produce a tabular representation of the variable
Tree operator
Returns a collection of trees conforming to the input pattern Equivalent to a grouping operation
Skolem functions
Create new identifier Perform value assignment
YAT XML Algebra( Con't)
Object algebra
Select/ Project / Join / Union / Intersection Group/ Sort/ Map / D-Join Applied on the top level of a Tab structure except Map
YAT XML Algebra( Con't)
A Bind operation and resulting Tab structure
The Tree operation
YATL Algebraic Translation
Steps
Named documents are the input MATCH clause is translated into a Bind operation The connection between the various inputs is materialized using a Join operation Where clauses are translated into a Select operation MAKE clause is translated using the Tree operation
The example shows the algebraic translation of the view definition of Figure 2 and example query
Query Example 1
Q1: What are the artifacts created at \Giverny" ?
MAKE $t MATCH artworks WITH doc.work.[ title.$t, more.cplace.$cl ] WHERE $cl = "Giverny"
Algebraization of YATL queries
Wrapping source query languages
Wrapping source operations in YAT is performed in two steps
Signature Essential Manual Semantics Example: function imported by the O2 wrapper
1 <operation name="external"> 2 <operation name="current_price"> 3 <input> 4 <value model="Artifact_Schema" pattern="Artifact"/></input> 5 <output> 6 <leaf label=Float /></output> 7 </operation> 8 </operation>
Describing OQL capabilities
YAT operational model borrows a large part OQL of algebra OQL binding capabilities are more restricted
Take this restriction into account by restrict Bind operation Bind is always the first operation in a query
Describing OQL capabilities( Con't )
Capturing Binding capabilities
Bind operation has 2 parameters: a filter and the data that has to be filtered/ bound Need to specify which are the acceptable filters for OQL E.g. valid filters, called Fpattern
Integration programmer does not need to see it Coded by YAT developers Embedded within the O2 wrapper
O2 Filter patterns exported in XML
1 <interface name="o2artifact"> 2 <operat> 3 <fmodel name="o2fmodel"> 4 <fpattern name="Fclass"> 5 <node label="class" bind="tree"> 6 <node label="Symbol" bind="none" inst="ground"> 7 <value pattern="Ftype"/></node></node> 8 </fpattern> 9 10 <fpattern name="Ftype"> 11 <union> 12 <leaf label="Bool"/> 13 <leaf label="Char"/> 14 <leaf label="Int"/> 15 <leaf label="Float"/> 16 <leaf label="String"/> 17 <node label="tuple" col="set" bind="tree"> 18 <star inst="ground"> 1 <node label="Symbol" bind="none"> 2 <value label="Ftype"/></node> 20 </star></node> 21 <node label="set" col="set" bind="tree"> 1 <star inst="none"><value label="Ftype"/></star></node> 23 <node label="bag" col="bag" bind="tree"> 1 <star inst="none"><value label="Ftype"/></star></node> 25 <node label="list" bind="tree"> 26 <star inst="none"><value label="Ftype"/></star></node> 27 <node label="array" bind="tree"> 28 <star inst="none"><value label="Ftype"/></star></node> 29 <ref pattern="Fclass"/> 30 </union> 31 </fpattern> 32 </fmodel> 33 </operat> 34 </interface>
Description for OQL. Below is a subset of the operational interface of the O2 wrapper:
1 <omodel name="o2omodel"> 2 <operation name="algebraic"> 3 <union> 4 <operation name="bind"> 5 <input> 6 <value model="o2model" pattern="Type"/> 7 <filter model="o2fmodel" pattern="Ftype"/> 8 </input> 9 <output> 10 <value model="yatstruc" pattern="Tab"/> 11 </output> 12 </operation> 13 <operation name="select"></operation> 14 <operation name="map"></operation> 15 ... 16 </union> 17 </operation> 18 19 <operation name="boolean"> 20 <union> 21 <operation name="shalow_eq"></operation> 22 <operation name="leq"></operation> 23 ... 24 </union> 25 </operation> 26 ... 27 </omodel>
Describing Wais capabilities
Three steps to wrap the query capabilities of the XML-Wais source
Specify the source Fpatterns Declare the source supporting Bind and Select Describe the full-text predicate contains supplied by Wais
Interface to the XML-Wais wrapper
Interface to the XML-Wais wrapper( Con't)
Optimization techniques
The algebra has two parts
Object algebra Two operations to manipulate XML data
Bind and Tree
Optimization is divided into two parts
Optimization techniques proposed for the relational or object models are directly applicable Rewriting techniques for the Bind/Tree operations
Optimize user queries with views locally or by pushing queries to the external sources
Bind Rewriting
Reason
A simpler Bind has a better chance to be pushed to a source Bind entails navigation that can be costly and should be transformed into more traditional associative access as much as possible
2 ways
Vertical navigation Horizontal navigation and type filtering
Bind and vertical navigation
Two ways
Split a complex Bind into elementary Binds, each one connecting together through DJoins Split a complex Bind into a linear sequence of elementary ones, each one navigating down the result of the previous one
From Bind to Join
Splitting Binds...