Unformatted text preview: Magic Sets
Optimization technique for recursive Datalog.
Also a win on some nonrecursive SQL
Mumick, Finkelstein, Pirahesh, and
Ramakrishnan, 1990 SIGMOD, pp. 247 258.
Combines bene ts of both top-down
backward chaining, recursive tree search
and bottom-up forward chaining, naive,
seminaive processing of logic, without
disadvantages of either. Example of Nonrecursive Use Find the programmers who are making less than
the average salary for their department.
FROM Emps e1
WHERE e1.job = 'programmer' AND
FROM Emps e2
WHERE e2.dept = e1.dept
; Naive implementation computes the average
salary for all departments.
Magic-sets" implementation rst determines
the departments that have programmers
perhaps very few. It can then use an index
on Emps.dept to avoid accessing the entire
Emps relation. Recursive Example
ancX,Y :- parX,Y
ancX,Y :- parX,Z & ancZ,Y Query: anc0; W .
Top-down search e.g., Prolog would:
1. Query the EDB for par0; Y .
2. By the rst rule: return all such answers,
say f0; 1; 0; 2g.
3. The same parent facts are also useful
in the second rule to set up calls" to
anc1; Y and anc2; Y .
4. Recursively solve these queries.
1 Advantage of Top-Down
We never even ask about individuals that are
not in the ancestry of individual 0. Advantage of Bottom-Up i.e., naive, seminaive
We don't go into in nite recursive loops. Example Both of the following Datalog programs loop if
ancX,Y :- parX,Y
ancX,Y :- ancX,Z & parZ,Y
ancX,Y :- parX,Y
ancX,Y :- ancX,Z & ancZ,Y Key Magic-Sets Ideas 1. Introduce magic predicates" to represent the
bound arguments in queries that a top-down
search would ask.
2. Introduce supplementary predicates" to
represent how answers are passed from leftto-right through a rule.
3. Technical details to get right:
a Predicate splitting : an IDB predicate
must be called" in top-down search
with only one binding pattern.
b Subgoal recti cation : avoid IDB subgoals
with repeated variables. Rule Goal Graphs
Needed to assure unique binding patterns for
Composed of rule and goal nodes, as follows. Goal Nodes
Predicate + adornment."
Adornment = list of b's and f 's, indicating
which arguments are bound, which are free.
Example: pbfb. First and third arguments of p
2 Rule Nodes
riS T represents the point in rule r after
seeing i subgoals, with variables in set S
bound, those in T free.
j Children of Goal Nodes Children of goal node p are those rule nodes
r0S T such that
1. Rule r has head predicate p.
2. S is the set of variables that appear in those
arguments of the head that says are bound.
3. T is the other variables of r.
j Children of Rule Nodes Children of the rule node rjS T are:
1. The goal node of the j + 1st subgoal of r,
with adornment that binds those arguments
whose only variables are in S .
2. The rule node rjS T , where S = S +
variables appearing the in j + 1st subgoal;
T is the other variables.
Exceptions: no rj +1 rule node if r has only
j + 1 subgoals. No goal child if j = 0 and r
has no subgoals.
j 0 j 0 0 0 Constructing the RGG
Start with goal node whose adornment
matches bindings of query.
Add nodes by constructing children as
required by rules from previous slides.
Reordering of subgoals of a rule is allowed:
helps maximize bound" arguments.
Reordering may be di erent for di erent rule
nodes. Example Here is a nonrecursive example, where the RGG is
r1: pX,Y :- qX,Z & rZ,Y
r2: rA,B :- sA,B
r3: rA,B :- tA,B
3 Query form pbf , e.g., p0; W ?
j r1X1;Z Y
: qbf j rbf
: sbf tbf j j Recursive Example
r1: ancX,Y :- parX,Y
r2: ancX,Y :- ancX,Z & ancZ,Y
Query; ancbb, e.g., ancjoe; sue?
: r2X0;Y Z
: j j parbb r2X1;Y;Z
: ancbf X
r1:0Y j r2X0 Y;Z
: j j r2X1;Z Y
: parbf j Splitting Predicates
For magic-sets to work, there must be a
unique binding pattern associated with each
No constraint on EDB predicates.
Key idea: For each adornment such that p
appears in the RGG, make a new predicate
4 p . Rules for p are the same as for p, but
predicates of IDB subgoals are the version
with the correct binding pattern.
RGG helps us gure out the needed binding
patterns. Example For RGG above:
anc bbX,Y :- parX,Y
anc bbX,Y :- anc bfX,Z &
anc bfX,Y :- parX,Y
anc bfX,Y :- anc bfX,Z &
anc bfZ,Y Rectifying Subgoals
All IDB subgoals must have arguments that
are distinct variables.
Feasible for datalog no function symbols.
Fixes some problems where RGG knows about
fewer bound arguments than the top-down
3 See p. 801 of PDKS-II.
Trick: replace an IDB subgoal G with
variables appearing in more than one
argument and or constant arguments by a
new predicate whose arguments are single
copies of the variables appearing in G.
Create rules for the new predicate by unifying
G with heads of rules for G's predicate.
Repetition may be needed because the
resulting rules may have unrecti ed subgoals. Example
r1: pX,Y :- aX,Y
r2: pX,Y :- bX,Z & pZ,Z & bZ,Y
pZ; Z is unrecti ed. Create qZ = pZ; Z .
Unify heads of rules with pZ; Z . Careful! Z
in body of r2 must be renamed.
r1 becomes pZ,Z :- aZ,Z or
qZ :- aZ,Z 5 r2 becomes
pZ,Z :- bZ,W & pW,W & bW,Z or qZ :- bZ,W & qW & bW,Z
Finally, in the original r2 we replace subgoal
pZ; Z by qZ . The resulting rules, with
pX,Y :- aX,Y
pX,Y :- bX,Z & qZ & bZ,Y
qX :- aX,X
qX :- bX,Y & qY & bY,X Magic Sets Transformation Start with a program and a binding pattern for a
1. Split predicates to get unique binding
2. Rectify subgoals.
3. Introduce magic and supplementary predicates
as follows. Magic Predicates For each IDB predicate p, introduce m p.
Arguments of m p correspond to bound
arguments of p in its unique binding pattern.
Intuition: m p is true of exactly those tuples
that are members of queries to some p-node in
the top-down expansion. Supplementary Predicates For each rule r of n subgoals, introduce
supplementary predicates supr:j for 0 j n.
Arguments are the bound and active variables
before the j + 1st subgoal of r.
3 A variable is active i it appears either in
the head or a subgoal from j + 1 on.
Intuition: true for a tuple i that tuple
represents a possible binding for the bound,
active variables at that point. 6 ...
View Full Document
- Spring '09
- Logic, Predicate, Datalog, IDB, binding pattern