Comprehensions in Haskell

Take Everything From Me,
But Leave Me
The Comprehension
DBPL — September 2017
Torsten Grust
db.inf.uni-tuebingen.de
Apologies, I am only
a database person
Torsten Grust
2
U Tübingen
Apologies, I am only
a database person
It is in this connection worth noticing
that in the Comm.ACM the papers
on data bases […] are of markedly lower
linguistic quality than the others.
—Edsger Dijkstra (EWD691)
Torsten Grust
2
U Tübingen
Apologies, I am only
a database person
The point is that the way in which the database
management experts tackle the problems seems to be
so grossly inadequate. They seem to form an inbred
crowd with very little knowledge of computing science
in general, who tackle their problems primarily
politically instead of scientifically.
Often they seemed to be mentally trapped by the
—Edsger
Dijkstra
(EWD577)
intricacies of early, rather ad hoc solutions to rather
accidental problems; as soon as such a technique has
Torsten Grust
U Tübingen
3
received a name, it becomes "a database concept".
so grossly inadequate. They seem to form an inbred
crowd with very little knowledge of computing science
in general, who tackle their problems primarily
politically instead of scientifically.
Apologies, I am only
a database person
Often they seemed to be mentally trapped by the
intricacies of early, rather ad hoc solutions to rather
accidental problems; as soon as such a technique has
received a name, it becomes "a database concept".
And a totally inadequate use of language, sharpening
their pencils with a blunt axe.
I learned a few things about Databases. I learned
—Edsger
Dijkstra
(EWD577)
—or: had my tentative impression confirmed— that
the term "Database Technology", although sometimes
Torsten Grust
3
U Tübingen
accidental problems; as soon as such a technique has
received a name, it becomes "a database concept".
And a totally inadequate use of language, sharpening
their pencils with a blunt axe.
Apologies, I am only
a database person
I learned a few things about Databases. I learned
—or: had my tentative impression confirmed— that
the term "Database Technology", although sometimes
used, is immature, for there is hardly any underlying
"science" that could justify the use of the term
"technology".
—Edsger Dijkstra (EWD577)
Torsten Grust
3
U Tübingen
Comprehension Syntax
[ h x | x ← xs, p x ]
Torsten Grust
4
U Tübingen
Comprehension Syntax
generator
[ h x | x ← xs, p x ]
head
filter
1. Successively draw bindings for x from domain xs,
2. for those bindings that pass filter p,
3. evaluate head h,
Torsten Grust
4
U Tübingen
Comprehension Syntax
generator
[ h x | x ← xs, p x ]
head
filter
1. Successively draw bindings for x from domain xs,
2. for those bindings that pass filter p,
3. evaluate head h,
4. collect results to form a list.
Torsten Grust
4
U Tübingen
Comprehension Syntax
generator
M
[ h x | x ← xs, p x ]
head
filter
1. Successively draw bindings for x from domain xs,
2. for those bindings that pass filter p,
3. evaluate head h,
4. collect results to form an M.
Torsten Grust
4
U Tübingen
Extension vs. Intension
{
Torsten Grust
}
5
U Tübingen
Extension vs. Intension
{ I, III, V, VII, IX }
Torsten Grust
5
U Tübingen
Extension vs. Intension
{ I, III, V, VII, IX }
set
[ roman x | x ← [1…10], odd x ]
Torsten Grust
5
U Tübingen
In the Beginning …
Relational Completeness of Data Base Sublanguages
E. F. Codd, IBM Research Report RJ987, 1972
Torsten Grust
6
U Tübingen
In the Beginning …
Relational Completeness of Data Base Sublanguages
E. F. Codd, IBM Research Report RJ987, 1972
generator
Torsten Grust
head
filter
6
U Tübingen
Today’s XQuery 3.0
XQuery 3.0: An XML Query Language
D. Chamberlin et al., W3C Recommendation, April 2014
Core of XQuery: versatile FLWOR expression
Torsten Grust
7
U Tübingen
Today’s XQuery 3.0
generator
filter
head
XQuery 3.0: An XML Query Language
D. Chamberlin et al., W3C Recommendation, April 2014
Core of XQuery: versatile FLWOR expression
Torsten Grust
7
U Tübingen
Today’s XQuery 3.0
generator
filter
head
XQuery 3.0: An XML Query Language
D. Chamberlin et al., W3C Recommendation, April 2014
generator
head
Core of XQuery: versatile FLWOR expression
Torsten Grust
7
U Tübingen
Early XQuery
A Data Model and Algebra for XQuery
M. Fernandez et al., October 2003
Torsten Grust
8
U Tübingen
Early XQuery
A Data Model and Algebra for XQuery
M. Fernandez et al., October 2003
generator
Torsten Grust
head
filter
8
U Tübingen
An XQuery Nucleus
1 module Query where
2 import Prelude hiding (elem,index)
3 4 -- Data Model: Constructors ---------------------------------------------------5 6 text :: String -> Node
7 elem :: Tag -> [Node] -> Node
8 ref :: Node -> Node
9 10 year0 :: Node
11 year0 = elem "@year" [ text "1999" ]
12 13 book0 :: Node
14 book0 = elem "book" [ 15 elem "@year" [ text "1999" ],
16 elem "title" [ text "Data on the Web" ],
17 elem "author" [ text "Abiteboul" ],
18 elem "author" [ text "Buneman" ],
19 elem "author" [ text "Suciu" ]]
20 21 bib0 :: Node
22 bib0 = elem "bib" [
23 elem "book" [ Torsten
Grust
U Tübingen
9
elem "@year" [ text "1999"
],
24 An XQuery Nucleus
•
•
≈ 430 lines of Haskell (300+ lines of examples)
•
List comprehensions express path navigation, FLOWR,
grouping/aggregation, quantification
Implements a complete XQuery core, including tree construction and traversal
Torsten Grust
9
U Tübingen
LINQ
The World According to LINQ
E. Meijer, October 2011
Comprehension syntax deeply embedded into
C#, with monad-based semantics organized
around SelectMany (aka >>=, flatmap)
Torsten Grust
10
U Tübingen
LINQ
generator
head
filter
The World According to LINQ
E. Meijer, October 2011
Comprehension syntax deeply embedded into
C#, with monad-based semantics organized
around SelectMany (aka >>=, flatmap)
Torsten Grust
10
U Tübingen
Emma
Implicit Parallelism through Deep Language Embedding
A. Alexandrov et al., SIGMOD 2015
Deep embedding of comprehensions in Scala,
compiles to Apache Flink / Spark
Torsten Grust
11
U Tübingen
Emma
generator
filter
head
Implicit Parallelism through Deep Language Embedding
A. Alexandrov et al., SIGMOD 2015
Deep embedding of comprehensions in Scala,
compiles to Apache Flink / Spark
Torsten Grust
11
U Tübingen
Pig Latin
Compiles to sequences of Map/Reduce jobs
Torsten Grust
12
U Tübingen
generator
Pig Latin
head
filter
Compiles to sequences of Map/Reduce jobs
Torsten Grust
12
U Tübingen
generator
Pig Latin
head
filter
Compiles to sequences of Map/Reduce jobs
Torsten Grust
12
U Tübingen
generator
filter
Pig Latin
head
Told you so.
Compiles to sequences of Map/Reduce jobs
Torsten Grust
12
U Tübingen
SQL
Query Q4 of the TPC-H OLAP benchmark
Torsten Grust
13
U Tübingen
head
generator
SQL
filter
Query Q4 of the TPC-H OLAP benchmark
Torsten Grust
13
U Tübingen
head
generator
SQL
filter
head
generator
filter
Query Q4 of the TPC-H OLAP benchmark
Torsten Grust
13
U Tübingen
head
generator
generator
SQL
head
filter
head
head
generator
generator
filter
filter
head
filter
generator
Torsten Grust
Query Q4 of the TPC-H OLAP benchmark
13
U Tübingen
Torsten Grust
14
U Tübingen
SQL
Torsten Grust
14
U Tübingen
One Way to Teach SQL
Torsten Grust
15
U Tübingen
One Way to Teach SQL
SELECT A, B
FROM
S
Torsten Grust
15
U Tübingen
One Way to Teach SQL
c ←∅
∅;
foreach x ∈ S do
c ← c ⨄ {(x.A,x.B)};
return c;
SELECT A, B
FROM
S
Torsten Grust
15
U Tübingen
One Way to Teach SQL
c ←∅
∅;
foreach x ∈ S do
c ← c ⨄ {(x.A,x.B)};
return c;
SELECT A, B
FROM
S
SELECT MAX(A)
FROM
S
Torsten Grust
15
U Tübingen
One Way to Teach SQL
SELECT A, B
FROM
S
c ←∅
∅;
foreach x ∈ S do
c ← c ⨄ {(x.A,x.B)};
return c;
SELECT MAX(A)
FROM
S
c ← –∞
–∞;
foreach x ∈ S do
c ← max2(c,x.A);
return c;
Torsten Grust
15
U Tübingen
One Way to Teach SQL
SELECT A, B
FROM
S
c ←∅
∅;
foreach x ∈ S do
c ← c ⨄ {(x.A,x.B)};
return c;
SELECT MAX(A)
FROM
S
c ← –∞
–∞;
foreach x ∈ S do
c ← max2(c,x.A);
return c;
0 < ALL(SELECT A
FROM
S)
Torsten Grust
15
U Tübingen
One Way to Teach SQL
SELECT A, B
FROM
S
c ←∅
∅;
foreach x ∈ S do
c ← c ⨄ {(x.A,x.B)};
return c;
SELECT MAX(A)
FROM
S
c ← –∞
–∞;
foreach x ∈ S do
c ← max2(c,x.A);
return c;
0 < ALL(SELECT A
FROM
S)
c ← true
true;
foreach x ∈ S do
c ← c ∧ (0 < x.A);
return c;
Torsten Grust
15
U Tübingen
One Way to Teach SQL
SELECT A, B
FROM
S
c ←∅
∅;
foreach x ∈ S do
c ← c⨄
⨄ {(x.A,x.B)};
return c;
SELECT MAX(A)
FROM
S
c ← –∞
–∞;
foreach x ∈ S do
c ← max
max22(c,x.A);
return c;
0 < ALL(SELECT A
FROM
S)
c ← true
true;
foreach x ∈ S do
c ← c∧
∧ (0 < x.A);
return c;
Torsten Grust
15
U Tübingen
One Program Form for SQL
Torsten Grust
16
U Tübingen
One Program Form for SQL
c ← z;
foreach x ∈ xs do
fold(z,f,xs) ≡
c ← f (c ,x);
return c;
Torsten Grust
16
U Tübingen
One Program Form for SQL
c ← z;
foreach x ∈ xs do
fold(z,f,xs) ≡
c ← f (c ,x);
return c;
Torsten Grust
M
carrier
liftM
zM
⊕M
bag
set
list
all
some
sum
max
min
bag t
set t
list t
bool
bool
num
t (ordered)
t (ordered)
{⋅}
{⋅}
[⋅]
id
id
id
id
id
∅
∅
[]
true
TRUE
false
FALSE
0
-∞
∞
⊎
∪
++
∧
∨
+
max2
min2
16
U Tübingen
One Program Form for SQL
Torsten Grust
17
U Tübingen
One Program Form for SQL
SELECT A
FROM
S
WHERE A > B
Torsten Grust
fold(∅,⊕,S) with
⊕(c,x) = c ⊎ (if (x.A > x.B) {x.A}
else ∅)
17
U Tübingen
One Program Form for SQL
SELECT A
FROM
S
WHERE A > B
fold(∅,⊕,S) with
⊕(c,x) = c ⊎ (if (x.A > x.B) {x.A}
else ∅)
fold(∅,⊕,R)
with
SELECT x.A,y.B
⊕(c,x) = c ⊎ fold(∅,⊗,S) with
FROM
R x,S y
⊗(d,y) = d ⊎ {(x.A,y.B)}
Torsten Grust
17
U Tübingen
fold(,,) Gets Ugly Quickly
Torsten Grust
18
U Tübingen
fold(,,) Gets Ugly Quickly
SELECT COUNT(*)
FROM
R x
WHERE EXISTS (SELECT y
FROM
S y
WHERE x.A = y.B)
fold(0,⊕,fold(∅,⊗,R)) with
with ⊕(c,_) = c + 1
⊗(d,x) = d ⊎ if (fold(false,⊙,S)) {x} else ∅
with ⊙(e,y) = e ∨ (x.A = y.B)
Torsten Grust
18
U Tübingen
fold(,,) Gets Ugly Quickly
SELECT COUNT(*)
FROM
R x
WHERE EXISTS (SELECT y
FROM
S y
WHERE x.A = y.B)
Algebraic
Wonderland.
fold(0,⊕,fold(∅,⊗,R)) with
with ⊕(c,_) = c + 1
⊗(d,x) = d ⊎ if (fold(false,⊙,S)) {x} else ∅
with ⊙(e,y) = e ∨ (x.A = y.B)
Torsten Grust
18
U Tübingen
fold(,,) Gets Ugly Quickly
SELECT COUNT(*)
FROM
R x
WHERE EXISTS (SELECT y
FROM
S y
WHERE x.A = y.B)
Algebraic
Wonderland.
fold(0,⊕,fold(∅,⊗,R)) with
with ⊕(c,_) = c + 1
REJECT!
⊗(d,x) = d ⊎ if (fold(false,⊙,S)) {x} else ∅
with ⊙(e,y) = e ∨ (x.A = y.B)
Torsten Grust
18
U Tübingen
Comprehension Semantics
Torsten Grust
19
U Tübingen
Comprehension Semantics
[ e | ]M
[ e | v1 ← e1, q ]M
[ e | p, q ]M
Torsten Grust
19
U Tübingen
Comprehension Semantics
[ e | ]M
≡
[ e | v1 ← e1, q ]M
≡
[ e | p, q ]M
≡
Torsten Grust
liftM(e)
19
U Tübingen
Comprehension Semantics
[ e | ]M
≡
liftM(e)
[ e | v1 ← e1, q ]M
≡
fold(zM,⊗,e1) with
⊗(c,v1) = c ⊕M [ e | q ]M
[ e | p, q ]M
≡
if (p) [ e | q ]M else zM
Torsten Grust
19
U Tübingen
Comprehensible SQL
Torsten Grust
20
U Tübingen
Comprehensible SQL
SELECT COUNT(*)
FROM
R x
WHERE EXISTS (SELECT y
FROM
S y
WHERE x.A = y.B)
Torsten Grust
20
U Tübingen
Comprehensible SQL
SELECT COUNT(*)
FROM
R x
WHERE EXISTS (SELECT y
FROM
S y
WHERE x.A = y.B)
[ y | y ← S, x.A = y.B ]bag
Torsten Grust
20
U Tübingen
Comprehensible SQL
SELECT COUNT(*)
FROM
R x
WHERE EXISTS (SELECT y
FROM
S y
WHERE x.A = y.B)
[ true | _ ← [ y | y ← S, x.A = y.B ]bag ]some
Torsten Grust
20
U Tübingen
Comprehensible SQL
SELECT COUNT(*)
FROM
R x
WHERE EXISTS (SELECT y
FROM
S y
WHERE x.A = y.B)
[ 1 | x ← R,
[ true | _ ← [ y | y ← S, x.A = y.B ]bag ]some ]sum
Torsten Grust
20
U Tübingen
Comprehensible SQL
SELECT COUNT(*)
FROM
R x
WHERE EXISTS (SELECT y
FROM
S y
WHERE x.A = y.B)
[ 1 | x ← R,
[ true | _ ← [ y | y ← S, x.A = y.B ]bag ]some ]sum
[ 1 | x ← R, [ x.A = y.B | y ← S ]some ]sum
Torsten Grust
20
U Tübingen
Comprehension Unnesting
[ e | qs1, v ← [ ]N, qs3 ]M
[ e | qs1, v ← [ e2 ]N, qs3 ]M
[ e | qs1, v ← [ e2 | qs2 ]N, qs3 ]M
[ e | qs1, [ e2 | qs2 ]some, qs3 ]M
Torsten Grust
21
U Tübingen
Comprehension Unnesting
[ e | qs1, v ← [ ]N, qs3 ]M
[ ]M
[ e | qs1, v ← [ e2 ]N, qs3 ]M
[ e[e2/v] | qs1, qs3[e2/v] ]M
[ e | qs1, v ← [ e2 | qs2 ]N, qs3 ]M
[ e[e2/v] | qs1, qs2, qs3[e2/v] ]M
[ e | qs1, [ e2 | qs2 ]some, qs3 ]M
[ e | qs1, qs2, e2, qs3 ]M
Torsten Grust
21
(⊕M idempotent)
U Tübingen
When Syntax Distracts
On Optimizing an SQL-like Nested Query
W. Kim, ACM TODS, 1982
Torsten Grust
22
U Tübingen
When Syntax Distracts
On Optimizing an SQL-like Nested Query
W. Kim, ACM TODS, 1982
Torsten Grust
22
U Tübingen
When Syntax Distracts
On Optimizing an SQL-like Nested Query
W. Kim, ACM TODS, 1982
Implemented in most RDBMSs to this day
Torsten Grust
22
U Tübingen
When Syntax Distracts
• Syntactic classification of nested SQL queries into
types N, Nx, D, J, A, JA, JA(NA), JA(AA), JA(AN), …
• Classes are associated with their particular
SQL–level unnesting rewrites.
Torsten Grust
23
U Tübingen
When Syntax Distracts
• Syntactic classification of nested SQL queries into
types N, Nx, D, J, A, JA, JA(NA), JA(AA), JA(AN), …
• Classes are associated with their particular
SQL–level unnesting rewrites.
Torsten Grust
23
U Tübingen
When Syntax Distracts
SELECT DISTINCT f(x)
FROM
R AS x
WHERE p(x) IN (SELECT g(y)
FROM
S AS y
WHERE q(x,y))
Torsten Grust
24
U Tübingen
When Syntax Distracts
SELECT DISTINCT f(x)
FROM
R AS x
WHERE p(x) IN (SELECT g(y)
FROM
S AS y
WHERE q(x,y))
[ f(x) | x ← R,
[ p(x) = v | v ← [ g(y)| y ← S, q(x,y) ]bag ]some ]set
Torsten Grust
24
U Tübingen
When Syntax Distracts
SELECT DISTINCT f(x)
FROM
R AS x
WHERE p(x) IN (SELECT g(y)
FROM
S AS y
WHERE q(x,y))
[ f(x) | x ← R,
[ p(x) = g(y) | y ← S, q(x,y) ]some ]set
Torsten Grust
24
U Tübingen
When Syntax Distracts
SELECT DISTINCT f(x)
FROM
R AS x
WHERE p(x) IN (SELECT g(y)
FROM
S AS y
WHERE q(x,y))
[ f(x) | x ← R, y ← S, q(x,y), p(x) = g(y) ]set
SELECT
FROM
WHERE
AND
Torsten Grust
DISTINCT f(x)
R AS x, S AS y
q(x,y)
p(x) = g(y)
24
U Tübingen
A Zoo of
Query Representations
Groupwise
Processing
of Relational
Queries
Damianos Chatziantoniou”
Kenneth A. ROSS*
Department of Computer Science, Columbia University
damia,nos,[email protected]
Groupwise Processing of Relational Queries
D. Chatziantoniou, K.A. Ross, VLDB 1997
1
Abstract
Torsten
In this paper, we define and examine a particular class of queries called group queries. Group
queries are natural queries in many decisionsupport applications. The main characteristic of a
group query is that it can be executed in a groupby-group fashion. In other words, the underlying
relation(s) can be partitioned (based on some set
of attributes) into disjoint groups, and each group
can be processed separately. We give a syntactic
Grustcriterion to identify these queries and prove its
Introduction
With the recent interest in decision support systems and
data warehousing has come a demand for techniques to
evaluate and optimize very complex relational queries involving both aggregation and joins. Current commercial
systems do not find good plans for many very complex
queries.
In this paper we examine a particular class of complex
Many complex decision
queries, called group queries.
support queries describe the following idea: for each value z
in a dimension D (e.g. for each customer), evaluate a query
Q’. This query Q’ can be something simple (e.g. compute
avg(sales)) or complex (e.g. include joins, selections, furU Tübingen
25 ther aggregations, etc). To specify this kind of complex
query in SQL, one has to embed D within many places
A Zoo of
Query Representations
Groupwise
Processing
of Relational
Queries
Damianos Chatziantoniou”
Kenneth A. ROSS*
Department of Computer Science, Columbia University
damia,nos,[email protected]
Groupwise Processing of Relational Queries
D. Chatziantoniou, K.A. Ross, VLDB 1997
1
Introduction
With the recent interest in decision support systems and
data warehousing has come a demand for techniques to
evaluate and optimize very complex relational queries involving both aggregation and joins. Current commercial
systems do not find good plans for many very complex
queries.
In this paper we examine a particular class of complex
Many complex decision
queries, called group queries.
support queries describe the following idea: for each value z
in a dimension D (e.g. for each customer), evaluate a query
Q’. This query Q’ can be something simple (e.g. compute
avg(sales)) or complex (e.g. include joins, selections, furU Tübingen
25 ther aggregations, etc). To specify this kind of complex
query in SQL, one has to embed D within many places
SELECT
f(x), agg(g(x))
In this paper, we define FROM
and examine a particuR
AS
x
lar class of queries called group queries. Group
queries are natural queries
in many decisionGROUP
BY f(x)
support applications. The main characteristic of a
Abstract
Torsten
group query is that it can be executed in a groupby-group fashion. In other words, the underlying
relation(s) can be partitioned (based on some set
of attributes) into disjoint groups, and each group
can be processed separately. We give a syntactic
Grustcriterion to identify these queries and prove its
A Zoo of
Query Representations
Groupwise
Processing
of Relational
Queries
Damianos Chatziantoniou”
Kenneth A. ROSS*
Department of Computer Science, Columbia University
damia,nos,[email protected]
Groupwise Processing of Relational Queries
D. Chatziantoniou, K.A. Ross, VLDB 1997
1
Abstract
In this paper, we define and examine a particular class of queries called group queries. Group
queries are natural queries in many decisionsupport applications. The main characteristic of a
group query is that it can be executed in a groupby-group fashion. In other words, the underlying
relation(s) can be partitioned (based on some set
of attributes) into disjoint groups, and each group
can be processed separately. We give a syntactic
Grustcriterion to identify these queries and prove its
Introduction
With the recent interest in decision support systems and
data warehousing has come a demand for techniques to
evaluate and optimize very complex relational queries involving both aggregation and joins. Current commercial
systems do not find good plans for many very complex
queries.
In this paper we examine a particular class of complex
Many complex decision
queries, called group queries.
support queries describe the following idea: for each value z
in a dimension D (e.g. for each customer), evaluate a query
Q’. This query Q’ can be something simple (e.g. compute
avg(sales)) or complex (e.g. include joins, selections, furU Tübingen
25 ther aggregations, etc). To specify this kind of complex
query in SQL, one has to embed D within many places
[ ⟨f(x), [ g(y) | y ← R, f(y) = f(x) ]agg⟩ | x ← R ]set
Torsten
A Zoo of
Query Representations
Groupwise
Processing
of Relational
Queries
Damianos Chatziantoniou”
Kenneth A. ROSS*
Department of Computer Science, Columbia University
damia,nos,[email protected]
Groupwise Processing of Relational Queries
D. Chatziantoniou, K.A. Ross, VLDB 1997
1
Abstract
Introduction
With the recent interest in decision support systems and
data warehousing has come a demand for techniques to
evaluate and optimize very complex relational queries involving both aggregation and joins. Current commercial
systems do not find good plans for many very complex
queries.
In this paper we examine a particular class of complex
Many complex decision
queries, called group queries.
support queries describe the following idea: for each value z
in a dimension D (e.g. for each customer), evaluate a query
Q’. This query Q’ can be something simple (e.g. compute
avg(sales)) or complex (e.g. include joins, selections, furU Tübingen
25 ther aggregations, etc). To specify this kind of complex
query in SQL, one has to embed D within many places
Q f g agg R ≡
In this paper, we define and examine a particular class of queries called group queries. Group
queries are natural queries in many decisionsupport applications. The main characteristic of a
group query is that it can be executed in a groupby-group fashion. In other words, the underlying
relation(s) can be partitioned (based on some set
of attributes) into disjoint groups, and each group
can be processed separately. We give a syntactic
Grustcriterion to identify these queries and prove its
[ ⟨f(x), [ g(y) | y ← R, f(y) = f(x) ]agg⟩ | x ← R ]set
Torsten
A Zoo of
Query Representations
Torsten Grust
26
U Tübingen
A Zoo of
Query Representations
Torsten Grust
26
U Tübingen
A Zoo of
Query Representations
Torsten Grust
26
U Tübingen
A Zoo of
Query Representations
Torsten Grust
26
U Tübingen
A Zoo of
Query Representations
Torsten Grust
26
U Tübingen
A Zoo of
Query Representations
P
P
P
SELECT agg(g(x))
FROM
P AS x
Torsten Grust
26
U Tübingen
A Zoo of
Query Representations
P
P
P
Q’ g agg P ≡
[ g(y) | y ← P ]agg
Torsten Grust
26
U Tübingen
A Zoo of
Query Representations
P
P
P
Q’ g agg P ≡
[ g(y) | y ← P ]agg
Torsten Grust
26
U Tübingen
A Zoo of
Query Representations
Torsten Grust
27
U Tübingen
be posed separately on each partition.
Query Q4 in Section 2.4.
2
We shall discuss
A Zoo of
Query
Framework
Theoretical Representations
In this section, we introduce our terminology and define
what we mean by a group query. We give a syntactic
this
t
tha
ve
pro
d
an
es
eri
qu
up
gro
g
yin
ntif
ide
for
on
eri
crit
condition is sufficient. We also show that every group query
can be expressed in a form that satisfies our criterion.
2.1
Assumptions
and
Terminology
We assume that queries are written in terms of views, with
no subqueries. This is a valid assumption since there are
many ways to rewrite a subquery as a join of two (or more)
the
t
tha
e
um
ass
y
iall
init
We
G].
LS
SP
,
y87
Da
2,
m8
[Ki
views
R. (Multiple relatio
Torsten
U ns
Tübingen
27ation
tabase contains a single rel
daGrust
e are
mbe
anypowsed
ll discuss
sha
ays sep
iona. joinWe
to ara
partitas
rewtely
rite on
a suea
bqchuery
of two (or more)
viQu
ewery
s [Kim
Day
ctio
87n, 2.4
Q482in, Se
SP.LSG]. We initially assume that the
database contains a single relation R.
(Multiple relations
will be considered in Section 2.4or
Framew .) k R may itself be a view
al
tic
re
eo
Th
2
or the result of another query, but from
ourgypointandof de
vi
ew
it
fine
olo
min
ter
r
ou
uce
intrulod
is Intreathis
ted sec
astion
an, enwe
caps
ated table. (I.e., if R was a view,
tactic
syn
a
e
giv
We
.
ery
qu
up
gro
a
by
an
me
whatwewe
then
don’t consider unfolding the definition
o
f
R
in
to
this
t
tha
ve
pro
d
an
es
eri
qu
up
gro
g
yin
ntif
ide
for
on
eri
qucrit
eries over R.)
that every group query
w
sho
o
als
We
t.
en
fici
suf
is
on
diti
con
We shall define below the notion of a
qu
er
y
ap. h. A
erigron
crit
r
ou
es
isfi
sat
t
tha
m
for
a
in
sed
res
exp
be
can
query graph has nodes that are relationa
l operations. We
consider three kinds of relational operat
io
ns
:
y
log
ino
rm
Te
d
an
ns
ptio
sum
As
2.1
Basic Blocks A basic block is some
combination of
projWe
ectioass
nsumeantha
, with
ws
d se
terinmsof ofrevie
ioes
ns are
applwri
qucteri
t le
iedttento ina jo
lations. In
are
SQno
re
the
L su
ce
sin
n
chquop
ptio
um
id seass
ations
This arise aexval
es.
erier
sub
pres
d as SELECT-FROM-WHERE
re)
mo
(or
two
querma
of
join
iesny wwa
a
as
ery
ithys
outto rew
aggrrite
egataessuboqu
r attribute renaming. A base
the
t
tha
e
um
ass
y
relavie
iall
init
tionws is[Kial
We
G].
LS
SP
,
y87
so2, trea
Date
m8
d as a basic block.
ns
relatio
ple
ulti
(M
R.
n
atio
rel
gle
A
Torsten
Grust
U
Tübingen
rease
27
gationcontainsBloacksin
tab
dagg
s An aggregation bloc
A Zoo of
Query Representations
e are
mbe
anypowsed
ll discuss
sha
ays sep
iona. joinWe
to ara
partitas
rewtely
rite on
a suea
bqchuery
of two (or more)
viQu
ewery
s [Kim
Day
ctio
87n, 2.4
Q482in, Se
SP.LSG]. We initially assume that the
database contains a single relation R.
(Multiple relations
will be considered in Section 2.4or
Framew .) k R may itself be a view
al
tic
re
eo
Th
2
or the result of another query, but from
ourgypointandof de
vi
ew
it
fine
olo
min
ter
r
ou
uce
intrulod
is Intreathis
ted sec
astion
an, enwe
caps
ated table. (I.e., if R was
tactic
syn
a
e
giv
We
.
ery
qu
up
gro
a
by
an
me
whatwewe
then
don’t consider unfolding the definition
o
f
R
in
to
this
t
tha
ve
pro
d
an
es
eri
qu
up
gro
g
yin
ntif
ide
for
on
eri
qucrit
eries over R.)
that every group query
w
sho
o
als
We
t.
en
fici
suf
is
on
diti
con
We shall define below the notion of a
qu
er
y
ap. h. A
erigron
crit
r
ou
es
isfi
sat
t
tha
m
for
a
in
sed
res
exp
be
can
query graph has nodes that are relationa
l operations. We
consider three kinds of relational operat
io
ns
:
y
log
ino
rm
Te
d
an
ns
ptio
sum
As
2.1
Basic Blocks A basic block is some
combination of
projWe
ectioass
nsumeantha
, with
ws
d se
terinmsof ofrevie
ioes
ns are
applwri
qucteri
t le
iedttento ina jo
lations. In
are
SQno
re
the
L su
ce
sin
n
chquop
ptio
um
id seass
ations
This arise aexval
es.
erier
sub
pres
d
as
S
E
LE
C
T
-FROM-WHERE
SQL
surface
syntax,
relational
algebra,
re)
mo
(or
two
querma
of
join
iesny wwa
a
as
ery
ithys
outto rew
aggrrite
egataessuboqu
r
at
tr
ib
ut
e re
naming. A base
query
graphs
+
annotations,
iteration
the
t
tha
e
um
ass
y
relavie
iall
init
tionws is[Kial
We
G].
LS
SP
,
y87
so2, trea
Date
m8
d as a basic block.
relations
ple
ulti
(M
R.
n
atio
rel
gle
A
rease
gationcontainsBloacksin
tab
dagg
s An aggregation bloc
s are wr
enting t
ntionally
with a
bqueries.
i
h
t
t
e
e
n
d
r
q
a
u
This is a
w the g
in terms
For singl
raph with ery result. We
of views,
e-relation
valid ass
ways to
rewrite a
umption
with
edges “p
databases
sink nod
subquery
since the
ointing
,
e
Kim82, D
there wil
at the to
re are
as a join
ay87, SP
l always
p of the
query gra
of two (o
LSG]. W
be a s
p
e contai
p
h
i
s for que
cture. F
r more)
e initially
ns a sin
r
o
igure 1
ies Ql, a
f V2 and
assume t
gle relati
shows
considere
n
V
d
3
Q
hat the
f
on R.
r
2
o
.
m
N
d in Se
q
o
uery Ql
tice the
and aggr
(Multiple
ction 2.4
e
separat
i
esult of
n
g
t
a
o
tion blo
relations
basic blo
.) R ma
another
c
k
cks (V2B
s (V2A,
y itself
query, bu
, V3
b
V
as an en
e
3
a
A
t
)
v
f
.
r
i
o
e
m
w
capsulate
our point
d table.
don’t co
of view i
nsider u
(I.e., if R
t
nfolding
w
a
ver R.)
s a view,
the defin
ition of
all define
R into
below th
e notion
ph has n
a view,
of a quer
odes tha
y graph.
t are rel
hree kind
a
t
i
o
A
nal ope
s of rela
r
t
a
i
o
t
i
n
o
al opera
ns. We
Blocks
tions:
A basic
block is
and selec
some co
tions app
mbination
lied to a
perations
of
join of re
are expre
lations.
ssed as
out agg
In
SELECTregates
FROM-W
or attribu
so treated
HERE
t
e
renaming.
as a basi
c block.
A base
n
Blocks
An aggre
on ope
gation b
ration s
lock is a
pecifying
d a list
sina set of
of aggreg
grouping
ate func
e groups
tions to
. The
be coma
ggregation
E), , O2:
a selectio
join cond
c
n.
a
n
o
p
I
n
t
i
itions.
o
n
S
a
QL, such
ELECT-FR
lly
OM-GRO
operations
0 : join c
UPBY-HA
use inclu
ondition.
a
r
e
V
ING
des all
queries w
(4 query Q
grouping
here
s), and
l
attributes
the FRO
Mclause
(plus
(b) query Q
contains
2
Figure
a single
Set block
1: Query
s expres
graphs
s the se
on, inters
t-oriented
ection a
operand differe
hema. T
Definition27
n
ce of rel
Torsten
U Tübingen
heGrust
correspon
2
a
.
2
t
i
:
o
n
s
(Pa
ECT
ding
fr
A Zoo of
Query Representations
A Uniform
Query Representation
Torsten Grust
28
U Tübingen
A Uniform
Query Representation
Q’ g agg P = [ g(y) | y ← P ]agg
partition f xs = [ ⟨f(x), [ y | y ← xs, f(x) = f(y) ]M⟩ | x ← xs ]set
map f xs = [ f(x) | x ← xs ]M
Torsten Grust
28
U Tübingen
A Uniform
Query Representation
Q’ g agg P = [ g(y) | y ← P ]agg
partition f xs = [ ⟨f(x), [ y | y ← xs, f(x) = f(y) ]M⟩ | x ← xs ]set
map f xs = [ f(x) | x ← xs ]M
map (λ⟨x,P⟩.⟨x, Q’ g agg P⟩ (partition f xs)
Torsten Grust
28
U Tübingen
A Uniform
Query Representation
Q’ g agg P = [ g(y) | y ← P ]agg
partition f xs = [ ⟨f(x), [ y | y ← xs, f(x) = f(y) ]M⟩ | x ← xs ]set
map f xs = [ f(x) | x ← xs ]M
map (λ⟨x,P⟩.⟨x, Q’ g agg P⟩ (partition f xs)
[ ⟨f(x), [ g(y) | y ← R, f(y) = f(x) ]agg⟩ | x ← R ]set
Torsten Grust
28
U Tübingen
A Uniform
Query Representation
Q’ g agg P = [ g(y) | y ← P ]agg
partition f xs = [ ⟨f(x), [ y | y ← xs, f(x) = f(y) ]M⟩ | x ← xs ]set
map f xs = [ f(x) | x ← xs ]M
map (λ⟨x,P⟩.⟨x, Q’ g agg P⟩ (partition f xs)
SELECT
f(x), agg(g(x))
FROM
R AS x
GROUP BY f(x)
Torsten Grust
28
U Tübingen
XPath
Torsten Grust
29
U Tübingen
XPath Comprehensions
Torsten Grust
30
U Tübingen
XPath Comprehensions
/descendant::a[following::b]/child::c
Torsten Grust
30
U Tübingen
XPath Comprehensions
/descendant::a[following::b]/child::c
1. Normalize, simplify, flip XPath step expressions
Torsten Grust
30
U Tübingen
XPath Comprehensions
/descendant::a[following::b]/child::c
1. Normalize, simplify, flip XPath step expressions
2. Compile XPath into queries over tabular XML encoding
Torsten Grust
30
U Tübingen
XPath Comprehensions
/descendant::a[following::b]/child::c
1. Normalize, simplify, flip XPath step expressions
2. Compile XPath into queries over tabular XML encoding
xpath (/p) c = xpath p (root c)
xpath (p1/p2) c = [ n’ | n ← xpath p1 c, n’ ← xpath p2 n ]X
xpath (p[q]) c = [ n | n ← xpath p c, [ true | _ ← xpath q n ]some ]X
xpath (ax::t) c = step (ax::t) c
Torsten Grust
30
U Tübingen
A Tabular XML Encoding
<a>
<b><c><d/>e</c></b>
<f><!--g-->
<h><i/><j/></h>
</f>
</a>
Torsten Grust
31
U Tübingen
A Tabular XML Encoding
a
b
f
h
c
g
d
Torsten Grust
e
i
j
31
U Tübingen
A Tabular XML Encoding
0a9
a
1b3
b
5f8
f
7h7
h
2c2
c
6g4
g
3d0
d
Torsten Grust
4e1
e
8i5
i
9j6
j
31
U Tübingen
A Tabular XML Encoding
post
0a9
a
1b3
b
a
5f8
f
6g4
g
4e1
e
j
5
8i5
i
9j6
j
c
b
d
Torsten Grust
h
7h7
h
2c2
c
3d0
d
f
31
g
e
5
i
pre
U Tübingen
A Tabular XML Encoding
post
0a9
a
1b3
b
a
5f8
f
6g4
g
4e1
e
j
5
8i5
i
9j6
j
c
b
d
Torsten Grust
h
7h7
h
2c2
c
3d0
d
f
31
g
e
5
i
pre
U Tübingen
A Tabular XML Encoding
post
0a9
a
1b3
b
a
5f8
f
h
7h7
h
2c2
c
6g4
g
3d0
d
f
4e1
e
j
5
8i5
i
9j6
j
c
b
d
g
i
e
pre
5
step (descendant::t) c =
[ n | n ← doc, pre c < pre n, post c > post n, tag n = t ]X
Torsten Grust
31
U Tübingen
A Tabular XML Encoding
post
0a9
a
1b3
b
a
5f8
f
h
7h7
h
2c2
c
6g4
g
3d0
d
f
4e1
e
j
5
8i5
i
9j6
j
c
b
d
g
i
e
pre
5
step (descendant::t) c =
[ n | n ← doc, pre c < pre n, post c > post n, tag n = t ]X
Torsten Grust
31
U Tübingen
A Tabular XML Encoding
post
0a9
a
1b3
b
a
5f8
f
h
7h7
h
2c2
c
6g4
g
3d0
d
f
4e1
e
j
5
8i5
i
9j6
j
c
b
d
g
i
e
pre
5
step (descendant::t) c =
[ n | n ← doc, pre c < pre n, post c > post n, tag n = t ]X
step (ancestor::t) c =
[ n | n ← doc, pre c > pre n, post c < post n, tag n = t ]X
Torsten Grust
31
U Tübingen
XPath: Looking Forward
XPath: Looking Forward
D. Olteanu et al., XMLDM (EDBT 2002), March 2002
Torsten Grust
32
U Tübingen
XPath: Looking Forward
XPath: Looking Forward
D. Olteanu et al., XMLDM (EDBT 2002), March 2002
a
b
f
h
c
g
d
e
Torsten Grust
i
j
32
U Tübingen
XPath: Looking Forward
XPath: Looking Forward
D. Olteanu et al., XMLDM (EDBT 2002), March 2002
a
b
f
/descendant::g/preceding::c
h
c
g
d
e
Torsten Grust
i
j
32
U Tübingen
XPath: Looking Forward
XPath: Looking Forward
D. Olteanu et al., XMLDM (EDBT 2002), March 2002
a
b
c
d
f
←
e
Torsten Grust
/descendant::g/preceding::c
h
g
i
j
32
U Tübingen
XPath: Looking Forward
XPath: Looking Forward
D. Olteanu et al., XMLDM (EDBT 2002), March 2002
a
b
d
←
c
f
e
Torsten Grust
/descendant::g/preceding::c
h
≡
g
i
j
/descendant::c[following::g]
32
U Tübingen
XPath: Looking Forward
Torsten Grust
33
U Tübingen
XPath: Looking Forward
Torsten Grust
33
U Tübingen
Comprehending XPath
Torsten Grust
34
U Tübingen
Comprehending XPath
/descendant::g/preceding::c
Torsten Grust
34
U Tübingen
Comprehending XPath
/descendant::g/preceding::c
[ v’ | v ← doc, tag v = ’g’, v’ ← doc,
pre v’ < pre v, post v’ < post v, tag v’ = ’c’ ]X
Torsten Grust
34
U Tübingen
Comprehending XPath
/descendant::g/preceding::c
[ v’ | v ← doc, tag v = ’g’, v’ ← doc,
pre v’ < pre v, post v’ < post v, tag v’ = ’c’ ]X
/descendant::c[following::g]
Torsten Grust
34
U Tübingen
Comprehending XPath
/descendant::g/preceding::c
[ v’ | v ← doc, tag v = ’g’, v’ ← doc,
pre v’ < pre v, post v’ < post v, tag v’ = ’c’ ]X
SELECT
FROM
WHERE
AND
ORDER BY
Torsten Grust
DISTINCT v’
doc v, doc v’
tag v = ’g’ AND tag v’ = ’c’
pre v’ < pre v AND post v’ < post v
pre v’
34
U Tübingen
BRING BACK
MONAD
COMPREHENSIONS
Torsten Grust
35
U Tübingen
Comprehensions in Haskell
Haskell
Torsten Grust
36
U Tübingen
Comprehensions in Haskell
Haskell
!
Torsten Grust
36
U Tübingen
Comprehensions in Haskell
Haskell
!
1990
Torsten Grust
1997
2003
36
2007
2011
U Tübingen
Comprehensions in Haskell
Haskell
!
1990
Torsten Grust
1997
2003
36
2007
2011
U Tübingen
Comprehensions in Haskell
Haskell
Comprehending Monads
!
1990
Torsten Grust
1997
2003
36
2007
2011
U Tübingen
Comprehensions in Haskell
Haskell
Comprehending Monads
27
!
1990
Torsten Grust
1997
2003
36
2007
2011
U Tübingen
Comprehensions in Haskell
Haskell
Comprehending Monads
27
!
Haskell 1.4
1990
Torsten Grust
1997
2003
36
2007
2011
U Tübingen
Comprehensions in Haskell
Haskell
Comprehending Monads
27
!
Haskell 98
Haskell 1.4
1990
Torsten Grust
1997
2003
36
2007
2011
U Tübingen
Comprehensions in GHC
Haskell
Comprehending Monads
27
!
Haskell 98
Haskell 1.4
1990
Torsten Grust
1997
2003
36
2007
2011
U Tübingen
Comprehensions in GHC
Haskell
Comprehending Monads
27
!
Haskell 98
Comprehensive
Comprehensions
Haskell 1.4
1990
Torsten Grust
1997
2003
36
2007
2011
U Tübingen
Comprehensions in GHC
Haskell
Comprehending Monads
Bringing Back
Monad Comprehensions
27
!
Haskell 98
Comprehensive
Comprehensions
Haskell 1.4
1990
Torsten Grust
1997
2003
36
2007
2011
U Tübingen
Movie Plot Line
Climax
Meeting
!
Pinch
Turning Point
Inciting Incident
5 min
Torsten Grust
120 min
36
U Tübingen
Comprehensi{ve, ons}
Comprehensive Comprehensions
P. Wadler, S. Peyton-Jones, Haskell Workshop, October 2007
Torsten Grust
37
U Tübingen
Comprehensi{ve, ons}
Comprehensive Comprehensions
P. Wadler, S. Peyton-Jones, Haskell Workshop, October 2007
[
|
,
,
,
,
]
Torsten Grust
(the dept, maximum salary)
(name, dept, salary) <- employees
then group by dept using groupWith
length dept > 10
then sortWith by Down (sum salary)
then take 5
37
U Tübingen
Comprehensi{ve, ons}
Comprehensive Comprehensions
P. Wadler, S. Peyton-Jones, Haskell Workshop, October 2007
AGGR
GROUP BY
[
|
,
,
,
,
]
(the dept, maximum salary)
(name, dept, salary) <- employees
then group by dept using groupWith
HAVING
length dept > 10
then sortWith by Down (sum salary)
then take 5
LIMIT
Torsten Grust
FROM
ORDER BY
37
ASC/DESC
U Tübingen
Comprehensi{ve, ons}
Comprehensive Comprehensions
P. Wadler, S. Peyton-Jones, Haskell Workshop, October 2007
[
|
,
,
]
row patterns!
sum salary
(name, “MS”, salary) <- employees
then group using runs 3
OVER
then take 5
Not shown: set operations, joins, WITH…RECURSIVE, …
Torsten Grust
37
U Tübingen
Database–Supported Haskell
Haskell Heap
DATA
DBMS
Torsten Grust
38
U Tübingen
Database–Supported Haskell
Haskell Heap
DATA
DATA
DBMS
Torsten Grust
38
U Tübingen
Database–Supported Haskell
Haskell Heap
DATA
Haskell
d
DATA
DBMS
Torsten Grust
38
U Tübingen
Database–Supported Haskell
Haskell Heap
Haskell
DATA
DBMS
Torsten Grust
38
U Tübingen
Database–Supported Haskell
Haskell Heap
SQL
DATA
DBMS
Torsten Grust
38
U Tübingen
Database–Supported Haskell
Haskell Heap
SQL
SQL
DATA
DBMS
Torsten Grust
38
U Tübingen
Database–Supported Haskell
Haskell Heap
DATA
SQL
SQL
d
DBMS
Torsten Grust
38
U Tübingen
Database–Supported Haskell
Haskell Heap
d
DATA
SQL
SQL
d
DBMS
Torsten Grust
38
U Tübingen
Database–Supported Haskell
Haskell Heap
haskell
DATA
SQL
SQL
d
d
DBMS
Torsten Grust
38
U Tübingen
Database–Supported Haskell
Torsten Grust
39
U Tübingen
Database–Supported Haskell
-— rolling minimum (mins [3,4,1,7] = [3,3,1,1])
mins :: Ord a => Q [a] -> Q [a]
mins xs =
[ minimum [ y | (y,j) <- #xs, j <= i ] | (_,i) <- #xs ]
-— margin: current value - minimum value up to now
margins :: (Ord a, Num a) => Q [a] -> Q [a]
margins xs = [ x - y | (x,y) <- zip xs (mins xs) ]
-— our profit is the maximum margin obtainable
profit :: (Ord a, Num a) => Q [a] -> Q [a]
profit xs = maximum (margins xs)
-— best profit obtainable for stock on given date
bestProfit :: Text -> Date -> Q [Trade] -> Q Double
bestProfit stock date trades =
profit [ price t | t <- sortWith ts trades,
id t == stock, day t == date ]
Torsten Grust
39
U Tübingen
Database–Supported Haskell
-— rolling minimum (mins [3,4,1,7] = [3,3,1,1])
mins :: Ord a => Q [a] -> Q [a]
mins xs =
[ minimum [ y | (y,j) <- #xs, j <= i ] | (_,i) <- #xs ]
-— margin: current value - minimum value up to now
margins :: (Ord a, Num a) => Q [a] -> Q [a]
margins xs = [ x - y | (x,y) <- zip xs (mins xs) ]
-— our profit is the maximum margin obtainable
Trades
profit :: (Ord a, Num a) => Q [a] -> Q [a]
id ts day
profit xs = maximum (margins xs)
ACME 1 7
ACME 2
date
ACME 3
ACME 4
Double
⠇
⠇
-— best profit obtainable for stock on given
bestProfit :: Text -> Date -> Q [Trade] -> Q
bestProfit stock date trades =
profit [ price t | t <- sortWith ts trades,
id t == stock, day t == date ]
Torsten Grust
39
/1/15
7/1/15
7/1/15
7/1/15
⠇
price
3.0
4.0
1.0
7.0
⠇
U Tübingen
Database–Supported Haskell
-— rolling minimum (mins [3,4,1,7] = [3,3,1,1])
mins :: Ord a => Q [a] -> Q [a]
mins xs =
[ minimum [ y | (y,j) <- #xs, j <= i ] | (_,i) <- #xs ]
-— margin: current value - minimum value up to now
margins :: (Ord a, Num a) => Q [a] -> Q [a]
margins xs = [ x - y | (x,y) <- zip xs (mins xs) ]
-— our profit is the maximum margin obtainable
profit :: (Ord a, Num a) => Q [a] -> Q [a]
profit xs = maximum (margins xs)
-— best profit obtainable for stock on given date
bestProfit :: Text -> Date -> Q [Trade] -> Q Double
bestProfit stock date trades =
profit [ price t | t <- sortWith ts trades,
id t == stock, day t == date ]
Torsten Grust
39
U Tübingen
Database–Supported Haskell
-— rolling minimum (mins [3,4,1,7] = [3,3,1,1])
mins :: Ord a => Q [a] -> Q [a]
mins xs =
[ minimum [ y | (y,j) <- #xs, j <= i ] | (_,i) <- #xs ]
-— margin: current value - minimum value up to now
margins :: (Ord a, Num a) => Q [a] -> Q [a]
margins xs = [ x - y | (x,y) <- zip xs (mins xs) ]
-— our profit is the maximum margin obtainable
profit :: (Ord a, Num a) => Q [a] -> Q [a]
profit xs = maximum (margins xs)
-— best profit obtainable for stock on given date
bestProfit :: Text -> Date -> Q [Trade] -> Q Double
bestProfit stock date trades =
profit [ price t | t <- sortWith ts trades,
id t == stock, day t == date ]
Torsten Grust
39
U Tübingen
Database–Supported Haskell
—- SQL code generated from Haskell source
SELECT MAX(margins.price - margins.min)
FROM
(SELECT t.price,
MIN(t.price)
OVER (ORDER BY t.ts ROW BETWEEN
UNBOUNDED PRECEDING
AND CURRENT ROW)
FROM trades AS t
WHERE t.id = ‘ACME’
AND
t.day = ’07/01/2015’
) AS margins(price,min)
Torsten Grust
39
U Tübingen
Comprehensions
Yield Independent Work
Torsten Grust
40
U Tübingen
Comprehensions
Yield Independent Work
[
[ f y | y ← g x ] | x ← xs ]
f :: a → b
Torsten Grust
40
U Tübingen
Comprehensions
Yield Independent Work
[ f [ f y | y ← g x ] | x ← xs ]
f :: a → b
Torsten Grust
40
U Tübingen
Comprehensions
Yield Independent Work
[ f 1 [ f y | y ← g x ] | x ← xs ]
f :: a → b
1
f :: [a] → [b]
Torsten Grust
40
U Tübingen
Comprehensions
Yield Independent Work
2
f [
[ f y | y ← g x ] | x ← xs ]
f :: a → b
1
f :: [a] → [b]
2
f :: [[a]] → [[b]]
Torsten Grust
40
U Tübingen
Comprehensions
Yield Independent Work
2
f [
g x | x ← xs ]
f :: a → b
1
f :: [a] → [b]
2
f :: [[a]] → [[b]]
Torsten Grust
40
U Tübingen
Comprehensions
Yield Independent Work
2
1
f (g
xs)
f :: a → b
1
f :: [a] → [b]
2
f :: [[a]] → [[b]]
Torsten Grust
40
U Tübingen
Comprehensions
Yield Independent Work
n
[ f e | x ← xs ]
Torsten Grust
⇝
40
f
n+1
[ e | x ← xs ]
U Tübingen
Nested Data Parallelism
n
[ f e | x ← xs ]
⇝
f
n+1
[ e | x ← xs ]
Implementation of a Portable Nested Data-Parallel Language
G. E. Blelloch et al., ACM PPoPP, May 1993
Torsten Grust
40
U Tübingen
The Flatter, the Better
Torsten Grust
41
U Tübingen
The Flatter, the Better
xss +2 yss
Torsten Grust
41
U Tübingen
The Flatter, the Better
xss +2 yss
[[19, 0],[30],[11,10, 7]]
+2 [[ 0, 4],[12],[13, 2, 3]]
Torsten Grust
41
U Tübingen
The Flatter, the Better
xss +2 yss
[[19, 0],[30],[11,10, 7]]
+2 [[ 0, 4],[12],[13, 2, 3]]
Torsten Grust
41
U Tübingen
The Flatter, the Better
[ 19, 0 , 30 , 11,10, 7 ]
1
+ [ 0, 4 , 12 , 13, 2, 3 ]
[
Torsten Grust
][ ][
41
]
forget
xss +2 yss
U Tübingen
The Flatter, the Better
[ 19, 4 , 42 , 24,12,10 ]
[
Torsten Grust
][ ][
41
]
forget
xss +2 yss
U Tübingen
The Flatter, the Better
Torsten Grust
41
imprint
[[19, 4],[42],[24,12,10]]
forget
xss +2 yss
U Tübingen
The Flatter, the Better
⇝
imprintn-1 (f 1 (forgetn-1 e))
[[19, 4],[42],[24,12,10]]
Torsten Grust
41
imprint
f e
forget
n
U Tübingen
The Flatter, the Better
n
f e
⇝
imprintn-1 (f 1 (forgetn-1 e))
[[19, 4],[42],[24,12,10]]
seg
Torsten Grust
pos
1
1
1
2
1
3
seg pos sum
1
1
2
3
3
3
41
1
2
3
4
5
6
19
4
42
24
12
10
U Tübingen
The Flatter, the Better
n
f e
⇝
imprintn-1 (f 1 (forgetn-1 e))
[[19, 4],[42],[24,12,10]]
seg
Torsten Grust
pos
1
1
1
2
1
3
seg pos sum
1
1
2
3
3
3
41
1
2
3
4
5
6
19
4
42
24
12
10
U Tübingen
The Flatter, the Better
n
f e
⇝
imprintn-1 (f 1 (forgetn-1 e))
[[19, 4],[42],[24,12,10]]
seg
Torsten Grust
pos
1
1
1
2
1
3
seg pos sum
1
1
2
3
3
3
41
1
2
3
4
5
6
19
4
42
24
12
10
U Tübingen
The Flatter, the Better
n
f e
⇝
imprintn-1 (f 1 (forgetn-1 e))
[[19, 4],[42],[24,12,10]]
seg
Torsten Grust
pos
1
1
1
2
1
3
seg pos sum
1
1
2
3
3
3
41
1
2
3
4
5
6
19
4
42
24
12
10
U Tübingen
Database Systems:
1
Designed to Implement _
Torsten Grust
42
U Tübingen
Database Systems:
1
Designed to Implement _
+
1
Torsten Grust
42
U Tübingen
Database Systems:
1
Designed to Implement _
seg ⋯ x
1
1
2
3
3
3
+
1
Torsten Grust
42
19
0
30
11
0
7
y
0
4
12
13
2
3
U Tübingen
Database Systems:
1
Designed to Implement _
+
1
Torsten Grust
πsum: x+y(
42
seg ⋯ x
1
1
2
3
3
3
19
0
30
11
0
7
y
0
4
12
13
2
3
)
U Tübingen
Database Systems:
1
Designed to Implement _
+
1
πsum: x+y(
seg ⋯ x
1
1
2
3
3
3
19
0
30
11
0
7
y
0
4
12
13
2
3
)
1
⋉p
Torsten Grust
42
U Tübingen
Database Systems:
1
Designed to Implement _
+
1
πsum: x+y(
seg1 ⋯ x
1
⋉p
Torsten Grust
1
1
2
3
3
3
seg ⋯ x
1
1
2
3
3
3
19
0
30
11
0
7
y
0
4
12
13
2
3
)
seg2 ⋯ y
19
0
30
11
0
7
1
1
2
3
3
3
42
0
4
12
13
2
3
U Tübingen
Database Systems:
1
Designed to Implement _
+
1
πsum: x+y(
seg1 ⋯ x
1
⋉p
Torsten Grust
1
1
2
3
3
3
19
0
30
11
0
7
seg ⋯ x
1
1
2
3
3
3
19
0
30
11
0
7
y
0
4
12
13
2
3
)
seg2 ⋯ y
⋉p
42
1
1
2
3
3
3
0
4
12
13
2
3
U Tübingen
Database Systems:
1
Designed to Implement _
+
1
πsum: x+y(
seg ⋯ x
1
1
2
3
3
3
19
0
30
11
0
7
y
0
4
12
13
2
3
)
seg1 ⋯ x
1
⋉p
Torsten Grust
1
1
2
3
3
3
19
0
30
11
0
7
seg2 ⋯ y
⋉p ∧ seg = seg
1
42
2
1
1
2
3
3
3
0
4
12
13
2
3
U Tübingen
Database Systems:
1
Designed to Implement _
+
1
πsum: x+y(
seg ⋯ x
1
1
2
3
3
3
19
0
30
11
0
7
y
0
4
12
13
2
3
)
seg1 ⋯ x
1
⋉p
Torsten Grust
1
1
2
3
3
3
19
0
30
11
0
7
seg2 ⋯ y
⋉p ∧ seg = seg
1
42
2
1
1
2
3
3
3
0
4
12
13
2
3
U Tübingen
Plan Bundles
Instead of Query Avalanches
[(Int,[Str],[(Bool,[(Int,Int)])])]
Torsten Grust
43
U Tübingen
Plan Bundles
Instead of Query Avalanches
[(Int,[Str],[(Bool,[(Int,Int)])])]
Torsten Grust
43
U Tübingen
Plan Bundles
Instead of Query Avalanches
[[][[]]]
Torsten Grust
43
U Tübingen
Plan Bundles
Instead of Query Avalanches
[[][[]]]
Query Plan Bundle
Torsten Grust
43
U Tübingen
Plan Bundles
Instead of Query Avalanches
[[][[]]]
Query Plan Bundle
Torsten Grust
43
U Tübingen
Torsten Grust
44
U Tübingen