Short Presentation Title

SAP HANA DATABASE
Mihnea Andrei
SAP Products & Innovation HANA Platform
July 8, 2014
Public
Agenda
SAP
SAP HANA DB Background
Architecture
Column Store & Compression
Snapshot Isolation
Outlook
© 2014 SAP AG or an SAP affiliate company. All rights reserved.
Public
2
SAP
Who was SAP (before HANA)?
Sales Order
Management
Financial/Mgmt
Accounting
Production Planning
© 2014 SAP AG or an SAP affiliate company. All rights reserved.
Business
Intelligence
Talent Management
Public
4
74% of the world’s
transaction revenue
touches an SAP system.
© 2014 SAP AG or an SAP affiliate company. All rights reserved.
5
SAP
Business Applications – Database & Technology – Analytics – Cloud – Mobile
Annual revenue (IFRS) of € 16,82 billion
More than 253,500 customers in 188 countries
More than 66,500 employees – and locations in more than 130 countries
A 42-year history of innovation and growth as a true industry leader
© 2014 SAP AG or an SAP affiliate company. All rights reserved.
Public
6
© 2014 SAP AG or an SAP affiliate company. All rights reserved.
7
Products & Innovation HANA Platform
California Campus – Worldwide
© 2014 SAP AG or an SAP affiliate company. All rights reserved.
Public
8
SAP HANA DB Background
Why did we build HANA?
How Did the SAP Use Database Before HANA?
See “The SAP Transaction Model: Know Your Applications”, SIGMOD 2008 Industrial
Talk
 Database was mainly a dumb store …
– Retrieve/Store data (Open SQL, no stored procedures)
– Transaction commit, with locks held very briefly
– Operational utilities
 … because SAP kept the following in the application server:
–
–
–
–
–
Application logic
Business object-level locks
Queued updates
Data buffers
Indexes
With the HANA platform, computation-intensive data-centric operations are moved to
the Database
© 2014 SAP AG or an SAP affiliate company. All rights reserved.
Public
10
DRAM Price/GB
Year
Price/GB
2013
2010
2005
2000
1995
1990
1985
1980
$5.50
$12.37
$189
$1,107
$30,875
$103,880
$859,375
$6,328,125
Source: http://www.statisticbrain.com/average-historic-price-of-ram/
© 2014 SAP AG or an SAP affiliate company. All rights reserved.
Public
11
In-Memory Computing
CPU
4-8 sockets
Core
Core
4-12 cores/CPU
L1 Cache
L1 Cache
1 NS, 64K/core
L2 Cache
L2 Cache
3 NS, 256k/core
L3 Cache
Main Memory
Disk
8 NS, >2M shared
80 NS, TBs
SSD: 100K NS
HD: 10M NS
Yes, DRAM is 125,000
times faster than disk, but
DRAM access is still 10-80
times slower than on-chip
caches
Using Intel Ivy Bridge for approximate values.
Actual numbers depends on specific hardware.
© 2014 SAP AG or an SAP affiliate company. All rights reserved.
Public
12
Enterprise Workloads are Read Dominated
Workload in Enterprise Applications consists of:
100%
90 %
80 %
70 %
60 %
50 %
40 %
30 %
20 %
10 %
0 %
Write:
Delete
Modification
Insert
Read:
Range Select
Table Scan
Lookup
OLTP
OLAP
© 2014 SAP AG or an SAP affiliate company. All rights reserved.
Workload
Workload
 Mainly read queries (OLTP 83%, OLAP 94%)
 Many queries access large sets of data
100%
90 %
80 %
70 %
60 %
50 %
40 %
30 %
20 %
10 %
0 %
Write:
Delete
Modification
Insert
Read:
Select
TPC-C
Public
13
Simplify Technology Stack with the SAP HANA
Platform
Insight to Action
Contextual. Real-time. Closed-loop.
Applications
Analytics
SAP HANA Platform
© 2014 SAP AG or an SAP affiliate company. All rights reserved.
Public
14
SAP HANA Database Background
BWA / BIA
Trex
Enterprise
Search
NewDB
/ HANA
BW on
HANA
Suite on
HANA
2011
2012
HANA
Platform
MaxDB
PTime
Sybase IQ/ASE/SA/RS/etc.
Ancient times
2000-2009
© 2014 SAP AG or an SAP affiliate company. All rights reserved.
2010
now
Public
15
SAP HANA DB Architecture
SAP HANA DB Processes
© 2014 SAP AG or an SAP affiliate company. All rights reserved.
Public
18
Business Applications
Connection and Session Management
SQL
Authorization
Manager
SQL Script
…
MDX
Calculation Engine
Optimizer and Plan Generator
Transaction
Manager
Execution Engine
In-Memory Processing Engines
Metadata
Manager
Persistency
Column Engine
Row Engine
Logging and Recovery
© 2014 SAP AG or an SAP affiliate company. All rights reserved.
Text Engine
Data Storage
19
Distributed Share-Nothing In-Memory Computing
NewDB
Database Server
NewDB
Database Server
Execution
Layer
Execution
Layer
R
In-Memory Stores
R
NewDB
Database Server
Execution
Layer
R
In-Memory Stores
R
R
Persistence Layer
Persistence Layer
In-Memory Stores
R
Persistence Layer
© 2014 SAP AG or an SAP affiliate company. All rights reserved.
Public
20
Column Store
& Compression
SAP HANA Technology
Hybrid Data Storage
SAP HANA Row Store
stores tables by row
Att1

Att2
Att3
Att4
SAP HANA Column Store
stores tables by column
Att1
Att5
Tuple 1
Tuple 1
Tuple 2
Tuple 2
Tuple 3
Tuple 3
Tuple n
Tuple n
Application often processes single records at once

many selects and /or updates of single records

Application typically accesses the complete record

Columns contain mainly distinct values

Aggregations and fast searching not required

Small number of rows (e.g. configuration tables)
© 2014 SAP AG or an SAP affiliate company. All rights reserved.
Att2
Att3
Att4
Att5

Search and calculation on values of a few columns

Big number of columns

Big number of rows and columnar operations


aggregate, scan, etc.
High compression rates possible

Most columns contain only few distinct values
Public
23
SAP HANA Technology
Dictionary Compression & N-bit Compression
Classical Row Store
Company
[CHAR50]
Region
[CHAR30]
Group
[CHAR5]
INTEL
USA
A
Siemens
Europe
Siemens
HANA Column Store
0
1
2
3
INTEL
Siemens
SAP
IBM
0 Europe
1 USA
0 A
1 B
2 C
0
1
0
B
1
0
1
Europe
C
1
0
2
SAP
Europe
A
2
0
0
SAP
Europe
A
2
0
0
IBM
USA
A
3
1
0
© 2014 SAP AG or an SAP affiliate company. All rights reserved.
Dictionary for attribute/
column „Group“
Index Vector
Stored in one memory chunk
=> data locality for fast scans
Public
24
SAP HANA Technology
Compression with run length encoding
Classical Row Store
Difficult to compress
HANA Column Store:
Dictionary compressed
0
1
2
3
INTEL
Siemens
SAP
IBM
HANA Column Store:
Run length compressed*
0 A
1 B
2 C
0
1
2
3
INTEL
Siemens
SAP
IBM
0 Europe
1 USA
0 A
1 B
2 C
1 x „0“
1 x „1“
1 x „0“
1
2 x „1“
4 x „0“
1 x „1“
0
2
2 x „2“
1 x „1“
1 x „2“
2
0
0
1 x „3“
A
2
0
0
A
3
1
0
Company
[CHAR50]
Region
[CHAR30]
Group
[CHAR5]
INTEL
USA
A
0
1
0
Siemens
Europe
B
1
0
Siemens
Europe
C
1
SAP
Europe
A
SAP
Europe
IBM
USA
0 Europe
1 USA
3 x „0“
* Note that there is a variety of compression methods
and algorithms like run-length compression
© 2014 SAP AG or an SAP affiliate company. All rights reserved.
Public
25
Snapshot Isolation
time
oldest reader tx1 commit tx2 begin
© 2014 SAP AG or an SAP affiliate company. All rights reserved.
tx3 commit
tx2 access
011000……………001101111100111
011001……………101101111011
111
000001……………100000000001000
inserted
11111
deleted
100010……………01000000010
base list of rows visible to all
111011……………111101
DATA-D……………DATA-D ATA-DA TA-
Initial Design – set oriented, optimized for OLAP
tx4 begin
& access
Public
30
New Design – OLTP friendly
valid valid
from to
tx3: delete where …
tx2: insert n rows
New
rows
tx1: insert 1 row
New row
– Valid from/to for every row?
tx3
tx1
tx3
Problems to solve
 Memory overhead
tx2
…
DATA
 Tx identity: TID vs. CID
– If TID: visibility rules, TCB memory overhead
– If CID: DML time ID, atomic commit, post-commit
 L2/3 cache friendly
– Stay local, avoid dereferencing pointers
 OLAP performance
© 2014 SAP AG or an SAP affiliate company. All rights reserved.
txn: reader
Public
31
Outlook
Where is HANA going next?
Continuing Challenges of Emerging Hardware
 Challenge 1: Parallelism: Take advantage of tens, hundreds, thousands of cores
 Challenge 2: Large memories & data locality/NUMA
– Yes, DRAM is 125,000 times faster than disk…
– But DRAM access is still 10-80 times slower than on-chip caches
© 2014 SAP AG or an SAP affiliate company. All rights reserved.
Public
33
HANA Platform On-Going Architectural Evolution
Data models
 Flexible schemas, graph functionality, geospatial, time series, historical data,
Big Data, external libraries
Resource and workload management
 Memory, threads, scheduling, admission control, service level management, data aging
Application services
 XS Engine, CDS and River
Continuing performance improvements
 Hardware advances, NUMA, improved modularization and architecture
Cloud and multi-tenancy
© 2014 SAP AG or an SAP affiliate company. All rights reserved.
Public
34
Co-innovating the Next Big Wave in Hardware Evolution
Multi-Core and Large “Memory” Footprints
Storage Class Memories / Non-Volatile Memory
 Leverage as DRAM and/or as persistent storage
On-Board DIMMs
 Very high density, byte-addressable
 DRAM like (< 3X) latency and bandwidth; similar endurance
 Compete with disk on cost/bit by 2020
Extreme Speed Network Fabric/Interconnects
 Inter-socket NUMA gets worse while inter-host NUMA gets better
 Inter-socket and Inter-host latencies converge
Exploiting Dark Silicon for Database Hardware Acceleration
 Also exploit GPUs for specific use cases, such as regression analysis
© 2014 SAP AG or an SAP affiliate company. All rights reserved.
Public
35
Thank you!
Contact information:
Arne Schwarz, [email protected]
Mihnea Andrei, [email protected]
Richard Pledereder, [email protected]
http://www.careersatsap.com/
http://jobs.sap.com
https://www.saphana.com
http://www.sap.com/pc/tech/in-memory-computing-hana.html
© 2014 SAP AG or an SAP affiliate company. All rights reserved.
© 2014 SAP AG or an SAP affiliate company.
All rights reserved.
No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG or an
SAP affiliate company.
SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG
(or an SAP affiliate company) in Germany and other countries. Please see http://global12.sap.com/corporate-en/legal/copyright/index.epx for additional
trademark information and notices.
Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.
National product specifications may vary.
These materials are provided by SAP AG or an SAP affiliate company for informational purposes only, without representation or warranty of any kind,
and SAP AG or its affiliated companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP AG or
SAP affiliate company products and services are those that are set forth in the express warranty statements accompanying such products and
services, if any. Nothing herein should be construed as constituting an additional warranty.
In particular, SAP AG or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related
presentation, or to develop or release any functionality mentioned therein. This document, or any related presentation, and SAP AG’s or its affiliated
companies’ strategy and possible future developments, products, and/or platform directions and functionality are all subject to change and may be
changed by SAP AG or its affiliated companies at any time for any reason without notice. The information in this document is not a commitment,
promise, or legal obligation to deliver any material, code, or functionality. All forward-looking statements are subject to various risks and uncertainties
that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking
statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions.
© 2014 SAP AG or an SAP affiliate company. All rights reserved.
Public
37