Reusable Data Access Patterns

Gary Helmling, Software Engineer @gario

HBaseCon 2015 - May 7

Agenda

• A brief look at data storage challenges

• How these challenges have influenced our work at Cask • Exploration of Datasets and how they help

• Review of some common data access patterns and how Datasets apply • A look underneath at the tech that makes it work

Data Storage Challenges

• Many HBase apps tend to solve similar problems with common needs

• Developers rebuild these solutions on their own, sometimes repeated for each app • Effective schema (row key) design is hard

• byte[] conversions, no real native types • No composite keys

Cask Data Application Platform

• CDAP is a scale-out application platform for Hadoop and HBase • Enables easy scaling of application components

• Combines real-time and batch data processing • Abstracts data storage details with Datasets

• Built from experience by Hadoop and HBase users and contributors

• CDAP and the components it builds on are open source (Apache License v2.0): • Tephra, a scalable transaction engine for HBase and Hadoop

• Encapsulate a data access pattern in a reusable, domain-specific API • Establishes best practices in schema definition

• Abstract away underlying storage platform

HBase

CDAP Datasets

Table 1

Table 2

Dataset

App 2

App 1

How CDAP Datasets Help

• Reusable as data storage templates • Easy sharing of stored data:

• Between applications

• Batch and real-time processing • Integrated testing

• Extensible to create your own solutions

• Leverage common services to ease development: • Transactions

Reusing Datasets

• CDAP integrates lifecycle management

• Centralized metadata provides key configuration • Transparent integration with other systems

• Hive metastore

• Map Reduce Input/OutputFormats • Spark RDDs

Reusing Datasets

/**

* Counter Flowlet. */

public class Counter extends AbstractFlowlet { @UseDataSet("wordCounts")

private KeyValueTable wordCountsTable; …

Testing Datasets

• CDAP provides three storage backends: in-memory, local (standalone), distributed

In-Memory

NavigableMap

Local Temp Files

Local

Distributed

Dataset APIs

Extending Datasets

• Existing datasets can be used as building blocks for your own patterns

• @EmbeddedDataset annotation injects wrapped instance in custom code • Operations seamlessly wrapped in same transaction, no need to re-implement

public class UniqueCountTable extends AbstractDataset { public UniqueCountTable(DatasetSpecification spec,

@EmbeddedDataset("unique") Table uniqueCountTable, @EmbeddedDataset("entry") Table entryCountTable) { …

} }

HBase Data Patterns & Datasets

•

Secondary Indexes

Object-mapping

•

Timeseries

Data cube

Secondary Indexing

• Example use case: Entity storage - store customer records indexed by location • HBase sorts data by row key

• Retrieving by a secondary value means storing a reference in another table • Two types: global and local

• Global: efficient reads, but updates can be inconsistent

• Local: updates can be made consistent, but reads require contacting all servers

• IndexedTable Dataset performs global indexing • Uses two tables: data table, index table

Object-Mapping

• Example use case: Entity storage - easily store User instances for user profiles • Easy serialization / deserialization of Java objects (think Hibernate)

• Maps property fields to HBase columns • No defined schemas in HBase

• Accessing data by other means requires knowledge of object structure

• ObjectMappedTable Dataset: automatically persists object properties as columns in HBase • Metadata managed by CDAP

• Stores the object's schema

Timeseries Data

• Example use case: any data organized around a time dimension • System metrics

• Stock ticker data

• Sensor data - smart meters

• Constructing keys to avoid hotspotting and support efficient retrieval can be tricky

• TimeseriesTable Dataset: for each data key, stores a set of (timestamp, value) records • Each stored value may have a set of tags used to filter results

• Each row represents a time bucket, individual values in that bucket stored as columns • When reading data, projects entries back into a simple Iterator for easy consumption

Data Cube

• Example use case: Retail product sales reports, web analytics

• Stores “fact” entries, with aggregated values along configured combinations of the “fact” dimensions

• Pre-aggregation necessary for efficient retrieval

• HBase increments can be costly in write-heavy workload • Querying requires knowledge of pre-aggregation structure

• Reconfiguration can be difficult

• Need metadata around configuration

• Cube Dataset: uses readless increments for efficient aggregation • Transactions keep pre-aggregations consistent

Transactions

• Provided by Tephra (http://tephra.io), an open-source, distributed, scalable transaction engine

designed for HBase and Hadoop

• Each transaction assigned a time-based, globally unique transaction ID • Transaction =

• Write Pointer: Timestamp for HBase writes

• Read pointer: Upper bound timestamp for reads

• Excludes: List of timestamps to exclude from reads

Tephra Architecture

RS 1

Client

Client

Client

_{RS 2}

Tx CP

Tx CP

Transactional Writes

• Client sets write pointer (transaction ID) as timestamp on all writes

• Maintains set of change coordinates (row-level or column-level granularity depending on needs) • On commit, client sends change set to Transaction Manager

• If any overlap with change sets of commits since transaction start, returns failure • On commit failure, attempts to rollback any persisted changes

• Deletes use special markers instead of HBase deletes • HBase deletes cannot be rolled back

Transactional Reads

• TransactionAwareHTable client sets the transaction state as an attribute on all read operations • Get, Scan

• Transaction Processor RegionObserver translates transaction state into request properties • max versions

• time range

• TransactionVisibilityFilter - excludes cells from: • Invalid transactions (failed but not cleaned up) • In-progress transactions

• “Delete” markers • TTL’d cells

Increment Performance

• HBase increments perform read-modify-write cycle

• Happens server-side, but read operation still incurs overhead • Read cost is unnecessary if we don't care about return value • Not a great fit for write-heavy workloads

HBase Increments

row:col timestamp value

hello:count 1001 1

1002 2

Example: Word count on “Hello, hello, world”

counting 2nd “hello”

1. read: value = 1

2. modify: value += 1 3. write: value = 2

Readless Increments

• Readless increments store individual increment values for each write • Mark cell value as Increment instead of normal Put

Readless Increments

row:col timestamp value

hello:count 1001 +1

1002 +1

1. write: increment = 1

Example: Word count on “Hello, hello, world”

Readless Increments

row:col timestamp value

hello:count 1001 +1

1002 +1

Example: Word count on “Hello, hello, world”

reading current count for “hello”

Readless Increments

• Increments become simple writes

• Good for write-heavy workloads (many uses of increments)

• Reads incur extra cost from reading all versions up to latest full sum • HBase RegionObserver merges increments on flush and compaction

• Limits cost of coalesce-on-read

Want to Learn More?

Open-source (Apache License v2) Website:

http://cdap.io

Mailing List:

[email protected] [email protected]

Open-source (Apache License v2) Website:

http://tephra.io

Mailing List:

[email protected] [email protected]

QUESTIONS?

Want to work on these and other challenges?

http://cask.co/careers/