Practical Data Integrity Protection in Network-Coded Cloud Storage

(1)

Practical Data Integrity Protection in

Network-Coded Cloud Storage

Henry C. H. Chen

Department of Computer Science and Engineering The Chinese University of Hong Kong

(2)

Outline

 Introduction

 FMSR in NCCloud

 FMSR-DIP

Publications

 Yuchong Hu, Henry C. H. Chen, Patrick P. C. Lee, and Yang Tang

NCCloud: Applying Network Coding for the Storage Repair in a Cloud of-Clouds

Proceedings of the 10th USENIX Conference on File and Storage Technology (FAST ’12)

 Henry C. H. Chen, and Patrick P. C. Lee

Practical Data Integrity Protection in Regenerating-Coding-Based Storage

(3)

Outline

 Introduction

 FMSR in NCCloud

(4)

Cloud Storage

 On-demand storage outsourcing

 Supports RESTful APIs:

(5)

(6)

Data Integrity Protection

 Corruption detection

• Addressed in this work

 Fault-tolerance and repair

• Addressed in NCCloud

 Desirable properties

• Minimize cost

• Works on thin clouds (i.e., clouds with only basic file access semantics)

(7)

Data Integrity Protection

• Addressed in this work

• Minimize cost

(8)

Data Integrity Protection

• Addressed in this work (FMSR-DIP)

• Minimize cost

(9)

Related Work

 Single node, smart clouds

• PDP [Ateniese et al. ’07]

• POR [Juels et al. ’07]

 Multi-node, different storage schemes

• MR-PDP [Curtmola et al. ’08]

(10)

Our Work

 Build FMSR-DIP, a corruption detection scheme that allows byte-sampling

• Works on thin clouds

• Works on functional minimum storage regenerating (FMSR) code

(11)

Outline

 Introduction

 FMSR in NCCloud

(12)

NCCloud

Proxy

Cloud 1 Cloud 2 Cloud 3 Cloud 4 Users

file upload

download file

(13)

Contributions of NCCloud

 Propose an implementable design of functional

minimum storage regenerating (FMSR) code

• Support basic read/write operations and the repair function on thin clouds

• Preserve storage requirements as in optimal erasure codes, while reducing repair traffic

(14)

Repairing a Failed Cloud

 How to repair:

Proxy

Cloud 1 Cloud 2 Cloud 3

Cloud 4

Cloud 5 _{Repair traffic =} _＋ _＋

(15)

Reed-Solomon Codes

 Conventional repair:

• Reconstruct whole file and generate data in new node

A B A+B A+2B B A+B A A A B File of size M Node 1 Node 2 Node 3 Node 4 Proxy

Reed Solomon codes Repair traffic = M

n = 4, k = 2

(n, k) MDS code: Any k out of n storage nodes (clouds) can rebuild original file.

(16)

FMSR in NCCloud

 Code chunk P_i = linear combination of original native chunks

 Repair in FMSR:

• Download one code chunk from each surviving node

• Reconstruct new code chunks (via random linear combination) in

new node P1 P2 P3 P4 P5 P6 P7 P8 P3 P5 P7 P1’ P2’ A B C D P1’ P2’ Node 1 Node 2 Node 3 Node 4 File of size M Proxy

n = 4, k = 2

F-MSR codes

(17)

FMSR Property

P1 P2 P3 P4 P5 P6 P7 P8 A B C D k(n-k) chunks Proxy partition encode P1 P2 P3 P4 P5 P6 P7 P8 n(n-k) chunks distribute File n=4, k=2 Storage nodes

(18)

FMSR Property

P1 P2 P3 P4 P5 P6 P7 P8 A B C D k(n-k) chunks Proxy partition encode P1 P2 P3 P4 P5 P6 P7 P8 n(n-k) chunks distribute File n=4, k=2 Storage nodes

c_1,1 c_1,2 c_1,3 c_1,4 .

. .

c_8,1 c_8,2 c_8,3 c_8,4

A B C D P1 P2 P3 P4 P5 P6 P7 P8 Encoding matrix

rank = k(n-k)

(19)

NCCloud: Experiments

 Testbed environment

• Local cloud

• Openstack Swift 1.4.2

• 1 proxy node connected to 15 storage nodes (LAN) • NCCloud deployed on proxy node

• Commercial cloud • Microsoft Azure

 Storage schemes

(20)

0 5 10 15 20 25 30 35

1 10 50 100 200 300 400 500

RAID-6(native) RAID-6(parity) F-MSR

Response time: Local Cloud

 FMSR has higher response time due to encoding/decoding overhead

 FMSR has slightly less response time in repair, due to less data download

20 0 10 20 30 40 50

1 10 50 100 200 300 400 500

RAID-6 F-MSR

File size (MB)

R es ponse tim e (s ) UPL OA D

File size (MB)

R es ponse tim e (s ) D OW N L OA D

File size (MB)

R es ponse tim e (s ) R EP A IR 0 2 4 6 8 10 12

1 10 50 100 200 300 400 500

RAID-6 F-MSR RS FMSR RS FMSR

RS (native chunk repair) RS (code chunk repair) FMSR

(21)

0 1 2 3 4 5 6

1 2 5 10

RAID-6(native) RAID-6(parity) F-MSR

Response time: Commercial Cloud

 No distinct response time difference, as network fluctuations play a bigger role in actual response time

21 File size (MB)

R es ponse tim e (s ) UPL OA D

File size (MB)

R es ponse tim e (s ) D OW N L OA D R es ponse tim e (s ) R EP A IR

File size (MB) 0

2 4 6

1 2 5 10

RAID-6 F-MSR 0 0.5 1 1.5 2 2.5

1 2 5 10

RAID-6 F-MSR RS FMSR RS FMSR

RS (native chunk repair) RS (code chunk repair) FMSR

(22)

Outline

 Introduction

 FMSR in NCCloud

(23)

FMSR-DIP: Design Goals

 Preserves advantage of FMSR

 Works on thin clouds

 Supports sampling to minimize cost

 Works against a Byzantine, mobile adversary

• Exhibits arbitrary behaviors

(24)

FMSR-DIP: Overview

FMSR-DIP

Cloud 1 Cloud 2 Cloud 3 Cloud 4 Users

file upload

download file

Proxy

(25)

FMSR-DIP: Upload

(26)

FMSR-DIP: Upload

Apply

error-correcting code (ECC)

(27)

FMSR-DIP: Upload

XOR each byte with a

(28)

FMSR-DIP: Upload

For each chunk, calculate

the MAC of the first 3 bytes

(29)

FMSR-DIP: Upload

 Upload the chunks to clouds

 Encrypt the metadata from NCCloud (which contains the encoding matrix)  Append all MACs to metadata

(30)

FMSR-DIP: Check

(31)

FMSR-DIP: Check

XOR with the previous pseudorandom

values, and check their consistency

(32)

Recall: FMSR Encoding

P₁ P₂ P₃ P₄ P₅ P₆ P₇ P₈ A B C D c_1,1 c_1,2 c_1,3 c_1,4

c_2,1 c_2,2 c_2,3 c_2,4 c_3,1 c_3,2 c_3,3 c_3,4 c_4,1 c_4,2 c_4,3 c_4,4 c_5,1 c_5,2 c_5,3 c_5,4 c_6,1 c_6,2 c_6,3 c_6,4 c_7,1 c_7,2 c_7,3 c_7,4 c_8,1 c_8,2 c_8,3 c_8,4

Encoding matrix

rank = k(n-k)

(33)

FMSR-DIP: Download

Download chunks from any 2 nodes

and verify with their MACs

(34)

FMSR-DIP: Download

Remove pseudorandom values

and pass to NCCloud for decoding

(35)

(36)

FMSR-DIP: Repair

Download 1 chunk from all other nodes

and verify with their MACs

(37)

FMSR-DIP: Repair

Remove pseudorandom values

and pass to NCCloud

(38)

FMSR-DIP: Repair

(39)

FMSR-DIP: Repair

Process the newly generated

chunks as before

(40)

FMSR-DIP: Repair

Upload chunks and update metadata

on all nodes

(41)

FMSR-DIP: Experiments

 Testbed environment

• Openstack Swift 1.4.2

• 1 proxy node connected to 15 storage nodes (LAN) • NCCloud and FMSR-DIP deployed on proxy node • NCCloud uses RAMDisk as storage

 Storage scheme

(42)

Running Time vs. File Size

 FMSR-DIP overhead comparable to network transfer time in a LAN environment U PL OA D D OW N L OA D R EP A IR 0 5 10 15 20 25

100MB 50MB 20MB 10MB 5MB 1MB

T ime taken (s) Transfer-Up DIP-Encode FMSR 0 2 4 6 8

T ime taken (s) Transfer-Down DIP-Decode FMSR 0 5 10 15 20

T ime taken (s) Transfer-Up Transfer-Down DIP-Encode DIP-Decode FMSR File size File size File size

(43)

The Check Operation

0 10 20 30 40 50 60 70 80

256B 1KB 4KB 7KB 25KB 256KB

T ime taken (s) _Misc. Transfer-Down Rank Checking PRF 0 5 10 15 20 25 30

100% 75% 50% 25% 10% 5% 1%

T ime taken (s) Misc. Transfer-Down Rank Checking PRF Download block size Checking percentage

 Bottleneck in network transfer

1% check

256KB download block size

(44)

Conclusions

 Propose a design for efficient data integrity protection using FMSR on thin clouds

 Implement and evaluate the efficiency of the design

 Source code:

• NCCloud

http://ansrlab.cse.cuhk.edu.hk/software/nccloud/

• FMSRDIP

(45)

(46)

Error Localization

(47)

Error Localization

Form a system with bytes from

k

other

nodes

(48)

Error Localization

Mark all involved bytes as correct if

system is consistent

(49)

Error Localization

(50)

Error Localization

(51)

Error Localization

(52)

Cloud Storage Pricing

S3 Rackspace Azure

Storage (per GB) $0.125 $0.15 $0.125

Data transfer in (per GB) free free Free

Data transfer out (per GB) $0.12 $0.18 $0.12

PUT (per 10,000 requests) $0.10 free $0.01

GET (per 10,000 requests) $0.01 free $0.01