• No results found

How SoundCloud scales

N/A
N/A
Protected

Academic year: 2021

Share "How SoundCloud scales"

Copied!
67
0
0

Loading.... (view fulltext now)

Full text

(1)

How SoundCloud scales

Alexander Grosse @klangberater

(2)

“Youtube for Audio”

(3)

Fakten

(4)

Alexa Ranking (233)

Fakten

(5)

Alexa Ranking (233)

Employees (190) - 80 engineers

Fakten

(6)

Alexa Ranking (233)

Employees (190) - 80 engineers

4 office locations (Berlin, London, San Francisco, Sofia)

Fakten

(7)

Alexa Ranking (233)

Employees (190) - 80 engineers

4 office locations (Berlin, London, San Francisco, Sofia)

>35 million registered users

Fakten

(8)

Alexa Ranking (233)

Employees (190) - 80 engineers

4 office locations (Berlin, London, San Francisco, Sofia)

>35 million registered users

partners (BBC, CNN)

Fakten

(9)

Alexa Ranking (233)

Employees (190) - 80 engineers

4 office locations (Berlin, London, San Francisco, Sofia)

>35 million registered users

partners (BBC, CNN)

> 10.000 Apps

Fakten

(10)
(11)

How SoundCloud scales

(12)

?

(13)

Organization

Technology

(14)

What is the foundation?

(15)
(16)

Principles

(17)

Top-Down vs. Bottom-Up

Principles

(18)

Top-Down vs. Bottom-Up Trust

Principles

(19)

Top-Down vs. Bottom-Up Trust

How to react to mistakes

Principles

(20)

Top-Down vs. Bottom-Up Trust

How to react to mistakes Continuous Improvement

Principles

(21)

Top-Down vs. Bottom-Up Trust

How to react to mistakes Continuous Improvement Innovation

Principles

(22)

Flat Hierarchies? Everybody has them...

Engineering teams responsible for architecture (not architecture teams)

Influence on the product

Top-Down vs. Bottom-Up

(23)

Deployment

Change Management

Process

Time for “technical debt”

Trust

(24)

What will be changed if a mistake happens?

Reaction to mistakes

(25)

R&D Department?

Only Product?

Innovation

(26)

SoundCloud‘s basic principles

(27)

No dedicated QA roles

SoundCloud‘s basic principles

(28)

No dedicated QA roles

“You build it - you run it”

SoundCloud‘s basic principles

(29)

No dedicated QA roles

“You build it - you run it”

Team Building (Rotation, constant changes)

SoundCloud‘s basic principles

(30)

No dedicated QA roles

“You build it - you run it”

Team Building (Rotation, constant changes)

Hacker Time

SoundCloud‘s basic principles

(31)

No dedicated QA roles

“You build it - you run it”

Team Building (Rotation, constant changes)

Hacker Time

Continuous Improvement

SoundCloud‘s basic principles

(32)

No dedicated QA roles

“You build it - you run it”

Team Building (Rotation, constant changes)

Hacker Time

Continuous Improvement

Move fast and break things

SoundCloud‘s basic principles

(33)

No QA

Does that mean we don‘t test?

(34)

“You build it - you run it”

(35)

Flat Hierarchy

(36)

Innovation

Hacker Time

(37)
(38)

Continuous Improvement

Retrospectives

Post Mortems

(Projekte)

(39)

Mistakes...

(40)

Scrum? Kanban?

Process?

(41)

What is the most

important thing you have

to do so that this works?

(42)
(43)

???

(44)

???

Initial Culture

(45)

???

Initial Culture

Hiring!!!

(46)
(47)

SoundCloud Hiring Process

(48)

How does it look like in

reality?

(49)

Organisation

(50)

HTML5

Mobile (native)

other

V1

System Engineering (Berlin / SF)

Search

T&S

Payment

Partner

Tools

Data

Activ.

Delivery

Platform

API

(51)

How do we structure teams?

cross functional?

(52)

Do teams stay the same?

Change!

(53)

Technology

(54)

Ruby/Scala/Go/Clojure/

Javascript

(55)

Scale Horizontally/

Vertically?

(56)

Rails->Mysql

RabbitMQ->Cassandra

http://backstage.soundcloud.com/2012/08/

evolution-of-soundclouds-architecture/

(57)

7 Master, 9 slaves, 900GB data

Additional slaves in EC2

Master Peak is 15K IOPS writes, 45k reads

one slave lags intentionally 1 hour behind

Je nach Konsistenzanforderung wird Slave oder Master gewählt (für reads - unterstützt read-only mode)

MySQL

(58)

3 Cluster

88 Nodes

15 TB of data

up to 70K reads/sec

Cassandra

Freitag, 5. April 13

(59)

Hadoop

Elastic Search

Proprietary Spam Detection

(60)

ElasticSearch

80 Mio Docs (gaining 1 Mio every 6

days)

175 QPS (gaining 15 QPS every month)

(61)

Hosted in S3/EC2

and physical DCs

(62)

Outages

(63)
(64)

Post Mortems

(65)

Move fast and break things

Don‘t get slow because you fear to make mistakes

Summary

(66)

backstage.soundcloud.com

(67)

Alexander Grosse

alexandergrosse.com

@klangberater

We are hiring!

www.soundcloud.com/jobs Freitag, 5. April 13

References

Related documents

80 TOP SELENIUM webdriver interview Questions for 5 years experience

Payments by iforex com complaints or experience about forex trading platform from your personally identifiable information depth information share your date of india!. Results but if

Pre-operative lateral cephalogram Post-operative lateral cephalogram D IAGNOSIS : Class III malocclusion (underbite), maxillary anterior-posterior hypoplasia,.

No paid reviews Get concrete on Denny's real problems Customer service an support information File a complaint at ComplaintsBoardcom.. If you're 55 or church save some success

Would like to enable the schedule yet poignant look at drastically cheaper prices ever primetime shows you can also share information?. Tv schedule for an office memo and how your

Want to select aaa com insurance really covers can access your home insurance more reasons to select coverages, insurance coverage and are a treasure.. Milestone that go www

Try Scribd FREE for 30 days to access over 125 million titles without ads or interruptions. Start

If the sufficient funds are not available If the sufficient funds are not available for the construction of standard or Broad for the construction of standard or Broad Gauge