Collaborative Software Development

(1)

Collaborative Software Development

and what we can learn from F/OSS development

Martin Kropp

Instute for Mobile and Distributed Systems

(2)

Some term clarifications

☺

Show how F/OSS development is organized

Show how we can profit from F/OSSD CSD practices

Describe some ingredients needed for effective CSD

(3)

Agenda

About Me

CSD and GSD - What does that mean?

GSD - Why should we care?

F/OSSD – The mother of CSD

l

How does it work?

Ingredients for CSD

Communication, Process, Project Management, CI,

Summary

swisslog, Grindelwald, 28. Mai 2009

(4)

Who am I?

(5)

Agenda

About Me

How does it work?

l

How does it work?

Ingredients for CSD

Summary

(6)

Collaborative Software Development

From Wikipedia, 26.05.2009:

The collaborative software development model is a style of

software development whose focus is on

public availability

and

communication

, usually via the Internet.

The software development model began widespread adoption

with the Linux kernel in 1991.

It is the

dominant model

used in development of

free software

.

It is very compatible with free software because free software

projects publish the source code of any published programs, so

they do not have the secrecy reason for hiding their

(7)

Global Software Development

“Software work undertaken at

geographically

separated

locations across national boundaries in

a

coordinated

fashion involving real time

(synchronous) and asynchronous

interaction

”

[Sundeep Sahay, University of Oslo]

(8)

Why should I do GSD?

To

Access to the world wide talent pool

Cost savings

Focus on where you get the best ROI

Market presence

End Users

Business Subject

Matter Experts

Technical

Budget Buyers

& Project Proxies

understanding local needs

being close to the customer

Mergers and acquisitions

Improved quality?

Follow-the-sun / -seasons development

gaining competitive advantage

Technical

Collaborators

(9)

Some reasons you might

not

want to do it

Strategic choices – timing, partners, …

how to maintain and gain competitive advantage?

Management / comms overhead – complexity

Cultural issues

Command chains in the org chart, attitude, individualism / collectivism, religions, politics,

…

Communication issues

time zones, distance, lack of informal communications, …

different backgrounds different “language”

information systems

Technical issues – does your software architecture support distributed development

model?

…

Institut für Mobile und Verteilte Systeme 9

(10)

Challenge Matrix

communication

temporal

geographical

sociocultural

distance

X

coordination

control

X

(11)

And how does this relate to CSD?

(12)

Agenda

About Me

F/OSSD – The mother of CSD

How does it work?

l

How does it work?

Ingredients for CSD

(13)

F/OSSD - The Mother of CSD

Plublic availability

Communication

… and there is more!

(14)

A Short History of F/OSSD

1984 – Richard Stallman founds the Free Software Foundation

(“FSF”) (www.fsf.org) in 1985 to develop “free” version of a UNIX

operating system

Promulgates GNU Public License (“GPL”)

1994 – Linux 1.0 is release under the GPL by Linus Torvalds

1998 – Open Source Initiative (“OSI”) is founded

www.opensource.org

Coins term “Open Source”

Certifies and lists open source licenses that conform to the OSD

2003 – Linux OS/Apache Web Server are mainstream (27%

(15)

Open Source Software – A Success Story?

How many successful projects can you name?

How many OS projects do exist?

(16)

Open Source Software – A Success Story!

Collaborative Development

(17)

F/OSSD (and CSD) – How it works?

Requirements and design

Configuration management and work coordination

Maintenance and Evolution

Project management/career development

Development Paradigm

(18)

F/OSSD – Requirements and Design

F/OSS Requirements/Designs

not explicit

not formal

F/OSS Requirements/Designs are embedded within “informalisms”

F/OSS Requirements/Design processes are different from their SE

(19)

F/OSSD „informalisms“

Project digests

Project

repositories

Issue tracking

databases

Email lists

Software bug

reports

News postings

IM/Internet

Relay Chat

Discussion

forums

reports

Relay Chat

forums

To-do lists

Scenarios of usage

How-to guides

FAQ’s and item

lists

Project Wikis

System

documentation

External

publications

Copyright

licenses

Architecture

diagrams

Plug-ins

Intra-app

scripting

Code from

other projects

Project Web

site

Multi-project

Web sites

Project source

code

etc.

(20)

Configuration Mgmt and Work Coordination

Use CM to coordinate and control who gets to update

what part of the system/online artifacts

Most F/OSSD projects use

version control systems

and

frequent releases (

daily releases

on active projects)

Apache: Single major release, with frequent “

patch

” releases

(21)

Maintenance and Evolution

Overall evolutionary dynamic of F/OSSD is reinvention

Reinvention enables continuous improvement

F/OSS evolve through minor mutations

Expressed, recombined, redistributed via incremental releases

F/OSS systems co-evolve with their development community

Success of one depends on the success of the other

(22)

Project Management and People

F/OSSD projects self-organize as a pyramid meritocracy via virtual project

management

Ad hoc alliances, free participation, work is not assigned,

Main motivation is learning and creation

Status by the practice of using

F/OSS developers want to have fun

learn about new stuff (tools, techniques, skills, etc.),

exercise their technical skill,

try out new kinds of systems to develop,

interconnect multiple F/OSSD projects (freedom of choice and expression)

exchange with others

(23)

F/OSSD Roles and Pyramid Meritocracy

Project Leader

Passive users (99% in Apache)

Readers

Bug reporters

Bug fixers

Peripheral developers

Active developers

Core members (<15)

Project Leader

(24)

Characteristics Summary of F/OSSS and CSD

Extensive involvement of user/developer community

Forums, mails, issues trackings, wiki

The developers must be users

Bug finding and fixing is done by users -> low post-release defect density and

high productivity due to independence of tasks.

“early and often Releases”

Nightly builds

Very lightweight

Intensive use of „informalism“

High modularized software

Common Code ownership

Automation

(25)

F/OSSD – CSD vs. Closed Source CSD

Public availability

(May be) a roadmap

Feature requests

Users

Closed source

Tight schedule

Strong requirements

A client

Losely organized

Strict organization

(26)

Agenda

About Me

How does it work?

l

How does it work?

Ingredients for CSD

(27)

Ingredients For Effective CSD

Good Communication

Strong Development Process

Good Project Management

Continuous Integration

In distributed

teams

(28)

Good Communication

Email

Skype

IM

Wikis

Web Plattform

Conference Calls

Video Conferencing

Direct Meeting

Exchange programs

Travelling

(29)

Strong Development Process

TDD

Code Reviews

Teams-of-Teams

Sprints

Mentoring

Discipline

(30)

Project Management

Bug Tracking

Feature Planning

Release Planning

Version Management

Cross Training

(31)

Continuous integration is the strategy of making sure that

changes to the project’s code base are built, tested and

reported on as soon as possible after they are introduced

CSD in practice

(32)

CI Prerequisites

Your code is maintained in a

central repository

You have

automated

the

complete

build of your system

You have included

regression tests

in the codebase as part of

the project.

you have included

automated reporting

about build success

you have included

automated reporting

about build success

you have a CI

infrastructure

with build server, test server,

(33)

Practices of Continuous Integration

Maintain a Single Source Repository.

Automate the Build

Make Your Build Self-Testing

Everyone Commits Every Day

Every Commit Should Build the Mainline on an Integration

Machine

Keep the Build Fast

Test in a Clone of the Production Environment

Make it Easy for Anyone to Get the Latest Executable

Everyone can see what's happening

Automate Deployment

From

http://martinfowler.com/articles/continuousIntegration.html

(34)

CI Infrastructure

vcs server

build server

Monitors projects/paths

in source repository

Development

Deployment Server

build report

Executes tasks on

schedule/event

Get local copies /

developers

-Get latest version

- Build

- Code Audit

-Unit Test

-Code Coverage

-Deploy

Build Script

(35)

CI Benefits

Finding Bugs is easier

Self-testing builds

Diff debugging

Less cumulative bugs

Reduced Risks

Easier predictions

Avoid blind spots in the project

Developers can concentrate on coding

Encourage more frequent deployment

(36)

Automate The Build

It‘s not just compiling …

The entire set of

ALL

steps needed to obtain the software

product

Use Tools like

(37)

When To Build

Every successful change causes a commit

Of course, after testing

Every commit triggers a build

Partial builds – any time

l

Full builds – nightly builds

(38)

Build Resources

Input

Anything that‘s needed for the product

Output

?

Anything that‘s needed for

Installation

Execution

Documentation

(39)

Self-Testing Builds

Automated tests

No manual execution

No clicks on dialog boxes

No config editing

Test on all levels

Unit test (use mocks for isolated tests) – Junit, easymock

Db test - dbTest

Integration test – Fit/Fitness, Selenium

Acceptance test – Fit/Fitnesse

GUI test – FEST

Web Test – WebTest

…

(40)

Build Tasks

Not just build and test code

Include auditing

Metrics

Code quality analysis

Code coverage

Test analysis

Statistics

(41)

Provide Feedback

Everybody sees what‘s going on

(42)

Summary

GSD is a must in any global acting company

It comes with a lot of challenges and issues …

distance,

culture, …

CSD provides some very helpful practices to address challenges

Common code owenship

Using „informalism“

Providing feedback

Establish automation

(43)

Resources

Books

Eric S. Raymond,

The Cathedral & the Bazaar

. O’Reilly, 2001

Musings on Linux and Open Source by an Accidental Revolutionary

James A. Highsmith III:

Adaptive Software Development: A Collaborative Approach to

Managing Complex Systems

- Dorset House Publishing Company, 1999.

Erran Carmel:

Global Software Teams: Collaborating Across Borders and Time Zones

.

Prentice Hall, 1998.

Dean Leffingwell:

Scaling Software Agility: Best Practices for Large Enterprises.

Addison-Wesley, 2003.

Edward Yourdon:

Competing in the Global Productivity Race

. Prentice Hall, 2004.

Dale Walter Karolak,

Global Software Development: Managing Virtual Teams and

Environments

. Wiley-IEEE Computer Society, 1998.

Raghvinder Sangwan, Matthew Bass, Neel Mullick, Daniel J. Paulish, Juergen Kazmeier,

Global Software Development Handbook

. AUERBACH, 2006.

(44)

Design and programming are human

activities; forget that and all is lost.

Bjarne Stroustrup, 1991

Thank you very much for your

attention!