Modern practices TIE-21100/

(1)

Modern practices

(2)

Today’s lecture

• Learn what some modern SW

engineering topics are about

• A peek to some research topic of our

department

(3)

(4)

(5)

(6)

(7)

How the lectures continue?

02.03 Modern practices Continuous Integration, Continuous Deployment, DevOps,..

9.03 No lectures - exam week

17.3 Improving Quality: review practices, testing and quality assurance Quality in general; Quality management systems 23.3 Dependable and safety-critical systems

30.3 Other methods; RUP, XP, Lean; Kanban. 6.4 Easter - no lectures

13.4 Role of software architecture; software maintenance; software evolution

20.4 Software business, software start-ups 27.4 Guest lecture 1

(8)

General matters

About the tools

– All information should be on the Web page (if something is missing, please report me) – Canvas is a collaboration tools

– Emails (POP-messages) are for quick updates and reminders

Sprint reviews

– Evaluation is based on several factors

(overall management, scrum process, implemented user stories)

Weekly exercises

– We had some technical problems that delayed finalization of Jenkins exercise => postponed to 16.3

– This week we look at Traditional, Agile and Lean through a scientific article

(9)

Agenda for today

• Recap of the effort estimation

• Global SW development

– And related research

• Optimized pipeline from requirements to use

feedback

(10)

STILL ABOUT EFFORT

ESTIMATION

(11)

Picture 12.8 in Haikala&Mikkonen, 23.9 in Sommerville Prestudy Requirement specification Architecture design Detailed design Implementation Real value Early prototyping Early design Post architecture

(12)

Effort estimation in Scrum

Sprint 2-4 weeks Sprint backlog 24 hours

Planning

Go / no-go decision Initial estimation Sprint planning

(13)

Effort estimation

• Necessary for planning: timetable and budget • Factors

– Complexity of the software – often hard to know in advance

– Productivity of the team • Skills

• Experience • Spirit

– Required quality

(14)

GSD -

(15)

Reasons for distributed development

• Size of the project

– One team cannot finish in time

– Not enough resources available otherwise • Need for additional competences

– HW, databases, UI – Domain knowledge • Business

– In some places development is cheaper

– Presence close to customer needed for the business

(16)

Global SW development

• SW developed by a geographically distributed organization

– Reasons like above

• Often main reason is lower salary costs – Also called off-shore development – Real savings debated

(17)

Business set-ups

Scale own development

By resources

from foreign company

Handle management Buy a

(18)

GSD problems

• Communication distance: (timezone, language cultural..).

• Serious issue since communication is major challenge in SW

• Spirit, motivation, committment – ”They and us” attitude

(19)

Guidelines for planning distributed projects

• Design for partitioning

– Or it will happen anyways

– Conway’s law: ” organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations”

• Facilitate communication • Add extra effort for

– Communication

– Solving problems due to bad communication in your plan

(20)

planning of a global SW project

Work packages & their relations

Teams

& their relations

GA

Work

distribution

(21)

GSD Work Distribution Model 21 Available resources System information Skill -area -level Team -cost -capacity -performance Slot Communication distance Component -estimated effort _{Responsibility} -skill System requirements * * * * * 0..* 1..*

(22)

Genetic Algorithms (GA) are adaptive heuristic search algorithms inspired by the natural selection and genetics.

The system runs through several generations

• A solution is encoded as genes and altered through mutations

• New solutions found also by combining parts of two existing solutions (parents)

• The best individuals survive from each generation.

(23)

A B C D A B C D A B C D A B C D A I C D A B E D A H C D O B C D A I C D A B E D A H C D O B C D A I C D A H C D A I E D A H C D A I C D A B E D A H C D O B C D Mutations Selection

(24)

scheduling in GSD (1/3) Genetic Algorithm Initial work distribution and schedule - Change team - Change development order Cost Duration

Set of optimal work distributions and

schedules

Mutations Fitness function

Result Team and WP

characteristics encoded in genes

(25)

12/09/2014

scheduling in GSD (3/3)

25

 Crossover exchanges genes between two parent chromosomes

 The fitness function evaluates the goodness of the solution (individual in a generation).

 Two factors: cost and lead-time determine the fitness

 Pareto optimality approach is used for selecting the survivors  In practice we seek points where improvement

of one criteria (e.g. cost) can only be optimized by compromising another (e.g. time).

TIE-21100/21106

Time Cost

(26)

London is faster but Bangalore is cheaper.

(27)

Time vs Cost

Time is expensive! Cost Time Minimum cost Minimum

(28)

Partitioning of the project

Cost/people

Impossible to partition

(29)

CONTINUOUS INTEGRATION,

DELIVERY, DEPLOYMENT

(30)

(http://www.martinfowler.com/articles/continuousIntegration.html)

• Suggested by many agile methods, for example XP. • The team integrates latest changes to complete

system frequently – even several times a day. • Often combined with test-driven development • Benefits

– Reduced risk (and earlier discovery) – Helps in getting rid of bugs

– Avoid chaos at the end (integration hell) – Changes behavior of programmers,

(31)

Practices of continuous integration

• Maintain a Single Source Repository. • Automate the Build

• Make Your Build Self-Testing

• Everyone Commits To the Mainline Every Day • Every Commit Should Build the Mainline on an

Integration Machine • Keep the Build Fast

• Test in a Clone of the Production Environment

• Make it Easy for Anyone to Get the Latest Executable • Everyone can see what's happening

(32)

Continuous integration, when not practical

• Target platform is too different – for example embedded SW

• System is big, and building takes long (e.g. hours) => Daily builds (often nightly builds)

– By agreed time of the day every developer needs to deliver a functional version of his/her

component

– Build server compiles and links the system – Automated tests are run and/or

(33)

Continuous integration

App Developes Version mgmt Build servers CI server 2. Signal change 3. Init build 4. Check out 5. Report results 6. Feedback Other stakeholders 1. Check in

(34)

Version Management Private Workspace Checkout Mainline Build and Test System Make Changes Build and Test System Commit changes Build Server Version Management Build and Test System fail Check-in to Build Server

(35)

Practices of continuous integration

• Automate Deployment

Limit number of branches

(36)

Practices of continuous integration

• Manage the dependencies for the source code

• Builds are based on a schedule or triggered by source code changes • Possibly run the build as

distributed processes

• Generating build notes and other documentation

(37)

Practices of continuous integration

Challenges:

• increased work effort needed to create the automated tests

• need to test user interaction or external interfaces that are difficult to automate

• the amount of processing

power needed to run the tests repeatedly.

(38)

Practices of continuous integration

NOTE: this is not always possible

• Prevents potential problems or conflicts to be hidden in the local copies of the software. • The rule of thumb in CI is to commit software

changes every day. In practice it is often beneficial to commit even more frequently.

(39)

Practices of continuous integration

Potential solutions to the problem: • Staged build or build pipeline • Staged testing

• Distributed build and testing • Raw processing power.

(40)

Practices of continuous integration

(41)

Practices of continuous integration

(42)

Practices of continuous integration

Status displays in separete and public screens (but maybe not in cafeteria)

(43)

Practices of continuous integration

(44)

An example tool for CI: Jenkins

• Very commonly used open source tool

• Run build (e.g. Ant) and automatic test (e.g. Junit) scripts • Integrates with very many version management systems • Can be triggered automatically by version management • Can be triggered by sending email

• Build running in batch mode – users can see the status • Let look at tutorial at:

http://www.vogella.com/tutorials/Jenkins/article.html

(45)

Continuous delivery and deployment

(http://blog.crisp.se/2013/02/05/yassalsundman/continuous-delivery-vs-continuous-deployment)

(46)

A/B Testing

Should our project have A or B? Implement a way to collect statistics Implement A Implement B Deploy B Usage statistics Usage statistics Compare

(47)

Background info – what is DevOps

• DevOps is about faster deployment of new innovations by combining technology, culture, practices and processes

• Some factors (sources: wikipedia, http://dev2ops.org)

– Use agile methodologies

– Demand for an increased rate of production releases – Cloud infrastructure from internal and external providers – Unified tools / unified processes

Business need Develop- ment Operation Agile DevOps • See: http://dev2ops.org

(48)

Summary of the demo

Integrated pipeline from coding to deployment and

operation

Edit Build _Test _Deploy

Operate

The developer just needs a browser. No SW

installation needed.

Deployment to brokered cloud: system finds the

most suitable provider and deploys there

(49)

Source code repository Commit Test Package Build Reporting Project configuration Deploy App COAP S Public or Private PaaS runtimes App users App Developer Develop code

(using web-based IDE)

Architecture that delivers

(50)

(51)

Concrete example Need-For-Speed

How to ensure fast pipeline for continuous deployment? We have worked on real data/projects/companies

Does the agile process work as it should? Project mgmt, Issue tracking Codin g Version mgmt Build Testing Deploy- ment Use Nature of changes, changing role of developers?

Are there particular reasons for problems

Are right tests done in right place to

(52)

(53)

Material

• Martin Fowler: Continuous Integration

http://martinfowler.com/articles/continuousIntegration.html

• Ashish Kumar: Development at the Speed and Scale of Google (Presentation + Video)

http://www.infoq.com/presentations/Development-at-Google#.TuYqU_kvSuI.email

• Tero Piirainen: Continuous Delivery or: How I Learned to Stop Worrying about Releasing and started to Love my Customers • Keynote by prof Jan Bosch,

http://www.tut.fi/fi/tietoa- yliopistosta/laitokset/tietotekniikka/laitoksen-esittely/tapahtumia/viksut-vekottimet/index.htm • Web page of need for speed project