Break It Before You Buy It!
Test Driven Development and Continuous Integration
Chris Hartjes -- CodeMash 2011 -- @chartjes http://www.littlehart.net/atthekeyboard
The Problem
•
“Build us a new fantasy games platform.”
•
“Oh yeah, you’re the only one who will be
doing the work.”
•
“Testing? Can’t we just click around on the
The Solution
•
Use a framework with a commitment to
testing so writing tests will be “easier”
•
Automate of as much stuff as I can
•
Do some pragmatic TDD
“Test-driven development (TDD) is a software development process that relies
on the repetition of a very short development cycle: first the developer writes a failing automated test case that defines a desired improvement or new function, then produces code to pass that test and finally refactors the new code to
acceptable standards. Kent Beck, who is credited with having developed or 'rediscovered' the technique, stated in 2003 that TDD encourages simple designs and inspires confidence.”
“In software engineering, continuous integration (CI) implements continuous processes of applying quality control — small pieces of effort, applied frequently. Continuous integration aims to improve the quality of software, and to reduce the time taken to deliver it, by replacing the traditional practice of applying quality control after completing all development.”
Someone did a study...
http://www.infoq.com/news/2009/03/TDD-Improves-Quality
15% - 35%
Pay now, or pay later, but
YOU WILL PAY
“Our general process for rails upgrades is pretty
consistent. First make the app load in the limited
sense, avoiding occurrences of LoadError whilst
starting rake. Focus on the minimum number of
changes needed to get rake running, then unit,
functional and integration tests.
The goal is to
have tests pass, not to convert to the newest API
or experiment with new Rails 3 stuff. You only add
what must be added to reach the goal.”
http://www.getharvest.com/blog/2011/01/harvest-is-running-rails-3/
I am not a
Testing Zealot!
•
I prefer to test things that are complicated
•
100% test coverage is nice but not practical
Behavior driven development (or BDD) is an agile software development technique that encourages collaboration between
developers, QA and non-technical or business participants in a software project. It was originally named in 2003 by Dan North as a response to
Test Driven Development, including Acceptance Test or Customer Test Driven Development practices as found in Extreme Programming. It has evolved over the last few years.
BDD focuses on obtaining a clear understanding of desired
software behaviour through discussion with stakeholders. It
extends
TDD
by writing test cases in a natural language that
non-programmers can read. Behavior-driven developers use
their native language in combination with the ubiquitous
language of
domain driven design
to describe the purpose
and benefit of their code. This allows the developers to
focus on why the code should be created, rather than the
technical details, and minimizes translation between the
technical language in which the code is written and the
domain language spoken by the business, users,
BDD using PHPUnit
<?php require_once 'PHPUnit/Extensions/Story/TestCase.php'; require_once 'BowlingGame.php';class BowlingGameSpec extends PHPUnit_Extensions_Story_TestCase
{
/**
* @scenario */
public function scoreForOneSpareIs16()
{
$this->given('New game')
->when('Player rolls', 5) ->and('Player rolls', 5) ->and('Player rolls', 3) ->then('Score should be', 16); } // ... } ?>
Your goal is to build the
best application possible
with the fewest bugs
making it out into
Automation is
your secret
TDD/BDD and CI add
automation to your
software development
workflow.
PHP 5.3.3
+
Zend Framework
+
PHPUnit
Python 2.6
+
py.test
What I Used
public function testStatsByDateRange() {
$expectedResults = unserialize(file_get_contents(TEST_PATH . '/fixtures/ rfs-1.txt')); $params = array( 'league' => 'testhockey', 'startDate' => '2010-10-07', 'endDate' => '2010-10-10' ); $this->rfs->getByDateRange($params); $results = $this->rfs->sortByFantasyPoints();
$this->assertTrue(count($results) > 0, 'Checking for results');
$this->assertEquals($results, $expectedResults, 'Got expected result set'); $this->assertTrue(isset($results[0]['goals']), 'Checking for goals');
$this->assertTrue(isset($results[0]['assists']), 'Checking for assists'); }
import py
import football_scoring
def test_football_empty_data_set(): expected_points = 0
test_data = dict()
assert expected_points == football_scoring.qb_points(test_data) assert expected_points == football_scoring.rb_points(test_data) assert expected_points == football_scoring.rec_points(test_data) assert expected_points == football_scoring.k_points(test_data) assert expected_points == football_scoring.def_points(test_data) def test_football_qb_points(): test_data = { 'league': 'football', 'yards': 300, 'interceptions': 1, 'touchdowns': 1, 'two_point_conversions': 1 } expected_points = 15
So how can you test
your PHP code and
your Python code at the
same time?
Do you also want to do this every
time
What is the bare minimum
required for continuous
Automated running of
your tests!
TELL ME WHAT
WENT WRONG!
Buildbot
•
PROS: It’s Python. Infinitely flexible.
•
CONS: It’s Python. Infinitely flexible.
(I highly recommend you read Jacob Kaplan-Moss’ series on his experiences
using Buildbot for Django testing)
Cruise Control
•
PROS: pioneer in CI solutions field, easy to
find help
•
CON: Unfriendly configuration, GUI shows
Hudson
(soon to be renamed Jenkins)
•
PROS: Very easy configuration, awesome
plugin ecosystem
•
CON: Forces some conventions, writing
Can you guess which
one I picked?
No screenshots of
Hudson, that’s just
Configure it to be as close to
your production setup as possible
Database
server
Web server
Interpreter
or VM
Support
libraries
Testing
Configure your version control
system to talk to Hudson every time
you commit code.
Plugins I Use, YMMV
•
Hudson Clover (for code coverage
reports)
•
Create Job Advanced (tells you who
triggered the job)
•
Hudson Email Extension (better email
More Plugins I Use
•
Subversion plugin
•
xUnit Plugin (to support PHPUnit
Write a shell script to
automatically run your tests
for you
mkdir /var/www/games-hudson/${BUILD_ID} cd ${WORKSPACE}/games
/usr/local/zend/bin/php doctrine-cli migrate cd ${WORKSPACE}/games/tests /usr/local/zend/bin/phpunit --log-junit=${WORKSPACE}/build/logs/phpunit-results.xml cd ${WORKSPACE}/games/scripts /usr/bin/py.test --junitxml=${WORKSPACE}/build/logs/pytest-xmlrunner.xml cp -R /var/lib/hudson/jobs/${JOB_NAME}/workspace/games/* /var/www/games-hudson/${BUILD_ID} chmod -R 777 /var/www/games-hudson/${BUILD_ID}/public/images rm -rf /var/www/games-hudson/current ln -sf /var/www/games-hudson/${BUILD_ID} /var/www/games-hudson/current
Configure Hudson to
send emails every time
Games-Corpdraft - Build # 635 - Successful:
Check console output at
http://
sekretserver.xmlteam.com:9999/job/Games-Corpdraft/635/
to view the results.
Changes for Build #635
[Chris Hartjes] Fixed date range for week 17
What Hudson Sends Me
Started by remote host 204.15.197.99
Reverting svn+ssh://leafs.xmlteam.com/opt/repos/games/trunk Updating svn+ssh://leafs.xmlteam.com/opt/repos/games/trunk U application/configs/application.ini
At revision 819
No emails were triggered.
[workspace] $ /bin/sh -xe /tmp/hudson7579535609700402003.sh + mkdir /var/www/games-hudson/2011-01-04_19-27-56
+ cd /var/lib/hudson/jobs/Games-Corpdraft/workspace/games + /usr/local/zend/bin/php doctrine-cli migrate
<snipped weirdness from running migrations>
+ cd /var/lib/hudson/jobs/Games-Corpdraft/workspace/games/tests
+ /usr/local/zend/bin/phpunit --log-junit=/var/lib/hudson/jobs/Games-Corpdraft/workspace/build/logs/phpunit-results.xml PHPUnit 3.4.14 by Sebastian Bergmann.
...
Time: 25 seconds, Memory: 32.25Mb [30;42m[2KOK (36 tests, 119 assertions)
[0m[2K+ cd /var/lib/hudson/jobs/Games-Corpdraft/workspace/games/scripts
+ /usr/bin/py.test --junitxml=/var/lib/hudson/jobs/Games-Corpdraft/workspace/build/logs/ pytest-xmlrunner.xml
============================= test session starts ============================== platform linux2 -- Python 2.5.2 -- pytest-1.3.2
test path 1: /var/lib/hudson/jobs/Games-Corpdraft/workspace/games/scripts test_baseball_scoring.py ....
test_football_scoring.py ...
generated xml file: /var/lib/hudson/jobs/Games-Corpdraft/workspace/build/logs/pytest-xmlrunner.xml
========================== 11 passed in 0.07 seconds ===========================
+ cp -R /var/lib/hudson/jobs/Games-Corpdraft/workspace/games/application /var/lib/hudson/ jobs/Games-Corpdraft/workspace/games/build.xml /var/lib/hudson/jobs/Games-Corpdraft/
workspace/games/doctrine-cli /var/lib/hudson/jobs/Games-Corpdraft/workspace/games/
doctrine_generator.php /var/lib/hudson/jobs/Games-Corpdraft/workspace/games/imports /var/ lib/hudson/jobs/Games-Corpdraft/workspace/games/library
/var/lib/hudson/jobs/Games-Corpdraft/workspace/games/public /var/lib/hudson/jobs/Games-Corpdraft/workspace/games/ scripts /var/lib/hudson/jobs/Games-Corpdraft/workspace/games/tests /var/lib/hudson/jobs/ Games-Corpdraft/workspace/games/tmp /var/www/games-hudson/2011-01-04_19-27-56
+ chmod -R 777 /var/www/games-hudson/2011-01-04_19-27-56/public/images + rm -rf /var/www/games-hudson/current
+ ln -sf /var/www/games-hudson/2011-01-04_19-27-56 /var/www/games-hudson/current Recording test results
[xUnit] [INFO] - Starting to record.
[xUnit] [INFO] - Processing PHPUnit-3.4 (default)
[xUnit] [INFO] - [PHPUnit-3.4 (default)] - 1 test report file(s) were found with the pattern 'build/logs/*-results.xml' relative to '/var/lib/hudson/jobs/Games-Corpdraft/workspace' for the testing framework 'PHPUnit-3.4 (default)'.
[xUnit] [INFO] - Converting '/var/lib/hudson/jobs/Games-Corpdraft/workspace/build/logs/ phpunit-results.xml' .
[xUnit] [INFO] - Stopping recording. Email was triggered for: Success Sending email for trigger: Success Sending email to: sekret@xmlteam.com Finished: SUCCESS
Games-Corpdraft - Build # 611 - Failure:
Check console output at http://sekretserver.xmlteam.com:9999/job/ Games-Corpdraft/611/ to view the results.
Changes for Build #611
[Chris Hartjes] Add category to league definition
