Technology
Report
2012
Web server nginx, Unicorn Client-side Backbone/Spine, JQueryUI, HTML5/CSS3 Authentication Devise/Sorcery Authorisation CanCan Database PostgreSQL Caching MemCache Persistence/ORM ActiveRecord Document store Mongoid, MongoDB Search Solr, Tire/ElasticSearch Auditing PaperTrail Long-running tasks Resque API REST JSON/HTTPS Scheduling Resque Scheduler Admin ActiveAdmin App server Rails 3.1, Ruby 1.9.2 Hosting EngineYard/Heroku App startup
foreman to start the processes defined in the profile
Testing
Cucumber, rspec, jasmine + jasmine_headless_webkit for javascript unit testing
Coverage
cane, rcov (1.8.7), simplecov (1.9.x), roodi, reek, flog and flay. Metrics_fu for a consolidated report but not good for check-in builds.
Monitoring Airbrake, Stethoscope, Nagios Web server Apache httpd (+mod_proxy) Client-side JQueryUI, HTML5/CSS3 Authentication Spring Security Authorisation Spring Security Database (any jdbc-driven db) Caching
Ehcache & Spring Cache
Persistence/ORM Hibernate Document store MongoDB Search Solr, ElasticSearch Auditing Envers Long-running tasks Quartz API REST JSON/HTTPS Scheduling Quartz Scheduler Admin Custom App server Apache Tomcat Hosting EC2 App startup Tomcat startup Testing JUnit, selenium Coverage Cobertura, CodeNarc Monitoring NewRelic Web server Apache httpd (+mod_proxy) Client-side JQueryUI, HTML5/CSS3 Authentication Spring Security Authorisation Spring Security Database (any jdbc-driven db) Caching
Ehcache & Spring Cache
Persistence/ORM GORM (Hibernate) Document store MongoDB Search Solr, ElasticSearch Auditing Envers Long-running tasks Quartz API REST JSON/HTTPS Scheduling Quartz Scheduler Admin Scaffolding App server Apache Tomcat Hosting EC2 App startup Tomcat startup Testing JUnit, cumber-jvm, geb, selenium Coverage Cobertura, GMetrics, CodeNarc Monitoring NewRelic
All facts, no fiction
As a technology company that builds
transactional web and mobile applications,
and software and devices for the Internet of
Things, we make data-driven decisions about
the areas on which we should be concentrating.
We use a framework for making those
evidence-based decisions internally, and from the feedback
we received from our customers we are confident
you would find a selection of that data useful too.
The DiUS Technology Report differs from others in that it is based
on empirical data rather than relying solely on our opinion. It brings
together, into a single view, data obtained by measuring the hours
we expend and what our customers and staff think. The report
contains observations of measurements of how our work profile
has changed over the last 4 years of the 7 years DiUS has operated.
It also contains highlights of feedback from surveys we conducted
of our customers and our staff.
We present an up-to-the-minute view of our technology stack
recipes – combinations of components we have proven to work well
together – that we have developed and carefully refined on real-world
projects we have delivered for our customers. We use and constantly
update these recipes for our projects; we offer them in the Technology
Report in the hope they can help you get a head start on your next
Rails, Grails, or Java project.
The Technology Report summarises the effort of our staff in Melbourne
and Sydney. My thanks to the DiUS founders Joe Losinno and Clency
Coutet for their support, to Mick Reidy and Ricky Yim, our Delivery
Managers, for their input and advice, to our Engagement Principals
Paula Ngov and Alistair McGregor for their help with the customer
survey, and to Jerome Lebel-Jones for his valued creative input.
By publishing this report, we have set out to add to your set of
useful points of reference for technology choices. However, applying
the principle of measuring value and avoiding assumptions, please
send your feedback directly to me at
[email protected].
Daryl Wilding-McBride
Chief Technology Officer
Daryl is the CTO at DiUS, where he is responsible for executing DiUS’s technology strategy: to expertly apply the latest innovations in technologies that increase developer productivity and help focus on delivering user-centered and more scalable enterprise-grade software applications and devices for customers.
Our recipes for success
Rails
Rails is increasing in popularity among developers and customers for its productivity benefits, maturing test frameworks, and vibrant ecosystem of gems and community support. Although in the past it was sometimes found to be lacking in performance and ability to horizontally scale, the Rails community has been busy making better performance a priority, to the point now where customers can confidently use Rails for their highest traffic applications.
Grails
Grails is gaining in popularity for customers who want the productivity benefits of Rails but who want to continue to deploy on their JVM infrastructure. Grails is also attractive for organisations who have a Java application heritage and want to leverage Java libraries and services.
Java
Java continues to be a requirement for many organisations who have established infrastructure and support processes based on the JVM, who have in-house skills in the Java language, and who want to leverage their collections of Java libraries built up over a number of years.
Pg. 3
HTML5: 2% Embedded C/C++: 11% Groovy/Grails: 6% Ruby/Rails: 43% Java: 36% Objective-C: 2%
only 5% of our effort is spent
in the advertising/media
domain, making us a more
diverse company than previously.
5
%
24,889
The total amount of
hours spent on hardware
development since November 2009,
now a significant part of DiUS’s work.
DiUS delivers its first
native app for the iPhone
within 6 months of the
Apple App Store opening
In November 2011 DiUS
delivers its first
cross-platform native app on
iPhone using Titanium
Monthly
hours of effort
in Rails first
overtook Java in
May 2008
Syst
em
T
ec
hno
logy
Jan 2008 MAY 2008 SEP 2008 Jan 2009 MAY 2009 SEP 2009 Jan 2010 MAY 2010 SEP 2010 Jan 2011 MAY 2011 SEP 2011 Jan 2012
44
%
of our business came
from the advertising/media
domain in 2008
D
omain
Agile comes of age: in March 2011
DiUS brings Agile methodology
into the highly regulated and
process-driven medical field.
DiUS expands
its repertoire
to a total of
12 technologies
DiUS mainly uses
only two core
technologies to
deliver applications
85
%
of all DiUS effort in
January 2008 is for
31
%
transactional web apps
DiUS’s focus has truly shifted
into multi- channel/emdebbed
delivery systems in addition
to traditional web apps.
the overall effort
for non-web app
work during
January 2012 was
In November ‘09 DiUS was the
first Australian company to work
for the electric car industry
producing charging technology.
14,000
+
Hours spent to date on
Embedded C/C++ since
November ‘09.
1461
Days of DiUS
november ‘09
DiUS initiates its
first hardware
development project.
Rails: 19% Other: 1% Java: 80% 19% 80% 1% 6% 2% 2% 43% 11% 36%Medical – 30%
Software Tools Development – 1%
Logistics – 9% Education – 6% Banking – 2% Travel – 1% Electric Vehicles – 5% Utilities - 30% Search – 9% advertising/Media – 5% Retail – 2%
Percentage of
work by domain
2012
Customer Insights
Over the coming 12 months, our clients will focus on...
At DiUS we believe it’s no use having great ideas or being creative if there’s no focus on shipping. Our customers are also attuned to this in their responses:Reducing time-to-market for new products and/or services (much interest = 72%). This increases to a whopping 80% beyond the next 12 months.
Removing deployment bottlenecks with frictionless/continuous delivery (much interest = 69%)
Reducing waste with Lean software development (much interest = 73%) The value of UX is being recognised more broadly, with 73% of respondents expressing “much interest” in “focussing on user-centred design”
An issue very close to our hearts, “initiating and maintaining your innovation culture” had the highest interest level of the entire survey with 100% of respondents expressing either some or much interest and 3 out of 4 respondents rating it of “much interest”.
Somewhat surprisingly...
The somewhat lower-than-expected overall result for improving stakeholder engagement. This might mean most customers already have it sorted, though surely it can always be improved?
There was a fairly split feeling about Agile architecture, with roughly the same number of respondents having ‘much’ interest as do those with ‘no’ interest. Whilst Agile delivery has come of age, the time is yet to come for Agile architecture.
Weaker interest than expected in cloud infrastructure. The most likely reason for this is that Australian organisations are still keeping their infrastructure in-house due to concerns about data jurisdiction.
Web application frameworks
Rails (Ruby) and Spring (Java) hold more interest than Grails, Roo, or Play. Roo and Play are still (b)leading edge and not production-ready, so those results are not a surprising outcome for the next 12 months. Grails, however, has been used by DiUS to deliver a few reasonably large apps in complex domains and we believe it is production-ready, so the strength of the ‘no’ interest response was a little surprising.
Building apps for mobile devices
Respondents expressed a strong interest in native apps and web apps for mobile devices, but there was less interest in cross-platform frameworks. This is consistent with DiUS’s conclusion; we feel we get more control over the user experience by using the native language of the platform. We recently delivered the Taylor iOS app using Titanium and then invested a few days of internal innovation resource to estimate the effort to create an Android version starting with the same code-base. To create a functionally-equivalent Android version, we estimated it would take another 40% of the effort taken to build the iOS version. Respondents, however, feel this will change over the medium term, with cross-platform becoming of interest beyond the next 12 months. This probably reflects a belief the maturity of these frameworks will improve in that timeframe.
noSQL data stores
It was surprising that there was not stronger interest in NoSQL databases in the short term, given their strengths in scalability, availability, and flexibility. This is probably because respondents’ business objectives are being adequately met with traditional RDBMSs, and their corresponding operational and support structures are in place and satisfactory. However, respondents feel the importance of NoSQL data stores will increase beyond the next 12 months.
A key input into our decisions about technology direction is what our
customers think. Rather than make assumptions, we asked them about
their main concerns, and their thoughts on technology choices in the
near- and medium-term. The respondents were from the full range
of business domains in which DiUS operates. The most-represented
role was Senior or Group Manager (28%), but we also had Program
Managers (15%), Team Leads (10%), Architects (8%), CEOs (8%),
CTOs (10%), and Delivery Managers (13%).
A reliable indication of what will become mainstream in the
future is what our staff are investigating out of their curiosity
and their drive to improve what they do. We surveyed our
staff to get another key input into our decision-making about
technology direction. Of the respondents, 65% were software
engineers or developers, 9% were business analysts, 13%
were team leads, and 7% were architects.
DiUS staff strongly favour convention-over-configuration frameworks like Rails and Grails for their productivity. However, 54% said they would like us to do more Rails, and 48% said more Grails. In contrast, 52% of respondents said we are doing about the right amount of Java.
Most staff thought we are doing about the right proportion of web apps, consulting assignments, infrastructure, and building shared services. However, a big majority would like to do more mobile apps (80%).
Our objective is to hire people who are always looking on the horizon for better ways of doing things and adopting them. This attribute is reflected in the broad range of answers to the question about what technologies or knowledge areas we should be learning about even more deeply:
Functional programming
UX, design thinking
Highly-available noSQL
CoffeeScript
Post-Java languages on the JVM, especially Scala
amazon Web Services
80
%
said they would like to do more mobile apps
52
%
said we are doing about the right amount of Java
48
%
would like us to do more Grails
54
%
would like us to do more Rails
Percentage of
work by domain
2008
Insurance – 2% Banking – 8%
Software Tools Development – 9% Logistics – 1% Social – 8% Retail – 2% Utilities – 20% Search – 6% advertising/Media – 44% Pg. 7 Pg. 6
Employee Insights
Top 10 technology tips
for the next 12 months
Melbourne
Paula Ngov
Engagement Principal
[email protected]
P +61 3 9008 5400
Level 10, 99 Queen Street
Melbourne 3000 VIC Australia
Sydney
Alistair MacGregor
Engagement Principal
[email protected]
P +61 2 8014 6640
Level 5, 220 George Street
Sydney 2000 NSW Australia
1
For their ability to help developers deliver features more quickly and with more agility, convention-over-configuration application frameworks such as Rails and Grails will continue to reduce Java’s past dominance of the web application development scene. We expect Rails and Grails to increase their share of our work and further eclipse the Java share over the next 12 months.2
Grails will increasingly find a place in the technology stacks of customers who don’t want to move away from the JVM.3
New Java-based convention-over-configuration frameworks such as Play will become production-ready.4
While in the past developers who only have mastery of Java would be in demand, today and into the future it will be critical for a developer to have mastery of two or three languages. We expect those to be drawn from Ruby, Objective-C, Groovy, JavaScript, and Java. We expect functional programming languages such as Scala to begin to play a part for specific problem domains.5
The use of NoSQL databases, especially MongoDB,will increase as customers are attracted to their light-weight approach to horizontal scalability and high availability compared with traditional RDBMSs such as Oracle, MySQL and Postgres.
