Spray – A quick way of creating Graphiti
Developer GuideAuthors: Jos Warmer, Karsten Thoms, Joerg Reichert Contents
1. Spray - A quick way of creating Graphiti 1. Target Runtime Framework
2. Reference Implementation 3. Framework Development 4. Language Development 5. Code Generation 6. Testing 7. User Interface
2. Setting up the developer environment 1. Source Control 2. Developer IDE 1. IDE 2. Update Sites 3. Features to Install 4. Troubleshooting
5. Add Spray Git repository to Eclipse 6. Add Spray task repository to Eclipse 7. Add Spray Jenkins connector to Eclipse 3. Target Platform
4. Workspace Setup
3. The Spray DSL Infrastructure 4. Spray Generator 1. Reference Implementation 2. Indentation 3. Import Organizing 4. Modularity 1. Template Methods 2. Hook Templates 3. Generated Code 5. Code Documentation 1. Overridden Methods 5. Graphiti 6. Project Setup 1. Naming 2. Source Management 3. Adding a new Plugin project
1. Manifest 2. build.properties
3. Project Specific Settings 4. Build
4. Dependency Management 1. Imported Packages 5. Plugin Development
1. plugin.xml 6. Spray Maven Build 7. Parent POM 1. Plugin Management 8. Profiles 1. Profile @skip-ui-tests@ 7. Continuous Integration 1. Introduction 2. Build Jobs 1. Spray CI Build
2. Spray Experimental Build 3. Administration
8. Release Process 9. Testing
1. Test Projects 2. Test aspects
1. Testing using the Spray DSL Editor enviroment 2. Testing the generated Graphiti editor
3. Test approaches 1. JUnit tests 2. Mocking 3. Xtext tests 4. JUnit plug-in tests 5. UI tests
6. Measuring test coverage 7. Test data organization 8. Generator template tests 4. Spray Language Tests 5. Runtime Tests
10.Issue Tracking 1. Tracking System 2. Git Commit Message
General rules for designing Spray
Target Runtime Framework
The target graphical modeling framework of Spray is Graphiti. Though other runtimes might be possible to describe with the DSLs, this is not the primary goal.
Reference Implementation
All proposed features that should be covered by code generation must be also covered by Reference Implementation code. This reference code must be developed in a way that it can be fully generated from Reference Models.
The quality of this reference code is crucial.
Framework Development
Although Graphiti has already a strong API it will be necessary to create framework classes also for Spray. The Spray framework API should be developed in a way that it will be potentially proposed to Graphiti as enhancement requests.
The framework layer of Spray should be as small as possible and as large as required.
Language Development
The DSLs developed in Spray should follow best practices in Language Design.
• Keep it simple: creating a simple Graphiti editor should be straightforward and quick to do
• More advanced features may not break the previous rule. They should be additive in the sense that you won’t see them if yo don’t need them.
• The Spray DSL does not need to enable everything possible with Graphiti. For advanced features, users will always be able to directly use the Graphiti API and combine such handwritten code with generated code. • If a language is hard to parse it is also likely hard to understand.
• Separation of concerns: If different concerns must be described in a DSL, it might require different DSLs
Code Generation
• Code Generator is designed to enable mixing generated code with handwritten code without losing the possibility for code generation.
• Code Generator uses the generation gap pattern to give maximum flexibility to mix handwritten code with generated code.
• The Code Generator must be executable standalone (Maven, Ant, Commandline) without an Eclipse installation • The Code Generator should be extensible by the user without the need to change the original plugin sources
Testing
All features of the DSLs will be covered by models that will be tested by unit tests.
User Interface
Setting up the developer environment
Source Control
We are using Git as source management system. The primary repository location is: https://code.google.com/a/ eclipselabs.org/p/spray/
See http://code.google.com/a/eclipselabs.org/p/spray/source/checkout for details.
You can new clone Git from the command line or use Eclipse, which is explained in a later section of this guide.
Developer IDE
IDE
Download latest Eclipse Indigo Classic appropriate for your platform: http://www.eclipse.org/downloads/packages/ eclipse-classic-371/indigosr1
Update Sites
The update sites used to install the features required to develop Spray can be imported in your IDE. As a base take any Eclipse 3.7 distribution. Then open workspace preferences Install / Update -> Available Software Site and click the Import button. When you already cloned the Spray repository, then choose the file
<path to your locally cloned Spray Git repository>/org.eclipselabs.spray.devenv/ updatesites-spray-dev.xml
otherwise import the file from the CI server:
https://spray.ci.cloudbees.com/job/spray-ci-build/lastSuccessfulBuild/artifact/ devtools/org.eclipselabs.spray.devenv/updatesites-spray-dev.xml
This xml file will add all required update sites to visit later when installing the plug-ins for the development environment.
Features to Install
Choose now Help -> Install New Software. Choose the following update site in „Work with”: Spray Development Environment, this will install
• MWE SDK 1.2.1
• MWE 2 language SDK 2.2.0 • MWE 2 runtime SDK 2.2.0
• EMF – Eclipse Modeling Framework SDK 2.7.0 • Graphiti SDK 0.8.1
• Xtext Antlr 2.0.0 • SWT Bot
• xtext-utils unittesting • Xtext SDK 2.2.0
• EclEmma Java Code Coverage 1.5.3 • Mylyn 3.6.2
• EGit 1.1.0 • Zest SDK 1.3.0
Troubleshooting
If you have already installed one of the above mentioned features you might experience problems installing the newer features. In this case uninstall the features in question before (Eclipse / About Eclipse / Installation Details / Installed Software -> Uninstall).
For example, if you need to upgrade Xtext, uninstall all Xtext features, and the install Xtext SDK again with Install New Software.
Add Spray Git repository to Eclipse
You can use our predefined team project set file to get the Spray projects into your workspace: • File > Import... > Team/Team Project Set, Next
• File: <path to your locally cloned Spray Git repository>/devtools/org.eclipselabs.spray.devenv/projectSet.psf, Finish
This will clone the Spray repository into your Eclipse workspace folder, create a number of working sets and add the projects to the working sets. To see the working sets go to the „View menu” icon of the Package Explorer and select Working Sets as Top Level Elements.
If you want to specify the location where to clone the Spray repository locally do the following: • File > Git/Projects from Git
• Add
• <Path to your locally cloned Git repository>, Next • Search, OK
If you had cloned the Spray repository locally and want to use that repository: • File > Git/Projects from Git
• Clone
• Paste https://code.google.com/a/eclipselabs.org/p/spray/ in the URI field and add your Google Account user name and your GoogleCode.com password (if your are logged in at the Spray Google Code site, click the Profiles link in the upper right corner and then go to the Settings tab) , Next
• select all branches, Next
• select your local path to store the Git repository clone, Finish
If you want to apply the Spray workset configuration without being forced to clone the Spray repository again, you can copy devtools/org.eclipselabs.spray.devenv/workingsets.xml to </path/to/workspace/.metadata>/.plugins/ org.eclipse.ui.workbench/workingsets.xml
Add Spray task repository to Eclipse
• Window > Show View > Other... > Mylyn/Task Repositories
• Open context menu in Task Repositories view > Add task repositories... • Google Code, Next
• URL: Paste http://code.google.com/a/eclipselabs.org/p/spray/ • Label: Spray
• unselect Anonymous, if you later want to push code
• enter your Google Account credentials (now use as password your Google Mail password, not the Google Code Password), Finish
• Add New Query
• use predefined query > Open issues
• Window > Show View > Other... > Mylyn/Task List
Add Spray Jenkins connector to Eclipse
• Window > Show View > Other... > Mylyn/Builds
• Click on the first of the icons in the right of the view header > Hudson (supports Jenkins), Next • Server: https://spray.ci.cloudbees.com
• Label: Spray • select Anonymous
• click on Refresh button • select job "spray-ci-build" • Finish
• click on spray-ci-build to show the current job status
Target Platform
We are targeting for Eclipse Indigo. A target platform definition is provided in releng/
org.eclipselabs.spray.targetplatform/spray.target. To set this target platform: • Import the project from path releng/org.eclipselabs.spray.targetplatform
• Open Workspace properties, go to Plug-in Development / Target Platform • Here you should see Spray Target Platform. Check it and press Apply.
• Eclipse Platform SDK 3.7.1 - http://download.eclipse.org/eclipse/updates/3.7
• EMF – Eclipse Modeling Framework SDK 2.7.0 - http://download.eclipse.org/releases/indigo • Graphiti SDK 0.8.1 - http://download.eclipse.org/graphiti/updates/0.8.0
• MWE SDK 1.2.1 - http://download.eclipse.org/modeling/tmf/xtext/updates/composite/releases/ • Xtext SDK 2.2.0 - http://download.eclipse.org/modeling/tmf/xtext/updates/composite/releases/ • Xtend SDK 2.2.0 - http://download.eclipse.org/modeling/tmf/xtext/updates/composite/releases/ • Xtext Antlr 2.0.0 - http://download.itemis.com/updates/releases/2.0.0/
• xtext-utils unittesting - http://xtext-utils.eclipselabs.org.codespot.com/git.distribution/releases/unittesting-0.9.x • SWT Bot - http://download.eclipse.org/technology/swtbot/helios/dev-build/update-site
Distributions with Eclipse Indigo + Xtext 2.0 can be found here: http://download.itemis.com/distros/
Workspace Setup
When creating a new workspace make sure to set the following settings: Resource encoding: UTF-8.
Project settings are stored in the devtools/org.eclipselabs.spray.devenv> project. Whenever possible, project specific settings should be applied and the settings file checked in.
In org.eclipselabs.spray.devenv there is a Java formatter configuration formatter.xml used in this project.
The Spray DSL Infrastructure
This secting will contain information about• mapping to the DSL concepts to Graphiti concepts • implementation of expressions
• scoping • nodel interferer • usage of injection • etc.
Spray Generator
Explanation of the Spray Xtend2 templates implementation and design decisions (architecture), how does (different) models and their combination with the templates map to the generated artifacts (what should be the expected output)
Reference Implementation
All proposed features that should be covered by code generation must be also covered by Reference Implementation code. This reference code must be developed in a way that it can be fully generated from Reference Models.
The quality of this reference code is crucial.
Indentation
All Xtend templates should use 4 spaces instead of tabs. Please check your template Xtend code for occurances of the tab character.
You can visualize them in your workspace by setting the preference option General / Editors / Text Editors / Show whitespace characters.
To replace all tabs in the current file open the Find/Replace dialog and enter (values without the ' character, this is just used here to visualize the whitespaces)
• Find: '\t'
• Replace with: ' '
• Check option "Regular expressions"
Import Organizing
Import statements that must be computed from classes used within generated code must be evaluated before actually printing out the class. To do so, we use a similar pattern like in Xtext’s Domainmodel Example: Use an Import organizer, evaluate the class body with all qualified type names, and after evaluating it print out the class header with the collected dynamic imports and the body afterwards.
This requires the following pattern in the Xtend classes: • Derive template class from FileGenerator
• Inject extension NamingExtensions
class DiagramTypeProvider extends FileGenerator { @Inject extension NamingExtensions naming
• Implement mainFile() with the following steps: • Print class header and package statement • Add static imports
• Add // MARKER_IMPORT. This will denote the position where dynamic imports will inserted. • Print the class body content
def mainFile(Diagram diagram, String className) ''' «header(this)»
package «diagram_package()»;
import org.eclipse.graphiti.tb.IToolBehaviorProvider; // MARKER_IMPORT
«body» '''
• Use the extension function shortName on any qualified name. It will print out the simple name of the qualified class name and collects the qualified name for the import manager.
«metaClass.javaInterfaceName.shortName» // javaInterfaceName computes the qualified class name of an EClass interface
// prints out 'IFeatureProvider'
"org.eclipse.graphiti.features.IFeatureProvider".shortName
Modularity
Template Methods
The templates should be organized into reasonable small template definition methods. Usually a good separation is to provide a template for each generated method, which are called when generating a class body.
def mainFile (MetaClass metaClass, String className) ''' ...
public class «className» extends AbstractCreateFeature { ... «generate_canCreate(metaClass)» «generate_create(metaClass)» «generate_createModelElement(metaClass)» «generate_getCreateImageId(metaClass)» «generate_hasDoneChanges(metaClass)» «generate_canUndo(metaClass)» } '''
def generate_canCreate (MetaClass metaClass) ''' «overrideHeader()»
public boolean canCreate(ICreateContext context) { ...
} '''
When the implementation of a template method grows too large, or is separatable into a sequence of logical parts, the template should delegate to subtemplate methods again.
This makes the template code more maintainable, and allows users later to override smaller parts of the generator if customization is needed.
Hook Templates
To allow insertion of additional fields, constants, methods etc. in (Java) class bodies by subclassed templates, the base class FileGenerator offers the hook template methods generate_additionalFields() and
generate_additionalMethods(). These methods should be called in a class body template. The user then can override these methods to insert additional code without the need to override the class template itself.
def mainFile(MetaReference reference, String className) ''' ...
public abstract class «className» extends AbstractUpdateFeature { «generate_additionalFields(reference)» // call hook template public «className»(IFeatureProvider fp) { super(fp); } «generate_canUpdate(reference)» ...
«generate_additionalMethods(reference)» // call hook template }
'''
Generated Code
Also generated methods should not grow too large. The user should be able to override smaller methods in the extensions files. To be able to do this, the generated methods in base classes should mostly be public or protected, and not static nor final.
When appropriate, callback methods should be generated where it is likely for the user to make changes without the need to override a complete method. For example, when creating a connection, the user might want to decorate it.
Example (AddReferenceAsConnection.xtend):
def mainFile(MetaReference reference, String className) ''' ...
public class «className» extends AbstractAddFeature { ...
«generate_add(reference)» // generate callback method
«generate_decorateConnection(reference)» }
'''
def generate_add (MetaReference reference) ''' ...
public PictogramElement add(IAddContext context) { ...
// call callback method in generated code
decorateConnection (addConContext, connection); }
'''
// template for callback method
def generate_decorateConnection (MetaReference reference) ''' /**
* Override this method to decorate the created connection */
protected void decorateConnection (IAddConnectionContext context, Connection connection) {}
Code Documentation
Overridden Methods
If a method from a base class is overridden by generated code, the method should declare the \@Override
annotation and the {@inheritDoc} Javadoc annotation. To shorten this, «overrideHeader» generates the necessary code.
Xtend Template:
«overrideHeader»
public void myMethod (...)
Generated code:
/**
* {@inheritDoc} */
@Override
Graphiti
This secting will contain information about • the generated Graphiti code
• additional static platform artifacts • architecture
Project Setup
Things to consider when developing Spray projects.
Naming
Base package / Bundle ID Prefix: org.ecliselabs.spray
Source Management
• Generated sources are not checked in
• The following resources must be added to .gitignore for each plugin • src-gen
• target • bin
• plugin.xml_gen
• Experimental features must be added on a branch until stabilized. This branch may be shared on the remote repository if it is of interest of other developers to contribute or review the changes.
Adding a new Plugin project
Manifest
• Add a plugin.properties # /**
# * <copyright>
# * Copyright (c) 2011 The Spray Project.
# * All rights reserved. This program and the accompanying materials # * are made available under the terms of the Eclipse Public License v1.0 # * which accompanies this distribution, and is available at
# * http://www.eclipse.org/legal/epl-v10.html # *
# * Contributors:
# * Spray Project Team # * </copyright>
# */
# NLS_MESSAGEFORMAT_VAR pluginName = ADD DESCRIPTION providerName = Eclipselabs Spray
• Open MANIFEST.MF, change/add entries (exchange 0.3.0 by current development version from other plugins)
Bundle-Name: %pluginName Bundle-Vendor: %providerName Bundle-Version: 0.3.0.qualifier Bundle-Localization: plugin
build.properties
Include the following files/directories, if they exist • Binary Build
• plugin.properties
• icons/ • images/ • Source Build • pom.xml • launch/ • */*.launch • log4j.properties
Project Specific Settings
To avoid confusion between different platforms and workspaces common settings should be defined as project specific settings. Those settings are checked in and thus shared.
• Resource: Text file encoding: Other / UTF-8 • Resource: New text file delimiter: Other / Unix • Java Code Style -> Formatter:
• Check "Enable project specific settings"
• select Active Profile "`spray_eclipse_formatter`" • Java Editor -> Save Actions
• Check "Enable project specific settings" • Check "Perform the selected actions on save" • Check „Format source code” / "Format all lines" • Check "Organize imports"
Build
• Copy the pom.xml from org.eclipselabs.spray.generator.graphiti. Exchange the plugin’s name in
<artifactId>org.eclipselabs.spray.generator.graphiti</artifactId> by the project’s name. • Open releng/org.eclipselabs.spray.distribution/pom.xml and add the plugin as additional
module
Dependency Management
Imported Packages
Require Bundle is sometimes a too tight dependency binding. It does not allow to exchange the implementing bundle, which sometimes might be desired. The following packages should imported instead of adding its bundles to the Required Bundles:
• org.eclipse.xtext.xbase.lib • org.eclipse.xtext.xtend2.lib • org.apache.log4j • org.apache.commons.logging • com.ibm.icu
Plugin Development
plugin.xmlXtext generates a plugin.xml just once, on second generator execution it creates a plugin.xml_gen. This is because Xtext’s generator cannot merge changes into the plugin.xml and manual modifications might be intended. Differences between both files must be merged manually.
The plugin.xml file is quite large and it can become difficult to decide, which differences are produced by the generator and which are manual changes. In order to identify the intended changes these places must be marked in the code by comments <!-- SPRAY BEGIN -->...<!-- SPRAY END -->. Example:
point="org.eclipse.ui.preferencePages"> <!-- SPRAY BEGIN --><!-- adding category --> <page category="org.eclipselabs.spray.xtext.Spray" class="org.eclipselabs.spray.styles.ui.StyleExecutableExtensionFactory:org.eclipse.xtext.ui.editor.preferences.LanguageRootPreferencePage" id="org.eclipselabs.spray.styles.Style" name="Style Language"> <keywordReference id="org.eclipselabs.spray.styles.ui.keyword_Style"/> </page> <!-- SPRAY END --> h1. Build Management
Spray Maven Build
Spray is using Maven Tycho for building. You need a Maven 3 installation.
To build Spray, change to the /releng/org.eclipselabs.spray.distribution directory and enter
mvn clean install -Pskip.ui.tests
Maven will download all required artifacts and plugins automatically and at the end you should get
[INFO] Reactor Summary: [INFO]
[INFO] Eclipselabs Spray ... SUCCESS [0.620s] [INFO] org.eclipselabs.spray.mm ... SUCCESS [14.537s] [INFO] org.eclipselabs.spray.generator.graphiti ... SUCCESS [2.444s] [INFO] org.eclipselabs.spray.runtime.graphiti ... SUCCESS [0.819s] [INFO] org.eclipselabs.spray.xtext ... SUCCESS [24.335s] [INFO] org.eclipselabs.spray.xtext.ui ... SUCCESS [1.720s] [INFO] org.mod4j.dsl.businessdomain.mm ... SUCCESS [1.143s] [INFO] org.eclipselabs.spray.examples.one ... SUCCESS [1.431s] [INFO] org.eclipselabs.spray.xtext.tests ... SUCCESS [5.161s] [INFO] org.eclipselabs.spray.examples.one.tests ... SUCCESS [48.009s] [INFO] org.eclipselabs.spray.feature ... SUCCESS [0.233s] [INFO] org.eclipselabs.spray.feature.source ... SUCCESS [0.178s] [INFO] org.eclipselabs.spray.feature.sdk ... SUCCESS [0.246s] [INFO] org.eclipselabs.spray.releng.repository ... SUCCESS [19.032s] [INFO] ---[INFO] BUILD SUCCESS
[INFO]
---In directory
releng/org.eclipselabs.spray.repository/target/repository
an Eclipse P2 repository is built, which contains all plugins and sources.
Parent POM
Spray’s Parent POM can be found at releng/org.eclipselabs.spray.parent/pom.xml.
Plugin Management
All Maven Plugins that are used in the build are declared in the <pluginManagement> section. Plugin versions are only managed here, module POMs must not specify any plugin version! If a common configuration can be defined for a plugin, this configuration is already added to the plugin in the plugin management section.
To check for possible updates run frequently
mvn versions:display-plugin-updates
Profiles
Profile skip-ui-tests
The build executes UI tests developed with SWTBot by default. If you want to skip executing these tests, activate the skip-ui-tests profile:
Continuous Integration
Introduction
We use Cloudbees' DEV@Cloud as infrastructure for a Continuous Integration build. The location of the CI system is https://spray.ci.cloudbees.com/
Build Jobs
Spray CI Build
Build URL: https://spray.ci.cloudbees.com/job/spray-ci-build/
Pulls branch master of the Git repository every 15 minutes. Build mail messages are sent to [email protected]. To receive build mails register to the spray-build Google Group.
Spray Experimental Build
Build URL: https://spray.ci.cloudbees.com/job/spray-ci-experimental/
Pulls branch experimental of the Git repository every 15 minutes. Build mail messages are sent to [email protected]. To receive build mails register to the spray-build Google Group.
You can push your repository state any time to the experimental branch if you want to test things on the build that might break it, like when changing the build configuration or introducing new dependencies.
Administration
The Cloudbees builds are administered by Karsten Thoms and Joerg Reichert. To request changes, add an issue and set the Label „Component-Build”.
Release Process
Follow these steps if you want to release a new version of Spray.
A shell script release.sh is prepared to execute all necessary steps. You will need to have checkout out the
spray.distribution git repository as a sibling directory to your main Spray repository; the process copies and uploads artifacts within this repository.
• Pull the latest state from the repository
• Open a command-line and go to the releng/org.eclipselabs.spray.distribution directory • Execute from this directory the scripts/release.sh script. The script takes 2 arguments:
• The first argument is the version to be released • The second argument is the next development version
sh scripts/release.sh 0.3.0 0.4.0
• Upload the file to the Downloads section of the project
Testing
Test Projects
All test projects can be found in the repository in the tests folder.
Test aspects
There are several aspects of Spray that have to be covered by tests:
Testing using the Spray DSL Editor enviroment
Testing the wizard
• Does the Spray wizard offer the expected configuration options?
• Does the Spray wizard produce a project with the expected configuration? • plugin.xml
• MANIFEST.MF • source folders
• created diagram and domain model with the expected file extensions • does the properties file exist and contain the expected values Testing the Spray DSL editor
• Is the imported domain model resolved?
• Do changes in the DSL editor trigger the code generation?
• Are semantic errors marked and probably even quick fixes offered for them (producing the expected change)? • Do the proposals work?
• Are the predefined DSL templates for e.g. containers and connections offered? • Is the color picker shown?
• Is access provided to JVM types in the class path?
• Do cross references inside the DSL work (e.g. referencing an already defined behavior group)? • Does the formatter produce the expected output?
• After save, is the DSL serialized correctly and can it be parsed afterwards? Testing the generated code
• Are all expected artifact generated for a given DSL? • correct package?
• correct name?
• base class as well as gap class generated? • expected file content?
• Does the generated code compile?
• Is a warning dialog shown, when it is tried to edit a generated file?
• Is there a context menu entry for moving a gap class from src-gen to src and does it work? • Is a gap class not regenerated when it has been moved to src?
Testing the generated Graphiti editor
• Can the Graphiti editor runtime be started out of the generated code? • Can a new diagram with the expected diagram be created?
• Does the generated Graphiti editor provide its expected abilities in compliance to what has been defined in the DSL?
• Does the palette contains all expected elements in the expected palette compartments?
• Creating elements: Does the domain model file contain a corresponding object with the expected properties after an element has been created in the diagram editor?
• Adding elements: Is the expected graphical representation with the expected properties created in the diagram model after the element from the palette has been selected and droped in the editor?
• Removing elements: Is the graphical representation removed without removing the domain element? • Updating elements: Are changes to the domain object done outside the diagram editor recognized in the
diagram editor?
• Deleting elements: Are the graphical representation as well as the domain element are deleted from their models?
• Resizing and layouting: Does the diagram elements have the expected coordinates after resizing (e.g. when a rectangle is resized, the contained text is aligned, too)?
• Constraints: Are the constraints applied when creating, adding, moving and resizing diagram elements (e.g. no connection allowed between certain objects)?
• Context buttons and menus: Are the expected context buttons and context menu entries available at the diagram elements?
• Properties View: Is the properties view shown when clicking on diagram elements and does it show the expected fields containing the expected values?
• Is the editor shown as dirty when there are changes in the diagram?
• When saved, are all changes persisted in the diagram file as well as in the domain model file and then the editor is not shown as dirty?
Test approaches
JUnit tests
• should be used when a functionality can be tested without the need to start an Eclipse instance (as they do not need a certain state only reachable by a running Eclipse)
• this of kind test should be preferred • we use JUnit4
• common test logic should be factored out, so that the actual test method states only the test parameters and passes them to a common test executer method, this makes the tests more readable
• following the suggestions of the XUnit Test Pattern book, the name of a test method should contain • test as prefix
• the name of the method to test, e.g. testGetSomething
• optional the condition under that the test is executed, e.g. testGetSomething_WhenNullIsPassed • optional the expected result, e.g. testGetSomething_WhenNullIsPassed__NullExpected
• the instance of the class to to test should be declared as global variable and assigned in a with @Before annotated setup method
• in a Maven pom these test should be execute either with the Maven-Surefire plugin or as alternative with the Tycho-Surefire plug-in, but then without the use of the UI test harness
Mocking
• to improve the isolation of tests but also as a way to avoid the need of a running Eclipse in some cases it is sensible to use a mocking framework to stub the interactions of the class to test
• we propose the use of Mockito in conjunction with Powermock as mocking framework, these libraries are exported by the plug-in org.eclipselabs.spray.testhelper and should be added to the MANIFEST.MF dependencies of the test plug-in if it want to use these mocking frameworks
• with Mockito you can stub interfaces as well as classes and put them in place of the classes the class under test would usually interact with, you can specify which values should be returned when a method of the stubbed class is called with certain parameters
• with Powermock the abilities of Mockito are extended to support mocking e.g. static method calls as well as constructor calls
• as most classes in Spray uses Google Guice injection, it is easy to inject the stubbed classes in place of the real classes, in all other cases sub classing the class to test may be an option
Xtext tests
• tests that exercises Xtext based classes should define the Xtext runner at the class header: @RunWith(XtextRunner.class)
• to replace the default injection configuration define a custom injection provider and also declare it in the header of the test class: @InjectWith(SprayTestsInjectorProvider.class)
• after that you are able inject the classes inside the test class itself as well as use them in the classes to test • to ease the writting of tests for Xtext based code, the xtext-utils are used
• beside checking the parsing and serializing of DSL models you can also test things like scoping and validation • for further insight in how to test Xtext DSLs also checkout Moritz Eysholdt’s presentation covering that topic • Xtext tests can be executed headless, i.e. as JUnit tests
JUnit plug-in tests
• in some cases it inevitable to start up an Eclipse instance to be able to execute tests successfully, as these tests depend on state only available in a running Eclipse (e.g. state is set by the Activator of the plug-in or the test queries the existing perspectives)
• some error states are not reproducable as JUnit test but only in a JUnit plug-in test (e.g. being able to import the domain model)
• some other error states only occur in a Eclipse RCP product having the Spray feature installed (e.g. build.properties does not include all required files)
• in a Maven pom these test should be execute with the Tycho-Surefire plug-in with setting the use of the UI test harness to true
• note that those non headless tests will not work on the Jenkins build server hosted by CloudBees, as this host on Linux machine, so their execution is disable there
UI tests
• as extension of JUnit plug-in tests UI tests executes operations a user would do usually manually, e.g. open menus, clicking on buttons, but also dragging elements in the Graphiti editor, do resize and so on
• we currently use SWT Bot, to define those tests, in contrast to a tool like WindowTester it is not able to record interactions with the UI, so you have to program those interactions by your own
• UI tests tend to be brittle and are hard to read and maintain, so they should be use when the functionality to test cannot be coverage by other test means
• depend on the performance of the Computer where they are executed (reactions too slow or too fast, have to work with sleeps and cope with threading issues)
• behave sometimes different on different platforms (UI elements cannot be found that have been found in a test run under a different operating system)
Measuring test coverage
• there has been added support of measuring code coverage in the Eclipse development environment as well as in the Maven build with using EMMA
• there are class loading issues when the code to instrument by EMMA uses Google Guice injection, so test coverage is currently mostly measure only for the test code that is excecuted but not for the code that is exercised by the test code (that would be actually the more interesting code coverage to measure)
Test data organization
• example domain models are provided in the model folder of the test plug-in
• the test models are named, numbered and organized in different packages to better associate them with the tests that uses them
• for smaller tests it may sufficient to build up the test models with using the generated EMF factory class instead of providing extra test models
• the test models can be used to compare expected and actual output, e.g. when testing the Spray DSL formatter • beside the test DSLs there can be also expected generator artifacts provided to be used in output comparison
Generator template tests
• beside testing the generated code by executing it and check its behavior, it is sometimes more sensible to additional provide tests for generator templates and helper methods, to be able to spot more easily where are failures in the generator logic, as some of those failures only occur under certain circumstances
• so instead of executing the hole generator with a bunch of different models, single templates and helper
methods can be execute with several variations of model parts only applicable to the template or helper method to test and verify their outputs
• you can check fault-tolerant does the templates and extensions act to invalid or incomplete model data
Spray Language Tests
The Spray Language is tested in project org.eclipselabs.spray.xtext.tests. This project makes use of the xtext-utils unittesting framework. These tests use example models that are read and processed and cover the following topics:
• Parsing of the models
• Resolving cross references / scoping • Validation
• Formatter • Serializing
Test models are kept below the /model folder. The examples should be as minimal as possible to demonstrate and test a language feature. This is especially important for debugging, since effort for debugging grows with the size of models.
The main test class is org.eclipselabs.spray.xtext.tests.ModelTests. The initialization part
@RunWith(XtextRunner2.class)
@InjectWith(SprayTestsInjectorProvider.class) public class ModelTests extends XtextTest { @Before
public void before() { super.before(); suppressSerialization(); EPackage.Registry.INSTANCE.put(GenModelPackage.eNS_URI, GenModelPackage.eINSTANCE); EcorePlugin.getEPackageNsURIToGenModelLocationMap().put(BusinessDomainDslPackage.eNS_URI, URI.createURI("classpath:/mod4j/BusinessDomainDsl.genmodel")); } ... }
A test method is simply invoking the testFile() method and passes the file under test. If additional resources are required to resolve cross references (typically at least the Ecore file used as metamodel) these are passed as additional arguments.
@Test
public void test_20_color() {
testFile("testcases/20-color.spray", "mod4j/BusinessDomainDsl.ecore"); }
Runtime Tests
The Spray generator components tests can be found in project
Issue Tracking
Tracking System
We use the Issue Tracking System from Google Code: http://code.google.com/a/eclipselabs.org/p/spray/issues/list
Git Commit Message
The tracking system allows automated linking of commits to the Git Repository with the Issue Tickets when following name conventions. When fixing an issue use a commit message with this format
fixed issue#<issuenumber>: <message>
