Climate. Release gce13bf6. Mirantis Inc.

(1)

Climate

Release 0.1.0.58.gce13bf6

Mirantis Inc.

(2)

(3)

Overview

1.1 Introduction

Idea of creating Climate originated with two different use cases:

• Compute host reservation (when user with admin privileges can reserve hardware resources that are dedicated to the sole use of a tenant)

• Virtual machine (instance) reservation (when user may ask reservation service to provide him working VM not necessary now, but also in the future)

Now these ideas have been transformed to more general view: with Climate user can request the resources of cloud environment to be provided (“leased”) to his project for specific amount on time, immediately or in future.

Both virtual (Instances, Volumes, Networks) and hardware (full hosts with specific characteristics of RAM, CPU and etc) resources can be allocated via “lease”.

In terms of benefits added, Resource Reservation Service will:

• improve visibility of cloud resources consumption (current and planned for future); • enable cloud resource planning based on current and future demand from end users; • automate the processes of resource allocation and reclaiming;

• provide energy efficiency for physical hosts (both compute and storage ones);

• potentially provide leases as billable items for which customers can be charged a flat fee or a premium price depending on amount of reserved cloud resources and their usage.

1.1.1 Glossary of terms

Reservation is an allocation of certain cloud resource (Nova instance, Cinder volume, compute host, etc.) to particular project. Speaking about virtual reservations, we may have not only simple, solid ones (like already mentioned instances and volumes), but also complex ones - like Heat stacks and Savanna clusters. Reservation is characterized by status, resource type and identifier and lease it belongs to.

Lease is a negotiation agreement between the provider (Climate, using OpenStack resources) and the consumer (user) where the former agrees to make some kind of resources (both virtual and physical) available to latter, based on a set of lease terms presented by the consumer. Here lease may be described as contract between user and reservation service about cloud resources to be provided right now or later. Technically speaking, lease is a group of reservations

(8)

granted to particular project upon request. Lease is characterized by start time, end time, set of individual reservations and associated events.

Event is simply something that may happen to lease. In most simple case event might describe lease start and lease end. Also it might be notification to user (e.g. about soon lease expiration) and some extra actions.

1.1.2 Rationale

Climate is created to:

• manage cloud resources not only right now, but also in the future; • have dedicated recourses on certain amount of time;

• prepare for the peak loads and perform capacity planning; • optimise energy consumption.

1.1.3 Lease types (concepts)

• Immediate reservation. Resources are provisioned immediately (like VM boot or moving host to reserved user aggregate) or not at all. If request can be fulfilled, lease is created and success status is returned. Lease should be marked as active or to_be_started. Otherwise (if request resource cannot be provisioned right now) failure status for this request should be returned.

• Reservation with retries. Mostly looks like previous variant, but in case of failure, user may want to have several (configurable number) retries to process lease creation action. In this case request will be processed till that will be possible to create lease but not more than set in configuration file number of times.

• effort reservation. Also might have place if lease creation request cannot be fulfilled immediately. Best-effort mechanism starts something like scavenger hunt trying to find resources for reservations. For compute hosts reservation that makes much sense, because in case there are instances belonging to other tenant on eligible hosts, and without them there will be possible to reserve these hosts, Climate may start instances migration. This operation can be timely and fairly complex and so different strategies may be applied depending on heuristic factors such as the number, type and state of the instances to be migrated. Also Climate should assert that there are at least enough potential candidates for the migration prior to starting the actual migration. If Climate decides to start migration, it returns success state and marks lease as in_progress, otherwise - failure. If this ‘hunting’ ends successfully before configurable timeout has passed, lease should be marked as active, otherwise its status is set to timedout.

• Delayed resource acquiring or scheduled reservation. In this reservation type lease is created successfully if Climate thinks there will be enough resources to process provisioning later (otherwise this request returns failure status). Lease is marked as inactive till all resources will be actually provisioned. That works pretty nice and predictable speaking about compute hosts reservation (because hosts as resources are got not from common cloud pool, but from admin defined pool). So Climate is possible to predict these physical resources usage and use that information during lease creation. If we speak about virtual reservations, here situation is more complicated, because all resources are got from common cloud resources pool, and Climate cannot guarantee there will be enough resources to provision them. In this failure case lease state will be marked as error with appropriate explanation.

1.2 Climate architecture

Climate design can be described by following diagram:

(9)

climate-client - provides the opportunity to communicate with Climate via REST API (climate-api service).

climate-api - waits for the REST calls from the outside world to redirect them to the manager. climate-api communi-cates with climate-manager via RPC. Runs as a separated process.

climate-manager - implements all logic and operations with leases, reservations and events. Communicates with Climate DB and stores there data structure of connected leases, reservations (both physical and virtual) and events. climate-manager service is responsible for running events created for lease and process all actions that should be done this moment. Manager uses resource-plugins to work with concrete resources (instances, volumes, compute hosts). climate-manager uses Keystone trusts to commit actions on behalf of user who has created lease before.

resource-plugin - responsible for exact actions to do with reserved resources (VMs, volumes, etc.) When working knows only about resource ID and token to use. All resource plugins work in the same process as climate-manager.

1.2.1 Virtual instance reservation

Virtual instance reservation mostly looks like usual instance booting for user - he/she only passes special hints to Nova containing information about future lease - lease start and end dates, its name, etc. Special Nova API extensions parse these parameter and use them to call Climate, passing to it ID of just created instance. If there is a need to reserve all instances in cloud (like in developer labs to automate process of resource reclaiming), default reservation extension might be used. By default it starts lease at the moment of request and gives it one month of lifetime.

During the time lease has not started yet, instance will be shelved.

1.2.2 Compute host reservation

Now process of compute hosts reserving contains two steps:

• admin marks hosts from common pool as possible to be reserved. That is implemented by moving these hosts to special aggregate;

• user asks for reserving of host with specified characteristics like: – the region

– the availability zone

(10)

– the host capabilities extra specs (scoped and non-scoped format is accepted) – the number of CPU cores

– the amount of free RAM – the amount of free disk space – the number of hosts

Technically speaking, resource ID here will be not host ID, because there might be many of them wanted. Resource here will be new aggregate containing reserved hosts. The time lease starts, user may use reserved compute capacity to run his/her instances on it passing special scheduler hint to Nova. When host is reserved, it’s not used for usual instance running, it might be used only when lease starts and only by passing reservation ID to Nova. That is implemented using special Nova Scheduler filter, that passes reservation ID to Climate and checks if user really can use reserved compute capacity.

(11)

CHAPTER

2

User guide

Installation

2.1 Installation using Devstack

This section includes instructions for Climate installation using Devstack. 1. Download Devstack:

git clone https://github.com/openstack-dev/devstack.git 2. Download Climate:

git clone https://github.com/stackforge/climate.git 3. Add climate files to Devstack:

cd climate/contrib/devstack DEVSTACK_DIR=../../../devstack cp lib/climate ${DEVSTACK_DIR}/lib

cp extras.d/70-climate.sh ${DEVSTACK_DIR}/extras.d

4. Configure devstack to run climate by adding climate, climate api and climate manager to the localrc file: cd ${DEVSTACK_DIR}

echo "enable_service climate" >> localrc echo "enable_service climate-a" >> localrc echo "enable_service climate-m" >> localrc

5. Run Devstack: ./stack.sh

2.2 Installation without Devstack

This section includes instructions for Climate installation. Download all Climate related repos:

(12)

git clone https://github.com/stackforge/climate.git git clone https://github.com/stackforge/climate-nova.git

git clone https://github.com/stackforge/python-climateclient.git Install all these projects to your working environment via:

python setup.py install

or

python setup.py develop

Next you need to configure Climate and Nova. Define /etc/climate/climate.conf file using the following example: [DEFAULT]

host=<climate_host>

os_auth_host=<auth_host> os_auth_port=<auth_port>

os_auth_protocol=<http, for example> os_admin_username=<username> os_admin_password=<password> os_admin_tenant_name=<tenant_name> [manager] plugins=basic.vm.plugin,physical.host.plugin [virtual:instance] on_start=on_start on_end=create_image, delete [physical:host] on_start=on_start on_end=on_end climate_username=<username> climate_password=<password> climate_tenant_name=<tenant_name>

Here os_admin_* flags refer to Climate service user. climate_* ones - to admin user created specially to work with physical reservations. If you have no these users, create them via Keystone:

keystone user-create --name=climate --pass=<service_password> --tenant_id=<service_tenant_id> [email protected] keystone user-role-add --tenant-id <service_tenant_id> --user-id <climate_user> --role-id <admin_role>

And the same procedure for special admin user to work with physical reservations.

Next you need to configure Nova. If you want to use virtual reservations, please add the following lines to nova.conf file:

osapi_compute_extension = nova.api.openstack.compute.contrib.standard_extensions

osapi_compute_extension = climatenova.api.extensions.default_reservation.Default_reservation osapi_compute_extension = climatenova.api.extensions.reservation.Reservation

If you want to use physical reservations add these ones:

scheduler_available_filters = nova.scheduler.filters.all_filters

scheduler_available_filters = climatenova.scheduler.filters.climate_filter.ClimateFilter

scheduler_default_filters=RetryFilter,AvailabilityZoneFilter,RamFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ClimateFilter Restart nova-api and nova-scheduler to use new configuration file.

(13)

Climate uses Keystone trusts to commit actions on behalf of user created lease. That’s why we need to create identityv3 service with appropriate endpoints:

keystone service-create --name keystonev3 --type identityv3 --description "Keystone Identity Service v3"

keystone endpoint-create --region <region> --service keystonev3 --publicurl "<auth_protocol>://<auth_host>:5000/v3" --adminurl "<auth_protocol>://<auth_host>:35357/v3" --internalurl "<auth_protocol>://<auth_host>:5000/v3" And, finally, we need to create reservation service in Keystone with its endpoints:

keystone service-create --name climate --type reservation --description “OpenStack reservation service.”

keystone endpoint-create --region <region> --service climate --publicurl "<auth_protocol>://<climate_host>:1234/v1" --adminurl "<auth_protocol>://<climate_host>:1234/v1" To start Climate services use:

climate-api climate-manager

Now you can use python-climateclient to communicate with Climate. APIs

2.3 Climate REST API docs

This page includes documentation for Climate APIs.

2.3.1 Climate REST API v1.0

1 General API information

This section contains base information about the Climate REST API design, including operations with different Cli-mate resource types and examples of possible requests and responses. CliCli-mate supports JSON data serialization format, which means that requests with non empty body have to contain “application/json” Content-Type header or it should be added ”.json” extension to the resource name in the request.

This should look like the following: GET /v1/leases.json HTTP/1.1 or GET /v1/leases HTTP/1.1 Accept: application/json 2 Leases Description

Lease is the main abstraction for the user in the Climate case. Lease means some kind of contract where start time, end time and resources to be reserved are mentioned.

Lease ops

(14)

Verb URI Description

GET /v1/leases Lists all leases registered in Climate. POST /v1/leases Create new lease with passed parameters. GET /v1/leases/{lease_id} Shows information about specified lease.

PUT /v1/leases/{lease_id} Updates specified lease (only name modification and prolonging are possible). DELETE /v1/leases/{lease_id} Deletes specified lease and frees all reserved resources.

2.1 List all leases

GET /v1/leases

• Normal Response Code: 200 (OK) • Returns the list of all leases. • Does not require a request body. Example request GET /v1/leases HTTP/1.1 response HTTP/1.1 200 OK Content-Type: application/json [ { "created_at": "2014-02-26 10:00:00", "end_date": "2345", "events": [ { "created_at": "2014-02-26 10:00:00", "event_type": "start_lease", "id": "event_id_1", "lease_id": "aaaa-bbbb-cccc-dddd", "status": "UNDONE", "time": "1234", "updated_at": null }, { "created_at": "2014-02-26 10:25:52", "event_type": "end_lease", "id": "event_id_2", "lease_id": "aaaa-bbbb-cccc-dddd", "status": "UNDONE", "time": "2345", "updated_at": null } ], "id": "aaaa-bbbb-cccc-dddd", "name": "lease_foo", "reservations": [ { "created_at": "2014-02-26 10:00:00", "id": "reservation_id", "lease_id": "aaaa-bbbb-cccc-dddd", "resource_id": "1234-1234-1234", "resource_type": "virtual:instance",

(15)

Climate, Release 0.1.0.58.gce13bf6 "status": "pending", "updated_at": null } ], "start_date": "1234", "tenant_id": "tenant_id", "trust_id": "trust_id", "updated_at": null, "user_id": "user_id" } ]

2.2 Create new lease

POST /v1/leases

• Normal Response Code: 202 (ACCEPTED) • Returns the information about created lease. • Requires a request body.

Example request POST /v1/leases HTTP/1.1 { "name": "lease_foo", "start_date": "1234", "end_date": "2345", "reservations": [ { "resource_id": "1234-1234-1234", "resource_type": "virtual:instance" } ], "events": [] } response HTTP/1.1 202 ACCEPTED Content-Type: application/json { "created_at": "2014-02-26 10:00:00", "end_date": "2345", "events": [ { "created_at": "2014-02-26 10:00:00", "event_type": "start_lease", "id": "event_id_1", "lease_id": "aaaa-bbbb-cccc-dddd", "status": "UNDONE", "time": "1234", "updated_at": null }, { "created_at": "2014-02-26 10:25:52",

(16)

"event_type": "end_lease", "id": "event_id_2", "lease_id": "aaaa-bbbb-cccc-dddd", "status": "UNDONE", "time": "2345", "updated_at": null } ], "id": "aaaa-bbbb-cccc-dddd", "name": "lease_foo", "reservations": [ { "created_at": "2014-02-26 10:00:00", "id": "reservation_id", "lease_id": "aaaa-bbbb-cccc-dddd", "resource_id": "1234-1234-1234", "resource_type": "virtual:instance", "status": "pending", "updated_at": null } ], "start_date": "1234", "tenant_id": "tenant_id", "trust_id": "trust_id", "updated_at": null, "user_id": "user_id" }

2.3 Show info about lease

GET /v1/leases/{lease_id} • Normal Response Code: 200 (OK)

• Returns the information about specified lease. • Does not require a request body.

Example request GET /v1/leases/aaaa-bbbb-cccc-dddd HTTP/1.1 response HTTP/1.1 200 OK Content-Type: application/json { "created_at": "2014-02-26 10:00:00", "end_date": "2345", "events": [ { "created_at": "2014-02-26 10:00:00", "event_type": "start_lease", "id": "event_id_1", "lease_id": "aaaa-bbbb-cccc-dddd", "status": "UNDONE", "time": "1234", "updated_at": null

(17)

Climate, Release 0.1.0.58.gce13bf6 }, { "created_at": "2014-02-26 10:25:52", "event_type": "end_lease", "id": "event_id_2", "lease_id": "aaaa-bbbb-cccc-dddd", "status": "UNDONE", "time": "2345", "updated_at": null } ], "id": "aaaa-bbbb-cccc-dddd", "name": "lease_foo", "reservations": [ { "created_at": "2014-02-26 10:00:00", "id": "reservation_id", "lease_id": "aaaa-bbbb-cccc-dddd", "resource_id": "1234-1234-1234", "resource_type": "virtual:instance", "status": "pending", "updated_at": null } ], "start_date": "1234", "tenant_id": "tenant_id", "trust_id": "trust_id", "updated_at": null, "user_id": "user_id" }

2.4 Update existing lease

PUT /v1/leases/{lease_id}

• Normal Response Code: 202 ACCEPTED • Returns the updated information about lease. • Requires a request body.

Example request PUT /v1/leases/aaaa-bbbb-cccc-dddd HTTP/1.1 { "name": "new_name", "end_date": "new_date", } response HTTP/1.1 202 ACCEPTED Content-Type: application/json { "created_at": "2014-02-26 10:00:00", "end_date": "new_date", "events": [

(18)

{ "created_at": "2014-02-26 10:00:00", "event_type": "start_lease", "id": "event_id_1", "lease_id": "aaaa-bbbb-cccc-dddd", "status": "UNDONE", "time": "1234", "updated_at": null }, { "created_at": "2014-02-26 10:25:52", "event_type": "end_lease", "id": "event_id_2", "lease_id": "aaaa-bbbb-cccc-dddd", "status": "UNDONE", "time": "2345", "updated_at": null } ], "id": "aaaa-bbbb-cccc-dddd", "name": "new_name", "reservations": [ { "created_at": "2014-02-26 10:00:00", "id": "reservation_id", "lease_id": "aaaa-bbbb-cccc-dddd", "resource_id": "1234-1234-1234", "resource_type": "virtual:instance", "status": "pending", "updated_at": null } ], "start_date": "1234", "tenant_id": "tenant_id", "trust_id": "trust_id", "updated_at": null, "user_id": "user_id" }

2.5 Delete existing lease

DELETE /v1/leases/{lease_id}

• Normal Response Code: 204 NO CONTENT • Does not require a request body.

Example request

DELETE /v1/leases/aaaa-bbbb-cccc-dddd HTTP/1.1 response

HTTP/1.1 204 NO CONTENT

Content-Type: application/json

(19)

3 Hosts Description

Host is the main abstraction for a Nova Compute host. It is necessary to enroll compute hosts in Climate so that the host becomes dedicated to Climate, and won’t accept other VM creation requests but the ones asked subsequently by leases requests for dedicated hosts within Climate. If no extra arguments but the name are passed when creating a host, Climate will take Nova specifications, like VCPUs, RAM or cpu_info. There is a possibility to add what we call arbitrary extra parameters (not provided within the Nova model) like number of GPUs, color of the server or anything that needs to be filtered for a user query.

Hosts ops

Verb URI Description

GET /v1/os-hosts Lists all hosts registered in Climate.

POST /v1/os-hosts Create new host with possibly extra parameters.

GET /v1/os-hosts/{host_id} Shows information about specified host, including extra parameters if existing. PUT /v1/os-hosts/{host_id} Updates specified host (only extra parameters are possible to change).

DELETE /v1/os-hosts/{host_id} Deletes specified host.

3.1 List all hosts

GET /v1/hosts

• Normal Response Code: 200 (OK) • Returns the list of all hosts. • Does not require a request body. Example request GET /v1/os-hosts HTTP/1.1 response HTTP/1.1 200 OK Content-Type: application/json [ {

"cpu_info": "{’some_cpu_info’: ’some_cpu_info’}",

"created_at": "2014-01-01 08:00:00", "hypervisor_hostname": "compute1", "hypervisor_type": "QEMU", "hypervisor_version": 1000000, "id": "1", "local_gb": 8, "memory_mb": 3954, "status": null, "updated_at": null, "vcpus": 2 }, {

"cpu_info": "{’some_cpu_info’: ’some_cpu_info’}",

"created_at": "2014-01-01 09:00:00",

"hypervisor_hostname": "compute2",

"hypervisor_type": "QEMU",

"hypervisor_version": 1000000,

(20)

"id": "2", "local_gb": 8, "memory_mb": 3954, "status": null, "updated_at": null, "vcpus": 2 } ] 3.2 Create host POST /v1/hosts

• Normal Response Code: 202 (ACCEPTED)

• Returns the information about created host, including extra parameters if any. • Requires a request body.

Example request POST /v1/os-hosts HTTP/1.1 { "name": "compute", "values": { "banana": "true" } } response HTTP/1.1 202 ACCEPTED Content-Type: application/json { "banana": "true",

"cpu_info": "{’vendor’: ’Intel’, ’model’: ’pentium’, ’arch’: ’x86_64’, ’features’: [

’lahf_lm’, ’lm’, ’nx’, ’syscall’, ’hypervisor’, ’aes’, ’popcnt’, ’x2apic’, ’sse4.2’, ’cx16’, ’ssse3’, ’pni’, ’ss’, ’sse2’, ’sse’, ’fxsr’, ’clflush’, ’pse36’, ’pat’, ’cmov’, ’mca’, ’pge’, ’mtrr’, ’apic’, ’pae’],

’topology’: {

’cores’: 1, ’threads’: 1, ’sockets’: 2}}",

"created_at": "2014-02-26 08:00:00", "hypervisor_hostname": "compute", "hypervisor_type": "QEMU", "hypervisor_version": 1000000, "id": "1", "local_gb": 8, "memory_mb": 3954, "status": null, "updated_at": null, "vcpus": 2 }

(21)

3.3 Show info about host

GET /v1/hosts/{host_id} • Normal Response Code: 200 (OK)

• Returns the information about specified host, including extra parameters if any. • Does not require a request body.

Example request GET /v1/os-hosts/1 HTTP/1.1 response HTTP/1.1 200 OK Content-Type: application/json { "banana": "true",

’topology’: {

3.4 Update existing host

PUT /v1/hosts/{host_id}

• Normal Response Code: 202 (ACCEPTED) • Returns the updated information about host. • Requires a request body.

Example request

PUT /v1/os-hosts/1 HTTP/1.1 {

"values": {

"banana": "false"

(22)

} } response HTTP/1.1 202 ACCEPTED Content-Type: application/json { "banana": "false",

’topology’: {

3.5 Delete existing host

DELETE /v1/hosts/{host_id}

• Normal Response Code: 204 (NO CONTENT) • Does not require a request body.

Example request DELETE /v1/os-hosts/1 HTTP/1.1 response HTTP/1.1 204 NO CONTENT Content-Type: application/json 4 Plugins Description

Plugins are working with different resources types. Technically speaking they are implemented using stevedore exten-sions. Currently plugins API requests are not implemented, listed below examples are their possible view.

Plugin ops

TBD -https://blueprints.launchpad.net/climate/+spec/create-plugin-api-endpoint

(23)

CHAPTER

3

Development guide

3.1 How to contribute

3.1.1 Getting started

• CreateGithubaccount if you don’t have one

• Make sure you have git options user.name and user.email set in git by typing: git config --list. If not, configure them.

• CreateLaunchpadaccount if you don’t have one • CreateOpenStack profile

• Login toOpenStack Gerritusing your Launchpad ID

– Sign up yourOpenStack Individual Contributor License Agreement

– Check that your email is listed inGerrit identities

• Subscribe to Climate-related projects onOpenStack Gerrit. Go to your settings and in the watched projects add stackforge/climate, stackforge/climate-nova and stackforge/python-climateclient

As all bugs/blueprints are listed inClimate Launchpad, you may keep track on them and choose some to work on.

3.1.2 How to keep in touch with community

• If you’re not subscribed toOpenStack general mailing listor toOpenStack development mailing list, do that. Climate-related emails must be send with [climate] in its subject.

• All questions may be asked on our IRC channel #openstack-climate onfreenode

• We also have weekly meetings on #openstack-meetingfreenode IRC channel

3.1.3 Your first commit to Climate

• Checkout corresponding Climate repository fromGithub

• Take a look on howGerrit review processgoes on in OpenStack (read carefullycommitting changespart)

(24)

• Apply and commit your changes

• Make sure all code checkings and tests have passed. See development guidelines to learn more • Send your patch to the review (you may usegit-reviewutility for that)

• Monitor status of your change onhttps://review.openstack.org/#/

3.2 Development guidelines

3.2.1 Coding Guidelines

PEP8checking should pass for all Climate code. You may check it using the following command:

tox -e pep8

Also you should keep your code clear using more code style checkings viapylint:

tox -e pylint

If you see any pep8/pylint errors in your code, it’s mandatory to fix them before sending your change on review.

3.2.2 Testing Guidelines

Climate repositories have unit tests that are run on all submitted code, and it is recommended for developers to execute them themselves to catch regressions early. Developers are also expected to keep the test suite up-to-date with any submitted code changes.

Unit tests might be ran inTOXenvironment via commands:

tox -e py27 tox -e py26

for Python 2.7 and Python 2.6 accordingly.

3.2.3 Documentation Guidelines

Currently Climate docs are partially written onOpenStack wikipages, and partially using Sphinx / RST located in the main repo in doc directory. In future all of them will be moved to Sphinx / RST (now these docs cannot be published on readthedocs.org, because there is already existing climate project created on it. Now Climate ATCs are voting to choose new name for Climate project and then all docs will be moved to new readthedocs project).

To build Sphinx / RST docs locally run the following command: tox -e docs

After it you can access generated docs in doc/build/ directory, for example, main page - doc/build/html/index.html.

(25)

Climate. Release gce13bf6. Mirantis Inc.

Climate

Release 0.1.0.58.gce13bf6

Mirantis Inc.

Contents

1

Overview

1.1 Introduction

1.1.1 Glossary of terms

1.1.2 Rationale

1.1.3 Lease types (concepts)

1.2 Climate architecture

1.2.1 Virtual instance reservation

1.2.2 Compute host reservation

2

User guide

2.1 Installation using Devstack

2.2 Installation without Devstack

2.3 Climate REST API docs

2.3.1 Climate REST API v1.0

3

Development guide

3.1 How to contribute

3.1.1 Getting started

3.1.2 How to keep in touch with community

3.1.3 Your first commit to Climate

3.2 Development guidelines

3.2.1 Coding Guidelines

3.2.2 Testing Guidelines

3.2.3 Documentation Guidelines