CASE STUDY BRIDGE.

CASE STUDY

#1

ABOUT THE CLIENT

TABLE OF CONTENTS

3

#2

ABOUT THE PROJECT

#3

OUR ROLE

#4

RESULT OF OUR COLLABORATION

#5

THE BUSINESS PROBLEM THAT WE SOLVED

#6

CHALLENGES

#7

VISUAL IDENTIFICATION

#8

WHAT HAVE WE LEARNED?

#9

PLANS FOR FUTURE

4

5

6-7

8

9

10

11

12

#10

REFERENCES

13-14

The organisations often closely cooperate; each one processes the data on NGOs to some degree. Their employees, who are involved in the BRIDGE project, know each other well and work together toward a clearer and better understanding of the non-profit sector. To make communicating about themselves easier, the group’s members decided to refer to themselves as Core Team Members (CTMs).

ABOUT THE CLIENT

The main stakeholders in the project are four

organisations from the non-profit sector:

GlobalGiving, GuideStar, Foundation Center,

and TechSoup Global.

The Basic Registry of Identified Global Entities (BRIDGE) is a new collaborative project that aims to revolutionise information sharing, in order to better understand the flows of philanthropic dollars and enhance transparency and effectiveness in the global social sector.

ABOUT THE PROJECT

BRIDGE is a system that assigns

nongovernmental organisations (NGOs),

NGO programs and projects, and other

entities in the social sector with a unique

global identifier, a “numerical fingerprint”

for non-profits.

The whole team worked on it for many months in collaboration with the Clients. Many deduplication functions were not identified at all at the beginning of the project. The approach to deduplication, the role of NoSQL databases, and the characteristics of queries or business rules determining when two organisations could be considered a duplicate were unknown. All of this was later identified, thanks to the commitment of the team and the Clients. Our role was to develop a complete solution (from the environment to the operating software and the documentation describing the system). This solution was integrated in the Amazon AWS cloud, and is able to store and deduplicate millions of organisations from the CTMs’ systems. We had to take care of performance issues, security, data backups or the appropriate redundancy of servers, and clusters of database systems.

We were also responsible for technical decisions and direction of the development. Thanks to that, the team had a strong sense of ownership for the deduplication solution and a sense of responsibility for the state and direction of its development. This really helped to organise work and focus on reaching the goals that had been set.

OUR ROLE

To reach a final solution, we had to solve

a specific problem: deduplication of data

about nonprofit organisations.

But let’s start from the beginning. The first phase of the project lasted about 18 months. Future Processing was responsible for creating a complete system that was able to deduplicate data on the approximately 3.3 million non-profit organisations provided by the CTMs. As part of the project, the vast majority of NGOs (97.4 percent) were assigned a unique identifier: a BRIDGE ID.

We created a highly efficient NGO search engine based on Elastic-Search (ES), which is very easy to scale within our infrastructure. MongoDB was used as the main database for storing everything that happened in the system. ElasticSearch and MongoDB operate the system together and complement each other to provide fast response times and, at the same time, a secure layer that stores the history of entries (that is, BRIDGE entries).

RESULTS OF

OUR COLLABORATION

Because of the exceptional nature of the

project, in which we worked with four Clients,

our role in BRIDGE was, in some respects,

different from the typical one.

In total, we’ve identified approximately

540,000

duplicates, resulting in

2.7 million

BRIDGE IDs in the system. The deduplication

engine has been fine-tuned to perform as

best as possible for NGOs.

We are now in the second phase of the project. Currently, BRIDGE can help CTMs exchange information efficiently and improve communication about NGOs.

Within the company, the project was supported by a technical advisor (TA), whose primary goal was to ensure the high quality of the development process and the solutions that were provided. The TA contributed a lot in terms of the structural quality of the product and frequently inspired us to achieve continuous, iterative improvement of the code.

Since the very beginning of the project, we have followed the Scrum framework as closely as possible. It seemed to be an ideal solution for both, us and our Clients.

Read more about

using Scrum in BRIDGE

The BRIDGE system could be a significant step toward connecting disparate sources of data and providing a richer access to this data to non-profit leaders, so that they can learn about social change at scale. It will also help donors to understand the sector and the organisations working for change.

The CTMs have a huge amount of data and there are many nongovernmental organisations in the world that could use it in countless ways.

PRIMARY

TECHNOLOGIES USED:

Other technologies and tools f fIIS f fElastic Search f fMongoDB f fSQL Server f

fCloud computing (Amazon AWS) f

fJMeter f

fTelerik testing framework

THE BUSINESS PROBLEM

THAT WE SOLVED

CHALLENGES

We were responsible for creating an innovative system that

allows unambiguous identification of charities around the world

by assigning them a BRIDGE ID.

Therefore, considering the impressive vision of the product, we can safely say that BRIDGE challenged us constantly.

At the beginning, there were trivial problems, such as choosing the tools to use to view and analyse the data about the organisations. It was also challenging to communicate with the CTMs about our test data. The real challenge was how to use the technology available to us in order to solve the problem we faced: deduplication of NGOs.

We had to make bold decisions to eliminate risks from the very beginning and to quickly test potential solutions. One example is that during the first sprint we created a fully working product (with very limited functionalities) based on Microsoft SQL. Then, during the first sprint review, together with the CTMs, we concluded that it would be a good idea to test the solution based on NoSQL. We accomplished that and during the second sprint review we demonstrated a system based on MongoDB (a nonrelational database).

During the project, as always, the need to meet with the stakeholders face-to-face was very important to us. The only problem was the location — most stakeholders were located in the United States. This was a challenge from the perspective of communication with the team, which was usually six hours ahead of the stakeholders. Synchronisation with the team often had to be carried out using documents, rather than conversations.

VISUAL

IDENTIFICATION

We initiated the effort to create a product logo from scratch. We

utilized members of our design department and (to a lesser extent)

some people responsible for design on the CTMs’ side, who gave

very valuable feedback.

What was the main idea behind the logo?

We wanted to create something that would be associated with stability. Therefore it was natural to design the logo in a form of a bridge connecting people — which is the name of the project after all. The fact that the people are holding hands is very important because it shows that people involved in the project support one another. The logo is blue because this is a colour associated with trust. Just as the sky is always blue, our bridge will always be a stable construction that you can safely rely upon. We decided to use only one colour, in order to highlight that everyone is of equal importance.

As the visual identification of the product was maturing, the logo helped us determine what colours we wanted to keep and has given the product a much more tangible shape.

EVOLUTION OF THE LOGO

first version

WHAT HAVE WE LEARNED?

Working with such large data sets, where each record is potentially

important and where every decision could be a source of valuable

information about the operation of the system was a new experience.

CTMs, we got a better understanding of their needs. Our Clients’ involvement allowed us to gather feedback, which was analysed by the entire team on a regular basis. Thanks to the fact that we worked closely with the Clients and put a lot of energy into the creation and improvement of our own processes, we suc-cessfully finished the project.

The whole team, in collaboration with the Clients, worked on the final solution for many months. Many deduplication functions were not identified at all at the beginning of the project. The approach to deduplication, the role of NoSQL databases, and the characteristics of queries or business rules determining when the two organisations could be considered a duplicate were unknown. All of this was later identified thanks to the commitment of the team and the Clients.

We had to carry out several experiments exploring alternative ways of approaching the deduplication of NGOs. Throughout this process, being transparent when talking to the Clients was the approach that worked best. We didn’t hide any problems or errors, which helped us to gain each other’s trust. We always relied on the Scrum framework (and really understood it) and we had a minimum structure around us.

While working on the project, we decided to take advantage of external training on modern, rapidly developing technologies at the core of the system. In order to improve quality, reduce risks, and accelerate development, the team participated in ElasticSearch training.

PLANS FOR

THE FUTURE

Ultimately, the CTMs want the BRIDGE

system to make it possible to understand

how the money intended for charity

flows between donors and NGOs around

the world.

BRIDGE will increase the transparency and effectiveness of actions taken in the NGO sector.

If, in the long run, BRIDGE actually helps the CTMs efficiently acquire information about NGOs from each other, it may be possible to extend the range of the system by expanding the CTM group to organisations of a similar nature from other countries — thus, get closer to the vision of a truly global system.

REFERENCES

Read our blog post

WHY DID YOU CHOOSE FUTURE PROCESSING?

Good value and good reputation from previous projects.

WHAT DISTINGUISHED US AGAINST THE COMPETITION?

Reputation from previous projects and a proposal that seemed to best understand the scope of the work.

ARE YOU SATISFIED WITH OUR COOPERATION?

Yes, working with Future Processing has been great.

WHAT DO YOU LIKE MOST IN FUTURE PROCESSING?

Very reliable and responsive and much cheaper than staffing a domestic team.

HOW DO YOU SEE THE FUTURE OF OUR COOPERATION?

We will continue to receive support and maintenance from FP as well as occasional project work. Eventually we will want to do some major version changes.

Chad McEvoy,

definitely work with Future Processing again and would recommend them to others as well.

Shane T. Ward,

GuideStar

A generally well-run project, showing good ownership and some good creativity on the product side. Technically producing good results. Good openness of activities and product progress guiding us where that was needed. Well engaging the CTM teams in various meetings weekly. Liked the demos and making progress concrete. Liked the openness to investigate or do things differently as we went through the months.

Paul van Haver,

TechSoup Global

Future Processing was extremely responsive and professional. Very organized and everyone seemed impressed by the quality of work. We stayed pretty much on track the whole time, and the team brought very good intellectual and technical capacity to the project. It seemed like you and your team genuinely took an interest in the project.

Communications were the only piece that could be improved. Your English was great, but we continually had trouble, especially in the beginning, figuring out the best way to meet online. That could be improved.

It has been a real pleasure working with you, and I hope we get the opportunity to work at that level again.

John Hecklinger,

Global Giving

The team was an example of Agile excellence: focused, committed, and adaptable.

Mirosław Drozdzowski,

Future Processing ul. Bojkowska 37A