Project 4: Camera as a Sensor, Life-Cycle Analysis, and
Employee Training Program
Team 7: Nathaniel Hunt, Chase Burbage, Siddhant Malani, Aubrey Faircloth, and
Kurt Nolte
March 15, 2013
Executive Summary:
The first part of this project required us to design software code for us to use for image processing on our Minesweeper 2 robot. The software needs to be able to detect the color and position of stickers place in the mine wells of the game board. A camera is mounted above the game board so that it can see all the mine wells. A mount was also fabricated to enable us to place filters in front of the camera in order to test the robustness of our software. The software takes a background picture with no stickers and then
subtracts that from a foreground picture with the stickers in place. The resulting picture only shows the elements that were added to the board. Image processing is then used to determine the color and position of the stickers. For the final design, those values would be fed into the puzzle solving program in order to determine where to place the mines.
The other parts of the project are a “cradle-to-grave” life cycle analysis of aluminum and vinyl siding to determine which one should be used to make the travel box for transporting and storing our robot. The decision of which material to use takes into account the economic performance and environmental impact of each one. A team training program was also created to ensure each member of our company will maintain and improve their skills throughout their career.
Engineering Requirements:
Customer Requirements Engineering Requirements Test
Use the provided game board w/out modification
Do not change board color or shape.
Camera sensor still operates with new game board installed.
Use xPC Work station Use Quanser Q4 board and Techron amplifier.
Ensure the camera system works from the xPC Work Station
Cost <$30 for sensing system Use inexpensive light sources. Add receipts to ensure that the cost is less than $30.
Fast Must read color and position
in <6s
Use stopwatch to calculate the time it takes the computer to read the color and position 10 times in under 6s.
Accurate
Reads the correct color and position of the mines 96% of time.
Ensure that 48 out of 50 tests are accurate
Works at a reasonable
distance
The camera must be able to
read the color and position
when the camera is within
24”.
Make sure the camera reads
the correct color and
position at 24”
Must be robust
Reads correct color and
position in multiple lighting
environments and with
multiple stickers
Must work with up to 2
transparency filters. Must
work with up to nine
stickers.
Must report color of stickers
in well
Camera will measure RGB
values of image and
determine color of sticker
Must correctly determine
green and red stickers 10
consecutive tries.
Must report position of
stickers
Camera will measure the
angular location of the
stickers (0, 30, 60 degrees)
Test that the camera reports
the correct location 10
times.
Document Hardware:
Calculation for ppi:
Number of Pixels = 640 pixels Width = 16.5 inches
Document Software:
The image processing algorithm for this system uses a structure containing important information to keep a track of the current state of the game board. The structure is called game_state and it has the following fields:
well_location: A 1x9 matrix containing the angles of each well relative to the reference point i.e. the rectangular well. The angles starting counter clock-wise are [30, 60, 90, 135, 180, -150, -120, -90, -45]
well_color: A 1x9 matrix which will store the colors of each well as an integer value 0, 1 or 2. key: This is a cell array which specifies what each integer in the well_color matrix represents. 0
means an empty well, 1 represents a green sticker and 2 represents a red sticker.
current_img: A 480x640x3 matrix which contains the RGB values for each pixel of the image after the pattern has been set. This can be considered to be the foreground.
diff_img: A 480x640x3 matrix which contains the RGB values for each pixel after the foreground or current_img has been subtracted from the background.
processed_img: A 480x640 matrix which contains the image after converting to binary and removing unwanted objects and noise.
The image processing algorithm consists of various modules which implement different parts of the algorithm. The modules and their functionality are:
get_image: This function returns a 480x640x3 matrix representing an image taken from a HP webcam. It has no input and can be called independently. The function performs a hardware reset to delete any image acquisition objects that exist in memory and unload all adaptors loaded by the toolbox. It then acquires a stream from the webcam. The video stream is previewed for 1 second to calibrate the camera. Then a snapshot is clicked and displayed on screen.
process_img: This function returns a 480x640 matrix which represents a binary image. It requires a subtracted image which is passed to it from the game_state structure. This function then converts the subtracted image into a binary image. Erosion and dilation are performed on the image to remove any noise and amplify wanted objects on the foreground. Erosion and dilation are performed with the help of a strel disk object.
get_colors: This function returns a Mx1 matrix where M is the number of wanted objects in the foreground image. It takes properties of the object regions and the foreground image as an input. It uses the centroid of these regions to make a 3x3 square around the centroid. The values for red, green and blue are averaged inside that square for the foreground image. Then it checks whether the color red is more prominent than the other two. If yes then it classifies the well as having a
red sticker. Otherwise if green is more prominent than the other two, it classifies the well as having a green sticker. These colors are stored as integer values; 1 – green sticker 2 – red sticker.
get_angles: This function returns an Mx1 matrix where M is the number of wanted objects in the foreground. The function takes as an input the properties of wanted objects. It creates a virtual triangle between the centroid of an object, the center of the game board and the reference point i.e. the center of the rectangular well. It uses The Law of Cosines or the SSS formula to calculate the angle between the line from the board center to the reference point and the line from the board’s center to the centroid of an object. The angles are then normalized to 30, 60, 90, 135, 180, -150, -120, -90, -45. Angles are calculated starting counter clockwise from the reference point.
5 .8 8” 5 .0 0” 3.0 0” 3 0 .0 0° 60 .00 ° 90.0 0° 135.00° 1.25” 1 .0 0 ” 0.25”
determine_game_state: This function returns the game_state structure with all the fields filled out. It acquires a foreground image and calls a function to process it. Then the centroids for wanted objects are calculated. According to the centroids, the colors and placement of each sticker is determined. The structure is filled with help from the return values of the get_angles and get_colors. The well location and color are reported.
Does the design meet the Requirements?
In our testing phase, we wanted to ensure that our group met all customer requirements. The first
customer requirement that we tested was to ensure the provided game board we used worked without any modification. We tested this ensuring the camera sensor still operated with a new game board installed. We built a platform to hold the game board which rested under the camera. This was to ensure we could successfully mount the game board without modification. The next requirement was to use the xPC work station. This was tested by making sure the camera system worked from the xPC work station.
Another requirement that was set for this project was to make sure the sensing system cost fewer than 30 dollars. At the end of the project, we were to add all the receipts and make sure we didn’t go over budget. At the end of the project, we spent a total of $21.23 in materials to mount our camera. We also wanted to make sure that our robot was easy to use.
Being fast was another requirement that we had to test for. We wanted to make sure that our robot would be able to read the color and position of the dots in under 6 seconds. Testing this 10 times, we got the speed of our robot to read the position and color to average to 4.65 seconds.
We tested the image processing algorithm for color and location identification along with the robustness of the algorithm. There were three different kinds of test cases:
1. With no filter 2. With one filter 3. With two filter
Test Case
Number of trials
Number of
successes
% Success
No filter
50
50
100%
One filter
50
25
50%
Two filters
50
10
20%
We knew that different lighting situations would affect our camera so we came in the morning, mid-day, and at night to test different lighting situations and made sure our camera could still correctly identify the colors and positions of the mines. However, when the filters were added the performance did not meet requirements. More investigation and experimentation will be needed to enhance the performance with filters in place. Perhaps adding an auxiliary light will improve the effectiveness of the algorithm. Working at a reasonable distance was another requirement we had to make sure we met. We wanted our camera to be able to work properly at a distance of 24 inches or greater away from the game board. When mounting the camera, we distanced the camera 25 inches away from the game board. Another requirement was to ensure our robot was robust.
Life Cycle Analysis:
It is important for engineers to not only design an economically sensible product but also an
environmentally responsible one. For that reason, we are conducting a life cycle analysis on the travel box for shipping and storing our robot. It is a “cradle-to-grave” analysis of aluminum and vinyl siding that takes into account the environmental aspects of using these materials from the time they are mined to being manufactured and delivered. One thousand units will be made so any slight difference between the materials will make a large difference in the total environmental impact.
Figure 1: Environmental Performance
Figure 1 shows the environmental performance of aluminum and vinyl siding. As you can see, aluminum has a slightly larger total environmental impact per unit of 0.0026 versus 0.0018. Aluminum uses more raw materials and has a larger impact while being manufactured. For this reason, it is obvious vinyl siding is a better choice from an environmental aspect.
Figure 2 shows the effect of the materials on contributing to global warming. Aluminum has a much larger impact of 1538gCO2 per unit versus 644gCO2 per unit for vinyl siding. Since we will be using one thousand units, this difference will be greatly amplified and it is obvious that vinyl siding would be the better choice.
Figure 3: Embodied Energy by Fuel Usage
Figure 3 shows the embodied energy by fuel usage which appears to be a negligible difference between the two. Aluminum has a higher score for fuel usage but vinyl has a higher score for feedstock energy.
Figure 4: Economic Performance
Figure 4 shows the economic performance of the two materials. Vinyl has an advantage in total economic performance and when combined with its environmental advantages, it is the clear choice as the
recommended material for the robot’s travel box to be used for shipping and storing. In order to make the best economical and environmental decisions, this life cycle analysis would be used for all aspects of the product design.
Training Program:
In every company it is imprint to support employee self-development and their learning. In order to ensure that each of our employees get the adequate training, our company has decided to use different forms of training. The first type of program we have decided to implement is professional development and leadership development. The professional development programs are designed to provide technical and general knowledge and experience to each employee. The leadership development programs are to ensure leaders continue to develop and “grow” the knowledge and skill necessary to effectively lead the projects and task they are involved in. These programs would be implemented through orientation consisting of formal coursework and on-the-job training assignments throughout the company.
Another approach that our company has chosen to take is to use targeted career training. In this type of program a centralized effort is used to provide effective and consistent training. This type of training combines formal coursework with rotations and is based on defined competencies. This would require an outside source to come into our company to do in-house training for each of our engineers. These types of programs are to help with professional, technical, and leadership training.
Our company approach is to consistently train our employees so that we continue to ensure that skills such as leadership, technical, and professionalism are continually being improved. To keep up with these types of skills, our company has decided that a total of two weeks out of the year would be used to implement these training programs. This would require an outside source to come into our company to do in-house training for each of our engineers. The total cost to do this would be $7,191.72 per engineer based on overhead calculation and their engineer salary. Based on our company having 5 engineers, the total of our training program would be $35,958.60.
We would also have each of our engineers join the IEEE society. With the cost of each membership at approximately 400 dollars that would be a total of 2,000 dollars. Having each of our engineers in the IEEE society would allow our engineers to be able to associate with our engineers across the world and learn new ideas and techniques.
Another aspect that we would incorporate in our training program would be to attend a year conference. The American Control Conference would be required to attend by all engineers so that they can see the state of the art control systems that are applied to robotics. The cost would be approximately 1,500 for travel, 1,000 for food and lodge, 600 for registration, and 2,157.52 for three days of hourly compensation or our engineers. That is a total of 26,287.60 for 5 engineers.
ECE495 - Project 4: Camera as a Sensor
Group Name and Members: ____________________________________________________
Score Pts ABET
Outcomes
5 General Report Format - Professional Looking Document/Preparation (whole document)
a) Fonts, margins (11pt, times new roman, single spaced. 1" margins on all sides). Follows the page limitations below.
b) Spelling and grammar are correct
c) Layout of pictures – all figures need captions and must be referenced in text d) References. Use IEEE reference format.
e) All report components are included in your website
Page 1: Title, Group Name, Group Members, and Date Customer Requirements
a) Description of what the Customer wants
g
5 Page 2: Engineering Requirements
a) Bulleted list of Engineering Requirements
c
20 The following should be a narrative report that describes your design decisions and final design,
e.g., don’t just have a flowchart without text that explains it.
Page 3: Document Hardware (1 page)
Connection diagrams
Equipment
Mounting hardware sketches and photograph. Calculation of resolution - pixels per
square on game board. Is the camera an appropriate sensor?
Page 4-6: Document Software (3 pages)
Flow charts, state diagrams, data structures, etc. that describe how the software is
implemented. Do not include a copy of the source code.
Page 7: Does the design meet the Requirements? (1 page)
Evaluate your system in regard to achieving the Requirements
Include statistics such as “found green sticker 80 times in 100 trials?
c
10 Pages 8-9 Life Cycle Analysis (2 pages)
You are proposing a design that consumes resources. Follow the “Life Cycle Assessment (LCA) Exercise” for the shipping box for your project to examine the life cycle for this one part of your design. Be sure to interpret the results of the computer program. Complete this section of the report by saying that a similar analysis could be done on the entire project to reduce
environmental impact.
h
20 Laboratory demonstration of your prototype (evaluated by instructor and TAs)
Total of 10 stickers will be placed on the board in two rounds.
Each ticker is defined by 2 variables: position and color.
Correct variable = 1 point => 2 per ticker x 10 tickers = 20 pts.
b
30 Laboratory demonstration of your prototype (evaluated by instructor and TAs)
Scenario 1: Five stickers (arbitrary color and location) will be placed on the game board
with no film over camera.
Scenario 2: Same stickers moved to other locations on the game board. One filter sheet
will be inserted in the mount and values of recorded variables (color and position) will be noted. There is no opportunity to acquire a new background image after the
filter is added.
Scenario 3: Repeat with two sheets.
Maximum points = 2 per sticker x 5 stickers x 3 scenarios = 30 pts.
c
10 Page 10 Employee Training Program
