Feature Extraction with persistence disabled

In order to properly corroborate the arguments made above regarding the influence of the feature save node being turned on (i.e. saving image features and other information

to file), we need to repeat the tests with this option disabled and evaluate whether or not they have an actual impact on performance.

The first graph, presented in Figure5.8, shows us, as before, the total execution time for thefeature extractor role pipeline, in seconds. As before, we still see an exponential

growth in the time it takes to process images in thefeature extractor as we increase the

number of images processed. Like before, this exponential increase is due to the fact that

Figure 5.7: Feature matcher average comparisons per second (persistence enabled)

as it becomes busier. This is a process that happens more often when larger amounts of images are being processed and, thus, the feature matching process takes longer.

We do, however, see a big difference between this graph and the graph present in Figure 5.3. The difference is the total execution time between each algorithm for the

same number of images in the case where persistence is enabled and the case where persistence is disabled. In the case of SURF, for 10000 images, the total execution time is almost 3 times less in the case where persistence is disabled versus the case where persistence is enabled. In the case of ORB, the difference is significantly different (around 440 milliseconds), which is quite expected. ORB features are significantly less memory intensive, which means writing them to a file is faster than is the case for SURF or HOG features, and so the feature save node does not introduce that many delays in the pipeline. For HOG, the difference is not so significant either. This is due to the fact that HOG has a very slow feature matching process, when compared to other algorithms. This means that thefeature extractor waits significantly more time for the feature matcher to be ready

to receive more features than in other algorithms and has plenty more CPU time to spare. This spare CPU time can be used for the writing of image features to a file, without too big an impact on performance.

When analysing the times for fewer images, the difference between total times be- tween the case with persistence enabled and persistence disabled is not so significant, because the lower amount of images does not allow the feature save node to introduce

5 . 4 . S I N G L E A L G O R I T H M T E S T S A N D R E S U LT S

Figure 5.8: Total feature extractor time (persistence disabled), in function of the number of images processed.

many delays in the system. This can be further evidenced in Figure5.9, that shows the average time each image spends in the whole feature extractor role pipeline. Between

this Figure and Figure5.4, the only significant time difference is when processing 10000

images, which makes sense, considering the eventually slower feature matching process, which in turn slows down the feature extractor and, if the feature extractor is also busy with saving image features, the whole feature extraction process will be slowed down. But in order to properly evaluate whether or not a slower feature extraction process (e.g.

with persistence enabled) also results in a slower feature matching process, we need to take a look at the feature matching results with persistence disabled and compare them with the feature matching results with persistence enabled. This is what we cover in the next Subsection.

This test gives us a more precise view over the actual time it takes to process an image, without the delay introduced by saving features, but it still does not allow us to compare the performance of the algorithms at the task of extracting features. Remember, this graph shows us the total time for the execution of thefeature extractor role pipeline, which

includes more than extracting features and is also delayed by thefeature matcher role. For

that, we need to analyse the time of the feature extraction node of the pipeline exclusively, as we did in Figure 5.5 (in that graph, we also consider the download and GPU load steps time, as they are required before extracting features). Repeating this step for the

Figure 5.9:Average time spent in the feature extractor (persistence disabled), per image

tests with no persistence enabled, we constructed a graph that shows the average feature extraction time, per image, for the different numbers of images processed and algorithms used. The times shown, in milliseconds, represent the average time it takes for an image URL to be retrieved from the stream, the respective image downloaded, uploaded to the GPU and have its features computed. The resulting graph is shown in Figure5.10. Here, the average feature extraction time is very similar and quite independent of the number of images processed. This is due to the no longer present CPU sharing phenomenon introduced by the saving of image features and information. This means that when processing larger amounts of images, there is no longer a step (feature save node) slowing down the other steps, which can execute normally and without delays. This is the case of feature matching with persistence enabled (Figure5.5), where there is a clear delay when processing 10000 images, when compared to the processing of fewer images. With persistence disabled, however, this is not the case. Hence, we can safely conclude that the delay presented by the saving of image features is in fact significant, and becomes worse with larger amounts of images.

5.4.3.1 Pipeline steps comparison

Table5.4shows us a similar table to what we have seen before. It contains the different times, in milliseconds, for each node of the feature extraction pipeline, and for each algorithm. As we can see, there is no significant time increase in the nodes from 5000

5 . 4 . S I N G L E A L G O R I T H M T E S T S A N D R E S U LT S

Figure 5.10: Average feature extraction time (persistence disabled), per image

to 10000 images, for any algorithm, as there was before. This shows that turning off the feature save and image information save nodes significantly improves performance. Now, CPU sharing is no longer a significant issue and each node can execute its computation in a more regular time, consistent to what we observe when dealing with fewer images.

This table shows the peak performance of our system’s feature extraction role. It means we can extract features at an extreme high rate. Without considering the source node (we also offer the optimization of processing images that are already downloaded, although its not our focus), we can extract around 125 SURF and HOG features per second and around 77 ORB features every second. This is extremely good performance, and follows from our design goal of processing images with high performance. A user can still benefit from not saving image features, if all he needs is the actual graph. If a user writes the graph (merely the image identifiers and the scores between them) along with the file that maps image names to image identifiers, he has all the information he needs to extract image similarity information regarding the images. He can also read from that graph and add more data to it, without having to deal with the features of all the images that were already processed. If, however, a user needs to compare newer images with past images, it may be wiser, in the long term, to save the image features for later comparison. It may be better to take the performance penalty right away, instead of running the whole process again, which will mean, undoubtedly, significantly more overall runtime.

Table 5.4: Feature Extraction pipeline (no persistence) steps average execution time, in milliseconds. Algorithm 500 Images 1000 Images 5000 Images 10000 Images Source Node SURF 314 301 326 347 HOG 328 309 299 330 ORB 309 302 307 312

GPU Load Node

SURF 4 4 4 4

HOG 4 4 4 5

ORB 4 4 4 4

Image Date Load Node

SURF 0.034 0.034 0.038 0.045

HOG 0.036 0.036 0.038 0.048

ORB 0.035 0.034 0.039 0.045

Feature Extraction Node

SURF 3 3 2 4

HOG 0.6 0.5 0.74 3.1

ORB 7 6 6 9

Image Date Comparison Node

SURF 0.008 0.007 0.005 0.005

HOG 0.009 0.009 0.005 0.005

ORB 0.008 0.007 0.005 0.004