6.3 Creating a performance evaluation model for an extended Infinica Process Engine
6.3.1 Performance model for an Infinica Process Engine Process with repository storage 53
For the extended use case, the previously used model has just been extended by the groups required for the repository. The extended model is shown in Figure 6.6. The entities described in subsection 6.2.1 still serve the same purposes as before, the new groups in the software contention model are:
Content Repository The group which processes incoming requests to the repository and performing database queries.
Database (Repository) The group in the database handling incoming queries from the repository.
In the device contention model the group Content Repository connects to the same disk and CPU as the groups of Infinica Process Engine, since they were hosted on the same machine during the test runs. The second introduced group for the database connects to the devices of the database machine, because the already existing database has been used. The average service times for the model again have been obtained from the log files, created during parametrisation runs, as described previously for the basic model.
54 Chapter 6: Definition and Test of an LQM using the Modelling Framework
Figure 6.6: Performance model for a PDF rendering process with repository template storage.
6.3.2 Results for the model with repository storage
The results of the extended architecture were evaluated in the same way as before for the basic one in subsection 6.2.2. Likewise to this, the following paragraphs discuss the results shown in the four charts in Figure 6.7.
Scheduler delay = 0ms
Similar to the basic use case the results with no scheduler delay and a usage of two pro-cess threads appear to match rather well. It is assumed this is the case, because the configuration is the closest to the one to which the model was parametrised with. How-ever, also in this configuration, the model would expect a significantly higher throughput when more process threads are active in parallel. Again this hints that an unidentified bottleneck is in place in the actual application which has not been modelled.
Chapter 6: Definition and Test of an LQM using the Modelling Framework 55
Figure 6.7: Result comparison for repository template storage with scheduler delays of 0ms (top left), 25ms (top right), 50ms (bottom left), and 100ms (bottom right).
Scheduler delay = 25ms
The measured values in this configuration are still very close to the one without any scheduler delays. This is similar to the modelled results for2 threads, while the calcu-lated throughput values for32threads remarkably decrease and fall below the measured values. This was not the case for the basic model before. These results also point out that there is a more restricting bottleneck in the individual process threads of the actual system.
Scheduler delay = 50ms
The measured values are still very similar to the ones for the previous scheduler delay settings, while the calculated throughputs continued to fall. This means that, although the scheduler takes breaks of 50msbefore running each process, another component is still taking more time in order to process the incoming requests. Similarly to the basic model discussed before, this bottleneck must block each thread individually. Otherwise the overall throughput would be anticipated to go down as the slow response time would sum up to the one of the delayed scheduler, which is not the case in the presented
56 Chapter 6: Definition and Test of an LQM using the Modelling Framework
graphs.
Scheduler delay = 100ms
With a delay of100msthe measured throughput values finally decreased. However, the model would have expected a far higher decrease of the throughput.
In overall comparison to the basic model, it is evident that one or more bottlenecks are missing in the created model, whereas the missing bottlenecks for the extended model are more significant as the ones in the basic model.
Chapter 7: Conclusion 57
7 Conclusion
The objective this work was to find a way to predict the performance of a microservice based application. After a research phase, it has been decided to use a layered queuing network approach as it has already been applied successfully to similar problems before.
Since no existing modelling framework was found to fit the necessary requirements, the first step was to implement an own framework, based on the Method of Layers [26].
Therefore a very basic implementation of the framework was chosen, only covering the features which were considered necessary to evaluate the overall approach on the actual application. For this evaluation two use cases were examined, a very basic one and a slightly extended one which actually consisted of two combined microservices.
In both use cases different parameters were changed and the results were compared.
Overall the results generated by the model deemed plausible. The model responded as expected to an artificially introduced bottleneck to the system, as the response times decreased. However, in comparison with the results measured on the actual application setup, the results deviated significantly. At this point it cannot be completely ruled out that the self implemented modelling framework is at least partially responsible for this, as some implementation details, especially for multiserver devices were not entirely clear.
However, the examined results indicate that the created models did miss at least one entity for the basic model and probably multiple entities for the extended architecture, which slowed the response times down in the studied system. More investigations will be needed in order to identify them. Network bandwidth has been already eliminated, as a network monitoring of the test runs did not imply shortages in this regard.
Generally deviations between the measured and the actual results were expected, not only because the model was very simplified and it is known that certain aspects are not covered by it, such as the Java garbage collector, or network latencies, but also because the accuracy of layered queuing networks in other appliances was between 10%and20%[23]. However, for the discussed use cases, the differences were higher in most cases, presumably due to missing software entities in the model, as discussed above. Still the results show that the applied method itself is a good tool to identify and analyse performance issues, in a way that it is possible to first build a simple model and then slowly extend it when the results do not fit with the measurements on the actual application. For instance, in the first try outs of the modelling framework, the process scheduler was not included in the model at all, it was only added after the measured results were always considerably lower than the calculated ones. After some research it turned out that the scheduler, only running in a single thread, caused this delay. Such knowledge can be applied when working on performance improvements on an already existing system.
58 Chapter 7: Conclusion
As for the microservices in general, the introduced method seems to be applicable as layered queuing networks support all of the major features required for modelling them, especially when adding additional functionalities such as two phase servers that allow to model a service centre on which requests are executed in synchronously while the caller is still waiting for them, and after the calls return they finish the request themselves. As already discussed above for the given use case analysis, the effort for creating reliable models may take considerable efforts though. Good knowledge about the individual mi-croservices is required. Nevertheless interesting approaches have been researched by others, such as for automatically creating layered queuing network models from Unified Modeling Language (UML) models [35]. This could help reducing performance mod-elling efforts, especially when well described UML models already exist.
Regarding the modelling efforts for microservices, it should also be considered, that the models for them can be re-used for the most part, when a microservice is moved to another architecture. However, naturally the service time at the devices will always needed to be adjusted depending on the hardware configuration in place.
While the given approach seems to be able to satisfy the requirements for performance modelling microservices, another possible approach, which would be interesting to be compared with are simulation models as summarised in section 3.3. Although the ef-forts have been considered too big for the given time frame, as the different application properties and behaviour have to be described in detail, it is expected to lead to ade-quate results as well and may be more flexible, especially for modelling components for which no simple representation in layered queuing networks can be found.
Known limitations
At the current implementation state of the modelling framework, multi customer support is known to be very restricted. For example it is not possible to assign different numbers of available threads to a multiserver instance per customer class.
As for now, only basic scheduler disciplines, such as first-come-first-served have been implemented. Additional disciplines need to be added if they are required in the future.
Limitations such as on network bandwidths or simultaneously allowed requests to a server cannot be modelled at this point.
Outlook and possible extensions
Besides the implementation of additional layered queuing network features, such as multiple phases, a major and useful step would be to include the modelling framework
Chapter 7: Conclusion 59
into Infinica Process Designer, an application for designing process flows. This way users could evaluate the performance of the processes they are working on during the design time of the process workflow.
To be able to provide a simplified user interface, the performance model should be deviated from the given process workflow for this. For the most part this is considered possible, as the major number of process elements can be linked to a device, e.g.
the urlReader activity typically only utilises a drive or network location, when reading resources, while other activities, such as for rendering documents mostly utilise only the local CPU (in a single thread), as summarised in Table 4.1.
In order to gain the individual execution times of the process elements, the process would need to be executed once on a hardware similar to the one in production. Then the service times at the devices can be automatically estimated from the execution time of the elements according to the process trace log, which can already be streamed automatically to Infinica Process Designer.
With this information it should be possible to automatically provide an already parametrised base model. Individual user input will likely be needed to provide fur-ther information about the hardware structure that will be used, e.g. when the urlReader activity reads from the local file system it usually does not know if it reads from the same physical device as from where the process may have been loaded before, or if it is a separated (network) device. Also it might read from a http service hosted on an arbitrary system, for which a different performance model may be required. Additionally, to run What-If analysis, the users should be able to change the model parameters, e.g.
by adding or removing CPU cores or changing other parameters.
60
Appendix A: Apache JMeter test execution script 61
Appendix A: Apache JMeter test execution script
Listing A.1: Shell script for executing the parametrised Apache JMeter tests.
# !/ bin / sh
62 Appendix A: Apache JMeter test execution script
Appendix A: Bibliography 63
Bibliography
[1] Hoda Amer. Automatic Transformation of UML software specification into LQN per-formance Models using Graph Grammar Techniques. PhD thesis, Carleton Univer-sity, 2001.
[2] Simonetta Balsamo, Antinisca Di Marco, Paola Inverardi, and Marta Simeoni.
Model-based performance prediction in software development: A survey. IEEE Transactions on Software Engineering, 30(5):295–310, 2004.
[3] Simonetta Balsamo and Andrea Marin. Queueing networks. In International School on Formal Methods for the Design of Computer, Communication and Software Sys-tems, pages 34–82. Springer, 2007.
[4] Forest Baskett, K Mani Chandy, Richard R Muntz, and Fernando G Palacios. Open, closed, and mixed networks of queues with different classes of customers. Journal of the ACM (JACM), 22(2):248–260, 1975.
[5] Raymond M Bryant, Anthony E Krzesinski, M Seetha Lakshmi, and K Mani Chandy.
The mva priority approximation. ACM Transactions on Computer Systems (TOCS), 2(4):335–359, 1984.
[6] A Buchmann, Christofer Dutz, and Samuel Kounev. Qpme-queueing petri net mod-eling environment. In Third International Conference on the Quantitative Evaluation of Systems-(QEST’06), pages 115–116. IEEE, 2006.
[7] Allan Clark, Stephen Gilmore, Jane Hillston, and Mirco Tribastone. Stochastic process algebras. volume 55, pages 132–179, 05 2007.
[8] B Terry Compton and Carol Withrow. Prediction and control of ada software de-fects. Journal of Systems and Software, 12(3):199–207, 1990.
[9] Shahir Daya, Nguyen Van Duy, Kameswara Eati, Carlos M Ferreira, Dejan Glozic, Vasfi Gucer, Manav Gupta, Sunil Joshi, Valerie Lampkin, Marcelo Martins, et al.
Microservices from Theory to Practice: Creating Applications in IBM Bluemix Using the Microservices Approach. IBM Redbooks, 2016.
[10] Nicola Dragoni, Saverio Giallorenzo, Alberto Lluch Lafuente, Manuel Mazzara, Fabrizio Montesi, Ruslan Mustafin, and Larisa Safina. Microservices: yesterday, today, and tomorrow. In Present and ulterior software engineering, pages 195–216.
Springer, 2017.
64 Appendix A: Bibliography
[11] Reiner Dumke, Claus Rautenstrauch, Andreas Schmietendorf, and Andre Scholz.
Performance engineering: state of the art and current trends, volume 2047.
Springer, 2003.
[12] Khaled El Emam, Saïda Benlarbi, Nishith Goel, Walcelio Melo, Hakim Lounis, and Shesh N Rai. The optimal class size for object-oriented software. IEEE Transac-tions on software engineering, 28(5):494–509, 2002.
[13] Greg Franks and Murray Woodside. Multiclass multiservers with deferred opera-tions in layered queueing networks, with software system applicaopera-tions. In The IEEE Computer Society’s 12th Annual International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunications Systems, 2004.(MASCOTS 2004). Proceedings., pages 239–248. IEEE, 2004.
[14] Samuel H Fuller and Lynette I Millett. The Future of Computing Performance:
Game Over or Next Level? National Academy Press, 2011.
[15] David C Galant. Queuing theory models for computer networks. 1989.
[16] Les Hatton. Reexamining the fault density component size connection. IEEE soft-ware, 14(2):89–97, 1997.
[17] Maysam Hedayati, Seyed Hossein Kamali, and Abdol Said Izadi. The monitoring of the network traffic based on queuing theory and simulation in heterogeneous network environment. In Proceedings of the 2009 International Conference on Information and Multimedia Technology, pages 396–402, 2009.
[18] Holger Hermanns, Ulrich Herzog, and Joost-Pieter Katoen. Process algebra for performance evaluation. Theoretical computer science, 274(1-2):43–87, 2002.
[19] Edward D Lazowska, John Zahorjan, G Scott Graham, and Kenneth C Sevcik.
Quantitative system performance: computer system analysis using queueing net-work models. Prentice-Hall, Inc., 1984.
[20] M Ajmone Marsan. Stochastic petri nets: an elementary introduction. In European workshop on applications and theory in Petri nets, pages 1–29. Springer, 1988.
[21] Manuel Mayer. Performance-Modellierung und Simulation eines SAP-Netweaver-Portal-Systems. Dissertation, Technische Universität München, München, 2013.
[22] J Meier, Carlos Farre, Prashant Bansode, Scott Barber, and Dennis Rea. Per-formance testing guidance for web applications: patterns & practices. Microsoft press, 2007.
Appendix A: Bibliography 65
[23] Dorina Petriu, Hoda Amer, Shikharesh Majumdar, and Istabrak Abdull-Fatah. Using analytic models predicting middleware performance. In Proceedings of the 2nd international workshop on Software and performance, pages 189–194, 2000.
[24] Martin Reiser. A queueing network analysis of computer communication networks with window flow control. IEEE Transactions on Communications, 27(8):1199–
1209, 1979.
[25] Stewart Robinson. Discrete-event simulation: From the pioneers to the present, what next? Journal of The Operational Research Society - J OPER RES SOC, 56:619–629, 06 2005.
[26] Jerome A. Rolia and Kenneth C. Sevcik. The method of layers. IEEE transactions on software engineering, 21(8):689–700, 1995.
[27] Jyothi Salibindla. Microservices api security. International Journal of Engineering Research & Technology, 7(1):277–281, 2018.
[28] Connie U Smith. Origins of software performance engineering: Highlights and out-standing problems. In Performance Engineering, pages 96–118. Springer, 2000.
[29] Andrew S Tanenbaum and Maarten Van Steen. Distributed systems: principles and paradigms. Prentice-Hall, 2007.
[30] Nidhi Tiwari and Prabhakar Mynampati. Experiences of using lqn and qpn tools for performance modeling of a j2ee application. In Int. CMG Conference, pages 537–548. Citeseer, 2006.
[31] Oliver Ullrich and Daniel Lückerath. An introduction to discrete-event modeling and simulation. Simulation Notes Europe, 27(1):9–16, 2017.
[32] Maarten van Steen and Andrew S Tanenbaum. A brief introduction to distributed systems. Computing, 98(10):967–1009, 2016.
[33] Murray Woodside and Greg Franks. Tutorial introduction to layered modeling of software performance, 2002.
[34] Murray Woodside, Dorin Petriu, and Khalid Siddiqui. Performance-related comple-tions for software specificacomple-tions. In Proceedings of the 24th International Confer-ence on Software Engineering, pages 22–32, 2002.
[35] Mana Hassanzadeh Zargari. Automatic Derivation of LQN Performance Models
66 Appendix A: Bibliography
from UML software models using Epsilon. PhD thesis, Master Thesis, Electrical and Computer Engineering Carleton University, 2016.
Glossary 67
Glossary
Apache Maven A tool for managing and building primarily Java-based projects..
API An Application Programming Interface is a specified interface to a software com-ponent which allows other comcom-ponents to access and use its services..
BPMN 2.0 Business Process Model and Notation 2.0.
CPU Central Processing Unit.
DOCX Microsoft Word Document.
HTML Hypertext Markup Language.
Java An object-oriented programming language and software platform.
JSON JavaScript Object Notation.
LDAP Lightweight Directory Access Protocol.
LQM Layered Queuing Model.
LQN Layered Queuing Network.
MVA Mean Value Analysis.
PCL Printer Command Language.
PDF Portable Document Format.
REST Representational State Transfer.
SOA Service-Oriented Architecture.
SOAP Simple Object Access Protocol.
UML Unified Modeling Language.
WebDAV Web Distributed Authoring and Versioning.
XML Extensible Markup Language.
XPath XML Path Language.
XSL Extensible Stylesheet Language.
XSL-FO XSL Formatting Objects.
68
Erklärung 69
Erklärung
Hiermit erkläre ich, dass ich meine Arbeit selbstständig verfasst, keine anderen als die angegebenen Quellen und Hilfsmittel benutzt und die Arbeit noch nicht anderweitig für Prüfungszwecke vorgelegt habe.
Stellen, die wörtlich oder sinngemäß aus Quellen entnommen wurden, sind als solche kenntlich gemacht.
Wien, 5. Juli 2020
HSMW-Thesis