An increasing number of websites, including internet banking, e-commerce, social network- ing are using the SSL protocols to secure their communication over the internet. The SSL protocols enable the authentication along with the protection of the application level data. The SSL protocol starts with its sub protocol called the handshake protocol. The handshake protocol is responsible for negotiating the cipher suites and compression method, the authen- tication and the key exchange. These processes involve cryptographic operations and are usually expensive in terms of CPU usage and memory requirements. This thesis analyzed the various attack vectors those can be exploited to degrade the server’s performance and ultimately produce the Denial of Service attack.
This thesis investigated the server based Denial of Service attacks on the SSL protocols. The existing SSL protocols [31, 30, 32, 49] are unbalanced in terms of the computational requirements between the server and the client. There are no existing explicit literatures that
10.6 Conclusion 111
cover all the aspects of the SSL protocols those models the DoS threat for the SSL protocols. The existing literature [6, 60, 63, 61] focuses on the reducing the load on the SSL server either by shifting some of the cryptographic computation to the client-side or requesting the client to work sufficiently more before the server performs any expensive operations. The literature [37] suggests that the extra load on the web servers due to the SSL-related operations can be reduced significantly by employing the faster and/or more CPUs.
This thesis focused on following aspects:
• Study the DoS attack strategies that are possible to successfully create DoS attacks on the SSL web servers.
• Study the various aspects of the SSL protocol and investigate the possibilities to ex- ploit any of these to execute the DoS attack. Measure the impact of the identified possibilities.
• Provide the effectiveness of some of the existing DoS strategies on the SSL protocol.
The DoS attacks on the SSL protocol cannot be completely avoided. The SSL protocol is an application layer protocol and runs transparently above the transport layer protocols such as TCP. The packet based mitigation techniques cannot sufficiently mitigate the DoS attacks which exploit the problems in the SSL protocol since such mitigation techniques act according to the signatures (definitions) of the attack those are provided to them. Generally, the attack traffic targeted at the SSL protocol appear to be the normal traffic at the transport level.
The main problem in the SSL protocols is the heavy cryptographic operations. These operations need to be performed at the starting of the SSL protocol in the phase called SSL handshake protocol. To make the situation worse, to perform the same SSL handshake, the client requires much lesser time and computational resources than the server. In addition, the client can request the server to perform many handshake requests. Therefore, this imbalance in the workload is the main cause of the DoS attacks. This kind of DoS attack depletes the server’s computational resources ultimately leading to the DoS condition.
The costs of operations other than the public key cryptography are relatively balanced in the SSL protocol. That is, operations such as symmetric key operations (symmetric key encryption-decryption), MAC calculations, random number generation and data handling (fragmentation, data copy) costs the same at the client as well as at the server side. This has been practically proven at [37].
The SSL protocol allows either communicating party to authenticate each other. In gen- eral, the server is the one who authenticates itself to the client, mostly by providing the certificates. The client can also authenticate itself to the server. The client can only au- thenticate itself to the server if the server requests the client to do so. The process of client authentication can add an extra work load on the server, degrading the server’s performance. The certificate verification consists of the process of certificate signature verification. The RSA signature verification need not be an inexpensive operation. The computational require- ments for the RSA certificate verification depend upon the size of the public exponent of the RSA key that is used for signing the certificate under consideration. Therefore, care should be taken that the certificates are signed with the RSA public key that does not have a higher public exponent. The extra work load on the server due to client certificate verification can lead to DoS attack. However, the likelihood of such attack is rare. Using certificates to attack the server cannot be safe for the attacker. Since, the certificate contains the identity of the user. However, maliciously created or the stolen certificate can allow the attacker to use the client authentication as the DoS attack vector. In addition, the legitimate users can degrade the server’s performance if the client’s certificates require higher time for verification due to
reasons explained so far in the thesis. Therefore, attention should be paid while accepting the certificate authorities (the public exponent of the key belonging to CAs should not be higher). In addition, the allowed certificate chain should be kept as short as possible.
The compression boundaries are systematically checked and followed in the OpenSSL implementation of the SSL protocol. Based on this implementation, there are no signs of a possibility of creation any buffer overflow attacks on the SSL server side.
The SSL renegotiation based DoS attack outperforms among the other two variants of the computationally intensive brute force DoS attacks. The other two variants follow the basic idea of simply opening the SSL connection with the SSL server. One DoS tool does not use the cryptography and does not send the valid client key exchange message to the server. Therefore, the server terminates the SSL connection immediately once decryption of the client key exchange message fails. Another DoS tool makes use of cryptography and is implemented using the standard OpenSSL library. With respect to the impact on the victim’s machine, this tool performs better than the tool that does not use cryptography. This is because, when the SSL handshake terminates just after decrypting the key exchange
message, the server does not have to wait for the cipher spec protocolmessage and the
finishedmessage from the client. In addition, it does not have to send these two messages from its side to the client. This saves the (waiting) time for the server and most importantly, the server saves few cryptographic operations. With these savings, server can serve more legitimate users.
Therefore, the brute force attack, where the attacker successfully completes the SSL handshake with the server is more effective than the brute force attack that uses canned messages and/or does not performance valid cryptographic operations. The most effective DoS attack on the SSL server remains to be the one that exploits the renegotiation feature in the SSL protocols. Therefore, the SSL renegotiation should be carefully configured on the server or should be disabled if not required by the business model.
Finally, this thesis reviewed some of the proposals those aim at improving the SSL server performance either by shifting the load to the client-side and reducing the load on the server side or requesting the client to work more before server starts performing expensive opera- tions. These optimizations have number of limitations and possibility of the DoS attack is not completely removed. In addition, some of these techniques introduce a new possibilities of DoS attacks. These possibilities are discussed in this thesis in respective sections.
Therefore, the SSL server should be carefully configured. The impact of the DoS attacks can be alleviated by employing sufficient bandwidth, processing power and correct usage of the session cache. When under attack, the network traffic should be actively monitored and attack sources should be identified according to the some of the techniques discussed in this thesis.
As a final remark of this research and experiments, there is a need of new protocol or change in the protocol where the server can change its behavior at run-time; for instance to request the clients to work more whenever needed.
CHAPTER
11
Acknowledgments
T
his thesis is written as part of the master project that I have performed at the Depart-ment of Mathematics and Computer Science at Eindhoven University of Technology, the Netherlands, in cooperation with Rabobank, Utrecht, the Netherlands. Upon successfully fin- ishing the Master project comes the conclusion of my study master of Science, specialized in the track Information Security Technology at Eindhoven University of Technology.I have enjoyed working on this project, mainly because of the freedom I was given to explore into the topic of DoS attacks and SSL protocols. I learnt a lot of things not only about the core topic of this thesis but also about a number of applications and technologies associated with it. I express my gratitude to my supervisors. I would like to thank Eric Verheul for initiating this project and giving numerous amounts of new insights and direc- tion along the route. I would like to thank Berry Schoenmakers for supervising my work from Eindhoven University of Technology, for providing great advice and giving magnificent support. I especially want to thank both Eric and Berry for teaching me how to perform a research and various aspects of writing a technical paper.
I would like to thank Henny van de Pavert and Remco Ruiter for providing me all the necessary information, platform and an opportunity to discuss the ideas and giving me all the freedom to perform this research. I would also like to thank Benne de Weger from TU/e and Yorick Koster, David Vaartjes and Pieter Anbeek from Rabobank for their helpful discussions and suggestions. Furthermore, I thank all the folks at Rabobank for the pleasant working and social environment. I would like to specially thank Hans Bielok from Rabobank for motivating me throughout my research.
Finally, a big thanks goes to my friends and fellow colleagues Roberto Lie and Alok Lele for advising me on various aspects of my studies.
Last, but not the least, I would like to thank my parents for their great support and encouragement throughout my study years.
References
[1] “thc-ssl-dos.” http://www.thc.org/thc-ssl-dos/, October 2011.
[2] “Prolexic Attack Report Q1 2012,” 2012.
[3] “Worldwide Infrastructure Security Report, ARBOR Networks,” 2011.
[4] R. Oppliger, SSL and TLS: Theory and Practice. Artech House Information Security and Privacy Series, Artech House, 2009.
[5] “SSL/TLS strong encryption: An introduction.” http://httpd.apache.org/docs/2. 2/ssl/ssl_intro.html.
[6] K. Bicacki, B. Crispo, and A. S. Tanenbaum, “Reverse ssl: Improved server performance and dos resistance for ssl handshakes.” Cryptology ePrint Archive, Report 2006/212, 2006. http://eprint.iacr.org/.
[7] V. Bernat, “SSL computational DoS mitigation.” http://vincent.bernat.im/en/ blog/2011-ssl-dos-mitigation.html, 2012.
[8] “OpenSSL, cryptography and SSL/TLS toolkit.” http://www.OpenSSL.org/, January 2012.
[9] J. Karia, “How DDoS attacks became the frontline tool of cyber-war.” http://thenextweb.com/media/2010/12/19/ how-ddos-attacks-became-the-frontline-tool-of-cyber-war/, December 2010.
[10] J. Emspak, “Anonymous launches DDoS attack on sony.” http://www.ibtimes.com/ articles/131421/20110406/anonymous-launches-ddos-attack-on-sony.htm, April 2011.
[11] M. Brian, “Anonymous’ operation : Payback campaign defends wikileaks, downs mastercard website.” http://thenextweb.com/media/2010/12/08/
anonymous-operationpayback-campaign-defends-wikileaks-downs-mastercard-website/, December 2010.
[12] A. Watters, “DDoS attacks take down mastercard and visa websites, "payback" for their stance on wikileaks.” http://www.readwriteweb.com/archives/ddos_attacks_take_ down_mastercard_and_visa_website.php, December 2010.
[13] G. Sandoval, “FBI probes 4chan’s ’anonymous’ DDoS attacks.”http://news.cnet.com/ 8301-31001_3-20022264-261.html, November 2010.
[14] C. B. Myers, “16-year-old boy arrested in the netherlands over pro-wikileaks attacks.” http://thenextweb.com/eu/2010/12/09/
16-year-old-boy-arrested-in-the-netherlands-over-mastercard-and-visa-website-attacks/, December 2010.
[15] “Online russian blackmail gang jailed for extorting $4m from gambling web- sites.” http://www.sophos.com/en-us/press-office/press-releases/2006/10/ extort-ddos-blackmail.aspx, October 2006.
[16] M. Shiels, “Web slows after jackson’s death.”http://news.bbc.co.uk/2/hi/8120324. stm, June 2009.
[17] “Openssl usage statistics.”http://trends.builtwith.com/Server/OpenSSL.
[18] M. Trojnara, “sslsqueeze.” http://pastebin.com/AgLQzL6L, November 2011.
[19] Y. Zhang, “Low-rate TCP-targeted dos attack disrupts internet routing,” in In Proc.
14th Annual Network and Distributed System Security Symposium, 2007.
[20] J. Leyden, “Phlashing attack thrashes embedded systems,” 2008.
[21] R. Abramov and A. Herzberg, “TCP ack storm DoS attacks,” in Future Challenges in Security and Privacy for Academia and Industry (J. Camenisch, S. Fischer-Hübner, Y. Murayama, A. Portmann, and C. Rieder, eds.), IFIP Advances in Information and Communication Technology, Springer Boston, 2011.
[22] J. Postel, “Transmission Control Protocol.” RFC 793 (Standard), Sept. 1981. Updated by RFCs 1122, 3168, 6093, 6528.
[23] W. Eddy, “TCP SYN Flooding Attacks and Common Mitigations.” RFC 4987 (Informa- tional), Aug. 2007.
[24] M. A. Jean-loup Gailly, “CERT advisory CA-1998−01 Smurf IP denial-of-service at- tacks.” http://www.cert.org/advisories/CA-1998-01.html, March 2000.
[25] M. A. Jean-loup Gailly, “Slowloris http dos.”http://ha.ckers.org/slowloris/, June 2011.
[26] “Checkmate with denial of service.” https://media.blackhat.com/bh-dc-11/ Brennan/BlackHat_DC_2011_Brennan_Denial_Service-Slides.pdf, June 2011.
[27] S. Shekyan, “Are you ready for slow reading?.” https://community.qualys.com/ blogs/securitylabs/2012/01/05/slow-read, January 2012.
[28] THN, “Another-DDoS-tool-from-anonymous-HOIC.” http://thehackernews.com/ 2012/03/another-ddos-tool-from-anonymous-hoic.html, March 2012.
[29] E. Rescorla,SSL and TLS: Designing and Building Secure Systems. Boston, MA, USA: Addison-Wesley Longman Publishing Co., Inc., 2001.
[30] T. Dierks and C. Allen, “RFC 2246 - The TLS protocol version 1.0.” http://tools. ietf.org/html/rfc2246, 1999.
REFERENCES 117
[31] P. C. K. Alan O. Freier, Philip Karlton, “RFC 6101 - the SSL protocol version 3.0.”
http://tools.ietf.org/html/rfc6101, 1996.
[32] T. Dierks and E. Rescorla, “RFC 4346 - the transport layer security (TLS) protocol version 1.1,” 2006.
[33] K. E. Hickman, “SSL0.2 PROTOCOL SPECIFICATION.” http://www.mozilla.org/ projects/security/pki/nss/ssl/draft02.html, 1995.
[34] “Openssl, cryptography and SSL/TLS toolkit.” http://www.OpenSSL.org/docs/ssl/ SSL_CTX_set_generate_session_id.html, 2012.
[35] S. Hollenbeck, “Transport layer security protocol compression methods.” http://www. ietf.org/rfc/rfc3749.txt, 2004.
[36] P. Deutsch, “DEFLATE compressed data format specification version 1.3.”http://www. ietf.org/rfc/rfc1951.txt, 1996.
[37] C. Coarfa, P. Druschel, and D. S. Wallach, “Performance analysis of tls web servers,”
ACM Trans. Comput. Syst., vol. 24, February 2006.
[38] A. Goldberg, R. Buff, and A. Schmitt, “Secure web server performance dramatically improved by caching ssl session keys,” in SSL Session Keys,” in Workshop on Internet Server Performance, 1998.
[39] M. Ray, “Renegotiating TLS,” November 2009.
[40] “Server-Gated Cryptography.” http://www.sslguard.com/datasheets/sgc_eng.pdf.
[41] E. Rescorla, M. Ray, S. Dispensa, and N. Oskov, “Transport Layer Security (TLS) Renegotiation Indication Extension.” RFC 5746 (Proposed Standard), Feb. 2010.
[42] E. Rescorla, “SSL/TLS and computational dos.”http://www.educatedguesswork.org/ 2011/10/ssltls_and_computational_dos.html, October 2011.
[43] R. Friend, “Transport Layer Security (TLS) Protocol Compression Using Lempel-Ziv- Stac (LZS).” RFC 3943 (Informational), Nov. 2004.
[44] C. Lingam, “Http compression for web applications.” http://software.intel.com/ en-us/articles/http-compression-for-web-applications/, January 2009.
[45] “Option codes.” http://www.rsa.com/products/bsafe/documentation/ mesuite21html/dev_guide/group__SSL__OP__DEF.html.
[46] M. A. Jean-loup Gailly, “zlib.” http://zlib.net/, May 2012.
[47] I. Ristic, “Internet ssl survey 2010.” http://blog.ivanristic.com/Qualys_SSL_ Labs-State_of_SSL_2010-v1.6.pdf.
[48] V. Bernat, “SSL-DoS.” https://github.com/vincentbernat/ssl-dos/blob/master/ server-vs-client.c, 2012.
[49] T. Dierks and E. Rescorla, “RFC 5246 - the transport layer security (TLS) protocol version 1.2,” 2008.
[50] M. Cooper, Y. Dzambasow, P. Hesse, S. Joseph, and R. Nicholas, “Internet X.509 Public Key Infrastructure: Certification Path Building.” RFC 4158 (Informational), Sept. 2005.
[51] “The go programming language.”http://golang.org/pkg/, July 2012.
[52] A. Langley, “RSA public exponent size.” http://www.imperialviolet.org/2012/03/ 16/rsae.html, March 2012.
[53] D. Cooper, S. Santesson, S. Farrell, S. Boeyen, R. Housley, and W. Polk, “Internet X.509 Public Key Infrastructure Certificate and Certificate Revocation List (CRL) Profile.” RFC 5280 (Proposed Standard), May 2008.
[54] “Netcraft web server survey.”http://news.netcraft.com/archives/2012/, July 2012.
[55] N. Mathewson and N. Provos, “Libevent.”http://libevent.org/, May 2012.
[56] “MONIT UNIX systems management.”http://www.mmonit.com/monit/, June 2012.
[57] Andre, “Understanding linux CPU load - when should you be worried?.”http://blog. scoutapp.com/articles/2009/07/31/understanding-load-averages, July 2009.
[58] S. Even, O. Goldreich, and S. Micali, “On-line/off-line digital signatures,” inProceedings on Advances in cryptology, CRYPTO ’89, (New York, NY, USA), pp. 263–275, Springer- Verlag New York, Inc., 1989.
[59] A. Juels and J. Brainard, “Client puzzles: a cryptographic defense against connection depletion attacks,” in Network and Distributed System Security Symposium, 1999.
[60] C. Castelluccia, E. Mykletun, and G. Tsudik, “Improving secure server performance by re-balancing ssl/tls handshakes,” in Proceedings of the 2006 ACM Symposium on Information, computer and communications security, ASIACCS ’06, (New York, NY, USA), pp. 26–34, ACM, 2006. http://doi.acm.org/10.1145/1128817.1128826.
[61] T. Matsumoto, K. Kato, and H. Imai, “Speeding up secret computations with insecure auxiliary devices,” inProceedings on Advances in cryptology, CRYPTO ’88, (New York, NY, USA), 1990.
[62] A. K. Lenstra and E. R. Verheul, “Selecting cryptographic key sizes,”Journal of Cryp- tology, vol. 14, pp. 255–293, 1999.
[63] D. Dean and A. Stubblefield, “Using client puzzles to protect tls,” inProceedings of the 10th conference on USENIX Security Symposium - Volume 10, SSYM’01, (Berkeley, CA, USA), USENIX Association, 2001.
[64] J. Salowey, H. Zhou, P. Eronen, and H. Tschofenig, “Transport Layer Security (TLS) Session Resumption without Server-Side State.” RFC 5077 (Proposed Standard), Jan. 2008.
[65] J. R. Thai Duong, “Here come the⊕ ninjas,” May 2011.
[66] V. Gupta, S. Gupta, S. Chang, and D. Stebila, “Performance analysis of elliptic curve cryptography for SSL,” inWiSE ’02: Proceedings of the 1st ACM workshop on Wireless security, (New York, NY, USA), pp. 87–94, ACM, 2002.
APPENDIX
A
Cryptographic Algorithms and their working
A.1
RSA
This section explains the difference in overhead i.e. why the server needs more computational power in the case of RSA.
During handshake, client encrypts pre-master secret using RSA public key obtained from server’s certificate message. The server decrypts this encrypted message using corresponding private key. These are most expensive operation in handshake when a cipher suite involves RSA as a key exchange algorithm. For instance, TLS_WITH_RSA_AES256_SHA.
From experimental results in this section, it is clear that RSA decryption is more expensive than encryption.Working of RSA is given to understand the difference in RSA encryption and decryption.
Following steps are followed to generate the RSA key pair (public and private keys) in the OpenSSL.
• Selecte. Whereeis a public exponent. Default value forein OpenSSL is 65535. This can be also changed to 3.
• Choose two distinct prime numbers such that they follow following properties.
1< e <Φ(n) (A.1)
And
GCD(Φ(n), e) = 1 (A.2)
Where,
Φ(n) = (p−1)×(q−1) (A.3)
• This means, a public exponenteis a value such that it is co-prime with Φ(n).
• A private exponentdis calculated as follows:
d=e−1modΦ(n). (A.4)
• The public key is represented as (n,e) and private key as (n,d).
• A messagem is encrypted to obtain cipher text cas follows:
c=memodn (A.5)
• This cipher textc is decrypted to obtainm as follows: