Evaluation and Testing Chapter

This  chapter  is  broken  up  into  two  parts  statistical  quantitative  results  and  industry  expert   evaluation.   The   Quantitative   results   where   calculated   based   on   the   systems   performance.   All  results  where  conducted  using  a  Huawei  Ideos  Android  phone.  The  phone  has  Edge  and   3G/HSDPA   connectivity   and   therefore   both   connection   types   were   tested.   The   PaymentPortal   server   was   run   off   an   Ubuntu   10.04   instance   running   in   the   amazon   EC2   environment  in  Ireland.  Not  much  is  know  about  the  actual  hardware  that  the  instance  is   run  on.  The  only  specifications  that  are  known  is  that  the  instance  has  been  allocated  613   MB  memory  and  2  EC2  Computing  Units.  One  EC2  Computing  Unit  is  equivalent  to  a  single   core   1.2GHz   Opteron   or   Xeon   processor   [38].   The   goal   of   this   chapter   is   to   establish   the   success   of   the   project.   The   aim   was   to   design   a   system   that   was   as   responsive   as   current   methods  and  more  cost  effective  than  current  mobile  banking  techniques.  The  quantitative   testing  is  broken  up  into  three  parts  the  latency  or  processing  time  for  a  set  of  requests,  the   amount  of  data  consumed  to  conduct  payments  and  the  robustness  of  the  system.  Can  the   system  handle  multiple  servlet  requests  at  once?  

Latency    

5.1

When  testing  a  web  based  mobile  phone  application  the  connectivity  time  is  affected  by  the   strength   of   the   signal.   There   are   two   main   types   of   data   connections   available.   They   are   EDGE  and  3G.  Currently  over  80%  of  South  Africa  has  3G  coverage.  But  this  is  only  when   using   the   mobile   phone   outdoors.   When   the   phone   is   taken   inside   and   behind   walls   the   signal   is   weakened.   When   testing   the   response   time   of   communication   with   the   server,   testing  had  to  be  done  on  both  EDGE  and  3G  connections.  The  first  test  that  was  conducted   was  to  establish  a  baseline  for  a  connection  to  the  server.  Therefore  an  application  was  used   on  the  mobile  phone  to  test  the  ping  (measure  of  the  round-­‐trip  time  for  a  packet)  from  the   mobile  phone  to  the  server.  This  same  measure  was  used  to  calculate  the  cost  of  moving  the   server  from  a  university  PC  to  the  Amazon  EC2  in  Ireland.  For  each  scenario  listed  below   200  measurements  were  taken  to  establish  a  median  and  average  value.  

1. EDGE  mobile  to  Lab  PC   2. EDGE  mobile  to  EC2   3. 3G  mobile  to  LAB  PC   4. 3G  mobile  to  EC2    

Table  4  Latency  Results  of  mobile  phones  

The  results  from  the  ping  tests  revealed  surprising  results.  The  3G/HSDPA  results  were  not   as  optimal  as  originally  thought  after  looking  at  the  actual  results  it  was  established  that  the   results   had   many   spikes.   These   spikes   occurred   when   the   network   changed   from   3G   to   HSDPA  the  cost  of  this  change  was  about  2000ms.  It  was  interesting  to  see  that  the  extra   latency  of  moving  the  server  to  Ireland  was  not  as  high  as  originally  thought.  In  some  cases   the  EC2  results  were  better  this  was  due  to  the  load  on  the  UCT  network.  From  Table  4  the   3G   network   does   not   seem   to   offer   an   improvement   in   latency   compared   to   EDGE.   The  

  Lab  PC  (South  Africa)   EC2  Server  (Ireland)  

EDGE  (Median)   403ms   509ms  

EDGE  (Average)   501ms   782ms  

3G/HSDPA  (Median)   483ms   532ms  

seen  in  Table  5  and  the  cost  of  moving  the  server  to  Ireland  can  be  seen  clearly.  This  is  an   interesting  result  because  the  PaymentPortal  system  does  once  off  calls  to  the  server  and   not  a  continuous  connection  and  therefore  this  type  of  result  shows  the  best  and  worse  case   results.  It  can  also  now  clearly  be  seen  that  the  cost  of  moving  the  server  to  Ireland  is  about   100ms   extra   in   latency.   Yet   by   looking   at   the   max   latency   on   the   different   connections   it   clearly   shows   that   the   EC2   server   connection   is   more   stable.   This   is   because   there   is   no   other  traffic  being  process  on  the  network  unlike  the  campus  network.    

Table  5  Max  and  Min  Latency  Results  

The   follow   results   conducted   on   the   PaymentPortal   system   was   to   establish   the   cost   of   using  a  HTTP  vs.  HTTPS  connection.  These  results  will  show  the  true  cost  of  encrypting  the   communication  channel.  For  these  the  time  was  measured  from  the  time  a  request  was  sent,   from   the   MyPay   application   to   the   server,   and   a   response   was   received   back   from   the   server.     For   this   case   of   the   testing   the   default   HTTPS   encryption   settings   were   used   as   mentioned   in   section   4.3.5.1.   The   default   encryption   used   is   128bit   RC4   encryption   with   MD5  message  authentication.    

  Figure  32  HTTP  vs.  HTTPS  Comparison  on  EDGE  

0   1000   2000   3000   4000   5000   6000   7000   8000   0   20   40   60   80   100   120   140   160   Response  Time  

Number  of  tests  

EDGE  with  HTTPS  to  EC2  and  128bit   Edge  HTTP  Results  EC2  

  EDGE  to  Lab  PC   EDGE  to  EC2   3G/HSDPA  to  Lab  PC   3G/HSDPA  to  EC2  

Min   241ms   401ms   136ms   310ms  

  Figure  33  HTTP  vs.  HTTPS  Response  Comparison  on  3G/HSDPA  

Table  6  Latency  of  HTTP  vs.  HTTPS  connections    

From  these  results,  in  Figure  32  and  33,  it  was  interesting  to  see  that  the  3G  results  were   more   consistent   than   the   EDGE   results   on   both   HTTP   and   HTTPS.   The   EDGE   results   had   spikes  in  response  times  this  could  be  due  to  the  weaker  signal  strength.  The  median  value   is  a  better  value  to  use  due  to  the  spikes  in  the  results.  These  spikes  skew  the  average  value   but   a   median   value   is   the   middle   value   in   an   ordered   list.   Table   6   shows   that   the   cost   of   adding  encryption  to  the  communication  channel  was  between  1000-­‐1300ms  for  EDGE  and   700-­‐800ms  on  3G.  This  is  an  increase  of  +-­‐90%  over  the  HTTP.  It  also  shows  that  the  cost  of   setting  up  an  URLConnection  is  much  more  on  EDGE  than  3G.  If  you  compare  the  median   ping  results  from  Table  4  with  the  median  results  from  Table  6.  It  shows  an  increase  of  over   100%  on  EDGE  but  only  56%  increase  on  3G.  This  increase  can  be  accounted  to  the  extra   bandwidth  needed  to  set  up  an  URLConnection  and  the  limits  of  EDGE.    

The  aim  of  this  project  was  to  make  a  secure  mobile  payments  platform.  But  the  question   then   became   how   secure   should   the   communication   be?   HTTPS   allows   for   a   multitude   of   encryption  techniques.  The  default  is  128bit  RC4  as  used  in  Table  6.  This  is  the  fastest  and   most   widely   used.   Yet   this   is   not   the   most   secure   connection   available.   Therefore   other   HTTPS   encryption   types   needed   to   be   tested.   The   compromise   in   performance   by   adding   extra  security  needed  to  be  calculated.    

  0   1000   2000   3000   4000   5000   6000   0   10   20   30   40   50   60   Response  time    

Number  of  tests  

3G/HSDPA  on  HTTP  to  EC2   3G-­‐HSDPA  HTTPS  to  EC2  

  EDGE  to  EC2   3G/HSDPA  to  EC2  

HTTP  (Average)   1677ms   983ms  

HTTPS  (Average)   2621ms   1664ms  

HTTP  (Median)   1195ms   833ms  

Figure  34  Comparison  between  128bit  and  256bit  encryption      

Table  7  Median  Latency  for  128  vs.  256bit  encryption  

In  Table  7  it  can  be  seen  that  the  performance  difference  versus  encryption  difference  is  +-­‐ 120ms   extra   in   response   time.   This   is   less   than   a   10%   for   an   exponential   increase   in   security.  For  this  reason  it  was  decided  to  use  the  more  secure  256bit  AES  encryption  for   the   communication   channel.   Since   the   server   location   and   encryption   level   have   been   evaluated   the   final   item   to   test   was   the   actual   response   time   for   given   requests.   These   would   be   everyday   requests   that   the   server   would   receive.   They   range   from   login   times,   check  balance  requests,  processing  a  payment  and  registering  a  new  card.  Table  8  contains   the  results  all  done  while  the  mobile  phone  had  a  3G  connection  an  EDGE  connection  added   between  3-­‐5  seconds  more  onto  the  response  time.  

Table  8  Median  Latency  for  MyPay  Requests  

It  takes  on  average  3.6  seconds  to  process  a  payment.  This  is  faster  than  the  average  of  10   seconds  it  takes  a  current  credit  card  machine  to  process  a  payment.  This  result  can  be  seen   as  a  acceptable  amount  of  time  to  process  a  payment.  And  is  on  par  or  faster  than  current   methods  of  payment.  Cash  needs  to  be  counted  and  change  calculated  this  will  take  longer  

  3G/HSDPA  to  EC2  

RC4_128  bit  Encryption   1675ms   AES_256  bit  Encryption   1803ms  

  Response  Time    

Check  Balance     3460ms  

Merchant  Login   4045ms  

Register  New  Merchant   4099ms  

Process  Payment   3600ms  

Register  New  Card   4442ms  

0   1000   2000   3000   4000   5000   6000   1   11   21   31   41   Latency  (ms)  

Number  of  readings  

RC4_128,  with  MD5  for  message  authentication  and  RSA  

than  3.6  seconds.  Based  on  these  results  it  can  be  established  that  the  response  time  of  the   system  is  satisfactory  for  a  mobile  payments  system.  The  response  is  also  guaranteed  unlike   SMS  where  it  can  take  up  to  several  hours  for  the  SMS  to  be  delivered.  The  reason  a  register   new  card  request  is  longer  than  any  other  request,  is  due  to  the  process  involved  in  creating   a   new   card.   When   a   new   card   is   created   there   are   two   external   API   calls   that   need   to   be   made.  The  first  is  a  bit.ly  request  to  shorten  the  download  URL  and  the  second  is  the  API  call   to  the  Clickatell  API  to  send  the  notification  SMS.    

Data  used  

5.2

In  order  to  calculate  the  exact  cost  of  a  payment  the  amount  of  data  used  per  request  was   needed.   Cellphone   data   is   charged   per   the   amount   of   bandwidth   used.   To   calculate   the   amount   of   data   used   for   each   request   an   application   was   installed   on   the   PaymentPortal   Server.  The  server  is  running  Ubuntu  10.04  and  there  is  a  bandwidth  monitoring  software   called  iftop.  The  reason  this  application  was  chosen  is  because  it  shows  the  bandwidth  per   device  connected.  Each  devices  connected  to  the  server  has  a  unique  IP  address  that  can  be   used  to  identify  the  device.  Figure  35  is  a  screenshot  of  the  iftop  application  running  on  the   server  with  a  mobile  device  connected  and  making  requests.    

The   IP   address   of   the   mobile   phone   can   be   seen   circled   in   red.   This   is   used   to   track   each   connection.   The   total   bandwidth   sent   and   received   for   the   session   can   be   seen   circled   in   blue.  The  top  figure  is  the  amount  of  data  sent  from  the  server  and  the  amount  below  is  the   amount  of  data  received  from  the  mobile  phone.  Figure  35  was  taken  during  a  login  request   the   amount   of   data   consumed   was   3.39Kb.   A   break   down   of   the   amount   of   data   for   each   request  can  be  seen  in  Table  9.    

Table  9  Cost  Per  Request    

Table  9  shows  that  the  amount  of  data  used  per  request  is  just  over  3Kb.  Using  the  average   price  of  R2/Mb  (0.19c/Kb)  the  cost  per  a  transaction  was  calculated.  The  cost  per  request  is   approximately   0.6cents   a   message.   This   is   below   the   desired   one   cent   a   transaction   originally   required   in   the   design   chapter.   Factors   that   contribute   to   this   low   data   consumption  are  the  use  of  JSON_Objects  and  the  HTTPS  protocol.  The  amount  of  data  could   have  maybe  been  reduced  more  if  compression  was  applied  to  the  data  before  sending.  But   this   would   have   required   more   processing   time.   At   a   price   of   0.6cents   a   transaction   a   merchant  can  process  roughly  fifty  transactions  per  day  for  less  than  35  cents.  Therefore  a   person  can  do  fifty  mobile  payments  for  the  less  than  the  cost  of  doing  one  using  SMS.  The   PaymentPortal   system   is   also   cheaper   than   USSD,   which   costs   20   cents   per   20   seconds   connected.    

Stress  Test  System